Animal abnormal state intelligent monitoring method and system based on biological behavioristics

Abstract

The invention belongs to the technical field of data processing and animal behavior recognition, and discloses an animal abnormal state intelligent monitoring method and system based on biological behavioristics, and the method comprises the following steps: S1, constructing and training an animal abnormal state monitoring model; s2, constructing an animal abnormal state monitoring network system; and S3, carrying out intelligent monitoring on the abnormal state of the animal. According to the method and the system provided by the invention, multi-model fusion is adopted, the animal behavior data is efficiently processed in real time, and the accuracy and the robustness of animal abnormal state monitoring are improved through more comprehensive and abundant feature representation data and twice judgment; the system can realize real-time monitoring of animal individual and group behaviors, timely discover abnormal states and perform early warning or intervention, can be widely applied to zoos, farms and the like, meets industrial requirements of large-scale and large-quantity animal synchronous monitoring, and effectively protects health and safety of animals.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing and animal behavior recognition, and in particular to a method and system for intelligently monitoring abnormal animal states based on biobehavior. Background Art

[0002] There are many different abnormal behaviors of animals, both individual and group, which generally involve movement, behavior, social interaction, appetite and body shape. The abnormal behaviors of individual animals can be manifested in many different forms, which may be caused by various factors such as disease, environmental changes, stress response or natural disasters; while the abnormal behaviors of animal groups are usually related to the overall health of the animal group, environmental changes or changes in the leadership of key individuals. Observing these abnormal behaviors and taking appropriate measures to intervene in time are crucial to maintaining the health and welfare of animals. Therefore, abnormal animal activities are a complex and diverse field. On the one hand, the specific manifestations of abnormal animal activities may vary due to internal factors such as species, age, gender and genetic background, and on the other hand, they are also related to external factors such as environmental changes and social behavior. Therefore, it is necessary to comprehensively consider multiple factors to make accurate analysis and judgment, which poses a huge challenge to the intelligent monitoring and identification technology of abnormal animal states.

[0003] In animal breeding, animal protection and other related industries, farms or management units often need to monitor a large number of animals in a large area at the same time. Traditional animal abnormal status monitoring methods usually rely on artificially designed features and rules, and there are problems such as the inability to timely discover and handle large-scale, large-volume and complex scenes and changes. Due to the lack of a sufficient number of professionally trained professionals, it is difficult for manpower to conduct all-round, no-dead-angle, all-weather monitoring and judgment of animals in zoos or livestock in the animal husbandry industry. It is impossible to grasp the changes in the health status of animals and livestock in a timely manner and intervene, and it is impossible to ensure the quality and efficiency of animal breeding and protection.

[0004] The existing biometric recognition technologies often have the following problems in practical applications: (1) Lack of relevant professional knowledge and experience makes it impossible to accurately identify abnormal conditions of animals; (2) In particular, the recognition of stereotyped behaviors of animals in abnormal conditions is still a blank; (3) Most of the current recognition technologies can only recognize more obvious abnormal conditions of animals such as wounds and attacks, but it is difficult to recognize abnormal conditions such as psychological abnormalities and depression.

[0005] For example, in the prior art, Chinese invention document CN111183917A discloses an animal abnormality monitoring, image processing method and device, the animal abnormality monitoring method includes: obtaining a breeding area image; monitoring the animal state corresponding to the animal feces from the breeding area image according to a pre-trained feces monitoring model, the animal state including an abnormal state and / or a normal state; determining the monitoring information corresponding to the breeding area according to the animal state. This prior art solution is based on computer vision, and uses a pre-trained model to monitor animal feces from images. Although it can automatically, accurately and timely monitor whether animals have diarrhea and other epidemic conditions, prevent the spread of the epidemic, and do not require manual monitoring, reducing labor costs and time costs, the existing animal abnormal behavior recognition technology is only based on one element of animal feces, and cannot achieve accurate recognition of various animal behaviors and timely detection of abnormal conditions; in addition, its real-time monitoring results need to be further handed over to manual judgment, and the overall processing efficiency is low and the accuracy is poor.

[0006] In addition, although various deep learning models have been widely used in tasks such as image classification, target monitoring, action recognition, and have achieved many breakthrough results, such as the SlowFast model, which is a deep learning model that performs well in the field of video understanding. It combines slow and fast paths to effectively capture the spatial and temporal information in the video, thereby achieving understanding and analysis of the video content. However, the judgment results output by the model still have shortcomings such as poor stability, and it is difficult to take into account the accuracy and robustness of the monitoring results.

[0007] Therefore, the existing technology cannot solve the above deficiencies, and cannot accurately define abnormal behaviors exhibited by individual and group animals, and therefore cannot accurately identify, monitor and intervene in a timely manner. It cannot meet the needs of related industries for simultaneous monitoring of large-scale and large numbers of animals, improving animal living conditions and protection measures, and controlling and avoiding risk events such as damage to animals themselves and mutual damage between animal groups. Summary of the invention

[0008] To solve the above technical problems, the present invention provides a method and system for intelligent monitoring of abnormal states of animals based on biobehavior, which judges complex animal behaviors as typified normal or abnormal states, and adopts the SlowFast dual-stream network model and abnormal score judgment model based on the deep learning framework to process animal behavior data in real time and efficiently, and improves the accuracy and robustness of monitoring abnormal states of animals through more comprehensive and rich feature representation data and two judgments, so as to meet the needs of related industries for synchronous monitoring of large-scale and large numbers of animals, improving living conditions and protection measures for animals, and controlling and avoiding risk events such as damage to animals themselves and mutual damage between animal groups.

[0009] The technical solution of the present invention is as follows:

[0010] A biobehavioral-based intelligent monitoring method for abnormal animal states comprises the following steps:

[0011] S1. Build and train animal abnormal state monitoring model

[0012] S1-1. Model construction

[0013] Construct a two-stream network SlowFast model based on a deep learning framework as a model for judging and monitoring animal abnormal states based on biobehavior and changes in animal behavior trajectories;

[0014] S1-2. Model training

[0015] Using the existing associated trajectory feature dataset that has been annotated with normal and abnormal states of animals, the two-stream network SlowFast model based on the deep learning framework is trained. After multiple rounds of training, an animal abnormal state monitoring model is obtained.

[0016] S2. Build an animal abnormal status monitoring network system

[0017] S2-1, constructing a network system including a cloud server, a management terminal and at least one animal behavior data collection unit;

[0018] S2-2, deploying the trained animal abnormal state monitoring model to the cloud server;

[0019] Deploy animal behavior data collection units to the field where the monitored animals are located;

[0020] S2-3, connecting the cloud server, the management terminal and the animal behavior data collection unit to each other through the network and enabling interactive communication;

[0021] S3. Intelligent monitoring of abnormal animal conditions

[0022] S3-1: Collect and upload the monitored animal behavior data: Animal behavior data collection unit, collects and uploads animal behavior data to the cloud server;

[0023] S3-2: Preliminary judgment: The uploaded animal behavior data is preprocessed by the trained animal abnormal state monitoring model to obtain a real-time animal behavior associated trajectory feature data set, and then the animal abnormal state is analyzed and calculated to obtain a preliminary judgment result. If the animal abnormal state monitoring model determines that it is abnormal, then:

[0024] S3-3: Judge again and output the result: input the preprocessed real-time animal behavior associated trajectory feature data set into the abnormal score judgment model formula 3, calculate the abnormal score score of each trajectory point, judge whether the abnormal score score falls within the preset normal state threshold range and output the final monitoring result: if the abnormal score exceeds the preset normal state threshold range, it is determined that the animal at the corresponding trajectory point is in an abnormal state; conversely, if the abnormal score does not exceed the preset normal state threshold, it is determined that the animal at the trajectory point is in a normal state:

[0025]

[0026] In Formula 3, score is the anomaly score of the trajectory point, e is the base of the natural logarithm, and z is the weighted sum of the feature vector of the trajectory point and the model parameter;

[0027] S3-4: Identification and monitoring of animal group behavior

[0028] Repeat steps S3-1 to S3-3, collect the behavior data of each animal in the monitored animal group and upload them to the cloud server, and judge the behavior status of each animal respectively. If two or more animals are in abnormal state at the same time, it is judged that there is an abnormal state of the animal group;

[0029] S4. Early warning and intervention of abnormal animal conditions

[0030] Repeat steps S2-S3 to continuously monitor the abnormal behavior of the animal, and use the trained animal abnormal state monitoring model and formula 3 to jointly monitor and analyze the animal state in real time. If the animal is found to be in an abnormal state, an early warning and intervention are carried out;

[0031] S5: Animal abnormal status monitoring network system operation monitoring and update

[0032] Regularly check the operating status of the animal abnormal state monitoring network system, process system logs and error information, and maintain safe and stable operation of the system; regularly use the newly acquired and verified associated trajectory feature data set to re-train the animal abnormal state monitoring model, and then redeploy it to the cloud server, and repeat steps S3-S4; through the above-mentioned real-time monitoring and analysis of animal behavior trajectory data, it is possible to identify abnormal behavior patterns and behavior state types of individual or group animals, including abnormal activities (lameness, paralysis, stereotyped behavior, rotation, jumping, twisting), illness (reduced activity, whole body or local tremor), injury, etc., and take corresponding measures in time to intervene and deal with it.

[0033] An animal abnormal state intelligent monitoring system based on biobehavior, which is a network system composed of a cloud server, a management terminal and one or more animal behavior data acquisition units, each part is interconnected and communicated through a network, and is used to implement the above-mentioned animal abnormal state intelligent monitoring method based on biobehavior;

[0034] The cloud server is built with: a data input unit, an animal abnormal state monitoring model, and a result output unit;

[0035] The data input unit is used to receive animal behavior data;

[0036] The animal abnormal state monitoring model is used for animal behavior recognition and abnormal state monitoring, and determines whether the animal behavior type corresponds to a normal state or an abnormal state based on the collected animal behavior data;

[0037] The result output unit is used to output the animal status monitoring results;

[0038] The animal behavior data acquisition unit includes: a video camera and a video data storage device;

[0039] The management terminal includes a computer or a mobile intelligent terminal device.

[0040] The animal abnormal state monitoring model includes a two-stream network SlowFast model based on a deep learning framework and an abnormal score judgment model, wherein the two-stream network SlowFast model includes a feature extraction module, a feature fusion module, and a classifier module;

[0041] The feature extraction module uses the SlowFast model to perform feature extraction on the preprocessed animal behavior data to extract the temporal characteristics and spatial relationship feature data of the animal behavior;

[0042] The feature fusion module fuses the feature data extracted from the slow path and the fast path to obtain feature representation data;

[0043] The classifier module uses the trained classification model to classify the fused feature representation data to determine whether the animal behavior type corresponds to a normal state or an abnormal state.

[0044] The biobehavioral-based intelligent monitoring method and system for abnormal animal states provided by the present invention have the following beneficial effects:

[0045] 1. The method and system provided by the present invention firstly judge the complex animal behavior as a typed normal or abnormal state, and adopt the SlowFast two-stream network model and the abnormal score judgment model based on the deep learning framework to process the animal behavior data in real time and efficiently, and improve the accuracy and robustness of the monitoring of the abnormal state of the animal through more comprehensive and rich feature representation data and two judgments; only the abnormal state corresponding to the abnormal behavior of the animal is warned and intervened, which saves the system computing resources and improves the system operation speed, so as to realize the real-time monitoring of the behavior of individual and group animals, timely discover the abnormal state and give warning or intervention, while the normal state keeps silent operation, so as to effectively reduce the system resource consumption and the system operation and response speed, and protect the health and safety of individual and group animals; compared with the traditional monitoring method, the present invention can take into account the monitoring accuracy and speed, take into account the individual and group animals, and can realize the monitoring of a large number of animal behaviors at low cost and high efficiency, and can meet the needs of the relevant industries for the synchronous monitoring of a large range and a large number of animals, the improvement of the living conditions and protection measures of animals, and the control and avoidance of the occurrence of risk events such as animal self-damage and mutual damage between animal groups.

[0046] 2. The present invention combines the SlowFast model based on the deep learning framework with the abnormal score judgment model for secondary judgment, and only performs alarm processing on abnormal animal states, which has the advantages of high monitoring accuracy, fast speed, large coverage, and a large number of synchronously monitored animals. The pre-trained deep learning model SlowFast is used as the latest research model in the dual-stream CNN, which has a high monitoring and analysis speed, and the abnormal score judgment model further improves the accuracy of the judgment results and greatly reduces the misjudgment rate. It can be widely used in industrial scenarios such as zoos and farms, and meets the industrial needs of synchronous monitoring of large numbers of animals in a large range, effectively protecting the health and safety of animals, and improving economic benefits.

[0047] 3. The method and system provided by the present invention judge the complex animal behaviors as normal or abnormal states of a type. By integrating two models and performing real-time monitoring and analysis of animal behavior trajectory data, the abnormal behavior patterns and behavior state types of individual or group animals can be accurately identified, including abnormal activities, illness, and injury. Then, corresponding measures can be taken in time to intervene and deal with them. Among them, the SlowFast dual-stream network structure is used as the main model. While ensuring the monitoring accuracy, a SlowFast behavior recognition method based on animal species recognition technology and abnormal behavior recognition for trajectories is designed, which overcomes the deficiency that the target monitoring of the prior art is only applied in the fields of vehicle monitoring, face monitoring, automatic driving, and security systems, but is rarely applied in combination with animal recognition technology.

[0048] 4. The method and system provided by the present invention typify and model the animal's movement trajectory, and can accurately identify the normal or abnormal stereotyped behavior of the animal; the present invention simplifies the animal's behavior trajectory with a point-line model, simulates the animal with points, simulates the movement trajectory with lines, and increases the monitoring of action duration, moving distance, moving speed, number of repetitions, etc. for training AI; so that AI can intelligently identify whether the animal is in an abnormal state based on the changes and records of the animal's behavior trajectory; no reports of related technologies and solutions have been found at home and abroad on the use of point-line models to train AI for identifying abnormal stereotyped behaviors of animals.

[0049] 5. The method and system provided by the present invention can automatically generate a report on the abnormal behavior status of animals within a certain period of time as needed, so as to better monitor the physical condition of animals in real time and conduct effective management, assist management personnel to investigate the causes of abnormal situations and solve problems, and ensure the physiological and psychological health of animals; the method and system provided by the present invention can be applied in scenes such as zoos, breeding farms, farms, scientific research laboratories, etc., to monitor animal behavior in real time and identify abnormal conditions in a timely manner, and has broad application prospects.

[0050] 6. The method and system provided by the present invention adopt multimodal feature fusion: the SlowFast model is used to simultaneously process the slow and fast paths in the video, thereby obtaining a more comprehensive and rich feature representation. This multimodal feature fusion method can better capture the temporal changes and spatial relationships of animal behavior, and improve the accuracy and robustness of abnormal state monitoring; slow and fast paths refer to two paths in the video that capture and process image sequences at different frame rates. The slow path is usually used to capture parts with slower movements and richer details, while the fast path is used to capture parts with faster movements and more obvious overall dynamics.

[0051] 7. The method and system provided by the present invention, based on an artificial intelligence model, automatically identify the abnormal states corresponding to changes in animal behavior trajectories, and only provide early warnings and interventions for the abnormal states corresponding to the abnormal behaviors of animals, thereby saving system computing resources and improving the system operation speed, so as to realize real-time monitoring of individual and group behaviors of animals, timely discover abnormal states and provide early warnings or interventions, and effectively protect the health and safety of individual and group animals. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 Schematic diagram of the network structure and flow of the intelligent monitoring system for abnormal animal status according to the embodiment of the present invention;

[0053] Figure 2 Schematic diagram of the main process of the method for intelligently monitoring abnormal state of animals according to an embodiment of the present invention;

[0054] Figure 3 It is a schematic diagram of the structure and flow of some modules of the animal abnormal state monitoring model according to an embodiment of the present invention;

[0055] Figure 4 It is another part of the module composition structure diagram and process flow diagram of the animal abnormal state monitoring model in the embodiment of the present invention.

[0056] Figure 5 This is an example diagram of the process of intelligent monitoring of abnormal animal status and outputting results according to an embodiment of the present invention;

[0057] Figure 6 Schematic diagram of the working process of the intelligent monitoring method for abnormal state of animals according to an embodiment of the present invention;

[0058] Figure 7 is another example diagram of the process of intelligently monitoring abnormal conditions of animals and outputting results according to an embodiment of the present invention;

[0059] Figure 8 Schematic diagram of an original image (OriginalImage) and a noisy image (Noisy Image) of a trajectory image in animal behavior data collected in an embodiment of the present invention;

[0060] Fig. 9 It is a schematic diagram of the animal state after data preprocessing in an embodiment of the present invention. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0062] Example 1

[0063] See also Figures 1 to 3 The animal abnormal state intelligent monitoring method and system based on biobehavioral science provided in this embodiment are suitable for industrial scenarios of large-scale and large-scale animal behavior monitoring, and can simultaneously monitor the health status of a large number of animals. It can dynamically and real-time monitor the abnormal state of animals of different species, body shapes, and growth stages. The animal abnormal state intelligent monitoring method based on biobehavioral science includes the following steps:

[0064] S1. Build and train animal abnormal state monitoring model

[0065] S1-1. Model construction

[0066] A two-stream network SlowFast model based on a deep learning framework is constructed as a model for judging and monitoring animal abnormal states based on biobehavior and changes in animal behavior trajectories; specifically:

[0067] Build a SlowFast model based on the deep learning TensorFlow or PyTorch framework. Based on biobehavior and animal behavior trajectory changes, perform feature extraction and space-time information fusion processing on the collected animal behavior trajectory video data to obtain the temporal changes and spatial relationship laws of animal behavior. Through dual-path design and deep learning, associate the extracted features with the state labels of animal behavior to achieve classification and identification of the state of animals in different behavior modes. As an animal abnormal state monitoring model, judge and monitor the abnormal state of animals.

[0068] S1-2. Model training

[0069] The existing data is obtained through the data input unit, and the existing data (associated trajectory feature data set) that has been labeled with the normal and abnormal states of animals is used to train the two-stream network SlowFast model based on the deep learning framework. After multiple rounds of training, an animal abnormal state monitoring model is obtained; specifically:

[0070] The existing data with marked normal and abnormal states of animals are obtained from an external database through a data input unit, and the animal abnormal state monitoring model is pre-trained with the existing data. The existing associated trajectory feature data set with marked normal and abnormal states of animals is used to train the two-stream network SlowFast model based on the deep learning framework, and the animal abnormal state monitoring model is obtained after multiple rounds of training;

[0071] The steps of processing the existing data into the existing associated trajectory feature data set are: extracting features from the pre-processed existing animal behavior trajectory data by manual annotation or by using the SlowFast model, matching and annotating the normal or abnormal state of the animal based on the spatiotemporal information in the biobehavioral and animal video sequences, associating the extracted trajectory data set with the animal state, and obtaining an associated trajectory feature data set that distinguishes the normal or abnormal state of the animal in the biobehavioral.

[0072] During training, the supervised deep learning TensorFlow or PyTorch framework is used to train the SlowFast model, including the cross entropy loss function and back propagation algorithm, to match the extracted trajectory feature data with the animal behavior labels, so that the model has the ability to automatically extract, match, classify and identify the state of different animal behavior data;

[0073] S1-3. Model Optimization

[0074] In the multi-round training process of step S1-2, the cross entropy loss function is used to optimize the animal abnormal state monitoring model, where the expression of the loss function is formula 2:

[0075]

[0076] In Formula 2, y represents the true label, Represents the prediction result of the model, and N represents the number of samples. Through the back propagation algorithm, the parameters of the model are adjusted so that the loss function reaches the minimum value, thereby improving the accuracy and generalization ability of the animal abnormal state monitoring model.

[0077] S1-4. Model testing

[0078] Using the trained animal abnormal state monitoring model, a small amount of video data collected by manual and animal behavior data acquisition units is input into the trained model. The model automatically classifies and identifies animal behaviors, outputs animal state judgment results, and compares the results with the animal state judgment results obtained by manual monitoring to test the accuracy of the animal abnormal state monitoring model.

[0079] S2. Build an animal abnormal status monitoring network system

[0080] S2-1. Construct an animal abnormal state monitoring network system connected through a network, consisting of a cloud server, a management terminal and one or more animal behavior data collection units;

[0081] The cloud server is used to receive monitoring data of monitored animals, perform status analysis, and output monitoring results. The cloud server is built with: a data input unit, an animal abnormal status monitoring model, and a result output unit; the animal abnormal status monitoring model is equipped with a feature extraction module, a feature fusion module, and a classifier module; the classifier module is built with a classification model;

[0082] The management terminal is used by the management personnel to input control instructions and obtain monitoring results;

[0083] The animal behavior data collection unit is used to collect the behavior trajectory data of the monitored animal on site, and includes: an audio and video camera, and a data storage device;

[0084] S2-2, deploy the trained animal abnormal state monitoring model to the cloud server, and deploy the animal behavior data collection unit to the site where the monitored animals are located; before deployment, the system can also be debugged and tested to ensure that the system can run stably and effectively identify the abnormal state of animals;

[0085] S2-3, the cloud server, management terminal and animal behavior data collection unit are connected to each other through the network and communicate with each other; the management personnel input the relevant control program and control signal through the management terminal to set the operation parameters, etc.;

[0086] S3. Intelligent monitoring of abnormal animal conditions

[0087] See attached Figure 2 , the main steps include:

[0088] Collect animal behavior data, including the animal's location information, movement path, action duration, etc. in different time periods;

[0089] Then the collected data is preprocessed, including data cleaning, denoising, trajectory extraction and other operations to ensure the quality and integrity of the data;

[0090] Use the trained SlowFast model to analyze and learn the preprocessed data and extract the characteristic representation of animal behavior;

[0091] Setting animal abnormal behavior monitoring rules and threshold ranges, such as judging the abnormal state corresponding to the animal's abnormal behavior based on indicators such as the animal's movement speed, frequency and activity range;

[0092] The SlowFast model and the abnormal score judgment model perform abnormal behavior monitoring calculations on the extracted feature representations, determine whether the animal is in an abnormal state, and output the monitoring results.

[0093] The specific steps include:

[0094] S3-1: Collect and upload the monitored animal behavior data and upload the animal behavior data to the cloud server;

[0095] One or more animal behavior data collection units installed at the animal activity site collect and upload the monitored animal behavior data to the cloud server;

[0096] The animal behavior data is the individual behavior data or group behavior data of the monitored animals that occur continuously, including: animal behavior video and audio data recorded in time sequence;

[0097] S3-2: Preliminary judgment: The trained animal abnormal state monitoring model pre-processes the uploaded animal behavior data, analyzes and calculates the animal abnormal state, and obtains the preliminary judgment results, which are as follows:

[0098] S3-2-1: Data preprocessing

[0099] The collected animal behavior data (including location information, movement path, etc.), that is, the trajectory image data in the video sequence, is preprocessed, including cleaning and denoising, to extract the animal behavior-related trajectory feature data set;

[0100] The feature extraction uses the following formula 1 to calculate the moving speed of the trajectory point:

[0101]

[0102] In Formula 1, ν represents the moving speed, x1, y1 and x2, y2 represent the coordinates of trajectory points 1 and 2 respectively, and t1, t2 represent the timestamps of trajectory points 1 and 2 respectively; by calculating the moving speed of each trajectory point, the motion feature data set of each trajectory point is accumulated to obtain the animal behavior associated trajectory feature data set; the associated trajectory feature data set specifically includes: abnormal activity pattern data, abnormal posture data, abnormal posture data, abnormal sound data, abnormal behavior data, etc.

[0103] S3-2-2: Simplified processing of animal abnormal characteristics

[0104] The animal's behavior trajectory is simplified using a point-line model, with points simulating animals and lines simulating movement trajectories. Monitoring data on action duration, moving distance, moving speed, and number of repetitions are added to train the two-stream network SlowFast model, so that the two-stream network SlowFast model has the ability to intelligently identify whether the animal is in an abnormal state based on changes in the animal's behavior trajectory and records.

[0105] S3-2-3: Animal abnormal feature data extraction

[0106] Using the feature extraction module in the pre-trained animal abnormal state monitoring model, feature extraction is performed on the input data to extract the temporal characteristics and spatial relationship feature data of the animal behavior;

[0107] S3-2-4: Animal abnormality feature data fusion

[0108] Using the feature fusion module in the pre-trained animal abnormal state monitoring model, the feature data extracted from the slow and fast paths are fused to obtain feature representation data;

[0109] S3-2-5: Animal feature representation data classification

[0110] Utilize the classifier module in the pre-trained animal abnormal state monitoring model, use the trained classification model to classify the fused feature representation data one by one according to the time alignment rules and the preset abnormal state time alignment threshold range, match the animal behavior state label with the feature representation data, if according to the time alignment rules, a feature representation data falls into the preset abnormal state time alignment threshold range, it is matched as abnormal state data and marked as animal behavior state, otherwise it is matched as normal state data and marked as normal state. If the animal abnormal state monitoring model determines that it is abnormal, then:

[0111] S3-3: Judge again and output the result: input the preprocessed animal behavior data into the abnormal score judgment model (Formula 3), calculate the abnormal score score of each trajectory point (i.e., the confidence of the prediction result of the SlowFast model), judge whether the abnormal score score falls within the preset normal state threshold range and output the final monitoring result: if the abnormal score exceeds the preset normal state threshold range, it is determined that the animal at the corresponding trajectory point is in an abnormal state; otherwise, if the abnormal score does not exceed the preset normal state threshold, it is determined that the animal at the trajectory point is in a normal state:

[0112]

[0113] In Formula 3, score is the anomaly score of the trajectory point, e is the base of the natural logarithm, and z is the weighted sum of the feature vector of the trajectory point and the model parameter;

[0114] S4. Early warning and intervention of abnormal animal conditions

[0115] Repeat steps S2-S3 to continuously monitor animal behavior, and use the trained animal abnormal state monitoring model and abnormal score judgment model to jointly monitor and analyze the animal state in real time. If the animal is found to be in an abnormal state, early warning and intervention will be carried out.

[0116] S5: Animal abnormal status monitoring network system operation monitoring and update

[0117] Regularly check the operating status of the animal abnormal state monitoring network system, process system logs and error information, and maintain safe and stable operation of the system; regularly use newly acquired and verified associated trajectory feature data sets to re-train the animal abnormal state monitoring model, and then redeploy it to the cloud server, and repeat steps S3-S4.

[0118] In the process of classifying the animal feature representation data in step S3-2-5, the classifier module in the pre-trained animal abnormal state monitoring model is used to classify the fused feature representation data one by one according to the time alignment rule and the preset abnormal state time alignment threshold range using the trained classification model, and the animal behavior state label is matched with the feature representation data. The time alignment rule can ensure that the animal behavior video data features and the corresponding animal behavior labels are aligned in time, that is, each video feature should correspond to the animal behavior label in the same period or continuous time period; the judgment process according to the time alignment rule is as follows: first, the animal behavior video data is segmented according to time, usually using a fixed length time window for division, and each time period corresponds to a continuous time period in the video; for each time period, the corresponding animal behavior label is determined, which can be achieved by manual labeling or automatic labeling algorithm to ensure that each time period has a corresponding behavior label; then the video data in each time period is feature extracted to obtain the corresponding video feature representation; finally, the extracted video features are matched and verified with the corresponding animal behavior labels to ensure that each video feature has a corresponding correct animal behavior label and they are aligned in time.

[0119] An intelligent monitoring system for abnormal animal status based on biobehavior, see attached Figure 1 , which is a network system consisting of a cloud server, a management terminal and one or more animal behavior data collection units, each part of which is interconnected and communicates with each other through a network, and is used to implement the aforementioned intelligent monitoring method for abnormal animal states based on biobehavior;

[0120] The cloud server is built with: a data input unit, an animal abnormal state monitoring model, and a result output unit;

[0121] The data input unit is used to receive animal behavior data;

[0122] The result output unit is used to output the animal status monitoring results;

[0123] The animal abnormal state monitoring model is used to extract rich feature representations from the spatiotemporal information in the video sequence of captured animal behavior data to perform animal behavior recognition and abnormal state monitoring, including a two-stream network SlowFast model and an abnormal score judgment model based on a deep learning framework, which are obtained after multiple rounds of training. The two-stream network SlowFast model includes a feature extraction module, a feature fusion module, and a classifier module that work in coordination with each other; specifically, the pre-trained animal abnormal state monitoring model and the pre-set time alignment threshold range are used to analyze and calculate the animal abnormal state to obtain a preliminary judgment result.

[0124] See attached Figure 3, the data input module is used to receive animal behavior trajectory data, including location information, movement path and other data; when the data input unit passes the received animal behavior data to the feature extraction module, the feature extraction module uses the SlowFast model to extract features from the pre-processed animal behavior data, and extracts the temporal features and spatial relationship feature data of the animal behavior;

[0125] The feature fusion module fuses the feature data extracted from the slow and fast paths to obtain a more comprehensive and accurate feature representation;

[0126] The classifier module uses the trained classification model to classify the fused feature representation data to determine whether the animal behavior type corresponds to a normal state or an abnormal state.

[0127] The animal behavior data collection unit includes: an audio and video camera and a data storage device; the audio and video camera is used to collect the monitored animal behavior data in the monitored area, including audio data and video data; the data storage device is used to store the audio data and video data and upload them to the cloud server;

[0128] The management terminal includes a computer or a mobile intelligent terminal device, such as a computer, a PAD, a smart phone, etc.

[0129] The above-mentioned method and system provided by the embodiment of the present invention, by using the trained animal abnormal state monitoring model and the abnormal score judgment model (Formula 3) to jointly monitor and analyze the animal behavior data in real time, after two judgments, can accurately identify the abnormal behavior of individual or group animals and the abnormal state corresponding to the behavior, including abnormal activities, illness, injury, social, psychological, etc., and can output the monitoring results and issue alarm information in time, so that the management personnel can take corresponding measures to intervene and deal with it, which can effectively protect the health and safety of animals and avoid possible losses.

[0130] In the data processing and analysis process of the embodiment of the present invention, by simultaneously processing the slow and fast paths, the SlowFast model can comprehensively consider the information captured by the two paths and fuse them together to obtain a more comprehensive and accurate feature representation; the multimodal feature fusion method of the model can better capture the temporal changes and spatial relationships of animal behavior, improve the accuracy and robustness of abnormal state monitoring, and enable the system to more effectively identify abnormal behaviors of animals.

[0131] The embodiment of the present invention can accurately identify the following abnormal behaviors of different types of animals and groups, and determine the trajectory points and time points where the abnormal behaviors occur as abnormal states:

[0132] The animal is exhibiting unusual behavior, including:

[0133] Lameness, paralysis: Animals may have difficulty walking or be completely unable to move due to defects in the spine, legs or feet, or damage to the nervous system.

[0134] Stereotyped behavior: Animals repeatedly perform certain abnormal activities in cages, such as holding their tails in their mouths for a long time, somersaults on cage bars, etc. This may be caused by a monotonous environment, lack of stimulation, or psychological stress.

[0135] Spinning, tumbling, and twisting: When an animal has problems with its balance, it may spin, somersault, or tumble while walking, which may be related to genetic problems, brain problems, or ear infections.

[0136] Behaviors that indicate a marked decrease in the animal's activity level, including:

[0137] Lethargy and immobility: Animals that appear unusually quiet and immobile may be a way to conserve energy after illness or injury.

[0138] Depression, unconsciousness, weakness: A decrease in the animal's overall activity level, which may be accompanied by changes in mental status, such as depression or loss of consciousness.

[0139] Whole body or partial tremors: The animal may experience involuntary shaking of the entire body or a part of the body, which may be caused by cold, neurological diseases, or defects in the areas of the brain that control behavior.

[0140] Other significant abnormal behaviors of animals, such as changes in appetite and drinking habits, abnormal sounds and postures, and changes in social behavior.

[0141] Example 2

[0142] See also Figures 4 to 5 The intelligent monitoring method and system for abnormal animal status based on biobehavior provided in this embodiment is an optimization based on Embodiment 1, and further simplifies the behavior trajectory of the animal using a point-line model.

[0143] See also Figure 5 The marked points and lines in the figure, the animal abnormal state intelligent monitoring method based on biobehavior also includes the following steps:

[0144] The training of animal abnormal state monitoring model is to use the SlowFast model to extract features and fuse space-time information of input animal behavior data (video trajectory, audio, etc.), so as to effectively capture the temporal changes and spatial relationships of animal behavior; through deep learning framework or neural network training, the extracted features are associated with the labels of animal behavior to achieve classification and recognition of different animal behavior patterns;

[0145] Animal abnormality feature extraction is to use the pre-trained animal abnormal state monitoring model to extract features from video data. The collected animal behavior video data is input into the animal abnormal state monitoring model to obtain the feature representation of each frame in the video; the extracted video features are matched with the corresponding animal behavior labels to construct a data set for model training; then the supervised learning method, including the cross entropy loss function and the back propagation algorithm, is used to train the model;

[0146] The feature extraction is to calculate the moving speed of the trajectory point through formula 1:

[0147]

[0148] In Formula 1, ν represents the moving speed, x1, y1 and x2, y2 represent the coordinates of trajectory points 1 and 2 respectively, and t1, t2 represent the timestamps of trajectory points 1 and 2 respectively. The motion characteristics of each trajectory point are obtained by calculating the moving speed of each trajectory point. The deep learning model is trained using the supervised learning method, and the extracted trajectory feature data is matched with the animal behavior label to achieve the classification and recognition of different behavior patterns.

[0149] Simplify the abnormal characteristics of animals, see Figure 5 The animal's behavior trajectory is simplified using a point-line model, with points simulating animals and lines simulating movement trajectories. The monitoring of action duration, moving distance, moving speed, number of repetitions, etc. is added to train the SlowFast model, so that the trained SlowFast model can judge whether the animal is in an abnormal state based on the changes and records of the animal's behavior trajectory for the purpose of intelligent identification.

[0150] To optimize the SlowFast model, the cross entropy loss function is used to optimize the model during the aforementioned model training process. The loss function is expressed as Formula 2:

[0151]

[0152] In Formula 2, y represents the true label, Represents the prediction result of the model, and N represents the number of samples. Through the back propagation algorithm, the parameters of the model are adjusted so that the loss function reaches the minimum value, thereby improving the accuracy and generalization ability of the model.

[0153] See attached Figure 4The optimized SlowFast model includes an encoder module, a decoder module, and a loss function module. The encoder module is used to encode the input animal behavior trajectory data into a feature representation in the latent space. The decoder module decodes the feature representation in the latent space into the original animal behavior trajectory data. The loss function module is used to calculate the reconstruction error between the encoder and the decoder to guide the training of the model. The following steps are used to optimize the SlowFast model:

[0154] Step 1: Input animal behavior trajectory data, encode it through the encoder module, and obtain feature representation in the latent space;

[0155] Step 2: Input the feature representation in the latent space into the decoder module and decode it into the original animal behavior trajectory data;

[0156] Step 3: Calculate the reconstruction error between the decoder output and the original input as the input of the loss function module;

[0157] Step 4: Optimize the encoder and decoder parameters through the back-propagation algorithm to minimize the reconstruction error;

[0158] Step 5: After training, the SlowFast model can be used to monitor abnormal behavior. When the reconstruction error exceeds the preset threshold, it is judged as abnormal behavior.

[0159] The slow and fast paths of this embodiment refer to two paths for capturing and processing image sequences at different frame rates in the video. The slow path is usually used to capture parts with slower movements and richer details, while the fast path is used to capture parts with faster movements and more obvious overall dynamics. In this embodiment, the SlowFast model is used to process these two paths simultaneously to obtain a more comprehensive and rich feature representation. Specifically, the slow path uses a lower frame rate, such as capturing 1 to 2 frames of images per second, to capture the details of slower movements, such as the slight movements and expression changes of animals. This ensures that enough detailed information is captured within a limited time, which helps to improve the detailed analysis of animal behavior. The fast path uses a higher frame rate, such as capturing 10 to 30 frames of images per second, to capture the overall dynamic changes of faster movements, such as large movements of animals such as running and jumping. This ensures that enough overall dynamic information is captured in a short time, which helps to grasp the overall behavior of animals.

[0160] The optimized method and system provided in this embodiment utilize the optimized animal abnormal state monitoring model to process the slow and fast paths in the video. By processing the slow and fast paths at the same time, the SlowFast model can comprehensively consider the information captured by these two paths and fuse them together to obtain a more comprehensive and rich feature representation. This multimodal feature fusion method can better capture the temporal changes and spatial relationships of animal behavior, improve the accuracy and robustness of abnormal state monitoring, and enable the system to more effectively identify the abnormal behavior of animals and then determine the corresponding abnormal state.

[0161] This embodiment models the animal's motion trajectory, simplifies the animal's behavior trajectory with a point-line model, simulates the animal with points, simulates the motion trajectory with lines, and adds monitoring of action duration, moving distance, moving speed, number of repetitions, etc. for training SlowFast; so that SlowFast can intelligently identify whether the animal is in an abnormal state based on the changes and records of the animal's behavior trajectory; the SlowFast model trained with a point-line model can only identify the abnormal stereotyped behavior of the animal, but cannot identify other abnormal behaviors and the abnormal states caused by them. Therefore, this embodiment greatly expands the application scope of the SlowFast model.

[0162] The method and system provided in the embodiments of the present invention can automatically generate a report on the abnormal behavior status of animals within a certain period of time as needed, so as to better monitor the physical condition of animals in real time and conduct effective management, assist management personnel to investigate the causes of abnormal situations and solve problems, and ensure the physiological and psychological health of animals; the method and system provided in the embodiments of the present invention can be widely used in industrial scenarios such as zoos, breeding farms, farms, scientific research laboratories, etc., to monitor animal behavior in real time and identify abnormal conditions in a timely manner, and has broad application prospects.

[0163] Example 3

[0164] See also Figure 6 to Figure 7 The animal abnormal state intelligent monitoring method and system based on biobehavioral science provided in this embodiment is a specific application and computer coding implementation example provided on the basis of the above embodiments, and is suitable for dynamic and real-time monitoring of abnormal behaviors of animals of different species, sizes, and growth stages. Based on the above embodiments, it is mainly divided into the following stages:

[0165] 1. Data collection and preprocessing stage

[0166] Step 1: Collect animal behavior data, including animal location information, movement path, action duration, etc.

[0167] In the living area of ​​the monitored animals, multiple high-definition cameras or infrared cameras are arranged to ensure that the animal's activity range is covered and the video image clarity and stability are ensured; audio and video data such as animal behavior trajectories are collected separately, and each frame in the video and the relevant animal behavior tags are marked with timestamps.

[0168] Step 2: Preprocess the collected data, including data cleaning, denoising, trajectory extraction and other operations, such as video format conversion (converting to .avi or .mp4 format), video frame extraction, video segmentation and noise removal;

[0169] 2. Feature extraction and model training stage

[0170] Step 3: Use SlowFast to analyze and learn the preprocessed data, and then use the pre-trained SlowFast deep learning model to extract features from the video data. First, input the video data into the SlowFast model to obtain the feature representation of each frame in the video; then match the extracted video features with the corresponding animal behavior labels to construct a training data set; then, use supervised learning methods such as cross entropy loss function and back propagation algorithm to train the deep learning model.

[0171] Step 4: Set abnormal behavior monitoring rules and threshold ranges, for example, to judge the abnormal state of animals based on indicators such as the animal's movement speed, frequency, and activity range. The rule for matching animal behavior tags adopted in this embodiment is the time alignment rule, which is one of the key steps in the process of matching animal behavior tags with video features. This rule ensures that the video features and the corresponding animal behavior tags are aligned in time, that is, each video feature should correspond to an animal behavior tag in the same period or continuous time period.

[0172] The matching process is as follows: First, the video data is segmented according to time, usually using a fixed-length time window, and each time period corresponds to a continuous time period in the video.

[0173] Then, for each time period, the corresponding animal behavior label is determined, which can be achieved through manual labeling or automatic labeling algorithms to ensure that each time period has a corresponding behavior label.

[0174] Again, feature extraction is performed on the video data in each time period to obtain the corresponding video feature representation.

[0175] Finally, the extracted video features are matched and verified with the corresponding animal behavior labels to ensure that each video feature has a corresponding correct animal behavior label and that they are aligned in time.

[0176] The SlowFast model in this embodiment is based on a deep learning framework. After training, the model can effectively capture the spatiotemporal information in video sequences and extract rich feature representations for subsequent behavior recognition and abnormal state monitoring. SlowFast is a two-stream neural network architecture for video understanding proposed by Facebook AI Research in 2019. The design of SlowFast is inspired by the biological visual system, which simulates the ability of human vision to process fast motion and details. The SlowFast network contains two parallel convolutional neural network (CNN) streams: a fast stream and a slow stream.

[0177] Fast Stream:

[0178] Fast stream runs at a higher frame rate, often twice as high, focusing on capturing motion information in the video.

[0179] The network structure of fast flow is simpler and has fewer parameters, so the computational cost is lower.

[0180] The purpose of fast stream is to capture subtle changes in time, which helps to identify fast actions.

[0181] Slow Flow:

[0182] The slow stream runs at a lower frame rate, usually half of it, and focuses on capturing the appearance information in the video; the network structure of the slow stream is more complex and has more parameters, so it has a higher representation ability; the purpose of the slow stream is to capture the overall changes in time, which helps to understand the context and content of the video; the two streams are fused at certain layers of the network, combining motion and appearance information to improve the performance of video understanding tasks such as video classification, action recognition, etc.

[0183] In this embodiment, the workflow of the trained SlowFast model is mainly as follows:

[0184] Input: The input video is divided into a fast stream and a slow stream. The fast stream processes video frames at a higher frame rate, and the slow stream processes video frames at a lower frame rate.

[0185] Feature extraction: The fast stream and the slow stream are respectively extracted with their own CNNs for spatiotemporal features.

[0186] Feature fusion: At certain layers of the network, features from the fast and slow streams are fused to combine motion and appearance information.

[0187] Multi-scale processing: By fusing features at different layers, SlowFast is able to capture information at different time scales.

[0188] Classification or regression: The last few layers of the network are used for classification (such as video category) or regression (such as temporal localization) and output the final prediction results.

[0189] Output: The output layer can be a softmax layer (for multi-class classification) or a linear layer (for regression tasks), outputting the probability or continuous value of each class.

[0190] The SlowFast network demonstrates strong performance in video understanding tasks through its unique two-stream design, especially when dealing with challenging fast actions and complex scenes.

[0191] The sample code used in this embodiment is as follows:

[0192]

[0193]

[0194]

[0195] loss=criterion(outputs,labels)

[0196] loss.backward()

[0197] optimizer.step()

[0198] running_loss+=loss.item()*inputs.size(0)

[0199] epoch_loss=running_loss / len(train_loader.dataset)

[0200] print('Epoch[{} / {}],Loss:{:.4f}'.format(epoch+1,num_epochs,epoch_loss))

[0201] #Behavior Recognition

[0202] defpredict(model,test_loader):

[0203] model.eval()

[0204] predictions=[]

[0205] with torch.no_grad():

[0206] for inputs in test_loader:

[0207] inputs = inputs.cuda()

[0208] outputs = model (inputs)

[0209] _,predicted=torch.max(outputs,1)

[0210] predictions.extend(predicted.cpu().numpy())

[0211] return predictions

[0212] 3. System integration and deployment phase

[0213] Integrate the above process code into the cloud server of the animal abnormal state intelligent monitoring system based on biobehavior, and then deploy the system to actual application scenarios, such as zoos, farms or scientific research laboratories; before deployment, perform system debugging and testing to ensure that the system can run stably and effectively identify the abnormal state of animals. The following are the specific implementation steps:

[0214] (1) Preparation:

[0215] Make sure you have installed the Python environment and the Django framework; prepare other dependent libraries required by the system, such as numpy, torch, etc.

[0216] (2) Create a Django project:

[0217] Create a new Django project using the command line tools provided by Django; create the required applications in the project, including front-end pages and back-end services.

[0218] (3) Write backend services:

[0219] Write backend service code in Django projects, including data processing, model loading, behavior recognition, and anomaly monitoring. Integrate behavior recognition models into backend services and provide corresponding API interfaces for frontend calls.

[0220] (4) Write the front-end page:

[0221] Use front-end technologies such as HTML, CSS, and JavaScript to write the user interface, including real-time video monitoring, abnormal status display, and early warning notification functions; call the API interface provided by the back-end service to achieve data interaction and display.

[0222] (5) Database configuration:

[0223] Configure the database and select the appropriate database type, including SQLite, MySQL, PostgreSQL, etc.

[0224] Define the data model, create database tables, and store the data required by the system, such as user information, video stream data, and behavior recognition results.

[0225] (6) Deploy to cloud server

[0226] Deploy the Django project to a cloud server or a local server; configure the server environment, including installing the Python environment, installing dependent libraries, setting up the database, etc.; use a Web server such as Nginx or Apache as a reverse proxy, and configure the forwarding of static files and dynamic requests.

[0227] (7) System testing:

[0228] Perform functional and performance tests on deployed systems to ensure that the systems can operate normally and meet user needs. Simulate user operations in actual scenarios to evaluate system stability and response speed.

[0229] (8) System maintenance:

[0230] Regularly monitor the system's operating status, process system logs and error information, and regularly update the system's software version and security patches to ensure the system's security and stability.

[0231] 4. Abnormal monitoring stage

[0232] Step 5: Monitor the extracted features for abnormal behavior and determine whether the animal is in an abnormal state.

[0233] Use the trained classification model to classify and identify the animal behaviors monitored in real time, input the real-time collected video data into the trained model, and obtain the classification results of the animal behaviors; set the appropriate threshold range and combine it with the confidence judgment model to make a two-step judgment on the classification results to determine whether there is abnormal behavior;

[0234] 5. Result output stage

[0235] Step 6: Output the type of abnormal behavior, time of occurrence, animal identity and other information;

[0236] Step 7: Visualize the frequency, trend and other information of abnormal behavior, such as through charts or statistical data;

[0237] 5. Early warning and intervention stage

[0238] Step 8: If an abnormal state of an animal is detected, corresponding early warning or intervention measures are triggered, such as sounding an alarm, notifying staff to intervene and deal with the situation, etc.

[0239] See also Figure 7 This embodiment is an example of applying the animal abnormal state intelligent monitoring method and system based on biobehavior provided in the previous embodiment to monitor bears in a wildlife park. On the basis of the previous embodiment, it also includes the following contents:

[0240] The audio and video data of the bears in the wildlife park are obtained, and the bears' activities in the wildlife park are recorded as a series of points, each of which represents the position of the bears at a certain moment; then, these points are connected with lines to form the bears' movement trajectory; at the same time, the duration of each action (such as the duration of each jump), the moving distance (such as the distance from one point to another), the moving speed (such as the distance moved per second), and the number of repetitions of the action (such as the number of consecutive jumps) are recorded; in this way, the simplified behavior trajectory data of the bears can be obtained and used to train the animal abnormal state monitoring model. The animal abnormal state monitoring model can learn the feature representations of different behavior patterns and judge whether the animal is in an abnormal state based on these features. For example, if the bear suddenly jumps frequently, for a long time and at a fast speed, it may mean that it is in an abnormal mood or has been stimulated by the outside world; the animal abnormal state monitoring model can detect abnormalities by analyzing these behavioral characteristics and issue an alarm in time so that the staff can conduct further observation and processing. This animal abnormal state monitoring model trained using a point-line model has good universality in the method of intelligently identifying animal abnormal states. It can provide a new networked monitoring and early warning method for farms, zoo management, etc., and help improve the health and safety level of animals in large numbers and over a large range.

[0241] The method and system provided by the embodiment of the present invention adopt multimodal feature fusion: the SlowFast model is used to simultaneously process the slow and fast paths in the video. The slow and fast paths refer to two paths that capture and process image sequences at different frame rates in the video. The slow path is usually used to capture parts with slower movements and richer details, while the fast path is used to capture parts with faster movements and more obvious overall dynamics. The slow path uses a lower frame rate, such as capturing 1 to 2 frames of images per second, to capture the details of slower movements, such as the slight movements and expression changes of animals; this ensures that sufficient detail information is captured within a limited time, which helps to improve the detailed analysis of animal behavior; while the fast path uses a higher frame rate, such as capturing 10 to 30 frames of images per second, to capture the overall dynamic changes of faster movements, such as large movements of animals such as running and jumping, thereby obtaining a more comprehensive and rich feature representation. This multimodal feature fusion method can better capture the temporal changes and spatial relationships of animal behavior, ensure that sufficient overall dynamic information is captured in a short time, help to make an overall comprehensive judgment on animal behavior and status, and improve the accuracy and robustness of abnormal state monitoring.

[0242] Example 4

[0243] See also Figures 8 to 9 This embodiment is an example of applying the intelligent animal abnormal state monitoring method and system based on biobehavior provided in the above embodiment to fox monitoring in a wildlife park. On the basis of the above embodiment, it further includes the following contents:

[0244] For the behavioral trajectory data of a fox in a wildlife park, first use cameras and other equipment to record its activity trajectories in different areas, such as running on the grass, walking by the lake, resting by the bench, etc.; these trajectory data can be marked as corresponding behavior categories, such as "running", "walking", "resting", etc.;

[0245] Supervised learning of models: Various behaviors of animals, such as walking, running, jumping, resting, etc., are associated with corresponding labels and the type of behavior is indicated; after these labeled data are collected and organized, they can be used as data sets for training supervised learning models; using these data sets, supervised learning models can be trained, such as convolutional neural networks (CNNs), where the input of the model is the animal behavior trajectory data and the output is the corresponding behavior label.

[0246] Through a large amount of labeled training data, the model can learn the characteristic representations of different behavioral patterns, thereby realizing the classification and identification of different animal behaviors.

[0247] Ultimately, the trained animal abnormal state monitoring model is applied to a variety of practical scenarios to achieve real-time monitoring and identification of animal behavior, provide a scientific basis for the management and display of wildlife parks, and promote animal protection and research.

[0248] For example, when the animal behavior data collection unit collects a video of a fox suddenly jumping around in a zoo, scratching its body frequently, and breathing rapidly, the animal abnormal state monitoring model determines that the fox has an abnormal behavior pattern, which may correspond to abnormal conditions such as being stimulated or infected with a disease. In this case, the system issues an early warning, notifying the staff to observe and examine it in time, and take appropriate treatment or isolation measures to ensure the health and safety of the fox.

[0249] In other embodiments, the method and system provided by the present invention can also monitor abnormal conditions of groups of animals in various scenarios, and can achieve abnormal state monitoring and alarm effects of animals based on biobehavior, which will not be introduced one by one here.

[0250] In summary, the method and system provided by the present invention use the SlowFast dual-stream network structure as the main model, construct a fusion monitoring model based on a deep learning framework, and combine animal behavior trajectory data for modeling and training, which can realize accurate identification of animal behavior and timely monitoring of abnormal states. While ensuring monitoring accuracy, a SlowFast behavior recognition method based on animal species recognition technology and abnormal behavior recognition for animal stereotyped behavior trajectories is designed. Compared with traditional monitoring methods, it has the advantages of high monitoring accuracy and high speed, and can better balance monitoring accuracy and processing speed. Its multimodal feature fusion method can better capture the temporal changes and spatial relationships of animal behavior, improve the accuracy and robustness of abnormal state monitoring, and enable the system to more effectively identify abnormal states of animals.

[0251] The above is a detailed description of the embodiments of the present invention in conjunction with the accompanying drawings, and the specific implementation methods herein are only used to help understand the method of the present invention. For those of ordinary skill in the art, according to the idea of ​​the present invention, changes and modifications can be made in the specific implementation methods and application scopes, so the present invention should not be understood as limiting the present invention.

Claims

1. A method for intelligent monitoring of abnormal animal states based on biobehavior, characterized in that: The steps include: S1. Build and train animal abnormal state monitoring model S1-1. Model construction Construct a two-stream network SlowFast model based on a deep learning framework as a model for judging and monitoring animal abnormal states based on biobehavior and changes in animal behavior trajectories; S1-2. Model training Using the existing associated trajectory feature dataset that has been annotated with normal and abnormal states of animals, the two-stream network SlowFast model based on the deep learning framework is trained. After multiple rounds of training, an animal abnormal state monitoring model is obtained. S2. Build an animal abnormal status monitoring network system S2-1, constructing a network system including a cloud server, a management terminal and at least one animal behavior data collection unit; S2-2, deploying the trained animal abnormal state monitoring model to the cloud server, and deploying the animal behavior data collection unit to the site where the monitored animal is located; S3. Intelligent monitoring of abnormal animal conditions S3-1: Collect and upload the monitored animal behavior data and upload the animal behavior data to the cloud server; S3-2: Preliminary judgment: The trained animal abnormal state monitoring model pre-processes the uploaded animal behavior data, analyzes and calculates the animal abnormal state, and obtains a preliminary judgment result. If the animal abnormal state monitoring model determines that it is abnormal, then: S3-3: judge again and output the result: input the preprocessed animal behavior data into the abnormal score judgment model formula 3, calculate the abnormal score score of each trajectory point, judge whether the abnormal score score falls within the preset normal state threshold range and output the final monitoring result: if the abnormal score exceeds the preset normal state threshold range, it is determined that the animal at the corresponding trajectory point is in an abnormal state; conversely, if the abnormal score does not exceed the preset normal state threshold, it is determined that the animal at the trajectory point is in a normal state: In Formula 3, score is the anomaly score of the trajectory point, e is the base of the natural logarithm, and z is the weighted sum of the feature vector of the trajectory point and the model parameters.

2. The method for intelligent monitoring of abnormal animal states based on biobehavior according to claim 1, characterized in that: It also includes the following steps: S4. Early warning and intervention of abnormal animal conditions Repeat steps S2-S3 to continuously monitor the animal behavior, and use the trained animal abnormal state monitoring model and formula 3 to jointly monitor and analyze the animal state in real time. If the animal is found to be in an abnormal state, early warning and intervention will be carried out.

3. The method for intelligently monitoring abnormal animal states based on biobehavior according to any one of claims 1 or 2, characterized in that: It also includes the following steps: S5: Animal abnormal status monitoring network system operation monitoring and update Regularly check the operating status of the animal abnormal state monitoring network system, process system logs and error information, and maintain safe and stable operation of the system; regularly use newly acquired and verified associated trajectory feature data sets to re-train the animal abnormal state monitoring model, and then redeploy it to the cloud server, and repeat steps S3-S4.

4. The method for intelligently monitoring abnormal animal states based on biobehavior according to any one of claims 1 or 2, characterized in that: Step S1 includes the following steps: S1-1. Construct a SlowFast model based on the deep learning TensorFlow or PyTorch framework. Based on biobehavior and changes in animal behavior trajectories, perform feature extraction and space-time information fusion processing on the collected animal behavior trajectory video data to obtain the temporal changes and spatial relationship laws of animal behavior. Through dual-path design and deep learning, associate the extracted features with the state labels of animal behavior to achieve classification and identification of the state of animals in different behavior modes. As an animal abnormal state monitoring model, the abnormal state of animals can be judged and monitored. S1-2, the animal abnormal state monitoring model is pre-trained with existing data, and the existing associated trajectory feature data set that has been labeled with the normal and abnormal states of animals is used to train the two-stream network SlowFast model based on the deep learning framework. After multiple rounds of training, the animal abnormal state monitoring model is obtained; S1-3. Model Optimization In the multi-round training process of step S1-2, the cross entropy loss function is used to optimize the animal abnormal state monitoring model, where the expression of the loss function is formula 2: In Formula 2, y represents the true label, Represents the prediction result of the model, and N represents the number of samples. Through the back propagation algorithm, the parameters of the model are adjusted so that the loss function reaches the minimum value, thereby improving the accuracy and generalization ability of the animal abnormal state monitoring model. S1-4. Model testing Using the trained animal abnormal state monitoring model, a small amount of video data collected by manual and animal behavior data acquisition units is input into the trained model. The model automatically classifies and identifies animal behaviors, outputs animal state judgment results, and compares the results with the animal state judgment results obtained by manual monitoring to test the accuracy of the animal abnormal state monitoring model.

5. The method for intelligent monitoring of abnormal animal states based on biobehavior according to claim 4, characterized in that: The step S1-2 uses existing data to pre-train the animal abnormal state monitoring model, and also includes the following steps: the step of processing the existing data into an existing associated trajectory feature data set is: extracting features from the pre-processed existing animal behavior trajectory data by manual annotation or by using a SlowFast model, and then matching and annotating the normal state or abnormal state of the animal based on the spatiotemporal information in biobehavior and animal video sequences, associating the extracted trajectory data set with the animal state, and obtaining an associated trajectory feature data set that distinguishes the normal state or abnormal state of the animal in biobehavior; During training, the supervised deep learning TensorFlow or PyTorch framework is used to train the SlowFast model, including the cross-entropy loss function and back-propagation algorithm, to match the extracted trajectory feature data with the animal behavior labels, so that the model has the ability to automatically extract, match, classify and recognize the state of different animal behavior data.

6. The method for intelligently monitoring abnormal animal states based on biobehavior according to any one of claims 1 or 2, characterized in that: Step S2 constructs an animal abnormal state monitoring network system, including the following steps: S2-1, constructing an animal abnormal state monitoring network system connected through a network, consisting of a cloud server, a management terminal and at least one animal behavior data collection unit; The cloud server is used to receive monitoring data of monitored animals, perform status analysis, and output monitoring results. The cloud server is built with: a data input unit, an animal abnormal status monitoring model, and a result output unit; the animal abnormal status monitoring model is equipped with a feature extraction module, a feature fusion module, and a classifier module; The management terminal is used by the management personnel to input control instructions and obtain monitoring results; The animal behavior data collection unit is used to collect the behavior trajectory data of the monitored animal on site, and includes: an audio and video camera, and a data storage device; S2-2, deploying the trained animal abnormal state monitoring model to the cloud server; deploying the animal behavior data collection unit to the site where the monitored animal is located; S2-3. The cloud server, the management terminal and the animal behavior data collection unit are interconnected and communicate with each other through the network.

7. The method for intelligently monitoring abnormal animal states based on biobehavior according to any one of claims 1 or 2, characterized in that: Step S3-2: Preliminary model judgment, further comprising the following steps: S3-2-1: Data preprocessing The collected animal behavior data (including location information, movement path, etc.), that is, the trajectory image data in the video sequence, is preprocessed, including cleaning and denoising, to extract the animal behavior-related trajectory feature data set; The feature extraction uses the following formula 1 to calculate the moving speed of the trajectory point: In formula 1, ν represents the moving speed, x1, y1 and x2, y2 represent the coordinates of trajectory points 1 and 2 respectively, and t1, t2 represent the timestamps of trajectory points 1 and 2 respectively. By calculating the moving speed of each trajectory point, the motion feature dataset of each trajectory point is obtained after accumulation, that is, the animal behavior associated trajectory feature dataset. S3-2-2: Simplified processing of abnormal characteristics of animals The animal's behavior trajectory is simplified using a point-line model, with points simulating the animal and lines simulating the movement trajectory. Monitoring data such as action duration, moving distance, moving speed, and number of repetitions are added to train AI. This allows AI to intelligently identify whether the animal is in an abnormal state based on changes in the animal's behavior trajectory and records. S3-2-3: Animal abnormal feature data extraction Using the feature extraction module in the pre-trained animal abnormal state monitoring model, feature extraction is performed on the input data to extract the temporal characteristics and spatial relationship feature data of the animal behavior; S3-2-4: Animal abnormality feature data fusion Using the feature fusion module in the pre-trained animal abnormal state monitoring model, the feature data extracted from the slow and fast paths are fused to obtain feature representation data; S3-2-5: Animal feature representation data classification Utilizing the classifier module in the pre-trained animal abnormal state monitoring model, the trained classification model is used to classify the fused feature representation data one by one according to the time alignment rules and the preset abnormal state time alignment threshold range, and the animal behavior state label is matched with the feature representation data. If a feature representation data falls into the preset abnormal state time alignment threshold range according to the time alignment rule, it is matched as abnormal state data and marked as animal behavior state. Otherwise, it is matched as normal state data and marked as normal state.

8. The method for intelligent monitoring of abnormal animal states based on biobehavior according to claim 1, characterized in that: The animal behavior data is the individual behavior data or group behavior data of the monitored animals that occurs continuously, including: animal behavior video and audio data recorded in time sequence.

9. An intelligent monitoring system for abnormal animal status based on biobehavior, characterized in that: It is a network system consisting of a cloud server, a management terminal and at least one animal behavior data acquisition unit, each part of which is interconnected and communicates through a network, and is used to implement the intelligent monitoring method for abnormal animal status based on biobehavior as described in any one of claims 1-8; The cloud server is built with: a data input unit, an animal abnormal state monitoring model, and a result output unit; The data input unit is used to receive animal behavior data; The animal abnormal state monitoring model is used for animal behavior recognition and abnormal state monitoring; The result output unit is used to output the animal status monitoring results; The animal behavior data collection unit includes: an audio and video camera and a data storage device; The management terminal includes a computer or a mobile intelligent terminal device.

10. The animal abnormal state intelligent monitoring system based on biobehavior according to claim 9 is characterized in that: The animal abnormal state monitoring model includes a two-stream network SlowFast model based on a deep learning framework and an abnormal score judgment model, wherein the two-stream network SlowFast model includes a feature extraction module, a feature fusion module, and a classifier module; The feature extraction module uses the SlowFast model to perform feature extraction on the preprocessed animal behavior data to extract the temporal characteristics and spatial relationship feature data of the animal behavior; The feature fusion module fuses the feature data extracted from the slow path and the fast path to obtain feature representation data; The classifier module uses the trained classification model to classify the fused feature representation data to determine whether the animal behavior type corresponds to a normal state or an abnormal state.

Images