Pet behavior anomaly visual detection device based on attention mechanism
By using a visual detection device for abnormal pet behavior based on an attention mechanism, real-time collection and analysis of pet video stream data is achieved, and key behavioral features are enhanced, enabling high-precision detection and real-time early warning of abnormal pet behavior. This solves the problems of time-consuming manual judgment and low detection accuracy in existing technologies.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- GUANGZHOU ABOVE THE CLOUD TECHNOLOGY CO LTD
- Filing Date
- 2026-03-04
- Publication Date
- 2026-06-05
AI Technical Summary
Existing pet behavior detection solutions rely on manual judgment, which is time-consuming and labor-intensive, has low detection accuracy, and is easily affected by background environmental interference, making it impossible to achieve real-time, high-precision abnormal behavior recognition.
The system employs an attention-based visual detection device, including a high-definition infrared camera, a data preprocessing module, an attention mechanism feature extraction module, and a deep neural network classification module. It collects video stream data in real time, enhances the pet's limb and posture features through the attention mechanism, combines deep learning to detect abnormal behavior, and provides real-time early warning through audible and visual alarms and wireless communication.
It achieves automated, real-time detection of abnormal pet behavior, with a total response time of ≤0.3s, a detection accuracy of ≥95%, and a low false positive rate, thus improving the level of intelligence in pet care.
Smart Images
Figure CN122157352A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of visual detection device technology, specifically to a visual detection device for abnormal pet behavior based on attention mechanisms. Background Technology
[0002] As people's living standards improve, pets are gradually becoming important family members, and their health and behavior are receiving much attention. However, due to busy work schedules, users are often unable to spend real time with their pets. When pets exhibit abnormal behavior (such as persistent restlessness, convulsions, refusal to eat, or hitting obstacles), it is difficult to detect in time, which may delay the opportunity for health intervention.
[0003] Existing pet behavior detection solutions have the following main shortcomings:
[0004] Relying on manual judgment: Most solutions only use cameras to record videos of pet activities, requiring users to manually review the videos to judge whether the behavior is abnormal, which is time-consuming and labor-intensive, and cannot achieve real-time monitoring;
[0005] Low detection accuracy: Some simple detection devices use inter-frame difference method or basic feature matching, which can only identify whether the pet is moving, but cannot distinguish between "normal activity" and "abnormal behavior", and are prone to misjudgment (such as misjudging the rapid movement of the pet while playing as abnormal).
[0006] Insufficient capture of key features: Traditional visual detection models do not pay enough attention to key features such as pet limb details and posture changes, and are easily affected by background environment (such as furniture, changes in lighting), leading to missed detection of abnormal behaviors.
[0007] To address the aforementioned issues, this invention proposes a visual detection device for abnormal pet behavior based on an attention mechanism. By enhancing the extraction of key behavioral features through an attention model and combining it with deep learning, the device achieves high-precision, real-time anomaly detection, thereby improving the level of intelligence in pet care. Summary of the Invention
[0008] The purpose of this invention is to provide a visual detection device for abnormal pet behavior based on attention mechanisms, so as to solve the problems mentioned in the background art.
[0009] To achieve the above objectives, the present invention provides the following technical solution: comprising the following modules: a visual acquisition module, a data preprocessing module, an attention mechanism feature extraction module, a behavior classification module, and an anomaly warning module and a power supply module electrically connected to the behavior classification module respectively; the visual acquisition module is used to acquire video stream data of the pet's activity area in real time and transmit the video stream data to the data preprocessing module; the data preprocessing module is used to perform noise reduction processing, keyframe extraction, and image size normalization on the received video stream data, and output preprocessed frame image data; the attention mechanism feature extraction module has a built-in improved convolutional attention network and spatial attention mechanism fusion model, used to extract pet behavior features from the frame image data, focusing on strengthening the weights of key features such as pet limb joints, posture changes, and activity trajectories, and outputting a high-dimensional behavior feature vector; the behavior classification module has a built-in trained deep neural network classification model, used to receive the high-dimensional behavior feature vector, compare it with a preset pet normal behavior feature library and abnormal behavior feature library, and output the behavior classification result; the anomaly warning module is used to output "abnormal" in the behavior classification module. Upon the result, an audible and visual alarm signal is triggered, and an abnormal warning message and corresponding behavioral fragment are pushed to the user's mobile terminal via a wireless communication unit; the power supply module is used to provide a stable operating voltage for the above modules.
[0010] Preferably, the visual acquisition module includes at least two high-definition infrared cameras. The shooting angle of the cameras is adjustable and supports switching between daytime color imaging and nighttime infrared imaging. The frame rate of the cameras is not less than 25fps and the resolution is not less than 1920×1080. The cameras are used to cover the main areas where pets are active and avoid blind spots in the shooting.
[0011] Preferably, the data preprocessing module uses an algorithm combining Gaussian filtering and median filtering to remove salt-and-pepper noise and Gaussian noise from the video stream; the key frame extraction uses an adaptive extraction strategy based on the inter-frame difference method. When the pixel difference between adjacent frames exceeds a preset threshold, it is determined to be a key frame and extracted. The threshold range is set to 5-15 pixel grayscale values.
[0012] Preferably, the fusion model of the attention mechanism feature extraction module includes a channel attention submodule and a spatial attention submodule. The channel attention submodule obtains the channel features of the frame image through global average pooling and global max pooling, processes them through a shared fully connected layer, and outputs channel attention weights through a sigmoid activation function to weight the feature importance of different channels. The spatial attention submodule compresses the channel dimension of the feature map after channel attention weighting, and outputs spatial attention weights through a convolutional layer and a sigmoid activation function to enhance the spatial features of the area where the pet is located and suppress the interference features of the background area. The fusion model applies the channel attention weights and spatial attention weights to the original feature map through element-wise multiplication to output enhanced key behavioral features.
[0013] Preferably, the deep neural network classification model of the behavior classification module is an improved model based on ResNet50. The improved model adds the high-dimensional behavior feature vector output by the attention mechanism feature extraction module before the fully connected layer of ResNet50. In the model training process, the cross-entropy loss function and Adam optimizer are used. The training dataset includes labeled images of at least 10 normal pet behaviors (walking, eating, sleeping, etc.) and at least 8 abnormal pet behaviors (continuous barking, twitching, hitting obstacles, etc.), and the number of labeled images for each behavior is not less than 500.
[0014] Preferably, the wireless communication unit of the abnormal warning module supports Wi-Fi, Bluetooth or 4G / 5G communication protocols. The warning information includes the type of abnormal behavior, detection time, and abnormal behavior segment (duration 10-30s). The sound decibel of the audible and visual alarm signal is adjustable (range 50-80dB), and the light color is red flashing (frequency 1-2Hz).
[0015] Preferably, it also includes a storage module, which is electrically connected to the data preprocessing module and the behavior classification module. The storage module is used to store the original video stream data, the preprocessed frame image data, the behavior classification results and abnormal behavior segments. The storage capacity is not less than 128GB and supports cyclic overwrite storage (when the storage volume reaches 90%, the earliest non-abnormal data is automatically deleted).
[0016] Preferably, it also includes a human-computer interaction module, which is electrically connected to the behavior classification module. The human-computer interaction module includes a touch screen and physical buttons, which are used to display real-time pet activity images and historical behavior statistics, and support users to manually set abnormal behavior judgment thresholds, warning methods, camera parameters, etc.
[0017] Preferably, the power supply module includes a rechargeable lithium battery (capacity not less than 5000mAh) and an external power interface (supporting 5V / 2A DC power supply). The power supply module also has a built-in power detection unit that pushes a low power reminder to the user's mobile terminal when the lithium battery power is below 20%.
[0018] Compared with the prior art, the beneficial effects of the present invention are:
[0019] 1. This invention features automation and real-time performance, eliminating the need for manual video viewing. The device automatically completes feature extraction, behavior classification, and anomaly warning, with a total response time of ≤0.3s, achieving real-time monitoring.
[0020] 2. High-precision detection of this invention: By strengthening the key behavioral features of pets (limbs, posture) through the attention mechanism and suppressing background interference, combined with the improved ResNet50 model, the accuracy of abnormal behavior detection is ≥95% with a low false positive rate. Attached Figure Description
[0021] Figure 1 This is a flowchart illustrating the steps of the visual detection device for abnormal pet behavior based on the attention mechanism of the present invention. Detailed Implementation
[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0023] Please see Figure 1 The present invention provides a technical solution: This visual detection device for abnormal pet behavior based on attention mechanism includes a visual acquisition module, a data preprocessing module, an attention mechanism feature extraction module, a behavior classification module, an abnormal warning module, a storage module, a human-computer interaction module, and a power supply module. Each module is electrically connected through a PCB circuit board and is encapsulated in an ABS flame-retardant shell. The shell is provided with a camera mounting hole, a touch display window, an audible and visual alarm hole, and a power interface.
[0024] The visual acquisition module employs two Hikvision DS-2CD3T25D-I3 high-definition infrared cameras, with a resolution of 1920×1080, a frame rate of 25fps, and support for 0.01Lux low-light color imaging and infrared night vision (effective night vision distance 10m). The two cameras are mounted on either side of the device, and the shooting angles are manually adjustable (horizontal adjustment range -45° to 45°, vertical adjustment range -30° to 30°), covering key areas such as the living room and balcony where pets are active, avoiding blind spots. The cameras connect to the data preprocessing module via a USB 3.0 interface for real-time transmission of video stream data.
[0025] Data preprocessing module: The NVIDIA Jetson Nano embedded development board is used as the core processor of the data preprocessing module. Its Quad-core ARM Cortex-A57 CPU can meet the real-time data processing requirements.
[0026] Noise reduction: An algorithm combining Gaussian filtering (convolution kernel size 5×5, standard deviation 1.2) and median filtering (kernel size 3×3) is implemented using the OpenCV library to first remove Gaussian noise from the video stream and then eliminate salt-and-pepper noise.
[0027] Keyframe extraction: Based on the inter-frame difference method, the difference in grayscale values between two adjacent frames is calculated. When the average difference exceeds 10 (pixel grayscale value, range 0-255), it is determined to be a keyframe and extracted to avoid redundant frames consuming computing power.
[0028] Size normalization: The extracted keyframes are scaled to 224×224 pixels (to adapt to the input requirements of the subsequent ResNet50 model) and pixel values are normalized (pixel values are mapped to the range of 0-1).
[0029] Attention mechanism feature extraction module: An improved CBAM attention fusion model is implemented on the Jetson Nano development board using the PyTorch framework. This model integrates channel attention submodules and spatial attention submodules.
[0030] Channel attention submodule: For the preprocessed 224×224×3 (RGB three-channel) frame image, global average pooling (outputting 1×1×3 feature map) and global max pooling (outputting 1×1×3 feature map) are performed respectively. The two pooling results are concatenated and input into a shared fully connected layer (64 neurons in the first layer and 3 neurons in the second layer). The sigmoid activation function outputs the attention weights of the three channels (e.g., "limb feature channel weight 0.8, background channel weight 0.2"). The features of each channel are weighted by element-wise multiplication.
[0031] Spatial Attention Submodule: The channel-weighted feature map is compressed by channel dimension (the maximum value of each channel is taken, and a 224×224×1 feature map is output). The feature map dimension is restored to 224×224×3 by a 3×3 convolutional layer (stride 1, padding 1). The spatial attention weights are output by the Sigmoid activation function (high weight for the pet area and low weight for the background area). The key spatial features are strengthened by element-wise multiplication.
[0032] The final output is a high-dimensional behavior feature vector of 224×224×3, which is then transmitted to the behavior classification module.
[0033] Behavior classification module: Based on the ResNet50 model, an attention feature vector input interface is added after the 5th convolutional block (conv5_x) of ResNet50. The feature vector output by the attention mechanism feature extraction module is fused with the inherent features of ResNet50, and then input into the fully connected layer (128 neurons) and the classification layer (outputting 19 classes, corresponding to 11 normal behaviors and 8 abnormal behaviors).
[0034] Model Training: The training dataset uses the publicly available PetAction dataset and a self-made labeled dataset, containing 19 classes of behavior, with 500 labeled images (224×224 resolution) for each class. The training process uses the cross-entropy loss function and the Adam optimizer (initial learning rate 0.001, decaying to 1 / 10 of the original rate every 10 epochs), with 50 training epochs. The final model achieves a classification accuracy of 96.2% on the test set.
[0035] Real-time classification: After receiving the attention feature vector, the behavior classification module classifies the data using the trained model. The classification time for a single class is ≤50ms. When the classification result is "normal", only the result is stored; when it is "abnormal", the abnormal warning module is triggered.
[0036] Anomaly warning module: includes an audible and visual alarm unit and a wireless communication unit.
[0037] Audible and visual alarm unit: It adopts a buzzer (model SFM-27) and an LED light (red). The buzzer decibel is set to 65dB (which can be adjusted through the human-machine interaction module), and the LED light flashes at a frequency of 1.5Hz. When an abnormality is detected, the alarm will continue for 30 seconds.
[0038] Wireless communication unit: It adopts an ESP8266 Wi-Fi module (supporting 802.11b / g / n protocols) and communicates with the user's mobile APP (based on Android / iOS development) via MQTT protocol to push early warning information (including abnormality type, such as "twitching"; detection time, such as "2024-05-20 14:30:25"; 15s abnormal behavior segment). After receiving the information, the APP will emit a vibration reminder.
[0039] Storage module
[0040] A Samsung 128GB microSD card (UHS-I U3 level) is used, connected to Jetson Nano via the SD card interface. The storage strategy is as follows: real-time storage of raw video stream (H.264 compressed format, 2Mbps bitrate) and pre-processed frame images (PNG format); storage of behavior classification results (CSV format, including time and behavior type); when the SD card storage reaches 90%, the oldest non-abnormal data is automatically deleted (abnormal data of the last 30 days is retained).
[0041] Human-computer interaction module: It adopts a 2.4-inch touch screen (320×240 resolution) and 3 physical buttons (power button, settings button, mute button):
[0042] The display screen shows real-time pet activity footage (split-screen display of dual camera feeds) and historical behavior statistics (pie chart of the number of normal / abnormal occurrences and the percentage of abnormal types in the past 24 hours).
[0043] The touchscreen allows you to set: anomaly detection thresholds (e.g., inter-frame difference threshold of 8-15), alert methods (APP push only / sound and light + APP), and camera parameters (exposure time, infrared mode).
[0044] Physical buttons are used for emergency power control, quick access to the settings interface, and temporarily disabling the audible and visual alarms.
[0045] Power supply module: Includes a 5000mAh 18650 rechargeable lithium battery (capacity density 2000mAh / g) and a DC 5V / 2A external power interface (Type-C).
[0046] The lithium battery is managed for charging and discharging through a power management chip (TI BQ24195), and can support the device to work independently for 8 hours when fully charged;
[0047] When powered by an external power source, it simultaneously charges the lithium battery.
[0048] The built-in power detection unit (collects battery voltage via ADC) sends a low battery reminder to the APP via wireless communication unit when the voltage is below 3.6V (corresponding to 20% battery level).
[0049] Device Working Process
[0050] Start-up and initialization: When the power is turned on (external power supply or lithium battery), the device starts automatically, the visual acquisition module starts shooting, and the data preprocessing module, attention feature extraction module, and behavior classification module complete initialization (loading model parameters and setting thresholds).
[0051] Real-time acquisition and preprocessing: The camera transmits video streams in real time to the data preprocessing module, which performs noise reduction, key frame extraction, and size normalization, and outputs preprocessed frame images.
[0052] Feature extraction and classification: The attention mechanism feature extraction module applies channel and spatial attention weights to the preprocessed frame image and outputs a high-dimensional feature vector; the behavior classification module inputs the feature vector into the improved ResNet50 model and outputs the behavior classification result.
[0053] Anomaly detection and early warning: If the classification result is "normal", the result and related data are stored; if it is "abnormal", an audible and visual alarm is triggered, and an early warning message and abnormal segment are pushed to the APP via Wi-Fi.
[0054] User interaction and data management: Users can view real-time images and statistics through the touchscreen, receive alerts and view abnormal segments through the APP; the storage module manages data according to a cyclical strategy to avoid storage overflow.
[0055] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.
[0056] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.
Claims
1. A visual detection device for abnormal pet behavior based on an attention mechanism, characterized in that, It includes the following modules: a visual acquisition module, a data preprocessing module, an attention mechanism feature extraction module, a behavior classification module, and an anomaly warning module and a power supply module that are electrically connected to the behavior classification module respectively; The visual acquisition module is used to acquire video stream data of the pet's activity area in real time and transmit the video stream data to the data preprocessing module. The data preprocessing module is used to denoise, extract keyframes, and normalize image size of the received video stream data, and output preprocessed frame image data. The attention mechanism feature extraction module has a built-in improved convolutional attention network and spatial attention mechanism fusion model, which is used to extract pet behavior features from the frame image data, focusing on strengthening the weights of key features such as pet limb joints, posture changes, and activity trajectories, and outputting high-dimensional behavior feature vectors. The behavior classification module has a built-in trained deep neural network classification model, which is used to receive the high-dimensional behavior feature vectors, compare them with preset pet normal behavior feature databases and abnormal behavior feature databases, and output behavior classification results. The abnormal warning module is used to trigger an audible and visual alarm signal when the behavior classification module outputs an "abnormal" result, and push abnormal warning information and corresponding behavior segments to the user's mobile terminal through the wireless communication unit. The power supply module is used to provide a stable operating voltage for the above modules.
2. The visual detection device for abnormal pet behavior based on an attention mechanism according to claim 1, characterized in that, The visual acquisition module includes at least two high-definition infrared cameras. The shooting angle of the cameras is adjustable and supports switching between daytime color imaging and nighttime infrared imaging. The frame rate of the cameras is no less than 25fps and the resolution is no less than 1920×1080. The cameras are used to cover the main areas where pets are active and avoid blind spots in the shooting.
3. The visual detection device for abnormal pet behavior based on an attention mechanism according to claim 1, characterized in that, The data preprocessing module employs a noise reduction algorithm combining Gaussian filtering and median filtering to remove salt-and-pepper noise and Gaussian noise from the video stream. The keyframe extraction uses an adaptive extraction strategy based on inter-frame difference. When the pixel difference between adjacent frames exceeds a preset threshold, it is determined to be a keyframe and extracted. The threshold range is set to 5-15 pixel grayscale values.
4. The visual detection device for abnormal pet behavior based on an attention mechanism according to claim 1, characterized in that, The fusion model of the attention mechanism feature extraction module includes a channel attention submodule and a spatial attention submodule. The channel attention submodule obtains the channel features of the frame image through global average pooling and global max pooling. After processing by a shared fully connected layer, the channel attention weights are output through the Sigmoid activation function to weight the feature importance of different channels. The spatial attention submodule compresses the channel dimension of the feature map after channel attention weighting, and outputs spatial attention weights through convolutional layers and sigmoid activation functions to enhance the spatial features of the area where the pet is located and suppress the interference features of the background area; the fusion model applies the channel attention weights and spatial attention weights to the original feature map by element-wise multiplication, and outputs enhanced key behavioral features.
5. The visual detection device for abnormal pet behavior based on an attention mechanism according to claim 1, characterized in that, The deep neural network classification model of the behavior classification module is an improved model based on ResNet50. The improved model adds the high-dimensional behavior feature vector output by the attention mechanism feature extraction module before the fully connected layer of ResNet50. In the model training process, the cross-entropy loss function and Adam optimizer are used. The training dataset includes labeled images of normal pet behavior and abnormal pet behavior, with no less than 500 labeled images for each behavior.
6. The visual detection device for abnormal pet behavior based on an attention mechanism according to claim 1, characterized in that, The wireless communication unit of the abnormal warning module supports Wi-Fi, Bluetooth or 4G / 5G communication protocols. The warning information includes the type of abnormal behavior, detection time, and abnormal behavior segment (duration 10-30s). The sound decibel of the audible and visual alarm signal is adjustable (range 50-80dB), and the light color is red flashing (frequency 1-2Hz).
7. The visual detection device for abnormal pet behavior based on an attention mechanism according to claim 1, characterized in that, It also includes a storage module, which is electrically connected to the data preprocessing module and the behavior classification module. The storage module is used to store the original video stream data, the preprocessed frame image data, the behavior classification results and the abnormal behavior segments. The storage capacity is not less than 128GB and supports cyclic overwrite storage (when the storage volume reaches 90%, the oldest non-abnormal data is automatically deleted).
8. The visual detection device for abnormal pet behavior based on an attention mechanism according to claim 1, characterized in that, It also includes a human-computer interaction module, which is electrically connected to the behavior classification module. The human-computer interaction module includes a touch screen and physical buttons, which are used to display real-time pet activity images and historical behavior statistics, and support users to manually set abnormal behavior judgment thresholds, warning methods, camera parameters, etc.
9. The visual detection device for abnormal pet behavior based on an attention mechanism according to claim 1, characterized in that, The power supply module includes a rechargeable lithium battery (capacity not less than 5000mAh) and an external power interface (supporting 5V / 2A DC power supply). The power supply module also has a built-in power detection unit, which pushes a low power reminder to the user's mobile terminal when the lithium battery power is lower than 20%.