Pet behavior monitoring method, apparatus, and medium

Abstract

The application relates to the technical field of monitoring, and discloses a pet behavior monitoring method, equipment and medium. The method is applied to an edge device and comprises the following steps: obtaining a latest target video frame in a monitoring video; performing pet skeleton key point detection on the target video frame to obtain skeleton key point features; identifying pet posture features of a pet to be monitored based on the skeleton key point features; performing furniture semantic detection on the target video frame to obtain furniture semantic features; performing behavior intention and behavior risk assessment on the pet to be monitored based on the skeleton key point features, the pet posture features and the furniture semantic features, to obtain a behavior assessment result; and performing a corresponding monitoring operation based on the behavior assessment result. The embodiments of the application can improve the real-time performance and joint understanding performance of pet behavior monitoring.

Description

Technical Field

[0001] This application relates to the field of monitoring technology, and in particular to a method, device and medium for monitoring pet behavior. Background Technology

[0002] In related technologies, pet monitoring solutions mainly rely on uploading video streams to the cloud for analysis and processing. For example, some products capture images through cameras and then send the data to a cloud server via Wi-Fi or 4G networks, where cloud-based AI models are used to identify and judge pet behavior. However, this type of pet monitoring solution suffers from high response latency and limited functionality, making it difficult to provide real-time warnings for sudden dangerous behaviors and unable to identify the pet's intentions, spatial goals, or behavioral trends. Summary of the Invention

[0003] The purpose of this application is to provide a method, device, and medium for monitoring pet behavior, aiming to improve the real-time performance and joint understanding performance of pet behavior monitoring.

[0004] This application provides a pet behavior monitoring method applied to an edge device, including: Acquire the latest target video frame from the monitoring video; the target video frame contains a video frame of the pet to be monitored within the monitoring area; Pet skeletal key point detection is performed on the target video frame to obtain skeletal key point features; Based on the skeletal key point features, the pet posture features of the pet to be monitored are identified; Furniture semantic detection is performed on the target video frame to obtain furniture semantic features; Based on the skeletal key point features, the pet posture features, and the furniture semantic features, the behavioral intentions and behavioral risks of the pet to be monitored are assessed, and the behavioral assessment results are obtained. Based on the behavioral assessment results, corresponding monitoring operations are performed.

[0005] In some embodiments, the pet skeletal keypoint detection on the target video frame includes: The target video frame is processed to crop out the corresponding pet region feature map; The pet target region image is input into a pre-trained keypoint detection model to predict the position of each pet skeletal keypoint in the pet target region image, thereby obtaining a set of skeletal keypoint coordinates. The skeletal keypoint coordinate set is mapped to the image coordinate system of the target video frame to obtain the skeletal keypoint features.

[0006] In some embodiments, identifying the pet posture features of the pet to be monitored based on the skeletal keypoint features includes: The skeletal keypoint features are normalized to obtain normalized keypoint features. Based on the spatial geometric relationship between the normalized keypoint features, the pose feature vector used to represent the pose is extracted from the normalized keypoint features. The posture feature vector is input into a pre-trained posture classification model to predict the posture category probability that maps the posture feature vector to the corresponding pet posture, and the pet posture feature is generated based on the posture category probability.

[0007] In some embodiments, performing furniture semantic detection on the target video frame includes: The target video frame is processed to obtain tensor features that meet preset conditions; Multi-scale feature extraction is performed on the tensor features to obtain a tensor feature map; The tensor feature map is input into a pre-trained furniture semantic detection model to perform feature fusion and enhancement on the tensor feature map, thereby obtaining the furniture semantic features.

[0008] In some embodiments, the assessment of the behavioral intent and behavioral risk of the pet under monitoring based on the skeletal keypoint features, the pet's posture features, and the furniture semantic features includes: Based on the location information of both the skeletal key point features and the furniture semantic features, the spatial geometric relationship features between the pet to be monitored and the furniture are calculated. Based on a preset intent inference rule base and risk assessment rule base, rule matching and logical reasoning are performed on the pet posture features, furniture semantic features and spatial geometric relationship features to obtain the behavior assessment result; the intent inference rule base and the risk assessment rule base are both rule sets configured based on a set of condition judgment functions.

[0009] In some embodiments, the rules in the intent inference rule base are defined as triggering the corresponding behavioral intent label and confidence level when conditions related to pet posture, spatial proximity, relative orientation and / or target furniture type are met; the rules in the risk assessment rule base are defined as triggering the corresponding risk level, risk type and risk score when conditions related to target furniture type, target spatial area, target posture and / or target movement state are met.

[0010] In some embodiments, when multiple rules in the intent inference rule base or multiple rules in the risk assessment rule base are triggered simultaneously, the rules are weighted and fused based on preset weights or confidence levels to generate the behavior assessment result.

[0011] In some embodiments, performing corresponding monitoring operations based on the behavior assessment results includes: Based on the behavior evaluation results, the target response strategy is determined from the preset response strategy mapping table; Perform at least one monitoring operation corresponding to the target response strategy; The response strategy mapping table defines at least the following strategies: For furniture involving the first risk level and the intent being the first behavior type, the monitoring operation is to record a behavior log in the storage area and update the activity statistics of the pet to be monitored; For furniture involving the second risk level and the intent being the second behavior type, the monitoring operation is to generate a prompt notification containing a snapshot of the current screen and a risk description, and push it to the associated user terminal; For furniture involving the third risk level and the intent being the third behavior type, the monitoring operation is to activate the local sound and light alarm device for real-time early warning and simultaneously send an emergency alarm message containing a real-time video stream access link to the user terminal; the risk levels of the first risk level, the second risk level, and the third risk level increase sequentially, and the activity levels of the behaviors corresponding to the first behavior type, the second behavior type, and the third behavior type increase sequentially.

[0012] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described pet behavior monitoring method.

[0013] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described pet behavior monitoring method.

[0014] The beneficial effects of this application are as follows: By processing data on an edge device and assessing pet behavior intent and risk based on skeletal keypoint features, pet posture features, and furniture semantic features extracted from a single video frame, corresponding monitoring operations are executed based on the behavior assessment results. Therefore, by deploying the pet behavior monitoring method on an edge device and extracting data from a single video frame, data processing is closer to the data source and the accumulated latency caused by temporal modeling is avoided, significantly reducing data transmission latency and enabling real-time response to pet behavior. Simultaneously, by comprehensively utilizing pet skeletal keypoint features, pet posture features, and furniture semantic features, a multi-dimensional and more refined behavior assessment system is constructed, providing richer and more accurate behavior assessment results. This enables more targeted monitoring operations, enhancing risk warning capabilities and joint understanding performance. Attached Figure Description

[0015] Figure 1 This is a diagram illustrating the application environment of the pet behavior monitoring method provided in the embodiments of this application.

[0016] Figure 2 This is a flowchart of the pet behavior monitoring method provided in the embodiments of this application.

[0017] Figure 3 This is a flowchart of a method for detecting key points of a pet skeleton in a target video frame, provided in an embodiment of this application.

[0018] Figure 4 This is a flowchart of a method for identifying the pet posture characteristics of a pet to be monitored, provided in an embodiment of this application.

[0019] Figure 5 This is a flowchart of a method for performing furniture semantic detection on target video frames provided in an embodiment of this application.

[0020] Figure 6 This is a flowchart of a method for assessing the behavioral intent and behavioral risk of a pet to be monitored, provided in an embodiment of this application.

[0021] Figure 7 This is a flowchart of a method for performing corresponding monitoring operations based on behavior assessment results, provided in an embodiment of this application.

[0022] Figure 8 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0024] It should be noted that although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first," "second," etc., in the specification, claims, and drawings are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0025] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application. Furthermore, the information, data, and signals involved in the embodiments of this application are all authorized by relevant parties or have been fully authorized by all parties, and the collection, use, and processing of related data comply with the relevant laws, regulations, and standards of the relevant countries and regions.

[0026] In traditional pet monitoring systems, video streams are uploaded to the cloud for analysis and processing, leading to a significant increase in response latency. This latency stems from the inherent characteristics of network transmission and cloud computing, making it difficult for the system to meet the demands of real-time alerts. Specifically, the cloud processing architecture cannot promptly identify a pet's intentions, spatial goals, or behavioral trends, thus impacting key system performance indicators, including real-time response capabilities and the accuracy of behavior recognition. The latency introduced during network transmission causes delays in detecting sudden dangerous behaviors, preventing the triggering of effective intervention mechanisms within critical time windows. For example, in a home environment, when a cat is active in the living room and gradually approaches an open window, the monitoring system needs to immediately detect this intention and assess the risk. In this scenario, video data is transmitted to a cloud server via Wi-Fi, processed, and the results are returned—a process that causes a delay in monitoring. Furthermore, the system cannot generate an alert signal at the critical moment when the pet is about to jump out of the window, thus failing to effectively prevent potential dangerous events. Specifically, the cloud model's recognition of pet posture features relies on the upload of the complete video stream, and network fluctuations further exacerbate processing latency, causing the judgment of behavioral intentions to deviate from the actual moment of occurrence.

[0027] If the aforementioned issues are not addressed, the system will be unable to intervene promptly in cases of sudden dangerous behavior, potentially leading to pet injury or property damage. Consequently, the limitation of its single-function architecture restricts the system's adaptability to complex behavioral scenarios and reduces the overall reliability of monitoring. Furthermore, the cloud-based processing architecture cannot integrate real-time analysis of pet posture characteristics and their relationship to furniture space, resulting in a lack of multi-dimensional data support for behavioral risk assessment, highlighting the urgency of resolving this problem.

[0028] Based on this, embodiments of this application provide a pet behavior monitoring method, apparatus, device, and medium. By performing data processing on an edge device and assessing pet behavior intent and risk based on skeletal key point features, pet posture features, and furniture semantic features extracted from a single video frame, this method effectively overcomes the defects of response delay and insufficient intent recognition, thereby improving response speed and enhancing risk warning capabilities.

[0029] Figure 1 This diagram illustrates the application environment of the pet behavior monitoring method provided in this embodiment. (See attached diagram.) Figure 1 This method is applied to a pet behavior monitoring system. The system includes a terminal 110 and an edge device 120. The terminal 110 and edge device 120 are connected via a network. The terminal 110 can be a desktop terminal or a mobile terminal; a mobile terminal can be a mobile phone, tablet, and / or laptop. The edge device 120 can be a router, routing switch, integrated access device (IAD), multiplexer, and / or various metropolitan area network (MAN) and wide area network (WAN) access devices. The terminal 110 sends the latest target video frame from the monitoring video to the edge device 120. The edge device 120 acquires the latest target video frame from the monitoring video, performs pet skeletal keypoint detection on the target video frame to obtain skeletal keypoint features, identifies the pet's posture features based on the skeletal keypoint features, performs furniture semantic detection on the target video frame to obtain furniture semantic features, and assesses the pet's behavioral intent and behavioral risk based on the skeletal keypoint features, pet posture features, and furniture semantic features to obtain behavioral assessment results. Based on the behavioral assessment results, corresponding monitoring operations are executed. The target video frame refers to a video frame containing the pet to be monitored within the monitoring area.

[0030] See Figure 2 In one embodiment, a pet behavior monitoring method is provided, wherein the execution subject of the method is an edge device, including but not limited to steps S201 to S206.

[0031] Step S201: Obtain the latest target video frame from the monitored video.

[0032] Edge devices refer to computing and storage environments located close to the data source, such as smart cameras, local servers, or dedicated edge computing devices. Processing data on edge devices reduces latency when data is transmitted to the cloud, improving response speed and data security.

[0033] The target video frame is a video frame containing the pet to be monitored within the monitored area. It can be understood that a target video frame is a single image frame extracted or selected from a continuous monitoring video stream. This video frame is the basis for subsequent analysis and processing, and its content contains real-time status information of the pet to be monitored within the monitored area.

[0034] The target video frame is a video frame containing the pet to be monitored within the monitored area. Acquiring the latest target video frame from the monitored video involves periodically capturing video frames for analysis from a continuous video stream or under specific conditions. For example, an edge device can be configured to acquire the latest video frame from a camera at a fixed frame rate and process it. Alternatively, the device can employ a simple motion detection algorithm to capture video frames only when significant motion is detected within the monitored area, reducing unnecessary processing.

[0035] Step S202: Perform pet skeleton key point detection on the target video frame to obtain skeleton key point features.

[0036] Pet skeletal keypoint detection is a computer vision technique used to identify and locate specific key points on a pet's body, such as the head, joints, and extremities. These key points allow for the construction of a skeletal model of the pet, enabling analysis of its posture and movement.

[0037] Skeletal keypoint features refer to the set of key points obtained through pet skeletal keypoint detection technology, used to describe a pet's body structure and posture. These features are usually represented in the form of coordinates or relative positions and serve as the foundational data for subsequent posture recognition and behavior assessment.

[0038] Performing pet skeletal keypoint detection on the target video frame involves identifying key body parts of the pet from the captured video frame, providing foundational data for subsequent posture and behavior analysis. For example, a general image feature extraction algorithm can be used to directly process the entire target video frame to identify any potential animal skeletal keypoints.

[0039] Step S203: Based on the skeletal key point features, identify the pet posture features of the pet to be monitored.

[0040] Pet posture features refer to abstract descriptions that characterize a pet's body posture, extracted and identified based on skeletal key point features. Examples include standing, sitting, jumping, and scratching. These features reflect the pet's current physical state and potential behavioral tendencies.

[0041] Based on the skeletal keypoint features, identifying the pet's posture characteristics involves transforming discrete skeletal keypoint information into a semantic description of the pet's posture. Specifically, a set of simple geometric rules can be preset. For example, by comparing the distances or relative angles between specific keypoints, it can be directly determined whether the pet is in a basic posture such as standing, sitting, lying down, or prone. For instance, when the vertical distance between the pet's head keypoint and its rump keypoint is detected to be less than a certain threshold, it can be identified as a sitting or lying posture.

[0042] Step S204: Perform furniture semantic detection on the target video frame to obtain furniture semantic features.

[0043] Furniture semantic detection is a computer vision technology used to identify furniture objects within a monitored area and understand their semantic information, such as category, location, and function. Examples include sofas, tables, and curtains.

[0044] Furniture semantic features refer to attribute information such as category, location, and size of furniture obtained through furniture semantic detection technology, used to describe the furniture within a monitored area. These features help in understanding the interaction between the pet being monitored and the monitored area.

[0045] Performing furniture semantic detection on the target video frame involves identifying the types and locations of furniture within the monitored area to understand the environmental context in which the pet is located. For example, image segmentation techniques based on color or texture features can be used to divide different regions in the video frame into different semantic categories, and common furniture such as sofas, tables, or chairs can be identified based on preset color or texture templates.

[0046] Step S205: Based on skeletal key point features, pet posture features, and furniture semantic features, conduct a behavioral intention and behavioral risk assessment of the pet to be monitored, and obtain the behavioral assessment results.

[0047] Behavioral intent and risk assessment refers to the process of comprehensively judging a pet's impending behavioral tendencies and the potential harm caused based on various information such as the pet's posture, environment, and furniture. Intent assessment focuses on what the pet wants to do, while risk assessment focuses on the potential negative impact of the behavior.

[0048] Behavioral assessment results refer to the conclusions drawn after assessing behavioral intent and risk. These results typically include information such as the pet's behavioral intent label, risk level, confidence level, or score, used to guide subsequent monitoring operations.

[0049] Based on the key features of the skeletal structure, the pet's posture features, and the semantic features of the furniture, a behavioral intention and risk assessment of the pet under monitoring is conducted. This involves a comprehensive analysis of the pet's behavior and environmental information to determine its potential intentions and risk levels. For example, a decision table can be constructed that lists how a specific posture (such as "scratching posture") is directly mapped to a preset behavioral intention (such as "damaging furniture") and risk level (such as "medium risk") when it is near specific furniture (such as "sofa").

[0050] Step S206: Based on the behavioral assessment results, perform the corresponding monitoring operations.

[0051] Monitoring operations refer to the corresponding measures taken based on the results of behavioral assessments. These operations may include logging, sending notifications, triggering alarms, or adjusting device settings, all aimed at intervening in or recording pet behavior.

[0052] Based on the behavioral assessment results, corresponding monitoring operations are performed, which involves taking practical actions according to the assessment results to achieve effective management of pet behavior. For example, when the assessment results indicate that the pet exhibits potentially risky behavior, the implementing entity can uniformly execute a preset monitoring operation, such as sending a general text message to the user's terminal informing them that "the pet may exhibit abnormal behavior." Alternatively, the implementing entity can display the assessment results on a local screen, awaiting manual intervention from the user. This single or manual response approach may not meet the differentiated and real-time needs under different risk levels and behavioral intentions.

[0053] The following example will provide a more detailed explanation of the above technical solution: Suppose user A has installed a smart edge camera in their home to monitor their pet dog's activities in the living room area. The dog sometimes scratches the sofa, and user A wants to be able to detect and intervene in a timely manner.

[0054] First, the executing unit continuously acquires the latest target video frame from the monitoring video. For example, one frame is captured every second and used as the current target video frame for analysis. This target video frame is ensured to contain the pet dog to be monitored.

[0055] Next, the target video frame is subjected to pet skeletal keypoint detection. For example, the executing entity uses its built-in image processing module to analyze the entire target video frame, identify multiple keypoints on the pet dog's body, such as the head, shoulders, elbows, hips, knees, and paws, and record the two-dimensional coordinates of these keypoints, thereby obtaining skeletal keypoint features.

[0056] Subsequently, based on these skeletal key point features, the pet dog's posture characteristics are identified. For example, by analyzing the relative positions and angles between key points, the operator determines whether the pet dog is currently in a "standing," "sitting," "lying," or "paw-raised" posture. When a key point of a pet dog's front paw is detected to be significantly raised and its center of gravity is shifted to one side, it may be identified as a "paw-raised posture."

[0057] Simultaneously, furniture semantic detection is performed on the target video frame. The executing agent identifies the main furniture in the living room area, such as "sofa," "coffee table," and "TV cabinet," and determines their positions and boundaries in the video frame, thereby obtaining furniture semantic features.

[0058] Then, the executing entity comprehensively analyzes the skeletal keypoint features, pet posture features, and furniture semantic features to assess the pet's behavioral intent and risk. For example, when the executing entity detects that the dog is in a "paw-raising posture" and the position of its paw keypoint is very close to the furniture semantic feature area of ​​the "sofa," the executing entity will make a judgment based on preset rules. At this time, the executing entity may infer that the dog's "behavioral intent" is "scratching the sofa" and assess its "behavioral risk" as "medium risk." Thus, the corresponding behavioral assessment result is generated.

[0059] Finally, based on the behavioral assessment results, corresponding monitoring operations are executed. For example, if the assessment results show that the pet dog is "scratching the sofa" and the risk is "medium risk," the executing entity can immediately send a notification to user A's mobile phone stating, "Your pet dog may be scratching the sofa, please be aware." By processing and assessing in real time on edge devices, this method can quickly respond to potential destructive behaviors of pets, effectively solving the problems of high response latency and limited functionality in traditional cloud solutions. It can also identify the pet's specific behavioral intentions and risks, thereby achieving intelligent, real-time monitoring and intervention of pet behavior.

[0060] Based on the above examples, the technical concept of this application demonstrates a significant technical contribution. Compared to existing solutions that upload video streams to the cloud for analysis and processing, this application deploys the pet behavior monitoring method on edge devices. This deployment method brings data processing closer to the data source, significantly reducing data transmission latency and enabling real-time responses to pet behavior. For example, when a dog raises its paw towards a sofa, the edge device can instantly perform skeletal keypoint detection, pose recognition, furniture semantic detection, and behavioral intent and risk assessment, and quickly issue an alarm notification, rather than waiting for the data to be uploaded to the cloud for processing before returning. This effectively avoids the shortcomings of traditional solutions that miss intervention opportunities due to high latency.

[0061] Furthermore, this application constructs a multi-dimensional and more refined behavior assessment system by comprehensively utilizing key features of pet skeletons, pet posture features, and semantic features of furniture. In the example above, it not only detects the movement of the pet dog, but also accurately infers the "behavioral intention" of "scratching the sofa" and the "behavioral risk" of "medium risk" by identifying its "paw-raising posture" and combining it with its "spatial proximity" to the "sofa." This ability to deeply integrate multiple features for logical reasoning enables this application to provide richer and more accurate behavior assessment results, thereby enabling more targeted monitoring operations, such as sending notifications containing specific behavioral descriptions, rather than just general "abnormal behavior" alerts.

[0062] See Figure 3In one embodiment, the method for detecting key points of a pet skeleton in a target video frame includes, but is not limited to, steps S301 to S303.

[0063] Step S301: Process the target video frame to crop out the corresponding pet region feature map.

[0064] Step S302: Input the pet target region image into the pre-trained keypoint detection model to predict the position of each pet skeletal keypoint in the pet target region image and obtain the set of skeletal keypoint coordinates.

[0065] Step S303: Map the set of skeletal keypoint coordinates to the image coordinate system of the target video frame to obtain skeletal keypoint features.

[0066] The target video frame is processed to crop out the corresponding pet region feature map. This aims to separate the pet from the complex video frame background, thereby narrowing the area of ​​interest for subsequent keypoint detection, reducing interference from irrelevant information, and improving processing efficiency. Specifically, this processing can employ object detection algorithms, such as the YOLO (You Only Look Once) series of models or Faster R-CNN models based on convolutional neural networks (CNNs), to identify and locate the pet target in the video frame. Then, based on the detected pet bounding box or segmentation mask, the portion containing only the pet is extracted from the original target video frame as the pet region feature map. Alternatively, background subtraction or motion detection techniques can be used to initially identify dynamic regions in the video frame that may contain the pet, and then these regions can be further refined to determine the pet region feature map.

[0067] This method inputs a pet target region image into a pre-trained keypoint detection model to predict the positions of various skeletal keypoints within the pet target region image. The aim is to perform detailed analysis of the cropped pet region feature map to accurately identify each skeletal keypoint. The pre-trained keypoint detection model is a deep learning model trained on a large dataset of pet images with skeletal keypoint annotations. Approaches such as bottom-up (e.g., OpenPose) or top-down (e.g., AlphaPose, HRNet) can be used. This model learns the structural features and interrelationships of various parts of the pet's body, thus outputting the two-dimensional coordinates of each skeletal keypoint (e.g., head, neck, shoulder, elbow, wrist, hip, knee, ankle, tail root, etc.) within the pet target region image after inputting the image.

[0068] Mapping the set of skeletal keypoint coordinates to the image coordinate system of the target video frame aims to transform the skeletal keypoint position information from the local (pet target area image) coordinate system to the global (target video frame) coordinate system. This allows subsequent behavior evaluation to accurately incorporate the pet's position information across the entire monitored area. Specifically, based on the coordinates and size of the top-left corner of the original bounding box used when cropping the pet target area image, the coordinates of each keypoint in the skeletal keypoint coordinate set can be translated and / or scaled to obtain its absolute position in the target video frame. These skeletal keypoint coordinates in the target video frame image coordinate system, along with their confidence scores and other information, constitute the skeletal keypoint features.

[0069] This application's solution achieves efficient and accurate pet skeletal keypoint detection through staged processing of the target video frame. First, instead of directly performing complex keypoint detection on the entire target video frame, the execution subject preprocesses the target video frame to crop out a pet target region image containing only the pet to be monitored. Then, the cropped pet target region image is input into a pre-trained model specifically optimized for pet skeletal keypoint detection. This model utilizes features learned from a large amount of pet data to accurately predict the positions of various skeletal keypoints of the pet within a local region, thus obtaining a set of skeletal keypoint coordinates. Finally, to effectively correlate this local keypoint information with other information in the target video frame (such as furniture semantic features), the execution subject precisely maps the set of skeletal keypoint coordinates in the local coordinate system back to the image coordinate system of the original target video frame, thereby generating complete skeletal keypoint features. This step-by-step, focused processing method makes the skeletal keypoint detection process more efficient and accurate, providing high-quality input data for subsequent pet pose recognition, behavioral intent, and risk assessment, thereby improving the performance and reliability of pet behavior monitoring.

[0070] The following is a concrete example to illustrate this. When the executing agent obtains a target video frame containing a pet, it first uses a deep learning-based object detection model, such as YOLOv11, to infer the bounding boxes of all pets in the video frame. Assume a pet bounding box is detected with its top-left corner coordinates (x, y), width w, and height h. Based on this bounding box information, the executing agent will crop the corresponding image region from the original target video frame, forming the pet target region image. Next, this pet region feature map is input into a pre-trained HRNet (High-Resolution Network) model, which has been trained on a large-scale pet skeletal keypoint dataset (e.g., images containing various pets such as cats and dogs in various poses). The HRNet model outputs the pixel coordinates of each skeletal keypoint (such as the head, limb joints, tail, etc.) in the pet target region image; for example, the coordinates of the head keypoint in the cropped image are (x_head_crop, y_head_crop). Finally, to transform these local coordinates into the global coordinate system of the original target video frame, the execution entity adds the coordinates of the top-left corner of the cropped region to the cropped image coordinates of each keypoint. That is, the coordinates of the head keypoints in the target video frame are (x_head_crop + x, y_head_crop + y). After this mapping, all keypoints form the skeletal keypoint features used for subsequent processing.

[0071] By employing the aforementioned technical solution, this application avoids computationally intensive detection directly on the entire complex target video frame when detecting key points on a pet's skeleton, effectively reducing computational resource consumption. By first cropping the pet target area image, the detection process can focus more on the pet itself, significantly reducing interference from background noise and irrelevant objects on key point detection, thereby greatly improving the accuracy and robustness of skeletal key point detection. This phased and refined processing method not only improves detection efficiency but also provides more accurate and reliable basic data for subsequent pet pose recognition, behavioral intent, and risk assessment based on skeletal key point features, thus enhancing the overall performance and real-time nature of pet behavior monitoring.

[0072] See Figure 4 In one embodiment, the method for identifying the pet posture characteristics of the pet to be monitored includes, but is not limited to, steps S401 to S403.

[0073] Step S401: Normalize the skeletal keypoint features to obtain normalized keypoint features.

[0074] Step S402: Based on the spatial geometric relationship between the normalized keypoint features, extract the pose feature vector used to represent the pose from the normalized keypoint features.

[0075] Step S403: Input the posture feature vector into the pre-trained posture classification model to predict the posture category probability that maps the posture feature vector to the corresponding pet posture, and generate pet posture features based on the posture category probability.

[0076] Normalizing skeletal keypoint features can eliminate differences in scale and location, making them more universal and comparable, thereby reducing the impact of background environment and individual pet differences on pose recognition. Specifically, normalization can be achieved using various methods. For example, all keypoint coordinates can be translated relative to a reference point on the pet's skeleton (such as the center point of the pet's torso or a specific joint), and then scaled by dividing by a feature length of the skeleton (such as torso length, diagonal length, or the average distance from all keypoints to the centroid). Another approach is to independently perform Z-score normalization on each dimension of the keypoint coordinates to achieve zero mean and unit variance, or to perform Min-Max normalization, mapping the coordinate values ​​to a preset range, such as [0, 1] or [-1, 1].

[0077] Based on the spatial geometric relationships between the features of each normalized keypoint, a posture feature vector is extracted to represent the posture. This allows for the extraction of core information that effectively distinguishes different postures from the normalized keypoints, forming a compact and expressive vector representation. This can be achieved by calculating various geometric properties between keypoints. For example, angles of specific joints (such as elbows, knees, and necks) can be calculated, directly reflecting the degree of limb bending; relative distances between keypoints can be calculated, such as the distance between the head and torso, or the distance between the extremities and the ground; angles between vectors composed of multiple keypoints, or the relative position vectors of keypoints relative to the pet's body coordinate system, can also be calculated. Furthermore, dimensionality reduction techniques such as Principal Component Analysis (PCA) can be used to extract low-dimensional posture feature vectors from high-dimensional normalized keypoint coordinates, preserving the most important posture information and removing redundancy.

[0078] The pose feature vectors are input into a pre-trained pose classification model to predict the probability of mapping the pose feature vectors to the corresponding pet poses, and pet pose features are generated based on the pose class probabilities. This can be achieved by using a machine learning model to classify the extracted pose feature vectors, thereby determining the specific pose type of the pet and its probability. The pre-trained pose classification model can be a variety of machine learning or deep learning models. For example, traditional machine learning classifiers such as Support Vector Machines (SVM), Random Forests, or Gradient Boosting Trees can be used. These models learn the mapping relationship between pose feature vectors and pose classes during the training phase. Alternatively, deep learning models such as Multilayer Perceptrons (MLP), Convolutional Neural Networks (CNN), or Recurrent Neural Networks (RNN) can be used. These models can automatically learn more complex patterns and feature representations from the pose feature vectors and output the probability distribution of each pose class. Based on these pose class probabilities, the class with the highest probability is usually selected as the final pet pose feature, or the probability distribution of all pose classes can be used as a richer pet pose feature.

[0079] To overcome the challenges of using raw skeletal keypoint features in pose recognition, this application first normalizes the skeletal keypoint features. This process eliminates scale and positional differences caused by external factors such as pet size, distance from the camera, and shooting angle, ensuring consistency and comparability of keypoint data across different scenes and pets. This ensures that subsequent pose analysis focuses on the intrinsic geometric features of the pet's body configuration, rather than being influenced by the external environment. Based on this, the execution agent further extracts a pose feature vector representing the pose based on the spatial geometric relationships between the normalized keypoint features. These geometric relationships, such as joint angles, relative limb lengths, and relative positions between keypoints, are the essential manifestation of pet pose and can effectively capture and distinguish different body poses. In this way, the raw, potentially noisy and redundant keypoint data is transformed into a compact and semantically rich pose feature vector. Subsequently, this pose feature vector is input into a pre-trained pose classification model. This model has learned the complex mapping relationship between the pose feature vector and various pet pose categories through a large amount of pet pose data. After receiving the pose feature vector, the model can predict the probability distribution of the vector corresponding to different pet pose categories. Ultimately, the implementing entity generates accurate pet posture features based on these posture category probabilities. Through this series of steps, the proposed solution can systematically transform raw and volatile skeletal keypoint data into stable and meaningful pet posture information, significantly improving the accuracy and robustness of posture recognition and providing high-quality input for subsequent behavioral intent and behavioral risk assessment.

[0080] The following is a concrete example. Suppose that through pet skeletal keypoint detection, a set of skeletal keypoint features of the pet in the target video frame has been obtained, represented as a series of two-dimensional coordinate points, such as 17 keypoints including the head, neck, shoulders, elbows, wrists, hips, knees, ankles, and tail root. First, these skeletal keypoint features are normalized. For example, the geometric center of all keypoints can be calculated as the pet's reference point, and all keypoint coordinates can be translated relative to this reference point. Next, the length of the pet's torso (e.g., the distance from the left shoulder to the right hip) is calculated as a scale factor, and then all translated keypoint coordinates are divided by this scale factor to obtain the normalized keypoint features. Second, based on the spatial geometric relationships between these normalized keypoint features, a pose feature vector is extracted. For example, the angle between the pet's elbow joints (shoulder-elbow-wrist) and knee joints (hip-knee-ankle) can be calculated, as well as the relative height and horizontal offset of the head keypoints relative to the center point of the torso. These angles and relative position information together constitute the pose feature vector used to characterize the posture. Finally, the posture feature vector is input into a pre-trained posture classification model. This model can be a deep learning-based convolutional neural network that has learned a large number of posture feature vectors for different pet postures (such as sitting, standing, lying down, running, scratching, etc.) during the training phase. After receiving the posture feature vector, the model outputs a vector containing the probability of each posture category. For example, the probability of "sitting" is 0.85, the probability of "standing" is 0.10, and the probability of "lying down" is 0.05. Based on this probability distribution, the agent can determine that "sitting" is the current pet posture feature, with a confidence score of 0.85.

[0081] Through the above technical solution, this application effectively addresses the problem that raw skeletal keypoint features are easily interfered with by external factors in pose recognition. Specifically, normalizing the skeletal keypoint features eliminates the interference of individual pet differences, shooting distance, and angle changes on pose recognition, allowing subsequent pose analysis to focus more on the pet's true body configuration, thereby improving the stability and comparability of pose features. Extracting pose feature vectors based on the spatial geometric relationships between normalized keypoint features effectively captures the essential features of pet poses, avoiding redundancy and noise caused by directly using raw coordinates, and significantly improving the expressive power and discriminative power of the features. Inputting the extracted pose feature vectors into a pre-trained pose classification model can accurately and efficiently identify the specific pose category and its confidence level of the pet to be monitored, providing accurate and robust pose information input for subsequent behavioral intent and behavioral risk assessment. This significantly improves the adaptability and accuracy of pet behavior monitoring in complex environments, enabling the implementing entity to more reliably understand the pet's behavioral state, thus laying a solid foundation for behavioral intent and risk assessment.

[0082] See Figure 5 In one embodiment, the method for performing furniture semantic detection on the target video frame includes, but is not limited to, steps S501 to S503.

[0083] Step S501: Process the target video frame to obtain tensor features that meet preset conditions.

[0084] Step S502: Perform multi-scale feature extraction on the tensor features to obtain the tensor feature map.

[0085] Step S503: Input the tensor feature map into the pre-trained furniture semantic detection model to perform feature fusion and enhancement on the tensor feature map to obtain furniture semantic features.

[0086] Processing the target video frame involves initially altering the original video frame, converting it into a numerical representation suitable for subsequent deep learning model processing, i.e., tensor features. Pre-processing conditions can include image resizing, color space conversion, normalization, and denoising to eliminate redundant information or noise in the original data and standardize the data format, thereby improving the efficiency and accuracy of subsequent feature extraction. For example, the target video frame can be resized to a fixed size using image scaling algorithms (such as bilinear interpolation or bicubic interpolation), then pixel value normalization can be performed (e.g., scaling pixel values ​​from the 0-255 range to the 0-1 range), and then converted to the tensor format required by deep learning frameworks such as PyTorch or TensorFlow. Alternatively, the target video frame can be grayscaled to reduce the number of channels, followed by histogram equalization to enhance image contrast, and then tensor transformation can be performed to adapt to the input feature requirements of a specific furniture semantic detection model.

[0087] Multi-scale feature extraction of tensor features captures features from different receptive fields (i.e., image region sizes) to address the diversity of furniture target sizes, shapes, and textures. By extracting features at different scales, the overall structure of large furniture and the fine texture of small furniture can be better identified, thereby improving the robustness of furniture semantic detection. For example, a multi-layer convolutional structure in a convolutional neural network (CNN) can be used, where each convolutional kernel has a different receptive field, thus extracting features at different scales at different depths. For instance, a Feature Pyramid Network (FPN) structure can be used to fuse feature maps from different levels through top-down and bottom-up paths, generating multi-scale feature maps with rich semantic information. Alternatively, dilated convolutions or pooling layers can be combined with convolutional layers to expand the receptive field of the convolutional kernels without increasing computational cost or reducing spatial resolution, thereby achieving multi-scale feature capture.

[0088] Pre-trained furniture semantic detection models are trained on a large amount of furniture image data and are capable of recognizing different furniture categories and their semantic information. After inputting multi-scale feature maps into the model, it further fuses (integrates information from different scales) and enhances (improves the expressive power of features through nonlinear transformations, attention mechanisms, etc.) these features, ultimately outputting high-level, semantically meaningful furniture features. For example, the pre-trained model can be based on semantic segmentation architectures such as U-Net and DeepLabV3+. These models typically contain an encoder-decoder structure, where the encoder extracts features, and the decoder restores spatial resolution and performs feature fusion, ultimately outputting pixel-level furniture category predictions. These predictions can serve as furniture semantic features. Alternatively, it can be based on instance segmentation models such as Mask R-CNN. This model, while detecting furniture targets, can also generate an accurate mask for each furniture instance and output its category and confidence score. Internal modules such as the Feature Pyramid Network (FPN) and Region Proposal Network (RPN) are responsible for feature fusion and enhancement.

[0089] The proposed solution for furniture semantic detection first preprocesses the original target video frames, transforming them into tensor features that meet preset conditions. This preprocessing step aims to unify the data format, remove noise, and lay the foundation for subsequent deep feature extraction. Subsequently, multi-scale feature extraction is performed on these tensor features. By capturing image information at different scales, it effectively addresses the significant differences in the size, shape, and texture of furniture targets in the monitored scene, ensuring that key visual information is fully captured, whether for large furniture or small items. This acquisition of multi-scale features significantly enhances the expressive power of the features and their adaptability to complex scenes. Finally, these tensor feature maps rich in multi-scale information are input into a pre-trained furniture semantic detection model. This model utilizes its knowledge learned from large amounts of data to deeply fuse and enhance the input features, thereby extracting high-level, highly robust furniture semantic features. These finely processed furniture semantic features, combined with skeletal keypoint features and pet posture features, can more accurately assess the spatial relationship between the monitored pet and the furniture, thereby improving the accuracy of behavioral intent and risk assessment, enabling efficient and accurate pet behavior monitoring even in edge device environments.

[0090] As a specific implementation method, when performing furniture semantic detection on target video frames, the target video frames can first be resized, for example, uniformly scaled to 512x512 pixels, and pixel values ​​normalized to convert them into floating-point tensors between 0 and 1, serving as tensor features that meet preset conditions. Next, a Feature Pyramid Network (FPN) structure based on ResNet-50 as the backbone network can be used for multi-scale feature extraction. This FPN structure can extract feature maps from different layers of ResNet-50 (e.g., C3, C4, C5 layers) and fuse them through top-down paths and lateral connections to generate multi-scale feature maps such as P3, P4, P5, P6, and P7, which are the tensor feature maps. Subsequently, these multi-scale tensor feature maps are input into a pre-trained DeepLabV3+ model. This DeepLabV3+ model is pre-trained on large semantic segmentation datasets such as COCO or ADE20K and fine-tuned for furniture categories. The model's internal Spatial Pyramid Pooling (ASPP) module and decoder module perform feature fusion and enhancement on the input tensor feature map, ultimately outputting pixel-level furniture category prediction maps, such as sofas, tables, chairs, cabinets, etc., along with the confidence score for each pixel belonging to the corresponding furniture category. These prediction maps and confidence scores together constitute the furniture semantic features.

[0091] Using the above technical solution, when performing furniture semantic detection on target video frames, preprocessing is first performed to obtain tensor features that meet preset conditions, effectively unifying the data format and reducing noise interference. Subsequently, through multi-scale feature extraction, visual information of furniture in different sizes and complex scenes can be comprehensively captured, significantly enhancing the expressive power of the features and their robustness to environmental changes. Finally, these rich multi-scale features are input into a pre-trained furniture semantic detection model for deep fusion and enhancement, greatly improving the accuracy and reliability of furniture semantic features. This allows subsequent pet behavior intent and risk assessment to be based on more accurate furniture context information, thereby improving the accuracy and reliability of pet behavior monitoring, especially in practical applications where edge device resources are limited and scenarios are varied, enabling more effective identification of potential inappropriate pet behaviors or dangerous situations.

[0092] See Figure 6 In one embodiment, the method for assessing the behavioral intent and behavioral risk of the pet to be monitored includes, but is not limited to, steps S601 to S602.

[0093] Step S601: Based on the location information of both skeletal keypoint features and furniture semantic features, calculate the spatial geometric relationship features between the pet to be monitored and the furniture; Step S602: Based on the preset intent inference rule base and risk assessment rule base, rule matching and logical reasoning are performed on pet posture features, furniture semantic features and spatial geometric relationship features to obtain behavior assessment results.

[0094] Both the intent inference rule base and the risk assessment rule base are rule sets configured based on a set of conditional judgment functions. They can be understood as knowledge sets used for advanced semantic understanding and risk assessment of pet behavior. Their function is to infer the pet's behavioral intent and potential risk level based on input feature data and predefined logical rules. Possible implementation methods include: implementing the rule base as a series of conditional-action (If-Then) statements, where the condition part is based on a combination of pet posture features, furniture semantic features, and spatial geometric relationship features, and the action part outputs the corresponding behavioral intent label, risk level, risk type, or risk score; or constructing the rule base as a decision tree, finite state machine, or ontology-based knowledge graph.

[0095] Based on the location information from both skeletal keypoint features and furniture semantic features, the spatial geometric relationship between the pet and the furniture can be calculated. This quantifies the relative position and distance between the pet and the furniture in space, providing crucial contextual information about the pet's interaction with the environment. For example, it can determine whether the pet is near the furniture, above or below it, and its relative orientation. Possible implementation methods include: calculating metrics such as Euclidean distance, minimum distance, and intersection-over-union (IoU) between the pet's skeletal keypoints (e.g., the pet's center point, head, and limb keypoints) and the furniture bounding box or segmentation mask obtained from furniture semantic detection; or inferring the relative orientation of the pet and the furniture by analyzing the relative coordinates, angles, and projection relationships of the pet's skeletal keypoints and the furniture semantic region in 3D space.

[0096] By performing rule matching and logical reasoning on pet posture features, furniture semantic features, and spatial geometric relationship features, a systematic integration of the pet's own state, environmental information, and the interaction information between the pet and the environment can be achieved, thus yielding comprehensive behavioral assessment results. Possible implementation methods include: employing an expert system-based reasoning engine that receives the above three features as input and matches rules one by one according to the priority and logical relationships defined in the rule base, executing conditional judgments until a matching rule is found and the corresponding behavioral assessment result is triggered; or utilizing fuzzy logic reasoning or probabilistic graphical models, using various features as input variables, and reasoning through preset fuzzy rules or probabilistic relationships to handle the uncertainty and fuzziness of features, ultimately outputting the probability distribution or confidence level of behavioral intent and risk assessment. The set of conditional judgment functions refers to the specific judgment logic units that constitute the condition part of the rule base. Their role is to provide the rule base with flexible and scalable condition definition capabilities, enabling rules to accurately capture various complex behavioral patterns. Possible implementations include: including a series of predefined mathematical functions and logical operators, such as comparison functions, range judgment functions, distance calculation functions, angle judgment functions, Boolean logic functions, etc.; or allowing users or the system to define their own functions, which can encapsulate specific business logic or domain knowledge.

[0097] This application's solution quantifies the relative position and distance between a pet and environmental objects (furniture), providing crucial spatial contextual information for subsequent behavioral analysis. For example, the distance, relative orientation, and contact between the pet and specific furniture are essential for determining the pet's behavioral intentions. Subsequently, the solution utilizes pre-defined intention inference and risk assessment rule bases to perform rule matching and logical reasoning on pet posture features, furniture semantic features, and calculated spatial geometric relationship features. Both the intention inference and risk assessment rule bases are rule sets configured based on a set of conditional judgment functions, meaning they can flexibly define complex behavioral patterns and risk conditions. By integrating the pet's posture, its environment (furniture type), and its interaction with the environment (spatial geometric relationships), the implementing entity can accurately identify the pet's behavioral intentions and assess the potential risk level based on predefined logical rules. This fusion of multi-dimensional features and rule-based reasoning mechanism can extract high-level semantic information from isolated perceptual data, thereby achieving accurate and intelligent assessment of pet behavioral intentions and risks, providing a reliable basis for subsequent monitoring operations.

[0098] The following is a concrete example to illustrate this. After the executing agent acquires the target video frame, it first obtains the coordinates of key points on various parts of the pet's body through skeletal keypoint detection. Simultaneously, it identifies various types of furniture in the video frame, along with their positions and outlines, through furniture semantic detection. For example, it can identify furniture objects such as "sofa," "coffee table," and "vase," and obtain their bounding boxes or pixel-level masks. Next, the executing agent calculates the spatial geometric relationship features between the pet's skeletal keypoints and these furniture objects. For example, it can calculate the minimum distance between the pet's head keypoint and the edge of the sofa, or determine whether the keypoints of the pet's limbs overlap with the bottom area of ​​the vase. These spatial geometric relationship features can include the distance between the pet and the furniture, their relative orientation (e.g., the pet facing the sofa, or with its back to the coffee table), and whether the pet is located in a specific area of ​​the furniture (e.g., above the sofa, or below the coffee table). Subsequently, the executing entity inputs pet posture characteristics (e.g., identifying the pet in a "standing," "lying," or "jumping ready" posture), furniture semantic characteristics (e.g., identifying "sofa" as fragile furniture and "cat tree" as safe furniture), and calculated spatial geometric relationship characteristics (e.g., the pet is close to the sofa and facing it) into the intent inference rule base and risk assessment rule base. These rule bases can be pre-configured. For example, an intent inference rule can be defined as: "If the pet's posture is 'jumping ready' and the distance between the pet and the sofa is less than a certain threshold and the pet is facing the sofa, then the intent is 'jump onto the sofa'." A risk assessment rule can be defined as: "If the pet's intent is 'jump onto the sofa' and the sofa material is 'fabric' (easily scratched), then the risk level is 'medium risk' and the risk type is 'furniture damage'." By matching these rules and performing logical reasoning, the executing entity finally outputs the behavior assessment result, such as "Intent: jump onto the sofa, confidence: 0.9; Risk: medium risk, type: furniture damage, score: 70." The set of conditional judgment functions can include functions such as "distance less than (X, Y)", "attitude equal to (Z)", and "material is (W)", which are used to construct the conditional expressions in the rules.

[0099] Through the aforementioned technical solution, this application effectively integrates independent pet posture, furniture semantics, and skeletal key point features. By calculating the spatial geometric relationship between the pet and the furniture, it provides rich contextual information for behavior assessment. Based on a pre-defined intent inference rule base and risk assessment rule base, rule matching and logical reasoning are performed to accurately identify the pet's behavioral intent from multi-dimensional data and quantify its potential risks. This structured reasoning method significantly improves the accuracy and reliability of behavior assessment, avoiding misjudgments or omissions that may result from judging based on a single feature. This allows for more precise capture of abnormal pet behavior or potential dangers, providing a solid foundation for subsequent monitoring operations and effectively enhancing the intelligence level and early warning capabilities of pet behavior monitoring.

[0100] In some embodiments, the rules in the intent inference rule base are defined as triggering the corresponding behavioral intent label and confidence level when conditions related to pet posture, spatial proximity, relative orientation and / or target furniture type are met; the rules in the risk assessment rule base are defined as triggering the corresponding risk level, risk type and risk score when conditions related to target furniture type, target spatial area, target posture and / or target movement state are met.

[0101] This application's solution, when assessing the behavioral intentions and risks of monitored pets, no longer relies solely on single or limited conditions. Instead, it employs a refined definition of intention inference rule bases and risk assessment rule bases, comprehensively considering multiple dimensions and factors, including pet posture characteristics, furniture semantic features, and the spatial geometric relationship between the pet and furniture. Specifically, the rules in the intention inference rule base are designed to identify multiple dimensions of information, such as pet posture, spatial proximity, relative orientation, and target furniture type. When these conditions are met, specific behavioral intention labels and their confidence levels are triggered, thus more accurately determining the actions the pet may take. Simultaneously, the rules in the risk assessment rule base focus on combining key factors such as target furniture type, the target spatial area where the pet is located, the pet's target posture, and its target movement state. When these conditions are met, corresponding risk levels, risk types, and risk scores are triggered. This multi-condition, multi-dimensional rule definition approach makes the behavioral assessment results more detailed and accurate. In this way, this application can effectively solve the problem that traditional rule bases may not provide accurate evaluation results, enabling subsequent monitoring operations to be based on more accurate evaluation results, thereby improving the intelligence and effectiveness of pet behavior monitoring.

[0102] The following is a concrete example to illustrate this. The rules in the intent inference rule base are defined as follows: When the pet's posture is "crouching," the spatial proximity is "very close" (e.g., the distance between the pet's center point and the edge of the furniture is less than 10cm), the relative orientation is "facing," and the target furniture type is "sofa," then the behavioral intent label "preparing to jump onto the sofa" is triggered, with a confidence level of 0.85. When the pet's posture is "head down," the spatial proximity is "in contact," the relative orientation is "facing," and the target furniture type is "flower pot," then the behavioral intent label "attempting to bite the plants" is triggered, with a confidence level of 0.90. When the pet's posture is "raising its paw," the spatial proximity is "in contact," the relative orientation is "sideways," and the target furniture type is "wooden table leg," then the behavioral intent label "attempting to scratch the furniture" is triggered, with a confidence level of 0.78. Meanwhile, the rules in the risk assessment rule base can be defined as follows: when the target furniture type is "electric wire," the target space is a "high-risk area" (e.g., near a power outlet), the target posture is "head down," and the target movement is "licking," then the risk level is "high risk," the risk type is "electric shock hazard," and the risk score is 95. When the target furniture type is "trash can," the target space is a "restricted area," the target posture is "peeking," and the target movement is "rummaging," then the risk level is "medium risk," the risk type is "accidental ingestion of foreign objects," and the risk score is 70. When the target furniture type is "sofa," the target space is a "safe area," the target posture is "jumping," and the target movement is "active," then the risk level is "low risk," the risk type is "normal play," and the risk score is 20. These rules can be pre-learned from expert experience or from large amounts of behavioral data using machine learning methods and stored in the rule base in the form of a set of conditional judgment functions.

[0103] Through the aforementioned technical solution, this application can conduct a more refined and accurate assessment of a pet's behavioral intentions and risks based on multi-dimensional information such as pet posture, spatial proximity, relative orientation, target furniture type, target spatial area, and target movement state. This multi-factor comprehensive judgment mechanism significantly improves the reliability and relevance of behavioral assessment results, avoiding misjudgments or omissions caused by overly simplistic rule definitions. Therefore, during subsequent monitoring operations, the implementing entity can take more appropriate and timely response measures based on more accurate assessment results, such as providing early warnings when a pet is about to engage in destructive behavior, or immediately issuing an alarm when a pet is in a dangerous area, thereby effectively protecting pet safety and family property, and improving the intelligence level and user experience of pet behavior monitoring.

[0104] In some embodiments, when multiple rules in the intentional inference rule base or multiple rules in the risk assessment rule base are triggered simultaneously, the rules are weighted and fused based on preset weights or confidence levels to generate behavioral assessment results.

[0105] This application's solution involves performing rule matching and logical reasoning on pet posture features, furniture semantic features, and spatial geometric relationship features based on a pre-defined intent inference rule base and a risk assessment rule base. After identifying scenarios where multiple rules from either the intent inference rule base or the risk assessment rule base are triggered simultaneously, a weighted fusion mechanism based on pre-defined weights or confidence levels is introduced. Specifically, when the executing entity detects that multiple rules simultaneously meet the triggering conditions, instead of simply listing all triggered rule results, the preliminary assessment results generated by these rules are quantitatively combined according to the pre-defined weights or confidence levels of each rule. For example, if one rule indicates "low-risk play" with a weight of 0.7, and another rule indicates "medium-risk biting" with a weight of 0.3, a comprehensive assessment result can be obtained through weighted fusion. This result may lean towards "low-risk play" but also consider the possibility of "medium-risk biting." This fusion mechanism effectively solves the problems of ambiguity, conflict, or uncertainty that may arise in the assessment results when multiple rules are triggered, ensuring the accuracy, consistency, and robustness of the behavior assessment results. In this way, the implementing body can understand the pet's behavioral intentions and potential risks more precisely, avoiding misjudgments or omissions caused by the limitations of a single rule, thereby improving the intelligence level of pet behavior monitoring.

[0106] As a specific implementation method, when the monitoring entity identifies a pet near a sofa, and its posture characteristics simultaneously satisfy both the intent inference rule of "jumping on the sofa" (e.g., labeled "actively playing," with a confidence level of 0.8) and the risk assessment rule of "biting the edge of the sofa" (e.g., risk level "medium risk," with a risk score of 0.6), the monitoring entity will not simply output two independent assessment results. Instead, it will perform a weighted fusion of these two results based on preset weights or confidence levels. For example, the intention rule weight for "actively playing" can be preset to 0.6, while the risk rule weight for "medium-risk biting" can be preset to 0.4. Through weighted fusion, the final behavioral assessment result may be a comprehensive judgment, such as "the pet is actively playing on the sofa, but there is a moderate risk of biting," accompanied by a comprehensive confidence level or risk score. This fusion method makes the assessment results more comprehensive and detailed, and can more accurately reflect the pet's true behavioral state and potential risks.

[0107] Through the above technical solution, when multiple rules in the intent inference rule base or the risk assessment rule base are triggered simultaneously, the evaluation results of each rule can be weighted and fused based on preset weights or confidence levels. This effectively solves the problems of ambiguity, conflict, or uncertainty that may exist in the evaluation results when multiple rules are triggered. This makes the final behavior evaluation results more accurate, consistent, and robust, avoiding misjudgments or omissions caused by the limitations of single-rule judgments. It significantly improves the intelligence and precision of pet behavior monitoring and provides a more reliable decision-making basis for subsequent monitoring operations.

[0108] See Figure 7 In one embodiment, the method for performing corresponding monitoring operations based on the behavior assessment results includes, but is not limited to, steps S701 to S702.

[0109] Step S701: Based on the behavior assessment results, determine the target response strategy from the preset response strategy mapping table.

[0110] Step S702: Execute at least one monitoring operation corresponding to the target response strategy.

[0111] The response policy mapping table should define at least the following policies: For furniture involving the first risk level and the intent being the first behavior type, the monitoring operation is to record behavior logs in the storage area and update the activity statistics of the pet to be monitored; For furniture involving the second risk level and the intent being the second behavior type, the monitoring operation is to generate a prompt notification containing a snapshot of the current screen and a risk description, and push it to the associated user terminal; For furniture involving the third risk level and with the intent being the third behavior type, the monitoring operation is to activate the local audible and visual alarm device for real-time warning, and simultaneously send an emergency alarm message containing a link to access the real-time video stream to the user terminal. The risk level of the first, second, and third risk levels increases sequentially, as does the activity level of the corresponding behaviors for the first, second, and third behavior types.

[0112] This application's solution introduces a refined monitoring and execution mechanism, enabling appropriate and timely responses based on different behavioral assessment results. After completing the intention and risk assessment of the pet's behavior and obtaining the assessment results, a response strategy mapping table is queried based on these results to determine the most suitable target response strategy for the current situation. Specifically, this response strategy mapping table divides monitoring operations into at least three levels based on increasing risk level and behavioral activity. For furniture involving the first risk level and the intention being the first behavioral type (typically representing low risk or daily behavior), the executing entity performs relatively mild monitoring operations, such as recording behavior logs in the storage area and updating the pet's activity statistics. When the assessment result involves furniture involving the second risk level and the intention being the second behavioral type (typically representing medium risk or potentially problematic behavior), the executing entity takes more alert measures, such as generating a notification containing a current screen snapshot and risk description, and pushing it to the associated user terminal. When the assessment results involve furniture at the third risk level and the intent is for a third behavioral type (typically representing high risk or an emergency), the implementing entity immediately activates the local audible and visual alarm device for real-time warning, and simultaneously sends an emergency alarm message containing a link to access the real-time video stream to the user terminal. Through this tiered response mechanism, the proposed solution transforms abstract behavioral assessment results into concrete and actionable monitoring actions, achieving a closed loop from behavior recognition to intelligent intervention. It enables the implementing entity to automatically select the most appropriate intervention method based on the severity and urgency of the behavior, avoiding the limitations of a single response mode, thereby significantly improving the intelligence level and practical value of pet behavior monitoring. This mechanism, closely integrated with the aforementioned behavioral assessment process, constitutes an efficient and intelligent pet behavior management system, effectively solving the problems of untimely or inaccurate responses in pet behavior monitoring.

[0113] The following is a concrete example. Assume a pet behavior monitoring device is installed in a home environment. This device continuously acquires and processes video frames. After the executing entity performs skeletal keypoint detection, pose recognition, and furniture semantic detection, it assesses that the monitored pet (e.g., a cat) is engaging in "minor scratching" behavior on the edge of a sofa, and the sofa is identified as "ordinary furniture," with a risk level determined to be "Level 1 Risk," and the behavioral intent as "Type 1 Behavior" (e.g., "minor interaction"). At this point, the executing entity queries a preset response strategy mapping table. According to the mapping table, the strategy corresponding to "Level 1 Risk" and "Type 1 Behavior" is to "record a behavior log in the storage area and update the activity statistics of the monitored pet." Therefore, the executing entity records the "minor scratching of the sofa" event, its occurrence time, duration, and other information in the device's local memory, and updates the statistical data of the cat's interaction with the sofa, for example, "Number of scratches on the sofa today +1." In another scenario, if the executing entity assesses that the cat is engaging in "continuous chewing" behavior on the TV cabinet, and the TV cabinet is identified as "valuable furniture," its risk level is determined to be "Level 2 Risk," and its behavioral intent is "Type 2 Behavior" (e.g., "destructive chewing"). In this case, the executing entity will determine the target response strategy based on the response strategy mapping table: "Generate a notification containing a snapshot of the current scene and a risk description, and push it to the associated user terminal." The executing entity will immediately capture a snapshot of the cat chewing the TV cabinet and generate a risk description, such as "Your pet is chewing the TV cabinet, please be aware!" Subsequently, this snapshot and risk description will be pushed to the user's associated smartphone via a mobile application. As another example, if the executing entity assesses that the cat is attempting to jump onto the gas stove, and the gas stove is identified as a "dangerous area," its risk level is determined to be "Level 3 Risk," and its behavioral intent is "Type 3 Behavior" (e.g., "climbing dangerous areas"). At this point, the executing entity will immediately implement the most urgent response strategy: "Activate the local audible and visual alarm device for real-time early warning, and simultaneously send an emergency alarm message containing a link to access the real-time video stream to the user terminal." The device's built-in buzzer and LED lights will immediately emit an alarm sound and flash. At the same time, the user's mobile phone will receive an emergency notification containing a clickable link. After clicking, the user can directly view the real-time video stream captured by the device to quickly understand the situation on site and take intervention measures.

[0114] Through the aforementioned technical solution, this application can intelligently select and execute different levels of monitoring operations based on the differences in pet behavior assessment results. This tiered response mechanism avoids a one-size-fits-all approach to all behaviors, thereby significantly improving the accuracy and effectiveness of pet behavior monitoring. Specifically, for low-risk behaviors, it can record and statistically analyze data seamlessly, providing users with long-term behavioral data without causing unnecessary disturbance; for medium-risk behaviors, it can promptly issue alerts with intuitive evidence, enabling users to intervene in a timely manner and effectively prevent potential problems from escalating; for high-risk behaviors, it can immediately activate audible and visual alarms and provide real-time video stream access, achieving rapid intervention and emergency warnings to maximize the protection of pet safety and family property. This refined response strategy, combined with the aforementioned behavioral intent and risk assessment, makes pet behavior monitoring more intelligent and human-centered, enabling more effective management and guidance of pet behavior and enhancing the user experience.

[0115] Figure 8 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application.

[0116] The following reference Figure 8 To describe an electronic device 800 according to such an embodiment of the present disclosure. Figure 8 The electronic device 800 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0117] like Figure 8 As shown, the electronic device 800 is presented in the form of a general-purpose computing device. The components of the electronic device 800 may include, but are not limited to: at least one processing unit 810, at least one storage unit 820, a bus 830 connecting different system components (including storage unit 820 and processing unit 810), a display unit 840, etc.

[0118] The storage unit stores program code, which can be executed by the processing unit 810, causing the processing unit 810 to perform the steps described in the above-described pet behavior monitoring method section of this specification according to various exemplary embodiments of this disclosure.

[0119] Storage unit 820 may include a readable medium in the form of a volatile storage unit, such as random access memory (RAM) 8201 and / or cache memory 8202, and may further include a read-only memory (ROM) 8203.

[0120] The storage unit 820 may also include a program / utility 8204 having a set (at least one) of program modules 8205, including but not limited to: an operating system, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0121] Bus 830 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0122] Electronic device 800 can also communicate with one or more external devices 800' (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 800, and / or with any device that enables electronic device 800 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 850. Furthermore, electronic device 800 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 860. Network adapter 860 can communicate with other modules of electronic device 800 via bus 830. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 800, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0123] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0124] The pet behavior monitoring method, device, and medium provided in this application process data on an edge device. Based on skeletal keypoint features, pet posture features, and furniture semantic features extracted from a single video frame, they assess pet behavior intent and risk, and ultimately execute corresponding monitoring operations based on the behavior assessment results. Therefore, by deploying the pet behavior monitoring method on an edge device and extracting data from a single video frame, data processing is closer to the data source, avoiding the accumulated latency caused by temporal modeling, significantly reducing data transmission latency, and enabling real-time response to pet behavior. Simultaneously, by comprehensively utilizing pet skeletal keypoint features, pet posture features, and furniture semantic features, a multi-dimensional and more refined behavior assessment system is constructed, providing richer and more accurate behavior assessment results. This allows for more targeted monitoring operations, enhancing risk warning capabilities and joint understanding performance.

[0125] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, and includes several instructions to cause a computing device to execute the methods described above according to the embodiments of this disclosure.

[0126] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0127] Computer-readable storage media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable storage medium may also be any readable medium other than a readable storage medium that can transmit, propagate, or transfer a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0128] Those skilled in the art will understand that the above modules can be distributed in the device as described in the embodiments, or they can be modified accordingly and placed in one or more devices that are unique to this embodiment. The modules in the above embodiments can be combined into one module, or they can be further divided into multiple sub-modules.

[0129] Exemplary embodiments of this disclosure have been specifically shown and described above. It should be understood that this disclosure is not limited to the detailed structures, arrangements, or implementations described herein; rather, this disclosure is intended to cover various modifications and equivalent arrangements contained within the spirit and scope of the appended claims.

Claims

1. A pet behavior monitoring method, applied to edge devices, characterized in that, include: Obtain the latest target video frame from the monitored video; The target video frame contains a video frame of the pet to be monitored within the monitoring area; Pet skeletal key point detection is performed on the target video frame to obtain skeletal key point features; Based on the skeletal key point features, the pet posture features of the pet to be monitored are identified; Furniture semantic detection is performed on the target video frame to obtain furniture semantic features; Based on the skeletal key point features, the pet posture features, and the furniture semantic features, the behavioral intentions and behavioral risks of the pet to be monitored are assessed, and the behavioral assessment results are obtained. Based on the behavioral assessment results, corresponding monitoring operations are performed.

2. The pet behavior monitoring method according to claim 1, characterized in that, The step of detecting key points of the pet skeleton in the target video frame includes: The target video frame is processed to crop out the corresponding pet region feature map; The pet target region image is input into a pre-trained keypoint detection model to predict the position of each pet skeletal keypoint in the pet target region image, thereby obtaining a set of skeletal keypoint coordinates. The skeletal keypoint coordinate set is mapped to the image coordinate system of the target video frame to obtain the skeletal keypoint features.

3. The pet behavior monitoring method according to claim 1, characterized in that, The step of identifying the pet posture features of the pet to be monitored based on the skeletal key point features includes: The skeletal keypoint features are normalized to obtain normalized keypoint features. Based on the spatial geometric relationship between the normalized keypoint features, the pose feature vector used to represent the pose is extracted from the normalized keypoint features. The posture feature vector is input into a pre-trained posture classification model to predict the posture category probability that maps the posture feature vector to the corresponding pet posture, and the pet posture feature is generated based on the posture category probability.

4. The pet behavior monitoring method according to claim 1, characterized in that, The step of performing furniture semantic detection on the target video frame includes: The target video frame is processed to obtain tensor features that meet preset conditions; Multi-scale feature extraction is performed on the tensor features to obtain a tensor feature map; The tensor feature map is input into a pre-trained furniture semantic detection model to perform feature fusion and enhancement on the tensor feature map, thereby obtaining the furniture semantic features.

5. The pet behavior monitoring method according to claim 1, characterized in that, The assessment of the behavioral intent and risk of the pet under monitoring, based on the skeletal key point features, the pet's posture features, and the furniture semantic features, includes: Based on the location information of both the skeletal key point features and the furniture semantic features, the spatial geometric relationship features between the pet to be monitored and the furniture are calculated. Based on a preset intent inference rule base and risk assessment rule base, rule matching and logical reasoning are performed on the pet posture features, furniture semantic features and spatial geometric relationship features to obtain the behavior assessment result; the intent inference rule base and the risk assessment rule base are both rule sets configured based on a set of condition judgment functions.

6. The pet behavior monitoring method according to claim 5, characterized in that, The rules in the intent inference rule base are defined as triggering the corresponding behavioral intent label and confidence level when conditions related to the pet's posture, spatial proximity, relative orientation, and / or target furniture type are met; the rules in the risk assessment rule base are defined as triggering the corresponding risk level, risk type, and risk score when conditions related to the target furniture type, target spatial area, target posture, and / or target movement state are met.

7. The pet behavior monitoring method according to claim 5, characterized in that, When multiple rules in the intent inference rule base or multiple rules in the risk assessment rule base are triggered simultaneously, the rules are weighted and fused based on preset weights or confidence levels to generate the behavior assessment result.

8. The pet behavior monitoring method according to claim 1, characterized in that, Based on the behavior assessment results, the corresponding monitoring operations are performed, including: Based on the behavior evaluation results, the target response strategy is determined from the preset response strategy mapping table; Perform at least one monitoring operation corresponding to the target response strategy; The response strategy mapping table defines at least the following strategies: For furniture involving the first risk level and the intent being the first behavior type, the monitoring operation is to record a behavior log in the storage area and update the activity statistics of the pet to be monitored; For furniture involving the second risk level and the intent being the second behavior type, the monitoring operation is to generate a prompt notification containing a snapshot of the current screen and a risk description, and push it to the associated user terminal; For furniture involving the third risk level and the intent being the third behavior type, the monitoring operation is to activate the local sound and light alarm device for real-time early warning and simultaneously send an emergency alarm message containing a real-time video stream access link to the user terminal; the risk levels of the first risk level, the second risk level, and the third risk level increase sequentially, and the activity levels of the behaviors corresponding to the first behavior type, the second behavior type, and the third behavior type increase sequentially.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the pet behavior monitoring method according to any one of claims 1 to 8.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the pet behavior monitoring method according to any one of claims 1 to 8.

Images