Pet data processing methods, devices, computer equipment and storage media

Abstract

Embodiments of the present application provide a pet data processing method and device, computer equipment and a storage medium. The method comprises: obtaining a target video frame; performing feature extraction based on the target video frame by calling a feature extraction layer to obtain multi-level features; performing object detection based on the first-level features of the target video frame by calling a detection head to obtain a target object detection result, the target object detection result being used to indicate whether the target video frame contains a preset object; performing pet behavior judgment based on the second-level features of the target video frame and the second-level features of the previous video frame of the target video frame in the target pet video by calling an effective behavior judgment head to obtain a target behavior probability score corresponding to the target video frame; and determining a target behavior judgment result of the target video frame according to the behavior probability score and a preset score threshold, the target behavior judgment result being used to indicate whether the target video frame contains an effective behavior of a pet. The method can improve multi-task processing efficiency.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, specifically to a pet data processing method, apparatus, computer equipment, and storage medium. Background Technology

[0002] With the continued growth in the number of pet owners and the increasing demand for pet behavior recognition in fields such as smart pet homes and health monitoring, pet monitoring technology is evolving from traditional single detection tasks to more complex multi-task, multi-scenario intelligent analysis. For example, it not only needs to detect pets but also needs to judge their behavior.

[0003] However, in related technologies, when performing multi-task processing such as detection and behavior judgment tasks based on long-duration continuous video, problems such as low processing efficiency are prone to occur. Summary of the Invention

[0004] The main objective of this application is to provide a pet data processing method, apparatus, computer device, and storage medium, which aims to improve the processing efficiency of detection and behavior judgment tasks.

[0005] To achieve the above objectives, a first aspect of this application proposes a pet data processing method, the method comprising:

[0006] Retrieve the target video frame from the target pet video;

[0007] A pre-trained task processing model is invoked, the task processing model including a feature extraction layer, a detection head, and an effective behavior judgment head;

[0008] Based on the target video frame, the feature extraction layer is invoked to extract features, thereby obtaining multi-level features of the target video frame, including first-level features and second-level features;

[0009] Based on the first-level features of the target video frame, the detection head is invoked to perform object detection, and the target object detection result corresponding to the target video frame is obtained. The target object detection result is used to indicate whether the target video frame contains a preset object, which includes a pet, or includes a pet and a device.

[0010] Based on the second-level features of the target video frame and the second-level features of the preceding video frames in the target pet video, the effective behavior judgment head is invoked to judge the pet behavior and obtain the target behavior probability score corresponding to the target video frame;

[0011] The target behavior judgment result of the target video frame is determined based on the behavior probability score and a preset score threshold. The target behavior judgment result is used to indicate whether the target video frame contains valid pet behavior.

[0012] In some embodiments, the training method of the task processing model includes:

[0013] Acquire sample pet videos and perform frame extraction on the sample pet videos based on the preset frame rate to obtain multiple sample pet images;

[0014] Based on multiple frames of the sample pet images, the feature extraction layer is invoked to extract features, thereby obtaining multi-level features for each frame of the sample pet images;

[0015] Based on the first-level features of each frame of the sample pet image, the detection head is invoked to perform object detection, and the sample object detection result corresponding to each frame of the sample pet image is obtained. The sample object detection result is used to indicate whether the sample pet image contains a preset object.

[0016] Calculate the object detection loss based on the sample object detection results and the object detection labels corresponding to the sample pet images;

[0017] Based on the second-level features of each sample pet image and the second-level features of adjacent frames of the sample pet image, the effective behavior judgment head is called to judge the pet behavior, obtain the sample behavior probability score corresponding to each sample pet image, and determine the sample behavior judgment result of each sample pet image according to the sample behavior probability score and the preset score threshold.

[0018] Calculate the behavior judgment loss based on the sample behavior judgment result and the behavior judgment label corresponding to the sample pet image;

[0019] The behavior judgment loss is weighted according to preset weights, and the parameters of the feature extraction layer, the detection head, and the effective behavior judgment head are adjusted according to the weighted behavior judgment loss and the object detection loss.

[0020] In some embodiments, the sample object detection result includes an object detection bounding box and an object detection confidence score;

[0021] The method for determining the behavior judgment label includes:

[0022] When the object detection confidence score of the sample pet image is greater than a preset confidence threshold, the sample pet image is determined as a candidate pet image.

[0023] Object location detection is performed based on object detection boxes corresponding to multiple adjacent candidate pet images, and behavior judgment labels for multiple adjacent candidate pet images are determined based on the object location detection results.

[0024] In some embodiments, the method for determining the behavior judgment label includes:

[0025] A teacher model is constructed based on the model information of the effective behavior judgment head;

[0026] For each training round, the model parameters of the teacher model are updated based on the model parameters of the effective behavior judgment head in the previous training round;

[0027] Based on the second-level features of each sample pet image and the second-level features of adjacent frames of the sample pet image, the teacher model with updated parameters is called to judge pet behavior and obtain the reference behavior probability score corresponding to each sample pet image.

[0028] The reference behavior judgment result for each frame of the sample pet image is determined based on the reference behavior probability score and the preset score threshold, and the reference behavior judgment result is determined as the behavior judgment label for the current training round.

[0029] In some embodiments, adjusting the parameters of the feature extraction layer, the detection head, and the effective behavior judgment head based on the weighted behavior judgment loss and the object detection loss includes:

[0030] The joint loss is calculated based on the weighted behavior judgment loss and the object detection loss.

[0031] Calculate the cross-task consistency loss based on the sample behavior probability score and the sample object detection result;

[0032] The parameters of the feature extraction layer, the detection head, and the effective behavior judgment head are adjusted based on the joint loss and the cross-task consistency loss.

[0033] In some embodiments, the sample object detection result includes an object detection bounding box and an object detection confidence score;

[0034] The step of calculating the cross-task consistency loss based on the sample behavior probability score and the sample object detection result includes:

[0035] The detection consistency loss is calculated based on the sample behavior probability score and the object detection confidence score.

[0036] The motion dynamic score of the object is determined based on the object detection boxes corresponding to multiple adjacent candidate pet images.

[0037] The action consistency loss is calculated based on the sample behavior probability score and the action dynamic score.

[0038] The cross-task consistency loss is calculated based on the detection consistency loss and the action consistency loss.

[0039] In some embodiments, the effective behavior determination head includes a spatiotemporal network, a pooling network, and a multilayer sensing network;

[0040] The process involves using the second-level features of the target video frame and the second-level features of the preceding video frames in the target pet video to call the effective behavior judgment head for pet behavior judgment, thereby obtaining the target behavior probability score corresponding to the target video frame, including:

[0041] Based on the second-level features corresponding to the target video frame and the preceding video frame, the spatiotemporal network is invoked to perform spatiotemporal attention calculation to obtain spatiotemporal features;

[0042] Based on the spatiotemporal features, the pooling network is invoked to perform pooling processing to obtain a global semantic vector;

[0043] Based on the global semantic vector, the multilayer perceptron is invoked to determine the pet's behavior and obtain the target behavior probability score of the target video frame.

[0044] To achieve the above objectives, a second aspect of this application provides a pet data processing apparatus, the apparatus comprising:

[0045] The video processing unit is used to acquire target video frames from the target pet video;

[0046] The model invocation unit is used to invoke a pre-trained task processing model, which includes a feature extraction layer, a detection head, and an effective behavior judgment head.

[0047] The feature extraction unit is used to call the feature extraction layer to perform feature extraction based on the target video frame, and obtain multi-level features of the target video frame, wherein the multi-level features include first-level features and second-level features;

[0048] An object detection unit is used to call the detection head to perform object detection based on the first-level features of the target video frame, and obtain the target object detection result corresponding to the target video frame. The target object detection result is used to indicate whether the target video frame contains a preset object, which includes a pet, or includes a pet and a device.

[0049] The behavior judgment unit is used to call the effective behavior judgment head to judge the pet behavior based on the second-level features of the target video frame and the second-level features of the preceding video frame of the target pet video, and to obtain the target behavior probability score corresponding to the target video frame.

[0050] The behavior result determination unit is used to determine the target behavior judgment result of the target video frame based on the behavior probability score and a preset score threshold. The target behavior judgment result is used to indicate whether the target video frame contains valid pet behavior.

[0051] To achieve the above objectives, a third aspect of this application provides a computer device including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the pet data processing method described in the first aspect.

[0052] To achieve the above objectives, a fourth aspect of the present application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the pet data processing method described in the first aspect.

[0053] To achieve the above objectives, a fifth aspect of this application provides a computer program product comprising a computer program that is read and executed by a processor of a computer device, causing the computer device to perform the pet data processing method described in the first aspect.

[0054] The pet data processing method provided in this application uses multi-level features extracted by the feature extraction layer for detection and behavior judgment tasks. That is, the detection and behavior judgment tasks share a set of feature extraction processes, which can reduce redundant calculations, realize feature sharing and complementarity, and improve the processing efficiency and accuracy of multi-task processing (i.e. detection and behavior judgment tasks). Attached Figure Description

[0055] The accompanying drawings are provided to further understand the technical solutions of this disclosure and constitute a part of the specification. They are used together with the embodiments of this disclosure to explain the technical solutions of this disclosure and do not constitute a limitation on the technical solutions of this disclosure.

[0056] Figure 1 A schematic diagram of the system used in the pet data processing method provided in this application;

[0057] Figure 2 A flowchart illustrating the pet data processing method provided in this application;

[0058] Figure 3AA schematic diagram of a pet data processing method during the inference stage provided in an embodiment of this application;

[0059] Figure 3B A schematic diagram of the pet data processing method during the training phase provided in this application;

[0060] Figure 4 A flowchart illustrating the task processing model training method provided in this application;

[0061] Figure 5 A schematic diagram of the pet data processing device provided in this application;

[0062] Figure 6 This is a schematic diagram of the hardware structure of the computer device provided in the embodiments of this application. Detailed Implementation

[0063] To make the objectives, technical solutions, and advantages of this disclosure clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and are not intended to limit the scope of this disclosure.

[0064] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0065] With the continued growth in the number of pet owners and the increasing demand for pet behavior recognition in fields such as smart pet homes and health monitoring, pet monitoring technology is evolving from traditional single detection tasks to more complex multi-task, multi-scenario intelligent analysis. For example, it not only needs to detect pets but also needs to judge their behavior.

[0066] However, in related technologies, the following problems are prone to occur when performing multi-task processing such as detection and behavior judgment tasks based on long-duration continuous videos:

[0067] 1. The separation of model structures leads to insufficient feature utilization. Related technologies typically employ independent model structures to handle detection and behavior judgment tasks separately, lacking feature sharing mechanisms and resulting in low overall inference efficiency. Furthermore, single-task models repeatedly extract redundant features, increasing computational load and resource consumption, making them unsuitable for efficient deployment in resource-constrained environments such as edge devices. Finally, this separation approach lacks multi-task collaborative learning, failing to achieve complementary optimization between detection and behavior judgment tasks, thus limiting overall performance improvement.

[0068] 2. Lack of spatiotemporal awareness leads to poor behavior recognition in long videos. Related technologies, based on static analysis of single-frame images, ignore the continuity and contextual dependence of pet behavior over time, making it difficult to accurately identify valid behavior moments. Furthermore, pet actions exhibit significant temporal dynamics (such as behavior persistence, action transitions, and interaction events), and models based on single-frame analysis cannot capture long-term contextual features, leading to missed detections and false positives. While some methods in related technologies introduce high-overhead structures such as 3D CNNs to model temporal sequences, this approach is computationally intensive and has high inference latency, making it unsuitable for real-time and large-scale deployment requirements.

[0069] 3. Image information redundancy leads to resource waste and low recognition efficiency. In long-duration surveillance videos, many consecutive frames show pets in a static, resting, or non-critical state, indicating low information density. Related technologies typically process all video frames equally, resulting in storage overhead, computational burden, and wasted resources for subsequent analysis. Furthermore, this method lacks an effective keyframe filtering mechanism, failing to filter out meaningless redundant frames while maintaining detection accuracy, thus impacting the overall system throughput and response speed.

[0070] In summary, the relevant technologies lack a multi-task collaborative learning mechanism that can simultaneously achieve pet detection and behavior validity judgment. They also lack a complete solution that is capable of efficiently modeling spatial and temporal dynamic features, supporting keyframe filtering, effectively reducing redundant image processing, adapting to end-to-end inference, and is suitable for intelligent processing of long-duration videos with a user-friendly deployment.

[0071] Based on this, to address the problems of low efficiency in multitasking in related technologies, this application provides a pet data processing method, apparatus, computer equipment, and storage medium. This method employs unified modeling of pet detection and behavior judgment tasks to achieve feature sharing and complementarity, thereby improving task processing efficiency and accuracy. The pet data processing method provided in this application embodiment will be described below.

[0072] Reference Figure 1 This is a schematic diagram of the system used in the pet data processing method provided in this application. Figure 1As shown, the system includes at least one smart device 110, at least one pet management client 130 corresponding to the smart device 110, and a pet device management server 120. The smart device 110 can be a smart litter box, a smart feeder, or a smart water dispenser, etc. The pet management client 130 is specifically a terminal that runs a pet management application. This terminal can be a mobile terminal, such as a smartphone, or a tablet, personal computer, headset, or vehicle terminal, etc. The pet management application can control smart devices such as smart litter boxes, smart feeders, and smart water dispensers. The pet device management server 120 is a server that provides management, data processing, and interaction services to multiple pet management clients 130. The pet device management server 120 can be a single high-performance computer, a cluster of multiple high-performance computers, a portion of a single high-performance computer (e.g., a virtual machine), or a combination of portions of multiple high-performance computers (e.g., virtual machines) within a network platform. The pet device management server 120 can be a physical server or a cloud server.

[0073] Reference Figure 2 This is a flowchart illustrating the pet data processing method provided in this application.

[0074] In some embodiments, the pet data processing method provided in this application includes, but is not limited to, steps S201 to S206.

[0075] Step S201: Obtain the target video frame from the target pet video;

[0076] Step S202: Invoke the pre-trained task processing model, which includes a feature extraction layer, a detection head, and an effective behavior judgment head;

[0077] Step S203: Based on the target video frame, the feature extraction layer is called to extract features to obtain multi-level features of each frame of the target pet image. The multi-level features include first-level features and second-level features.

[0078] Step S204: Based on the first-level features of the target video frame, call the detection head to perform object detection and obtain the target object detection result corresponding to the target video frame. The target object detection result is used to indicate whether the predicted target video frame contains a preset object. The preset object includes a pet, or includes a pet and a device.

[0079] Step S205: Based on the second-level features of the target video frame and the second-level features of the preceding video frames in the target pet video, the effective behavior judgment head is called to judge the pet behavior and obtain the target behavior probability score corresponding to the target video frame.

[0080] Step S206: Determine the target behavior judgment result of the target video frame based on the behavior probability score and the preset score threshold. The target behavior judgment result is used to indicate whether the target pet image contains valid pet behavior.

[0081] In step S201 of some embodiments, the target pet video can refer to a long, continuous video (e.g., several hours or days long) to be multitasking processed, and can be acquired from a smart device equipped with a camera module, such as a smart litter box or a smart feeder. A video frame can refer to an independent frame image extracted from a continuous video stream at a frequency of one or more frames per second, thus forming the video stream to be processed. The target video frame can refer to the current video frame in the video stream. For example, the video stream corresponding to the target pet video includes video frames t1, t2, t3, t4, t5, etc., which arrive frame by frame in a streaming manner (e.g., first arriving at t1, then t2, and so on). Thus, the target video frame can refer to the most recently arrived video frame (e.g., t5).

[0082] In step S202 of some embodiments, a pre-trained task processing model is invoked. The task processing model can refer to a model with multi-task processing capabilities, meaning it can include multiple task heads. For example, when "multi-task" includes detection and behavior judgment tasks, the task processing model can include a detection task head (i.e., a detection head, which can serve as the main task head) and a behavior judgment task head (i.e., an effective behavior judgment head, which can serve as an auxiliary task head). Furthermore, the task processing model can also include a feature extraction layer, which can be a common feature extraction model for both the detection head and the effective behavior judgment head; that is, the detection head and the effective behavior judgment head share the features extracted by the feature extraction layer.

[0083] In step S203 of some embodiments, such as Figure 3A As shown, the target video frame can be used as input data for the feature extraction layer. Based on this layer, features are extracted from shallow to deep layers, resulting in a multi-level feature map (i.e., multi-level features) corresponding to the target video frame. Shallow features have high spatial resolution (e.g., large size) and contain rich detail information, but weak semantic information. Deep features have low spatial resolution (e.g., small size) and contain stronger semantic information, but lose more detail information. Specifically, the feature extraction layer can employ a backbone network and a multi-scale feature fusion structure (Neck). The multi-scale feature fusion structure can be a Feature Pyramid Network (FPN). The FPN can fuse deep semantic information with shallow detail features through a top-down path, forming a multi-scale feature pyramid (i.e., multi-level features). For example, multi-level features can include P5 features (i.e., high-order features), P4 features (i.e., mid-order features), and P3 features (i.e., low-order features).

[0084] Furthermore, in this embodiment, the multi-level features can be divided into first-level features and second-level features. The first-level features can refer to features used for detection tasks, such as P3, P4, and P5 features. The second-level features can refer to features used for behavior judgment tasks, such as P5 features.

[0085] In step S204 of some embodiments, the detection task may refer to the task of object detection. The object may include a pet entity, or a pet entity and environmental devices. Environmental devices may refer to related devices contained in the pet image, such as smart water fountains, smart feeders, pet gates, cameras, air purifiers, pet toys, and other devices related to pet behavior. Thus, when the first-level features (such as P3 and P4 features) corresponding to the target video frame are input into the detection head, the obtained target detection result may include a detection box, category, and confidence score. The detection box may refer to a bounding box used to locate and identify the position of objects (i.e., pets and devices) in the target video frame. The category may refer to the category label of the object identified by the detection box, such as pet categories (e.g., cats, dogs) and device categories (e.g., smart water fountains, smart feeders). The confidence score can represent the score of the presence of an object in the detection box (the value can range from 0 to 1). For example, when the confidence score of a detection box is 0.9, it indicates that the detection head predicts that the probability of the presence of an object in the detection box is 90%.

[0086] In practical applications, environmental device detection helps improve the accuracy of behavior recognition and scene understanding. For example, when a pet is detected approaching a water fountain for a certain period of time, it can help determine that the pet is engaging in "drinking" behavior, thereby improving the accuracy of behavior tagging. When a pet is detected approaching a door or the door is open, it can help determine whether there is "going out" or "coming in," and further analyze the pet's activity path and habits by combining time-series information. In addition, by detecting the period of interaction between the pet and the automatic feeder, it can help determine "eating" behavior and feeding frequency, which is helpful for health management. Thus, the detection head of this application embodiment can realize the method of environmental device detection, which not only helps multimodal information fusion and improves overall scene understanding, but also provides more comprehensive and accurate data support for subsequent intelligent applications such as health management, behavior analysis, and abnormal behavior early warning.

[0087] In step S205 of some embodiments, the behavior judgment task may refer to the task of determining whether a video frame contains valid pet behavior. Valid pet behavior may refer to behavior that meets other analysis needs (such as health management, behavior analysis, abnormal behavior warning, etc.), such as behavior with significant motion characteristics and / or completeness. Significant motion characteristics may refer to motion characteristics that meet at least one of the following conditions: 1. The pet makes visible displacement in the spatial dimension (such as running, jumping, etc.); 2. The action lasts for a sufficient duration in the temporal dimension (such as greater than 0.5 seconds); 3. The pet's body parts undergo morphological changes in the morphological dimension (such as lowering its head to eat, etc.). Behavior completeness may refer to a complete segment that includes the start, process, and end of the behavior.

[0088] In this embodiment, a space-time modeling effective behavior judgment head is built based on high-order shared features (such as P5 features). Thus, the target video frame and its preceding video frames can be used as input data for the effective behavior judgment head. The preceding video frames can refer to cached video frames in the video stream, specifically the n frames preceding the target video frame (i.e., a sliding window of size n). For example, taking the target video frame as frame t5, when n=2, the preceding video frames can include frames t4 and t3. The value of n is not specifically limited. It can be connected that in the historical cases where each preceding video frame is the current frame, multi-level features corresponding to the preceding video frame have already been extracted based on the feature extraction layer, and these multi-level features can be cached in a FIFO buffer. Thus, when the target video frame t5 is the current frame, the second-level features corresponding to frames t4 and t3 can be extracted from the buffer, and combined with the second-level features corresponding to the target video frame t5, the effective behavior judgment head is called to judge pet behavior, obtaining the target probability score corresponding to the target video frame t5.

[0089] In some embodiments, the effective behavior determination head includes a spatiotemporal network, a pooling network, and a multilayer sensing network. Step S205 may include, but is not limited to, the following sub-steps:

[0090] Construct a pet image set based on the target pet image and its adjacent frames;

[0091] Based on the second-level features corresponding to the target video frame and the preceding video frame, a spatiotemporal network is invoked to perform spatiotemporal attention calculation to obtain spatiotemporal features;

[0092] The global semantic vector is obtained by using a pooling network based on spatiotemporal features.

[0093] Pet behavior is judged by calling a multilayer perceptron based on global semantic vectors, and the target behavior probability score of the target video frame is obtained.

[0094] In this embodiment, the second-level features of the target video frame and preceding video frames are used as input data for a spatiotemporal network (such as a lightweight spatiotemporal Transformer module). The spatiotemporal network is used to model inter-frame temporal dependencies and cross-frame feature interactions, enhancing the model's ability to recognize continuous behavioral patterns (such as action continuation and posture changes). In other words, the spatiotemporal network integrates inter-frame temporal modeling and spatial semantic understanding capabilities to enhance the effective behavior judgment head's ability to understand the temporal continuity of pet behavior. Specifically, the spatiotemporal network includes three processing stages: input processing, temporal dimension position encoding, and spatiotemporal attention processing. Input processing means that the spatiotemporal network input data consists of high-order spatial feature maps (i.e., second-level features, or P5 features) corresponding to N consecutive video frames (i.e., including the target video frame and preceding video frames), with each feature map having a size of C×H×W (C represents the number of channels, and H and W represent the spatial dimensions of the feature map). By flattening the feature map of each frame in two dimensions, a sequence of patches (i.e., spatial positions) of length H×W is obtained, thus representing the input sequence as N×(H·W)×C. Temporal positional encoding representation: To model inter-frame sequence information, learnable temporal positional encoding is added to the embedding of each frame to preserve the structural features of temporal order (i.e., adding positional encoding to each frame in the input sequence to preserve temporal information), assisting the effective behavior judgment head in perceiving the sequential relationship in the evolution of behavior. Spatiotemporal attention processing includes two attention mechanisms: intra-spatial attention and inter-frame temporal attention. Intra-spatial attention refers to modeling the spatial dependencies between features within a single frame, thereby helping to enhance the perception ability of the behavior occurrence area (i.e., within a single frame, attention calculation is performed on features at different spatial locations to capture spatial relationships, thereby focusing on key areas within a single frame). Inter-frame temporal attention refers to introducing fully connected attention calculation between frames to capture the evolution trend of actions and cross-frame correlations (i.e., attention calculation is performed between the same spatial location (or all spatial locations) in different frames to capture temporal relationships, thereby analyzing the association between the current frame and the preceding and following n frames). This application's embodiments utilize a method for extracting behavioral context information based on a lightweight spatiotemporal Transformer. Because the lightweight spatiotemporal Transformer has extremely low computational overhead and controllable inference latency, this application is applicable to real-time analysis and deployment on resource-constrained devices (such as smart cameras and edge boxes). Furthermore, this application's embodiments employ local and global temporal dependency modeling to effectively capture the persistence, transition, and interactivity features of pet behavior, improving the accuracy and robustness of complex behavior recognition.

[0095] The spatiotemporal features corresponding to the feature sequences output by the spatiotemporal network (feature maps corresponding to each video frame) are used as input data for the pooling network. The pooling network is used to compress the spatial dimension into a global semantic vector, thereby reducing computational complexity and preserving key spatiotemporal information. In other words, the global semantic vectors of the target video frame and the preceding video frames can be obtained based on the pooling network.

[0096] The output sequence of the pooling network is used as the input data of the multilayer perceptron. The multilayer perceptron is used to perform frame-level validity prediction and outputs a valid behavior probability score (i.e., target behavior probability score) between 0 and 1 via the sigmoid function, thereby realizing the frame-level keyframe screening function.

[0097] The lightweight spatiotemporal modeling provided in this application combines spatial static features and temporal dynamic features, and employs efficient temporal awareness. Thus, compared to related technologies that use high-overhead 3D convolutional networks, this application achieves a balance between recognition accuracy and inference speed, making it suitable for edge computing and large-scale application scenarios.

[0098] In step S206 of some embodiments, the behavior probability score corresponding to the target video frame is compared with a preset score threshold to obtain the corresponding target behavior judgment result. Specifically, when the behavior probability score is greater than or equal to the preset score threshold, it indicates that there is effective pet behavior in the target video frame, that is, the target behavior judgment result of "containing effective behavior" can be obtained. In this way, the target video frame can be filtered out as a keyframe. In subsequent processing, detailed behavior recognition, abnormal behavior warning, and other tasks can be performed based on the filtered keyframes, and / or the keyframes can be pushed to the pet management client to form a log for users to view.

[0099] This application employs a unified backbone network and a multi-scale feature fusion structure, allowing the extracted multi-level features to be shared between the detection task and the behavior judgment detection task. The detection head for the detection task primarily utilizes low, medium, and high-order features for spatial local region recognition (such as pet and device detection), while the effective behavior judgment head for the behavior judgment task relies on higher-order semantic features to model frame-level behavior effectiveness, enhancing the understanding and modeling capabilities of pet behavior semantics. This achieves feature sharing and complementarity, thereby reducing inference redundancy and improving overall computational efficiency and system throughput. Furthermore, by introducing an effective behavior keyframe discrimination mechanism, high-information-density keyframes can be automatically filtered from lengthy, continuous video streams, thus reducing resource overhead in storage, communication, and subsequent inference.

[0100] The training process of the task processing model is explained below.

[0101] Reference Figure 3B and Figure 4In some embodiments, the training method for the task processing model includes, but is not limited to, steps S401 to S407.

[0102] Step S401: Obtain sample pet videos and perform frame extraction on the sample pet videos based on a preset frame rate to obtain multiple sample pet images.

[0103] Step S402: Based on the multi-frame sample pet images, the feature extraction layer is called to extract features and obtain multi-level features of each frame sample pet image;

[0104] Step S403: Based on the first-level features of each frame of sample pet image, call the detection head to perform object detection, and obtain the sample object detection result corresponding to each frame of sample pet image. The sample object detection result is used to indicate whether the predicted sample pet image contains a preset object.

[0105] Step S404: Calculate the object detection loss based on the sample object detection results and the object detection labels corresponding to the sample pet images;

[0106] Step S405: Based on the second-level features of each sample pet image and the second-level features of the adjacent frames of the sample pet image, call the effective behavior judgment head to judge the pet behavior, obtain the sample behavior probability score corresponding to each sample pet image, and determine the sample behavior judgment result of each sample pet image according to the sample behavior probability score and the preset score threshold.

[0107] Step S406: Calculate the behavior judgment loss based on the sample behavior judgment result and the behavior judgment label corresponding to the sample pet image;

[0108] Step S407: The behavior judgment loss is weighted according to the preset weights, and the parameters of the feature extraction layer, the detection head, and the effective behavior judgment head are adjusted according to the weighted behavior judgment loss and the object detection loss.

[0109] In step S401 of some embodiments, the sample pet video may refer to a long-duration continuous video (such as one lasting several hours or days), which can be obtained from historical data in smart devices equipped with camera modules, such as smart litter boxes or smart feeders. The preset frame rate may refer to a pre-set frame extraction frequency, used to indicate how many frames are extracted from the video per second. This application embodiment does not specifically limit the value of the preset frame rate. Extracting multiple sample pet images from the sample pet video according to the preset frame rate reduces redundant processing of the sample pet video.

[0110] In step S402 of some embodiments, each frame of sample pet image can be used as input data for the feature extraction layer to extract features from shallow to deep layers based on the feature extraction layer, thereby obtaining a multi-level feature map (i.e., multi-level features) corresponding to each frame of sample pet image. The multi-level features may include P5 features (i.e., high-order features), P4 features (i.e., mid-order features), and P3 features (i.e., low-order features).

[0111] In step S403 of some embodiments, the detection task may refer to the task of object detection. The object may include a pet entity, or a pet entity and environmental devices. Environmental devices may refer to related devices contained in the pet image, such as smart water fountains, smart feeders, pet gates, cameras, air purifiers, pet toys, and other devices related to pet behavior. Thus, when the multi-level features corresponding to a single-frame sample pet image are input into the detection head, the obtained sample detection result may include a detection box, category, and confidence score. The detection box may refer to a bounding box used to locate and identify the position of objects (i.e., pets and devices) in the sample pet image. The category may refer to the category label of the object identified by the detection box, such as pet categories (e.g., cats, dogs, etc.) and device categories (e.g., smart water fountains, smart feeders, etc.). The confidence score can represent the score of the presence of an object in the detection box (the value can range from 0 to 1). For example, when the confidence score of a detection box is 0.9, it indicates that the detection head predicts that the probability of the presence of an object in the detection box is 90%.

[0112] In step S404 of some embodiments, the object detection label may refer to a result label obtained in advance through manual annotation or other methods. The object detection label is used to indicate whether the sample pet image contains a preset object in real-world conditions. That is, the object detection label may include the detection box that accurately locates the object contained in the sample pet image, the true category of the contained object, and the confidence level (usually 0 or 1). Thus, the object detection loss of the detection head can be calculated based on the sample object detection results (including the detection box, category, and confidence level predicted by the model) and the object detection label. In other words, the detection head is trained using labeled supervised training. The object detection loss L_det may include the detection box regression loss (such as CloU, calculated based on the true detection box and the predicted detection box), the category cross-entropy loss (calculated based on the true category and the predicted category), and the target confidence loss (calculated based on the true confidence level and the predicted confidence level).

[0113] In step S405 of some embodiments, a single-frame sample pet image is processed based on a preset sliding window. For example, the sliding window size is 2n+1, meaning that the current single-frame sample pet image and the adjacent frame images that are temporally close to the sample pet image (e.g., n frames before and after, constituting a total of 2n+1 frames; for example, when n=2, it means taking 5 frames each time, and the value of n is not specifically limited) are used as input data for the effective behavior judgment head. In this way, the effective behavior judgment head can combine the spatiotemporal context information of the adjacent frame images to perform pet behavior judgment on the current sample pet image and the adjacent frame images, and obtain the target behavior probability scores of the current sample pet image and the adjacent frame images, that is, obtain 2n+1 sample behavior probability scores.

[0114] In step S406 of some embodiments, the effective behavior judgment head is trained under weak supervision, and pseudo-labels (i.e., behavior judgment labels) are generated based on sample pet images. Thus, the behavior judgment loss L_beh of the effective behavior judgment head can be calculated based on these pseudo-labels and the prediction results of the effective behavior judgment head (i.e., sample behavior judgment results). It is understood that the behavior judgment loss L_beh can be a binary classification loss, such as using binary cross-entropy loss (BCE Loss) to determine the frame-level judgment error of effective behavior.

[0115] The following explains the method for generating pseudo-tags (i.e., behavior judgment tags).

[0116] In some embodiments, the method for determining behavior judgment labels includes, but is not limited to, the following steps:

[0117] When the object detection confidence score of a sample pet image is greater than a preset confidence threshold, the sample pet image is identified as a candidate pet image.

[0118] Object location detection is performed based on the object detection boxes corresponding to multiple adjacent candidate pet images, and behavior judgment labels for multiple adjacent candidate pet images are determined based on the object location detection results.

[0119] In this embodiment, the sample object detection result obtained by calling the detection head based on the sample pet image may include an object detection bounding box and an object detection confidence score. Thus, the sample pet images with object detection confidence scores greater than a preset confidence threshold can be filtered out using the sample object detection results, and these filtered sample pet images are used as behavior-related candidate frames (i.e., candidate pet images). In other words, sample pet images with object detection confidence scores less than or equal to the preset confidence threshold can be considered as frames without pets or backgrounds, and therefore these sample pet images can be excluded (i.e., these sample pet images will certainly not contain valid pet behavior), thereby reducing noisy samples.

[0120] A candidate frame sequence can be constructed based on the selected multiple candidate pet images. Using the object detection box information (such as position and size) for each frame in this candidate frame sequence, the position of the object (such as a pet) can be detected, i.e., whether there is a significant positional change in the object, thus obtaining the object's position and location result. Specifically, multiple adjacent candidate pet images can be determined from the candidate frame sequence using a preset sliding window. The pet's motion vector (such as motion amplitude, specifically displacement) or IoU change (i.e., detecting the overlap of adjacent frame detection boxes) is calculated based on the object detection box information of these adjacent candidate pet images. When the motion vector or IoU change is greater than a preset position threshold, it indicates that the pet has a significant positional change (i.e., there is a behavioral action) among these adjacent candidate pet images, and the object position detection result indicates a significant positional change. Conversely, if the change is less than a preset threshold, it indicates that the pet's positional change is not significant, and the object position detection result indicates no significant positional change.

[0121] Thus, when the object location detection result indicates a significant location change, the candidate pet images in the adjacent frames can be marked as "valid frames," that is, the pseudo-labels (i.e., behavior judgment labels) of these candidate pet images in the adjacent frames are determined as "valid frames (or keyframes)." When the object location detection result indicates no significant location change, the pseudo-labels (i.e., behavior judgment labels) of these candidate pet images in the adjacent frames are determined as "invalid frames (or non-keyframes)." It can be understood that non-candidate pet images (i.e., sample pet images with an object detection confidence score less than or equal to the confidence threshold) are pseudo-labeled as "invalid frames (or non-keyframes)." This application does not specifically limit the specific values ​​of the confidence threshold and the location threshold in its embodiments.

[0122] The method for generating pseudo-labels based on the detection results of the detection task in this application embodiment can train an effective behavior judgment head even when only the detection task is labeled. This reduces the reliance on manual annotation for frame-level behavior judgment, thereby improving the system's scalability and deployment efficiency. Furthermore, the method of smoothing the generated pseudo-labels using a sliding window (i.e., multiple adjacent candidate pet images have the same pseudo-label) in this application embodiment can reduce discontinuities and misjudgments in frame-level annotation.

[0123] In other embodiments, the method for determining the behavior judgment label includes, but is not limited to, the following steps:

[0124] Construct a teacher model based on the model information from the effective behavior judgment head;

[0125] For each training round, the model parameters of the teacher model are updated based on the model parameters of the effective behavior judgment head in the previous training round.

[0126] Based on the second-level features of each sample pet image and the second-level features of adjacent frames of the sample pet image, the updated teacher model is called to judge pet behavior and obtain the reference behavior probability score corresponding to each sample pet image.

[0127] The reference behavior judgment result for each frame of the pet image is determined based on the reference behavior probability score and the preset score threshold, and the reference behavior judgment result is determined as the behavior judgment label for the current training round.

[0128] In this embodiment, pseudo-labels are updated round by round during training. Through a teacher-model network structure and an exponential moving average (EMA) strategy, the effective behavior judgment head gradually self-corrects and converges on the pseudo-labels. The EMA strategy is a method to improve the generalization ability and stability of a model by smoothing model parameters or weights. By maintaining a moving average of the model parameters instead of directly using the original parameters, the EMA strategy reduces fluctuations during training, making the model more robust. In the initial stage of model training, a teacher model can be constructed based on the model information of the effective behavior judgment head (such as a historical version of the current effective behavior judgment head or a stable version maintained by the EMA strategy). The teacher model generates corresponding reference behavior probability scores based on the higher-order features (i.e., second-level features) corresponding to multiple adjacent sample pet images (e.g., 2n+1 frames). Comparing these reference behavior probability scores with a preset score threshold yields the reference behavior judgment result. Thus, based on the reference behavior judgment result (the reference behavior probability score is greater than or equal to a preset score threshold, or the reference behavior probability score is less than a preset score threshold), behavior judgment labels (i.e., pseudo-labels) representing "valid frames" or "invalid frames" can be obtained. The student model (i.e., the valid behavior judgment head of the current training round) can be trained based on these pseudo-labels.

[0129] As training progresses, before each training round, the teacher model's parameters can be updated based on the EMA strategy and the model parameters of the student model from the previous training round (i.e., the effective behavior judgment head from the previous training round). That is, after each training round, the teacher model updates its own parameters using the EMA strategy and the updated model parameters of the student model in this round. The updated teacher model is then used to generate better pseudo-labels in the next training round. The method for generating pseudo-labels here is the same as the method for determining pseudo-labels based on reference behavior judgment results described earlier, and will not be repeated here. Through repeated training, the teacher model can generate pseudo-labels of progressively higher quality, and the student model (i.e., the effective behavior judgment head) can be trained using these progressively higher quality pseudo-labels, achieving self-correction and gradual convergence.

[0130] In summary, in this embodiment, pseudo-labels are determined based on the teacher model, and the model parameters of the teacher model are updated using an EMA strategy. The student model is then trained based on the pseudo-labels determined by the teacher model, thereby gradually improving model performance.

[0131] In step S407 of some embodiments, the joint loss (i.e., the joint loss of the detection task and the behavior judgment task, also known as the multi-task loss) L_joint = L_det + λ·L_beh can be calculated based on the object detection loss L_det and the behavior judgment loss L_beh. Here, λ represents a preset weight used to adjust the weight ratio of the detection loss L_det and the behavior judgment loss L_beh during model training. The parameters of the overall model (including the feature extraction layer, the detection head, and the effective behavior judgment head) are adjusted based on this joint loss to improve the accuracy of the detection and behavior judgment tasks.

[0132] In some embodiments, step S407 may include, but is not limited to, the following sub-steps:

[0133] The joint loss is calculated based on the weighted behavior judgment loss and the object detection loss.

[0134] Calculate cross-task consistency loss based on sample behavior probability scores and sample object detection results;

[0135] The parameters of the feature extraction layer, detection head, and effective behavior judgment head are adjusted based on joint loss and cross-task consistency loss.

[0136] The method for calculating the joint loss L_joint based on the object detection loss L_det and the behavior judgment loss L_beh is the same as described above, and will not be repeated here.

[0137] In this embodiment, to reduce the deviation in learning direction between the detection task and the behavior judgment task during the model training phase, a cross-task consistency loss L_consistency can be introduced during the training phase. The cross-task consistency loss L_consistency is used to constrain the consistency of the output results of the detection task and the behavior judgment task. Specifically, it can be calculated based on the output results of the detection task (i.e., the sample object detection result) and the output results of the behavior judgment task (i.e., the sample behavior probability score). Thus, the total model loss L_total = L_joint + μ · L_consistency, where μ is the weight coefficient. The parameters of the overall model (including the feature extraction layer, detection head, and effective behavior judgment head) are adjusted based on this total model loss.

[0138] In some embodiments, calculating the cross-task consistency loss based on the sample behavior probability score and the sample object detection result may include, but is not limited to, the following sub-steps:

[0139] The detection consistency loss is calculated based on the sample behavior probability score and the object detection confidence score.

[0140] The motion dynamic score of an object is determined based on the object detection bounding boxes corresponding to multiple adjacent candidate pet images.

[0141] The action consistency loss is calculated based on the sample behavior probability score and action dynamic score.

[0142] The cross-task consistency loss is calculated based on the detection consistency loss and the action consistency loss.

[0143] In this embodiment, the prediction result of the behavior judgment task (keyframe validity judgment) needs to be highly consistent with the confidence score output by the detection task and the dynamic signals related to the pet's actions (such as the pet's movement amplitude, IoU changes, etc.). This can reduce the directional deviation that occurs when these two tasks are optimized separately, and improve the quality of pseudo-labels and the overall model performance. Specifically, the cross-task consistency loss can include detection consistency loss and dynamic consistency loss. Detection consistency loss requires that the validity probability score of the behavior judgment task (i.e., the sample behavior probability score, denoted as p_valid) be consistent with the detection confidence score of the detection task (i.e., the object detection confidence score, denoted as c_det). Thus, the mean squared error (MSE) or KL divergence can be calculated on the sample behavior probability score p_valid and the object detection confidence score c_det to obtain the detection consistency loss. Action consistency loss requires that the validity probability score be consistent with the action dynamic signals (such as movement amplitude, IoU changes, etc., denoted as c_motion). Therefore, the action dynamic score of the pet can be calculated first based on the object detection boxes corresponding to multiple adjacent candidate pet images (for example, the motion amplitude calculation result or IoU change result can be normalized and the normalized result can be used as the action dynamic score). Then, the mean square error of the sample behavior probability score and the action dynamic score can be calculated to obtain the action consistency loss.

[0144] Thus, based on the calculated detection consistency loss and action consistency loss, the cross-task consistency loss L_consistency = MSE(p_valid, c_det) + MSE(p_valid, c_motion) can be calculated.

[0145] The embodiments of this application introduce a method of cross-task consistency loss during training, which achieves consistency between the frame-level "validity" prediction results and the detection confidence and action dynamic signals, thereby improving the robustness and effectiveness of pseudo-labels.

[0146] The pet data processing method provided in this application improves the recognition accuracy and processing efficiency of key behavior frames in long videos by constructing a unified multi-task network structure for detection and behavior judgment tasks, combined with a lightweight Transformer module and a weakly supervised pseudo-label generation mechanism. Specifically, by adopting an end-to-end joint optimization structure, the main and auxiliary tasks (i.e., detection and behavior judgment tasks) share features and are jointly trained, improving the context modeling capability and inference efficiency of the effective behavior judgment head, and adapting to various deployment environments (such as edge devices, smart cameras, etc.). Through the design of a weakly supervised pseudo-label generation and self-learning mechanism, effective training can still be achieved in the absence of manual frame-level annotation, exhibiting good transfer and generalization performance. The model structure of the task processing model has good modular decoupling, facilitating subsequent integration of extended functions such as health management, behavior analysis, and abnormal behavior early warning, possessing the ability to be implemented in practical applications and promising prospects for industrial promotion. Therefore, it can be seen that the pet data processing method provided in this application has broad application potential in scenarios such as smart pet raising, pet behavior analysis, and smart monitoring. It can be embedded as a core capability into various smart terminals and video analysis platforms, promoting the technological evolution and product innovation in the field of intelligent pet behavior recognition.

[0147] In addition, to adapt to the natural changes in pet appearance over time (such as changes in posture, changes in lighting, etc.), feature extraction can be dynamically adjusted (for example, when the method provided in this application embodiment is deployed on a smart device in a home setting, since pets in a home setting are usually fixed, the task processing model can be fine-tuned and optimized periodically based on pet videos collected by the smart device), thereby improving the task processing model's adaptability to environmental changes in practical applications.

[0148] Reference Figure 5 In some embodiments, this application also provides a pet data processing device 500, which includes:

[0149] Video processing unit 501 is used to acquire target video frames in the target pet video;

[0150] The model invocation unit 502 is used to invoke the pre-trained task processing model, which includes a feature extraction layer, a detection head, and an effective behavior judgment head.

[0151] The feature extraction unit 503 is used to call the feature extraction layer to perform feature extraction based on the target video frame, and obtain multi-level features of the target video frame, including first-level features and second-level features.

[0152] The object detection unit 504 is used to call the detection head to perform object detection based on the first-level features of the target video frame, and obtain the target object detection result corresponding to the target video frame. The target object detection result is used to indicate whether the predicted target video frame contains a preset object, which includes a pet or includes a pet and a device.

[0153] The behavior judgment unit 505 is used to call the effective behavior judgment head to judge the pet behavior based on the second-level features of the target video frame and the second-level features of the preceding video frame of the target pet video, and to obtain the target behavior probability score corresponding to the target video frame.

[0154] The behavior result determination unit 506 is used to determine the target behavior judgment result of the target video frame based on the behavior probability score and a preset score threshold. The target behavior judgment result is used to indicate whether the target video frame contains valid pet behavior.

[0155] Reference Figure 6 , Figure 6 The hardware structure of a computer device according to another embodiment is illustrated. The computer device includes:

[0156] The processor 601 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0157] The memory 602 can be implemented as a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 602 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 602 and is called and executed by the processor 601 using the pet data processing method of the embodiments of this application.

[0158] The input / output interface 603 is used to implement information input and output;

[0159] The communication interface 604 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0160] Bus 605 transmits information between various components of the device (e.g., processor 601, memory 602, input / output interface 603, and communication interface 604);

[0161] The processor 601, memory 602, input / output interface 603, and communication interface 604 are connected to each other within the device via bus 605.

[0162] This application also provides a storage medium storing a computer program, which, when executed by a processor, implements the pet data processing method provided in this application.

[0163] This application also provides a computer program product, which includes a computer program. A processor of a computer device reads and executes the computer program, causing the computer device to perform the pet data processing method described above.

[0164] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in this disclosure and the foregoing drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “including,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatuses.

[0165] It should be understood that in this disclosure, "at least one item" means one or more, and "more than one" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0166] It should be understood that in the description of the embodiments of this application, "multiple" means two or more, "greater than", "less than", "exceeding" etc. are understood to exclude the number itself, and "above", "below", "within" etc. are understood to include the number itself.

[0167] In the several embodiments provided in this disclosure, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0168] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0169] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0170] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0171] It should also be understood that the various implementation methods provided in this application can be combined arbitrarily to achieve different technical effects.

[0172] The above is a detailed description of the embodiments of this disclosure. However, this disclosure is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of this disclosure. All such equivalent modifications or substitutions are included within the scope defined by the claims of this disclosure.

Claims

1. A pet data processing method, characterized by, The method comprises: acquiring a target video frame in a target pet video; calling a pre-trained task processing model, the task processing model comprising a feature extraction layer, a detection head and an effective behavior judgment head; based on the target video frame, calling the feature extraction layer to perform feature extraction, obtaining multi-level features of the target video frame, the multi-level features comprising first-level features and second-level features; based on the first-level features of the target video frame, calling the detection head to perform object detection, obtaining a target object detection result corresponding to the target video frame, the target object detection result being used to indicate whether the target video frame contains a preset object, the preset object comprising a pet or comprising a pet and a device; based on the second-level features of the target video frame and the second-level features of a previous video frame of the target video frame in the target pet video, calling the effective behavior judgment head to perform pet behavior judgment, obtaining a target behavior probability score corresponding to the target video frame; determining a target behavior judgment result of the target video frame according to the behavior probability score and a preset score threshold, the target behavior judgment result being used to indicate whether the target video frame contains an effective behavior of a pet.

2. The method of claim 1, wherein, The training method of the task processing model comprises: acquiring a sample pet video and performing frame extraction processing on the sample pet video based on a preset frame rate, obtaining a plurality of sample pet images; based on the plurality of sample pet images, calling the feature extraction layer to perform feature extraction, obtaining multi-level features of each sample pet image; based on the first-level features of each sample pet image, calling the detection head to perform object detection, obtaining a sample object detection result corresponding to each sample pet image, the sample object detection result being used to indicate whether the sample pet image contains a preset object; calculating an object detection loss according to the sample object detection result and an object detection label corresponding to the sample pet image; based on the second-level features of each sample pet image and the second-level features of an adjacent frame image of the sample pet image, calling the effective behavior judgment head to perform pet behavior judgment, obtaining a sample behavior probability score corresponding to each sample pet image, and determining a sample behavior judgment result of each sample pet image according to the sample behavior probability score and the preset score threshold; calculating a behavior judgment loss according to the sample behavior judgment result and a behavior judgment label corresponding to the sample pet image; performing parameter adjustment on the feature extraction layer, the detection head and the effective behavior judgment head according to the weighted behavior judgment loss and the object detection loss after the weighted processing of the behavior judgment loss according to a preset weight.

3. The method of claim 2, wherein, The sample object detection result comprises an object detection box and an object detection confidence; The determination method of the behavior judgment label comprises: when the object detection confidence of the sample pet image is greater than a preset confidence threshold, determining the sample pet image as a candidate pet image; The object position detection is performed based on object detection boxes corresponding to the plurality of adjacent candidate pet images, and a behavior judgment label of the plurality of adjacent candidate pet images is determined according to an object position detection result.

4. The method of claim 2, wherein, The method for determining the behavior judgment label comprises: a teacher model is constructed according to model information of the effective behavior judgment head; for each training round, model parameters of the teacher model are updated based on model parameters of the effective behavior judgment head in a previous training round; a pet behavior is judged by calling the teacher model after the parameter update based on second-level features of each sample pet image and second-level features of adjacent frame images of the sample pet image, to obtain a reference behavior probability score corresponding to each sample pet image; a reference behavior judgment result of each sample pet image is determined according to the reference behavior probability score and a preset score threshold, and the reference behavior judgment result is determined as a behavior judgment label of a current training round.

5. The method of claim 3, wherein, The parameter adjustment of the feature extraction layer, the detection head and the effective behavior judgment head according to the behavior judgment loss after the weighting processing and the object detection loss comprises: a joint loss is calculated according to the behavior judgment loss after the weighting processing and the object detection loss; a cross-task consistency loss is calculated according to the sample behavior probability score and the sample object detection result; the parameter adjustment of the feature extraction layer, the detection head and the effective behavior judgment head is performed according to the joint loss and the cross-task consistency loss.

6. The method of claim 5, wherein, The calculation of the cross-task consistency loss according to the sample behavior probability score and the sample object detection result comprises: a detection consistency loss is calculated based on the sample behavior probability score and the object detection confidence; an action dynamic score of an object is determined based on object detection boxes corresponding to the plurality of adjacent candidate pet images; an action consistency loss is calculated based on the sample behavior probability score and the action dynamic score; the cross-task consistency loss is calculated according to the detection consistency loss and the action consistency loss.

7. The method of claim 1, wherein, The effective behavior judgment head comprises a space-time network, a pooling network and a multi-layer perception network; the pet behavior judgment is performed by calling the effective behavior judgment head based on the second-level features of the target video frame and the second-level features of a previous video frame of the target video frame in the target pet video, to obtain a target behavior probability score corresponding to the target video frame, which comprises: a space-time feature is obtained by performing space-time attention calculation on the target video frame and the previous video frame based on the second-level features corresponding to the target video frame and the previous video frame by calling the space-time network; a global semantic vector is obtained by performing pooling processing on the space-time feature by calling the pooling network; the target behavior probability score of the target video frame is obtained by performing pet behavior judgment on the global semantic vector by calling the multi-layer perception network.

8. A pet data processing apparatus, characterized by comprising: The device comprises: a video processing unit configured to obtain a target video frame in a target pet video; The model calling unit is configured to call a pre-trained task processing model, the task processing model comprising a feature extraction layer, a detection head, and an effective behavior judgment head; The feature extraction unit is configured to call the feature extraction layer based on the target video frame to perform feature extraction, and obtain multi-level features of the target video frame, the multi-level features comprising first-level features and second-level features; The object detection unit is configured to call the detection head based on the first-level features of the target video frame to perform object detection, and obtain a target object detection result corresponding to the target video frame, the target object detection result being used to indicate whether the target video frame contains a preset object, the preset object comprising a pet or comprising a pet and a device; The behavior judgment unit is configured to call the effective behavior judgment head based on the second-level features of the target video frame and second-level features of a previous video frame of the target video frame in the target pet video to perform pet behavior judgment, and obtain a target behavior probability score corresponding to the target video frame; The behavior result determination unit is configured to determine a target behavior judgment result of the target video frame according to the behavior probability score and a preset score threshold, the target behavior judgment result being used to indicate whether the target video frame contains an effective behavior of a pet. 9.A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the computer device is configured to perform the method according to any one of claims 1-8 when the computer program is executed by the processor. The processor executes the computer program to implement the pet data processing method in any one of claims 1 to 7.

10. A computer readable storage medium, the storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the pet data processing method in any one of claims 1 to 7.

11. A computer program product, characterised in that, The computer program product comprises a computer program, which is read and executed by a processor of a computer device, so that the computer device executes the pet data processing method in any one of claims 1 to 7.

Images