Pet behavior data acquisition and emotion monitoring method and integrated system

Abstract

The invention provides a pet behavior data acquisition and emotion monitoring method and an integrated system. The method comprises the following steps: firstly, collecting heart rate, body temperature, environment temperature and humidity and videos, unifying a time base by matching short-distance wireless timestamp broadcasting with local time service, performing face mask, foreground separation and skeleton key point extraction on the videos by an end side, and only uploading structural features including key point coordinates and foreground area ratios; time lag is obtained through normalized cross-correlation of the skeleton key point speed sequence and the heart rate sequence, phase alignment is completed, and a first feature sequence is obtained; frequency domain analysis is conducted on the tail trajectory, the auricle pitch angle and the first feature are connected in series to form a second feature sequence, the second feature sequence is input into a cross-modal attention sequential network, and continuous coordinates and discrete categories such as relaxation, excitation and anxiety are output in a combined mode; a deviation degree is calculated based on an individualized rhythm baseline, and a graded alarm is output when the threshold is exceeded and the standard is continuously reached, so that pet emotion monitoring which is privacy-friendly, reliable in alignment and explainable is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of video image processing, in particular to a pet behavior data acquisition and emotion monitoring method and integrated system. BACKGROUND

[0002] In recent years, the number of pet-keeping households has grown rapidly, and pet owners have paid more attention to behavioral health and emotional well-being, leading to the emergence of intelligent pet care products. Existing solutions can be broadly divided into two categories: wearable devices that rely on accelerometers, gyroscopes, heart rate (PPG), and body temperature sensors to count steps, activity levels, or sleep duration; and fixed cameras or mobile cameras that use target detection and tracking for remote pet care and simple behavior recognition. While these solutions can cover basic health indicators and location-based care, they still have significant shortcomings in stable monitoring and interpretation of emotional states.

[0003] First, single-modality or weak fusion is common. Using only IMU or only video cannot cover both physiological and behavioral representations of emotions; even if multiple modalities coexist, there is a lack of reliable alignment and segment division under a unified time base, making it difficult to effectively integrate cross-source information. Second, the scarcity and subjectivity of labels constrain model learning. Emotional labels are costly and inconsistent, and existing methods often use a small number of artificial labels or empirical rules, which can lead to overfitting and domain failure. Third, individual differences and diurnal / weekly rhythms are not fully modeled. Different breeds, sizes, ages, and routines cause significant differences in heart rate, body temperature, and activity patterns, making it difficult to transfer universal thresholds or static models, and ignoring rhythm baselines can amplify false positives and false negatives.

[0004] On the visual side, low light, occlusion, and multiple pets in parallel are common problems in home environments; on the physiological side, PPG is easily affected by hair color, tightness, and motion artifacts, and data loss and noise are frequent. Most products do not provide adaptive sensing and quality-aware weighting, and once a single modality is abnormal, the entire system degrades. Privacy and compliance are also a concern: many camera-based solutions upload raw audio and video to the cloud for analysis, increasing privacy exposure and bandwidth costs; lightweight, de-identified processing on the edge, and only uploading structured features have not yet formed an industry consensus.

[0005] At the algorithm level, existing emotion recognition often stops at direct judgment of discrete categories, lacks continuous characterization and uncertainty estimation in the valence-arousal two-dimensional space, and is difficult to support threshold adaptation and risk grading. The use of fine-grained cues such as tail swing spectrum and ear posture, which are highly related to animal emotions, is insufficient, and cross-modal temporal dependencies are not explicitly modeled. In terms of intervention, existing systems are mostly one-way alarms or remote shouting, lacking a closed loop of "trigger-feedback evaluation-strategy adjustment"; early identification and grading of abnormal behaviors (such as suspected convulsions and persistent high stress) are also not perfect. Finally, as the user base expands, how to achieve continuous model evolution without uploading raw data lacks mature edge-cloud collaboration and federated learning practices. Existing recognition models do not consider pet tail swing and ear activity, nor do they incorporate specific pet tail swing and ear activity data into the network model, resulting in low recognition accuracy.

[0006] In summary, the industry urgently needs a systematic technical solution that completes privacy-friendly feature extraction on the edge side, achieves multi-modal time alignment and robust fusion, combines individualized rhythm baseline and continuous emotion representation, and has closed-loop intervention and abnormal grading alarm capabilities, to stably and explainably monitor pet emotions in complex home environments and weakly labeled conditions. SUMMARY

[0007] To solve the above problems in the prior art, the present application provides a pet behavior data acquisition and emotion monitoring method, which can realize privacy-friendly, reliable and explainable pet emotion monitoring.

[0008] The present application provides a pet behavior data acquisition and emotion monitoring method, comprising the steps of: S1: collecting pet data by wearable devices and fixed cameras, the pet data including heart rate value data, pet body temperature, environmental temperature and humidity, and video data; unifying the timestamps of the pet data to the same time base through short-range wireless timestamp broadcasting and local time service correction; S2: performing privacy preprocessing on the video data on the edge side, including face detection and masking, foreground and background separation, pet skeleton key point extraction, forming a structured feature sequence containing key point coordinates and foreground area ratio, and uploading to the server, the original video frames are not uploaded and deleted on the edge side; S3: within a fixed-length sliding window, based on the heart rate change exceeding a preset adaptive threshold, the heart rate value data, pet body temperature, and environmental temperature and humidity are divided into multiple segments, the normalized cross-correlation of the skeleton key point speed sequence and the heart rate sequence is calculated, the time lag with the maximum correlation coefficient is taken to shift the pet data features as a whole to achieve phase alignment, and a cross-modal first feature sequence is generated; S4: performing frequency domain analysis on the time sequence trajectory of the tail key point in the first feature sequence to obtain a tail swing height, and calculating an auricle pitch angle with the head-neck connecting line as a reference, and concatenating the tail swing height and the auricle pitch angle with the first feature sequence to form a second feature sequence; S5: inputting the second feature sequence into the trained cross-modal attention time sequence network model, and outputting a continuous emotion coordinate and a discrete emotion category; S6: establishing an individualized baseline based on the diurnal rhythm of the target pet, and calculating a deviation degree of the continuous emotion coordinate and the discrete emotion category output in step S5 from the baseline; S7: outputting a graded alarm when the deviation degree exceeds a set threshold and a preset duration is met.

[0009] Preferably, the unification of the pet data timestamps to the same time base through short-range wireless timestamp broadcasting and correction by the local time service includes: S11: the gateway device of the local time service as a master clock sends a synchronization message containing a master clock timestamp and a sequence number through short-range wireless broadcasting at a fixed period, and sends a subsequent message carrying the accurate sending time of the synchronization message; S12: the wearable device and the fixed camera record local receiving time respectively when receiving the synchronization / subsequent message, and calculate the initial time difference from the master clock according to the message; S13: the wearable device and the fixed camera respectively send a delay request to the gateway and receive a delay response, and calculate the time difference of the link and the master clock based on the four time points 、 、 、 S14: taking the time difference as a clock offset and correcting the local clock according to a preset linear drift rate, and performing offset and drift compensation on the original timestamps of the data collected by each modality, so as to unify the timestamps to the same time base with the gateway as a reference.

[0010] Preferably, step S2 includes: performing face detection by using a first convolutional neural network, and performing irreversible mask processing on the pixels in the detection frame to de-identify; performing foreground and background separation by using background subtraction to obtain a binary foreground mask, and retaining the pet target by morphological opening and closing operation and median filter denoising; performing calculation in the foreground area by using a pet pose estimation network, and outputting pixel coordinates and confidence of at least 16 skeleton key points; ​The ratio of the number of pixels in the foreground mask to the total number of pixels in the frame is used to calculate the foreground area ratio. The unified time base timestamp, key point coordinates, and foreground area ratio are then combined in chronological order to form a structured feature sequence. The terminal only uploads this structured feature sequence to the server, and the original video frames are not uploaded and are deleted on the terminal.

[0011] Preferably, dividing the heart rate data, pet body temperature, and ambient temperature and humidity into multiple segments includes: using window length... and step length Sliding statistics are performed on the heart rate value sequence to calculate the mean and standard deviation for each window. An adaptive threshold is set between the window mean and the standard deviation. The sum of the standard deviations of the windows; when the heart rate change exceeds the adaptive threshold within any window, the window boundary is marked, and the heart rate value, body temperature and ambient temperature and humidity are divided into multiple segments according to the adjacent boundaries.

[0012] Preferably, generating the cross-modal first feature sequence includes: performing first-order difference on the pixel coordinate sequence of key points of the pet skeleton to obtain the velocity sequence of each key point, and weighting and summing the velocity moduli according to the confidence of the key points to form the velocity sequence of the skeleton key points. At the same time, bandpass filtering is performed to eliminate noise. Within a given time delay search range, the normalized cross-correlation function between the velocity sequence of the skeleton key points and the heart rate sequence is calculated, and the time delay with the largest correlation coefficient is obtained. The key point coordinates, skeleton key point velocity sequence and foreground area ratio time series are translated along the time axis and resampled to a unified time base, thereby completing the phase alignment with the heart rate sequence. The aligned pet data features are then concatenated in time order to form the cross-modal first feature sequence.

[0013] Preferably, forming the second feature sequence includes: extracting the coordinates of the tail root key point and the tail tip key point from the first feature sequence to construct a tail lateral displacement signal. ,right Segmented windowing and Welch power spectrum estimation were performed to calculate the energy in the high-frequency band of 3-8Hz. With low frequency band 0.3-3Hz energy The tail swings high ; Using the head-neck line vector For reference, take the auricle vector. Calculate the pitch angle of the auricle And its mean is calculated over the same time window as the power spectrum; The results obtained in each time window , Align the corresponding time windows in the first feature sequence with a unified time base and concatenate them according to dimensions, so that the feature vector corresponding to each time window contains at least a timestamp and the first feature vector. , The window-level feature vectors arranged in chronological order constitute the second feature sequence.

[0014] Preferably, the cross-modal attention temporal network model includes: Input adaptation layer: The second feature sequence is divided into visual subsequence and environmental subsequence according to its source, and resampled according to a uniform sampling period. Missing positions are marked with a mask. The visual subsequence includes the trajectory coordinates of skeleton key points, foreground area ratio, high and low frequency energy ratio of tail swing, and ear pitch angle. The environmental subsequence includes heart rate value, body temperature, and ambient temperature and humidity. Visual branch temporal encoder: performs temporal encoding on visual subsequences and explicitly outputs the visual features at each time step, including the mean and variance of the skeleton keypoint coordinates in the current window; Environmental branch temporal encoder: performs temporal encoding on the environmental subsequence and outputs the environmental features at each time step, including the mean and variance of pet body temperature and environmental temperature and humidity; Cross-modal multi-head attention fusion layer: Using the visual branch output as the query and the environmental branch output as the key and value, cross-modal attention is calculated, and the output fusion temporal features include the relative time delay that maximizes the attention weight, so as to explicitly characterize the temporal dependency between visual and environmental signals; Temporal aggregation layer: Attention pooling is performed on the fused temporal features to obtain fragment-level sentiment embedding vectors; Dual output heads: one is a regression head that outputs two-dimensional continuous emotion coordinates; the other is a classification head that outputs the probability distribution of discrete emotion categories, where the maximum probability is used as the confidence level of the category. Discrete emotion categories include relaxation, excitement, anxiety, and stress.

[0015] This invention also provides an integrated system for pet behavior data collection and emotion monitoring, comprising: Data acquisition module: Pet data is collected by wearable devices and fixed cameras. The pet data includes heart rate data, pet body temperature, ambient temperature and humidity, and video data. Pet data timestamps are unified to the same time base by broadcasting short-range wireless timestamps and correcting them with local time service. The edge processing module performs privacy preprocessing on the video data on the edge, including face detection and masking, foreground and background separation, pet skeleton key point extraction, forming a structured feature sequence containing key point coordinates and foreground area ratio and uploading it to the server. The original video frames are not uploaded and are deleted on the edge. First feature sequence generation module: Within a fixed-length sliding window, based on the heart rate change exceeding a preset adaptive threshold, the heart rate data, pet body temperature, and ambient temperature and humidity are divided into multiple segments. The normalized cross-correlation between the velocity sequence of the skeleton key points and the heart rate sequence is calculated. The time lag with the largest correlation coefficient is used to shift the pet data features as a whole to achieve phase alignment and generate a cross-modal first feature sequence. Second feature sequence generation module: Perform frequency domain analysis on the temporal trajectory of the tail key points in the first feature sequence to obtain the tail swing height, and calculate the ear pitch angle with the head-neck line as a reference. Connect the tail swing height and ear pitch angle with the first feature sequence to form the second feature sequence. Processing module: Inputs the second feature sequence into the trained cross-modal attention temporal network model and outputs continuous emotion coordinates and discrete emotion categories; Deviation Calculation Module: Establishes an individualized baseline based on the target pet's diurnal rhythm, and calculates the deviation between the continuous emotion coordinates and discrete emotion categories output in step S5 and the baseline; Tiered alarm module: When the deviation exceeds the set threshold and meets the preset duration, a tiered alarm is output.

[0016] The beneficial technical effects achievable by this invention are as follows: 1. This application performs face detection and masking, foreground / background separation, and skeleton key point extraction on the edge, uploading only structured features such as "key point coordinates / confidence and foreground area ratio." The original video frames are not uploaded and are deleted on the edge, significantly reducing the exposure of identifiable information about family members and the home environment, thus meeting privacy compliance requirements. Simultaneously, the structured features are smaller and less redundant than the original video, resulting in an order-of-magnitude reduction in uplink bandwidth and cloud storage costs, allowing for continuous operation at the edge. By using short-range wireless timestamp broadcasting in conjunction with local time services, heart rate, body temperature, environmental, and visual features are unified to the same time base, avoiding the risks of redundant calculations and missynchronization caused by cloud re-alignment. The entire link is lighter, more stable, and easier to deploy.

[0017] 2. This application solves the information loss caused by misalignment of multi-source data by using an adaptive threshold of mean-standard deviation of heart rate within a fixed-length sliding window, and obtaining time lag by normalized cross-correlation between the skeletal keypoint velocity sequence and the heart rate sequence, and achieving phase alignment by overall translation of visual features. Furthermore, the tail swing height and ear pitch angle (referenced to the head-neck line) are derived from the first feature sequence and concatenated with the original sequence to form a second feature sequence, explicitly introducing fine-grained cues highly correlated with emotional positivity and activation. A cross-modal attentional temporal network encodes visual and physiological / environmental subsequences respectively and performs multi-head attention fusion in the temporal dimension, outputting a two-dimensional continuous coordinate of valence-awakening and a discrete emotion category in one go, improving recognition accuracy and interpretability through two paths: "trajectory-level continuous expression + category-level decision-making." In a preferred embodiment, a dynamic temperature-based activation function driven by the ear angle and tail energy ratio is introduced, improving convergence and scene generalization without changing the network size.

[0018] 3. This application establishes a rhythmic baseline for each pet, standardizes valence and arousal by time period, and obtains a deviation measurement that matches its own daily routine, significantly reducing the interference of breed, size, age differences, and diurnal fluctuations on threshold determination. The alarm side adopts a hierarchical strategy of "deviation threshold + shortest duration" and configures hysteresis removal / degradation rules to effectively suppress false triggers under short-term noise, transient excitement, or occasional occlusion conditions; at the same time, it retains at least two levels of output, such as normal and high priority, to facilitate differentiated response and resource scheduling in subsequent systems. Based on a modular link of "data acquisition and time alignment—edge-side privacy processing—two-level feature sequence—model inference—baseline and hierarchical alarm," the system has inherent robustness to sensor loss or short-term anomalies, low on-site installation and maintenance costs, and is easy to deploy quickly in various scenarios such as homes, boarding facilities, and medical front-ends.

[0019] 4. This application introduces a "temperature-driven" activation function, highly driven by ear pitch angle and tail wagging, into the hidden layer of a cross-modal attention temporal network. This function adapts the nonlinear intensity to behavioral physiological states: temperature rises during high activation / tension, suppressing excessive amplification of noise and vigorous movement; sensitivity is restored during relaxation. This mechanism, without altering the network structure or inference latency, acts as context-aware regularization and gain control, while suppressing gradient saturation / explosion, accelerating convergence, significantly reducing false alarms such as "excitement misjudged as anxiety," and improving robustness and generalization across scenarios and individuals. Furthermore, the temperature is driven by interpretable metrics, facilitating operation and review, and works in conjunction with rhythmic baselines and hierarchical alarms to further stabilize alarm triggering. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a flowchart of the steps of a pet behavior data collection and emotion monitoring method according to the present invention; Figure 2 This is a schematic diagram of an integrated system for pet behavior data collection and emotion monitoring according to the present invention. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] Example 1: In view of the aforementioned problems mentioned in the prior art, and in order to solve the above technical problems, as shown in the appendix. Figure 1 As shown: This application provides a method for collecting pet behavior data and monitoring emotions, including the following steps: S1: Pet data is collected by a wearable device and a fixed camera. This pet data includes heart rate data, pet body temperature, ambient temperature and humidity, and video data. The pet data timestamps are unified to a common time base via short-range wireless timestamp broadcasting and local time service correction. In some embodiments, the system composition and sampling frequency are as follows: Wearable device (collar): PPG heart rate module (built-in IIR noise reduction, outputting frame-by-frame heart rate and signal quality), sampling and downsampling to 25Hz; body surface temperature 1Hz; battery and wireless module (BLE 5.0). Fixed camera: 1080p, 30fps, built-in edge computing unit (for edge-side video processing, covered in S2). Gateway: Home router or mobile app (one of the two), supporting BLE and Wi-Fi, serving as the master clock for the local time service. Environmental sensor (integrated in the collar or camera base): Temperature and humidity 0.2Hz (every 5s). Unified time base: UNIX Epoch (milliseconds), 64-bit integer timestamp. Time synchronization method and parameters: Master and slave roles: The gateway is the master clock, and the collar and camera are slave clocks. The gateway broadcasts a synchronization message every 2 seconds via BLE, following the PTP concept but implemented on BLE: SYNC: carries the master clock's coarse time and sequence number; FOLLOW_UP: immediately following SYNC, carrying the precise transmission time of SYNC; The slave device receives two messages, records the local reception time, and obtains an initial time difference estimate. Link delay estimation (unicast every 10 seconds): The slave device sends DELAY_REQ to the gateway, and the gateway immediately replies with DELAY_RESP. Four time points are marked at both ends (t1 for transmission, t2 for reception, t3 for reply, t4 for reception). One-way delay and clock offset are calculated and smoothed using a first-order **PLL / exponential moving average (α=0.2)**. Clock correction strategy: If the offset |offset| ≤ 50ms, a slew (gradual adjustment) method is used to equalize at a rate not exceeding 100ppm to avoid time jumps; if |offset| > 50ms (e.g., upon initial power-on), step (one-time correction) is allowed, and "synchronization quality = degraded" is marked. Resynchronization cycle: Normally 10–15s; reduced to 5s when drift > 20ppm or temperature surge (> 3℃ / min) is detected.

[0024] The timestamp marking specifications are as follows: Heart rate uses the time of the frame-by-frame detection (PPG peak) as the "sample time," which is marked by the driver at the interrupt layer and then mapped to a unified time base; Body temperature / humidity: the sampling thread marks the timestamp and attaches the sensor serial number; Video frames: the frame timestamp is the hard interrupt time of SensorStart-of-Frame, with higher accuracy than application layer timing; All data frames include a time base version and synchronization quality flag (good / degraded / out of synchronization). Out of synchronization and conformal preservation: If no synchronization message is received within 30 seconds, the system enters Holdover mode: extrapolation is performed using the most recent offset and drift rate, and "synchronization quality = out of synchronization" is set; acquisition does not stop, but the upstream algorithm downweights or skips such data according to rules during the phase alignment (S3) stage. After synchronization is restored, the system is first pulled back in slew mode to avoid a break in the timeline between video and physiological data. Security and energy consumption: Synchronization messages include HMAC verification to prevent forgery; the collar end relaxes the unicast delay request interval to 20–30s during non-interactive periods, with a typical power consumption increase of <1mA.

[0025] Wearable devices and cameras cache small batches of data locally in TLV (Type-Length-Value) format (heart rate / body temperature / environment data per 1–5 seconds, video features per frame). Each packet contains: a unified time base timestamp, synchronization quality, sensor ID, sampling period, and serial number. Data is transmitted to the gateway via BLE / Wi-Fi, which either forwards it to the server or caches it locally (retransmitting after network disconnection) to ensure proper timing. Alignment accuracy and normal acceptance alignment error (collar ↔ camera) ≤ ±20ms; error within Holdover 60s ≤ ±40ms. During production line operation or initial installation, a one-time verification is performed using a consensus event triggered by the camera's flash / buzzer: the relative time difference between the heart rate peak, frame time, and flash event should all fall within the above thresholds.

[0026] Abnormal scenarios are handled as follows: If temperature drift is detected causing crystal oscillator offset (>50ppm for 10s), the system automatically increases the resynchronization frequency and records the maintenance log; if multiple resynchronization failures occur, the mobile app is switched to the master clock, and "synchronization source = mobile device" is written to the metadata; if synchronization is lost for a long time (>10min), data acquisition continues, but it is marked as "unusable for cross-modal phase alignment" and is only used for low-value statistics (such as steps / activity). Through the above embodiments, the "data acquisition and unified time base" required in S1 is engineered and implemented: in a home environment without additional wiring, a lightweight synchronization centered on the gateway and combining BLE broadcast + unicast latency measurement is achieved, ensuring that heart rate / body temperature / environment and video features can be accurately aligned in the subsequent cross-modal processing stages of S2–S7.

[0027] S2: Perform privacy preprocessing on the video data on the device side, including face detection and masking, foreground and background separation, pet skeleton key point extraction, forming a structured feature sequence containing key point coordinates and foreground area ratio and uploading it to the server, while the original video frames are not uploaded and are deleted on the device side. In some embodiments, the operating environment and input fixed camera capture video streams at 1080p, 30 frames per second, with a built-in edge computing unit; the device and gateway have achieved a unified time base in S1. The edge only processes the current and very short historical frames in local memory, without writing the original frames to disk. Stream capture and timestamp annotation: each frame obtains a precise start timestamp at the sensor layer, along with a "synchronization quality" flag, which is retained throughout processing. Face / body detection and masking (de-identification): a lightweight object detection is performed once per frame, with detection categories including adult and child faces and upper body bodies. Irreversible processing is applied to the detection bounding box regions: fixed blackening is preferred; if selected by the user, it can be switched to pixelated mosaic. Masked regions are recorded in metadata for auditing purposes. This step occurs before any image content is exported, ensuring that all subsequent intermediate and final data have been de-identified.

[0028] Foreground and background separation is based on a combination of background modeling and inter-frame differencing: During initialization, a static background is learned within 2 seconds. During runtime, a foreground mask is calculated for each frame, and temporal median filtering and morphological opening / closing operations are used to remove noise and fill small holes. In low-light or nighttime infrared mode, the process switches to grayscale processing, still outputting a binary foreground mask. If multiple large foreground areas appear in the scene, only the largest connected component consistent with the historical trajectory is retained to reduce false detections.

[0029] Pet target tracking and multi-pet differentiation: For families with multiple pets, lightweight tracking is additionally enabled. Body proportions and stripe descriptions are extracted from the foreground region, and short-term IDs are maintained based on spatial continuity to ensure that skeletal key points and foreground areas originate from coherent segments of the same pet. Pet skeletal key point extraction: A pet pose estimation network is run in the foreground region, outputting pixel coordinates of at least 16 key points, covering the head, neck, shoulders, hips, limb joints, ears, tail root, and tail tip. Occluded or low-confidence key points are interpolated or marked as missing, and a frame-level quality flag is written. If the quality of key points in an entire frame is below a threshold, the entire frame is marked as "invalid," and only the foreground area ratio is retained for statistical purposes. Quality control and anomaly handling: Sharpness indicators and motion blur indicators are calculated for each frame. If multiple consecutive frames fail to meet quality standards, the skeleton inference frequency is reduced to save energy, and a "degradation mode" is recorded in the metadata. When the camera is obstructed or the screen is completely black, empty structured features are directly output, and an anomaly event is reported, facilitating upstream strategy downgrading or skipping.

[0030] The foreground area ratio is calculated by dividing the number of pixels in the foreground mask by the total number of pixels in the frame. If multiple people or large objects exist in the same frame but are not connected to the pet, interference is eliminated using the tracking constraints from the previous step. Structured feature sequence assembly generates a structured record at the frame level, with fields including at least: a unified time base timestamp, frame number, a list of keypoint coordinates, foreground area ratio, processing quality flag, mask region summary, and device identifier. To save bandwidth, keypoint coordinates can be quantized into an integer grid, and the sequence can be incrementally encoded. This structured data does not contain any original pixels or reversible image content.

[0031] Encrypted transmission and local deletion: Structured features are packaged in small batches on the device side, with integrity verification and signature added, and then uploaded to the server or gateway cache via a secure channel. Original video frames always exist only in the memory buffer, retained for no more than one second, and are immediately overwritten and released after the above steps are completed. In case of an abnormal crash, the device will first perform a secure erasure process upon restart. Operating parameters and power consumption: Face detection is executed every frame by default, but can be reduced to every other frame in static scenes to save energy; skeleton inference can dynamically adjust its frequency as needed, such as 30, 15, or 10 frames per second. The entire chain runs in real-time on a regular edge NPU or CPU, with controllable maximum power consumption on the device side. Output and guarantees to upstream: Output content is only the structured feature sequence (metadata such as keypoint coordinates and foreground area ratio), excluding original audio and video.

[0032] Alignment ensures that each record carries a unified time base timestamp and synchronization quality, guaranteeing consistency with the time alignment requirements of S1 and providing a reliable foundation for the fragmentation and phase alignment of S3. Privacy compliance: Any area that might identify family members is masked on the device side; if detection fails, the structured output of the frame is discarded directly according to the policy, and an alarm is recorded to prevent the outward transmission of unmasked information. Through the above embodiments, the closed loop of S2, "device-side privacy preprocessing—skeleton and foreground area extraction—structured feature uploading—original frame not uploaded and deleted," can be directly implemented in engineering, satisfying both privacy and bandwidth constraints, and providing stable and interpretable input for subsequent multimodal fusion and sentiment inference.

[0033] S3: Within a fixed-length sliding window, based on the heart rate change exceeding a preset adaptive threshold, the heart rate data, pet body temperature, and ambient temperature and humidity are divided into multiple segments. The normalized cross-correlation between the velocity sequence of the skeleton key points and the heart rate sequence is calculated. The time lag with the largest correlation coefficient is used to shift the pet data features as a whole to achieve phase alignment and generate the first feature sequence across modalities. In some embodiments, the input data comes from the structured visual features of S2 (frame-by-frame keypoint coordinates, keypoint confidence, foreground area ratio, all with a unified time base timestamp), as well as heart rate values, pet body temperature, and environmental temperature and humidity sequences (with the same time base). Sampling and Windowing: Segment detection uses a fixed-length sliding window with a window length of 4 seconds and a step size of 1 second; the phase-aligned time-delay search range is set to 2 seconds before and after. Quality Flag: Any missing or low-confidence frames are marked with a quality flag for subsequent downweighting or skipping.

[0034] Based on heart rate fragmentation, the recent level and fluctuation amplitude of heart rate are calculated within a sliding window, and an adaptive threshold (e.g., "window mean plus a certain number of window standard deviations") is generated accordingly. When the heart rate change continuously exceeds the adaptive threshold to a preset number of samples within any window, it is determined to be an "event window". Adjacent or overlapping event windows are merged into fragments; the starting point of each fragment is extended forward by a small buffer (e.g., 2 seconds), and the ending point is extended backward by a small buffer (e.g., 4 seconds). Body temperature, ambient temperature and humidity, and visual features within the same time range are extracted to form candidate fragments. Extremely short fragments (e.g., less than 6 seconds) are discarded, and extremely long fragments are segmented according to the maximum length (e.g., 60 seconds) to ensure the stability of subsequent alignment and modeling.

[0035] A skeleton velocity sequence is constructed to approximate "velocity" by the temporal changes in keypoint coordinates, and the displacement amplitude of keypoints is calculated frame by frame. Using keypoint confidence levels as weights, the overall skeleton "velocity" index for that frame is obtained. To suppress jitter, this index undergoes mild temporal smoothing and denoising. Within the segment range, a skeleton velocity time series with the same path length as the heart rate is obtained, and the quality indicators for each time point are preserved.

[0036] The optimal time delay is determined by normalized cross-correlation. Within the set time delay search range, the similarity between the "skeleton velocity sequence" and the "heart rate sequence" is evaluated one by one to obtain the similarity curves that change with time delay. The time delay with the highest similarity is selected as the "optimal time delay," and the magnitude of the peak value is recorded as the alignment quality score. If the quality score is lower than the threshold (e.g., 0.3), the segment is considered difficult to align reliably: it can be regressed to no translation or a suboptimal alignment strategy can be adopted, and "low confidence alignment" is marked in the segment metadata.

[0037] Visual features are translated and resampled using the optimal time delay. Only the visual features (keypoint coordinates, skeleton velocity derived from the coordinates, and foreground area ratio) are translated along the time axis to align with the heart rate sequence within the segment; the heart rate, body temperature, and environmental sequences are not translated. The translated visual features, heart rate, body temperature, and environmental data are then resampled to the same sampling period (e.g., 20 Hz) while maintaining a unified time base. Insufficient data at boundaries is handled using interpolation or missing data markers, and the quality flag is inherited. The optimal time delay, alignment quality score, and boundary interpolation ratio used for this segment are recorded in the segment metadata for subsequent selection and weight training.

[0038] A cross-modal first feature sequence is generated. Using a uniform sampling period as the time step, the cross-modal vector of the segment is progressively organized, including heart rate value and its variation, body temperature, ambient temperature and humidity, keypoint coordinates after translation and alignment or their low-dimensional representation, skeleton velocity index, foreground area ratio, and alignment metadata (optimal time lag, quality score). These time step vectors are concatenated in chronological order to obtain the cross-modal first feature sequence of the segment. Segment identifiers, start and end timestamps, and quality statistics are written to both the beginning and end of the sequence. For consecutive segments with short gaps (e.g., less than 2 seconds), splicing can be selected, with transition markers inserted in the middle; if the gaps are large, they are output as independent sequences.

[0039] For anomaly and degradation handling, if a segment shows a persistent absence of heart rate data, alignment is not performed; only the visual and environmental subsequences are retained and marked "unaligned." If the visual data shows a continuous low confidence level, the heart rate and environmental subsequences are retained and marked "visual degradation." For drift caused by prolonged high temperatures or extreme humidity, the alignment quality score threshold can be lowered or the time lag search range can be narrowed within the segment to avoid misalignment.

[0040] The output and the guaranteed output to the upstream are several fragment-level cross-modal first feature sequences, each with unified time base, phase alignment information and quality indicators, providing direct input for tail and auricle feature enhancement in S4 and cross-modal attention modeling in S5. The entire process runs online, with new fragments generated as they are detected, having zero dependence on historical data; the window and threshold can be dynamically adjusted according to the variety and age at the factory or through cloud-based strategies. Through this embodiment, the engineering path in S3, "fragmentation—cross-correlation to calculate lag—translation of visual side features by lag—unified resampling—output of first feature sequence", is concretized, ensuring cross-modal temporal consistency and providing executable degradation strategies for noise, missing data and boundary conditions.

[0041] S4: Perform frequency domain analysis on the time-series trajectory of the key points of the tail in the first feature sequence to obtain the tail swing height, and calculate the ear pitch angle with the head-neck line as a reference. Connect the tail swing height and ear pitch angle with the first feature sequence to form the second feature sequence. In some embodiments, the input premise, the first feature sequence from S3, has been aligned to a unified time base and includes, frame by frame, the coordinates and confidence scores of skeletal keypoints (including tail root, tail tip, ear root, ear tip, head, neck, etc.), foreground area ratio, and timestamps and quality indicators aligned with physiological / environmental quantities such as heart rate. Time windows and preprocessing are performed, with a 2-second analysis window and a 1-second step size (50% overlap). De-jittering and interpolation are performed on the keypoint trajectories within each window: if the confidence score of a single frame's keypoint is below a threshold, it is marked as missing; if the missing value persists for no more than 200 milliseconds, linear interpolation is performed; otherwise, the window is marked as "invalid tail / ear feature".

[0042] Tail wagging frequency domain characteristics ("tail energy ratio"), establishing tail displacement signals: Within each time window, the movement of the tail tip relative to the tail root is taken, and the displacement sequence changing with time is calculated along the transverse direction of the body (the vertical direction of the body axis determined by head-neck-hip). Frequency domain analysis: Power spectrum estimation is performed on this displacement sequence, and the cumulative energy of two frequency bands is statistically analyzed: Low frequency band (approximately 0.3–3 Hz): corresponds to slow, large-amplitude tail wagging; High frequency band (approximately 3–8 Hz): corresponds to fast, tense, or highly activated tail wagging. Formation index: The ratio of high-frequency energy to low-frequency energy is used as the "tail energy ratio," and logarithmic compression of the dynamic range is performed when necessary; the dominant frequency and spectral peak clarity of this window are recorded as quality references (only for annotation, not mandatory for judgment). Quality control: When the tail key points are severely obscured or the spectral energy is too low, the "tail energy ratio" of this window is marked as invalid, and a missing marker is used as a placeholder in subsequent concatenation.

[0043] The reference direction for the auricle pitch angle feature is represented by the head-neck line, indicating head orientation. Ear vectors are obtained by taking the vectors from the ear root to the ear tip. Both ears can be used, but the side with higher confidence is preferred, or the average of both ears is taken as the auricle direction for that frame. Angle calculation and statistics: Within the window, the pitch angle of the auricle relative to the head-neck reference is calculated frame by frame, obtaining the mean and dispersion (e.g., quantile difference or standard deviation) of the window, reflecting the ear's forward / backward tilt and stability. Quality control: If the missing rate of ear keypoints is too high (e.g., exceeding half the number of frames in the window), the auricle features for that window are considered invalid.

[0044] Time alignment and "concatenation" generate the second feature sequence. The alignment rules are as follows: the start and end times of the window are strictly aligned with the time steps of the first feature sequence; if the first feature uses high-frequency sampling (e.g., 20Hz), the first features within the window are aggregated according to the window (mean / median / extreme values, etc.) to obtain a vector corresponding one-to-one with the window. Field concatenation: Construct a window-level feature vector for each time window, which includes at least: timestamp (window center or start and end times), window duration; aggregated values ​​of the first feature sequence (heart rate, body temperature, environmental quantities, low-dimensional skeletal representation, foreground area ratio, etc. within the window); tail energy ratio (invalid ones are marked missing); mean and dispersion of auricular pitch angle (invalid ones are marked missing); quality indicators (whether the tail / auricular / aggregation is valid, occlusion ratio, spectral peak clarity). Sequential output: Output the above window-level vectors in chronological order to form the second feature sequence. This sequence maintains a one-to-one correspondence with the first feature in time, facilitating direct consumption by subsequent models. For multi-pet tracking and anomaly handling, if multi-pet tracking is enabled, the second feature sequence is generated separately for each pet ID; when pet switching or tracking drift occurs within the window, the entire window is set to "invalid". In nighttime infrared or low-light scenes, if the quality of ear / tail keypoints generally degrades, a window vector containing only the aggregated value of the first feature can still be output without forcibly interrupting the sequence.

[0045] The frequency band boundaries, window length, and step size can all be adjusted according to breed or age differences through policy distribution. To avoid interference from extreme values, the "tail energy ratio" and "ear pitch angle" can be gently pruned or scaled by quantiles. The second feature sequence can retain the original value or be accompanied by a dimensionless version (such as normalized according to the historical median and quantile width) so that the S5 model and the S6 baseline can be used simultaneously.

[0046] Through the above steps, without increasing the amount of original video sent out, the system structures the tail spectral features and ear pose features that are strongly correlated with the positive or negative nature of emotions and activation, and aligns them with the first feature sequence to form a second feature sequence that can be directly used by cross-modal attention temporal networks, thereby improving the interpretability and robustness of emotion determination.

[0047] S5: Input the second feature sequence into the trained cross-modal attention temporal network model and output continuous emotion coordinates and discrete emotion categories; In some embodiments, the input and preprocessing take the second feature sequence generated by S4 as the model input. This sequence is arranged by time steps, each time step containing: heart rate and its variation, body temperature, ambient temperature and humidity; low-dimensional representation of the visual skeleton, keypoint confidence aggregation values, skeleton "velocity" index, foreground area ratio; high / low frequency energy ratio of tail wagging and ear pitch angle; and alignment quality score and time-lag metadata from S3. The system first performs segmented normalization on all numerical fields (calibrating the cache according to the device and individual's "time-of-day / week") and generates a missing mask: fields with low confidence or unavailable are marked as missing without interpolation. Simultaneously, the sequence is pruned or padded to a fixed duration (e.g., 32 or 64 time steps), time location encoding is added, and it is packaged into a window for a single forward inference.

[0048] The model structure and input mapping consist of two parallel branches and a cross-modal fusion layer: the visual branch primarily uses stacked one-dimensional temporal convolutional networks (TCNs), with the first two layers extracting short-term local dynamics and the last two layers expanding the receptive field to cover a range of 10–15 seconds; the branch outputs time-step-level visual embeddings while preserving quality gating (weighting low-confidence time steps). The physiological / environmental branch also uses a TCN structure, but with longer convolutional kernels to better capture the slow trends in heart rate and temperature / humidity; this branch outputs time-step-level physiological / environmental embeddings.

[0049] Cross-modal multi-head attention fusion layer: Using visual embeddings as queries and physiological / environmental embeddings as keys and values, fine-grained interactions are achieved through local temporal window attention (e.g., 8 time steps before and after), resulting in fused temporal embeddings. The output of this layer also returns the relative time delay and weight distribution of the attention peak as interpretability supplementary information. Dynamic activation (optional subordinate implementation): Temperature-based activation functions are enabled in the two fully connected hidden layers after the fusion layer. The temperature is calculated from the auricle pitch angle and tail energy ratio at the current time step, thereby adaptively adjusting the nonlinear intensity and suppressing noise amplification in high-activation segments.

[0050] The process involves temporal aggregation and dual-head output. The fused temporal embeddings are then attention-pooled (giving higher weights to more important time steps) to obtain fragment-level emotion embedding vectors. This is followed by two parallel output heads: a continuous coordinate regression head, which outputs two continuous values ​​(range 0–1) for valence and arousal, used to characterize the positive or negative nature of emotion and the strength of activation levels; and a discrete classification head, which outputs the probability distributions for four categories: relaxation, excitement, anxiety, and stress, and provides the maximum probability and its corresponding category as the discrete prediction and confidence level for this fragment.

[0051] To improve deployment usability, Monte Carlo dropout inference is initiated several times during the inference phase, and the variance of the output is used as an uncertainty estimate. Simultaneously, temperature calibration is applied to the classification probabilities to ensure comparability between probabilities of different devices and individuals. The output of continuous coordinates is lightly corrected again based on the individual baseline (S6) to ensure consistency across days for the same pet. If a field is missing in the physiological / environmental branch, a mask guides the branch to not generate effective gradients (training) or participate in aggregation (inference) at that time step, while the fusion layer automatically reduces its dependence on this time step. If the visual branch quality remains low and the alignment quality score is below a threshold, the system degrades to primarily using the physiological / environmental branch, and the fusion layer retains only limited attention to adjacent domains. When both branches are of low quality simultaneously, the model still outputs results, but the uncertainty increases significantly, providing a threshold strategy to suppress false triggers. The model runs in real time using the NPU / CPU on the edge or gateway side. The typical step size of the sliding window is 1 second, and the single-window inference latency is controlled within 50–100 milliseconds. When the home network is unstable, the inference is completed entirely locally, and only the structured results and alarm events are uploaded to ensure privacy and real-time performance.

[0052] The output of each segment in the result encapsulation and upstream linkage includes: unified time base start and end times, valence and arousal values, discrete category and confidence level, uncertainty index, relative lag of attention peak, alignment quality score, and branch quality indicator. This output is directly used for baseline deviation calculation in S6 and graded alarms in S7; when the continuous emotional trajectory shows a deteriorating trend but does not reach the alarm threshold, a light prompt or suggested intervention can be triggered on the product side.

[0053] The initial offline training of the model adopts a hybrid strategy of self-supervised and weak labeling (weak supervision is composed of tail energy ratio, auricle angle, heart rate deviation and user light feedback). After going online, the gradient difference is obtained periodically through federated learning mechanism for small updates. The update only changes the weight file and does not change the network topology and end-side interface to ensure compatibility.

[0054] Through the above embodiments, the engineering path of S5, which is to "input the second feature sequence into a cross-modal attention temporal network and output continuous coordinates and discrete categories", has been clarified, realizing fine-grained, interpretable, and deployable real-time inference of pet emotions under the premise of unified time base and privacy protection.

[0055] S6: Establish an individualized baseline based on the target pet's diurnal rhythm, and calculate the deviation of the continuous emotion coordinates and discrete emotion categories output in step S5 from the baseline; In some embodiments, the baseline learning period is segmented by time. After the device goes online, it first enters the baseline learning period (e.g., 7 days). Based on the local time zone, a day is divided into fixed-length time periods (e.g., 30 minutes per segment, 48 segments in total). At the same time, it is distinguished from Monday to Sunday in the week dimension, thus forming a combination bucket of "day of the week × time period in a day" (e.g., 7 × 48 buckets). Each bucket maintains historical statistics for the pet separately. When switching to daylight saving time, it automatically aligns according to the local time zone. If there is one less or more time period on the day of the switch, it is processed by merging adjacent time periods or interpolation. During the learning period, the system collects the output from S5 for each time period: valence, arousal (continuous value), and probability distribution of four types of emotions (discrete output and confidence). The data for the time period is only included in the baseline if it meets the following quality thresholds: the synchronization quality of S1 is "good" or "degraded but within the threshold"; the alignment quality score of S3 is not lower than the set value; the uncertainty of S5 is lower than the set upper limit, and at least one side of the visual / physiological branches is of normal quality. For periods of persistent occlusion, sensory abnormalities, or obvious illness with high fever, these are directly marked as "unsuitable for baseline." For each bucket, the following are maintained: Continuous component: Central level and dispersion of valence and arousal (e.g., using robust mean and quantile width to avoid the influence of extreme values), and the number of valid samples is recorded. Discrete component: Normal probability distribution of the four emotion categories appearing in this bucket. To improve stability, incremental updates are used: new observations are weighted into history with smaller weights (e.g., decaying daily), allowing the baseline to gradually take shape over several days, avoiding short-term abnormal skew. For buckets with insufficient samples (e.g., very little activity at night), a regression to the "weighted average of adjacent time periods within the same day" or the "weekly average of the same time period" is used, and marked as "low confidence." After the learning period, online updates are implemented: daily, qualified data from the previous day are weighted and placed into the corresponding buckets locally, and the time of the most recent update is recorded. If a significant change in daily routine is detected (e.g., a rise or fall in median price and wake-up counts over the same period in the past three days, exceeding a preset range), the update weight of that bucket is increased to allow the baseline to adapt to the new routine more quickly. When a user actively switches "scenario modes" (e.g., relocation, long-term travel), the bucket merging range can be temporarily expanded, and the finer buckets can be restored after the new environment stabilizes. At the inference time, the system locates the current bucket, reads the continuous baseline parameters of that bucket, and performs segmented standardization on the price and wake-up counts at this time to obtain the offset intensity compared to the normal range for that time period. The two-dimensional offset can be combined into a single value (e.g., taking the larger value or a weighted sum) to obtain the "continuous deviation." When the uncertainty given by S5 is high, the weight of this deviation is automatically reduced to prevent low-confidence outputs from triggering false alarms.

[0056] The system calculates the "surprise level relative to normal" by comparing the current probability distribution of the four negative categories with the normal distribution of the corresponding bucket. When the current probability of a negative category (such as anxiety or stress) is significantly higher than the normal level for that period, the surprise level increases significantly; if the current distribution is close to the normal level, the surprise level is lower. To improve usability, this surprise level is gently cropped and numerically calibrated to keep it at a similar level across different individuals.

[0057] The overall deviation is obtained by superimposing "continuous deviation" and "discrete surprise" with preset weights. The weights are adaptively adjusted based on the following factors: which side (continuous or discrete) has historically been more stable during this period, the more stable side is given higher weight; which side has higher quality and lower uncertainty in the current window is also given higher weight. To suppress instantaneous spikes, the overall deviation is smoothed within a short sliding window; if a single high value appears but then immediately falls back, it is not directly triggered downstream, ensuring noise resistance. For boundary and missing value handling, if the current bucket sample is severely insufficient (e.g., the first day of a new installation), estimation is regressed to "adjacent time periods within the same day" or "weekly average of the same time period," and the upper limit of the overall deviation is tightened to avoid excessive sensitivity initially. If either the continuous or discrete side is missing (e.g., vision is temporarily unavailable), the deviation is calculated based solely on the other side, while the overall confidence level is lowered.

[0058] When transitioning between midnight and weekdays, a smooth transition between the preceding and following bucket positions is maintained to avoid abrupt boundary changes. The output of the system integrated with S7 outputs the following at each time step: current bucket position identifier, continuous deviation, discrete deviation, overall deviation, used fallback level, and quality flag. These values ​​are input into S7's tiered alarm strategy: a Level 1 or Level 2 alarm is triggered only when the overall deviation continuously exceeds the Level 1 or Level 2 threshold for a preset duration; when the deviation falls below the cancellation threshold and remains below it for a set duration, the alarm is automatically cancelled or downgraded. The threshold can be dynamically fine-tuned based on individual historical false alarm rates and owner feedback.

[0059] This implementation significantly reduces misjudgments caused by individual differences and diurnal fluctuations without altering the S5 model structure. It employs a combination of time-based baselines, robust statistics, and online fine-tuning. The comprehensive deviation reflects both shifts in continuous emotional trajectories and captures abnormal increases in discrete categories during specific time periods, meeting the stability requirements for subsequent tiered alerts and closed-loop interventions. During deployment, it is recommended to conservatively set thresholds and enable stronger rollback and smoothing during the first two weeks to ensure a smooth user experience during the cold start phase.

[0060] S7: When the deviation exceeds a set threshold and meets the preset duration, a tiered alarm is output. In some embodiments, the edge device evaluates the overall deviation D in 1-second increments. Two threshold levels are set: T1 and T2 (T2 > T1). Within a 60-second sliding window, and with normal synchronization / alignment quality and model uncertainty below the upper limit: if D is continuously ≥ T1 and cumulatively ≥ 90 seconds, a level 1 alarm is triggered; if D is continuously ≥ T2 and cumulatively ≥ 60 seconds, a level 2 alarm is directly triggered and overrides the level 1 alarm. Hysteresis debouncing is used: to remove level 1, D must be continuously below T1-Δ for 60 seconds; to downgrade from level 2 to level 1, D must be continuously below T2-Δ for 90 seconds. The alarm payload includes a unified time base timestamp, duration, peak D, price / wake-up, discrete category and confidence level, and alignment / branch quality flag. After triggering, a 2-minute cooldown period is entered, during which only upgrades are allowed and no repeated reminders are permitted; when level 2 persists for ≥ 5 minutes, intervention suggestions are pushed synchronously and user / device feedback is recorded for subsequent threshold adaptive adjustment.

[0061] Preferably, unifying the pet data timestamps to the same time base through short-range wireless timestamp broadcasting and correction using a local time service includes: S11: The gateway device that provides local time service acts as the master clock and sends a synchronization message containing the master clock timestamp and sequence number via short-range wireless broadcast at fixed intervals, and sends subsequent messages carrying the precise time of transmission of the synchronization message. S12: When the wearable device and the fixed camera receive the synchronization / follow-up message, they respectively record the local reception time and calculate the initial time difference with the master clock based on the message; S13: The wearable device and the fixed camera respectively send a delay request to the gateway via unicast and receive a delay response, based on four time points. , , , Calculate the time difference between the link delay and the master clock; S14: Using the time difference as the clock offset and correcting the local clock according to a preset linear drift rate, offset and drift compensation are performed on the original timestamps of the data collected in each mode, thereby unifying the timestamps to the same time base based on the gateway.

[0062] Preferably, step S2 includes: performing face detection using a first convolutional neural network, and performing irreversible masking processing on the pixels within the detection box to de-identify them; Background subtraction is used to separate the foreground and background, resulting in a binary foreground mask. Morphological opening and closing operations and median filtering are then used to denoise and retain the pet target. A pet pose estimation network is used within the foreground region to calculate the pixel coordinates and confidence scores of at least 16 skeleton key points. The ratio of the number of pixels in the foreground mask to the total number of pixels in the frame is used to calculate the foreground area ratio. The unified time base timestamp, key point coordinates, and foreground area ratio are combined into a structured feature sequence in chronological order. Only this structured feature sequence is uploaded to the server on the client side; the original video frames are not uploaded and are deleted on the client side.

[0063] Preferably, dividing the heart rate data, pet body temperature, and ambient temperature and humidity into multiple segments includes: using window length... and step length Sliding statistics are performed on the heart rate value sequence to calculate the mean and standard deviation for each window. An adaptive threshold is set between the window mean and the standard deviation. The sum of the standard deviations of the windows; when the heart rate change exceeds the adaptive threshold within any window, the window boundary is marked, and the heart rate value, body temperature and ambient temperature and humidity are divided into multiple segments according to the adjacent boundaries.

[0064] Preferably, generating the cross-modal first feature sequence includes: performing first-order difference on the pixel coordinate sequence of key points of the pet skeleton to obtain the velocity sequence of each key point, and weighting and summing the velocity moduli according to the confidence of the key points to form the velocity sequence of the skeleton key points. At the same time, bandpass filtering is performed to eliminate noise. Within a given time delay search range, the normalized cross-correlation function between the velocity sequence of the skeleton key points and the heart rate sequence is calculated, and the time delay with the largest correlation coefficient is obtained. The key point coordinates, skeleton key point velocity sequence and foreground area ratio time series are translated along the time axis and resampled to a unified time base, thereby completing the phase alignment with the heart rate sequence. The aligned pet data features are then concatenated in time order to form the cross-modal first feature sequence.

[0065] Preferably, forming the second feature sequence includes: extracting the coordinates of the tail root key point and the tail tip key point from the first feature sequence to construct a tail lateral displacement signal. ,right Segmented windowing and Welch power spectrum estimation were performed to calculate the energy in the high-frequency band of 3-8Hz. With low frequency band 0.3-3Hz energy The tail swings high ; Using the head-neck line vector For reference, take the auricle vector. Calculate the pitch angle of the auricle And average it over the same time window as the power spectrum; the values ​​obtained in each time window , Align the corresponding time windows in the first feature sequence with a unified time base and concatenate them according to dimensions, so that the feature vector corresponding to each time window contains at least a timestamp and the first feature vector. , The window-level feature vectors arranged in chronological order constitute the second feature sequence.

[0066] Preferably, the cross-modal attention temporal network model includes: an input adaptation layer: dividing the second feature sequence into a visual subsequence and an environmental subsequence according to its source, resampling it according to a uniform sampling period, and marking missing positions with a mask; the visual subsequence includes the trajectory coordinates of skeleton key points, foreground area ratio, high and low frequency energy ratio of tail wagging, and ear pitch angle; the environmental subsequence includes heart rate value, body temperature, and ambient temperature and humidity. Visual branch temporal encoder: performs temporal encoding on visual subsequences and explicitly outputs the visual features at each time step, including the mean and variance of the skeleton keypoint coordinates in the current window; Environmental branch temporal encoder: performs temporal encoding on the environmental subsequence and outputs the environmental features at each time step, including the mean and variance of pet body temperature and environmental temperature and humidity; Cross-modal multi-head attention fusion layer: Using the visual branch output as the query and the environmental branch output as the key and value, cross-modal attention is calculated, and the output fusion temporal features include the relative time delay that maximizes the attention weight, so as to explicitly characterize the temporal dependency between visual and environmental signals; Temporal aggregation layer: Attention pooling is performed on the fused temporal features to obtain fragment-level sentiment embedding vectors; Dual output heads: one is a regression head that outputs two-dimensional continuous emotion coordinates; the other is a classification head that outputs the probability distribution of discrete emotion categories, where the maximum probability is used as the confidence level of the category. Discrete emotion categories include relaxation, excitement, anxiety, and stress.

[0067] Preferably, calculating the deviation of continuous emotion coordinates, discrete emotion categories, and the baseline includes: (i) Divide historical data into time periods, such as a day or a week, and establish the continuous emotional coordinate value, arousal baseline mean and standard deviation, and discrete emotional category baseline probability distribution for the target pet in each time period. (ii) Standardize the output continuous emotional coordinates and arousal over the corresponding time period to obtain their respective standard residuals; the larger of the two absolute values ​​is recorded as the continuous deviation. (iii) Using the baseline probability distribution from step (i) as a reference, calculate the cross-entropy between the output category probability distribution and the baseline distribution to obtain the category deviation. (iv) The continuous deviation and the category deviation are linearly combined according to preset weights to obtain the final deviation for alarm determination; The continuous emotion coordinates are two-dimensional numerical pairs of valence and arousal, where valence represents the positive or negative nature of the emotion and arousal represents the level of physiological activation. The temporal features after cross-modal attention fusion are output by the regression head, which outputs the values ​​of valence and arousal respectively.

[0068] Preferably, the output of graded alarms includes: outputting a first-level alarm when the deviation exceeds a first deviation threshold continuously within a sliding time window and the duration is not less than the first minimum duration; and outputting a second-level alarm when the deviation exceeds a second deviation threshold continuously and the duration is not less than the second minimum duration.

[0069] This invention also provides an integrated system for pet behavior data collection and emotion monitoring, comprising: Figure 2 As shown, this invention provides a set of implementable hardware and connection topology for an integrated pet behavior data acquisition and emotion monitoring system, including a wearable terminal (collar / bridle), sensors, PPG heart rate (outputting frame-by-frame heart rate values ​​and quality indicators), and skin temperature probes. Communication is BLE 5.0 (broadcast + GATT), supporting encryption. A low-power MCU with small-capacity RAM / Flash is used for timestamps and data packaging. Power supply: rechargeable lithium battery with power management (overcharge and over-discharge protection). Waterproof and sweatproof (IP67 recommended), quick-release strap. Vibration motor / LED indicator for local prompts. Fixed camera (integrated with end-side processing unit) imaging, RGB / IR sensor, 1080p / 30fps (or higher), night vision infrared illumination and IR-CUT. Built-in SoC / NPU performs face detection masking, foreground / background separation, pet skeleton key point extraction and feature packaging; only briefly caches the current frame in memory. Wi-Fi (2.4 / 5GHz) and / or Ethernet; built-in BLE (for local synchronization / network configuration). Only caches structured features and short-term queues (resumes downloads after network outages), does not write raw video to disk. DC 5V / 12V adapter. Environmental sensor temperature and humidity integrated module (can be integrated into the camera base via I / O). 2 C interface, or as a standalone BLEBeacon. Local gateway (time service and aggregation) form: home router, edge gateway box, or mobile app (any of which serves as the master clock). Functions: publish local timestamp broadcast (BLE), handle latency measurement, unify time base; aggregate collar / camera / environment data and forward it to the cloud; local offline caching. Communication: BLECentral + Wi-Fi / Ethernet uplink. Server / cloud secure access (TLS), device management, feature storage, model and rule engine, federated learning aggregation (only receiving model differences or feature prototypes).

[0070] Linked devices include smart feeders, smart speakers / soothing devices, and lights, which can be used for subsequent intervention strategies (without affecting the core monitoring function of this invention). The gateway acts as the master clock, periodically broadcasting synchronization / follow-up messages via BLE; the collar and camera receive and record local time; the device performs unicast round-trip measurements with the gateway via DELAY_REQ / DELAY_RESP to estimate link latency and clock offset; the device locally corrects its clock using a "slow-tuning" method and writes a unified time base timestamp to each data record. Collar → Gateway: BLEGATT transmits heart rate and skin temperature (with timestamp and quality indicator), with high-frequency reporting in small packets. Environmental Sensor → Camera / Gateway: I 2 C (built-in) or BLE (independent sensor). Camera (edge): The camera performs face masking, foreground separation, skeleton key point extraction, and foreground area ratio calculation locally; it only uploads structured feature sequences (timestamp, key point coordinates / confidence, foreground area ratio, processing quality indicators, etc.). Gateway → Server: Secure uplink via HTTPS / MQTT over TLS 1.2+; local FIFO buffering and sequential retransmission in case of network outage to prevent out-of-order transmission. Control / Configuration Link: The cloud sends device configuration (sampling frequency, window length, threshold coefficient, privacy policy) to the gateway, and then distributes it to the terminal via BLE / Wi-Fi. Supports OTA firmware upgrades; upgrade package signature verification and fragmented breakpoint resume. Cloud-triggered intervention command → gateway → linked device execution. Device factory certificate or key pair, link-level encryption and message integrity verification (HMAC); the camera does not output raw audio and video to the outside, and the frame buffer is released immediately after processing in the edge-side memory; structured features and physiological / environmental data adopt the principle of minimizing fields, and are accompanied by synchronization quality and processing quality indicators for upstream risk control. The collar, camera, and environmental sensors are all centered around the gateway; the camera uses Wi-Fi / Ethernet, while the collar and independent temperature and humidity sensors use BLE; the gateway handles all uplink connections. When the home router is unavailable, the mobile app acts as a temporary gateway and master clock; it automatically switches back to the home gateway upon recovery. The camera's field of view covers the main activity areas (sofa / dining area / doorway), at a height of 0.8–1.2m; the environmental sensors avoid air conditioning vents; the collar fits snugly against the skin without being constricting.

[0071] Key interfaces and timing ensure that all reports carry: unified time base timestamp, device / sensor ID, synchronization quality, sampling period / frame rate, and data sequence number; the gateway performs arrival order rearrangement and lightweight consistency checks (time jump, duplicate packet loss marking) on ​​cross-source data to ensure that S3 and subsequent modules can directly consume it. The above hardware and connection methods guarantee: (1) privacy and bandwidth friendliness (only structured features are output on the end side); (2) cross-modal time consistency (local synchronization with unified time base); (3) robust transmission and easy deployment (star topology, network interruption resume transmission, mobile gateway backup), which can be quickly implemented in home environments.

[0072] Data acquisition module: Pet data is collected by wearable devices and fixed cameras. The pet data includes heart rate data, pet body temperature, ambient temperature and humidity, and video data. Pet data timestamps are unified to the same time base by broadcasting short-range wireless timestamps and correcting them with local time service. The edge processing module performs privacy preprocessing on the video data on the edge, including face detection and masking, foreground and background separation, pet skeleton key point extraction, forming a structured feature sequence containing key point coordinates and foreground area ratio and uploading it to the server. The original video frames are not uploaded and are deleted on the edge. First feature sequence generation module: Within a fixed-length sliding window, based on the heart rate change exceeding a preset adaptive threshold, the heart rate data, pet body temperature, and ambient temperature and humidity are divided into multiple segments. The normalized cross-correlation between the velocity sequence of the skeleton key points and the heart rate sequence is calculated. The time lag with the largest correlation coefficient is used to shift the pet data features as a whole to achieve phase alignment and generate a cross-modal first feature sequence. Second feature sequence generation module: Perform frequency domain analysis on the temporal trajectory of the tail key points in the first feature sequence to obtain the tail swing height, and calculate the ear pitch angle with the head-neck line as a reference. Connect the tail swing height and ear pitch angle with the first feature sequence to form the second feature sequence. Processing module: Inputs the second feature sequence into the trained cross-modal attention temporal network model and outputs continuous emotion coordinates and discrete emotion categories; Deviation Calculation Module: Establishes an individualized baseline based on the target pet's diurnal rhythm, and calculates the deviation between the continuous emotion coordinates and discrete emotion categories output in step S5 and the baseline; Tiered alarm module: When the deviation exceeds the set threshold and meets the preset duration, a tiered alarm is output.

[0073] Preferably, the hidden layer of the trained cross-modal attention temporal network model uses an improved activation function, the expression of which is:

[0074]

[0075] in, For pet activity factors, This represents the Sigmoid nonlinear operation. (t) represents the pitch angle of the auricle at time t. When the pet's tail swings high at time t, This serves as the input to the hidden layer of a cross-modal attention temporal network model. , These are the first learnable parameter and the second learnable parameter, respectively.

[0076] In some embodiments, a "gyroscope node" refers to an integrated miniature IMU node containing a three-axis gyroscope and a three-axis accelerometer. The module consists of one aggregation host (installed on the collar) and 10 distributed gyroscope nodes, for a total of 11 gyroscope units. Time-slotted sampling and reporting are performed via a 2.4 GHz low-power ad hoc network (master-slave TDMA). The host then interfaces with the local gateway via BLE 5.0, and the time base is unified by the "local time service" in S1. The goal is to output kinematic features such as segment attitude / joint angles and angular velocities as input enhancements for "skeleton velocity / phase alignment" in S3 and S4–S5.

[0077] Hardware configuration and wearing arrangement: The main unit (neckband) includes an MCU / NPU, BLE 5.0, 2.4 GHz radio frequency, PPG heart rate and skin temperature sensors (which can be integrated with the original wearable device). It is responsible for sending synchronization time slots, collecting data from each node, encrypting and packaging the data, and local buffering. Ten gyroscope nodes (each <5 g, IP67): head (above the skull or forehead), cervical spine (C7), thoracic back (anterior scapular border / withers), sacrum (above the pelvis), tail root (L7–S1), mid-tail (for tail swing spectrum and phase), upper left forelimb (proximal humerus), upper right forelimb, upper left hindlimb (proximal femur), and upper right hindlimb. Each node is secured with a medical adhesive / elastic strap and has built-in orientation markers to reduce assembly errors.

[0078] Sampling and Communication Sampling Frequency: 200 Hz for both gyroscope and accelerometer (node ​​end), aggregated by the host and transmitted to the upper layer at 100 Hz; adaptive frequency reduction is used to save energy when necessary. Air Interface Protocol: Master-Slave TDMA, the host allocates fixed time slots, and nodes only transmit in their allocated time slots to avoid conflicts from 10 concurrent nodes; automatic retransmission is performed for abnormal packet loss. Time Synchronization: The host broadcasts a "micro-synchronization frame" at the beginning of each frame, and nodes use hardware counters for phase locking; the host simultaneously accesses the S1's local time service, aligns the module clocks to a unified time base, and writes a timestamp and a "synchronization quality" flag.

[0079] In-situ calibration and assembly registration: Static calibration: After initial wear, stand still for ≥5 seconds to correct the zero offset and installation tilt angle of each node using gravity direction. Dynamic registration: Guided to complete 3 simple movements: left and right head swaying, raising forelimbs, and natural tail wagging. The system performs "skeleton joint constraint" fitting on the correlation between angular velocity and acceleration of each node, automatically establishing the alignment matrix and segment length estimation of "node → body segment". Self-check and alarm: If a node is found to be saturated / unresponsive / installed backwards, the host provides a local LED / vibration prompt and prompts for re-wearing in the application.

[0080] Attitude and joint estimation, segment attitude calculation: First-order denoising is performed at the node end, and lightweight inertial attitude calculation (such as complementary filtering / attitude filtering) is used on the host side to fuse angular velocity and gravity direction, eliminating the need for geomagnetism and avoiding indoor magnetic interference. Joint angles and angular velocities: Neck, shoulder / hip, and tail hinge angles are obtained by "adjacent segment difference", and joint angles, angular velocities, and coherence indices are output; gait period and foot landing events are obtained through angular velocity zero-crossing and energy peak detection. Quality assessment: "Node quality" and "Joint calculation quality" are attached to each time step for weight reduction and missing value handling in S3–S5.

[0081] Feature extraction and visual verification: Kinematic features such as segmental angular velocity modulus, joint angle range, gait frequency, gait symmetry, tail swing dominant frequency and energy proportion are downsampled at 10–20 Hz and aligned with visual / physiological flow. Visual verification: When the skeletal key points of S2 are available, the system evaluates the similarity between the "tail root / tail tip" direction and the "tail mid-segment IMU" principal axis; consistency improves the reliability of the tail swing spectrum, while inconsistency uses the IMU as the standard and is marked "visual degradation". Alignment strategy: The timestamp of this module is synchronized with S1, and the normalized cross-correlation of S3 only needs to be fine-tuned within a small time delay range, resulting in more stable alignment and lighter computation.

[0082] The interface relationship with the S-link is as follows: S1: All module data uses a unified time base and participates in the "Good / Degraded / Out of Sync" flag. S2: The visual side still only uploads structured features; the module data does not involve images, making it inherently privacy-friendly. S3: "Skeleton velocity (IMU)" is incorporated into the alignment calculation; when heart rate is missing, it can degenerate into "IMU-video" alignment; when video is unavailable, "IMU-heart rate" alignment can still be used to generate the first feature sequence. S4: In addition to visual keypoints, the tail features add "angular velocity spectrum ratio of the mid-tail IMU" as a redundant channel to improve usability in nighttime / occluded scenes. S5: The model input adds "joint angle and angular velocity, cadence, and symmetry" features; cross-modal attention can learn the phase relationship of "IMU↔ heart rate / body temperature". S6 / S7: In long-term statistics, the step frequency / symmetry of the IMU is used to improve the individualized rhythm; when a high step frequency and high tail swing energy ratio are continuously observed and the value decreases, the reliability of the graded alarm is increased.

[0083] Power consumption and battery life: Typical node current <5 mA (200 Hz sampling, intermittent transmission); coin cell battery can operate for 8–10 hours; main unit 100–150 mA·h battery can meet 1–2 days of moderate use. Supports "quiet self-sleep / wake-up" and nighttime frequency reduction strategy.

[0084] If any node goes offline due to anomalies or degradation, the missing node will be filled using skeletal constraints and marked as missing; if consecutive disconnections exceed a threshold, a "node maintenance" prompt will be triggered. Comfort and safety: All contact parts use soft bases and breathable straps, with chamfered edges, and the tail node uses a flexible tail clip to ensure that natural movement is not affected.

[0085] This module provides high temporal resolution kinematic facts for scenes with poor vision, low light, occlusion, or high-speed motion without altering them. After alignment and feature fusion, it can significantly improve the stability and interpretability of valence / arousal determination and reduce the risk of "excitement / stress confusion".

[0086] The above provides a detailed description of a method and integrated system for collecting pet behavior data and monitoring emotions. Specific examples have been used to illustrate the principles and implementation methods of this invention. The descriptions of the above embodiments are only for the purpose of helping to understand the core ideas of this invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas and methods of this invention. Therefore, the content of this specification should not be construed as a limitation of this invention.

Claims

1. A method for collecting pet behavior data and monitoring emotions, characterized in that, Including the following steps: S1: Pet data is collected by wearable devices and fixed cameras. The pet data includes heart rate data, pet body temperature, ambient temperature and humidity, and video data. The pet data timestamps are unified to the same time base by broadcasting short-range wireless timestamps and correcting them with local time service. S2: Perform privacy preprocessing on the video data on the device side, including face detection and masking, foreground and background separation, pet skeleton key point extraction, forming a structured feature sequence containing key point coordinates and foreground area ratio and uploading it to the server, while the original video frames are not uploaded and are deleted on the device side. S3: Within a fixed-length sliding window, based on the heart rate change exceeding a preset adaptive threshold, the heart rate data, pet body temperature, and ambient temperature and humidity are divided into multiple segments. The normalized cross-correlation between the velocity sequence of the skeleton key points and the heart rate sequence is calculated. The time lag with the largest correlation coefficient is used to shift the pet data features as a whole to achieve phase alignment and generate the first feature sequence across modalities. S4: Perform frequency domain analysis on the time-series trajectory of the key points of the tail in the first feature sequence to obtain the tail swing height, and calculate the ear pitch angle with the head-neck line as a reference. Connect the tail swing height and ear pitch angle with the first feature sequence to form the second feature sequence. S5: Input the second feature sequence into the trained cross-modal attention temporal network model and output continuous emotion coordinates and discrete emotion categories; S6: Establish an individualized baseline based on the target pet's diurnal rhythm, and calculate the deviation of the continuous emotion coordinates and discrete emotion categories output in step S5 from the baseline; S7: When the deviation exceeds the set threshold and meets the preset duration, output a graded alarm.

2. The method for collecting pet behavior data and monitoring emotions as described in claim 1, characterized in that, The process of unifying pet data timestamps to the same time base via short-range wireless timestamp broadcasting and local time service correction includes: S11: The gateway device that provides local time service acts as the master clock and sends a synchronization message containing the master clock timestamp and sequence number via short-range wireless broadcast at fixed intervals, and sends a subsequent message carrying the precise time of transmission of the synchronization message. S12: When the wearable device and the fixed camera receive the synchronization / follow-up message, they respectively record the local reception time and calculate the initial time difference with the master clock based on the message; S13: The wearable device and the fixed camera respectively send a delay request to the gateway via unicast and receive a delay response, based on four time points. , , , Calculate the time difference between the link delay and the master clock; S14: Using the time difference as the clock offset and correcting the local clock according to the preset linear drift rate, offset and drift compensation are performed on the original timestamps of the data collected in each mode, thereby unifying the timestamps to the same time base based on the gateway.

3. The method for collecting pet behavior data and monitoring emotions as described in claim 1, characterized in that, Step S2 includes: A first convolutional neural network is used for face detection, and irreversible masking is performed on the pixels within the detection box to remove the identifiers. Background subtraction is used to separate the foreground and background, resulting in a binary foreground mask. Morphological opening and closing operations and median filtering are then used to denoise the pet target. A pet pose estimation network is used in the foreground region to calculate and output the pixel coordinates and confidence scores of at least 16 skeleton keypoints. The ratio of the number of pixels in the foreground mask to the total number of pixels in the frame is used to calculate the foreground area ratio. The unified time base timestamp, key point coordinates, and foreground area ratio are then combined in chronological order to form a structured feature sequence. The terminal only uploads this structured feature sequence to the server, and the original video frames are not uploaded and are deleted on the terminal.

4. The method for collecting pet behavior data and monitoring emotions as described in claim 1, characterized in that, The process of dividing heart rate data, pet body temperature, and ambient temperature and humidity into multiple segments includes: based on window length... and step length Sliding statistics are performed on the heart rate value sequence to calculate the mean and standard deviation for each window. An adaptive threshold is set between the window mean and the standard deviation. The sum of the standard deviations of the windows; when the heart rate change exceeds the adaptive threshold within any window, the window boundary is marked, and the heart rate value, body temperature and ambient temperature and humidity are divided into multiple segments according to the adjacent boundaries.

5. The method for collecting pet behavior data and monitoring emotions as described in claim 1, characterized in that, The generation of the cross-modal first feature sequence includes: The pixel coordinate sequence of key points on the pet skeleton is first-order differencing to obtain the velocity sequence of each key point. The velocity magnitudes are then weighted and summed according to the confidence level of each key point to form the skeleton key point velocity sequence. Bandpass filtering is applied to eliminate noise. Within a given time delay search range, the normalized cross-correlation function between the skeleton key point velocity sequence and the heart rate sequence is calculated, and the time delay with the highest correlation coefficient is obtained. The key point coordinates, skeleton key point velocity sequence and foreground area ratio time series are translated along the time axis and resampled to a unified time base, thereby completing the phase alignment with the heart rate sequence. The aligned pet data features are then concatenated in time order to form the cross-modal first feature sequence.

6. The method for collecting pet behavior data and monitoring emotions as described in claim 1, characterized in that, The formation of the second feature sequence includes: The coordinates of the tail root key point and the tail tip key point are extracted from the first feature sequence to construct the tail lateral displacement signal. ,right Segmented windowing and Welch power spectrum estimation were performed to calculate the energy in the high-frequency band of 3-8Hz. With low frequency band 0.3-3Hz energy The tail swings high ; Using the head-neck line vector For reference, take the auricle vector. Calculate the pitch angle of the auricle And its mean is calculated over the same time window as the power spectrum; The results obtained in each time window , Align the corresponding time windows in the first feature sequence with a unified time base and concatenate them according to dimensions, so that the feature vector corresponding to each time window contains at least a timestamp and the first feature vector. , The window-level feature vectors arranged in chronological order constitute the second feature sequence.

7. The method for collecting pet behavior data and monitoring emotions as described in claim 1, characterized in that, The cross-modal attention temporal network model includes: Input adaptation layer: The second feature sequence is divided into visual subsequence and environmental subsequence according to its source, and resampled according to a uniform sampling period. Missing positions are marked with a mask. The visual subsequence includes the trajectory coordinates of skeleton key points, foreground area ratio, high and low frequency energy ratio of tail swing, and ear pitch angle. The environmental subsequence includes heart rate value, body temperature, and ambient temperature and humidity. Visual branch temporal encoder: performs temporal encoding on visual subsequences and explicitly outputs the visual features at each time step, including the mean and variance of the skeleton keypoint coordinates in the current window; Environmental branch temporal encoder: performs temporal encoding on the environmental subsequence and outputs the environmental features at each time step, including the mean and variance of pet body temperature and environmental temperature and humidity; Cross-modal multi-head attention fusion layer: Using the visual branch output as the query and the environmental branch output as the key and value, cross-modal attention is calculated, and the output fusion temporal features include the relative time delay that maximizes the attention weight, so as to explicitly characterize the temporal dependency between visual and environmental signals; Temporal aggregation layer: Attention pooling is performed on the fused temporal features to obtain fragment-level sentiment embedding vectors; Dual output heads: one is a regression head that outputs two-dimensional continuous emotion coordinates; the other is a classification head that outputs the probability distribution of discrete emotion categories, where the maximum probability is used as the confidence level of the category. Discrete emotion categories include relaxation, excitement, anxiety, and stress.

8. The method for collecting pet behavior data and monitoring emotions as described in claim 1, characterized in that, Calculating the deviation of continuous emotion coordinates, discrete emotion categories, and baselines includes: (i) Divide historical data into time periods, such as a day or a week, and establish the continuous emotional coordinate value, arousal baseline mean and standard deviation, and discrete emotional category baseline probability distribution for the target pet in each time period. (ii) Standardize the output continuous emotional coordinates and arousal over the corresponding time period to obtain their respective standard residuals; the larger of the two absolute values ​​is recorded as the continuous deviation. (iii) Using the baseline probability distribution from step (i) as a reference, calculate the cross-entropy between the output category probability distribution and the baseline distribution to obtain the category deviation. (iv) The continuous deviation and the category deviation are linearly combined according to preset weights to obtain the final deviation for alarm determination; The continuous emotion coordinates are two-dimensional numerical pairs of valence and arousal, where valence represents the positive or negative nature of the emotion and arousal represents the level of physiological activation. The temporal features after cross-modal attention fusion are output by the regression head, which outputs the values ​​of valence and arousal respectively.

9. The method for collecting pet behavior data and monitoring emotions as described in claim 1, characterized in that, The output of graded alarms includes: when the deviation exceeds the first deviation threshold continuously within the sliding time window and the duration is not less than the first minimum duration, a level 1 alarm is output; when the deviation exceeds the second deviation threshold continuously and the duration is not less than the second minimum duration, a level 2 alarm is output.

10. An integrated system for pet behavior data collection and emotion monitoring, characterized in that, include: Data acquisition module: Pet data is collected by wearable devices and fixed cameras. The pet data includes heart rate data, pet body temperature, ambient temperature and humidity, and video data. Pet data timestamps are unified to the same time base by broadcasting short-range wireless timestamps and correcting them with local time service. The edge processing module performs privacy preprocessing on the video data on the edge, including face detection and masking, foreground and background separation, pet skeleton key point extraction, forming a structured feature sequence containing key point coordinates and foreground area ratio and uploading it to the server. The original video frames are not uploaded and are deleted on the edge. First feature sequence generation module: Within a fixed-length sliding window, based on the heart rate change exceeding a preset adaptive threshold, the heart rate data, pet body temperature, and ambient temperature and humidity are divided into multiple segments. The normalized cross-correlation between the velocity sequence of the skeleton key points and the heart rate sequence is calculated. The time lag with the largest correlation coefficient is used to shift the pet data features as a whole to achieve phase alignment and generate a cross-modal first feature sequence. Second feature sequence generation module: Perform frequency domain analysis on the temporal trajectory of the tail key points in the first feature sequence to obtain the tail swing height, and calculate the ear pitch angle with the head-neck line as a reference. Connect the tail swing height and ear pitch angle with the first feature sequence to form the second feature sequence. Processing module: Inputs the second feature sequence into the trained cross-modal attention temporal network model and outputs continuous emotion coordinates and discrete emotion categories; Deviation Calculation Module: Establishes an individualized baseline based on the target pet's diurnal rhythm, and calculates the deviation between the continuous emotion coordinates and discrete emotion categories output in step S5 and the baseline; Tiered alarm module: When the deviation exceeds the set threshold and meets the preset duration, a tiered alarm is output.

Images