Pet behavior reconstruction interaction method and system

Abstract

The application relates to a pet behavior reconstruction interaction method and system, the method collects nine-axis inertial measurement unit data through a wearable device, performs attitude calculation through a quaternion combined with Kalman filtering, and generates a smooth Euler angle sequence; motion data is divided into time sequence windows, and a few class samples are enhanced through a physical simulation strategy pool; a cascaded one-dimensional convolutional neural network and a double-layer bidirectional long short-term memory network are constructed as a behavior recognition model, local space and global time sequence features are extracted, and training is performed in combination with a dynamic class weight and a learning rate annealing; a digital avatar of the pet is generated in real time on a remote terminal according to a recognition result, and the wearable device is driven to perform physical tactile output in response to a user interaction instruction. The application realizes full-time and full-space pet state monitoring without a visual blind area, establishes a bidirectional tactile interaction loop, and improves the immersion and real-time performance of remote human-pet emotional connection.

Description

Technical Field

[0001] This application relates to the technical field of intelligent pet monitoring, and in particular to an interactive method and system for reconstructing pet behavior. Background Technology

[0002] In the field of remote pet monitoring and human-pet emotional interaction, with the acceleration of urbanization and the expansion of the population of young people living alone and frequent business travelers, pet owners' demand for remotely sensing their pets' status and engaging in emotional interaction is growing. Currently, the pet monitoring technology solutions commonly used in the industry mainly rely on visual monitoring devices, such as smart cameras and mobile monitoring robots. These devices allow users to view their pets' live footage on a remote terminal through real-time video streaming.

[0003] However, existing visual monitoring solutions suffer from several long-standing technical problems in practical applications: Regarding blind spots, pets have a natural tendency to hide and are frequently active at night. Traditional cameras have inherent limitations in their field of view, making it difficult to cover corners such as under beds and on top of cabinets, and unable to track pets that have disappeared into confined spaces. Empty monitoring screens can actually exacerbate separation anxiety for users. In terms of privacy and security, users need to install cameras in private spaces such as bedrooms and living rooms. Uploading home image and audio data to the cloud poses a serious risk of leakage, leaving users in a constant psychological struggle between checking on their pets and protecting their privacy. Regarding interaction, existing products only offer one-way video transmission or voice commands, limiting interaction to the visual and auditory levels and lacking tactile feedback channels, thus failing to meet users' deeper psychological need for a sense of presence.

[0004] In existing technologies, for example, CN121415134A discloses a method for pet behavior recognition and emotion detection based on the fusion of an improved YOLOv11 model and a large language model. This method includes constructing a two-level labeling system, using the improved YOLOv11 for behavior detection and emotion inference, and combining it with a large language model to generate professional responses. However, this solution has the following drawbacks: it relies on visual image data, which creates blind spots when pets crawl under the bed or in low-light conditions; it requires collecting family images and uploading them to the cloud, posing a privacy risk; and it only provides one-way status feedback, lacking two-way tactile interaction functionality, thus failing to meet users' deeper needs for remote emotional connection. For example, CN121438404A discloses a pet behavior recognition and image translation method based on image features, which mainly includes: preprocessing image frames extracted from the input video; extracting key point features, appearance features, and motion features through a multi-branch network and fusing them to obtain fused temporal features; comparing the features at each time step with the behavior codebook to form a code string; outputting behavior labels and confidence scores through temporal decoding, and generating a mask based on key points; inputting the original image, conditional vector, and mask into a condition generator to output a behavior interpretation image. However, this scheme has the following drawbacks: it relies on visual image data, resulting in blind spots when pets crawl under the bed or in dark environments at night; it requires collecting image and video data from inside the home, posing a privacy risk; it only achieves one-way state recognition and image interpretation output from pet to user, lacking a two-way interaction channel from user to pet, especially lacking a tactile feedback mechanism, preventing users from conveying comfort and care to their pets through the system; and it has limited processing capabilities for occluded scenes, with recognition accuracy significantly decreasing when key points of the pet's body are occluded. Summary of the Invention

[0005] The purpose of this invention is to provide a pet behavior reconstruction interaction method and system, which partially solves or alleviates the above-mentioned shortcomings in the prior art, enables all-time and space pet status monitoring without visual blind spots, establishes a two-way tactile interaction loop, enhances the immersiveness, real-time performance and privacy security of remote human-pet emotional connection, and improves the accuracy of identifying rare behaviors.

[0006] To solve the aforementioned technical problems, the present invention specifically adopts the following technical solution:

[0007] A first aspect of the present invention is to provide a pet behavior reconstruction interaction method, comprising the following steps:

[0008] Pet movement data is collected by wearable devices worn on the pet. The movement data includes three-axis acceleration data, three-axis angular velocity data and three-axis geomagnetic data, which are arranged in chronological order to form an original movement time series.

[0009] The collected motion data is used to calculate the attitude and generate a smooth Euler angle sequence, including:

[0010] Using quaternions as the system state vector, Kalman filter state equations and measurement equations are constructed; wherein, the state equations are based on the triaxial angular velocity data to construct a state transition matrix, the measurement equations are based on the triaxial acceleration data to construct an acceleration measurement matrix, and the measurement equations are based on the triaxial geomagnetic data to construct a geomagnetic measurement matrix.

[0011] The current system state vector is predicted using the state transition matrix to obtain the predicted quaternion.

[0012] The predicted quaternion is fused with the acceleration measurement matrix and the acceleration measurement value, and the predicted quaternion is fused with the geomagnetic measurement value using the geomagnetic measurement matrix, so as to optimize the prediction result and obtain the corrected quaternion.

[0013] The corrected quaternion is converted into a smooth Euler angle sequence;

[0014] The original motion time series is divided into time-series windows to generate multiple window data with fixed time lengths;

[0015] Behavior recognition based on the window data includes:

[0016] The window data is input into a pre-trained behavior recognition model, and the spatiotemporal features of the window data are extracted by the spatiotemporal feature extraction network in the behavior recognition model. The spatiotemporal feature extraction network includes a cascaded one-dimensional convolutional neural network and a bidirectional long short-term memory network. The one-dimensional convolutional neural network is used to extract the local spatial collaborative features of each channel in the window data within the temporal neighborhood and output a local feature sequence. The bidirectional long short-term memory network is used to receive the local feature sequence and extract spatiotemporal features with global semantics along the temporal direction.

[0017] The spatiotemporal features are input into the classification network of the behavior recognition model, and the corresponding behavior semantic labels are output.

[0018] Based on the smooth Euler angle sequence and behavioral semantic tags, a digital clone of the pet is generated in real time on a remote terminal; and / or, user feedback signals are sent to the remote terminal based on the behavioral semantic tags so that the user can perceive the pet's real-time dynamics.

[0019] In response to an interactive command initiated by a user through a remote terminal, a tactile feedback signal is sent to the wearable device, and the wearable device is driven to perform physical tactile output for interaction.

[0020] Furthermore, the step of dividing the original motion time series into time-series windows to generate multiple window data of fixed time lengths includes:

[0021] First, all types of data in the original motion time series are uniformly resampled to a preset reference frequency so that all types of data streams have a uniform time step on the time axis.

[0022] Set a window of fixed time length, and the window length corresponds to a preset number of time steps, so that each window covers the same length of motion data;

[0023] The original motion time series is divided along the time axis by a preset sliding step size, where the sliding step size is less than the window length.

[0024] Furthermore, the one-dimensional convolutional neural network is used to extract the local spatial collaborative features of each channel in the temporal neighborhood of the window data, and outputs a local feature sequence including:

[0025] The one-dimensional convolutional neural network includes a cascaded first convolutional block, a second convolutional block, and a third convolutional block;

[0026] The first convolutional block performs a one-dimensional convolution operation on the window data to extract the local spatial collaborative features of each channel in the temporal neighborhood. After batch normalization and ReLU activation, the temporal dimension is reduced by max pooling, and the first intermediate feature is output.

[0027] The second convolutional block performs a one-dimensional convolution operation on the first intermediate feature to further extract local spatial collaborative features. After batch normalization and ReLU activation, it achieves temporal dimension reduction again through max pooling and outputs the second intermediate feature.

[0028] The third convolutional block performs a one-dimensional convolution operation on the second intermediate feature to extract deep local spatial collaborative features. After batch normalization and ReLU activation, the time dimension remains unchanged, and the local feature sequence is output.

[0029] Furthermore, the bidirectional long short-term memory network is used to receive the local feature sequence and extract spatiotemporal features with global semantics along the temporal direction, including:

[0030] The bidirectional long short-term memory network is a two-layer bidirectional structure, which includes a forward long short-term memory layer and a reverse long short-term memory layer.

[0031] The local feature sequence is input into the forward long short-term memory layer along the time axis to extract forward temporal features;

[0032] The local feature sequence is input into the reverse long short-term memory layer in reverse along the time axis to extract reverse temporal features;

[0033] The forward time series feature and the reverse time series feature are concatenated at the same time step to obtain the bidirectional time series feature at the corresponding time step;

[0034] The bidirectional temporal features of the last time step are extracted as spatiotemporal features.

[0035] Furthermore, the classification network includes a first dropout layer, a first fully connected layer, a ReLU activation layer, a second dropout layer, and a second fully connected layer connected in sequence;

[0036] Furthermore, the process of inputting the spatiotemporal features into the classification network of the behavior recognition model and outputting corresponding behavior semantic labels includes:

[0037] The spatiotemporal features are input into the first dropout layer, and some neurons are randomly dropped at a preset first dropout rate to obtain the first regularized features.

[0038] The first regularized feature is input into the first fully connected layer to map the dimensions of the spatiotemporal features from the first dimension to the second dimension, thereby obtaining the first fully connected feature;

[0039] The first fully connected feature is nonlinearly transformed through the ReLU activation layer to obtain the activation feature;

[0040] The activation features are input into the second dropout layer, and some neurons are randomly dropped at a preset second dropout rate to obtain the second regularization features.

[0041] The second regularized feature is input into the second fully connected layer to map the feature dimension from the second dimension to the number of categories dimension, thus obtaining the Logits vector;

[0042] The Logits vector is activated by the Softmax function, the predicted probability of each behavior category is calculated, and the category with the highest probability is selected as the behavior semantic label for output.

[0043] Furthermore, the step of generating a digital clone of the pet in real time on a remote terminal based on the smoothed Euler angle sequence and behavioral semantic tags includes:

[0044] Obtain a preset motion mapping library, which contains the mapping relationship between behavior semantic tags and skeletal animation instructions;

[0045] The corresponding skeletal animation instructions are obtained by querying the action mapping library based on the behavioral semantic tags.

[0046] The rotation angles of each joint in the pet skeletal model are calculated in real time based on the smoothed Euler angle sequence to generate posture driving parameters;

[0047] The skeletal animation instructions are fused with the posture driving parameters to drive the skeletal animation engine of the 3D pet model, so that the overall movement of the 3D pet model matches the behavioral semantic label, and the local posture is synchronized with the smooth Euler angle sequence in real time.

[0048] The animation of the 3D pet model is rendered and output in real time through the display interface of the remote terminal, forming a digital clone of the pet.

[0049] Furthermore, the step of sending user feedback signals to a remote terminal based on the behavioral semantic tags, so that the user can perceive the pet's real-time dynamics, includes:

[0050] The behavioral semantic tags are acquired in real time, and the corresponding behavioral category and confidence level are determined based on the behavioral semantic tags.

[0051] The behavior category is matched with a preset feedback mapping table to determine the feedback type and feedback intensity corresponding to the behavior category;

[0052] A user feedback signal is generated based on the feedback type and feedback intensity, and the user feedback signal is sent to a remote terminal;

[0053] After receiving the user feedback signal, the remote terminal performs corresponding prompt output according to the feedback type and feedback intensity.

[0054] The feedback type includes at least one of vibration prompts, sound prompts, and light effect prompts; the feedback intensity is positively correlated with the confidence level of the behavior semantic tag, or matches the preset importance level corresponding to the behavior category.

[0055] Furthermore, the behavior recognition model is trained using a dynamic category weight penalty strategy, including:

[0056] Count the number of samples in each category in the training set to determine the sample number distribution of each category;

[0057] For each category, a category weight is calculated, which is inversely proportional to the number of samples in that category.

[0058] A weighted cross-entropy loss function is constructed as the loss function of the behavior recognition model. The weighted cross-entropy loss function is based on the standard cross-entropy loss function, where the loss term for each category is multiplied by the corresponding category weight.

[0059] Furthermore, before training the behavior recognition model, data augmentation steps are performed:

[0060] Count the number of samples of each category in the training set to determine the target equilibrium quantity;

[0061] For a minority of categories where the number of samples is less than the target equilibrium number, at least two strategies are randomly selected from a preset physical simulation strategy pool and nested in series to generate supplementary samples.

[0062] The physics simulation strategy pool includes at least:

[0063] A scaling factor with a random value within a preset range is introduced. The scaling factor is used to change the time axis density of the original time series. The new time axis index vector after mapping is calculated, and the augmented time series is generated by linear interpolation or cubic spline interpolation.

[0064] For each sensor measurement vector in the original motion time series, generate independent random small rotation angles around the three-dimensional spatial coordinate axes. Construct a spatial rotation perturbation matrix based on the three rotation angles. Multiply the current sensor measurement vector by this spatial rotation perturbation matrix to generate the augmented sensor vector.

[0065] For the triaxial angular velocity data, combined with the white noise level measured by the static calibration characteristics of the wearable device, Gaussian white noise following a normal distribution is injected into each data axis. The amplitude of the injected noise is adjusted by a preset noise intensity control factor to generate augmented triaxial angular velocity data.

[0066] Deep feature vectors are extracted from a few categories. Multiple nearest neighbor samples of the deep feature vectors in the feature space are calculated. One of the nearest neighbor samples is randomly selected, and an interpolation weight with a random value between zero and one is introduced. The difference vector between the deep feature vector and the nearest neighbor sample is multiplied by the interpolation weight and then superimposed on the deep feature vector to synthesize a new feature vector sample.

[0067] Secondly, this application also discloses a pet behavior reconstruction interactive system, the system comprising:

[0068] The motion data acquisition module is configured to collect pet motion data through a wearable device worn on the pet. The motion data includes three-axis acceleration data, three-axis angular velocity data, and three-axis geomagnetic data, which are arranged in chronological order to form an original motion time series.

[0069] The attitude calculation module is configured to perform attitude calculation on the acquired motion data and generate a smooth Euler angle sequence, including:

[0070] Using quaternions as the system state vector, Kalman filter state equations and measurement equations are constructed; wherein, the state equations are based on the triaxial angular velocity data to construct a state transition matrix, the measurement equations are based on the triaxial acceleration data to construct an acceleration measurement matrix, and the measurement equations are based on the triaxial geomagnetic data to construct a geomagnetic measurement matrix.

[0071] The current system state vector is predicted using the state transition matrix to obtain the predicted quaternion.

[0072] The predicted quaternion is fused with the acceleration measurement matrix and the acceleration measurement value, and the predicted quaternion is fused with the geomagnetic measurement value using the geomagnetic measurement matrix, so as to optimize the prediction result and obtain the corrected quaternion.

[0073] The corrected quaternion is converted into a smooth Euler angle sequence;

[0074] The time-series window segmentation module is configured to perform time-series window segmentation on the original motion time series to generate multiple window data with fixed time lengths;

[0075] The behavior recognition module, configured to perform behavior recognition based on the window data, includes:

[0076] The window data is input into a pre-trained behavior recognition model, and the spatiotemporal features of the window data are extracted by the spatiotemporal feature extraction network in the behavior recognition model. The spatiotemporal feature extraction network includes a cascaded one-dimensional convolutional neural network and a bidirectional long short-term memory network. The one-dimensional convolutional neural network is used to extract the local spatial collaborative features of each channel in the window data within the temporal neighborhood and output a local feature sequence. The bidirectional long short-term memory network is used to receive the local feature sequence and extract spatiotemporal features with global semantics along the temporal direction.

[0077] The spatiotemporal features are input into the classification network of the behavior recognition model, and the corresponding behavior semantic labels are output.

[0078] The status awareness module is configured to render and generate a digital clone of the pet in real time on a remote terminal based on the smooth Euler angle sequence and behavioral semantic tags; and / or to send user feedback signals to the remote terminal based on the behavioral semantic tags so that the user can perceive the pet's real-time dynamics.

[0079] The physical tactile output module is configured to respond to an interactive command initiated by a user through a remote terminal, send a tactile feedback signal to the wearable device, and drive the wearable device to perform physical tactile output for interaction.

[0080] Beneficial technical effects:

[0081] 1. This invention utilizes a nine-axis inertial measurement unit to collect acceleration, angular velocity, and geomagnetic data of a pet. Combined with a quaternion and Kalman filtering attitude calculation method, it achieves high-precision and robust estimation of the pet's three-dimensional spatial attitude. Compared to traditional camera-based visual monitoring solutions, this invention completely overcomes the limitations of line-of-sight obstruction and lighting conditions. Whether the pet is in blind spots such as under the bed, on top of a cabinet, or in a sofa crevices, or even in darkness at night, the system can accurately perceive its posture and movement, reconstructing the pet's three-dimensional behavioral model in real time. This fundamentally solves the long-standing problem of blind spots in existing monitoring technologies.

[0082] 2. This invention constructs a cascaded one-dimensional convolutional neural network and a two-layer bidirectional long short-term memory network as the behavior recognition model, forming a deep learning architecture with decoupled spatiotemporal features. The one-dimensional convolutional neural network specifically extracts local spatial collaborative features of multi-channel data in the temporal neighborhood, capturing transient abrupt change patterns such as a sharp increase in angular velocity and tilt angle. The two-layer bidirectional long short-term memory network fuses temporal information in both forward and reverse directions, while integrating historical preparatory actions and future buffers for the current action. This allows it to accurately distinguish between micro-movements with extremely similar local waveforms, such as squatting before jumping and preparing to lie down, completely eliminating semantic ambiguity caused by single-directional temporal information and significantly improving the accuracy and robustness of behavior recognition.

[0083] 3. This invention addresses the long-tail distribution problem prevalent in pet behavior data by designing a two-stage heuristic precision augmentation mechanism and a dynamic category weight penalty strategy. In the data augmentation stage, this invention randomly selects multiple strategies from eight physical simulation strategies—Gaussian noise injection, signal jitter, amplitude scaling, smooth amplitude distortion, time translation, nonlinear time distortion, three-dimensional random rotation, and channel deactivation—and concatenates and nests them to generate supplementary samples with high physical plausibility and diversity, thus balancing the number of samples in each category. In the model training stage, this invention constructs a weighted cross-entropy loss function inversely proportional to the number of samples in each category, assigning higher loss penalty coefficients to rare behaviors such as falling and being startled, forcing the gradient to tilt towards minority class samples during backpropagation. Through the combined effect of these two mechanisms, this invention significantly improves the accuracy of identifying rare behaviors, providing a reliable technical foundation for early warning of abnormal pet health.

[0084] 4. This invention constructs a complete two-way emotional interaction closed loop. In the pet-to-user direction, the invention maps the identified behavioral semantic tags and smooth Euler angle sequences to a three-dimensional digital avatar in real time, and renders the output in real time through the display interface of a remote terminal, allowing the user to intuitively perceive the pet's real-time status and behavioral changes, gaining a strong sense of "presence." In the user-to-pet direction, the invention responds to interactive commands such as petting, hugging, calling, and comforting initiated by the user through a remote terminal. Based on the command type, it determines the corresponding tactile feedback mode, driving a linear resonant actuator, eccentric rotating mass motor, shape memory alloy, or micro-airbag structure in the wearable device to execute physical tactile output, generating diverse tactile stimuli such as vibration, compression, and hugging, enabling the pet to perceive the owner's remote care. This two-way tactile interaction loop fills the long-standing gap in the tactile interaction dimension of existing pet monitoring technologies, significantly enhancing the emotional connection and companionship experience between humans and pets in remote scenarios. Attached Figure Description

[0085] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. In all the drawings, similar elements or parts are generally identified by similar reference numerals. The elements or parts in the drawings are not necessarily drawn to scale. Obviously, the drawings described below are some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings without any creative effort.

[0086] Figure 1 This is a flowchart of an interactive method for reconstructing pet behavior according to this application.

[0087] Figure 2 This is a time-domain schematic diagram of the reconstruction of the tactile waveform of a touch interaction in one embodiment of the present invention.

[0088] Figure 3 This is a schematic diagram of an airbag used to perform physical tactile output in one embodiment of the present invention; wherein, Figure 3 (a) is a pressure-deformation angle response curve of the airbag used in this invention under different wall thickness parameters. Figure 3 (b) is a schematic diagram of the three-dimensional finite element mesh model of the airbag of the present invention.

[0089] Figure 4 This is a schematic diagram of a pet behavior reconstruction interaction system in one embodiment of this application. Detailed Implementation

[0090] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0091] In this document, suffixes such as "module," "part," or "unit" used to denote elements are used only for the purpose of illustrative purposes and have no specific meaning in themselves. Therefore, "module," "part," or "unit" may be used interchangeably.

[0092] In this document, the terms "upper," "lower," "inner," "outer," "front," "rear," "one end," and "the other end," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the present invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the present invention. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0093] In this document, unless otherwise explicitly specified and limited, the terms "installed," "equipped with," "connected," etc., should be interpreted broadly. For example, "connection" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection, a direct connection, or an indirect connection through an intermediate medium; it can be a connection within two components. Those skilled in the art can understand the specific meaning of the above terms in this invention based on the specific circumstances.

[0094] In this document, "and / or" includes any and all combinations of one or more of the listed related items.

[0095] In this article, "multiple" means two or more, that is, it includes two, three, four, five, etc.

[0096] Figure 1 A flowchart of an interactive method for reconstructing pet behavior according to this application is shown. (Refer to...) Figure 1 The method specifically includes the following steps:

[0097] S1. Collect pet movement data through a wearable device worn on the pet. The movement data includes three-axis acceleration data, three-axis angular velocity data and three-axis geomagnetic data, and are arranged in chronological order to form an original movement time sequence.

[0098] In some embodiments, a nine-axis inertial measurement unit (IMU) sensor module built into a wearable smart collar worn around the pet's neck collects real-time motion data of the pet in three-dimensional space at a preset sampling frequency; the motion data includes linear acceleration data collected by a three-axis accelerometer. Angular velocity data acquired by a three-axis gyroscope and geomagnetic direction data collected by a triaxial magnetometer. The discrete sampled values ​​output by the three types of sensors are aligned by timestamps and arranged sequentially in chronological order to form an original motion time sequence of length N.

[0099] S2. Perform attitude calculation on the collected motion data to generate a smooth Euler angle sequence, including:

[0100] Using quaternions as the system state vector, Kalman filter state equations and measurement equations are constructed; wherein, the state equations are based on the triaxial angular velocity data to construct a state transition matrix, the measurement equations are based on the triaxial acceleration data to construct an acceleration measurement matrix, and the measurement equations are based on the triaxial geomagnetic data to construct a geomagnetic measurement matrix.

[0101] The current system state vector is predicted using the state transition matrix to obtain the predicted quaternion.

[0102] The predicted quaternion is fused with the acceleration measurement matrix and the acceleration measurement value, and the predicted quaternion is fused with the geomagnetic measurement value using the geomagnetic measurement matrix, so as to optimize the prediction result and obtain the corrected quaternion.

[0103] The corrected quaternion is converted into a smooth Euler angle sequence;

[0104] In some embodiments, this invention proposes an attitude determination scheme that integrates multi-source sensor information and employs a Kalman filtering mechanism, aiming to calculate a high-precision and smooth Euler angle time series from data acquired by an inertial measurement unit (IMU). The core of this scheme lies in defining the system state vector as a quaternion. This represents the rotational transformation from the navigation coordinate system (n-frame) to the vehicle coordinate system (b-frame). Based on quaternion theory, the corresponding direction cosine matrix (i.e., the rotation matrix) is... This can be specifically expressed as:

[0105] ;

[0106] In each sampling period Inside, the system receives three-axis angular velocity observations from the gyroscope. And it performs state prediction using quaternion differential equations. Specifically, the prior estimate at the current time step... Based on the posterior estimate of the previous time step The derivation leads to:

[0107] ;

[0108] in It is a skew-symmetric matrix constructed from angular velocities. This represents a 4x4 identity matrix. It can be represented as a state transition matrix Its unfolded form is as follows:

[0109] ;

[0110] At the same time, the error covariance matrix is ​​also predicted in one step based on the state transition model:

[0111] ;

[0112] in The aforementioned state transition matrix, The system process noise covariance matrix characterizes the noise characteristics of the gyroscope.

[0113] To address the drift error generated during gyroscope integration, this scheme introduces accelerometers and magnetometers as external observation tools for measurement updates. First, utilizing the physical property of gravity being constant and vertically downward in the navigation system, the pitch and roll angles are corrected. In the NED (Northeast Earth) navigation system, the standard gravity vector is defined as... By using prior quaternions Convert to direction cosine matrix This allows for the prediction of the gravity vector in the carrier coordinate system. :

[0114] ;

[0115] in, The accelerometer measurement function is represented by the actual collected and normalized accelerometer readings. Compared with the predicted value By comparison, the gravity measurement residuals are generated:

[0116] ;

[0117] To effectively apply this residual to state correction, it is necessary to calculate the Jacobian matrix of the observation function with respect to the quaternion state. :

[0118] ;

[0119] Secondly, to correct for the yaw angle, geomagnetic data is introduced to provide an absolute geographic north reference. The measured and normalized geomagnetic vector is then used. Rotate to the navigation system using prior quaternions: By forcibly projecting the geomagnetic vector in the navigation system onto the north-pointing XZ plane (i.e., eliminating the eastward component), an ideal reference magnetic field is obtained:

[0120] ;

[0121] in, The northward component (X-axis) of the reference magnetic field. For the perpendicular component (Z-axis) of the reference magnetic field, m nx m represents the component of the Earth's magnetic field in the direction of geographic north. ny This represents the component of the Earth's magnetic field in the geographic east direction. Then, the ideal reference magnetic field... Rotate back to the carrier coordinate system to obtain the system-predicted geomagnetic vector. :

[0122] ;

[0123] in, Represents geomagnetic measurement functions;

[0124] And calculate the geomagnetic measurement residuals:

[0125] ;

[0126] and the corresponding Jacobian matrix :

[0127] ;

[0128] After obtaining the two sets of independent measurement residuals and their Jacobian matrices, this scheme performs joint data fusion. This is achieved by constructing a... Global measurement matrix With one global residual vector And combined with prior covariance and including the noise variance of accelerometers and magnetometers Diagonal measurement noise covariance matrix Calculate the Kalman gain:

[0129] ;

[0130] By using the calculated Kalman gain and global residual, the prior state and covariance are corrected to obtain the posterior quaternion state at the current time step:

[0131] ;

[0132] After the calculation is completed, it is necessary to... Perform a normalization operation to ensure that it satisfies the unit quaternion constraint. Simultaneously update the error covariance matrix for use in the next cycle.

[0133] ;

[0134] Finally, the converged posterior quaternion The Euler angles are solved intuitively using standard mathematical transformations. These correspond to the roll angle, pitch angle, and yaw angle, respectively. The pitch angle... Roll angle Yaw angle This generates continuous, smooth, and high signal-to-noise ratio Euler angle time series, providing a solid and reliable attitude foundation for subsequent behavior recognition and interaction applications.

[0135] S3. Perform time-series window segmentation on the original motion time series to generate multiple window data with fixed time lengths.

[0136] In some embodiments, the step of dividing the original motion time series into time-series windows to generate multiple window data of fixed time lengths includes:

[0137] First, resample all types of data in the motion data to a preset reference frequency so that all types of data streams have a uniform time step on the time axis.

[0138] Set a window of fixed time length, and the window length corresponds to a preset number of time steps, so that each window covers the same length of motion data;

[0139] The motion data is slidably segmented along the time axis with a preset sliding step size, where the sliding step size is less than the window length.

[0140] In some embodiments, the various types of data in the motion data are first resampled to a preset reference frequency, so that the various types of data streams have a uniform time step on the time axis. Since the raw data collected by the sensor may have inconsistent sampling rates, the resampling process unifies all data streams such as acceleration, angular velocity, geomagnetism, and calculated Euler angles to the same sampling frequency, ensuring that the data of each channel are strictly aligned in the time dimension when the window is subsequently segmented, providing a synchronized input sequence for spatiotemporal feature extraction.

[0141] A fixed-length window is set, corresponding to a preset number of time steps, ensuring each window covers the same duration of motion data. The window length can be set based on the duration of the target behavior; for example, a window length of 0.5 seconds corresponds to 50 time steps at a base frequency of 100Hz. This window length fully covers the complete process of common pet behaviors, including transient actions such as jumping, running, and turning, avoiding incomplete behavioral semantics due to an excessively short window, or introducing too much irrelevant background information due to an excessively long window.

[0142] Motion data is segmented along the time axis using a preset sliding step size, which is less than the window length. The sliding step size is typically set to half the window length; for example, if the window length is 50 time steps, the sliding step size is set to 25 time steps. By setting overlapping sliding windows, there is partial overlap between adjacent windows. This high overlap segmentation method ensures that transient actions are not semantically corrupted by hard cutting, while effectively expanding the number of training samples and improving the model's generalization ability. For a motion data sequence with a total length of M time steps, the time step index range of the i-th data window is from i×S to i×S+L. 1, where L is the window length and S is the sliding step size, thereby generating a series of window data samples with a fixed time span.

[0143] Through the above-mentioned temporal window segmentation process, continuous long-term motion signals are transformed into multiple fixed-length short-term window samples. Each window sample contains complete attitude motion information and can be used as a standardized input for the subsequent spatiotemporal feature decoupling network, thus realizing an effective conversion from continuous signals to discrete samples.

[0144] S4. Performing behavior recognition based on the window data, including:

[0145] The window data is input into a pre-trained behavior recognition model, and the spatiotemporal features of the window data are extracted by the spatiotemporal feature extraction network in the behavior recognition model. The spatiotemporal feature extraction network includes a cascaded one-dimensional convolutional neural network and a bidirectional long short-term memory network. The one-dimensional convolutional neural network is used to extract the local spatial collaborative features of each channel in the window data within the temporal neighborhood and output a local feature sequence. The bidirectional long short-term memory network is used to receive the local feature sequence and extract spatiotemporal features with global semantics along the temporal direction.

[0146] The spatiotemporal features are input into the classification network of the behavior recognition model, and the corresponding behavior semantic labels are output.

[0147] In some embodiments, the one-dimensional convolutional neural network is used to extract local spatial co-features of each channel in the window data within the temporal neighborhood, and outputs a local feature sequence including:

[0148] The one-dimensional convolutional neural network includes a cascaded first convolutional block, a second convolutional block, and a third convolutional block;

[0149] The first convolutional block performs a one-dimensional convolution operation on the window data to extract the local spatial collaborative features of each channel in the temporal neighborhood. After batch normalization and ReLU activation, the temporal dimension is reduced by max pooling, and the first intermediate feature is output.

[0150] The second convolutional block performs a one-dimensional convolution operation on the first intermediate feature to further extract local spatial collaborative features. After batch normalization and ReLU activation, it achieves temporal dimension reduction again through max pooling and outputs the second intermediate feature.

[0151] The third convolutional block performs a one-dimensional convolution operation on the second intermediate feature to extract deep local spatial collaborative features. After batch normalization and ReLU activation, the time dimension remains unchanged, and the local feature sequence is output.

[0152] In some embodiments, the bidirectional long short-term memory network is used to receive the local feature sequence and extract spatiotemporal features with global semantics along the temporal direction, including:

[0153] The bidirectional long short-term memory network is a two-layer bidirectional structure, which includes a forward long short-term memory layer and a reverse long short-term memory layer.

[0154] The local feature sequence is input into the forward long short-term memory layer along the time axis to extract forward temporal features;

[0155] The local feature sequence is input into the reverse long short-term memory layer in reverse along the time axis to extract reverse temporal features;

[0156] The forward time series feature and the reverse time series feature are concatenated at the same time step to obtain the bidirectional time series feature at the corresponding time step;

[0157] The bidirectional temporal features of the last time step are extracted as spatiotemporal features.

[0158] The classification network includes a first dropout layer, a first fully connected layer, a ReLU activation layer, a second dropout layer, and a second fully connected layer connected in sequence.

[0159] Furthermore, the process of inputting the spatiotemporal features into the classification network of the behavior recognition model and outputting corresponding behavior semantic labels includes:

[0160] The spatiotemporal features are input into the first dropout layer, and some neurons are randomly dropped at a preset first dropout rate to obtain the first regularized features.

[0161] The first regularized feature is input into the first fully connected layer to map the dimensions of the spatiotemporal features from the first dimension to the second dimension, thereby obtaining the first fully connected feature;

[0162] The first fully connected feature is nonlinearly transformed through the ReLU activation layer to obtain the activation feature;

[0163] The activation features are input into the second dropout layer, and some neurons are randomly dropped at a preset second dropout rate to obtain the second regularization features.

[0164] The second regularized feature is input into the second fully connected layer to map the feature dimension from the second dimension to the number of categories dimension, thus obtaining the Logits vector;

[0165] The Logits vector is activated by the Softmax function, the predicted probability of each behavior category is calculated, and the category with the highest probability is selected as the behavior semantic label for output.

[0166] In some embodiments, after dividing the motion data into temporal windows to generate multiple window data of fixed time lengths, this step inputs the window data into a pre-trained behavior recognition model, and performs spatiotemporal feature extraction and behavior classification on the window data through the behavior recognition model, outputting corresponding behavior semantic labels.

[0167] The behavior recognition model comprises a cascaded spatiotemporal feature extraction network and a classification network. The spatiotemporal feature extraction network extracts deep spatiotemporal features from the window data, employing a cascaded architecture of a one-dimensional convolutional neural network and a bidirectional long short-term memory network. Specifically, the one-dimensional convolutional neural network extracts local spatial co-features of each channel in the window data within the temporal neighborhood and outputs a local feature sequence; the bidirectional long short-term memory network receives the local feature sequence and extracts spatiotemporal features with global semantics along the temporal direction, effectively eliminating temporal ambiguity in actions.

[0168] In some embodiments, a one-dimensional convolutional neural network includes a cascaded first convolutional block, a second convolutional block, and a third convolutional block. The window data input to the one-dimensional convolutional neural network has a preset dimensionality, such as being organized as a three-dimensional tensor in the channel and time dimensions. The first convolutional block performs a one-dimensional convolution operation on the window data to extract the local spatial cooperative features of each channel in the temporal neighborhood. The convolutional kernel size is set to 3. By sliding the convolutional kernel, the cooperative change pattern of multi-channel data between adjacent time steps is captured. After batch normalization and ReLU activation, the temporal dimension is reduced by max pooling, and the first intermediate feature is output. The pooling layer uses max pooling with a kernel size of 2 and a stride of 2 to achieve the first temporal dimensionality reduction, while providing fault tolerance for small temporal shifts. The second convolutional block performs a one-dimensional convolution operation on the first intermediate feature to further extract higher-dimensional local spatial collaborative features. It also uses a one-dimensional convolution with a kernel size of 3. After batch normalization and ReLU activation, max pooling is used to further reduce the temporal dimension, outputting the second intermediate feature. After two pooling operations, the temporal dimension is compressed to one-quarter of its original length, while the number of feature channels expands layer by layer, achieving a progressive extraction from shallow detailed features to deep abstract features. The third convolutional block performs a one-dimensional convolution operation on the second intermediate feature to extract deeper local spatial collaborative features. The kernel size remains at 3. After batch normalization and ReLU activation, the temporal dimension remains unchanged, outputting a local feature sequence. This convolutional block omits pooling to prevent excessive compression of the temporal dimension and loss of contextual information, preserving sufficient temporal resolution for subsequent recurrent neural networks.

[0169] In some embodiments, the bidirectional long short-term memory network employs a two-layer bidirectional structure, consisting of a first-layer bidirectional LSTM and a second-layer bidirectional LSTM stacked together. Each layer contains a forward LSTM layer and a backward LSTM layer. The first-layer bidirectional LSTM receives the local feature sequence output by a one-dimensional convolutional neural network as input. Its forward LSTM layer extracts historical dependency information of the sequence along the forward time axis, while its backward LSTM layer extracts future dependency information of the sequence along the backward time axis. The forward and backward hidden states at the same time step are concatenated and output to the second-layer bidirectional LSTM. The second-layer bidirectional LSTM takes the concatenated hidden state sequence output by the first layer as input and performs forward and backward temporal feature extraction again. This two-layer stacking achieves a higher level of abstract encoding of temporal data, enabling the model to capture more complex temporal dependencies. This two-layer bidirectional structure achieves a bidirectional disambiguation mechanism by concatenating the positive and negative hidden states at each time step. Since local micro-actions exhibit strong temporal ambiguity—for example, a pre-jump squat and a preparatory prone position are extremely similar in their local waveforms—it's difficult to accurately distinguish them using only positive temporal information. The bidirectional LSTM, however, integrates the historical preparatory phase and future buffer of the current action, fusing past and future contextual information at the same time step, thus completely eliminating semantic ambiguity. Finally, the concatenated hidden state vector from the last time step of the sequence is extracted as a spatiotemporal feature. This feature, which integrates both positive and negative temporal information, has an output dimension of 128 dimensions, serving as a macroscopic global feature encoding covering the entire 0.5-second time window, used as input for subsequent classification networks.

[0170] In some embodiments, the classification network includes a first dropout layer, a first fully connected layer, a ReLU activation layer, a second dropout layer, and a second fully connected layer connected in sequence. Spatiotemporal features are input into the first dropout layer, and some neurons are randomly dropped at a preset first dropout rate to obtain first regularized features; this dropout operation effectively suppresses model overfitting and enhances the model's generalization ability. The first regularized features are input into the first fully connected layer to map the dimension of the spatiotemporal features from a first dimension to a second dimension, resulting in first fully connected features; this fully connected layer performs nonlinear combination of global features to further refine discriminative features highly correlated with behavioral categories. The first fully connected features are nonlinearly transformed by the ReLU activation layer to obtain activation features; the ReLU activation function introduces nonlinear expressive power and also has sparse activation characteristics, which helps improve the computational efficiency of the model. The activation features are input into the second dropout layer, and some neurons are randomly dropped at a preset second dropout rate to obtain second regularized features; this dropout operation further strengthens the regularization effect. The second regularized features are input into the second fully connected layer to map the feature dimension from a second dimension to the category number dimension, resulting in a Logits vector. The Logits vector is activated by the Softmax function, the predicted probability of each behavior category is calculated, and the category with the highest probability is selected as the behavior semantic label for output, thereby realizing the automatic recognition and classification of pet behavior.

[0171] In some embodiments, training the behavior recognition model using a dynamic category weight penalty strategy includes:

[0172] Count the number of samples in each category in the training set to determine the sample number distribution of each category;

[0173] For each category, a category weight is calculated, which is inversely proportional to the number of samples in that category.

[0174] A weighted cross-entropy loss function is constructed as the loss function of the behavior recognition model. The weighted cross-entropy loss function is based on the standard cross-entropy loss function, where the loss term for each category is multiplied by the corresponding category weight.

[0175] In some embodiments, during the training of the behavior recognition model, in order to address the common class imbalance problem in pet behavior data, namely the situation where the number of background class samples such as "still" is much greater than the number of minority class samples such as "rolling" and "frightened", this invention adopts a dynamic class weight penalty strategy to optimize the model training process.

[0176] Specifically, the first step is to count the number of samples in each category in the training set to determine the sample distribution for each category. Although the training set has been segmented using the aforementioned temporal window and heuristic augmentation, which has alleviated class imbalance to some extent, significant differences in the frequency of different behaviors in natural states may still exist in the number of samples in each category. For example, the number of samples in the stationary category is much larger than that in the running category, and the number of samples in the running category is much larger than that in the fallen category. Accurately counting the sample distribution for each category is the foundation for subsequent weight calculations.

[0177] For each category, calculate the category weight, which is inversely proportional to the number of samples in that category. Specifically, suppose there are C behavior categories in the training set, and the number of samples in the i-th category is Ni. Then the weight wi of that category can be expressed as: Or adopt a normalized form This is to control the weight scale. By setting the weights inversely proportional to the number of samples, the class with fewer samples receives a higher weight coefficient. This causes the model to impose a higher penalty on incorrect predictions of minority class samples during training, thereby guiding the model to pay more attention to rare behavioral categories that are difficult to learn fully.

[0178] A weighted cross-entropy loss function is constructed as the loss function for the behavior recognition model. The weighted cross-entropy loss function multiplies the loss term for each class by the corresponding class weight, based on the standard cross-entropy loss function. The expression for the weighted cross-entropy loss function is:

[0179]

[0180] Where yi is the one-hot encoding of the true label, yi=1 when the sample belongs to class i, otherwise yi=0; Let wi be the probability that a sample belongs to class i, predicted by the model; and let wi be the class weight of class i. During backpropagation, the gradient value generated by the weighted cross-entropy loss function is scaled according to the class weights. For minority class samples, due to their larger weights, the corresponding gradient value is amplified, making the model parameters more biased towards reducing the classification error of the minority class during updates. For majority class samples, due to their smaller weights, the corresponding gradient value is weakened, preventing the model from overfitting to the background class or common classes. Through the above dynamic class weight penalty mechanism, the model can automatically balance the learning priority between classes during training, effectively alleviating the model bias problem caused by the long-tail distribution of data, significantly improving the recognition accuracy of rare behavior classes, while maintaining stable recognition ability of common behavior classes.

[0181] In some embodiments, the training process of the behavior recognition model employs a joint decay and annealing scheduling algorithm. Specifically, the Adam optimizer is used for gradient updates of the model parameters, and an initial learning rate is set. And L2 regularized weight decay is introduced, with the weight decay coefficient set to By limiting the L2 norm of network weights, excessive node activation values ​​caused by excessive network depth are suppressed, effectively preventing model overfitting. Simultaneously, a dynamic learning rate annealing scheduler based on validation set performance monitoring is integrated. The monitoring metric is set to the validation set loss function value, with a patience value of five training epochs and a decay factor of 0.5. When the validation set loss function value no longer decreases within five consecutive training epochs, the scheduler automatically triggers a learning rate decay operation, multiplying the current learning rate by the decay factor to obtain the updated learning rate. This joint decay and annealing scheduling strategy allows the network to quickly cross the saddle point of the loss surface with a large learning rate in the early stages of training, accelerating the convergence process. In the later stages of training, it can finely search for the minimum region of the optimal loss surface with a very small step size, achieving fine-tuning of model parameters and thus obtaining a model with high generalization ability under limited sample conditions.

[0182] In some embodiments, a data augmentation step is performed before training the behavior recognition model:

[0183] Count the number of samples of each category in the training set to determine the target equilibrium quantity;

[0184] For a minority of categories where the number of samples is less than the target equilibrium number, at least two strategies are randomly selected from a preset physical simulation strategy pool and nested in series to generate supplementary samples.

[0185] In one specific implementation, before training the behavior recognition model, in order to further alleviate the class imbalance problem that is common in pet behavior data and improve the model's ability to recognize minority behaviors, the present invention performs a data augmentation step by expanding the minority samples through a physical simulation strategy.

[0186] Specifically, firstly, the number of samples in each category in the training set is counted to determine the target equilibrium quantity. By counting the number of samples in all categories in the training set, the distribution of sample numbers for each category is obtained. Let there be C behavior categories, and the number of samples in the i-th category be Ni. Then the target equilibrium quantity can be set as the maximum number of samples in each category, i.e. Alternatively, other preset values ​​can be set according to actual needs. By determining the target balance quantity, it is clear which categories need to be enhanced and the number of supplementary samples to be generated.

[0187] For minority classes with fewer samples than the target equilibrium number, at least two strategies are randomly selected from a pre-defined pool of physical simulation strategies and cascaded together to generate supplementary samples. Let Nc be the current number of samples for minority class c, then the number of supplementary samples to be generated is... For each supplementary sample to be generated, two to five enhancement strategies are randomly selected from the physical simulation strategy pool and applied to the original sample in a random sequence to generate new samples with diversity and semantic consistency. By randomly combining multiple strategies, the overfitting problem caused by a single enhancement method is avoided, enabling the model to learn more robust feature representations.

[0188] The physics simulation strategy pool can include the following enhancement strategies: Gaussian noise injection, which involves superimposing random noise following a Gaussian distribution onto the original signal. The model employs several techniques to enhance its robustness to sensor noise. These include: simulating sensor background thermal noise; injecting minute fluctuations in a specific frequency band to simulate high-frequency interference caused by mechanical vibrations during pet movement; amplitude scaling by multiplying by a scaling factor x′=αx, where α ranges from [0.8, 1.2] to simulate the difference in force amplitude exerted by pets of different sizes and weights performing the same action; smoothing amplitude distortion by multiplying a smooth curve generated by cubic spline interpolation by the original signal to simulate the uneven force exertion during a single action; time translation by cyclically shifting left or right along the time axis for several time steps, giving the model tolerance to translation at the moment of action occurrence, making the model independent of the absolute position of the action within the time window; nonlinear time distortion by randomly stretching and compressing local parts of the time axis, giving the model time-scale invariance to changes in action execution speed, enabling the model to recognize the same type of action at different speeds; and three-dimensional random rotation by multiplying the Euler angle sequence by a small random rotation matrix R(Δ). Δθ, Δψ) simulates the small physical offset of the collar or wearable device at the pet's wearing position, enhancing the model's adaptability to changes in sensor installation position; channel deactivation sets the sequence of a specific channel to zero with a preset probability, simulating the situation of single-axis sensor failure under extreme conditions, improving the system's robustness when the sensor partially fails.

[0189] In some embodiments, to improve the robustness of the pet behavior recognition model to temporal dynamic changes, a data augmentation strategy based on time axis density adjustment is introduced. The physical meaning of this strategy is that the same pet performing the same action (e.g., running, jumping, or walking) may exhibit different movement rates due to factors such as physical condition, emotional fluctuations, or environmental stimuli. To simulate this natural variability, the system employs a time-domain scaling method to controllably stretch or compress the original multidimensional time series.

[0190] Specifically, let the raw sensor data window corresponding to a certain action be represented as... ,in The number of time steps. For feature dimensions (such as acceleration, angular velocity, geomagnetic data, etc., nine axes), a random scaling factor is introduced. Typical values This factor is used to control the scaling factor. Based on this factor, a new timeline index vector is constructed after mapping. Subsequently, first-order linear interpolation (or cubic spline interpolation to improve smoothness) is used to generate the augmented sequence. , its first The values ​​at each time point are calculated as follows:

[0191] ;

[0192] This allows for the effective simulation of pet movement patterns at different rhythms while maintaining the semantics of the actions.

[0193] Furthermore, considering that wearable smart collars inevitably experience positional shifts during pets' daily activities (such as collar rotation, sliding, or tilting), this application also introduces a spatial rotation enhancement strategy to improve the model's adaptability to sensor attitude disturbances. The physical basis of this strategy is that the vector data measured by the accelerometer and magnetometer depend on the device's orientation in the carrier coordinate system, which is uncertain in actual use. Therefore, the original measured vector... (can be acceleration) or geomagnetic vector Apply tiny random rotations in three-dimensional space. , , They are respectively The components on the x, y, and z axes in a three-axis coordinate system. In the specific implementation, the perturbation angles around the three axes are sampled from a Gaussian distribution: roll angle... Pitch angle Yaw angle And based on this, construct a composite rotation matrix. The augmented vector representation is as follows: It is worth noting that since the gyroscope output is angular velocity (i.e., the time derivative of the Euler angle), when a rotational perturbation is applied to the attitude vector, the corresponding bias compensation must be introduced into the angular velocity sequence simultaneously to maintain kinematic consistency and avoid introducing non-physical motion artifacts.

[0194] Furthermore, to enhance the model's generalization ability to real-world sensor noise environments, the system also integrates a Gaussian noise injection mechanism. This strategy aims to simulate signal distortion caused by factors such as electromagnetic interference, circuit glitches, device aging, or temperature drift in complex real-world scenarios. This is applied to angular velocity data sequences. Combined with the white noise level covariance matrix determined by static calibration (such as Allan variance analysis) before the equipment leaves the factory. Zero-mean Gaussian white noise is injected into the data of each axis at each time step. Specifically, the noise term... ,in This is an adjustable noise intensity factor used to control the disturbance amplitude. The augmented angular velocity sequence is expressed as follows: This method not only improves the model's tolerance to low signal-to-noise ratio data, but also effectively prevents overfitting to idealized, clean data.

[0195] Finally, addressing the prevalent class imbalance problem in pet behavior datasets (e.g., scarce minority class samples such as "eating" and "sneezing"), the system employs the SMOTE (Synthetic Minority Over-sampling Technique) oversampling strategy to synthesize physically plausible new samples in the feature space. The physical meaning of this strategy lies in the fact that on a continuous behavioral manifold, there are numerous unobserved "transition states" or "boundary states" between adjacent poses. Although these states are not explicitly recorded, they can be reasonably inferred through interpolation. Specifically, for minority classes... deep feature vectors First, search within its Euclidean neighborhood. The nearest neighbor samples constitute a set Then, a nearest neighbor is randomly selected from them. And introduce random interpolation weights. New feature samples are synthesized.

[0196] The synthetic sample is located on the line connecting the original sample and its nearest neighbor, representing a reasonable intermediate state between two similar behavioral postures. This effectively increases the sample density of the minority class near the decision boundary without destroying the data manifold structure, significantly improving the classification network's recognition accuracy and generalization performance for rare behaviors.

[0197] Through the aforementioned two-stage heuristic precision enhancement mechanism—namely, the first stage statistically determines the number of samples needed for each category, and the second stage selects multiple strategies from the physical simulation strategy pool and generates supplementary samples through concatenation and nesting—this invention can effectively expand the number of minority class samples while maintaining the original behavioral semantics, thus balancing the number of samples in each category. This enhancement method fully considers the physical characteristics of pet behavior data and the actual working conditions of the sensor environment, generating enhanced samples with high physical rationality and diversity, significantly improving the training effect and generalization ability of subsequent behavior recognition models.

[0198] S5. Based on the smooth Euler angle sequence and behavioral semantic tags, a digital clone of the pet is generated in real time on a remote terminal; and / or, a user feedback signal is sent to the remote terminal based on the behavioral semantic tags so that the user can perceive the pet's real-time dynamics.

[0199] In some embodiments, the step of generating a digital clone of the pet in real time on a remote terminal based on the smoothed Euler angle sequence and behavioral semantic tags includes:

[0200] Obtain a preset motion mapping library, which contains the mapping relationship between behavior semantic tags and skeletal animation instructions;

[0201] The corresponding skeletal animation instructions are obtained by querying the action mapping library based on the behavioral semantic tags.

[0202] The rotation angles of each joint in the pet skeletal model are calculated in real time based on the smoothed Euler angle sequence to generate posture driving parameters;

[0203] The skeletal animation instructions are fused with the posture driving parameters to drive the skeletal animation engine of the 3D pet model, so that the overall movement of the 3D pet model matches the behavioral semantic label, and the local posture is synchronized with the smooth Euler angle sequence in real time.

[0204] The animation of the 3D pet model is rendered and output in real time through the display interface of the remote terminal, forming a digital clone of the pet.

[0205] In some embodiments, sending a user feedback signal to a remote terminal based on the behavioral semantic tag to enable the user to perceive the pet's real-time dynamics includes:

[0206] The behavioral semantic tags are acquired in real time, and the corresponding behavioral category and confidence level are determined based on the behavioral semantic tags.

[0207] The behavior category is matched with a preset feedback mapping table to determine the feedback type and feedback intensity corresponding to the behavior category;

[0208] A user feedback signal is generated based on the feedback type and feedback intensity, and the user feedback signal is sent to a remote terminal;

[0209] After receiving the user feedback signal, the remote terminal performs corresponding prompt output according to the feedback type and feedback intensity.

[0210] The feedback type includes at least one of vibration prompts, sound prompts, and light effect prompts; the feedback intensity is positively correlated with the confidence level of the behavior semantic tag, or matches the preset importance level corresponding to the behavior category.

[0211] In one specific implementation, after identifying the pet's behavior and obtaining the smoothed Euler angle sequence and behavioral semantic tags, this step generates a digital clone of the pet in real time on a remote terminal based on the smoothed Euler angle sequence and behavioral semantic tags; and / or, sends user feedback signals to the remote terminal based on the behavioral semantic tags, so that the user can perceive the pet's real-time dynamics, thereby realizing two-way interaction and emotional connection between humans and pets in a remote scenario.

[0212] In the embodiment of generating a digital pet avatar, a pre-defined motion mapping library is first obtained. This library contains the mapping relationship between behavioral semantic labels and skeletal animation instructions. This motion mapping library is pre-built through annotation by professional animators or training based on motion capture data, mapping abstract behavioral semantic labels such as "running," "jumping," "lying down," and "rolling" to specific skeletal animation instructions, such as running loop animations and jumping and landing animations. By establishing the mapping from behavioral semantics to animation instructions, the semantic conversion from high-level behavior recognition results to low-level animation control is achieved.

[0213] The system queries the action mapping library based on the behavioral semantic tags to obtain the corresponding skeletal animation instructions. When the behavior recognition model outputs the behavioral semantic tags for the current time window, the system uses these tags as keys to retrieve the animation instructions corresponding to that behavior from the action mapping library. For example, when the recognition result is "running," the system retrieves the running loop animation instruction; when the recognition result is "jumping," it retrieves the jumping animation instruction. These animation instructions define the macroscopic movement pattern of the overall actions of the digital pet avatar, ensuring that the overall behavior of the digital avatar is consistent with the behavior categories of a real pet.

[0214] The rotation angles of each joint in the pet's skeletal model are calculated in real time based on a smoothed Euler angle sequence to generate posture driving parameters. The smoothed Euler angle sequence contains the 3D spatial posture information of the location of the wearable device on the pet's neck. Through inverse kinematics algorithms, the neck posture information is extrapolated to the rotation angles of each joint in the pet's skeletal model, including the pitch, yaw, and roll angles of the head and neck joints, as well as the constraint postures transmitted to related joints such as the torso and limbs through the kinematic chain. These posture driving parameters provide refined local posture information of the digital avatar at the current moment, enabling the digital avatar to reflect subtle changes in the head and neck movements of a real pet in real time.

[0215] In some specific implementations, the rotation angles of each joint in the pet skeletal model are calculated in real time based on a smooth Euler angle sequence, and the resulting posture driving parameters can be:

[0216] Each pet skeletal model has a corresponding preset standard state template. For example, for the standard adult domestic cat model, the system presets its skeletal anatomy in a natural standing position, such as neck length, spine length from thoracic vertebrae to pelvis, upper arm and forearm length, thigh and lower leg length, relative offset vectors of each joint, rotation axis direction of each joint, and physiological range of joint angles (e.g., elbow pitch angle range of 0). Up to 120 When a user uses the system for the first time, they only need to select the pet category (such as "cat-standard"), and the system will automatically load the corresponding parameters. Then, it will calculate the rotation angles of each joint in the pet's skeletal model in real time based on a smooth Euler angle sequence to generate posture driving parameters.

[0217] Taking a pet cat as an example, to simplify calculations, the head and neck in its skeletal model are usually abstracted into two key nodes: the cervical joint and the head joint. The pose and position of the cervical joint are directly measured by the IMU (Inertial Measurement Unit) of a wearable device as known input; the pose of the head joint is solved through inverse kinematics iteration to satisfy the constraint of head orientation.

[0218] First, based on the smoothed Euler angle sequence calculated in step S2, the system extracts the complete attitude information of the neck joints at the current moment, including the yaw angle. Pitch angle Roll angle and position vector .in, , , They are respectively The components along the x, y, and z axes. The neck rotation matrix is ​​constructed using these three Euler angles. It is usually constructed in the ZYX axis order: Neck pose and position are known quantities, serving as the root node input of the kinematic chain, and are not involved in the inverse kinematics solution.

[0219] Secondly, the system sets the head joints as end effectors with inverse kinematics. Desired head position. Calculated by adding a preset head offset vector along the neck's orientation to the neck position: ,in This is a predefined neck-to-head offset vector, representing a fixed spatial offset of the skull center relative to the neck. Simultaneously, the desired head orientation is determined based on the current behavioral semantic label or natural standing posture. For example, when facing forward, the desired orientation is an identity matrix.

[0220] The system assigns the currently assumed angle vector to the head joints. ,in The yaw angle corresponding to the head, Corresponding pitch angle, The corresponding roll angle. The initial value can be set to zero (natural eye-level posture) or the solution result from the previous time step can be used to accelerate convergence by taking advantage of time continuity.

[0221] Current actual position of head The calculation is obtained from the local transformation of the cervical root node through the head joints. Since the head is considered a rigid extension of the neck, its position does not change with head rotation (rotation only changes orientation, not the position of the skull center relative to the neck), therefore... , and the assumed angle of the head Irrelevant. The current head orientation is obtained by multiplying the neck rotation matrix by the head local rotation matrix: ,in Constructed by assuming the angles follow the same ZYX axis order as the neck: .

[0222] Since the rotation of the head itself does not change the position of the skull center, the Jacobian matrix of the position of the end effector (head) with respect to the joint angles is zero: Therefore, the head angle is calculated based on orientation error rather than position error. Orientation error is defined as the difference between the desired head orientation and the current orientation, and can be represented by a rotation matrix or a quaternion. Let the orientation error rotation matrix be... Convert it to axis-angle or Euler angle form to obtain the error vector. Then, the inverse kinematics is used for iterative solution. To minimize the orientation error, the iterative update formula is similar to that of the position-damped least squares method: ,in This is the Jacobian matrix of head orientation versus joint angle, describing the relationship between the rate of change of head orientation and joint angular velocity. It is the identity matrix. It is continuously updated through iteration. and recalculate This continues until the orientation error is less than a preset threshold or the maximum number of iterations is reached.

[0223] For the torso, taking the aforementioned pet cat as an example, the spine in its skeletal model is abstracted as a kinematic chain composed of multiple joints, and the neck posture information is transmitted to the pelvis step by step through forward kinematics and inverse kinematics.

[0224] First, based on the smoothed Euler angle sequence calculated in step S2, the absolute pose of the neck joints in three-dimensional space is determined, including the rotation matrix. and position vector , which serves as the root node input of the spinal kinematic chain.

[0225] Secondly, spinal kinematic chain parameters are obtained from a preset standard state template, including: a fixed offset vector from the cervical spine to the thoracic spine. Length of bones in each joint of the spine ( , (This refers to the number of spinal joints) and the direction of rotation of each joint.

[0226] Transformation matrix of thoracic vertebra root nodes Calculated based on neck posture and preset offset:

[0227] ;

[0228] in It is by and The homogeneous transformation matrix represents the pose of the neck joints in the world coordinate system. It is a 4×4 matrix, and its top-left 3×3 submatrix is ​​a rotation matrix. The 3×1 sub-vector in the upper right corner is the position vector. The bottom row is [0,0,0,1]. For the preset offset vector The resulting translation transformation matrix is ​​a 4×4 matrix. Its upper left corner is a 3×3 identity matrix representing no rotation, and its upper right corner is a 3×1 offset vector representing the translation amount. The bottom row is [0,0,0,1], which is used to ensure the homogeneity of the matrix.

[0229] Local transformation matrix of each joint of the spine Given the joint angles to be solved and preset bone length Construction. Each local transformation matrix has the following form:

[0230] ;

[0231] in That is, a translation vector that includes the length of the bone, usually set as This represents the distance traveled from the current joint to the next joint along the local X-axis, a distance equal to one bone length. The root node transformation is propagated stepwise along the spinal chain to the pelvis via positive kinematics.

[0232] ;

[0233] from Extracting the rotational and translational components: Take its top left corner. The submatrix obtains the current pelvic pose. Take its upper right corner The subvector obtains the current pelvic position. .

[0234] In the inverse kinematics solution, the system is based on the desired pelvic position. For the end target. The estimation method is as follows:

[0235] ;

[0236] in The preset total length of the spine is obtained by summing the preset bone lengths of each spinal joint, i.e.:

[0237] ;

[0238] This is the location of the root node of the thoracic vertebra. Take the default orientation (or an approximation of neck posture) in a natural standing position. Then minimize iteratively. and The error between them is used to solve the spinal joint angle. Let the vector of the spinal joint angles be... , of which components ( ) indicates the first The rotation angle of each spinal joint. The iterative update formula uses the damped least squares method:

[0239] ;

[0240] in The angle of the pelvic position relative to the spinal joints The Jacobian matrix.

[0241] After iterative convergence, the result obtained from the last forward kinematics calculation Extract and This means that the information about the neck posture is transmitted to the trunk (pelvis) through the kinematic chain. This result reflects, on the one hand, the constraint of the neck posture on the overall orientation of the trunk, and on the other hand, provides a reference pose for the subsequent calculation of the limb joints.

[0242] For the limbs, taking the aforementioned pet cat as an example, the constraint posture calculation of the limb joints is divided into two independent branches: the forelimb chain and the hindlimb chain, with different starting points.

[0243] The forelimb chain originates from the root node of the thoracic vertebrae. The pose of this node... (including location) and posture The forward kinematic transmission process of the forelimb chain is calculated by combining the neck posture with a preset neck-to-thoracic offset, and is the same as the spinal transmission process. The shoulder joint position is obtained by adding a preset thoracic-to-shoulder offset vector to the thoracic root node. The position of the elbow joint is obtained by adding the length of the upper arm extending along the direction of the upper arm to the shoulder joint; the position of the forearm end is obtained by adding the length of the forearm extending along the direction of the forearm to the elbow joint.

[0244] The hindlimb chain originates from the pelvic reference point. The location of this node... and posture It is obtained from the spinal kinematic chain. The positive kinematic transmission process of the hindlimb chain is as follows: the hip joint position is obtained by adding a preset pelvic-to-hip joint offset vector to the pelvic reference point, that is... The knee joint position is obtained by adding the thigh length along the direction of the hip joint; the position of the rear end of the foot is obtained by adding the lower leg length along the direction of the calf.

[0245] The inverse kinematics solutions for both the forelimb and hindlimb chains employ the same damped least squares iterative principle as those for the spine. With the desired foot position as the end-effector, the angles of each joint are solved by iteratively minimizing the error between the current foot position and the desired position. During the solution process, physiological range constraints on joint angles are simultaneously applied (e.g., the range of elbow pitch angles is...). to ) and ground contact constraints (the foot position during the support phase satisfies) The remaining parts (Jacobi matrix calculation, damping factor adjustment, convergence condition judgment) can be consistent with the inverse kinematics of the spine.

[0246] By fusing skeletal animation commands with pose-driven parameters, the skeletal animation engine of the 3D pet model is powered, ensuring that the overall movement of the 3D pet model matches its behavioral semantic labels, and that local poses are synchronized in real time with smooth Euler angle sequences. The fusion process employs a hierarchical fusion strategy: skeletal animation commands control the basic behavioral loops and macroscopic movement sequences of the digital pet avatar, ensuring that the overall movement conforms to behavioral semantics; pose-driven parameters then overlay real-time pose adjustments of key joints such as the head and neck, enabling the digital avatar to respond in real-time to subtle changes in the movements of the real pet. Through this fusion mechanism, the digital avatar maintains both the accuracy of behavioral categories and achieves real-time pose following.

[0247] The system renders and outputs a 3D animated pet model in real time through a remote terminal's display interface, creating a digital avatar of the pet. Remote terminals include, but are not limited to, personal computer desktop software, smartphone applications, smartwatch faces, augmented reality glasses, or holographic displays. The 3D pet model uses a standard skeletal animation format and is rendered in real time by a graphics rendering engine. Users can intuitively observe the pet's real-time status and behavioral changes through the terminal's display interface, gaining a sense of presence and remote companionship.

[0248] In an embodiment where user feedback signals are sent to a remote terminal, behavioral semantic labels are acquired in real time, and the corresponding behavioral category and confidence level are determined based on these labels. The behavior recognition model outputs the behavioral semantic label along with the predicted probability value corresponding to that label as a confidence level, reflecting the model's degree of certainty regarding the recognition result. For example, if the model outputs the label "running" with a confidence level of 0.92, it indicates that the model has a high degree of confidence in the recognition result.

[0249] The system matches behavior categories with a pre-defined feedback mapping table to determine the corresponding feedback type and intensity. The feedback mapping table predefines feedback strategies for various behaviors: for active behaviors such as "running" and "jumping," vibration or sound cues may be used; for behaviors requiring user attention, such as "startled" or "abnormal," a combination of strong vibration and special sound effects may be used; for quiet behaviors such as "stillness" and "sleep," weak vibration or light effects may be used, or no feedback may be triggered. Through the feedback mapping table, the semantics of behavior are translated into user-perceived signals.

[0250] User feedback signals are generated based on the feedback type and intensity, and then sent to a remote terminal. Upon receiving the feedback signal, the remote terminal executes corresponding prompts according to the feedback type and intensity. Feedback types include at least one of vibration, sound, and light effects, which can be used individually or in combination. For example, vibration can be generated on a user's smartwatch or wristband, a specific sound effect can be played on a smartphone, or a breathing light effect can be displayed on a desktop terminal. Feedback intensity is positively correlated with the confidence level of the behavioral semantic label; that is, the higher the confidence level, the greater the feedback intensity, allowing the user to perceive the certainty of the model's judgment. Alternatively, it can be matched with a preset importance level corresponding to the behavior category. For example, "abnormal behavior" corresponds to the highest feedback intensity, "normal behavior" to a medium feedback intensity, and "no change" to a low feedback intensity or no feedback. Through this feedback mechanism, users can perceive changes in their pet's status in real time through tactile and auditory channels without continuously watching the digital avatar screen, achieving a multimodal remote pet monitoring experience.

[0251] In some embodiments, this application may further include: S6, in response to an interaction command initiated by a user through a remote terminal, sending a tactile feedback signal to the wearable device and driving the wearable device to perform physical tactile output for interaction.

[0252] In one specific implementation, based on the realization of pet status monitoring and real-time rendering of digital clones, this step further responds to the interactive commands initiated by the user through a remote terminal, sends tactile feedback signals to the wearable device, and drives the wearable device to perform physical tactile output to conduct remote two-way emotional interaction, filling the gap in the tactile interaction dimension of existing pet monitoring technology.

[0253] Specifically, the interactive commands initiated by the user through a remote terminal include at least one of the following: petting commands, hugging commands, calling commands, or soothing commands. The remote terminal may be a personal computer desktop software, a smartphone application, a smartwatch face, or a dedicated remote control device. The user triggers the corresponding interactive commands through interactive controls or gestures on the terminal interface. For example, the user can simulate a petting action by sliding the mouse on the digital avatar screen of the desktop software, or send a hug command by clicking the hug icon on the smartwatch. The interactive commands are transmitted in real time to the wearable device worn around the pet's neck via a cloud server or a peer-to-peer communication link.

[0254] In response to received interactive commands, the system determines the haptic feedback mode based on the type of the command and generates a corresponding haptic feedback signal. There is a preset mapping relationship between the haptic feedback mode and the type of interactive command: a stroking command is mapped to a continuous, low-frequency vibration waveform, simulating the feel of fingers gently stroking fur; a hug command is mapped to a gradual increase followed by a decrease in pressure, simulating the warmth of a hug; a call command is mapped to a short pulse vibration, simulating the effect of a gentle pat; and a soothing command is mapped to a smooth, intermittent low-frequency vibration, simulating the rhythm of a soothing pat. The haptic feedback signals corresponding to different interactive commands have differentiated characteristics in parameters such as waveform, frequency, amplitude, and duration to provide a diverse haptic experience.

[0255] The tactile feedback signal is sent to the wearable device, which then drives the device to perform physical tactile output. The wearable device has a built-in tactile actuator that converts electrical signals into physical mechanical motion, generating tactile stimulation that the pet can perceive. The tactile actuator includes at least one of a linear resonant actuator, an eccentric rotary mass motor, a shape memory alloy, or a micro-airbag structure. The linear resonant actuator and the eccentric rotary mass motor generate vibration-type tactile feedback, adjusting the intensity and rhythm of the vibration by controlling the frequency and amplitude of the driving voltage. The shape memory alloy utilizes the physical property of material contraction when energized to simulate a softer, slower, embracing sensation, providing a near-realistic hugging experience. The micro-airbag structure uses a micro-pump to inflate and deflate, changing the pressure inside the collar to provide non-vibrational contact feedback, suitable for pets sensitive to vibration. Driven by these tactile actuators, the wearable device generates physical tactile stimulation on the pet's neck that matches the user's interaction intention, allowing the pet to perceive the owner's remote care.

[0256] Reference Figure 2 This is a time-domain schematic diagram of the reconstruction of the tactile waveform of the touch interaction in one embodiment of the present invention; the horizontal axis is time (unit: seconds), and the vertical axis is the normalized motor drive amplitude (dimensionless). Figure 2 In the middle: The gray area represents a sinusoidal carrier signal with a frequency of 150Hz, whose frequency is set to the mechanical resonant frequency of the vibration motor to improve energy conversion efficiency; the red dashed line represents a low-frequency envelope signal with a frequency of 1.5Hz, corresponding to the physiological rhythm cycle of a gentle human touch, used to characterize the temporal characteristics of the interaction intention; the blue area represents the synthesized tactile output signal generated after amplitude modulation, which is the drive command ultimately sent to the wearable device actuator. Its instantaneous amplitude is obtained by real-time modulation of the carrier signal by the envelope signal, satisfying... .in The envelope function is 1.5Hz. The carrier frequency is used. This waveform structure achieves synergistic optimization of interactive intent semantics and actuator physical characteristics, ensuring that haptic feedback has natural rhythm, high responsiveness, and low power consumption.

[0257] Reference Figure 3 This is a schematic diagram of an airbag used to perform physical tactile output in one embodiment of the present invention. Figure 3(a) is a pressure-deformation angle response curve of the airbag used in this invention under different wall thickness parameters; the horizontal axis is the internal inflation pressure (unit: kPa), and the vertical axis is the single-sided deployment angle of the airbag (unit: °); the four curves in the figure correspond to the static mechanical characteristics of the airbag with wall thicknesses of 1.5 mm (blue square line), 2 mm (orange dotted line), 2.5 mm (gray triangle line), and 3 mm (yellow diamond line), respectively: as the wall thickness increases, the deployment angle of the airbag decreases under the same pressure, and the stiffness increases significantly, indicating that the wall thickness is a key structural parameter for controlling the compliance and load-bearing capacity of the airbag. This application preferably uses an airbag wall thickness of 2.0 mm. This wall thickness ensures sufficient compliance to achieve significant deformation while possessing good structural stiffness, avoiding uncontrolled deformation due to excessive thinness or sluggish response due to excessive thickness. Figure 3 (b) is a schematic diagram of the three-dimensional finite element mesh model of the airbag of the present invention. It is modeled using non-uniform tetrahedral elements. The structure consists of alternating high-density inner reinforcing ribs and outer wavy flexible zones, aiming to achieve a local stiffness gradient distribution, allowing the airbag to unfold orderly along a preset path during inflation, avoiding local stress concentration and asymmetric bulging; the structural design combines... Figure 3 The mechanical response characteristics shown in (a) ensure that the airbag has predictable and repeatable deformation behavior within the working pressure range of 0-60 kPa, making it suitable for controllable deformation actuators in human-computer interaction haptic feedback devices.

[0258] In some embodiments, the tactile feedback signal is adaptively adjusted based on the current wearing status of the wearable device. The wearable device has a built-in wear detection sensor to detect whether the device is worn correctly and the tightness of the fit between the device and the pet's neck. When the device is detected as not being worn or loosely, the system reduces or pauses the tactile feedback output to avoid ineffective interaction or energy waste. When the device is detected as being worn correctly, the tactile feedback intensity is dynamically adjusted according to the fit; if the fit is too tight, the output intensity is appropriately reduced to avoid overstimulation; if the fit is too loose, the output intensity is appropriately increased to ensure that the tactile stimulation is effectively transmitted to the pet.

[0259] In some embodiments, the interaction commands are continuous interaction commands. The system generates corresponding continuous tactile feedback signals in real time based on the continuous interaction parameters input by the user. For example, when a user performs a swipe operation on the touchscreen of a remote terminal, the system acquires parameters such as the swipe speed, direction, and pressure in real time, and dynamically generates tactile feedback signals based on these parameters. This ensures that the tactile feedback output by the wearable device is synchronized in real time with the trajectory and force of the user's finger swipe, simulating a realistic tactile interaction experience. The tactile feedback corresponding to the continuous interaction commands adopts a streaming transmission mode, that is, the user continuously generates a command stream during the interaction process, and the system responds in real time and continuously outputs tactile feedback until the user terminates the interaction.

[0260] Through the aforementioned two-way tactile feedback mechanism, this invention realizes a remote tactile interaction loop from user to pet. The user initiates interaction commands through a remote terminal, and the pet perceives physical tactile stimulation through a wearable device, receiving real-time care from its owner. Combined with the aforementioned motion monitoring and feedback mechanism from pet to user, a complete two-way emotional interaction loop is formed, significantly enhancing the emotional connection and companionship experience between humans and pets in remote scenarios, and solving the long-standing pain point of the lack of tactile interaction dimension in existing pet monitoring technologies.

[0261] Further reference Figure 4 As an implementation of the above-described method, this application provides an embodiment of a pet behavior reconstruction interaction system, which is consistent with... Figure 1 Corresponding to the method embodiments shown, the system can be specifically applied to various electronic devices.

[0262] refer to Figure 4 A pet behavior reconstruction interactive system, comprising:

[0263] The motion data acquisition module 210 is configured to collect pet motion data through a wearable device worn on the pet. The motion data includes three-axis acceleration data, three-axis angular velocity data and three-axis geomagnetic data, which are arranged in chronological order to form an original motion time series.

[0264] Attitude calculation module 220, configured to perform attitude calculation on the acquired motion data and generate a smooth Euler angle sequence, including:

[0265] Using quaternions as the system state vector, Kalman filter state equations and measurement equations are constructed; wherein, the state equations are based on the triaxial angular velocity data to construct a state transition matrix, the measurement equations are based on the triaxial acceleration data to construct an acceleration measurement matrix, and the measurement equations are based on the triaxial geomagnetic data to construct a geomagnetic measurement matrix.

[0266] The current system state vector is predicted using the state transition matrix to obtain the predicted quaternion.

[0267] The predicted quaternion is fused with the acceleration measurement matrix and the acceleration measurement value, and the predicted quaternion is fused with the geomagnetic measurement value using the geomagnetic measurement matrix, so as to optimize the prediction result and obtain the corrected quaternion.

[0268] The corrected quaternion is converted into a smooth Euler angle sequence;

[0269] The time-series window segmentation module 230 is configured to perform time-series window segmentation on the original motion time series to generate multiple window data with fixed time lengths;

[0270] Behavior recognition module 240, configured to perform behavior recognition based on the window data, includes:

[0271] The window data is input into a pre-trained behavior recognition model, and the spatiotemporal features of the window data are extracted by the spatiotemporal feature extraction network in the behavior recognition model. The spatiotemporal feature extraction network includes a cascaded one-dimensional convolutional neural network and a bidirectional long short-term memory network. The one-dimensional convolutional neural network is used to extract the local spatial collaborative features of each channel in the window data within the temporal neighborhood and output a local feature sequence. The bidirectional long short-term memory network is used to receive the local feature sequence and extract spatiotemporal features with global semantics along the temporal direction.

[0272] The spatiotemporal features are input into the classification network of the behavior recognition model, and the corresponding behavior semantic labels are output.

[0273] The state awareness module 250 is configured to render and generate a digital clone of the pet in real time on a remote terminal based on the smooth Euler angle sequence and behavioral semantic tags; and / or to send user feedback signals to the remote terminal based on the behavioral semantic tags so that the user can perceive the pet's real-time dynamics.

[0274] The physical tactile output module 260 is configured to send tactile feedback signals to the wearable device in response to an interaction command initiated by the user through a remote terminal, and drive the wearable device to perform physical tactile output for interaction.

[0275] In another aspect, this application also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable storage medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the following... Figure 1 The method shown.

[0276] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0277] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a computer terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0278] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.

Claims

1. A pet behavior reconstruction interaction method, characterized in that, Includes the following steps: Pet movement data is collected by wearable devices worn on the pet. The movement data includes three-axis acceleration data, three-axis angular velocity data and three-axis geomagnetic data, which are arranged in chronological order to form an original movement time series. The collected motion data is used to calculate the attitude and generate a smooth Euler angle sequence, including: Using quaternions as the system state vector, Kalman filter state equations and measurement equations are constructed; wherein, the state equations are based on the triaxial angular velocity data to construct a state transition matrix, the measurement equations are based on the triaxial acceleration data to construct an acceleration measurement matrix, and the measurement equations are based on the triaxial geomagnetic data to construct a geomagnetic measurement matrix. The current system state vector is predicted using the state transition matrix to obtain the predicted quaternion. The predicted quaternion is fused with the acceleration measurement matrix and the acceleration measurement value, and the predicted quaternion is fused with the geomagnetic measurement value using the geomagnetic measurement matrix, so as to optimize the prediction result and obtain the corrected quaternion. The corrected quaternion is converted into a smooth Euler angle sequence; The original motion time series is divided into time-series windows to generate multiple window data with fixed time lengths; Behavior recognition based on the window data includes: The window data is input into a pre-trained behavior recognition model, and the spatiotemporal features of the window data are extracted by the spatiotemporal feature extraction network in the behavior recognition model. The spatiotemporal feature extraction network includes a cascaded one-dimensional convolutional neural network and a bidirectional long short-term memory network. The one-dimensional convolutional neural network is used to extract the local spatial collaborative features of each channel in the window data within the temporal neighborhood and output a local feature sequence. The bidirectional long short-term memory network is used to receive the local feature sequence and extract spatiotemporal features with global semantics along the temporal direction. The spatiotemporal features are input into the classification network of the behavior recognition model, and the corresponding behavior semantic labels are output. Based on the smooth Euler angle sequence and the behavioral semantic tags, a digital clone of the pet is generated in real time on a remote terminal; and / or, a user feedback signal is sent to the remote terminal based on the behavioral semantic tags so that the user can perceive the pet's real-time dynamics. In response to an interactive command initiated by a user through a remote terminal, a tactile feedback signal is sent to the wearable device, and the wearable device is driven to perform physical tactile output for interaction.

2. The pet behavior reconstruction interaction method according to claim 1, characterized in that, The step of dividing the original motion time series into time-series windows to generate multiple window data of fixed time lengths includes: First, all types of data in the original motion time series are uniformly resampled to a preset reference frequency so that all types of data streams have a uniform time step on the time axis. Set a window of fixed time length, and the window length corresponds to a preset number of time steps, so that each window covers the same length of motion data; The original motion time series is divided along the time axis by a preset sliding step size, where the sliding step size is less than the window length.

3. The pet behavior reconstruction interaction method according to claim 1, characterized in that: The one-dimensional convolutional neural network is used to extract the local spatial collaborative features of each channel in the temporal neighborhood of the window data, and outputs a local feature sequence including: The one-dimensional convolutional neural network includes a cascaded first convolutional block, a second convolutional block, and a third convolutional block; The first convolutional block performs a one-dimensional convolution operation on the window data to extract the local spatial collaborative features of each channel in the temporal neighborhood. After batch normalization and ReLU activation, the temporal dimension is reduced by max pooling, and the first intermediate feature is output. The second convolutional block performs a one-dimensional convolution operation on the first intermediate feature to further extract local spatial collaborative features. After batch normalization and ReLU activation, it achieves temporal dimension reduction again through max pooling and outputs the second intermediate feature. The third convolutional block performs a one-dimensional convolution operation on the second intermediate feature to extract deep local spatial collaborative features. After batch normalization and ReLU activation, the time dimension remains unchanged, and the local feature sequence is output.

4. The pet behavior reconstruction interaction method according to claim 3, characterized in that: The bidirectional long short-term memory network is used to receive the local feature sequence and extract spatiotemporal features with global semantics along the temporal direction, including: The bidirectional long short-term memory network is a two-layer bidirectional structure, which includes a forward long short-term memory layer and a reverse long short-term memory layer. The local feature sequence is input into the forward long short-term memory layer along the time axis to extract forward temporal features; The local feature sequence is input into the reverse long short-term memory layer in reverse along the time axis to extract reverse temporal features; The forward time series feature and the reverse time series feature are concatenated at the same time step to obtain the bidirectional time series feature at the corresponding time step; The bidirectional temporal features of the last time step are extracted as spatiotemporal features.

5. The pet behavior reconstruction interaction method according to claim 4, characterized in that: The classification network includes a first dropout layer, a first fully connected layer, a ReLU activation layer, a second dropout layer, and a second fully connected layer connected in sequence. Furthermore, the process of inputting the spatiotemporal features into the classification network of the behavior recognition model and outputting corresponding behavior semantic labels includes: The spatiotemporal features are input into the first dropout layer, and some neurons are randomly dropped at a preset first dropout rate to obtain the first regularized features. The first regularized feature is input into the first fully connected layer to map the dimensions of the spatiotemporal features from the first dimension to the second dimension, thereby obtaining the first fully connected feature; The first fully connected feature is nonlinearly transformed through the ReLU activation layer to obtain the activation feature; The activation features are input into the second dropout layer, and some neurons are randomly dropped at a preset second dropout rate to obtain the second regularization features. The second regularized feature is input into the second fully connected layer to map the feature dimension from the second dimension to the number of categories dimension, thus obtaining the Logits vector; The Logits vector is activated by the Softmax function, the predicted probability of each behavior category is calculated, and the category with the highest probability is selected as the behavior semantic label for output.

6. The pet behavior reconstruction interaction method according to claim 1, characterized in that: The step of generating a digital clone of the pet in real time on a remote terminal based on the smoothed Euler angle sequence and behavioral semantic tags includes: Obtain a preset motion mapping library, which contains the mapping relationship between behavior semantic tags and skeletal animation instructions; The corresponding skeletal animation instructions are obtained by querying the action mapping library based on the behavioral semantic tags. The rotation angles of each joint in the pet skeletal model are calculated in real time based on the smoothed Euler angle sequence to generate posture driving parameters; The skeletal animation instructions are fused with the posture driving parameters to drive the skeletal animation engine of the 3D pet model, so that the overall movement of the 3D pet model matches the behavioral semantic label, and the local posture is synchronized with the smooth Euler angle sequence in real time. The animation of the 3D pet model is rendered and output in real time through the display interface of the remote terminal, forming a digital clone of the pet.

7. The pet behavior reconstruction interaction method according to claim 1, characterized in that: The step of sending user feedback signals to a remote terminal based on the behavioral semantic tags, so that the user can perceive the pet's real-time dynamics, includes: The behavioral semantic tags are acquired in real time, and the corresponding behavioral category and confidence level are determined based on the behavioral semantic tags. The behavior category is matched with a preset feedback mapping table to determine the feedback type and feedback intensity corresponding to the behavior category; A user feedback signal is generated based on the feedback type and feedback intensity, and the user feedback signal is sent to a remote terminal; After receiving the user feedback signal, the remote terminal performs corresponding prompt output according to the feedback type and feedback intensity. The feedback type includes at least one of vibration prompts, sound prompts, and light effect prompts; the feedback intensity is positively correlated with the confidence level of the behavior semantic tag, or matches the preset importance level corresponding to the behavior category.

8. The pet behavior reconstruction interaction method according to claim 5, characterized in that, The behavior recognition model is trained using a dynamic category weight penalty strategy, including: Count the number of samples in each category in the training set to determine the sample number distribution of each category; For each category, a category weight is calculated, which is inversely proportional to the number of samples in that category. A weighted cross-entropy loss function is constructed as the loss function of the behavior recognition model. The weighted cross-entropy loss function is based on the standard cross-entropy loss function, where the loss term for each category is multiplied by the corresponding category weight.

9. A pet behavior reconstruction interaction method according to any one of claims 1-8, characterized in that, Before training the behavior recognition model, perform data augmentation steps: Count the number of samples of each category in the training set to determine the target equilibrium quantity; For a minority of categories where the number of samples is less than the target equilibrium number, at least two strategies are randomly selected from a preset physical simulation strategy pool and nested in series to generate supplementary samples. The physics simulation strategy pool includes at least: A scaling factor with a random value within a preset range is introduced. The scaling factor is used to change the time axis density of the original time series. The new time axis index vector after mapping is calculated, and the augmented time series is generated by linear interpolation or cubic spline interpolation. For each sensor measurement vector in the original motion time series, generate independent random small rotation angles around the three-dimensional spatial coordinate axes. Construct a spatial rotation perturbation matrix based on the three rotation angles. Multiply the current sensor measurement vector by this spatial rotation perturbation matrix to generate the augmented sensor vector. For the triaxial angular velocity data, combined with the white noise level measured by the static calibration characteristics of the wearable device, Gaussian white noise following a normal distribution is injected into each data axis. The amplitude of the injected noise is adjusted by a preset noise intensity control factor to generate augmented triaxial angular velocity data. Deep feature vectors are extracted from a few categories. Multiple nearest neighbor samples of the deep feature vectors in the feature space are calculated. One of the nearest neighbor samples is randomly selected, and an interpolation weight with a random value between zero and one is introduced. The difference vector between the deep feature vector and the nearest neighbor sample is multiplied by the interpolation weight and then superimposed on the deep feature vector to synthesize a new feature vector sample.

10. A pet behavior reconstruction interactive system, characterized in that, The system includes: The motion data acquisition module is configured to collect pet motion data through a wearable device worn on the pet. The motion data includes three-axis acceleration data, three-axis angular velocity data, and three-axis geomagnetic data, which are arranged in chronological order to form an original motion time series. The attitude calculation module is configured to perform attitude calculation on the acquired motion data and generate a smooth Euler angle sequence, including: Using quaternions as the system state vector, Kalman filter state equations and measurement equations are constructed; wherein, the state equations are based on the triaxial angular velocity data to construct a state transition matrix, the measurement equations are based on the triaxial acceleration data to construct an acceleration measurement matrix, and the measurement equations are based on the triaxial geomagnetic data to construct a geomagnetic measurement matrix. The current system state vector is predicted using the state transition matrix to obtain the predicted quaternion. The predicted quaternion is fused with the acceleration measurement matrix and the acceleration measurement value, and the predicted quaternion is fused with the geomagnetic measurement value using the geomagnetic measurement matrix, so as to optimize the prediction result and obtain the corrected quaternion. The corrected quaternion is converted into a smooth Euler angle sequence; The time-series window segmentation module is configured to perform time-series window segmentation on the original motion time series to generate multiple window data with fixed time lengths; The behavior recognition module, configured to perform behavior recognition based on the window data, includes: The window data is input into a pre-trained behavior recognition model, and the spatiotemporal features of the window data are extracted by the spatiotemporal feature extraction network in the behavior recognition model. The spatiotemporal feature extraction network includes a cascaded one-dimensional convolutional neural network and a bidirectional long short-term memory network. The one-dimensional convolutional neural network is used to extract the local spatial collaborative features of each channel in the window data within the temporal neighborhood and output a local feature sequence. The bidirectional long short-term memory network is used to receive the local feature sequence and extract spatiotemporal features with global semantics along the temporal direction. The spatiotemporal features are input into the classification network of the behavior recognition model, and the corresponding behavior semantic labels are output. The status awareness module is configured to render and generate a digital clone of the pet in real time on a remote terminal based on the smooth Euler angle sequence and behavioral semantic tags; and / or to send user feedback signals to the remote terminal based on the behavioral semantic tags so that the user can perceive the pet's real-time dynamics. The physical tactile output module is configured to respond to an interactive command initiated by a user through a remote terminal, send a tactile feedback signal to the wearable device, and drive the wearable device to perform physical tactile output for interaction.

Images