An AI-based pet emotion recognition system

Abstract

The present invention relates to the field of pet emotion recognition technology, and in particular to an AI-based pet emotion recognition system. It includes a pet data acquisition module, a feature extraction module, an emotion recognition module, an emotion dynamics tracking module, and an interactive feedback module. The pet data acquisition module acquires multimodal data through a sensor network, and the feature extraction module uses sparse coding and dictionary learning algorithms to extract key features, including facial expressions, sound spectra, physiological changes, and movement behaviors. The emotion recognition module uses capsule networks and generative adversarial networks for emotion recognition and classification. The emotion dynamics tracking module uses graph neural networks to predict emotion trends. The interactive feedback module generates emotion management strategies through reinforcement learning. The present invention not only improves the accuracy and efficiency of pet emotion recognition, provides more effective pet emotion management and interactive support, but also provides pet owners with practical management tools to promote the health and happiness of their pets.

Description

Technical Field

[0001] The present invention relates to the technical field of pet emotion recognition, and in particular to an AI-based pet emotion recognition system. Background Art

[0002] In today's society, pets are increasingly becoming family members, and their owners' attention to their emotions and well-being is becoming increasingly important. However, since pets cannot directly express their emotions through language, owners often rely on their behavioral and physiological characteristics to make judgments, a process that is susceptible to subjective factors and lacks accuracy. Currently, pet emotion recognition primarily relies on the observation and judgment of experienced veterinarians or pet trainers. However, this approach is not only highly subjective but also requires extensive expertise and experience. Current issues also exist: existing technologies often rely on a single or limited type of data to determine pet emotions, lacking the comprehensive collection and utilization of multimodal data, resulting in insufficient accuracy and comprehensiveness in emotion recognition. Existing technologies often rely on simple classification algorithms, which cannot effectively handle the diversity and complexity of pet emotions, resulting in low recognition accuracy. Existing technologies are generally limited to static emotion judgments, unable to track and predict dynamic changes in pet emotions, or to promptly identify emotional fluctuations, making it difficult to provide effective support for long-term pet health and behavior management. Summary of the Invention

[0003] To solve the above problems, the present invention provides an AI-based pet emotion recognition system, which solves the problem of how to accurately identify, dynamically track and predict pet emotions through multimodal data and advanced algorithms, thereby improving the accuracy and comprehensiveness of pet emotion recognition and providing more effective support for pet health and behavior management.

[0004] To achieve the above object, the technical solution adopted by the present invention is:

[0005] An AI-based pet emotion recognition system includes a pet data acquisition module, a feature extraction module, a pet emotion recognition module, an emotion dynamic tracking module, and an interactive feedback module that are sequentially connected in communication;

[0006] The pet data acquisition module is used to collect multimodal data of the pet through a sensor network; the multimodal data includes video images, sounds, body temperature, heart rate, position and activity level of the pet;

[0007] The feature extraction module is used to receive the multimodal data of the pet and extract key features through sparse coding and dictionary learning algorithms; the key features include the pet's facial expression features, sound spectrum features, physiological index change features and movement behavior features;

[0008] The pet emotion recognition module is used to recognize and classify pet emotions, including happiness, excitement, anxiety, fear, and anger, through a capsule network algorithm and a generative adversarial network.

[0009] The emotional dynamics tracking module is used to represent the pet's emotional state as nodes through a graph neural network, and to track and predict emotional change trends by learning the relationships between nodes and graph structure features;

[0010] The interactive feedback module is used to generate emotion management strategies and interaction suggestions through a reinforcement learning algorithm based on the results of emotion recognition and dynamic tracking, and interact with pet owners through smart devices.

[0011] Furthermore, the operation process of the feature extraction module includes the following steps:

[0012] Performing format conversion and normalization processing on the multimodal data, including grayscale conversion of video images, frequency domain analysis of sound signals, and filtering and smoothing of body temperature and heart rate;

[0013] Use sparse coding algorithm to encode the features of preprocessed data, construct the initial feature matrix, and use sparse constrained optimization algorithm to select key technical features;

[0014] A dictionary learning algorithm is used to train the feature matrix and generate a feature dictionary suitable for pet emotion recognition;

[0015] The extracted features are selected and combined, and the feature selection algorithm is used to construct the final key feature set based on the importance and relevance of the features, which provides input for the emotion recognition module.

[0016] Furthermore, the formula of the sparse coding algorithm is as follows:

[0017]

[0018] Among them, x * represents the optimal sparse representation, i.e., the key features extracted from the multimodal data; y represents the feature vector of the preprocessed multimodal data; D represents the basis vector matrix of the feature space, i.e., the combination of all features in emotion recognition; x represents the sparse representation of the input data in the dictionary, i.e., the key feature combination corresponding to the pet's emotions; argmin x It means finding the parameter x that minimizes the objective function; The reconstruction error term represents the square error between the dictionary matrix D reconstructed data through sparse representation x and the original input data y. If Dx is very close to y, it means that the extracted sparse features fully describe the current emotion of the pet; λ||x||1 represents the sparsity constraint term that controls the sparsity degree through λ.

[0019] Furthermore, the operation process of the pet emotion recognition module includes the following steps:

[0020] receiving the key features from the feature extraction module;

[0021] Based on the key features, an emotion recognition model is constructed using a capsule network, and model training is performed in combination with a generative adversarial network. The capsule network is used to capture the hierarchical relationship and spatial information between features, and a dynamic routing mechanism is used to improve the model's ability to understand complex emotion features.

[0022] The processed multimodal data is input into the emotion recognition model to classify and identify the pet's emotional state, output the initial emotion classification result, and use the generative adversarial network to strengthen the emotion recognition process and generate the final emotion recognition result.

[0023] Furthermore, the formula of the emotion recognition model is as follows:

[0024]

[0025] in, represents the total loss function, which combines the losses of the capsule network and the generative adversarial network to optimize the emotion recognition model; α represents the weight factor used to control the influence of the capsule network feature reconstruction loss; β represents the weight factor used to control the influence of the generative adversarial network loss; v j represents the output vector of the capsule network, i.e., the key features of pet emotions extracted; t j represents the target emotion feature vector, which is used to guide the accuracy of the output features of the capsule network; D(x) represents the output probability of the discriminator for the real data x; D(G(z)) represents the output probability of the discriminator for the generated data (G(z)); x represents the multimodal data sample; z represents the noise vector sampled from the latent space, which is used to generate the potential emotion features.

[0026] Furthermore, the formula of the capsule network is as follows:

[0027]

[0028] Among them, v j Represents the final recognized emotion feature, the output vector of the capsule network; s j Represents the sum of the input vectors of the j-th emotion node.

[0029] Furthermore, the operation process of the emotion dynamics tracking module includes the following steps:

[0030] The emotion recognition results generated by the pet emotion recognition module are represented as nodes of a graph neural network, where each node corresponds to an emotional state.

[0031] Use graph neural networks to build a relationship graph between emotional states. The edges in the graph represent the transformation relationships between emotional states, and the weights of the edges reflect the probability or similarity of the transformation.

[0032] Based on the current emotional state and its adjacent emotional states, the features of each node are updated through a graph neural network to capture the mutual influence and dynamic changes between emotions;

[0033] Perform time series analysis on the features of nodes in the graph neural network to predict future trends in pet emotions and generate emotional state sequences;

[0034] Based on the emotional state sequence, potential emotional abnormalities are identified, and an automatic alarm mechanism for emotional changes is implemented by setting thresholds;

[0035] The predicted emotion change trend and current emotion state are passed to the interactive feedback module.

[0036] Furthermore, the formula of the graph neural network is as follows:

[0037]

[0038] in, Represents the feature vector of node i at time step t, that is, the emotional state characteristics of the pet at that moment; represents the updated feature vector of node i at time step t+1; represents the set of neighbor nodes of node i, that is, other emotional states directly related to the current emotional state; c ij represents the square root of the number of connections or degree between nodes i and j; W represents the influence weight between emotional states; W0 represents the influence of the emotional state itself; σ represents the activation function.

[0039] Furthermore, the operation process of the interactive feedback module includes the following steps:

[0040] The pet's current emotional state and predicted emotional change trends are communicated to the pet owner in real time via smart devices, including smartphones, tablets, smart speakers, and smart home devices.

[0041] Based on the identified emotional state, a reinforcement learning algorithm is used to generate personalized emotional management strategies, provide behavioral suggestions and environmental adjustment plans;

[0042] Generate interaction suggestions and management strategies based on the pet's emotional state, including play activities, rest schedules, and emotional communication;

[0043] By analyzing and collecting user feedback, the emotion recognition model and interaction strategy are dynamically adjusted and optimized.

[0044] The beneficial effects of the present invention are:

[0045] Through the pet data acquisition module, the present invention utilizes a sensor network to acquire multimodal data such as video images, sounds, body temperature, heart rate, location, and activity level of pets, providing rich context and details, helping to fully understand the pet's condition. The feature extraction module employs sparse coding and dictionary learning algorithms to extract key features from multimodal data, including facial expressions, sound spectra, physiological indicators, and motor behavior characteristics. This method effectively extracts core information related to emotions, ensuring accurate and effective recognition. The pet emotion recognition module utilizes a capsule network algorithm and a generative adversarial network to identify and classify pet emotions, accurately distinguishing between various emotional states such as happiness, excitement, anxiety, fear, and anger, providing more precise emotional understanding. The emotion dynamics tracking module uses a graph neural network to represent emotional states as nodes and, by learning the relationships between nodes and graph structure features, tracks and predicts emotional trends in real time. This function can promptly identify emotional fluctuations, providing strong support for pet health and behavior management. The interactive feedback module uses reinforcement learning algorithms to generate personalized emotion management strategies and interaction suggestions based on the results of emotion recognition and dynamic tracking, and interacts with pet owners through smart devices to help owners better understand and respond to their pets' needs and improve the human-pet relationship. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 This is a module schematic diagram of an AI-based pet emotion recognition system of the present invention.

[0047] Figure 2 This is a flowchart of the operation process of the pet emotion recognition module provided by one embodiment of the present invention.

[0048] Figure 3 This is a flowchart of the operation process of the emotion dynamics tracking module provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0049] See also Figure 1-3 As shown, the present invention relates to an AI-based pet emotion recognition system.

[0050] Example

[0051] An AI-based pet emotion recognition system includes a pet data acquisition module, a feature extraction module, a pet emotion recognition module, an emotion dynamic tracking module, and an interactive feedback module that are sequentially connected in communication;

[0052] The pet data acquisition module is used to collect multimodal data of the pet through a sensor network; the multimodal data includes video images, sounds, body temperature, heart rate, position and activity level of the pet;

[0053] It should be noted that the pet data acquisition module is composed of multiple sensors, forming a complex sensor network to ensure comprehensive and accurate pet status monitoring. Each sensor is responsible for different types of data collection. The following is a detailed description of each sensor:

[0054] Video image sensor: A camera placed in the pet's activity area captures the pet's facial expressions and body posture. The camera should have night vision to ensure clear images in all lighting conditions. The camera should be placed in areas where the pet frequently moves, such as the pet's bed, food area, and play area, to ensure that the pet's behavior can be captured from multiple angles. The image data collected by the camera undergoes preliminary processing using edge computing devices, such as image denoising and brightness adjustment.

[0055] Sound sensor: Installed on the pet's collar or near the pet's activity area, it captures the pet's vocalizations, breathing, and other sound characteristics. The sound data undergoes pre-processing by the front-end device, including noise removal, gain adjustment, and sampling rate conversion.

[0056] Physiological sensors: Wearable devices that integrate multiple sensors and are worn around your pet's neck to monitor their physiological indicators in real time. Temperature sensors: Monitor your pet's surface temperature to help determine their health and emotional state. Heart rate sensors: Monitor your pet's heart rate using photoplethysmography (PPG) or electrocardiography (ECG) technology. Respiration sensors: Monitor your pet's breathing rate and pattern.

[0057] GPS module: The smart collar's built-in GPS module records your pet's location and activity path in real time. This location data is transmitted to the central processing unit via a wireless network and combined with other sensor data for comprehensive analysis.

[0058] Accelerometer: The smart collar's built-in three-axis accelerometer is used to detect your pet's activity and movement patterns, including walking, running, jumping, and other behaviors. By extracting and analyzing the acceleration data, we can identify the pet's behavior patterns and activity intensity.

[0059] All sensor data is transmitted to the central processing unit via Bluetooth Low Energy (BLE), Wi-Fi, or other wireless communication technologies. For long-distance transmission, cellular networks may be used. Some data is processed locally using edge computing to reduce transmission burden and latency. After receiving the data, it is stored in a distributed database to ensure secure and reliable data storage. The data storage system should support efficient read and write operations and backup and recovery mechanisms.

[0060] All sensor data must be encrypted during transmission to protect confidentiality and integrity. Common encryption methods include AES (Advanced Encryption Standard) and TLS (Transport Layer Security). The system should incorporate strict access control mechanisms to ensure that only authorized users and devices can access and process pet data. Data privacy regulations (such as GDPR) must be followed to protect the privacy of users and pets during data collection and processing. The system should also provide functionality for managing and deleting user data.

[0061] The feature extraction module is used to receive the multimodal data of the pet and extract key features through sparse coding and dictionary learning algorithms; the key features include the pet's facial expression features, sound spectrum features, physiological index change features and movement behavior features;

[0062] The operation process of the feature extraction module includes the following steps:

[0063] Performing format conversion and normalization processing on the multimodal data, including grayscale conversion of video images, frequency domain analysis of sound signals, and filtering and smoothing of body temperature and heart rate;

[0064] In one embodiment, the video image processing is as follows:

[0065] Grayscale processing: Convert RGB images to grayscale using YUV or HSV color space conversion. Grayscale conversion simplifies computation and reduces noise interference with feature extraction. During the grayscale conversion process, local contrast enhancement techniques are used to enhance image detail.

[0066] Image denoising and smoothing: Apply a Gaussian filter to remove high-frequency noise from an image. The Gaussian filter parameters can be adaptively adjusted based on the image noise level. A bilateral filter is used to remove noise while preserving image edge information.

[0067] Scale Normalization: Resize images of different sizes to a uniform size (e.g., 224x224 pixels) to fit the subsequent feature extraction network. Use bilinear interpolation to scale the images to maintain image clarity and proportion.

[0068] The sound signal is processed as follows:

[0069] Frequency Domain Analysis: The fast Fourier transform (FFT) is used to convert the time-domain sound signal into a spectrogram. The short-time Fourier transform (STFT) is then used to obtain a frequency-domain representation for analyzing the frequency characteristics of the sound. To enhance the readability of the spectrogram, Mel-frequency cepstral coefficients (MFCCs) are used to extract sound features. MFCCs effectively represent the spectral shape of sound signals and are suitable for analyzing emotional characteristics.

[0070] Denoising: Use spectral subtraction to eliminate background noise. This improves the signal-to-noise ratio by estimating the noise spectrum and subtracting it from the signal. Adaptive filtering techniques (such as Wiener filtering) are implemented to dynamically adjust filter parameters to accommodate changes in ambient noise.

[0071] Physiological indicators are processed as follows:

[0072] Temperature and heart rate filtering: A Kalman filter is used to smooth data, eliminating sensor noise and occasional measurement errors. The Kalman filter uses a dynamic model to predict the current state and combines it with observed values ​​for optimal estimation. Peak detection and RR interval analysis are performed on the heart rate signal to extract emotion-related features such as heart rate variability (HRV).

[0073] Trend Analysis: Linear regression is used to analyze time series trends in heart rate and body temperature to identify potential abnormal fluctuations. Time series decomposition methods (such as STL decomposition) are used to separate seasonality, trend, and residual components for in-depth analysis of changes in physiological indicators.

[0074] Use sparse coding algorithm to encode the features of preprocessed data, construct the initial feature matrix, and use sparse constrained optimization algorithm to select key technical features;

[0075] Specifically, a sparse autoencoder is used to construct a three-layer network structure, in which the input layer corresponds to the original features, and the sparse hidden layer in the middle maintains sparsity through regularization constraints (L1 norm). Sparse coding solves the sparse solution so that the error when the output layer reconstructs the input layer features is minimized, thereby extracting sparse features with high recognition. The sparse representation result is constructed as the initial feature matrix. Each column of the feature matrix represents a sparse feature vector, and each row corresponds to a sample. Dimensionality reduction techniques (such as PCA or t-SN E) are used to compress the feature matrix to reduce redundant dimensions and enhance the distinguishing ability of features. The orthogonal matching pursuit (OMP) algorithm is used to solve the sparse coding problem. OMP iteratively selects the features most correlated with the residual error, thereby gradually constructing a sparse solution. The alternating direction multiplier method (ADMM) is applied for sparse optimization to accelerate the solution process of the sparse solution.

[0076] A dictionary learning algorithm is used to train the feature matrix and generate a feature dictionary suitable for pet emotion recognition;

[0077] Specifically, samples are randomly selected from the sparse feature matrix as the initial dictionary for dictionary learning. The initial dictionary should cover the feature distribution of different emotional states to improve the generalization ability of the model. The dictionary is updated using the K-SVD algorithm. K-SVD continuously optimizes the quality of the dictionary by alternately minimizing sparse coding and dictionary update errors. In each iteration, K-SVD uses singular value decomposition to update the dictionary to improve its feature reconstruction ability. Through the sparse representation characteristics of the dictionary, the input features are sparsely reconstructed to ensure that the dictionary can efficiently represent the structure of the input data. The reconstruction error of the dictionary for the input data is calculated, and the reconstruction ability of the dictionary is measured by the mean square error (MSE). The reconstruction effects of different dictionaries are compared to select the optimal feature dictionary. The sparse representation coefficient is used to analyze the sparse representation ability of the dictionary to ensure that the dictionary can effectively distinguish different emotion categories.

[0078] The extracted features are selected and combined, and the feature selection algorithm is used to construct the final key feature set based on the importance and relevance of the features, which provides input for the emotion recognition module.

[0079] Furthermore, the formula of the sparse coding algorithm is as follows:

[0080]

[0081] Among them, x * represents the optimal sparse representation, that is, the key features extracted from multimodal data; y represents the feature vector of the preprocessed multimodal data, which provides multimodal information about the pet's current state and provides a comprehensive data foundation for emotion analysis; D represents the basis vector matrix of the feature space, that is, the combination of all features in emotion recognition. The richness and accuracy of the dictionary matrix directly affect the effect of sparse coding. The more accurately the dictionary can express emotional features, the more helpful it is to accurately identify emotions; x represents the sparse representation of the input data in the dictionary, that is, the key feature combination corresponding to the pet's emotions. Sparsity ensures the efficiency of the system and reduces computational complexity by selectively focusing on the most representative features; arg min x It means finding the parameter x that minimizes the objective function; The reconstruction error term represents the square error between the dictionary matrix D reconstructed data through sparse representation x and the original input data y. If Dx is very close to y, it means that the extracted sparse features fully describe the current emotion of the pet; λ||x||1 represents the sparsity constraint term that controls the sparsity degree through λ.

[0082] The pet emotion recognition module is used to recognize and classify pet emotions, including happiness, excitement, anxiety, fear, and anger, through a capsule network algorithm and a generative adversarial network.

[0083] The operation process of the pet emotion recognition module includes the following steps:

[0084] receiving the key features from the feature extraction module;

[0085] Based on the key features, an emotion recognition model is constructed using a capsule network, and model training is performed in combination with a generative adversarial network. The capsule network is used to capture the hierarchical relationship and spatial information between features, and a dynamic routing mechanism is used to improve the model's ability to understand complex emotion features.

[0086] Specifically, the input multimodal feature vector is converted into an initial capsule through a convolutional layer. Each capsule contains a feature vector, representing a specific attribute or sub-feature of a feature. The initial capsule layer is responsible for capturing low-level feature information and performing preliminary spatial encoding. The initial capsule layer is connected to the high-level capsule layer through a dynamic routing mechanism. Dynamic routing selectively transmits information through weight updates to form high-level feature capsules. The high-level capsule layer encodes the hierarchical relationship between features and can represent the complex structure and pattern of emotions. A dynamic routing algorithm is used between capsules to dynamically adjust the routing coefficient. The output of each capsule is updated through iterative calculation to ensure that important features receive higher attention. The steps of the dynamic routing algorithm are: initialize the routing coefficient to an equal value; calculate the prediction vector of each initial capsule for the high-level capsule; update the routing coefficient to minimize the prediction error; and finally output the feature representation of the high-level capsule through iterative solution.

[0087] Generative adversarial networks (GANs) are used to enhance the emotion recognition capabilities of capsule networks. They improve the model's generalization performance through adversarial training between generated and real samples. The generator network generates simulated feature samples that mimic real emotion features within a multimodal feature space. The generator, consisting of a series of fully connected layers and activation functions, generates high-dimensional feature vectors using random noise. The discriminator network is responsible for distinguishing between real and generated features. The discriminator's inputs are the features output by the capsule network and the simulated features generated by the generator. The discriminator uses a multi-layer perceptron (MLP) structure to perform binary classification on the input samples. The goal of GANs is to gradually improve the generator's generative power and the discriminator's discriminative power through a game of chess between the generator and the discriminator. During training, the generator and discriminator loss functions are alternately optimized: Discriminator loss: measures the ability to distinguish between real and generated samples. Generator loss: measures the ability of generated samples to be mistaken by the discriminator as real samples.

[0088] The processed multimodal data is input into the emotion recognition model to classify and identify the pet's emotional state, output the initial emotion classification result, and use the generative adversarial network to strengthen the emotion recognition process and generate the final emotion recognition result.

[0089] Specifically, the processed multimodal data is input into a capsule network to obtain preliminary emotion classification results. Each emotion category corresponds to an output vector of a high-level capsule. The vector magnitude is used to represent the probability of each emotion category, and the capsule with the largest magnitude is selected as the initial emotion classification result. The output of the capsule network and the samples generated by the generator are input into the discriminator to enhance emotion recognition through adversarial training. By continuously optimizing the generator and discriminator, the capsule network can generate more accurate emotion feature representations, improving the model's recognition capabilities. During adversarial training, the parameters of the capsule network are continuously adjusted to enhance its ability to distinguish different emotional states. Combined with feedback from the discriminator, the initial emotion classification results are revised to generate the final emotion recognition results. Based on the final emotion recognition results, the pet's emotional state is output, including happiness, excitement, anxiety, fear, and anger. Through multimodal information fusion, the accuracy and reliability of emotion recognition are improved. The emotion recognition results are presented to the pet owner via a smart device. The changes in the pet's emotional state are displayed through charts or visual interfaces, and corresponding suggestions and measures are provided.

[0090] Furthermore, the formula of the emotion recognition model is as follows:

[0091]

[0092] in, represents the total loss function, which combines the losses of the capsule network and the generative adversarial network to optimize the emotion recognition model; α represents the weight factor used to control the influence of the capsule network feature reconstruction loss; β represents the weight factor used to control the influence of the generative adversarial network loss; v j represents the output vector of the capsule network, i.e., the key features of pet emotions extracted; t j represents the target emotion feature vector, which is used to guide the accuracy of the output features of the capsule network; D(x) represents the output probability of the discriminator for the real data x; D(G(z)) represents the output probability of the discriminator for the generated data (G(z)); x represents the multimodal data sample; z represents the noise vector sampled from the latent space, which is used to generate the potential emotion features.

[0093] The formula of the capsule network is as follows:

[0094]

[0095] Among them, v j Represents the final recognized emotion feature, the output vector of the capsule network; s j The sum of the input vectors of the jth emotion node is obtained by aggregating the key features output by the feature extraction module:

[0096]

[0097] Among them, c ij represents the dynamic routing coefficient, reflecting the correlation between the key features output by the feature extraction module and the emotion node j; It represents the prediction vector of the i-th key feature extracted from the feature extraction module for the j-th emotion node. This feature includes facial expression features, sound spectrum features, physiological index change features and movement behavior features, and is obtained by calculation:

[0098]

[0099] Among them, W ij Represents the weight matrix from feature to emotion node; u i Represents the i-th key feature vector output by the feature extraction module.

[0100] The emotional dynamics tracking module is used to represent the pet's emotional state as nodes through a graph neural network, and to track and predict emotional change trends by learning the relationships between nodes and graph structure features;

[0101] The operation process of the emotion dynamics tracking module includes the following steps:

[0102] The emotion recognition results generated by the pet emotion recognition module are represented as nodes of a graph neural network, where each node corresponds to an emotional state.

[0103] It should be noted that nodes: each node represents the emotional state of a pet at a specific point in time.

[0104] Edges: Edges represent the transformation relationship between emotional states, and the weight of the edge reflects the probability or similarity of the transformation.

[0105] Use graph neural networks to build a relationship graph between emotional states. The edges in the graph represent the transformation relationships between emotional states, and the weights of the edges reflect the probability or similarity of the transformation.

[0106] Specifically, the emotion recognition results received from the pet emotion recognition module. Each recognition result contains an emotional state and its corresponding feature vector, such as the probability distribution of the emotion category. Each recognition result constitutes a node. The node feature vector includes the emotion category (such as happiness, excitement, anxiety, fear, anger) and its probability. The initial value of the node feature can be the vector representation in the emotion recognition result, including emotion probability, facial expression features, sound spectrum features, etc. Each node is accompanied by a timestamp to represent time series information in the graph. A directed graph is constructed by analyzing the sequence of emotion recognition results, in which the node represents the emotional state and the edge represents the transition of the emotional state. The weight of the edge is calculated based on the transition probability or state similarity. Common calculation methods include cosine similarity or Euclidean distance. According to the historical emotional state sequence, the frequency of transition from one emotional state to another is counted to calculate the transition probability. The similarity between the feature vectors of two nodes is measured using cosine similarity or Euclidean distance as the weight of the edge.

[0107] Based on the current emotional state and its adjacent emotional states, the features of each node are updated through a graph neural network to capture the mutual influence and dynamic changes between emotions;

[0108] Perform time series analysis on the features of nodes in the graph neural network to predict future trends in pet emotions and generate emotional state sequences;

[0109] Based on the emotional state sequence, potential emotional abnormalities are identified, and an automatic alarm mechanism for emotional changes is implemented by setting thresholds;

[0110] Specifically, anomaly detection algorithms (such as density-based clustering and isolation forest algorithms) are used to identify anomalies in emotional state sequences. Anomaly detection is based on the suddenness and inconsistency of emotional state changes. A threshold for emotional change is set, and an alarm is triggered when it exceeds this threshold. Alarm messages are generated based on the identified abnormal emotional states, notifying pet owners to take appropriate measures.

[0111] Automatic alarm process: define the normal range of emotional changes and the abnormal detection threshold; perform anomaly detection on the predicted emotional state sequence; when abnormal emotional changes are detected, automatically trigger an alarm notification.

[0112] The predicted emotion change trend and current emotion state are passed to the interactive feedback module.

[0113] Furthermore, the formula of the graph neural network is as follows:

[0114]

[0115] in, Represents the feature vector of node i at time step t, that is, the emotional state characteristics of the pet at that moment; represents the updated feature vector of node i at time step t+1; represents the set of neighbor nodes of node i, that is, other emotional states directly related to the current emotional state; c ij represents the square root of the number of connections or degree between nodes i and j; W represents the influence weight between emotional states; W0 represents the influence of the emotional state itself; σ represents the activation function.

[0116] The interactive feedback module is used to generate emotion management strategies and interaction suggestions through a reinforcement learning algorithm based on the results of emotion recognition and dynamic tracking, and interact with pet owners through smart devices.

[0117] The operation process of the interactive feedback module includes the following steps:

[0118] The pet's current emotional state and predicted emotional change trends are communicated to the pet owner in real time via smart devices, including smartphones, tablets, smart speakers, and smart home devices.

[0119] It should be noted that the results of emotion recognition and dynamic tracking are transmitted to smart devices via Wi-Fi or Bluetooth. Smart devices include smartphones, tablets, smart speakers and smart home devices. Using mobile applications (APP) or smart home platforms, the pet's current emotional state and predicted emotional change trends are updated in real time. The APP interface provides a graphical display, including historical records of emotional states, real-time updates and trend forecasts. When the emotional state changes significantly or an abnormality is detected, the pet owner is notified through push notifications or voice alarms. Provide quick response suggestions for emergencies, such as soothing measures or professional help.

[0120] Based on the identified emotional state, a reinforcement learning algorithm is used to generate personalized emotional management strategies, provide behavioral suggestions and environmental adjustment plans;

[0121] Specifically, select an appropriate reinforcement learning algorithm, such as the Deep Q-Network (DQN), the Advantaged Actor-Critic (A3C), or the Deep Deterministic Policy Gradient (DDPG). Use historical emotional state data for offline training to simulate different emotional change scenarios and optimize the emotion management strategy. In actual operation, utilize online learning to dynamically update the strategy to adapt to changes in the environment and emotions. Regularly evaluate the effectiveness of the strategy and adjust algorithm parameters and strategy selection based on feedback. Combined with online learning results, optimize the strategy to improve the effectiveness of emotion management.

[0122] Generate interaction suggestions and management strategies based on the pet's emotional state, including play activities, rest schedules, and emotional communication;

[0123] Specifically, appropriate behavioral activities are recommended based on the current emotional state. For example, when a pet is anxious, relaxation training or massage is recommended. Personalized training plans and game plans are provided to help pets release excess energy. Environmental conditions are adjusted based on environmental monitoring data (such as temperature and humidity). For example, when a pet feels hot, it is recommended to adjust the room temperature or provide a cool rest area. Use smart home devices to achieve automated environmental adjustments, such as adjusting smart lights or air conditioning. Play specific music or sounds through smart speakers to soothe pets' emotions. Provide suggestions for interacting with owners, such as increasing companionship time or having intimate communication.

[0124] By analyzing and collecting user feedback, the emotion recognition model and interaction strategy are dynamically adjusted and optimized.

[0125] Specifically, users are collected to provide feedback on emotion management strategies and interaction suggestions. This feedback can be in the form of ratings, comments, or direct communication. This feedback data is analyzed to identify the strengths and weaknesses of the strategies. Based on user feedback and emotion recognition results, emotion recognition model parameters are adjusted to enhance model accuracy. The parameter configuration of the reinforcement learning algorithm is optimized to improve the effectiveness and adaptability of the strategies.

[0126] In summary, the present invention utilizes multiple sensors (such as video, sound, body temperature, heart rate, GPS, and accelerometers) to collect multimodal data on pets, which can comprehensively capture the pet's behavior and physiological state. This comprehensive data collection can improve the accuracy of emotion recognition and ensure that the system can understand the pet's emotional changes from multiple perspectives. Through sparse coding and dictionary learning algorithms, key features of pets, such as facial expressions, sound spectra, physiological changes, and movement behaviors, are extracted. This feature extraction method can effectively reduce data redundancy, retain the most representative features for emotion recognition, and improve the recognition and efficiency of the model.

[0127] This paper uses capsule networks and generative adversarial networks (GANs) to capture the hierarchical relationships and spatial information between features, improving the ability to understand complex emotional features. GAN further enhances the generalization performance of the model through adversarial training of the generative and discriminative networks, making it more effective in identifying complex emotions in pets (such as happiness, excitement, anxiety, fear, and anger). Emotional states are represented as nodes, and by learning the relationships between nodes and graph structure features, it is possible to track and predict emotional trends. GNN can identify the mutual influence and dynamic changes between emotional states, thereby providing more accurate emotion prediction results.

[0128] This system combines emotion recognition and dynamic tracking results, conveying them to pet owners in real time via smart devices. The system utilizes reinforcement learning algorithms to generate personalized emotion management strategies, providing targeted behavioral recommendations and environmental adjustments to help owners better manage their pets' emotions. By collecting and analyzing user feedback, the system dynamically adjusts and optimizes emotion recognition models and interaction strategies, ensuring they continually adapt to the changing needs of pets and owners.

[0129] The above embodiments are merely descriptions of preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary engineering technicians in this field should fall within the scope of protection determined by the claims of the present invention.

Claims

1. An AI-based pet emotion recognition system, characterized in that: It includes a pet data acquisition module, a feature extraction module, a pet emotion recognition module, an emotion dynamic tracking module and an interactive feedback module which are sequentially connected in communication; The pet data acquisition module is used to collect multimodal data of the pet through a sensor network; the multimodal data includes video images, sounds, body temperature, heart rate, position and activity level of the pet; The feature extraction module is used to receive the multimodal data of the pet and extract key features through sparse coding and dictionary learning algorithms; the key features include the pet's facial expression features, sound spectrum features, physiological index change features and movement behavior features; The pet emotion recognition module is used to recognize and classify pet emotions, including happiness, excitement, anxiety, fear, and anger, through a capsule network algorithm and a generative adversarial network. The emotional dynamics tracking module is used to represent the pet's emotional state as nodes through a graph neural network, and to track and predict emotional change trends by learning the relationships between nodes and graph structure features; The interactive feedback module is used to generate emotion management strategies and interaction suggestions through reinforcement learning algorithms based on the results of emotion recognition and dynamic tracking, and interact with pet owners through smart devices; The operation process of the pet emotion recognition module includes the following steps: receiving the key features from the feature extraction module; Based on the key features, an emotion recognition model is constructed using a capsule network, and model training is performed in combination with a generative adversarial network. The capsule network is used to capture the hierarchical relationship and spatial information between features, and a dynamic routing mechanism is used to improve the model's ability to understand complex emotion features. The processed multimodal data is input into the capsule network to obtain preliminary emotion classification results. The vector modulus is used to represent the probability of each emotion category, and the capsule with the largest modulus is selected as the initial emotion classification result. The output of the capsule network and the samples generated by the generator are input into the discriminator to enhance emotion recognition through adversarial training. By continuously optimizing the generator and discriminator, the parameters of the capsule network are continuously adjusted to enhance its ability to distinguish different emotional states. The initial emotion classification results are corrected based on the feedback information of the discriminator to generate the final emotion recognition result. Based on the final emotion recognition result, the emotional state of the pet is output, including happiness, excitement, anxiety, fear and anger. The formula of the emotion recognition model is as follows: ； in, represents the total loss function, which combines the losses of capsule network and generative adversarial network and is used to optimize the emotion recognition model; Represents the weight factor, which is used to control the impact of capsule network feature reconstruction loss; represents the weight factor used to control the impact of the generative adversarial network loss; represents the output vector of the capsule network, i.e., the key features of pet emotions extracted; Represents the target emotion feature vector, which is used to guide the accuracy of the capsule network output features; Represents the output probability of the discriminator for the real data x; represents the output probability of the discriminator for the generated data (G(z)); x represents the multimodal data sample; z represents the noise vector sampled from the latent space for generating potential emotion features; The operation process of the emotion dynamics tracking module includes the following steps: The emotion recognition results generated by the pet emotion recognition module are represented as nodes of a graph neural network, where each node represents the emotional state of the pet at a specific time point. A graph neural network is used to construct a relationship graph between emotional states. The edges of the graph represent the transformation relationships between emotional states, and the weights of the edges reflect the probability or similarity of the transformations. Specifically, the emotion recognition results received from the pet emotion recognition module each contain an emotional state and its corresponding feature vector. Each recognition result constitutes a node, and the node feature vector includes the emotion category and its probability. The initial value of the node feature is the vector representation in the emotion recognition result, including emotion probability, facial expression features, and sound spectrum features. A directed graph is constructed by analyzing the sequence of emotion recognition results, in which nodes represent emotional states and edges represent the transition of emotional states. The weights of the edges are calculated based on the transition probability or state similarity. Based on the historical emotional state sequence, the frequency of transitions from one emotional state to another is counted to calculate the transition probability. The similarity between the feature vectors of two nodes is measured using cosine similarity or Euclidean distance as the weight of the edge. Based on the current emotional state and its adjacent emotional states, the features of each node are updated through a graph neural network to capture the mutual influence and dynamic changes between emotions; Perform time series analysis on the features of nodes in the graph neural network to predict future trends in pet emotions and generate emotional state sequences; Based on the emotional state sequence, potential emotional abnormalities are identified, and an automatic alarm mechanism for emotional changes is implemented by setting thresholds; Pass the predicted emotion change trend and current emotion state to the interactive feedback module; The formula of the graph neural network is as follows: ； in, Represents the feature vector of node j at time step t, that is, the emotional state feature of the pet at time step t; represents the updated feature vector of node i at time step t+1; represents the set of neighbor nodes of node i, that is, other emotional states directly related to the current emotional state; represents the square root of the number of connections or degree between nodes i and j; Indicates the influence weight between emotional states; Indicates the impact of the emotional state itself; Represents the activation function.

2. The AI-based pet emotion recognition system according to claim 1, characterized in that: The operation process of the feature extraction module includes the following steps: Performing format conversion and normalization processing on the multimodal data, including grayscale conversion of video images, frequency domain analysis of sound signals, and filtering and smoothing of body temperature and heart rate; Use sparse coding algorithm to encode the features of preprocessed data, construct the initial feature matrix, and use sparse constrained optimization algorithm to select key technical features; A dictionary learning algorithm is used to train the feature matrix and generate a feature dictionary suitable for pet emotion recognition; The extracted features are selected and combined, and the feature selection algorithm is used to construct the final key feature set based on the importance and relevance of the features, which provides input for the emotion recognition module.

3. The AI-based pet emotion recognition system according to claim 2, characterized in that: The formula of the sparse coding algorithm is as follows: ； in, represents the optimal sparse representation, i.e., the key features extracted from the multimodal data; y represents the feature vector of the preprocessed multimodal data; D represents the basis vector matrix of the feature space, i.e., the combination of all features in emotion recognition; x represents the sparse representation of the input data in the dictionary, i.e., the key feature combination corresponding to the pet's emotion; It means finding the parameter x that minimizes the objective function; Represents the reconstruction error term, which represents the square error between the dictionary matrix D reconstructed data through sparse representation x and the original input data y. If It is very close to y, indicating that the extracted sparse features fully describe the pet’s current emotion; It indicates that the sparsity constraint term controls the sparsity degree through λ.

4. The AI-based pet emotion recognition system according to claim 1, characterized in that: The formula of the capsule network is as follows: in, Represents the final recognized emotion feature, the output vector of the capsule network; Represents the sum of the input vectors of the j-th emotion node.

5. The AI-based pet emotion recognition system according to claim 1, characterized in that: The operation process of the interactive feedback module includes the following steps: The pet's current emotional state and predicted emotional change trends are communicated to the pet owner in real time via smart devices, including smartphones, tablets, smart speakers, and smart home devices. Based on the identified emotional state, a reinforcement learning algorithm is used to generate personalized emotional management strategies, provide behavioral suggestions and environmental adjustment plans; Generate interaction suggestions and management strategies based on the pet's emotional state, including play activities, rest schedules, and emotional communication; By analyzing and collecting user feedback, the emotion recognition model and interaction strategy are dynamically adjusted and optimized.

Images