AI-based pet emotion recognition system
Through the AI-based pet emotion recognition system, using multimodal data and advanced algorithms, accurate identification and dynamic tracking of pet emotions are achieved, solving the problems of insufficient identification accuracy and inability to track emotional changes in the existing technology, and making pet health and behavior management provides effective support.
Patent Information
- Application Number
- CN202411115983.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-14
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2044-08-14
AI Technical Summary
The lack of comprehensive collection and utilization of multimodal data in pet emotions recognition in the prior art has led to insufficient identification accuracy and comprehensiveness, inability to effectively handle the diversity and complexity of pet emotions, and inability to track and predict dynamic changes in emotions, It is difficult to provide effective support for pet health and behavior management.
Using an AI-based pet emotion recognition system, multimodal data is collected through sensor networks, such as video images, sounds, body temperature, heart rate and location, and using sparse encoding and dictionary learning algorithms to extract key features, combining capsule networks and generative adversarial networks to perform emotions. Identify, graph neural networks perform dynamic emotions tracking and generate emotional management strategies through reinforcement learning algorithms.
Accurate identification and dynamic tracking of pet emotions are achieved, the accuracy and comprehensiveness of recognition are improved, and more effective support for pet health and behavior management is provided, and emotional fluctuations can be identified in a timely manner and personalized management strategies are generated.
Smart Images

Figure CN119049086A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of pet emotion recognition, and in particular to an AI-based pet emotion recognition system. Background Art
[0002] In today's society, pets are increasingly important as family members, and pet owners are paying more and more attention to their emotions and health status. However, since pets cannot express emotions directly through language, owners usually rely on their behavior and physiological characteristics to make judgments, which is easily affected by subjective factors and is not accurate enough. At present, pet emotion recognition mainly relies on the observation and judgment of experienced veterinarians or pet trainers. However, this method is not only highly subjective, but also requires a lot of professional knowledge and experience. There are still the following problems: existing technologies often rely on only a single or limited type of data to judge pet emotions, lack of comprehensive collection and utilization of multimodal data, resulting in insufficient accuracy and comprehensiveness of emotion recognition; existing technologies often rely on simple classification algorithms, which cannot effectively handle the diversity and complexity of pet emotions, and have low recognition accuracy; existing technologies can usually only perform static emotion judgments, cannot track and predict the dynamic changes of pet emotions, cannot identify emotional fluctuations in a timely manner, and are difficult to provide effective support for the long-term health and behavior management of pets. Summary of the invention
[0003] To solve the above problems, the present invention provides an AI-based pet emotion recognition system, which solves the problem of how to accurately identify, dynamically track and predict pet emotions through multimodal data and advanced algorithms, thereby improving the accuracy and comprehensiveness of pet emotion recognition and providing more effective support for pet health and behavior management.
[0004] To achieve the above object, the technical solution adopted by the present invention is:
[0005] An AI-based pet emotion recognition system comprises a pet data acquisition module, a feature extraction module, a pet emotion recognition module, an emotion dynamic tracking module and an interactive feedback module which are sequentially connected in communication;
[0006] The pet data collection module is used to collect multimodal data of the pet through a sensor network; the multimodal data includes video images, sounds, body temperature, heart rate, position and activity of the pet;
[0007] The feature extraction module is used to receive the multimodal data of the pet and extract key features through sparse coding and dictionary learning algorithms; the key features include facial expression features, sound spectrum features, physiological index change features and movement behavior features of the pet;
[0008] The pet emotion recognition module is used to recognize and classify pet emotions, including happiness, excitement, anxiety, fear and anger, through a capsule network algorithm and a generative adversarial network.
[0009] The emotional dynamics tracking module is used to represent the emotional state of the pet as nodes through a graph neural network, and to track and predict the trend of emotional changes by learning the relationship between nodes and graph structure features;
[0010] The interactive feedback module is used to generate emotion management strategies and interactive suggestions through a reinforcement learning algorithm based on the results of emotion recognition and dynamic tracking, and interact with pet owners through smart devices.
[0011] Furthermore, the operation process of the feature extraction module includes the following steps:
[0012] The multimodal data is formatted and normalized, including grayscale processing of video images, frequency domain analysis of sound signals, and filtering and smoothing processing of body temperature and heart rate;
[0013] Use sparse coding algorithm to encode the features of preprocessed data, construct the initial feature matrix, and use sparse constrained optimization algorithm to select key technical features;
[0014] Using dictionary learning algorithm, the feature matrix is trained to generate a feature dictionary suitable for pet emotion recognition;
[0015] The extracted features are selected and combined, and the feature selection algorithm is used to construct the final key feature set based on the importance and relevance of the features, which provides input for the emotion recognition module.
[0016] Furthermore, the formula of the sparse coding algorithm is as follows:
[0017]
[0018] Among them, x * represents the optimal sparse representation, i.e., the key features extracted from the multimodal data; y represents the feature vector of the preprocessed multimodal data; D represents the basis vector matrix of the feature space, i.e., the combination of all features in emotion recognition; x represents the sparse representation of the input data in the dictionary, i.e., the key feature combination corresponding to the pet's emotions; argmin x It means finding the parameter x that minimizes the objective function; The reconstruction error term represents the square error between the dictionary matrix D reconstructed data through the sparse representation x and the original input data y. If Dx is very close to y, it means that the extracted sparse features fully describe the current emotion of the pet; λ||x||1 represents the sparsity constraint term that controls the sparsity degree through λ.
[0019] Furthermore, the operation process of the pet emotion recognition module includes the following steps:
[0020] receiving the key features from the feature extraction module;
[0021] Based on the key features, an emotion recognition model is constructed through a capsule network, and the model is trained in combination with a generative adversarial network; the capsule network is used to capture the hierarchical relationship and spatial information between features, and the model's ability to understand complex emotion features is improved through a dynamic routing mechanism;
[0022] The processed multimodal data is input into the emotion recognition model to classify and identify the emotional state of the pet, output the initial emotion classification result, and use the generative adversarial network to strengthen the emotion recognition process and generate the final emotion recognition result.
[0023] Furthermore, the formula of the emotion recognition model is as follows:
[0024]
[0025] in, represents the total loss function, which combines the losses of the capsule network and the generative adversarial network and is used to optimize the emotion recognition model; α represents the weight factor, which is used to control the influence of the capsule network feature reconstruction loss; β represents the weight factor, which is used to control the influence of the generative adversarial network loss; v j represents the output vector of the capsule network, i.e., the key features of pet emotions extracted; t j represents the target emotion feature vector, which is used to guide the accuracy of the output features of the capsule network; D(x) represents the output probability of the discriminator for the real data x; D(G(z)) represents the output probability of the discriminator for the generated data (G(z)); x represents a multimodal data sample; z represents a noise vector sampled from the latent space, which is used to generate potential emotion features.
[0026] Furthermore, the formula of the capsule network is as follows:
[0027]
[0028] Among them, v j Represents the final recognized emotion feature, the output vector of the capsule network; s j Represents the sum of the input vectors of the jth emotion node.
[0029] Furthermore, the operation process of the emotion dynamics tracking module includes the following steps:
[0030] The emotion recognition results generated by the pet emotion recognition module are represented as nodes of the graph neural network, where each node corresponds to an emotional state;
[0031] Use graph neural networks to build a relationship graph between emotional states. The edges in the graph represent the transformation relationships between emotional states, and the weights of the edges reflect the probability or similarity of the transformation.
[0032] Based on the current emotional state and its adjacent emotional states, the features of each node are updated through the graph neural network to capture the mutual influence and dynamic changes between emotions;
[0033] Perform time series analysis on the features of nodes in the graph neural network to predict the future trend of pet emotions and generate emotional state sequences;
[0034] According to the emotional state sequence, potential emotional abnormalities are identified, and an automatic alarm mechanism for emotional changes is implemented by setting thresholds;
[0035] The predicted emotion change trend and current emotion state are passed to the interactive feedback module.
[0036] Furthermore, the formula of the graph neural network is as follows:
[0037]
[0038] in, Represents the feature vector of node i at time step t, that is, the emotional state characteristics of the pet at that moment; represents the updated feature vector of node i at time step t+1; represents the set of neighbor nodes of node i, that is, other emotional states directly related to the current emotional state; c ij represents the number of connections or the square root of the degree between nodes i and j; W represents the influence weight between emotional states; W0 represents the influence of the emotional state itself; σ represents the activation function.
[0039] Furthermore, the operation process of the interactive feedback module includes the following steps:
[0040] The pet's current emotional state and predicted emotional change trends are communicated to the pet owner in real time via smart devices; the smart devices include smart phones, tablet computers, smart speakers, and smart home devices;
[0041] Based on the identified emotional state, a reinforcement learning algorithm is used to generate personalized emotion management strategies, provide behavioral suggestions and environmental adjustment plans;
[0042] Generate interaction suggestions and management strategies based on the pet’s emotional state, including play activities, rest schedules, and emotional communication;
[0043] By analyzing and collecting user feedback, the emotion recognition model and interaction strategy can be dynamically adjusted and optimized.
[0044] The beneficial effects of the present invention are:
[0045] The present invention uses a pet data acquisition module and a sensor network to obtain multimodal data such as video images, sounds, body temperature, heart rate, position and activity of pets, providing rich background information and details, which helps to fully understand the status of pets. The feature extraction module uses sparse coding and dictionary learning algorithms to extract key features from multimodal data, including facial expressions, sound spectra, physiological indicators and movement behavior characteristics. This method can effectively extract core information related to emotions and ensure the accuracy and effectiveness of recognition. The pet emotion recognition module uses capsule network algorithms and generative adversarial networks to identify and classify pet emotions, and can accurately distinguish various emotional states of pets such as happiness, excitement, anxiety, fear and anger, providing more accurate emotional understanding. The emotional dynamic tracking module uses a graph neural network to represent emotional states as nodes, and by learning the relationship between nodes and graph structure features, it can track and predict emotional change trends in real time. This function can identify emotional fluctuations in a timely manner and provide strong support for pet health and behavior management. The interactive feedback module generates personalized emotion management strategies and interactive suggestions based on the results of emotion recognition and dynamic tracking using a reinforcement learning algorithm, and interacts with pet owners through smart devices to help owners better understand and respond to their pets’ needs and improve the human-pet relationship. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] Figure 1 It is a module schematic diagram of an AI-based pet emotion recognition system of the present invention.
[0047] Figure 2 It is a flowchart of the operation process of the pet emotion recognition module provided by one embodiment of the present invention.
[0048] Figure 3 It is a flowchart of the operation process of the emotion dynamics tracking module provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0049] See also Figure 1-3 As shown, the present invention relates to an AI-based pet emotion recognition system.
[0050] Example
[0051] An AI-based pet emotion recognition system comprises a pet data acquisition module, a feature extraction module, a pet emotion recognition module, an emotion dynamic tracking module and an interactive feedback module which are sequentially connected in communication;
[0052] The pet data collection module is used to collect multimodal data of the pet through a sensor network; the multimodal data includes video images, sounds, body temperature, heart rate, position and activity of the pet;
[0053] It should be noted that the pet data acquisition module is composed of a variety of sensors, forming a complex sensor network to ensure comprehensive and accurate pet status monitoring. Each sensor is responsible for different types of data collection. The following is a detailed description of each sensor:
[0054] Video image sensor: A camera placed in the pet's activity area to capture the pet's facial expressions and body postures. The camera should have night vision to ensure clear images under various lighting conditions. The camera should be placed in places where pets often move, such as pet beds, food areas, play areas, etc., to ensure that the pet's behavior can be captured from multiple angles. The image data collected by the camera is initially processed by the edge computing device, such as image denoising and brightness adjustment.
[0055] Sound sensor: Installed on the pet's collar or around the pet's activity area, it captures the pet's voice, breathing and other sound characteristics. The sound data is pre-processed by the front-end device, including denoising, gain adjustment and sampling rate conversion.
[0056] Physiological sensor: A wearable device that integrates multiple sensors and is worn around the pet's neck to monitor the pet's physiological indicators in real time. Body temperature sensor: Monitors the pet's body temperature to help determine the pet's health and emotional state. Heart rate sensor: Monitors the pet's heart rate changes through photoplethysmography (PPG) or electrocardiogram (ECG) technology. Respiration sensor: Monitors the pet's breathing rate and pattern.
[0057] GPS module: The GPS module built into the smart collar is used to record the pet's geographic location and activity path in real time. The location data is transmitted to the central processing unit via a wireless network and combined with other sensor data for comprehensive analysis.
[0058] Accelerometer: The three-axis accelerometer built into the smart collar is used to detect the pet's activity and movement patterns, including walking, running, jumping, etc. By extracting and analyzing the acceleration data, the pet's behavior pattern and activity intensity can be identified.
[0059] All sensor data is transmitted to the central processing unit via Bluetooth Low Energy (BLE), Wi-Fi, or other wireless communication technologies. For long-distance transmission, cellular networks may be used. Some data is processed by edge computing on local devices to reduce transmission burden and latency. After receiving the data, it is stored in a distributed database to ensure the security and reliable storage of the data. The data storage system should support efficient read and write operations and backup and recovery mechanisms.
[0060] All sensor data is encrypted during transmission to protect the confidentiality and integrity of the data. Common encryption methods include AES (Advanced Encryption Standard) and TLS (Transport Layer Security). The system should be designed with strict access control mechanisms to ensure that only authorized users and devices can access and process pet data. Comply with data privacy protection regulations (such as GDPR) to protect the privacy of users and pets during data collection and processing. The system should provide functions for user data management and deletion.
[0061] The feature extraction module is used to receive the multimodal data of the pet and extract key features through sparse coding and dictionary learning algorithms; the key features include facial expression features, sound spectrum features, physiological index change features and movement behavior features of the pet;
[0062] The operation process of the feature extraction module includes the following steps:
[0063] The multimodal data is formatted and normalized, including grayscale processing of video images, frequency domain analysis of sound signals, and filtering and smoothing processing of body temperature and heart rate;
[0064] In one embodiment, the video image processing is as follows:
[0065] Grayscale processing: Use YUV or HSV color space conversion to convert RGB images into grayscale images. Grayscale can simplify calculations and reduce the interference of noise on feature extraction. In the grayscale process, local contrast enhancement technology is used to enhance the details of the image.
[0066] Image denoising and smoothing: Apply a Gaussian filter to remove high-frequency noise from the image. The Gaussian filter parameters can be adaptively adjusted based on the image noise level. Use a bilateral filter to denoise while retaining the edge information of the image.
[0067] Scale normalization: Resize images of different sizes to a uniform size (e.g., 224x224 pixels) to fit the subsequent feature extraction network. Use bilinear interpolation to scale the image to maintain image clarity and proportion.
[0068] The sound signal is processed as follows:
[0069] Frequency domain analysis: The sound signal in the time domain is converted into a spectrogram using the Fast Fourier Transform (FFT). The frequency domain representation is obtained by calculating the Short Time Fourier Transform (STFT) to analyze the frequency characteristics of the sound. To enhance the readability of the spectrogram, the Mel Frequency Cepstral Coefficients (MFCC) are used to extract the sound features. MFCCs can effectively represent the spectral shape of the sound signal and are suitable for the analysis of emotional characteristics.
[0070] De-noising: Use spectral subtraction to remove background noise. Improve the signal-to-noise ratio by estimating the noise spectrum and subtracting it from the signal. Implement adaptive filtering techniques (such as Wiener filtering) to dynamically adjust filter parameters to adapt to changes in environmental noise.
[0071] Physiological indicators are processed as follows:
[0072] Temperature and heart rate filtering: Kalman filter is used for data smoothing to eliminate sensor noise and occasional measurement errors. Kalman filter predicts the current state through dynamic model and makes optimal estimation based on observed values. Peak detection and RR interval analysis are performed on heart rate signal to extract emotion-related features such as heart rate variability (HRV).
[0073] Trend analysis: Apply linear regression to analyze the time series trends of heart rate and body temperature to identify potential abnormal fluctuations. Use time series decomposition methods (such as STL decomposition) to separate seasonality, trend and residual components and deeply analyze changes in physiological indicators.
[0074] Use sparse coding algorithm to encode the features of preprocessed data, construct the initial feature matrix, and use sparse constrained optimization algorithm to select key technical features;
[0075] Specifically, a sparse autoencoder is used to construct a three-layer network structure, in which the input layer corresponds to the original features, and the sparse hidden layer in the middle maintains sparsity through regularization constraints (L1 norm). Sparse coding extracts sparse features with high recognition by solving the sparse solution so that the error is minimized when the output layer reconstructs the input layer features. The sparse representation result is constructed as the initial feature matrix. Each column of the feature matrix represents a sparse feature vector, and each row corresponds to a sample. Dimensionality reduction techniques (such as PCA or t-SN E) are used to compress the feature matrix to reduce redundant dimensions and enhance the distinguishing ability of features. The orthogonal matching pursuit (OMP) algorithm is used to solve the sparse coding problem. OMP gradually constructs a sparse solution by iteratively selecting the features most relevant to the residual error. The alternating direction method of multipliers (ADMM) is applied for sparse optimization to accelerate the process of solving the sparse solution.
[0076] Using dictionary learning algorithm, the feature matrix is trained to generate a feature dictionary suitable for pet emotion recognition;
[0077] Specifically, samples are randomly selected from the sparse feature matrix as the initial dictionary for dictionary learning. The initial dictionary should cover the feature distribution of different emotional states to improve the generalization ability of the model. The dictionary is updated using the K-SVD algorithm. K-SVD continuously optimizes the quality of the dictionary by alternately minimizing the sparse coding and dictionary update errors. In each iteration, K-SVD updates the dictionary using singular value decomposition to improve its ability to reconstruct features. Through the sparse representation characteristics of the dictionary, the input features are sparsely reconstructed to ensure that the dictionary can efficiently represent the structure of the input data. The reconstruction error of the dictionary for the input data is calculated, and the reconstruction ability of the dictionary is measured by the mean square error (MSE). The reconstruction effects of different dictionaries are compared to select the optimal feature dictionary. The sparse representation ability of the dictionary is analyzed by the sparse representation coefficient to ensure that the dictionary can effectively distinguish different emotion categories.
[0078] The extracted features are selected and combined, and the feature selection algorithm is used to construct the final key feature set based on the importance and relevance of the features, which provides input for the emotion recognition module.
[0079] Furthermore, the formula of the sparse coding algorithm is as follows:
[0080]
[0081] Among them, x * represents the optimal sparse representation, that is, the key features extracted from multimodal data; y represents the feature vector of the preprocessed multimodal data, which provides multimodal information of the pet's current state and provides a comprehensive data basis for emotion analysis; D represents the basis vector matrix of the feature space, that is, the combination of all features in emotion recognition. The richness and accuracy of the dictionary matrix directly affect the effect of sparse coding. The more accurately the dictionary can express emotional features, the more helpful it is to accurately identify emotions; x represents the sparse representation of the input data in the dictionary, that is, the key feature combination corresponding to the pet's emotions. Sparsity ensures the efficiency of the system and reduces the computational complexity by selectively focusing on the most representative features; arg min x It means finding the parameter x that minimizes the objective function; The reconstruction error term represents the square error between the dictionary matrix D reconstructed data through the sparse representation x and the original input data y. If Dx is very close to y, it means that the extracted sparse features fully describe the current emotion of the pet; λ||x||1 represents the sparsity constraint term that controls the sparsity degree through λ.
[0082] The pet emotion recognition module is used to recognize and classify pet emotions, including happiness, excitement, anxiety, fear and anger, through a capsule network algorithm and a generative adversarial network.
[0083] The operation process of the pet emotion recognition module includes the following steps:
[0084] receiving the key features from the feature extraction module;
[0085] Based on the key features, an emotion recognition model is constructed through a capsule network, and the model is trained in combination with a generative adversarial network; the capsule network is used to capture the hierarchical relationship and spatial information between features, and the model's ability to understand complex emotion features is improved through a dynamic routing mechanism;
[0086] Specifically, the input multimodal feature vector is converted into an initial capsule through a convolutional layer. Each capsule contains a feature vector, which represents a specific attribute or sub-feature of a feature. The initial capsule layer is responsible for capturing low-level feature information and performing preliminary spatial encoding. The initial capsule layer is connected to the advanced capsule layer through a dynamic routing mechanism. Dynamic routing selectively transmits information through weight updates to form advanced feature capsules. The advanced capsule layer encodes the hierarchical relationship between features and can represent the complex structure and pattern of emotions. A dynamic routing algorithm is used between capsules to dynamically adjust the routing coefficient. The output of each capsule is updated through iterative calculation to ensure that important features receive higher attention. The steps of the dynamic routing algorithm are: initialize the routing coefficient to an equal value; calculate the prediction vector of each initial capsule for the advanced capsule; update the routing coefficient to minimize the prediction error; and finally output the feature representation of the advanced capsule through iterative solution.
[0087] Generative adversarial networks are used to enhance the emotion recognition ability of capsule networks, and improve the generalization performance of the model through adversarial training between generated samples and real samples. The generator network generates simulated feature samples that simulate real emotion features in a multimodal feature space. The generator consists of a series of fully connected layers and activation functions, and generates high-dimensional feature vectors through random noise. The discriminator network is responsible for distinguishing between real features and generated features. The input of the discriminator is the features output from the capsule network and the simulated features generated by the generator. The discriminator performs binary classification on the input samples through a multi-layer perceptron (MLP) structure. The goal of GAN is to gradually improve the generation ability of the generator and the discrimination ability of the discriminator through the game between the generator and the discriminator. During the training process, the loss functions of the generator and the discriminator are optimized alternately: Discriminator loss: measures the ability to distinguish between real samples and generated samples. Generator loss: measures the ability of generated samples to be mistaken for real samples by the discriminator.
[0088] The processed multimodal data is input into the emotion recognition model to classify and identify the emotional state of the pet, output the initial emotion classification result, and use the generative adversarial network to strengthen the emotion recognition process and generate the final emotion recognition result.
[0089] Specifically, the processed multimodal data is input into the capsule network to obtain a preliminary emotion classification result. Each emotion category corresponds to an output vector of an advanced capsule. The vector magnitude is used to represent the probability of each emotion category, and the capsule with the largest magnitude is selected as the initial emotion classification result. The output of the capsule network and the samples generated by the generator are input into the discriminator, and emotion recognition is enhanced through adversarial training. By continuously optimizing the generator and the discriminator, the capsule network can generate more accurate emotion feature representations and improve the recognition ability of the model. During the adversarial training process, the parameters of the capsule network are continuously adjusted to enhance its ability to distinguish different emotional states. Combined with the feedback information of the discriminator, the initial emotion classification result is corrected to generate the final emotion recognition result. According to the final emotion recognition result, the emotional state of the pet is output, including happiness, excitement, anxiety, fear and anger. Through multimodal information fusion, the accuracy and reliability of emotion recognition are improved. The emotion recognition results are presented to the pet owner through a smart device. The changes in the emotional state of the pet are displayed through a chart or a visual interface, and corresponding suggestions and measures are provided.
[0090] Furthermore, the formula of the emotion recognition model is as follows:
[0091]
[0092] in, represents the total loss function, which combines the losses of the capsule network and the generative adversarial network and is used to optimize the emotion recognition model; α represents the weight factor, which is used to control the influence of the capsule network feature reconstruction loss; β represents the weight factor, which is used to control the influence of the generative adversarial network loss; v j represents the output vector of the capsule network, i.e., the key features of pet emotions extracted; t j represents the target emotion feature vector, which is used to guide the accuracy of the output features of the capsule network; D(x) represents the output probability of the discriminator for the real data x; D(G(z)) represents the output probability of the discriminator for the generated data (G(z)); x represents a multimodal data sample; z represents a noise vector sampled from the latent space, which is used to generate potential emotion features.
[0093] The formula of the capsule network is as follows:
[0094]
[0095] Among them, v j Represents the final recognized emotion feature, the output vector of the capsule network; s j The sum of the input vectors of the jth emotion node is obtained by aggregating the key features output by the feature extraction module:
[0096]
[0097] Among them, c ij represents the dynamic routing coefficient, reflecting the correlation between the key features output by the feature extraction module and the emotion node j; It represents the prediction vector of the i-th key feature extracted from the feature extraction module for the j-th emotion node. The feature includes facial expression features, sound spectrum features, physiological index change features and movement behavior features, which are obtained by calculation:
[0098]
[0099] Among them, W ij Represents the weight matrix from feature to emotion node; u i Represents the i-th key feature vector output by the feature extraction module.
[0100] The emotional dynamics tracking module is used to represent the emotional state of the pet as nodes through a graph neural network, and to track and predict the trend of emotional changes by learning the relationship between nodes and graph structure features;
[0101] The operation process of the emotion dynamics tracking module includes the following steps:
[0102] The emotion recognition results generated by the pet emotion recognition module are represented as nodes of the graph neural network, where each node corresponds to an emotional state;
[0103] It should be noted that nodes: each node represents the emotional state of a pet at a specific point in time.
[0104] Edges: Edges represent the transformation relationship between emotional states, and the weight of the edge reflects the probability or similarity of the transformation.
[0105] Use graph neural networks to build a relationship graph between emotional states. The edges in the graph represent the transformation relationships between emotional states, and the weights of the edges reflect the probability or similarity of the transformation.
[0106] Specifically, the emotion recognition results received from the pet emotion recognition module. Each recognition result contains an emotional state and its corresponding feature vector, such as the probability distribution of emotion categories. Each recognition result constitutes a node. The node feature vector includes emotion categories (such as happiness, excitement, anxiety, fear, anger) and their probabilities. The node feature initial value can be a vector representation in the emotion recognition result, including emotion probability, facial expression features, sound spectrum features, etc. Each node is accompanied by a timestamp for representing time series information in the graph. A directed graph is constructed by analyzing the sequence of emotion recognition results, in which the node represents the emotional state and the edge represents the transition of the emotional state. The weight of the edge is calculated based on the transition probability or state similarity. Common calculation methods include cosine similarity or Euclidean distance. According to the historical emotional state sequence, the frequency of transition from one emotional state to another is counted to calculate the transition probability. The similarity between two node feature vectors is measured using cosine similarity or Euclidean distance as the weight of the edge.
[0107] Based on the current emotional state and its adjacent emotional states, the features of each node are updated through the graph neural network to capture the mutual influence and dynamic changes between emotions;
[0108] Perform time series analysis on the features of nodes in the graph neural network to predict the future trend of pet emotions and generate emotional state sequences;
[0109] According to the emotional state sequence, potential emotional abnormalities are identified, and an automatic alarm mechanism for emotional changes is implemented by setting thresholds;
[0110] Specifically, anomaly detection algorithms (such as density-based clustering algorithms, isolation forests, etc.) are used to identify anomalies in the emotional state sequence. Anomaly detection is based on the suddenness and inconsistency of emotional state changes. A threshold for emotional changes is set, and an alarm is triggered when the threshold is exceeded. Alarm information is generated based on the identified abnormal emotional state, notifying the pet owner to take corresponding measures.
[0111] Automatic alarm process: define the normal range of emotional changes and the abnormal detection threshold; perform anomaly detection on the predicted emotional state sequence; when abnormal emotional changes are detected, automatically trigger an alarm notification.
[0112] The predicted emotion change trend and current emotion state are passed to the interactive feedback module.
[0113] Furthermore, the formula of the graph neural network is as follows:
[0114]
[0115] in, Represents the feature vector of node i at time step t, that is, the emotional state characteristics of the pet at that moment; represents the updated feature vector of node i at time step t+1; represents the set of neighbor nodes of node i, that is, other emotional states directly related to the current emotional state; c ij represents the number of connections or the square root of the degree between nodes i and j; W represents the influence weight between emotional states; W0 represents the influence of the emotional state itself; σ represents the activation function.
[0116] The interactive feedback module is used to generate emotion management strategies and interactive suggestions through a reinforcement learning algorithm based on the results of emotion recognition and dynamic tracking, and interact with pet owners through smart devices.
[0117] The operation process of the interactive feedback module includes the following steps:
[0118] The pet's current emotional state and predicted emotional change trends are communicated to the pet owner in real time via smart devices; the smart devices include smart phones, tablet computers, smart speakers, and smart home devices;
[0119] It should be noted that the results of emotion recognition and dynamic tracking are transmitted to smart devices via Wi-Fi or Bluetooth. Smart devices include smartphones, tablets, smart speakers and smart home devices. Use mobile applications (APP) or smart home platforms to update the pet's current emotional state and predicted emotional change trends in real time. The APP interface provides a graphical display, including historical records of emotional states, real-time updates and trend forecasts. When the emotional state changes significantly or an abnormality is detected, the pet owner is notified through push notifications or voice alarms. Provide quick response suggestions for emergencies, such as comfort measures or professional help.
[0120] Based on the identified emotional state, a reinforcement learning algorithm is used to generate personalized emotion management strategies, provide behavioral suggestions and environmental adjustment plans;
[0121] Specifically, select a suitable reinforcement learning algorithm, such as the Deep Q Network (DQN), the Advantaged Actor-Critic (A3C) algorithm, or the Deep Deterministic Policy Gradient (DDPG) algorithm. Use historical emotional state data for offline training, simulate different emotional change scenarios, and optimize emotional management strategies. In actual operation, use online learning to dynamically update strategies to adapt to changes in the environment and emotions. Regularly evaluate the effectiveness of strategies, and adjust algorithm parameters and strategy selection based on feedback. Combined with online learning results, optimize strategies to improve the effectiveness of emotional management.
[0122] Generate interaction suggestions and management strategies based on the pet’s emotional state, including play activities, rest schedules, and emotional communication;
[0123] Specifically, appropriate behavioral activities are recommended based on the current emotional state. For example, when the pet is anxious, relaxation training or massage is recommended. Personalized training plans and game plans are provided to help pets release excess energy. Environmental conditions are adjusted based on environmental monitoring data (such as temperature and humidity). For example, when the pet feels hot, it is recommended to adjust the room temperature or provide a cool rest area. Use smart home devices to achieve automated environmental adjustments, such as adjusting smart lights or air conditioning. Play specific music or sounds through smart speakers to soothe the pet's emotions. Provide suggestions for interacting with the owner, such as increasing companionship time or having intimate communication.
[0124] By analyzing and collecting user feedback, the emotion recognition model and interaction strategy can be dynamically adjusted and optimized.
[0125] Specifically, collect feedback from owners on emotion management strategies and interaction suggestions. Feedback forms include ratings, opinions and suggestions, or direct communication. Analyze feedback data to identify the strengths and weaknesses of the strategy. Adjust emotion recognition model parameters based on user feedback and emotion recognition results to enhance the accuracy of the model. Optimize the parameter configuration of the reinforcement learning algorithm to improve the effectiveness and adaptability of the strategy.
[0126] In summary, the present invention uses a variety of sensors (such as video, sound, body temperature, heart rate, GPS, accelerometer) to collect multimodal data of pets, which can comprehensively capture the behavior and physiological state of pets. This comprehensive data collection can improve the accuracy of emotion recognition and ensure that the system can understand the emotional changes of pets from multiple angles. The key features of pets, such as facial expressions, sound spectra, physiological changes and motor behaviors, are extracted through sparse coding and dictionary learning algorithms. This feature extraction method can effectively reduce data redundancy, retain the most representative features for emotion recognition, and improve the recognition and efficiency of the model.
[0127] The present invention adopts capsule network and generative adversarial network (GAN), which can capture the hierarchical relationship and spatial information between features and improve the ability to understand complex emotional features. GAN further enhances the generalization performance of the model through adversarial training of generative and discriminative networks, making it more outstanding in identifying the complex emotions of pets (such as happiness, excitement, anxiety, fear, anger). The emotional state is represented as a node, and the trend of emotional change is tracked and predicted by learning the relationship between nodes and graph structure features. GNN can identify the mutual influence and dynamic changes between emotional states, thereby providing more accurate emotion prediction results.
[0128] The present invention combines emotion recognition and dynamic tracking results, and conveys them to pet owners in real time through smart devices. The system uses reinforcement learning algorithms to generate personalized emotion management strategies, provide targeted behavioral suggestions and environmental adjustment plans, and help owners better manage their pets' emotions. By collecting and analyzing user feedback, the system can dynamically adjust and optimize emotion recognition models and interaction strategies to ensure that they continue to adapt to the changing needs of pets and owners.
[0129] The above implementation modes are merely descriptions of the preferred implementation modes of the present invention, and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary engineering and technical personnel in the field shall fall within the protection scope determined by the claims of the present invention.
Claims
1. A pet emotion recognition system based on AI, characterized in that: It includes a pet data collection module, a feature extraction module, a pet emotion recognition module, an emotion dynamic tracking module and an interactive feedback module which are sequentially connected in communication; The pet data collection module is used to collect multimodal data of the pet through a sensor network; the multimodal data includes video images, sounds, body temperature, heart rate, position and activity of the pet; The feature extraction module is used to receive the multimodal data of the pet and extract key features through sparse coding and dictionary learning algorithms; the key features include facial expression features, sound spectrum features, physiological index change features and movement behavior features of the pet; The pet emotion recognition module is used to recognize and classify pet emotions, including happiness, excitement, anxiety, fear and anger, through a capsule network algorithm and a generative adversarial network. The emotional dynamics tracking module is used to represent the emotional state of the pet as nodes through a graph neural network, and to track and predict the trend of emotional changes by learning the relationship between nodes and graph structure features; The interactive feedback module is used to generate emotion management strategies and interactive suggestions through a reinforcement learning algorithm based on the results of emotion recognition and dynamic tracking, and interact with pet owners through smart devices.
2. The AI-based pet emotion recognition system according to claim 1, characterized in that: The operation process of the feature extraction module includes the following steps: The multimodal data is formatted and normalized, including grayscale processing of video images, frequency domain analysis of sound signals, and filtering and smoothing processing of body temperature and heart rate; Use sparse coding algorithm to encode the features of preprocessed data, construct the initial feature matrix, and use sparse constrained optimization algorithm to select key technical features; Using dictionary learning algorithm, the feature matrix is trained to generate a feature dictionary suitable for pet emotion recognition; The extracted features are selected and combined, and the feature selection algorithm is used to construct the final key feature set based on the importance and relevance of the features, which provides input for the emotion recognition module.
3. The AI-based pet emotion recognition system according to claim 2, characterized in that: The formula of the sparse coding algorithm is as follows: Among them, x * represents the optimal sparse representation, i.e., the key features extracted from the multimodal data; y represents the feature vector of the preprocessed multimodal data; D represents the basis vector matrix of the feature space, i.e., the combination of all features in emotion recognition; x represents the sparse representation of the input data in the dictionary, i.e., the key feature combination corresponding to the pet's emotions; arg min x It means finding the parameter x that minimizes the objective function; The reconstruction error term represents the square error between the dictionary matrix D reconstructed data through the sparse representation x and the original input data y. If Dx is very close to y, it means that the extracted sparse features fully describe the current emotion of the pet; λ||x||1 represents the sparsity constraint term that controls the sparsity degree through λ.
4. A pet emotion recognition system based on AI according to claim 1, characterized in that: The operation process of the pet emotion recognition module includes the following steps: receiving the key features from the feature extraction module; Based on the key features, an emotion recognition model is constructed through a capsule network, and the model is trained in combination with a generative adversarial network; the capsule network is used to capture the hierarchical relationship and spatial information between features, and the model's ability to understand complex emotion features is improved through a dynamic routing mechanism; The processed multimodal data is input into the emotion recognition model to classify and identify the emotional state of the pet, output the initial emotion classification result, and use the generative adversarial network to strengthen the emotion recognition process and generate the final emotion recognition result.
5. A pet emotion recognition system based on AI according to claim 4, characterized in that: The formula of the emotion recognition model is as follows: in, represents the total loss function, which combines the losses of the capsule network and the generative adversarial network and is used to optimize the emotion recognition model; α represents the weight factor, which is used to control the influence of the capsule network feature reconstruction loss; β represents the weight factor, which is used to control the influence of the generative adversarial network loss; v j represents the output vector of the capsule network, i.e., the key features of pet emotions extracted; t j represents the target emotion feature vector, which is used to guide the accuracy of the output features of the capsule network; D(x) represents the output probability of the discriminator for the real data x; D(G(z)) represents the output probability of the discriminator for the generated data (G(z)); x represents a multimodal data sample; z represents a noise vector sampled from the latent space, which is used to generate potential emotion features.
6. The AI-based pet emotion recognition system according to claim 5, characterized in that: The formula of the capsule network is as follows: Among them, v j Represents the final recognized emotion feature, the output vector of the capsule network; s j Represents the sum of the input vectors of the jth emotion node.
7. The AI-based pet emotion recognition system according to claim 1, characterized in that: The operation process of the emotion dynamics tracking module includes the following steps: The emotion recognition results generated by the pet emotion recognition module are represented as nodes of the graph neural network, where each node corresponds to an emotional state; Use graph neural networks to build a relationship graph between emotional states. The edges in the graph represent the transformation relationships between emotional states, and the weights of the edges reflect the probability or similarity of the transformation. Based on the current emotional state and its adjacent emotional states, the features of each node are updated through the graph neural network to capture the mutual influence and dynamic changes between emotions; Perform time series analysis on the features of nodes in the graph neural network to predict the future trend of pet emotions and generate emotional state sequences; According to the emotional state sequence, potential emotional abnormalities are identified, and an automatic alarm mechanism for emotional changes is implemented by setting thresholds; The predicted emotion change trend and current emotion state are passed to the interactive feedback module.
8. The AI-based pet emotion recognition system according to claim 7, characterized in that: The formula of the graph neural network is as follows: in, Represents the feature vector of node i at time step t, that is, the emotional state characteristics of the pet at that moment; represents the updated feature vector of node i at time step t+1; represents the set of neighbor nodes of node i, that is, other emotional states directly related to the current emotional state; c ij represents the number of connections or the square root of the degree between nodes i and j; W represents the influence weight between emotional states; W0 represents the influence of the emotional state itself; σ represents the activation function.
9. The AI-based pet emotion recognition system according to claim 1, characterized in that: The operation process of the interactive feedback module includes the following steps: The pet's current emotional state and predicted emotional change trends are communicated to the pet owner in real time via smart devices; the smart devices include smart phones, tablet computers, smart speakers, and smart home devices; Based on the identified emotional state, a reinforcement learning algorithm is used to generate personalized emotion management strategies, provide behavioral suggestions and environmental adjustment plans; Generate interaction suggestions and management strategies based on the pet’s emotional state, including play activities, rest schedules, and emotional communication; By analyzing and collecting user feedback, the emotion recognition model and interaction strategy can be dynamically adjusted and optimized.
Citation Information
Patent Citations
SAR image change detection method based on sparse representation and capsule network
CN112270285A
Multi-channel physiological time sequence emotion recognition method based on ordinal number division network
CN116269386A
Pet emotion prediction method and device, storage medium and pet collar
CN117357115A
Pet state recognition device based on artificial intelligence technology
CN117668648A
Multifunctional education robot
CN118079182A
Cited By
Pet emotion recognition system and method
CN119314522A
Intelligent agent emotion generation method and system and storage medium
CN119599055A
Animal language conversion method and device, electronic equipment and storage medium
CN119943059A
Animal language conversion methods, devices, electronic equipment and storage media
CN119943059B
System platform for psychological assessment and emotion feedback
CN120124085A
