Pet calming method

Abstract

The present application discloses a pet calming method, comprising: acquiring pet behavior data, and inputting the pet behavior data into a pet emotion recognition model so as to identify a current emotion type of a pet; according to the current emotion type of the pet, determining a calming strategy corresponding to the current emotion type; and calming the pet on the basis of the calming strategy.

Description

Pet comforting methods

[0001] This application claims priority to Chinese patent application No. 202411531547.4, filed on October 30, 2024, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of emotion recognition technology, and more particularly to a method for soothing a pet. Background Technology

[0003] As society progresses and people's living standards improve, more and more families are keeping pets. Pets play an important role in life, providing not only emotional companionship but also helping people relieve stress and loneliness. Interacting with pets can bring happiness, improve mental health, and even encourage owners to participate in more social activities, thus improving their quality of life.

[0004] However, a pet's emotions are often easily influenced by its environment, health status, and owner. When a pet's emotions fluctuate, it may exhibit anxiety, aggression, or restlessness, which can trouble the owner and even increase emotional stress.

[0005] Experienced pet owners can accurately judge their pets' emotions by observing the tone, volume, and frequency of their cries, and then provide effective ways to soothe them. However, not every pet owner possesses this ability. Therefore, accurately understanding a pet's emotions in order to better soothe and interact with them has become a problem that pet owners want to solve. Technical issues

[0006] The main purpose of this application is to provide a pet soothing method that addresses the issue of identifying and comforting a pet's emotions. Technical solutions

[0007] To achieve the above objectives, this application proposes a pet soothing method, the method comprising:

[0008] Acquire pet behavior data and input it into a pet emotion recognition model to identify the pet's current emotion type;

[0009] Based on the pet's current emotional type, determine the soothing strategy corresponding to the current emotional type;

[0010] The pet is soothed using the aforementioned soothing strategy.

[0011] In one embodiment, the pet soothing device includes a speaker, a vibrator, a massager, an ultrasonic generator, and a high-voltage generator. In the step of determining a soothing strategy corresponding to the pet's current emotional type:

[0012] The soothing strategy includes playing soothing phrases generated based on the pet owner's voice and controlling the vibrator to operate in the corresponding vibration mode according to the pet's current emotional state.

[0013] In one embodiment, the pet emotion recognition model is established using the following steps:

[0014] Obtain a first pet behavior dataset, which includes pet sound data, action data, facial expression data, and environmental information;

[0015] A second pet behavior dataset is obtained by labeling the sound data, the action data, and the facial expression data in the first pet behavior dataset with emotion tags.

[0016] The second pet behavior dataset was cleaned and augmented to obtain the third pet behavior dataset.

[0017] A pre-trained model is obtained by pre-training a general neural network using an animal behavior dataset;

[0018] The pet emotion recognition model was obtained by fine-tuning the pre-trained model using the third pet behavior dataset.

[0019] In one embodiment, after the step of labeling the sound data, the action data, and the facial expression data in the first pet behavior dataset with emotion tags to obtain the second pet behavior dataset, the method further includes:

[0020] A behavior classifier is constructed to extract pet behavior features, identify pet emotions, and classify pet emotions.

[0021] In one embodiment, the step of constructing a behavior classifier, extracting pet behavior features through the behavior classifier, identifying pet emotions, and classifying pet emotions includes:

[0022] Convolutional neural networks are used to extract audio features, facial expression features, and action features that represent emotions;

[0023] Long Short-Term Memory (LSTM) networks are used to process the time-series characteristics of audio and / or video.

[0024] A fusion layer is constructed to fuse pet audio features, facial expression features, and motion features to obtain fused features;

[0025] A behavior classifier is constructed based on the fused features.

[0026] In one embodiment, the step of extracting audio features, facial expression features, and action features representing emotions using a convolutional neural network includes:

[0027] Perform a short-time Fourier transform on the audio signal to obtain its spectrum.

[0028] Convert the frequency axis in the spectrum plot to the Mel frequency scale to obtain the Mel spectrum plot;

[0029] Cepstral analysis was applied to the Mel spectrum, and Mel frequency cepstral coefficients were generated through discrete cosine transform.

[0030] Audio features representing emotions are extracted from Mel spectrograms or Mel frequency cepstral coefficients using convolutional neural networks.

[0031] In one embodiment, the step of extracting audio features, facial expression features, and action features representing emotions using a convolutional neural network includes:

[0032] Extracting the pet's facial region from a video or image to obtain a pet expression image;

[0033] Adjust the size of the pet's facial expression image and perform normalization processing;

[0034] The facial features of a pet are obtained by extracting features from the pet's facial region in the pet's facial expression image layer by layer using a convolutional neural network.

[0035] In one embodiment, the step of extracting audio features, facial expression features, and action features representing emotions using a convolutional neural network includes:

[0036] Image features of each frame in a pet video are extracted using a convolutional neural network.

[0037] Image features are input into a long short-term memory network to process time series data and obtain pet action features;

[0038] or,

[0039] By applying a 3D convolutional neural network to perform convolution operations between multiple frames, motion features at different time points are extracted to obtain pet action features.

[0040] In one embodiment, the step of fine-tuning the pre-trained model using the third pet behavior dataset to obtain a pet emotion recognition model includes:

[0041] The behavioral data from the third pet behavior dataset is input into the pre-trained model, and the pre-trained model outputs a prediction result based on the input behavioral data.

[0042] The loss function is used to calculate the error between the pre-trained model output and the true label, thus obtaining the loss value.

[0043] Based on the loss value, the gradient of each parameter in the pre-trained model is calculated using the backpropagation algorithm, wherein the parameters of the pre-trained model include the convolutional kernel weights, the weights of the fully connected layers, and the biases of the fully connected layers;

[0044] The optimization algorithm is used to adjust the model parameters according to the gradient, and / or adjust the hyperparameters, to perform the steps of inputting the behavior data in the third pet behavior dataset into the pre-trained model, and the pre-trained model outputting the prediction result according to the input behavior data, until the loss value reaches the preset value;

[0045] The hyperparameters include learning rate, batch size, number of network layers, kernel size, and regularization coefficient.

[0046] In one embodiment, the step of fine-tuning the pre-trained model using the third pet behavior dataset to obtain a pet emotion recognition model includes:

[0047] A multi-task model is constructed using a multi-task learning algorithm. This multi-task model is used to identify the pet's emotions, actions, and environment.

[0048] In one embodiment, the step of fine-tuning the pre-trained model using the third pet behavior dataset to obtain a pet emotion recognition model includes:

[0049] Regularization methods are used to control and constrain the parameters or structure of the model to prevent the model from overfitting to the training data. Beneficial effects

[0050] The technical solution proposed in this application has at least the following technical effects:

[0051] This application acquires pet behavior data, including body movements, sounds, and facial expressions. This data is input into a pet emotion recognition model to identify the pet's current emotion type. Based on this current emotion type, a corresponding soothing strategy is determined. The pet is then soothed using this strategy. By identifying the pet's emotions from multiple dimensions, the accuracy of emotion recognition is improved, allowing the model to determine whether the pet is angry, happy, or calm. By selecting appropriate strategies based on the pet's different emotional states, the model can more effectively alleviate anxiety and tension in pets. Attached Figure Description

[0052] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0053] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0054] Figure 1 is a flowchart illustrating an embodiment of the pet soothing method of this application;

[0055] Figure 2 is a flowchart illustrating an embodiment of the pet emotion recognition model establishment method of this application.

[0056] Figure 3 is a flowchart illustrating another embodiment of the pet emotion recognition model establishment method of this application;

[0057] Figure 4 is a flowchart illustrating another embodiment of the pet emotion recognition model establishment method of this application.

[0058] Figure 5 is a flowchart illustrating another embodiment of the pet emotion recognition model establishment method of this application.

[0059] Figure 6 is a flowchart illustrating another embodiment of the pet emotion recognition model establishment method of this application;

[0060] Figure 7 is a flowchart illustrating an embodiment of the pet emotion recognition model establishment method of this application.

[0061] Figure 8 is a flowchart illustrating another embodiment of the pet emotion recognition model establishment method of this application.

[0062] Figure 9 is a flowchart illustrating another embodiment of the pet emotion recognition model establishment method of this application;

[0063] Figure 10 is a flowchart illustrating another embodiment of the pet emotion recognition model establishment method of this application.

[0064] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Embodiments of the present invention

[0065] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0066] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0067] As society progresses and people's living standards improve, more and more families are keeping pets. Pets play an important role in life, providing not only emotional companionship but also helping people relieve stress and loneliness. Interacting with pets can bring happiness, improve mental health, and even encourage owners to participate in more social activities, thus improving their quality of life.

[0068] However, a pet's emotions are often easily influenced by its environment, health status, and owner. When a pet's emotions fluctuate, it may exhibit anxiety, aggression, or restlessness, which can trouble the owner and even increase emotional stress.

[0069] Experienced pet owners can accurately judge their pets' emotions by observing the tone, volume, and frequency of their cries, and then provide effective ways to soothe them. However, not every pet owner possesses this ability. Therefore, accurately understanding a pet's emotions in order to better soothe and interact with them has become a problem that pet owners want to solve.

[0070] To address the above issues, this application proposes a pet soothing method aimed at identifying and comforting a pet's emotions.

[0071] Referring to Figure 1, which is a flowchart illustrating an embodiment of the pet soothing method of this application, the pet soothing method proposed in this application includes steps S100 to S300, wherein:

[0072] Step S100: Obtain pet behavior data and input the pet behavior data into the pet emotion recognition model to identify the pet's current emotion type;

[0073] Step S200: Determine a soothing strategy corresponding to the current emotional type of the pet.

[0074] Step S300: Soothe the pet based on the soothing strategy.

[0075] In this embodiment, pet behavior data includes the pet's audio, video, and heart rate data. The pet's voice can be collected via a microphone, its facial expressions and movements can be captured via a camera (images or videos), and its heart rate can be collected via a heart rate sensor.

[0076] After acquiring pet behavior data, the data is input into a pet emotion recognition model. This model identifies the pet's current emotion type, helping pet owners recognize their pet's emotions based on its behavior. This allows for effective ways to soothe the pet or improve the quality of interaction between the pet owner and the pet, resulting in better companionship.

[0077] In daily life, a pet's emotions are influenced by its environment, physical condition, and the behavior of its owner. If a pet feels anxious, fearful, or overly stressed, it may exhibit aggressive or destructive behavior or health problems such as decreased appetite or insomnia. Owners also experience worry or anxiety when their pets are sick, anxious, or behaving abnormally. Pets that are overly dependent on their owners may exhibit separation anxiety when their owners are not around, impacting their quality of life. In these situations, if pet owners can employ appropriate soothing strategies to comfort their pets, it can help them maintain emotional stability and continue to enjoy the positive influences their pets bring; however, this requires considerable experience.

[0078] Therefore, in one embodiment, the pet soothing device includes a speaker, a vibrator, a massager, an ultrasonic generator, and a high-voltage generator. In the step of determining a soothing strategy corresponding to the pet's emotion recognition result:

[0079] The soothing strategy includes playing soothing phrases generated based on the pet owner's voice and controlling the vibrator to operate in the corresponding vibration mode according to the pet's current emotional state.

[0080] In this embodiment, the pet soothing device uses a pet emotion recognition model to infer the input pet behavior data, identify the pet's current emotion, and then generate corresponding soothing strategies based on the pet's emotional state to help the pet alleviate negative emotions.

[0081] Specifically, the pet soothing device includes a speaker, vibrator, massager, ultrasonic generator, and high-voltage generator. When a pet exhibits anxiety, fear, or other abnormal emotions, this application first collects the pet's voice through a microphone, captures images or videos of the pet's facial expressions and movements through a camera, and collects behavioral data such as the pet's heart rate through a heart rate sensor. This behavioral data is then input into a pet emotion recognition model. The model infers from the input pet behavioral data to identify the pet's current emotional state and, combined with past emotional data, determines the current emotion type. Then, the pet soothing device generates a gentle phrase based on the pet owner's voice and plays it through the speaker, mimicking the owner's tone to soothe the pet. Simultaneously, the vibrator operates in a specific vibration mode based on the pet's current emotional type; for example, a slow vibration mode may help the pet relax when anxious, while a faster vibration mode may be suitable when the pet needs stimulation or interaction.

[0082] In some implementations, pet soothsayers also include AI models. The functionality of pet soothsayers is further enhanced by employing AI models, which generate personalized soothing phrases and dynamically adjust soothing strategies.

[0083] Based on the pet's emotion recognition results, the AI ​​model can generate soothing phrases that match the current emotional state. For example, when a pet shows anxiety, the AI ​​can generate soothing and calming phrases, using simple and gentle language to comfort it.

[0084] AI can not only generate appropriate sentences, but also use audio synthesis technology to mimic the voice of a pet owner to play these sentences. Using the owner's voice helps pets recognize familiar sounds when they are not around, thus alleviating separation anxiety.

[0085] Using sensors or cameras, the system can monitor a pet's responses to soothing statements in real time, including changes in the pet's behavior, physiological data (such as heart rate and respiratory rate), and emotional fluctuations. This allows the system to determine whether the current soothing strategy is effective.

[0086] Based on the pet's real-time reactions, the AI ​​model can dynamically adjust the content and playback of soothing phrases. If the pet's emotions are not effectively relieved, the system will generate new, more suitable soothing phrases and play them again. The frequency and content of the adjustments can be gradually optimized based on the pet's emotional changes, forming a highly personalized soothing strategy.

[0087] The AI ​​model continuously optimizes the generation process of soothing phrases through machine learning. The pet soothing device records the pet's emotional changes and the effectiveness of soothing strategies. By analyzing this data, the accuracy and soothing effect of future phrase generation are gradually improved, forming a highly adaptable and personalized pet soothing device.

[0088] In this way, the soothing device not only helps pets alleviate separation anxiety when their owners are away, but also soothes and calms them when they exhibit excessive tension or stress. Furthermore, the combination of the pet owner's voice and vibration patterns simulates a real-life comforting scenario, providing the pet with more authentic emotional support.

[0089] This AI-based smart soothsayer combines the owner's voice and personalized phrases to provide immediate emotional support and more accurately meet the pet's emotional needs over time, thereby significantly improving the pet's quality of life and ensuring its emotional health and behavioral stability.

[0090] In one embodiment, referring to Figure 2, Figure 2 is a flowchart illustrating an embodiment of the pet emotion recognition model establishment method of this application. The pet emotion recognition model is established using the following steps:

[0091] Step S110: Obtain the first pet behavior dataset, which includes pet sound data, action data, facial expression data, and environmental information;

[0092] Step S120: Label the sound data, action data, and facial expression data in the first pet behavior dataset with emotion tags to obtain a second pet behavior dataset;

[0093] Step S130: Perform data cleaning and data augmentation on the second pet behavior dataset to obtain the third pet behavior dataset;

[0094] Step S140: Use the animal behavior dataset to pre-train the general neural network to obtain a pre-trained model;

[0095] Step S150: Fine-tune the pre-trained model using the third pet behavior dataset to obtain a pet emotion recognition model.

[0096] In this embodiment, a first pet behavior dataset is first formed by collecting a large amount of pet behavior data. This dataset includes multimodal data about pets. The first pet behavior dataset includes:

[0097] Sound data: such as pet barks, vocalizations, or other sounds;

[0098] Motion data: Capturing pets' body movements through sensors or cameras;

[0099] Facial expression data: Obtain changes in a pet's facial expressions through visual analysis;

[0100] Text data: Owners or pet caregivers record their pets' daily behaviors and related events;

[0101] Environmental information: This includes external factors such as temperature, light, and noise, which can affect a pet's mood.

[0102] Secondly, the collected sound, movement, and facial expression data are labeled with emotion tags, and the pet's behavior is associated with its corresponding emotion (such as happiness, anxiety, fear, relaxation, etc.) to form a second pet behavior dataset with emotion tags.

[0103] Next, data cleaning and data augmentation are performed on the second pet behavior dataset. Data cleaning includes removing invalid, duplicate, or outlier data to ensure the quality of the dataset. Data augmentation involves transforming the training data in various ways, such as rotating, flipping, cropping, and adding noise, to generate new training samples. This not only expands the dataset size but also allows the model to be trained on more diverse data, preventing overfitting, increasing the model's generalization ability, and resulting in a richer third pet behavior dataset.

[0104] In some implementations, different weights can be assigned to the categories of multimodal data between data cleaning and data augmentation of the second pet behavior dataset, thereby balancing the amount of each type of data. For example, 40% weight can be assigned to audio data, 30% to video data, 10% to text data, and 20% to image data.

[0105] Then, using the animal behavior dataset, a general neural network is pre-trained to obtain a basic model capable of extracting animal behavioral features. The animal behavior dataset includes animal audio and video data. At this stage, the model learns how to extract useful features from input data such as sound, movement, and facial expressions.

[0106] In this embodiment, the general neural network can be Wav2Vec or Transformer. Wav2Vec is a deep learning model for speech recognition, originally proposed by Facebook AI Research. It is a self-supervised learning approach primarily used for pre-training speech signals. Wav2Vec learns acoustic features by predicting missing parts in speech segments, allowing the model to learn useful representations from large amounts of unlabeled audio data. Wav2Vec can be viewed as a feature extractor, whose output can be further used as input for downstream tasks such as speech recognition, sentiment analysis, etc.

[0107] The Transformer is an architecture that has achieved a major breakthrough in the field of Natural Language Processing (NLP). It first appeared in the paper "Attention is All You Need". Unlike traditional RNNs or LSTMs, the Transformer uses a self-attention mechanism to process sequential data in parallel, greatly improving training efficiency. The Transformer has become the foundation for many NLP tasks, including machine translation, text summarization, and question answering systems.

[0108] Finally, the pre-trained general neural network was combined with a third pet behavior dataset, and the model parameters were fine-tuned to adapt it to the specific pet emotion recognition task. In this way, a deep learning model specifically for pet emotion recognition was obtained, which can automatically determine the emotional state of a pet based on the input pet behavior data.

[0109] In one embodiment, referring to Figure 3, Figure 3 is a flowchart illustrating another embodiment of the pet emotion recognition model establishment method of this application. Step S120, after labeling the sound data, action data, and facial expression data in the first pet behavior dataset with emotion tags to obtain the second pet behavior dataset, further includes:

[0110] Step S160: Construct a behavior classifier, extract pet behavior features through the behavior classifier, identify pet emotions, and classify pet emotions.

[0111] In this embodiment, the behavior classifier is a deep learning-based model used to automatically analyze and understand pet behavior data. Based on the multimodal data (such as sounds, movements, and expressions) extracted in previous steps, this classifier accurately extracts behavioral features by learning the complex relationship between pet behavior and its emotional state.

[0112] The primary function of a behavior classifier is to extract features from pet behavior data. These features may include the pet's movement patterns, vocalization patterns, facial expression changes, and so on. By extracting these features, the classifier can transform complex behaviors into easily analyzable numerical or vector representations.

[0113] Action feature extraction: By analyzing a pet's body posture, walking trajectory, jumping or running behavior, the classifier can identify the type of specific action (such as active, still, chasing, etc.).

[0114] Voice feature extraction: Extracting features such as pitch, volume, and frequency changes from a pet's barking, vocalizations, or other sound signals to identify emotional information in the voice.

[0115] Facial expression feature extraction: Using visual recognition technology, changes in a pet's facial expressions, especially the movement of the ears, eyes, and mouth, are extracted to determine the pet's emotions.

[0116] Once behavioral features are extracted, the behavior classifier uses these features to categorize the pet's emotions. The classifier is typically based on predefined emotion categories (such as happiness, anxiety, fear, excitement, etc.) and uses machine learning algorithms (such as convolutional neural networks (CNNs), long short-term memory networks (LSTMs), etc.) to predict these categories.

[0117] The classification process can be divided into several steps:

[0118] Feature input: The extracted features are used as input and fed into the classifier.

[0119] Model inference: The classifier processes the input data through neural networks or other machine learning algorithms to infer the category of the current emotion.

[0120] Emotion Classification Output: The model ultimately outputs the pet's emotional state and provides its classification result. For example, the classifier might output "The pet's current emotion is anxiety, with a confidence level of 85%".

[0121] In practical applications, behavior classifiers can adaptively adjust themselves by continuously learning and updating data. For example, a classifier can gradually optimize its classification accuracy based on the pet's historical behavior data and the owner's feedback, ensuring that its emotion recognition for each pet is more personalized.

[0122] A well-constructed behavior classifier can be applied in real-time to a pet emotion monitoring system. When a pet exhibits a specific behavior, the system can automatically identify the pet's emotional state through the classifier, helping the owner or automated devices to respond promptly. Based on the classification results, the owner can take appropriate soothing strategies or behavioral interventions to improve the pet's emotional state.

[0123] In one embodiment, referring to Figure 4, Figure 4 is a flowchart illustrating another embodiment of the pet emotion recognition model establishment method of this application. Step S160, constructing a behavior classifier, extracting pet behavior features through the behavior classifier, identifying pet emotions, and classifying pet emotions includes the following steps:

[0124] Step S161: Use a convolutional neural network to extract audio features, facial expression features, and action features that represent emotions;

[0125] Step S162: Use a Long Short-Term Memory network to process the time-series characteristics of audio and / or video;

[0126] Step S163: Construct a fusion layer to fuse pet audio features, facial expression features, and motion features to obtain fused features;

[0127] Step S164: Construct a behavior classifier based on the fused features.

[0128] In this embodiment, a convolutional neural network (CNN) is used to extract various behavioral features of the pet, including audio features, facial expression features, and action features.

[0129] By performing time-frequency analysis on the vocal signals of pets, CNNs can extract emotion-related features from the spectrogram, such as pitch, tone, and amplitude variations.

[0130] By using CNNs to analyze images of pets' faces, we can extract features of displacement or state changes in facial organs such as ears, eyes, and mouth. These features can help infer the pet's emotional state.

[0131] By analyzing a pet's movement trajectory and posture changes, CNNs can extract representative behavioral features from videos or other sensor data, such as jumping, walking, and stationary movement patterns.

[0132] Since pet sound and video data are time-series signals, this embodiment employs a Long Short-Term Memory (LSTM) network to handle the long-term dependencies of the sound data. For example, LSTM can identify continuous change patterns in sound, thereby determining whether there are emotion-related sound features, such as continuous barking which may indicate anxiety.

[0133] Pets' actions and expressions are also time-series data, and LSTM can capture the dynamic changes in these behaviors. For example, LSTM can identify the gradual transition of a pet from calm to excitement, combining data from multiple time steps to determine emotional changes.

[0134] LSTM is well-suited for handling long-term dependencies in these sequence data, thus accurately capturing the temporal dynamics of emotional changes.

[0135] Through the fusion layer, audio features, facial expression features, and motion features extracted from the CNN are integrated into a comprehensive feature vector. The fusion process can use concatenation, weighted summation, or more complex fusion strategies to ensure that information from different modalities complements each other, forming a more accurate representation of emotional features.

[0136] By combining features from different sensory channels, the system can gain a more comprehensive understanding of a pet's emotions. For example, a pet's facial expressions may indicate slight tension, but by combining vocal and motor features, the system can further confirm whether this emotion is anxiety or excitement.

[0137] The classifier can use fully connected layers or other common classification methods to input fused features into the model and ultimately output the pet's emotion category. Based on the weights obtained during training, the classifier can accurately classify the pet's emotions, such as identifying them as "anxious," "happy," or "angry."

[0138] After inputting pet behavior data, the classifier will output the corresponding classification result based on the fused features and predefined emotion labels, and use the confidence index to represent the model's certainty about the classification result.

[0139] This embodiment combines a Convolutional Neural Network (CNN) with a Long Short-Term Memory (LSTM) network to extract and process the audio, facial, and motor features of a pet, and integrates the multimodal data through a fusion layer. Finally, a behavior classifier accurately categorizes the pet's emotions. This method not only captures static behavioral features but also handles dynamic changes over time, thus more accurately identifying the pet's emotional state. This multimodal fusion emotion recognition system significantly improves the accuracy and reliability of pet emotion analysis.

[0140] In one embodiment, referring to Figure 5, Figure 5 is a flowchart illustrating another embodiment of the pet emotion recognition model establishment method of this application. Step S161, the step of extracting audio features, facial expression features, and action features representing emotions using a convolutional neural network, includes:

[0141] Step S1611: Perform a short-time Fourier transform on the audio signal to obtain a spectrum.

[0142] Step S1612: Convert the frequency axis in the spectrum diagram to Mel frequency scale to obtain the Mel spectrum diagram;

[0143] Step S1613: Apply cepstral analysis to the Mel spectrum and generate Mel frequency cepstral coefficients through discrete cosine transform;

[0144] Step S1614: Extract audio features representing emotions from the Mel spectrogram or Mel frequency cepstral coefficients using a convolutional neural network.

[0145] In this embodiment, the pet's sound signal is first converted into a spectrum using a short-time Fourier transform (STFT). STFT, by windowing the time-series signal, can convert the audio signal from the time domain to the frequency domain, thereby capturing the frequency changes of the sound at different points in time.

[0146] A spectrogram can represent the frequency components of an audio signal over a specific time period, which helps to capture the time-frequency characteristics of sound, especially revealing the dynamic fluctuations in sound frequency when a pet's emotions change.

[0147] The frequency axis of the spectrogram obtained in step S1611 is then converted to a Mel frequency scale to obtain a Mel spectrogram. The Mel frequency scale is a frequency scale that is more in line with human auditory perception. It represents frequencies in a non-linear way to better reflect the perceptual differences between low-frequency and high-frequency sounds.

[0148] Mel spectrograms are better able to capture emotion-related audio features because they highlight areas of the sound signal that humans or animals are sensitive to, making them particularly useful for processing audio data related to pet emotions.

[0149] Next, cepstral analysis is applied to the Mel spectrogram to generate Mel frequency cepstral coefficients (MFCCs) using discrete cosine transform (DCT). MFCCs are widely used features in audio processing and can effectively capture the spectral envelope information of sound.

[0150] MFCCs extract complex spectrogram information into a set of compact feature vectors. These features contain the main spectral information of the sound and can effectively represent the emotional features in the sound. MFCCs have been widely used in speech recognition and sentiment analysis, demonstrating high sentiment recognition accuracy.

[0151] Finally, a convolutional neural network (CNN) is used to extract emotion-related audio features from Mel spectrograms or Mel frequency-cephalic coefficients (MFCCs). CNNs can automatically identify emotion-related audio patterns by capturing local features and spatial dependencies in the input data through their convolutional layers. A convolutional neural network (CNN) consists of convolutional layers, pooling layers, and fully connected layers, where:

[0152] Convolutional layers: The convolutional layers of a CNN perform convolution operations on the input spectrogram or MFCCs to identify specific frequency patterns and time-series changes in the audio signal that are related to emotion.

[0153] Pooling layer: The pooling operation further reduces the feature map, extracts key emotional features from the audio signal, and removes redundant information.

[0154] Fully connected layer: Finally, the features extracted by the convolutional neural network are integrated into a feature vector through a fully connected layer for subsequent emotion classification or behavior analysis.

[0155] In this way, convolutional neural networks can automatically extract high-order features representing emotions from audio signals, which play an important role in subsequent pet emotion recognition.

[0156] In one embodiment, referring to Figure 6, Figure 6 is a flowchart illustrating another embodiment of the pet emotion recognition model establishment method of this application. Step S161, the step of extracting audio features, facial expression features, and action features representing emotions using a convolutional neural network includes:

[0157] Step S1615: Extract the pet's facial region from the video or image to obtain a pet expression image;

[0158] Step S1616: Adjust the size of the pet's facial expression image and perform normalization processing;

[0159] Step S1617: Extract the features of the pet's facial region from the pet's expression image layer by layer using a convolutional neural network to obtain expression features.

[0160] In step S1615: Extract the pet's facial region from the video or image to obtain a pet expression image. The convolutional neural network identifies and extracts the pet's facial region from the video stream or still image. Using object detection algorithms (such as a Haar feature cascade classifier or a deep learning face detection model), the pet's face can be accurately identified, and the region can be extracted as a separate image.

[0161] By extracting facial regions from a pet, the system can focus on the most expressive visual information about emotions, such as changes in the eyes, ears, and mouth. Subtle changes in these areas can often reflect a pet's emotional state.

[0162] In step S1616: Resizing and Normalizing the Pet Expression Image, the extracted pet expression image is resized to a uniform size to fit the input requirements of the convolutional neural network. This typically involves scaling the image to a fixed size (e.g., 224x224 pixels or 128x128 pixels).

[0163] Normalization, on the other hand, normalizes image pixel values, scaling them to between 0 and 1 (or between -1 and 1), thereby reducing numerical range bias and improving model convergence speed and stability. Normalization ensures more efficient network training while avoiding the influence of abnormal pixel values.

[0164] In step S1617: The features of the facial region in the pet expression image are extracted layer by layer by the convolutional neural network to obtain expression features. The pet expression image after normalization is input into the convolutional neural network (CNN) to extract the expression features of the pet's facial region in the image layer by layer.

[0165] The convolutional layer scans the pet's facial expression image by applying convolutional kernels (filters) to capture local features in the image. These features may include subtle changes in the pet's facial muscles, the opening and closing of the eyes, the position of the ears, and changes in the shape of the mouth. The convolutional layer can effectively extract these subtle emotional cues.

[0166] After the convolution operation, nonlinearity is introduced through an activation function (such as ReLU), enabling the network to capture complex nonlinear relationships in the image and further enhance its ability to express emotional features.

[0167] Pooling layers are used to reduce the size of feature maps, retaining the most important features while reducing data redundancy. A common pooling method is max pooling, which can preserve local maxima in an image, thereby extracting key emotional features, such as ears perking up when nervous or eyes half-closed when relaxed.

[0168] After convolutional and pooling layers, the feature maps are flattened and fed into fully connected layers. These fully connected layers integrate the extracted facial features into a high-dimensional feature vector. This feature vector summarizes the emotional characteristics of the pet's face and is used for subsequent emotion classification and behavior analysis.

[0169] Through this layer-by-layer feature extraction, convolutional neural networks can automatically extract deep emotional information from facial expression images, forming feature vectors with emotional discriminative power.

[0170] In one embodiment, referring to Figure 7, Figure 7 is a flowchart illustrating an embodiment of the pet emotion recognition model establishment method of this application. Step S161, the step of extracting audio features, facial expression features, and action features representing emotions using a convolutional neural network includes:

[0171] Step S1618: Extract image features of each frame in the pet video using a convolutional neural network, and input the image features into a long short-term memory network to process the time series to obtain pet action features;

[0172] or,

[0173] Step S1619: Apply a 3D convolutional neural network to perform convolution operations between multiple frames to extract motion features at different time points and obtain pet action features.

[0174] In this embodiment, based on convolutional neural networks (CNN) and their combination with time series processing, two different methods can be used to extract pet action features.

[0175] Step S1618: Extract image features for each video frame using a convolutional neural network (CNN), and process the time series using a Long Short-Term Memory (LSTM) network to obtain pet action features. The video is decomposed into multiple frames, and each frame is input as a separate image into the CNN. The CNN extracts image features related to the pet's actions from each frame. These features include the pet's posture, limb movements, and dynamic changes in the head or tail at each moment.

[0176] Among them, the convolutional layer performs a convolution operation on each frame of the image to extract spatial features in the image, such as the pet's posture and local motion features.

[0177] Pooling layer: Reduces the dimensionality of the feature map, retaining the most important action-related features.

[0178] The extracted image features of each frame are used as time-series data and input into a Long Short-Term Memory (LSTM) network. LSTM excels at processing sequential data and can capture the temporal dependencies between video frames, thereby extracting the overall motion features of the pet.

[0179] LSTM can capture the continuity and dynamic changes of actions through memory and forgetting mechanisms, recognizing the process by which a pet transitions from one action to another. For example, LSTM can learn action patterns such as running, jumping, or slow movement from multiple consecutive frames.

[0180] By using LSTM to process the time series, the system can extract complete motion features, reflecting the pet's movement patterns and emotional dynamics throughout the video.

[0181] Step S1619: Apply a 3D convolutional neural network (3D-CNN) to perform convolution operations between multiple frames to extract motion features at different time points, obtaining the pet's action features. The 3D convolutional neural network (3D-CNN) directly performs convolution operations between multiple consecutive video frames. Unlike traditional 2D CNNs, 3D CNNs can perform convolutions simultaneously in both temporal and spatial dimensions, thereby capturing motion information between video frames.

[0182] 3D-CNN extracts the motion features of a pet at different points in time by performing three-dimensional convolution operations on a sequence of video frames. This method can not only capture the spatial features in each frame (such as the pet's posture at a specific moment), but also capture the dynamic changes between consecutive frames (such as the process of the pet moving from one position to another).

[0183] 3D convolution can simultaneously take into account the pet's temporal movement changes and spatial posture changes, integrating multidimensional information from time and space. This makes the method more efficient in extracting complex movement patterns of pets, such as fast chasing or slow crawling.

[0184] Motion features extracted through 3D convolution operations can reflect the pet's movement changes and dynamic features throughout the video, providing strong motion information support for emotion recognition.

[0185] In one embodiment, referring to Figure 8, Figure 8 is a flowchart illustrating another embodiment of the pet emotion recognition model establishment method of this application. Step S150, the step of fine-tuning the pre-trained model using the third pet behavior dataset to obtain the pet emotion recognition model includes:

[0186] Step S151: Input the behavior data from the third pet behavior dataset into the pre-trained model, and the pre-trained model outputs the prediction result based on the input behavior data.

[0187] In this step, behavioral data (such as audio, actions, facial expressions, and other multimodal data) from the third pet behavior dataset are input into the pre-trained model. The pre-trained model has been pre-trained using other general animal behavior datasets and possesses certain feature extraction and emotion recognition capabilities.

[0188] Input behavioral data: This data is preprocessed (such as normalization, data augmentation, etc.) and then fed into the model. The model outputs prediction results based on the input data, such as the pet's current emotional state (such as anxiety, happiness, fear, etc.).

[0189] Step S152: Use the loss function to calculate the error between the pre-trained model output and the true label to obtain the loss value.

[0190] In this step, the model's predicted output is compared with the predefined real emotion labels in the third pet behavior dataset, and the error between the two is calculated using a loss function to obtain the loss value.

[0191] Loss functions: Common loss functions include cross-entropy loss (for classification tasks) or mean squared error (for regression tasks). A loss function quantifies the difference between the model's predictions and the actual sentiment labels; the smaller the error, the more accurate the model's predictions.

[0192] Loss value: The loss value is a metric that measures the current performance of the model, reflecting the degree of deviation between the model's output and the true label. A higher loss value indicates that the model's performance needs further optimization.

[0193] Step S153: Based on the loss value, calculate the gradient of each parameter in the pre-trained model using the backpropagation algorithm, wherein the parameters of the pre-trained model include the convolutional kernel weights, the weights of the fully connected layers, and the biases of the fully connected layers.

[0194] In this step, the gradients of the parameters of each layer in the model are calculated using the backpropagation algorithm. The parameters of the pre-trained model include the weights of the convolutional kernels, the weights of the fully connected layers, and the biases of the fully connected layers in the convolutional neural network.

[0195] Backpropagation: This is an algorithm used to train neural networks. By backpropagating the loss value to each layer of the network, the gradients of the model parameters with respect to the loss value are calculated. These gradients represent the direction and magnitude of the loss value as the parameters change, guiding the direction of model parameter adjustments.

[0196] Gradient calculation: For the convolutional kernel weights, fully connected layer weights, and biases, calculate their gradients relative to the loss value. These gradients are the basis for the optimization algorithm to adjust the model parameters.

[0197] Step S154: Use an optimization algorithm to adjust the model parameters according to the gradient, and / or adjust the hyperparameters, and execute the step of inputting the behavior data in the third pet behavior dataset into the pre-trained model, and the pre-trained model outputting the prediction result according to the input behavior data, until the loss value reaches the preset value;

[0198] The hyperparameters include learning rate, batch size, number of network layers, kernel size, and regularization coefficient.

[0199] In this step, optimization algorithms (such as Adam, SGD, etc.) are used to adjust the model parameters based on the calculated gradients, with the aim of minimizing the loss value.

[0200] Parameter tuning: Based on the gradient value of each parameter, the optimization algorithm updates the model's weights and biases, so that the model output is closer to the true label.

[0201] Adjusting hyperparameters: During model fine-tuning, some hyperparameters can also be adjusted, including:

[0202] Learning rate: Controls the size of the step size in model parameter updates. A higher learning rate can speed up training but may lead to unstable convergence; a lower learning rate may slow down model convergence.

[0203] Batch size: refers to the number of data samples used for each model parameter update. A larger batch size helps stabilize model training, but requires more computational resources.

[0204] Network layer number: The depth of the model. Too deep a network may lead to overfitting, while too shallow a network may prevent the model from learning enough features.

[0205] Kernel size: The size of the convolution kernel affects the precision of feature extraction.

[0206] Regularization coefficient: A hyperparameter used to prevent model overfitting and can adjust the model's generalization ability.

[0207] Repeat the input and adjustment process: After parameter adjustment, continue to input behavioral data into the model, recalculate the loss value and gradient, and adjust the parameters. This process iterates until the loss value reaches a preset threshold or the model converges.

[0208] In this embodiment, data from a third pet behavior dataset is input into a pre-trained model. The loss value between the predicted result and the true label is calculated. The backpropagation algorithm is used to calculate the parameter gradient, and the model parameters are adjusted through an optimization algorithm to finally obtain a pet emotion recognition model. In this process, the adjustment of hyperparameters can further optimize the model's performance, ensuring that the model can accurately and stably recognize the pet's emotional state in practical applications.

[0209] In one embodiment, referring to Figure 9, Figure 9 is a flowchart illustrating another embodiment of the pet emotion recognition model establishment method of this application. Step S150, the step of fine-tuning the pre-trained model using the third pet behavior dataset to obtain the pet emotion recognition model includes:

[0210] Step S155: A multi-task model is constructed using a multi-task learning algorithm. The multi-task model is used to identify the pet's emotions, the pet's actions, and the pet's environment.

[0211] Task learning is a machine learning technique that improves a model's generalization ability by allowing it to learn multiple related tasks simultaneously, share information, and improve its generalization ability.

[0212] In this embodiment, the multi-task learning algorithm is used to construct a model capable of processing multiple tasks simultaneously, including:

[0213] Pet Emotion Recognition: Identifies a pet's current emotional state, such as happiness, anxiety, anger, relaxation, etc.

[0214] Pet motion recognition: Recognizes pet actions in videos or images, such as running, standing still, jumping, eating, etc.

[0215] Environmental recognition: Identify the pet's current environment, such as indoors, outdoors, noisy environment, quiet environment, etc., because changes in the environment have a great impact on the pet's emotions and behavior.

[0216] This embodiment utilizes a multi-task learning algorithm to construct a multi-task model, achieving multi-dimensional recognition of pet emotions, actions, and the environment. By sharing a feature layer and task-specific layers, the model can effectively extract emotion, action, and environment-related features from pet behavioral data, while simultaneously optimizing performance across various tasks. This approach not only improves the accuracy of emotion recognition but also provides more comprehensive behavioral analysis and environmental awareness, helping pet owners better understand their pets' behavior and emotional states.

[0217] In one embodiment, referring to Figure 10, Figure 10 is a flowchart illustrating another embodiment of the pet emotion recognition model establishment method of this application. Step S150, the step of fine-tuning the pre-trained model using the third pet behavior dataset to obtain the pet emotion recognition model includes:

[0218] Step S156: Use regularization methods to control and constrain the parameters or structure of the model to prevent the model from overfitting on the training data.

[0219] During the training of a pet emotion recognition model, the model may perform well on training data but poorly on new data, a phenomenon known as overfitting. To prevent this, this embodiment uses regularization methods to control and constrain the model's complexity, thereby improving the model's generalization ability.

[0220] In this embodiment, L1 regularization, L2 regularization, and Dropout regularization can be used to prevent the model from overfitting on the training data.

[0221] L1 regularization constrains the absolute values ​​of model parameters by adding a penalty term to the loss function, causing some weights to tend to zero, thus achieving parameter sparsity. This is suitable for feature selection and reducing model complexity. In pet emotion recognition models, it can remove unimportant features, thereby reducing the risk of overfitting.

[0222] L2 regularization constrains the squares of the model parameters by adding a penalty term to the loss function. By limiting the magnitude of the weights, L2 regularization prevents the model's weights from becoming excessively large, thus preventing over-reliance on specific features. This smooth weight distribution helps the model perform better on new data, improving its generalization ability.

[0223] Dropout is a structural regularization method that randomly discards a certain percentage of neurons during training to prevent the model from becoming dependent on specific neurons or features. In each training iteration, some neurons are randomly removed, allowing the model to learn more robust features. It effectively prevents overfitting, especially in deep neural networks. For pet emotion recognition models, Dropout allows the model to avoid over-reliance on a particular type of input feature when extracting audio, facial, or motion features, thus improving the model's generalization ability.

[0224] The technical solution proposed in this application has at least the following technical effects:

[0225] This application acquires pet behavior data, including body movements, sounds, and facial expressions. This data is input into a pet emotion recognition model to identify the pet's current emotion type. Based on this current emotion type, a corresponding soothing strategy is determined. The pet is then soothed using this strategy. By identifying the pet's emotions from multiple dimensions, the accuracy of emotion recognition is improved, allowing the model to determine whether the pet is angry, happy, or calm. By selecting appropriate strategies based on the pet's different emotional states, the model can more effectively alleviate anxiety and tension in pets.

Claims

1. A pet soothing method, applied to a pet soothing device, wherein, The method includes: Acquire pet behavior data and input it into a pet emotion recognition model to identify the pet's current emotion type; Based on the pet's current emotional type, determine the soothing strategy corresponding to the current emotional type; The pet is soothed using the aforementioned soothing strategy.

2. The method as described in claim 1, wherein, The pet soothing device includes a speaker, a vibrator, a massager, an ultrasonic generator, and a high-voltage generator. In the step of determining a soothing strategy corresponding to the pet's current emotional type: The soothing strategy includes playing soothing phrases generated based on the pet owner's voice and controlling the vibrator to operate in the corresponding vibration mode according to the pet's current emotional state.

3. The method as described in claim 1, wherein, The pet emotion recognition model was established using the following steps: Obtain a first pet behavior dataset, which includes pet sound data, action data, facial expression data, and environmental information; A second pet behavior dataset is obtained by labeling the sound data, the action data, and the facial expression data in the first pet behavior dataset with emotion tags. The second pet behavior dataset was cleaned and augmented to obtain the third pet behavior dataset. A pre-trained model is obtained by pre-training a general neural network using an animal behavior dataset; The pre-trained model was fine-tuned using the third pet behavior dataset to obtain a pet emotion recognition model.

4. The method of claim 3, wherein, After the step of labeling the sound data, motion data, and facial expression data in the first pet behavior dataset with emotion tags to obtain the second pet behavior dataset, the method further includes: A behavior classifier is constructed to extract pet behavior features, identify pet emotions, and classify pet emotions.

5. The method of claim 4, wherein, The steps of constructing a behavior classifier, extracting pet behavior features through the behavior classifier, identifying pet emotions, and classifying pet emotions include: Convolutional neural networks are used to extract audio features, facial expression features, and action features that represent emotions; Long Short-Term Memory (LSTM) networks are used to process the time-series characteristics of audio and / or video. A fusion layer is constructed to fuse pet audio features, facial expression features, and motion features to obtain fused features; A behavior classifier is constructed based on the fused features.

6. The method of claim 5, wherein, The steps of extracting audio features, facial expression features, and action features representing emotions using a convolutional neural network include: Perform a short-time Fourier transform on the audio signal to obtain its spectrum. Convert the frequency axis in the spectrum plot to the Mel frequency scale to obtain the Mel spectrum plot; Cepstral analysis was applied to the Mel spectrum, and Mel frequency cepstral coefficients were generated through discrete cosine transform. Audio features representing emotions are extracted from Mel spectrograms or Mel frequency cepstral coefficients using convolutional neural networks.

7. The method of claim 5, wherein, The steps of extracting audio features, facial expression features, and action features representing emotions using a convolutional neural network include: Extracting the pet's facial region from a video or image to obtain a pet expression image; Adjust the size of the pet's facial expression image and perform normalization processing; The facial features of a pet are obtained by extracting features from the pet's facial region in the pet's facial expression image layer by layer using a convolutional neural network.

8. The method of claim 5, wherein, The steps of extracting audio features, facial expression features, and action features representing emotions using a convolutional neural network include: Image features of each frame in a pet video are extracted using a convolutional neural network. These image features are then input into a long short-term memory network to process the time series and obtain pet action features. or, By applying a 3D convolutional neural network to perform convolution operations between multiple frames, motion features at different time points are extracted to obtain pet action features.

9. The method of claim 3, wherein, The step of fine-tuning the pre-trained model using the third pet behavior dataset to obtain the pet emotion recognition model includes: The behavioral data from the third pet behavior dataset is input into the pre-trained model, and the pre-trained model outputs a prediction result based on the input behavioral data. The loss function is used to calculate the error between the output of the pre-trained model and the true label to obtain the loss value; Based on the loss value, the gradient of each parameter in the pre-trained model is calculated using the backpropagation algorithm, wherein the parameters of the pre-trained model include the convolutional kernel weights, the weights of the fully connected layers, and the biases of the fully connected layers; The optimization algorithm is used to adjust the model parameters according to the gradient, and / or adjust the hyperparameters, to perform the steps of inputting the behavior data in the third pet behavior dataset into the pre-trained model, and the pre-trained model outputting the prediction result according to the input behavior data, until the loss value reaches the preset value; The hyperparameters include learning rate, batch size, number of network layers, kernel size, and regularization coefficient.

10. The method of claim 3, wherein, The step of fine-tuning the pre-trained model using the third pet behavior dataset to obtain the pet emotion recognition model includes: A multi-task model is constructed using a multi-task learning algorithm. This multi-task model is used to identify the pet's emotions, actions, and environment. Regularization methods are used to control and constrain the parameters or structure of the model to prevent the model from overfitting to the training data.

Images