Pet accompanying robot state control system and method capable of dynamically updating character

Abstract

The invention discloses a pet accompanying robot state control system and method capable of dynamically updating characters, and the method comprises the steps: sensing the current state of a user, and recognizing the emotion state of the user through a multi-mode emotion calculation method based on the current state of the user; each feedback state of the pet robot is generated; based on the selected specific feedback state, the appropriate pet robot feedback expression, action and sound effect are intelligently matched; and synchronously controlling the pet robot to realize integral state output of man-machine interaction. According to the method, the specific feedback state is output by combining the multi-mode emotion perception with the personality of the pet robot, so that the personalized interaction experience of the pet robot is improved. Meanwhile, through a dynamic updating mechanism of the character of the pet robot, the pet robot can evolve continuously, and the emotional accompanying experience of thousands of people in long-term use of the user is improved.

Description

Technical Field

[0001] This invention relates to the field of robotics, specifically to a dynamic personality-updating status control system and method for a companion pet robot. Background Technology

[0002] As people's living standards improve and society ages, the demand for emotional companionship is growing. However, the fast pace of work and life means less time for family members and friends to spend together. While traditional pet ownership has addressed this need to some extent, many families are still unable to keep pets due to limitations such as time, space, and hygiene. These limitations include the risk of parasite transmission from pet contact, bacterial growth from improperly cleaned feces, environmental hygiene or allergy problems caused by shedding, and the inability to take sick pets to the vet promptly while traveling.

[0003] With the miniaturization and micro-miniaturization of robots, and the rapid development of technologies such as artificial intelligence, various highly biomimetic and intelligent pet robots have gradually emerged on the market, attracting increasing attention as a new form of emotional companionship. However, most existing pet robots lack a deep understanding of user emotions and the ability to adapt. Their personalities are often fixed and cannot be dynamically updated based on user behavior and emotional states, resulting in less authentic and natural emotional interactions with users, making it difficult to meet users' diverse and personalized emotional companionship needs. Summary of the Invention

[0004] The purpose of this invention is to provide a dynamic personality-updating status control system and method for a companion pet robot. This system enables the pet robot to perceive the user's emotional state in real time through sound and visual recognition, and to perform comprehensive calculations based on the pet's own personality to generate diverse pet feedback states. This achieves a more natural, intimate, and personalized emotional interaction between the user and the pet robot. Simultaneously, the pet robot's personality is dynamically updated through periodic data analysis and learning algorithms, allowing the pet robot to develop its own personality and enhancing the long-term emotional companionship experience for the user.

[0005] The present invention solves the above problems through the following technical solution:

[0006] A dynamic personality-updating companion pet robot state control system includes: a perception module, an emotion computing module, a feedback state generation module, a state control module, and an execution module connected in sequence; and a perception module, an intelligent dialogue module, a dynamic personality update module, and a feedback state generation module also connected in sequence. The emotion computing module is further connected to the dynamic personality update module, and the intelligent dialogue module is further connected to the execution module.

[0007] The sensing module is used to obtain the user's current state and includes at least a microphone, camera, and touch sensor as sensing units.

[0008] The emotion computing module is used to identify the user's emotional state from the perceived user's voice and facial expressions;

[0009] The feedback status generation module combines the current user's emotional state and the current pet robot's personality according to a weighted ratio to generate the pet robot's feedback status.

[0010] The intelligent dialogue module is used to recognize the content and semantics of the user's speech, process and generate response content based on the context of the speech, and then convert it into speech to achieve intelligent dialogue;

[0011] The personality dynamic update module is used to dynamically update the pet robot's personality based on historical user emotion recognition results and intelligent dialogue content.

[0012] The status control module is used to match and select specific feedback expressions, actions and sound effects based on the generated pet robot feedback status in order to control the execution of the pet robot's feedback behavior;

[0013] The execution module is used to distribute the pet robot's feedback expressions, movements, and sound effects to the eye screen unit, the head and the motor units of each joint, and the speaker unit, respectively, to achieve human-computer interaction output.

[0014] As a further improvement, the microphone includes a ring of six microphones for sensing the direction of the user's speech; the touch sensors include multiple body parts of the robot.

[0015] As a further improvement, the emotion computing module stores historical user emotion recognition results and tags these results as one of the dimensions for dynamic personality updates.

[0016] And / or the intelligent dialogue module stores the user's historical dialogue content with the robot, and tags the historical dialogue content as one of the dimensions for dynamic personality updates.

[0017] As a further improvement, the pet robot's feedback states include at least three categories: positive feedback, neutral feedback, and negative feedback. Positive feedback includes activity, curiosity, and relaxation; neutral feedback includes alertness, hunger, and rest; and negative feedback includes anxiety, loneliness, and pain.

[0018] As a further improvement, the first mapping relationship established between each user emotion and the pet robot's feedback state is a probability distribution of the occurrence of various feedback states during each human-computer interaction, with the total probability being 100%.

[0019] The second mapping relationship between the personality described for each pet robot and the pet robot's feedback state is the probability distribution of the occurrence of various feedback states during each human-computer interaction, with the total probability being 100%.

[0020] Furthermore, the combined weight of the current user's emotion and the current pet robot's personality is 100%.

[0021] Furthermore, the present invention also solves the above problems through the following technical solutions:

[0022] A method for controlling the state of a companion pet robot that dynamically updates its personality includes:

[0023] Sensing the user's current state includes observing the user's facial expressions and gestures through the camera, listening to the user's voice through the microphone, and obtaining the user's touch actions and locations through the touch sensor;

[0024] Identify user emotional states using multimodal sentiment analysis methods based on user voice and facial expressions.

[0025] Generate each feedback state of the pet robot, including establishing a first mapping relationship between the current user's emotion and each feedback state of the pet robot, and establishing a second mapping relationship between the current pet robot's personality and each feedback state of the pet robot. Both mapping relationship tables include the distribution probability of each feedback state. The distribution probability values ​​of the two mapping relationship tables are combined according to a specified weight ratio to form the final distribution probability of each feedback state, and the specific feedback state is selected according to the final distribution probability.

[0026] Based on the selected specific feedback state, the system intelligently matches appropriate pet robot feedback expressions, actions, and sound effects from the built-in expression library, action library, and sound effect library.

[0027] The system synchronously controls the pet robot's eye screen to display matching feedback expressions, controls the motors of its head and joints to execute matching feedback action sequences, and controls the speaker to play matching feedback sound effects, thus achieving the overall state output of human-computer interaction.

[0028] As a further improvement, the control method specifically includes the following steps:

[0029] Step S01. The pet robot observes the user's facial expressions and collects video stream data through a camera;

[0030] Step S02. The pet robot listens to the user's voice through the microphone and collects audio stream data;

[0031] Step S03. Visual and speech-based multimodal emotion recognition: Based on the video stream data collected in step S01, visual facial expressions of the user are recognized; and based on the audio stream data collected in step S02, the timbre and content of the user's speech are recognized to obtain the emotional state.

[0032] Step S04. Obtain the mapping table between user emotions and pet robot states;

[0033] Step S05. Query the corresponding pet robot state distribution probability based on the user's emotion recognition results;

[0034] Step S06. Obtain the state calculation weight setting parameters; the parameters include the weight values ​​of the current user's emotion and the current pet robot's personality, which correspond to the probability distribution of the pet robot's state, and the sum of the two weight values ​​is 100%;

[0035] Step S07. Obtain the mapping table between the current pet robot's personality and the pet robot's state;

[0036] Step S08. Calculate the final probability distribution of each state of the pet robot based on the weights, according to the probability distribution of each state of the pet robot corresponding to the current user's emotion and the current pet robot's personality.

[0037] Step S09. Based on the final probability distribution, select the pet robot feedback state that should be output this time;

[0038] Step S10. Obtain the library of facial expressions, actions, and sound effects available for the pet robot's feedback status;

[0039] Step S11. Based on the feedback status, select the final output emoticons, actions, and sound effects from the available emoticon, action, and sound effect library;

[0040] Step S12. After arranging facial expressions, actions, and sound effects in a reasonable timing sequence, execute the output.

[0041] As a further improvement, the control method further includes the following steps:

[0042] Step S21. Perform speech recognition on the acquired user voice and convert it into text;

[0043] Step S22. Process the text using a large model to generate response content for the dialogue;

[0044] Step S23. Convert the reply to voice;

[0045] Step S31. Display feedback facial animations on the eye display screen;

[0046] Step S32. The action is executed by motors in the head and limbs;

[0047] Step S33. Play feedback sound effects and the reply voice from the intelligent dialogue module through the speaker.

[0048] As a further improvement, the control method also includes: a dynamic update mechanism;

[0049] The dynamic update mechanism includes: the system has a built-in default pet robot personality; acquiring historical user emotion recognition results and historical intelligent dialogue content; constructing pet robot personality-related tags, and performing data analysis on historical user emotion recognition results and historical intelligent dialogue content within a specified time period, and updating the tag data; at the end of each time period, updating the probability values ​​of the distribution of each feedback state involved in the current pet robot personality according to the corresponding tag data and a specified mapping relationship, so as to realize the dynamic update of the pet robot personality.

[0050] As a further improvement, the dynamic update mechanism includes the following specific steps:

[0051] Step S41. Save the user's historical emotional data;

[0052] Step S51. Save the history data of the intelligent dialogue;

[0053] Step S61. Based on the historical data saved in steps S41 and S51, generate tag data according to the specified time period;

[0054] Step S62. Obtain the mapping relationship between tags and pet robot states;

[0055] Step S63. Based on the count of each tag, query and statistically analyze the new distribution of the status of each pet robot corresponding to the tag;

[0056] Step S64. Based on the new state distribution, compare it with the current state distribution corresponding to the pet robot's personality, and generate pet robot personality correction data according to the specified rules;

[0057] Step S65. Update the pet robot's personality data according to the personality correction data, and save it as the new current pet robot personality data.

[0058] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0059] This invention enhances the personalized interactive experience of pet robots by using multimodal emotion perception combined with the robot's own personality to output specific feedback states. Furthermore, through a dynamic personality update mechanism, the pet robot can continuously evolve, improving the emotional companionship experience for users over the long term. Attached Figure Description

[0060] Figure 1 This is a schematic diagram of the pet robot state control system framework according to an embodiment of the present invention.

[0061] Figure 2 This is a schematic diagram of the pet robot state control process according to an embodiment of the present invention.

[0062] Figure 3 This is a schematic diagram illustrating the probability distribution of user emotions and pet robot states according to an embodiment of the present invention.

[0063] Figure 4 This is a schematic diagram illustrating the probability distribution of the built-in personality and state of the pet robot described in an embodiment of the present invention.

[0064] Figure 5 This is a schematic diagram of the dynamic status update process of the pet robot according to an embodiment of the present invention.

[0065] Attached reference numerals: 101, Perception module; 201, Affective computing module; 202, Feedback state generation module; 203, Intelligent dialogue module; 204, Dynamic personality update module; 205, State control module; 301, Execution module. Detailed Implementation

[0066] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0067] Example 1:

[0068] Combined with appendix Figure 1 As shown, a dynamic personality-updating companion pet robot state control system includes: a perception module 101, an emotion computing module 201, a feedback state generation module 202, an intelligent dialogue module 203, a personality dynamic update module 204, a state control module 205, and an execution module 301. The perception module 101, emotion computing module 201, feedback state generation module 202, state control module 205, and execution module 301 are connected in sequence. The perception module 101 is also connected in sequence to the intelligent dialogue module 203, the personality dynamic update module 204, and the feedback state generation module 202. The emotion computing module 201 is also connected to the personality dynamic update module 204, and the intelligent dialogue module 203 is also connected to the execution module 301.

[0069] The perception module 101 is used to obtain the user's current state, including at least voice and facial expression, and includes perception units such as microphone, camera, and touch sensor.

[0070] In one embodiment, the microphone includes a ring of six microphones, which can be used to sense the approximate location of the user speaking. Touch sensors include multiple body parts of the robot, such as the head, back, chin, and abdomen.

[0071] The emotion computing module 201 is used to identify the user's emotional state from the perceived user's voice and facial expressions.

[0072] In one embodiment, user emotional states include calm, joy, anger, worry, contemplation, sadness, fear, and surprise. The emotion computing module stores historical user emotion recognition results and tags these results as one of the dimensions for dynamic personality updates.

[0073] The feedback status generation module 202 generates the pet robot's feedback status based on the current user's emotional state and the current pet robot's personality, and combines them according to a weight ratio.

[0074] In one embodiment, the system has pre-set some initial pet robot personalities, including optimistic and positive, quiet and peaceful, and curious. At the initial setup of a specific device, one of these pre-set personalities is selected as the current pet robot personality for that device. The pet robot's feedback states include three categories: positive feedback, neutral feedback, and negative feedback. Positive feedback includes activity, curiosity, and relaxation; neutral feedback includes alertness, hunger, and rest; and negative feedback includes anxiety, loneliness, and pain.

[0075] Furthermore, the first mapping relationship established between each user emotion and the pet robot's feedback state is the probability distribution of the occurrence of various feedback states during each human-computer interaction, with the total probability being 100%.

[0076] Furthermore, the second mapping relationship established between each pet robot personality and the pet robot's feedback state is the probability distribution of various feedback states during each human-computer interaction, with the total probability being 100%.

[0077] The combined weight of the current user's emotion and the current pet robot's personality is 100%. Optionally, the weight of both the current user's emotion and the current pet robot's personality is set to 50% by default.

[0078] The intelligent dialogue module 203 is used to recognize the user's speech content and semantics, process and generate response content based on the context of the speech, and then convert it into speech to achieve intelligent dialogue.

[0079] In one embodiment, the intelligent dialogue module saves the user's historical dialogue content with the robot and tags the historical dialogue content as one of the dimensions for dynamic personality updates.

[0080] The personality dynamic update module 204 is used to dynamically update the pet robot's personality based on historical user emotion recognition results and intelligent dialogue content.

[0081] In one embodiment, using a natural week as the time period, the number of tag repetitions is calculated based on historical user emotion recognition result tags in the emotion computing module and historical dialogue content tags in the intelligent dialogue module. Simultaneously, the number of tag repetitions within the period is correlated with the pet robot's specific feedback state to improve the probability distribution. For example, if the user emotion recognition result shows "surprise" multiple times within a week, and the historical dialogue content shows "why" multiple times, the probability value of "curiosity" in the pet robot's personality is increased, thereby achieving dynamic updates to the pet robot's personality. Subsequent human-computer interaction will then display more expressions, actions, and sound effects corresponding to the "curiosity" state.

[0082] The status control module 205 is used to match and select specific feedback expressions, actions and sound effects based on the generated pet robot feedback status in order to control the execution of the pet robot's feedback behavior.

[0083] In one embodiment, for the pet robot's feedback states, positive feedback such as activity, curiosity, and relaxation; neutral feedback such as alertness, hunger, and rest; and negative feedback such as anxiety, loneliness, and pain, a mapping table is constructed with feedback expressions, actions, and sound effects. Each state corresponds to multiple expressions, actions, and sound effects, and after being selected according to a certain distribution probability, they are handed over to the execution module in a programmed sequence.

[0084] Furthermore, the probability distribution can be initially set to a uniform distribution, and the probability distribution value can be optimized and adjusted based on user emotional feedback from multiple rounds of interaction.

[0085] The execution module 301 is used to deliver the pet robot's feedback expressions, movements and sound effects to the eye screen unit, the head and the motor units of each joint, and the speaker unit respectively, so as to realize human-computer interaction output.

[0086] In one embodiment, there are two eye screens, which can simultaneously display relevant facial expression animations or display different facial expression animations separately, depending on the feedback status. The speaker can be a single speaker for mono playback or multiple speakers for stereo playback.

[0087] Example 2:

[0088] Combined with appendix Figure 2 As shown, a method for controlling the state of a companion pet robot that dynamically updates its personality includes:

[0089] Sensing the user's current state includes observing the user's facial expressions and gestures through the camera, listening to the user's voice through the microphone, and acquiring the user's touch actions and locations through the touch sensor;

[0090] It can identify users' emotional states using multimodal emotion computing methods based on users' voices, facial expressions, etc., regardless of whether there is one or multiple modalities of perceptual data.

[0091] Generate various feedback states for the pet robot, including establishing a mapping relationship between the current user's emotion and the various feedback states of the pet robot, and also establishing a mapping relationship between the current pet robot's personality and the various feedback states of the pet robot. Both mapping relationship tables include the distribution probability of each feedback state. The distribution probability values ​​of the two mapping relationship tables are combined according to a specified weight ratio to form the final distribution probability of each feedback state, and the specific feedback state is selected and generated according to the final distribution probability.

[0092] Based on the selected specific feedback state, the system intelligently matches appropriate pet robot feedback expressions, actions, and sound effects from the built-in expression library, action library, and sound effect library.

[0093] The system synchronously controls the pet robot's eye screen to display matching feedback expressions, controls the motors of its head and joints to execute matching feedback action sequences, and controls the speaker to play matching feedback sound effects, thus achieving the overall state output of human-computer interaction.

[0094] Specific steps:

[0095] Step S01. The pet robot observes the user's facial expressions and collects video stream data through a camera. The perception module 101 provides the collected video stream to the emotion computing module 201.

[0096] Step S02. The pet robot listens to the user's voice through a microphone and collects audio stream data. The perception module 101 provides the collected audio stream to the emotion computing module 201. Steps S02 and S01 can be performed simultaneously.

[0097] Step S03. Multimodal emotion recognition based on vision and speech.

[0098] The emotion computing module 201 obtains the user's emotional state by visually recognizing the user's facial expressions based on the video stream data collected in step S01, and by speech recognizing the user's tone and content based on the audio stream data collected in step S02. Visual recognition and speech recognition, as two recognition modalities, can be combined into a multimodal approach or used as a single modal approach to calculate the user's emotional state.

[0099] Step S04. Obtain the mapping table between user emotions and pet robot states. See [link / reference] Figure 3This is a schematic diagram illustrating the probability distribution of user emotions and pet robot states according to one embodiment. This probability distribution forms a mapping table.

[0100] Step S05. Query the corresponding pet robot state distribution probability based on the user emotion recognition results.

[0101] Step S06. Obtain the state calculation weight setting parameters. The parameters include the weight values ​​of the probability distribution of the current user's emotion and the current pet robot's personality corresponding to the pet robot's state, and the sum of the two weight values ​​is 100%. In one embodiment, the parameters can be saved locally on the device by default, or the device can obtain updates from the server through the application programming interface.

[0102] Step S07. Obtain the mapping table between the current pet robot's personality and its state. See also Figure 4 This is a schematic diagram of the probability distribution of the built-in personality and state of a pet robot according to one embodiment. The probability distribution forms a mapping table.

[0103] Step S08. Calculate the final distribution probability of each state of the pet robot based on the weights, according to the distribution probability of each state of the pet robot corresponding to the current user's emotion and the current pet robot's personality.

[0104] In one embodiment, the pet robot includes nine states: active, curious, relaxed, alert, hungry, resting, anxious, lonely, and in pain. Therefore: the final probability distribution of each state = probability distribution of the current user's emotion corresponding to the current state × weight of the current user's emotion calculation + probability distribution of the current pet robot's personality corresponding to the current state × weight of the current pet robot's personality calculation.

[0105] Taking the current pet robot's personality as "optimistic and positive" and the user's emotion recognition result in step S05 as "happy" as an example, see [link / reference]. Figure 3 and Figure 4 In one embodiment, if both weight values ​​of the parameter in step S06 are 50%, then in the final state distribution probability, the distribution probability of the "active" state is 60%×50%+50%×50%=55%, the distribution probability of the "curious" state is 5%×50%+15%×50%=10%, and so on.

[0106] Step S09. Based on the final distribution probability, select the pet robot feedback status that should be output this time.

[0107] In one embodiment, the positive or negative bias relative to the final distribution probability can be calculated based on the actual number of outputs of each state in historical data, where positive indicates a higher number of outputs and negative indicates a lower number of outputs. Among the states with negative bias, one is randomly selected, for example, "active" is selected as the pet robot feedback state for output.

[0108] Step S10. Obtain the available expression, action, and sound effect library for the pet robot's feedback status.

[0109] In one embodiment, the library of facial expressions, actions, and sound effects corresponding to the feedback status can be saved locally on the device by default, and can also be updated by the device from the server through the application programming interface.

[0110] Step S11. Based on the feedback status, select the final output expressions, actions, and sound effects from the available expression, action, and sound effect library using an optimization algorithm.

[0111] In one embodiment, the preferred algorithm can be based on positive changes in user sentiment in historical feedback states.

[0112] Furthermore, positive changes include moving from negative to neutral emotions: for example, from "worry" to "calm"; from negative to positive emotions: for example, from "worry" to "joy"; and from neutral to positive emotions: for example, from "calm" to "joy".

[0113] Step S12. After arranging facial expressions, actions, and sound effects in a reasonable timing sequence, execute the output.

[0114] Step S21. Perform speech recognition on the acquired user voice and convert it into text.

[0115] In one embodiment, both steps S21 and S03 use the audio stream data collected in S02, and the two steps can be processed simultaneously.

[0116] Step S22. Process the text using a large model to generate the dialogue response content.

[0117] Step S23. Convert the reply to voice.

[0118] Step S31. Display feedback facial animations on the eye display screen.

[0119] Step S32. The action is performed by motors in the head and limbs.

[0120] Step S33. Play feedback sound effects and the reply voice from the intelligent dialogue module through the speaker.

[0121] In one embodiment, step S33 may combine the audio stream data of S12 and S23 and play them simultaneously through a speaker.

[0122] Example 3:

[0123] Combined with appendix Figure 5 As shown, a method for dynamically updating the status of a companion pet robot with dynamically updated personality includes the following steps:

[0124] Step S01. The pet robot observes the user's facial expressions through a camera. The perception module 101 provides the acquired video stream to the emotion computing module 201.

[0125] Step S02. The pet robot listens to the user's voice through a microphone. The perception module 101 provides the acquired audio stream to the emotion computing module 201. Steps S02 and S01 can be performed simultaneously.

[0126] Step S03. Multimodal emotion recognition based on vision and speech.

[0127] The emotion computing module 201 obtains the user's emotional state by visually recognizing the user's facial expressions based on the video stream data collected in step S01, and by speech recognizing the user's tone and content based on the audio stream data collected in step S02. Visual recognition and speech recognition, as two recognition modalities, can be combined into a multimodal approach or used as a single modal approach to calculate the user's emotional state.

[0128] Step S41. Save the user's historical emotional data.

[0129] The emotion computing module 201 saves historical data of the results of each emotion recognition. In one embodiment, this historical data can be synchronized by the device to a server for storage.

[0130] Step S21. Perform speech recognition on the acquired user voice and convert it into text.

[0131] In one embodiment, both steps S21 and S03 use the audio stream data collected in S02, and the two steps can be processed simultaneously.

[0132] Step S51. Save the history data of the intelligent dialogue.

[0133] The intelligent dialogue module 203 saves the historical text data of each dialogue. In one embodiment, the historical data can be synchronized by the device to a server for storage.

[0134] Step S61. Generate tag data based on historical data and according to the specified time period.

[0135] In one embodiment, the personality dynamic update module 204 generates tag data based on the historical data of emotion recognition results from the emotion calculation module 201 and the historical dialogue text data from the intelligent dialogue module 203, with a time period of one natural week, and counts the number of each tag. For example, the count of the emotion recognition result tag "thinking" is 32, and the count of the dialogue text tag "why" is 57.

[0136] Step S62. Obtain the mapping relationship between tags and pet robot status.

[0137] The personality dynamic update module 204 obtains the mapping relationship between each tag and the pet robot's state. For example, the emotion recognition result tag "thinking" and the dialogue text tag "why" are both mapped to the pet robot's state "curious". In one embodiment, the state mapping relationship can be saved locally on the device by default, or it can be updated by the device from the server through the application programming interface.

[0138] Step S63. Based on the count of each tag, query and statistically analyze the new distribution of the status of each pet robot corresponding to the tag.

[0139] Step S64. Based on the new state distribution, compare it with the current state distribution corresponding to the pet robot's personality, and generate pet robot personality correction data according to the specified rules.

[0140] In one embodiment, the specified rule is that if the probability distribution of each state increases or decreases by 3%, the personality correction data will increase or decrease by 1 accordingly; at the same time, the maximum correction amount for a single time period can be limited for the personality correction data, for example, ±3.

[0141] Step S65. Update the pet robot's personality data according to the personality correction data, and save it as the new current pet robot personality data.

[0142] In one embodiment, the pet robot's personality data is updated at the end of a one-week period.

[0143] This invention enhances the personalized interactive experience of pet robots by combining multimodal emotion perception with the robot's own personality to output specific feedback states. Simultaneously, through a dynamic personality update mechanism, the pet robot can continuously evolve, improving the emotional companionship experience for users over the long term.

[0144] Although the present invention has been described herein with reference to illustrative embodiments, the above embodiments are merely preferred embodiments of the present invention, and the implementation of the present invention is not limited to the above embodiments. It should be understood that those skilled in the art can devise many other modifications and implementations, which will fall within the scope and spirit of the principles disclosed in this application.

Claims

1. A dynamic personality-updating status control system for a companion pet robot, characterized in that, include: The system consists of a perception module, an emotion computing module, a feedback state generation module, a state control module, and an execution module, all connected in sequence. It also includes a perception module, an intelligent dialogue module, a personality dynamic update module, and a feedback state generation module, all connected in sequence. Furthermore, the emotion computing module is also connected to the personality dynamic update module, and the intelligent dialogue module is also connected to the execution module. The sensing module is used to obtain the user's current state and includes at least a microphone, camera, and touch sensor as sensing units. The emotion computing module is used to identify the user's emotional state from the perceived user's voice and facial expressions; The feedback status generation module combines the current user's emotional state and the current pet robot's personality according to a weighted ratio to generate the pet robot's feedback status. The intelligent dialogue module is used to recognize the content and semantics of the user's speech, process and generate response content based on the context of the speech, and then convert it into speech to achieve intelligent dialogue; The personality dynamic update module is used to dynamically update the pet robot's personality based on historical user emotion recognition results and intelligent dialogue content. The status control module is used to match and select specific feedback expressions, actions and sound effects based on the generated pet robot feedback status in order to control the execution of the pet robot's feedback behavior; The execution module is used to distribute the pet robot's feedback expressions, movements, and sound effects to the eye screen unit, the head and the motor units of each joint, and the speaker unit, respectively, to achieve human-computer interaction output.

2. The status control system for a companion pet robot with dynamically updated personality according to claim 1, characterized in that, The microphone includes a ring of six microphones for sensing the direction of the user's speech; the touch sensors include multiple body parts of the robot.

3. The status control system for a companion pet robot with dynamically updated personality according to claim 1, characterized in that, The emotion computing module stores historical user emotion recognition results and tags these results as one of the dimensions for dynamic personality updates. And / or the intelligent dialogue module stores the user's historical dialogue content with the robot, and tags the historical dialogue content as one of the dimensions for dynamic personality updates.

4. The status control system for a companion pet robot with dynamically updated personality according to claim 1, characterized in that, The pet robot's feedback states include at least three categories: positive feedback, neutral feedback, and negative feedback. Positive feedback includes activity, curiosity, and relaxation; neutral feedback includes alertness, hunger, and rest; and negative feedback includes anxiety, loneliness, and pain.

5. The status control system for a companion pet robot with dynamically updated personality according to claim 1, characterized in that, The first mapping relationship established between each user emotion and the pet robot's feedback state is the probability distribution of the occurrence of various feedback states during each human-computer interaction, with the total probability being 100%. The second mapping relationship between the personality described for each pet robot and the pet robot's feedback state is the probability distribution of the occurrence of various feedback states during each human-computer interaction, with the total probability being 100%. Furthermore, the combined weight of the current user's emotion and the current pet robot's personality is 100%.

6. A method for controlling the state of a companion pet robot with dynamically updated personality, characterized in that, include: Sensing the user's current state includes observing the user's facial expressions and gestures through the camera, listening to the user's voice through the microphone, and obtaining the user's touch actions and locations through the touch sensor; Identify user emotional states using multimodal sentiment analysis methods based on user voice and facial expressions. Generate each feedback state of the pet robot, including establishing a first mapping relationship between the current user's emotion and each feedback state of the pet robot, and establishing a second mapping relationship between the current pet robot's personality and each feedback state of the pet robot. Both mapping relationship tables include the distribution probability of each feedback state. The distribution probability values ​​of the two mapping relationship tables are combined according to a specified weight ratio to form the final distribution probability of each feedback state, and the specific feedback state is selected according to the final distribution probability. Based on the selected specific feedback state, the system intelligently matches appropriate pet robot feedback expressions, actions, and sound effects from the built-in expression library, action library, and sound effect library. The system synchronously controls the pet robot's eye screen to display matching feedback expressions, controls the motors of its head and joints to execute matching feedback action sequences, and controls the speaker to play matching feedback sound effects, thus achieving the overall state output of human-computer interaction.

7. The method for controlling the state of a companion pet robot with dynamically updated personality according to claim 6, characterized in that, The control method specifically includes the following steps: Step S01. The pet robot observes the user's facial expressions and collects video stream data through a camera; Step S02. The pet robot listens to the user's voice through the microphone and collects audio stream data; Step S03. Visual and speech-based multimodal emotion recognition: Based on the video stream data collected in step S01, visual facial expressions of the user are recognized; and based on the audio stream data collected in step S02, the timbre and content of the user's speech are recognized to obtain the emotional state. Step S04. Obtain the mapping table between user emotions and pet robot states; Step S05. Query the corresponding pet robot state distribution probability based on the user's emotion recognition results; Step S06. Obtain the state calculation weight setting parameters; the parameters include the weight values ​​of the current user's emotion and the current pet robot's personality, which correspond to the probability distribution of the pet robot's state, and the sum of the two weight values ​​is 100%; Step S07. Obtain the mapping table between the current pet robot's personality and the pet robot's state; Step S08. Calculate the final probability distribution of each state of the pet robot based on the weights, according to the probability distribution of each state of the pet robot corresponding to the current user's emotion and the current pet robot's personality. Step S09. Based on the final probability distribution, select the pet robot feedback status that should be output this time; Step S10. Obtain the library of facial expressions, actions, and sound effects available for the pet robot's feedback status; Step S11. Based on the feedback status, select the final output emoticons, actions, and sound effects from the available emoticon, action, and sound effect library; Step S12. After arranging facial expressions, actions, and sound effects in a reasonable timing sequence, execute the output.

8. The method for controlling the state of a companion pet robot with dynamically updated personality according to claim 7, characterized in that, The control method further includes the following steps: Step S21. Perform speech recognition on the acquired user voice and convert it into text; Step S22. Process the text using a large model to generate response content for the dialogue; Step S23. Convert the reply to voice; Step S31. Display feedback facial animations on the eye display screen; Step S32. The action is performed by motors in the head and limbs; Step S33. Play feedback sound effects and the reply voice from the intelligent dialogue module through the speaker.

9. The method for controlling the state of a companion pet robot with dynamically updated personality according to claim 6, characterized in that, The control method further includes: a dynamic update mechanism; The dynamic update mechanism includes: the system has a built-in default pet robot personality; acquiring historical user emotion recognition results and historical intelligent dialogue content; constructing pet robot personality-related tags, and performing data analysis on historical user emotion recognition results and historical intelligent dialogue content within a specified time period, and updating the tag data; at the end of each time period, updating the probability values ​​of the distribution of each feedback state involved in the current pet robot personality according to the corresponding tag data and a specified mapping relationship, so as to realize the dynamic update of the pet robot personality.

10. The method for controlling the state of a companion pet robot with dynamically updated personality according to claim 9, characterized in that, The dynamic update mechanism includes the following steps: Step S41. Save the user's historical emotional data; Step S51. Save the intelligent dialogue history data; Step S61. Based on the historical data saved in steps S41 and S51, generate tag data according to the specified time period; Step S62. Obtain the mapping relationship between tags and pet robot states; Step S63. Based on the count of each tag, query and statistically analyze the new distribution of the status of each pet robot corresponding to the tag; Step S64. Based on the new state distribution, compare it with the current state distribution corresponding to the pet robot's personality, and generate pet robot personality correction data according to the specified rules; Step S65. Update the pet robot's personality data according to the personality correction data, and save it as the new current pet robot personality data.

Images