Method and system for realizing deep interaction of virtual electronic pet

Abstract

The invention relates to a deep interaction implementation method and system for a virtual electronic pet, and the method comprises the steps: achieving the local semantic deep analysis based on a Raspberry Pi 4B hardware platform through the fusion of a large language model, and constructing a dynamic behavior optimization mechanism through the combination of an Alicloud sentiment analysis API and a reinforcement learning algorithm; the system can generate emotional adaptability response in real time according to the emotional state of the user; a sound and picture synchronization engine ensures that Coqui-TTS speech synthesis and OLED animation rendering realize hardware-level synchronization, and meanwhile, an RAG vector database is established to store historical interaction characteristics so as to support personalized behavior memory; a modular PCB (printed circuit board) is adopted to design and expand a GPIO (general purpose input / output) interface to support sensor expansion, and dynamic load distribution of local basic functions and cloud complex processing is realized through an edge-cloud computing hybrid architecture, so that the technical bottlenecks in the aspects of intelligent depth, interaction continuity and expandability in the prior art are essentially solved.

Description

Technical Field

[0001] The present invention relates to the technical field of virtual pet simulation, and in particular to a method and system for realizing deep interaction of a virtual electronic pet. Background Art

[0002] The development of virtual pet technology began in the 1990s, with games like Tamagotchi and Tamagotchi. Their core functionality, feeding and cleaning, was achieved through simple button interactions, pioneering digital companionship. With the widespread adoption of the internet and smart devices, virtual pets shifted from hardware to software applications, exemplified by Neopets in the early 2000s and Talking Tom Cat in 2009. The latter enhanced the user experience through multimodal interaction (such as display screen response and voice imitation), but the depth of interaction was limited by pre-set scripts, resulting in a low level of intelligence. In recent years, the integration of artificial intelligence (AI) has driven the development of virtual pets towards intelligence and emotional expression. For example, the intelligent robot Anki Vector supports simple conversational expressions, but its limited functionality, limited scalability, and high cost have limited its application. VR / AR products like the metaverse Horizon Worlds offer immersive experiences, but rely on high-performance devices and pose security risks (such as player addiction).

[0003] In summary, existing virtual pet technology has the following defects: mobile application products (such as Talking Tom Cat) only support preset actions and simple voice imitation but lack deep semantic understanding capabilities; the language models of intelligent robot products do not integrate emotion recognition technology, resulting in a lack of emotional adaptability in interactive feedback; VR / AR products have a serious disconnect between voice interaction and behavioral actions, resulting in a fragmented multimodal experience.

[0004] Based on this, this case is brought forward. Summary of the Invention

[0005] One of the purposes of the present invention is to provide a method for realizing deep interaction of a virtual electronic pet, which essentially solves the technical bottlenecks of the prior art in terms of intelligence depth, interaction continuity and scalability.

[0006] In order to achieve the above object, the technical solution of the present invention is as follows:

[0007] A method for implementing deep interaction of a virtual electronic pet comprises the following steps:

[0008] S10. Get the information entered by the user for the first time;

[0009] S20. Converting the input information into text or instructions recognizable by the system;

[0010] S30. Perform sentiment analysis based on the converted text or instruction in S20 to generate a text or instruction for the reply content;

[0011] S40. According to the text or command in reply content S30, converted into voice and virtual electronic pet's actions and expressions to express to the user;

[0012] S50. Get the information that the user enters again based on the reply content;

[0013] S60. Convert the re-entered information into text or instructions that the system can recognize;

[0014] S70. Perform an initial emotional screening to determine the user's emotional state and use this emotional state as a contextual prompt for the next round of dialogue;

[0015] S80. Perform sentiment analysis based on the converted text or instruction in S60 and the prompt word formed by the initial emotional screening in S70 to generate a text or instruction for the reply content;

[0016] S90. According to the text or command in the reply content of S80, it is converted into voice and the action and expression of the virtual electronic pet to express to the user;

[0017] S100. Loop steps S50 to S90 until the user ends the conversation.

[0018] Furthermore, the information input by the user includes any one or more of the user's voice, gestures, text, expressions, and body movements.

[0019] Furthermore, the information input by the user is voice information. In step S20 or S60, the Whisper model is used to convert the voice information into text that can be recognized by the system.

[0020] Furthermore, the sentiment analysis in step S30 or S80 adopts the LLaMA-2 language model.

[0021] Furthermore, the initial emotional screening includes the following steps: in the previous round of dialogue, the reply content is a sentence containing punctuation marks at the end of the sentence. According to the punctuation marks, the tone is judged by querying the comparison table to determine the user's emotional state, and the emotional state is used as the context prompt word for the next round of dialogue.

[0022] Furthermore, the content of each conversation is saved and a summary is generated as the basis for the next sentiment analysis and response content generation.

[0023] A second object of the present invention is to provide a virtual electronic pet deep interaction system based on the above-mentioned virtual electronic pet deep interaction implementation method, the system comprising a PCB, a Raspberry Pi 4B is used as a controller on the PCB, and an interaction module is integrated;

[0024] The controller includes:

[0025] Information conversion module 1, used to convert user input information into text or instructions that can be recognized by the system;

[0026] Information conversion module 2 is used to convert the text or instructions replied by the system into voice and the movements and expressions of the virtual electronic pet, and express them to the user through the display screen and speaker;

[0027] The system includes a cloud server, which includes a sentiment analysis model. The cloud server performs sentiment analysis based on the converted text or instructions to generate text or instructions for reply content.

[0028] Furthermore, the controller includes an emotion screening module, which judges the tone based on the punctuation mark query table in the previous round of dialogue, determines the user's emotional state, and uses this emotional state as the context prompt word for the next round of dialogue and uploads it to the cloud server.

[0029] Furthermore, the cloud server includes a database and a summary model. The database is used to save the content of each conversation between the system and the user. The summary model generates summary content for each conversation content as a basis for the next sentiment analysis and reply content generation.

[0030] Furthermore, the interaction module includes at least a display screen, a touch button, a microphone and a speaker.

[0031] The advantages of the present invention are:

[0032] 1. Efficient Hardware Architecture and Performance Advantages: The device utilizes a Raspberry Pi 4B as its core hardware, achieving compact modular integration through an innovative PCB design, simplifying the connection of peripherals such as displays, buttons, and audio modules. Compared to existing products, this design significantly improves operational efficiency: the Raspberry Pi 4B's quad-core processor supports multi-tasking parallel processing (such as real-time audio acquisition and network communication) while maintaining low power consumption, addressing the reliance of traditional virtual pet products (such as VR devices) on high-performance local hardware. A cloud-based collaborative mechanism (Alibaba Cloud API + GPU server) further offloads complex computations (such as sentiment analysis and content generation, including summarization), ensuring the system remains stable and responsive even under high loads.

[0033] 2. Intelligent interaction technology fusion: The device integrates multiple AI technologies, uses the Whisper model to achieve high-precision speech-to-text for voice interaction, and uses the fine-tuned LLaMA-2 model combined with reinforcement learning to dynamically optimize response strategies for emotion recognition; the RAG database supports historical interaction memory. Compared with existing products, the system realizes personification companionship through multi-modal collaboration (voice, animation, and text feedback). Tests show that the emotion recognition module can accurately capture user emotions (such as triggering comforting responses when feeling down) and enhance user stickiness through personalized adaptation (such as adjusting game content based on user preferences).

[0034] 3. User-centered function design and practicality: The device provides a full-process closed-loop interaction with low-cost hardware (Raspberry Pi solution). It covers voice input, emotion recognition, intelligent response, and multi-modal output, supports dynamic expressions and cat-style speech synthesis, and users can customize the appearance and voice style. Multi-user mode ensures data isolation. Compared with high-priced devices such as Anki Vector, this design reduces the threshold for use, especially for urban dwellers or users with limited space, solving the high-cost problem of traditional pet care. The closed-loop design breaks through the limitations of fragmented functions of existing products.

[0035] 4. Social value and technology promotion potential: The system meets the needs of special groups through emotional companionship, and tests show that it can effectively relieve stress. Cross-platform compatibility expands application scenarios. The modular architecture facilitates future expansion, setting a new paradigm for intelligent companion devices. Actual deployment has verified its stability in diverse environments, making it more versatile than existing products (such as VR devices relying on specialized hardware). BRIEF DESCRIPTION OF DRAWINGS

[0036] Figure 1 To construct a schematic diagram in the embodiment. DETAILED DESCRIPTION

[0037] The present application will be further described in conjunction with the embodiments. It should be understood that the terms "upper", "lower", "front", "back", "left", "right", "top", "bottom", "inner", "outer" and the like indicate the orientation or positional relationship shown in the drawings, and are only used to facilitate the description of the present application and simplify the description, and therefore cannot be understood as indicating or implying that the devices or elements must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the present application.

[0038] The present embodiment proposes a deep interaction implementation method for a virtual electronic pet, comprising the following steps:

[0039] S10. Obtain the information input by the user for the first time;

[0040] S20. Converting the input information into text or instructions recognizable by the system;

[0041] S30. Perform sentiment analysis based on the text or instruction converted in S20 to generate a text or instruction for the reply content;

[0042] S40. According to the text or command in reply content S30, converted into voice and virtual electronic pet's actions and expressions to express to the user;

[0043] S50. Get the information that the user enters again based on the reply content;

[0044] S60. Convert the re-entered information into text or instructions that the system can recognize;

[0045] S70. Perform an initial emotional screening to determine the user's emotional state and use this emotional state as a contextual prompt for the next round of dialogue;

[0046] S80. Perform sentiment analysis based on the converted text or instruction in S60 and the prompt word formed by the initial emotional screening in S70 to generate a text or instruction for the reply content;

[0047] S90. According to the text or command in the reply content of S80, it is converted into voice and the action and expression of the virtual electronic pet to express to the user;

[0048] S100. Loop steps S50 to S90 until the user ends the conversation.

[0049] In step S10, the information input by the user includes any one or more of the user's voice, gestures, text, expressions, and body movements. In this embodiment, the information input by the user is preferably voice information, so in step S20 or S60, the Whisper model is used to convert the voice information into text that can be recognized by the system. The Whisper model is based on the Transformer architecture and includes two main parts: an encoder and a decoder. In the Whisper model, the input voice signal is converted into a Log-Mel spectrogram. The encoder extracts features from the Log-Mel spectrogram through a multi-layer Transformer block (including a multi-head self-attention mechanism and a feedforward neural network) to generate a semantic representation. The decoder uses a sequence-to-sequence learning method, combined with multi-task training (such as language recognition, timestamp prediction, etc.), to gradually generate a target text sequence. During the decoding process, the Whisper model combines the output of the encoder with the current decoding state through a multi-head cross-attention mechanism, and finally outputs the recognized text content, for example, converting the user's voice "Kitty, I am unhappy and depressed today" into corresponding text.

[0050] Of course, when the user inputs other types of information such as gestures, text, expressions, body movements, etc., corresponding models can also be selected for conversion, such as multimodal deep learning models, visual technology models, etc.

[0051] For sentiment analysis in step S30 or S80, this embodiment adopts the LLaMA-2 language model, which is used to provide sentiment analysis and content generation APIs, and can be combined with Alibaba Cloud services to enhance emotion recognition capabilities.

[0052] In step S70, the emotional screening includes the following steps: in the previous round of dialogue, the reply content is a sentence containing punctuation marks at the end of the sentence. According to the punctuation marks, the tone is judged by querying the comparison table to determine the user's emotional state, and the emotional state is used as the context prompt word for the next round of dialogue. In this embodiment, the emotional screening plays the role of preliminary analysis of the user's emotions, which can improve the speed and accuracy of subsequent emotional analysis, thereby generating more personalized and thoughtful replies, thereby improving the user experience. For example, in the previous round of dialogue, the last punctuation mark of the reply of the LLaMA-2 language model is an exclamation mark, indicating an emotion of surprise. Then, the user's emotions can be reversely inferred, and the user may have a confused emotion. The inferred emotion is then fed back to the LLaMA-2 language model and integrated into the next round of dialogue. By deploying a comparison table or mapping table of sentence-end punctuation marks and emotional state prompt words in the system, the above-mentioned emotional screening process can be achieved.

[0053] Since the LLaMA-2 language model has a limited token (minimum unit) input, an excessively long context will cause the attention effect of the large model to decrease. This embodiment adds a small model in parallel with the LLaMA-2 language model, namely the summary model. The summary model saves the content of each conversation and generates a summary content as the basis for the next sentiment analysis and reply content generation. Replacing lengthy historical conversation information with simple summary content and inputting it into the LLaMA-2 language model can improve the accuracy and speed of emotion recognition and enhance user experience.

[0054] For the above method, this embodiment also provides Figure 1The system architecture shown in the figure below comprises three layers: human-computer interaction, embedded exploration layer, and cloud server. A modular design enables clear division of labor and efficient collaboration among these components. The system utilizes a layered architecture, organically combining user interaction, local processing, and cloud computing to ensure a balance between intelligent functionality and real-time requirements. The human-computer interaction interface serves as the front-end entry point, the embedded exploration layer as the intermediate processing layer, and the cloud server as the back-end support. These three layers utilize the HTTP protocol for data exchange, ensuring overall system stability and scalability. The design emphasizes the coordination between user experience and technical implementation, fully embodying the development philosophy of modern intelligent systems.

[0055] As the carrier of the embedded exploration layer, the hardware of this architecture adopts a modular PCB integrated design, using a Raspberry Pi 4B as the core controller (CPU: 1.5GHz quad-core ARM Cortex-A72, memory: 2GB / 4GB LPDDR4), connected through GPIO pins, integrating multiple interactive modules to achieve human-computer interaction, and realizing compact hardware modules through PCB optimization layout, reducing signal interference and improving response stability.

[0056] The Raspberry Pi 4B's quad-core processor supports multi-tasking parallel processing (such as real-time audio acquisition and network communication) while maintaining low power consumption, solving the reliance of traditional virtual pet products (such as VR devices) on high-performance local hardware. The controller includes information conversion module 1, information conversion module 2, and an emotion screening module. Information conversion module 1 includes the Whisper model, which is used to convert voice information into text that the system can recognize; information conversion module 2 uses Coqui-TTS, which is used to convert the text or instructions replied by the system into voice and the movements and expressions of the virtual electronic pet; the emotion screening module can determine the user's emotional state by setting a local comparison table between punctuation marks and prompt words, extracting the end-of-sentence punctuation marks of the reply content in the previous round of conversation, and judging the tone by querying the comparison table. This is then used as the context prompt word for the next round of conversation and uploaded to the cloud server.

[0057] In this embodiment, the interaction module includes at least an OLED display screen, a touch button, a microphone and a speaker.

[0058] The cloud server includes an LLaMA-2 sentiment analysis model. According to the converted text or instructions, the sentiment analysis is performed to generate reply content text or instructions. As an optimization, the cloud server further includes a RAG vector database and a summary model. The summary model can be a Seq2Seq model with an attention mechanism, which can better focus on important parts of the dialogue, or a machine learning model, etc. The summary model is preferably a lightweight model to improve running speed. In the cloud server, the summary model generates summary content for each dialogue content, which serves as the basis for the next sentiment analysis and reply content generation. The RAG vector database not only remembers the user's past dialogue content, but also enhances the relevance and personalization of the response through an efficient retrieval mechanism.

[0059] Application process example:

[0060] User voice input: I am a little unhappy today.

[0061] The system converts the input information and uploads it to the cloud server. The language model of the cloud server analyzes and replies, "What's wrong, my master?"

[0062] The system converts the reply information and sends instructions to the virtual electronic pet to express a puzzled expression (such as blinking).

[0063] User voice input: I had an argument with my friend today.

[0064] The system preliminarily judges that the user's mood is not good according to the punctuation mark "?" at the end of the last reply.

[0065] The system converts the input information and uploads it to the cloud server along with the hint word "mood is not good".

[0066] The cloud server replies, "I am very sorry to hear that", and instructs the virtual electronic pet to make a sympathetic expression (such as drooping ears and blinking eyes to show concern), which is output to the user through the speaker and display screen.

[0067] In the actual deployment of the intelligent pet, the method described in this embodiment is verified to be effective through the following implementation scheme:

[0068] In the voice interaction scenario, the reliability of voice recognition (Whisper model) and synthesis (SummerTTS) in complex environments is tested, covering accuracy, noise resistance, and real-time performance. 100 instructions are recorded in 3 types of environments (quiet indoor / 65dB background noise / cross language), and noise is injected using standard audio analysis tools to verify the recognition effect of the voice recognition module.

[0069] In the human-computer dialogue scene, the verification emotion recognition (LLaMA-2 model) and the cooperative reliability of the multi-modal output (voice, expression, action) are verified. By inputting 5 categories of emotional texts (such as "sadness" and "excitement"), it is checked whether the system outputs matching voice tone and animation. The matching degree is quantified by artificial evaluation + sensor data. Its usability is proved by evaluating the consistency of emotional feedback and the abnormal recovery ability.

[0070] The above examples are only used to explain the concept of the present application, and are not limited to the protection of the present application. Any non-essential modification of the present application using this concept shall fall within the scope of protection of the present application.

Claims

1. A method for realizing deep interaction of a virtual electronic pet, characterized in that: The following steps are involved: S10. Get the information entered by the user for the first time; S20. Converting the input information into text or instructions recognizable by the system; S30. Perform sentiment analysis based on the text or instruction converted in S20 to generate a text or instruction for the reply content; S40. According to the text or command in reply content S30, converted into voice and virtual electronic pet's actions and expressions to express to the user; S50. Get the information that the user enters again based on the reply content; S60. Convert the re-entered information into text or instructions that the system can recognize; S70. Perform an initial emotional screening to determine the user's emotional state and use this emotional state as a contextual prompt for the next round of dialogue; S80. Perform sentiment analysis based on the converted text or instruction in S60 and the prompt word formed by the initial emotional screening in S70 to generate a text or instruction for the reply content; S90. According to the text or command in the reply content of S80, it is converted into voice and the action and expression of the virtual electronic pet to express to the user; S100. Loop steps S50 to S90 until the user ends the conversation.

2. A method for implementing deep interaction of a virtual electronic pet according to claim 1, characterized in that: The information input by the user includes any one or more of the user's voice, gestures, text, expressions, and body movements.

3. A method for implementing deep interaction of a virtual electronic pet according to claim 1, characterized in that: The information input by the user is voice information. In step S20 or S60, the Whisper model is used to convert the voice information into text that can be recognized by the system.

4. A method for implementing deep interaction of a virtual electronic pet according to claim 1, characterized in that: The sentiment analysis in step S30 or S80 adopts the LLaMA-2 language model.

5. The method for realizing deep interaction of a virtual electronic pet according to claim 1, wherein: The initial emotional screening includes the following steps: in the previous round of dialogue, the reply content is a sentence containing punctuation marks at the end of the sentence, and the tone is judged based on the punctuation marks by querying a comparison table to determine the user's emotional state, and the emotional state is used as the context prompt word for the next round of dialogue.

6. A method for implementing deep interaction of a virtual electronic pet as claimed in claim 1, characterized in that: The content of each conversation is saved and a summary is generated as the basis for the next sentiment analysis and response content generation.

7. A virtual electronic pet deep interaction system based on the method for implementing deep interaction of a virtual electronic pet according to any one of claims 1 to 6, characterized in that: The system includes a PCB, which uses a Raspberry Pi 4B as a controller and integrates an interactive module; The controller includes: Information conversion module 1, used to convert user input information into text or instructions that can be recognized by the system; Information conversion module 2, used to convert the text or instructions replied by the system into voice and the movements and expressions of the virtual electronic pet; The system includes a cloud server, which includes a sentiment analysis model. The cloud server performs sentiment analysis based on the converted text or instructions to generate text or instructions for reply content.

8. The virtual electronic pet deep interaction system according to claim 7, characterized in that: The controller includes an emotion screening module, which determines the tone of voice based on the sentence-ending punctuation marks of the reply content in the previous round of dialogue by querying a comparison table, thereby determining the user's emotional state, and uses this emotional state as a context prompt word for the next round of dialogue and uploads it to the cloud server.

9. The virtual electronic pet deep interaction system according to claim 8, characterized in that: The cloud server includes a database and a summary model. The database is used to save the content of each conversation between the system and the user. The summary model generates summary content for each conversation content as the basis for the next sentiment analysis and response content generation.

10. The virtual electronic pet deep interaction system according to claim 7, characterized in that: The interactive module at least includes a display screen, touch buttons, a microphone and a speaker.

Images