Asynchronous anthropomorphic communication method and system based on voice transfer

Abstract

The invention provides an asynchronous anthropomorphic communication method and system based on voice transfer, which are suitable for various terminals such as wearable equipment, earphones, dolls, bolsters and the like. The method comprises the steps of user voice input, edge or cloud recognition, playback confirmation, content translation and voice synthesis, asynchronous transmission and broadcast and the like. The system does not depend on a specific hardware form, emphasizes a user confirmation mechanism and anthropomorphic voice broadcast and supports multilingual translation and personalized voice styles, the communication process is bound based on a device ID or nickname identity, social account login is not needed, and interaction privacy security is guaranteed. The method is widely applicable to various asynchronous social application scenes such as children, old people, lovers, autism rehabilitation and the like. The system supports nickname binding and friend relationship establishment, users can complete social connection through voice instructions or two-dimensional codes, and controllability and interestingness of communication interaction are enhanced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of voice communication, human-computer interaction, edge computing and Internet of Things, and in particular to a social communication method and system supporting asynchronous voice interaction, content confirmation, personified broadcasting and cross-language communication, which is suitable for multiple types of terminal devices, including but not limited to wearable devices, dolls, figurines, earphones, smart cloth products, etc. BACKGROUND

[0002] Existing voice interaction devices such as smart speakers, educational robots, and translation earphones mainly focus on voice control, information query, real-time translation, etc., and most of them are based on "human-device" interaction. When used by children, the elderly or special groups, the following problems exist: Communication requires both parties to be online at the same time, and does not support asynchronous messaging and emotional delay expression; Lack of confirmation mechanism in the interaction process, prone to misidentification and miscommunication; The output voice lacks a personified style, making it difficult to create an immersive interaction atmosphere; Insufficient support for multiple languages, limiting cross-language social experience; Communication is based on mobile phone numbers or social accounts, which poses a risk of privacy leakage and external harassment; Parents or guardians lack control over the communication process.

[0003] Currently, there is no complete communication solution that integrates "edge recognition + content confirmation + personified broadcasting + asynchronous push + multi-language translation + account decentralization + security supervision" to meet the needs of different user groups for non-real-time, emotionally rich, safe and controllable voice communication. SUMMARY

[0004] The present application aims to provide a social communication method and system structure supporting asynchronous voice communication, user confirmation and personified broadcasting, suitable for wearable devices, dolls, earphones, clothing and other types of carriers. This method does not limit the processing location of voice recognition and voice synthesis (which can be located on the terminal or in the cloud), but through the triple mechanism of "content confirmation + personified voice output + asynchronous reach", it realizes a safe, controllable and emotionally expressive voice social communication method.

[0005] The key features of the present application include: Independent of specific device structure, deployable on clothes, earphones, hats, dolls, pillows, etc. User confirmation mechanism prevents incorrect information from being uploaded, improving interaction safety; Voice style can be customized to enhance emotional expression and immersion; Support for multi-language recognition and translation, barrier-free cross-cultural communication; Asynchronous voice delivery, supporting timing playing, silent transmission, confirmation return and other control strategies; The communication process is bound based on device ID or nickname identity, without mobile phone number or social account, so as to protect the privacy of users. The cloud supports content supervision and parental control function, and ensures that voice interaction is safe and reliable.

[0006] The application comprises the following core links: (1) Voice input collection: the user inputs voice content through a terminal device (including but not limited to wearable and embedded devices), and the device takes the original voice as a data starting point; (2) Voice recognition and synthesis: the voice data can complete semantic recognition, language translation and voice synthesis in a local terminal edge device or a cloud platform to form interactive content for feedback; (3) User confirmation mechanism: the recognition and processing results are returned to the user end, and the user confirms whether the expression is accurate and whether it needs to be modified, so as to prevent ambiguous information flow caused by misrecognition; (4) Confirmation uploading and storage: the voice data (which can include target language text, synthesized voice or multi-mode information package) confirmed by the user is uploaded to a cloud communication relay system and is bound to the identity of a target receiver; (5) Asynchronous voice delivery: the target receiver device pulls and plays in a self-defined voice style at a set time node or under an active trigger condition. The play has a personification feature, such as a child voice, an elderly voice, a cartoon character voice, a virtual character voice and the like, so as to enhance the emotional immersion experience.

[0007] The application system supports: Multi-language recognition and translation; Configurable voice style template; User identity is matched based on device ID or nickname, and does not depend on mobile phone or account system; Asynchronous strategies include timing playing, silent playing, delayed message, holiday greetings and the like.

[0008] The application system is particularly suitable for: Cross-language children socialization; Long-distance emotional communication between lovers; Daily companionship for the elderly; Emotional counseling for special groups such as autistic children and rehabilitation patients.

[0009] In order to adapt to different device computing power conditions and network environments, the application constructs a flexible data processing and communication path, supports a complete interactive link of "terminal-cloud-terminal", and mainly comprises the following two switchable recognition and confirmation paths:

[0010] One is edge recognition path: the sending end device completes voice recognition and semantic understanding through the local edge processing module, and replays the recognition result in a personified voice mode locally for the user to confirm.

[0011] Two is cloud recognition path: the sending end device directly uploads the original voice data to the cloud platform after encryption, and the cloud platform completes voice recognition, semantic analysis and voice synthesis, and then returns the recognition result to the user device through voice for confirmation.

[0012] The above paths rely on the communication module for information transmission, support mainstream protocols including but not limited to MQTT, HTTPS, WebSocket, etc., and ensure the safety and reliability of voice data in the recognition, confirmation, transmission, translation and broadcast links.

[0013] The application realizes the fusion goal of "emotional expression + safety control + cross-device asynchronous socialization" of voice communication based on the definition of behavior logic in the core process without relying on the device form, and has significant practicality and feasibility.

[0014] In order to realize the personalized expression and social identity recognition of the voice communication system, the application adopts a device user nickname binding mechanism and a safe and controllable social relationship establishment method.

[0015] User and device binding mechanism: Each terminal device (such as wearable devices, dolls, etc.) needs to complete the main user identity setting through voice or supporting platform setting in the first use or initialization stage: The user can input the device nickname (such as "Xiaoming") through voice, and the system binds it as the "main user nickname" of the device after voice recognition confirmation; Or set by the guardian through the supporting platform App, such as "device ID A binds nickname Xiaoming"; After the binding is completed, the nickname is used for cloud social graph establishment and identity recognition, and all communication interactions take the nickname as the personalized identification; The system supports unique nickname detection to prevent duplication, and supports later changes and identity unbinding.

[0016] User friend relationship establishment mechanism: The user can establish social relationships between devices (actually between users) in the following ways: Active addition: When a user says to a device, "I want to add Xiaoming as a friend," the system matches the device ID with "nickname=Xiaoming" in its social graph and sends a friend request to the target device. Friend recommendation: User A can recommend User C as a friend to User B. The device will announce in a human-like manner, "XX recommends YY to become your friend. Do you agree?" After B confirms, the binding is completed. Add via QR code: The device supports QR code or ID scanning. Users can establish a binding relationship by scanning the target device's QR code through the platform app. Security Mechanism: All friend requests require confirmation from the receiving device user to be effective. Guardians can set restrictions on the sources of friends they can add and set review rules.

[0017] Once the binding is complete, users can conduct asynchronous voice communication through the bound device. All communication content and identity are identified and forwarded based on "device nickname + encrypted ID", without exposing the user's real identity information.

[0018] This mechanism effectively ensures the safety and credibility of the social process, and also enhances the anthropomorphic atmosphere of the interaction and the sense of immersion in the system.

[0019] This invention, by constructing an asynchronous anthropomorphic voice communication method and system, solves the problems of high real-time performance, limited expression, stiff interaction, and poor security in existing technologies, and has the following beneficial effects: Enables asynchronous voice-based social interaction, allowing users to exchange information and interact without both parties being online simultaneously; A voice recognition result confirmation mechanism is adopted to prevent the direct transmission of misidentified information and improve the accuracy of interaction; The voice broadcast style is selectable, including children's voice, elderly voice, cartoon character voice, etc., to enhance the humanized and immersive experience; Supports multilingual input and output; the system can automatically recognize languages ​​and complete cross-language translation and broadcasting, making it suitable for cross-cultural user social interaction. User identity binding is based on device ID or nickname mapping, avoiding reliance on mobile phone numbers or social media accounts and improving privacy and security; the system has identity binding and friend recommendation mechanisms to ensure reasonable and secure communication paths; The cloud-based communication relay platform has content monitoring capabilities, allowing parents / guardians to review and control their children's communication content, thus reducing risks. The system is flexible in deployment, does not depend on specific physical devices, and is suitable for various terminals such as clothing, headphones, dolls, and pillows.

[0020] In summary, this invention can be widely applied to various scenarios such as children's social interaction, elderly companionship, couples' communication, and rehabilitation of children with autism, and has significant social value and industrial application prospects. Attached Figure Description

[0021] Figure 1 System structure diagram of the system described in this invention; Figure 2 Asynchronous voice interaction flowchart; Figure 3 Flowchart of the "User Binding Mechanism"; Figure 4 Flowchart of "Friend Recommendation Logic"; Detailed Implementation

[0022] To make the objectives, technical solutions, and beneficial effects of this invention clearer and more explicit, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. This invention is not limited to the specific embodiments described below; any equivalent substitutions or improvements made within the spirit and principle of this invention should be included within the scope of protection of this invention.

[0023] Example 1: Multi-Doll Voice Message System within a Family

[0024] In a family, the older sister Xiaoya, the younger brother Xiaozhi, and the parents each have a voice device (such as a doll, headphones, or voice backpack) that is associated with a nickname.

[0025] After Xiaoya got home from school, she said to her doll, "Little brother, don't forget to bring your English book to your schoolbag." The system recognized and linked the target to Xiaozhi.

[0026] The system automatically translates the audio in the cloud (e.g., if the older sister speaks Mandarin, the younger brother's device is set to Cantonese), and then plays the message the next morning through the younger brother's voice device: "Your sister is reminding you to bring your English book."

[0027] Parents can set guardianship permissions and review message content, which is suitable for asynchronous communication and collaboration within families with multiple children.

[0028] Example 2: Strangers' Mood Messages on the Voice Chat Wall

[0029] Deploy a "voice chat wall" in a public library or youth activity center, with multiple virtual doll cards that can be scanned and linked to the device.

[0030] The user scanned the QR code to bind card A and set the nickname "Chenxi" to leave a voice message: "I'm really annoyed lately, but I also want to try my best."

[0031] The system will review the message and then recommend it to another device that matches the nickname "Adao".

[0032] After the other party agreed, the device automatically played: "Chenxi said she has encountered some difficulties recently, but she is still trying."

[0033] We can build a voice-based, anonymous, and low-barrier-to-entry social mechanism for strangers, suitable for psychological catharsis and empathetic social interaction.

[0034] Example 3: School teachers use dolls to conduct group broadcasts.

[0035] The elementary school homeroom teacher used a voice-activated device to connect to the nickname "Teacher Li" and said to the student: "After the red group finishes their math exercises today, we will check the mistakes."

[0036] The system uses all student device IDs bound to the "Red Group" as the target group, translates the voice information into the language settings of the corresponding student device, and generates human-like voice.

[0037] After school, each member of the Red Team heard through their doll: "Teacher Li said, remember to check your math mistakes today."

[0038] The system records whether the video has been played and the user's response, which is used for teacher behavior tracking and closed-loop teaching reminders.

[0039] Example 4: Voice Broadcast Communication in Post-Disaster Resettlement Scenarios

[0040] At temporary resettlement sites in the disaster area, each child or elderly person is given a pre-configured voice device, which is automatically associated with the nickname "volunteer".

[0041] A volunteer announced over the loudspeaker: "It's going to rain tonight, so make sure to secure the tent openings tightly."

[0042] The system can asynchronously push broadcast information by device group and language distribution. When the doll broadcasts, it automatically adds a guiding text, such as: "The volunteer sister reminds you to be careful at the tent entrance when it rains tonight."

[0043] It is suitable for special scenarios where network infrastructure is unstable and people cannot be online in real time to provide security reminders, emotional support, or task communication.

[0044] Example 5: Non-Account-Based Stranger Social Interaction and Language Practice

[0045] When a user scans a QR code displaying a doll in the public exhibition area, the system prompts: "Would you like to communicate with this doll?"

[0046] Users set their nickname as "Xiaodong" and input "I want to learn Tibetan" via voice. The system then binds the nickname and guides users into the "Language Practice Community".

[0047] The platform linked Xiaodong with another user who used a Tibetan-speaking doll. The next day, Xiaodong received a message from the doll: "Hello from Xizang! Let's practice together."

[0048] It supports anonymous binding and does not expose mobile phone numbers, making it suitable for language interest groups or interactive scenarios at exhibitions.

[0049] Example 6: User and Device Nickname Binding and Friend Recommendation Process

[0050] User Xiaomei says to the doll A, "Your name is Xiaomei." After the system confirms via voice, it sets "Xiaomei" as the main user nickname for the device, thus completing the binding process.

[0051] Xiao Mei said to Doll A, "I want to recommend Xiao Ming to Xiao Jun." The system binds Xiao Ming's device ID to the recommendation information based on the user's social graph and pushes a voice message to Xiao Jun's device, "Would you like to become friends with Xiao Ming?"

[0052] After Xiaojun confirms, the system completes the mapping of the social relationships among the three, allowing them to communicate with each other.

[0053] Example 7: Nickname Binding and Voice Confirmation Method

[0054] When a user named Xiaoyi uses a voice device (such as headphones or a doll) for the first time, she says to the device, "Your name is Xiaoyi."

[0055] The system performs local speech recognition and plays back the voice in a human-like voice: "You want me to be called Xiaoyi, right?"

[0056] After the user answers "yes", the system will bind "Xiaoyi" to the current device's unique ID and generate a user nickname tag for that device.

[0057] Afterwards, all communication identities in the system will be identified as "Xiaoyi", and the binding information can be saved in the cloud and allowed to be modified or restored in the parent's account.

[0058] Example 8: QR code binding and personalized voice settings

[0059] After parents purchase a voice-activated pillow for their child, they can scan the QR code on the device packaging with their mobile phone to be automatically redirected to the binding page.

[0060] The system prompts: "Please name this device." The parent enters "Xiaoyu" on the page and selects "cartoon tone" as the voice style.

[0061] Once completed, the system will bind the nickname "Xiaoyu" to the device ID and configure the voice broadcast parameters.

[0062] When the child interacts with the device, the device automatically plays content in the role of "Xiaoyu," enabling personalization and linking with the family account.

[0063] Example 9: Voice Recommendation Mechanism in a Playroom

[0064] User Xiao An wanted to add her good friend Xiao Jie as a friend. She said to the doll, "I want to recommend Xiao Jie."

[0065] The system recognizes that "Xiao Jie" in the voice is another user nickname and searches the database for the device ID that is bound to that nickname.

[0066] Xiao Jie's doll broadcasts during its free time: "Xiao An wants to be your friend, would you like to?"

[0067] When Xiao Jie says "I do" to the doll, the system establishes a friend relationship mapping between the two devices, and subsequent voice communication is possible, with the ability to relay the message.

[0068] Example 10: Friend QR Code Invitation Mechanism

[0069] User Xiaotong generated a "friend QR code" in her doll system, and her parents helped her send the QR code to her cousin Xiaoyu who lived far away.

[0070] After Xiaoyu's parents scanned the QR code, the system prompted "Do you want to bind your friend Xiaotong?" and played a voice confirmation.

[0071] After clicking "Confirm," Xiaoyu's doll will announce during the designated time period: "Xiaotong is now your good friend."

[0072] The two devices establish a communication relationship, allowing interaction, messaging, and story sharing via asynchronous voice.

[0073] Example 11: Interactive Scenario in a Three-Person Doll Group Chat

[0074] Users Xiaomi, Xiaojie, and Xiaolei each have a voice-interactive avatar with the nicknames "Taotao," "Feifei," and "Mengmeng," respectively.

[0075] Xiaomi, through her doll "Tao Tao," said, "I want to leave a message for Fei Fei and Meng Meng. I won first place in the drawing competition at school today!"

[0076] After the doll "Tao Tao" confirms the voice content, it uploads the voice data to the cloud platform.

[0077] The cloud identifies the device ID corresponding to the recipients "Feifei" and "Mengmeng" and performs Chinese character recognition, emotion judgment, and voice style matching.

[0078] The platform is set up for synchronous scheduling, and broadcasts to "Feifei" and "Mengmeng" at 9:00 a.m. the following day, according to the time zone of the recipient and the set quiet time.

[0079] What Feifei heard was: "Taotao said: She won first place in the drawing competition today, that's great!"

[0080] What Mengmeng heard was: "Taotao sent a message: She won an award today and was just thinking of telling you all~"

[0081] This type of group interaction mechanism can support one-to-many social relationships, expand children's social networks, and is suitable for building children's social networks such as "class circles", "interest groups", and "sibling groups".

[0082] Example 12: Explanation of Terminal-Cloud-Terminal Communication Path and Data Interaction

[0083] User Xiaozhi uses the voice interaction module embedded in his smart earphones to say, "Tell Xiaolan that you did well on your exam today."

[0084] The earphone's built-in edge processing unit does not enable local voice recognition mode, so the raw voice data is encrypted and uploaded to the cloud via the MQTT protocol.

[0085] The cloud platform completes speech recognition, semantic parsing, and speech synthesis, and pushes the generated speech content back to Xiaozhi's terminal device, where the headset's voice module reads: "You should say to Xiaolan: 'Good luck on your exam today,' right?"

[0086] After Xiao Zhi listened and confirmed, the device sent a confirmation command to the cloud.

[0087] The cloud platform then sent the translated and synthesized anthropomorphic voice pack to Xiaolan's voice doll via HTTPS protocol.

[0088] Xiaolan's device is set to automatically broadcast at 3 PM, at which time the voice pack will repeat in a cartoonish style: "Xiaozhi said he did well on his exam today, and he wants you to be happy too."

Claims

1. An asynchronous anthropomorphic communication method based on speech paraphrasing, characterized in that, Includes the following steps: (1) The sending user inputs voice content through a wearable or embedded voice interaction terminal device; (2) The voice data is processed for semantic recognition and speech synthesis at the terminal or uploaded to the cloud; (3) The recognition result or synthesized speech is sent back to the user terminal for content confirmation; (4) The confirmed voice content is uploaded to the cloud communication platform and the recipient's device identity is bound; (5) The receiving device retrieves voice data according to the set time or triggering conditions and broadcasts it in a preset human-like voice style.

2. The method according to claim 1, characterized in that: The terminal device may take the form of, but is not limited to, clothing, hats, headphones, scarves, dolls, pillows, etc.

3. The method according to claim 1, characterized in that: The voice styles include, but are not limited to, children's voices, elderly voices, cartoon character voices, and virtual character voices.

4. The method according to claim 1, characterized in that: The communication process is based on device ID or user nickname for identity mapping and binding, without the need for mobile phone number or social media account.

5. The method according to claim 1, characterized in that: The speech recognition and synthesis can be performed either on a local terminal or uploaded to a cloud platform.

6. The method according to claim 1, characterized in that: The asynchronous delivery method includes various strategies such as timed sending, delayed broadcasting, silent transmission, and receiving receipt confirmation.

7. The method according to claim 1, characterized in that: The system supports multilingual recognition and translation, automatically identifying the source language and synthesizing the target language speech packet for transmission and broadcasting.

8. An asynchronous voice communication system constructed based on the method of any one of claims 1 to 7, characterized in that: It includes a voice interaction terminal, an edge processing module, a cloud recognition platform, a translation and speech synthesis module, a communication relay platform, an identity binding mechanism, and a voice broadcasting module.

9. The method according to claim 1, characterized in that: The binding relationship between the sender and the receiver is established through any of the following methods: voice confirmation, QR code scanning, or third-party user recommendation with voice confirmation by the receiver.

10. The method according to claim 1, characterized in that: The user establishes an identity mapping with the voice interaction device by setting a nickname, which is used to identify the user's source and support voice routing during communication.

11. The method according to claim 1, characterized in that, The speech recognition and semantic parsing process can be completed in the edge processing module of the sending device, or it can be completed by the cloud after the original speech data is uploaded to the cloud. The processing result is transmitted back to the sending device via voice for user confirmation.

12. The method according to claim 1, characterized in that, The communication module uses protocols such as MQTT, HTTPS, or WebSocket to transmit voice data and confirmation status between the sending end and the cloud, and completes identity mapping and communication routing based on device ID or nickname.

Images