AI digital wisdom interaction method and system

Abstract

The invention relates to an AI digital intelligence interaction method and system, and the method comprises the steps: monitoring the voice input of a user in an interaction state; recognizing the voice data of the user, and converting the voice data of the user into a question text; performing semantic correction on the question text through a semantic model in combination with historical question and answer interaction data in current interaction; inputting the question text after semantic correction into an artificial intelligence model to obtain a reply text output by the artificial intelligence model; according to the technical scheme, by deeply understanding the context background of the question asked by the user, the natural and smooth question asking text which better fits the intention of the user can be generated, and the continuity and immersion of user experience are greatly improved.

Description

Technical Field

[0001] The present application relates to the field of human-computer interaction technology, and in particular to an AI digital-human interaction method and system. Background Art

[0002] Digital humans are defined as digital people with evolved data intelligence capabilities and methods. Today, they have developed into a "digital human" framework system of "multi-source data + intelligent algorithms + domain experts". Digital humans not only have the strong monetization advantages of digital humans, but also have intelligent and interactive features, which makes them perform well in handling complex tasks and providing personalized services.

[0003] Digital people are digital characters that are close to human images and created using digital technology. They are based on speech recognition and speech synthesis technology, with large language models as the core of information processing, and use digital character video synthesis technology to optimize the human-computer interaction experience.

[0004] In the entire system, speech recognition is used as a module for information collection and front-end processing to complete the conversion of speech to text. The large language model is trained on massive language data and is proficient in understanding, generating and translating human language. It serves as the core of the digital human system and outputs text answers. The speech synthesis module converts the text content output by the large model into speech and passes it to the digital human video synthesis module for synthesis.

[0005] The defects of the prior art are:

[0006] The user conversation is directly input into the big model to obtain the reply text, but language is complex and changeable. It is difficult to ensure the accuracy and coherence of the reply text in the existing technology, and it is difficult to completely replace real interpersonal communication as a whole. Summary of the invention

[0007] In order to overcome, at least to some extent, the problem in related technologies that it is difficult to ensure the accuracy and coherence of reply texts, the present application provides an AI digital human interaction method and system.

[0008] The scheme of this application is as follows:

[0009] According to a first aspect of an embodiment of the present application, there is provided an AI digital human interaction method, comprising:

[0010] When in the interactive state, listen to the user's voice input;

[0011] Recognize the user's voice data and convert the user's voice data into question text;

[0012] Performing semantic correction on the question text by combining the semantic model with the historical question-answer interaction data in the current interaction;

[0013] Inputting the semantically corrected question text into the artificial intelligence model to obtain the answer text output by the artificial intelligence model;

[0014] The reply text is converted into audio and outputted through a digital character.

[0015] Preferably, the method further comprises:

[0016] When in standby mode and detecting that the user's voice input contains preset wake-up keywords, the system switches to interactive mode.

[0017] Preferably, the method further comprises:

[0018] When the digital character outputs audio and detects that the user's voice input contains preset interruption keywords, the audio output is stopped.

[0019] Preferably, after the audio output is stopped because the user's voice input contains a preset interruption keyword, the method further includes:

[0020] Outputs the preset interruption feedback reply.

[0021] Preferably, the method further comprises:

[0022] When in the interactive state, if the user's voice input is monitored to contain preset standby keywords, or if no question-and-answer interaction occurs within the set time, switch to the standby state.

[0023] Preferably, the method further comprises:

[0024] Obtain conversation scene corpora from various disciplines as training data;

[0025] Training the semantic model according to the training data;

[0026] The semantic model is used to perform semantic correction on the current interactive text according to the historical interactive data.

[0027] Preferably, the method further comprises:

[0028] After the interaction state ends, the question-answer interaction data generated in the interaction state is integrated and stored in the local knowledge base.

[0029] Preferably, the method further comprises:

[0030] According to the hot spots and trends of users' questions in the question-and-answer interaction data, adjust the weights and priorities of various types of data in the local knowledge base.

[0031] Preferably, the method further comprises:

[0032] The digital character is generated according to the custom interaction interface and interaction logic entered by the user.

[0033] According to a second aspect of an embodiment of the present application, an AI digital-human interaction system is provided, comprising:

[0034] Processor and memory;

[0035] The processor and the memory are connected via a communication bus:

[0036] Wherein, the processor is used to call and execute the program stored in the memory;

[0037] The memory is used to store a program, and the program is at least used to execute an AI digital human interaction method as described in any one of the above items.

[0038] The technical solution provided by this application may have the following beneficial effects:

[0039] The AI ​​digital-human interaction method in the present application includes: when in an interactive state, monitoring the user's voice input; recognizing the user's voice data and converting the user's voice data into a question text; semantically correcting the question text through a semantic model combined with historical question-and-answer interaction data in the current interaction; inputting the semantically corrected question text into an artificial intelligence model to obtain a reply text output by the artificial intelligence model; and converting the reply text into audio and outputting it through a digital character.

[0040] In this technical solution, in addition to the large model, a semantic model is also trained separately. This semantic model will perform semantic correction on the question text in human-computer interaction in combination with the historical question-and-answer interaction data in the current interaction. By deeply understanding the context of the user's question, it is possible to generate a question text that is more in line with the user's intention and natural and fluent, greatly improving the consistency and immersion of the user experience. This semantic correction strategy based on historical question-and-answer interaction data has made human-computer question-and-answer interaction no longer just a mechanical response, but has evolved into a warm and intelligent conversation experience.

[0041] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0043] Figure 1 This is a flowchart of an AI digital human interaction method provided by an embodiment of the present application;

[0044] Figure 2It is a structural diagram of an AI digital-human interaction system provided by an embodiment of the present application.

[0045] Reference numerals: processor-21; memory-22. DETAILED DESCRIPTION

[0046] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0047] Embodiment 1

[0048] Figure 1 This is a flowchart of an AI digital human interaction method provided by an embodiment of the present application, referring to Figure 1 , an AI digital human interaction method, comprising:

[0049] S11: When in the interactive state, monitoring the user's voice input;

[0050] It should be noted that the interactive state refers to one of the states of the system. When in the interactive state, the system can be regarded as being in a working state, and voice interaction with the user can be performed at this time.

[0051] It should be noted that when the system is in an interactive state, it needs to continuously monitor the user's voice input.

[0052] S12: Recognize the user's voice data and convert the user's voice data into question text;

[0053] In the whole system, speech recognition is used as a module for information collection and front-end processing to complete the conversion of speech to text. Speech recognition and text conversion are relatively conventional existing technologies and will not be described in detail here.

[0054] S13: semantically correct the question text by combining the semantic model with the historical question-answer interaction data in the current interaction;

[0055] It should be noted that the method also includes:

[0056] Obtain conversation scene corpora from various disciplines as training data;

[0057] Train the semantic model based on the training data;

[0058] The semantic model is used to semantically correct the current interaction text based on historical interaction data.

[0059] It should be noted that the AI ​​digital human in this technical solution refers to a digital character image that is close to the human image and created using digital technology.

[0060] In this technical solution, conversation scene corpus in various disciplines is obtained as training data to ensure that the system can support human-computer interaction in various disciplines. A large amount of daily conversation scene corpus is used to train the semantic model, which is used to rewrite and correct the speech recognition results so that the rewritten results are more inclined to written formal terms.

[0061] The key to the implementation of this technical solution is that the system can efficiently integrate the questions to be rewritten and their related historical question-answer interaction data as a reference for rewriting correction. In this way, the semantic model can combine the historical question-answer interaction data for accurate semantic analysis and reconstruction, thereby generating a question text that is both accurate and in line with the conversation context.

[0062] S14: inputting the semantically corrected question text into the artificial intelligence model to obtain a reply text output by the artificial intelligence model;

[0063] Artificial intelligence models are "big models", such as the Wenyan Yixin big model. With the continuous improvement of computer hardware performance and the rapid optimization of deep learning algorithms, the development of big models is changing with each passing day. A series of artificial intelligence applications based on big models have been launched one after another, among which ChatGPT and "Wenxin Yiyan" have had a wide impact on social production and life.

[0064] In this technical solution, the existing artificial intelligence model can be directly used.

[0065] In specific implementation, the semantically complete and formatted question text after the semantic model is rewritten is passed to the cloud-based artificial intelligence model for further processing to obtain the response result of the artificial intelligence model.

[0066] S15: Convert the reply text into audio and output it through the digital character.

[0067] In specific practice, a real-time voice cloning software based on SoVITS is built. After the artificial intelligence model generates a reply text, the reply text is converted into audio and sent to the digital character for playback.

[0068] Preferably, in the present technical solution, the data is transmitted in a streaming manner.

[0069] The data transmission process in this technical solution includes:

[0070] 1. In the process of converting the user's voice data into question text, the system builds a real-time streaming audio-to-text model based on Kaldi and Zipformer, which can convert the user's input audio into text while ensuring low latency.

[0071] 2. The system builds real-time voice cloning software based on SoVITS, which generates audio corresponding to the input text in a streaming form. Combined with the text responses generated by the artificial intelligence model in a streaming manner, it can convert them into audio responses with extremely low latency and transmit them to the digital character for video synthesis, making the entire dialogue interaction logic smoother.

[0072] In this technical solution, the streaming transmission method is adopted, which can significantly improve the speed of speech synthesis.

[0073] It should be noted that the method also includes:

[0074] Generate a digital character based on the custom interaction interface and interaction logic entered by the user.

[0075] In this technical solution, users are supported to customize the interactive interface and interactive logic of digital characters, which improves the flexibility of interactive strategies and the customization of interactions, and can better meet user needs.

[0076] In this technical solution, in addition to the large model, a semantic model is also trained separately. This semantic model will perform semantic correction on the question text in human-computer interaction in combination with the historical question-and-answer interaction data in the current interaction. By deeply understanding the context of the user's question, it is possible to generate a question text that is more in line with the user's intention and natural and fluent, greatly improving the consistency and immersion of the user experience. This semantic correction strategy based on historical question-and-answer interaction data has made human-computer question-and-answer interaction no longer just a mechanical response, but has evolved into a warm and intelligent conversation experience.

[0077] Embodiment 2

[0078] In this embodiment, other states of the system are described.

[0079] It should be noted that the method also includes:

[0080] When in standby mode and detecting that the user's voice input contains preset wake-up keywords, the system switches to interactive mode.

[0081] The system presets two operating states: standby state and interactive state. Preferably, each state also corresponds to different action posture videos of digital characters and is matched with corresponding background interactive logic.

[0082] The state switching logic is as follows:

[0083] Automatically switch to the corresponding state according to the user's questions and the current operating state. For example, in the standby state, it continuously monitors the ambient sound, and when it monitors the user's voice input containing the preset wake-up keyword, it switches to the interactive state. The system switches from the idle state to the interactive state and responds when entering the interactive state, such as answering "Are you there?" or "Hello."

[0084] Configure a specific wake-up word (such as "Xiaoyu Xiaoyu."), and the user can start the digital character service by speaking the wake-up word. After the digital character is awakened, it can support continuous conversation, without the user having to say the wake-up word before each conversation, reducing the number of times the user has to wake up repeatedly.

[0085] It should be noted that the method also includes:

[0086] When the digital character outputs audio and detects that the user's voice input contains preset interruption keywords, the audio output is stopped.

[0087] The human-computer interaction strategy in this technical solution also includes an interruption mechanism. During the digital character's answering process, if the user says specific interruption keywords, such as "OK, OK" or "Stop," to interrupt the answer and ask a new question, the system can recognize and interrupt the current answer and instead process the new question.

[0088] Furthermore, after the audio output is stopped because the user's voice input contains a preset interruption keyword, the method further includes:

[0089] Outputs the preset interruption feedback reply.

[0090] It should be noted that the method also includes:

[0091] When in the interactive state, if the user's voice input is monitored to contain preset standby keywords, or if no question-and-answer interaction occurs within the set time, switch to the standby state.

[0092] It should be noted that when the user's voice input is monitored to contain the preset standby keyword, it means that the interaction is completed. At this time, the device switches back to the standby state and continues to monitor whether the user's voice input contains the preset wake-up keyword.

[0093] If no question-and-answer interaction occurs within the set time, it means that the user may have forgotten to end the current question-and-answer interaction. The system needs to actively end the question-and-answer interaction and switch to standby mode.

[0094] It can be understood that in this technical solution, the use of real-life digital characters to interact with users through voice can give people a greater sense of intimacy, and a variety of wake-up and interruption strategies also make the use of digital characters closer to daily conversation patterns.

[0095] Embodiment 3

[0096] It should be noted that the method also includes:

[0097] After the interaction state ends, the question-answer interaction data generated in the interaction state is integrated and stored in the local knowledge base.

[0098] In this technical solution, the processed historical question-and-answer data is integrated into the local knowledge base and indexed to improve retrieval efficiency. At the same time, the security and privacy of the data are ensured.

[0099] Since the local knowledge base supports localized small-scale model processing, the risk of data leakage is reduced.

[0100] Furthermore, the method also includes:

[0101] According to the hot spots and trends of users' questions in the question-and-answer interaction data, adjust the weights and priorities of various types of data in the local knowledge base.

[0102] In this technical solution, the local knowledge base and model are continuously optimized and iterated. Specifically, according to the hot spots and trends of user questions in the question-and-answer interaction data, the weights and priorities of various types of data in the local knowledge base are adjusted, and the model and local knowledge base can be continuously iterated according to user usage to adapt to more complex environments.

[0103] Preferably, the parameters and training data of the large language model are continuously adjusted and optimized based on user feedback and actual application effects to improve the accuracy and generalization ability of the model.

[0104] Preferably, the data in the local knowledge base is updated regularly to ensure that the information in the knowledge base remains up-to-date and accurate.

[0105] Embodiment 4

[0106] Figure 2 This is a schematic diagram of the structure of an AI digital human interaction system provided by an embodiment of the present application, referring to Figure 2 , an AI digital-human interaction system, comprising:

[0107] Processor 21 and memory 22;

[0108] The processor 21 and the memory 22 are connected via a communication bus:

[0109] The processor 21 is used to call and execute the program stored in the memory 22;

[0110] The memory 22 is used to store programs, and the programs are at least used to execute an AI digital human interaction method as described in the above embodiments.

[0111] It can be understood that the same or similar parts of the above embodiments can be referenced to each other, and the contents not described in detail in some embodiments can refer to the same or similar contents in other embodiments.

[0112] It should be noted that, in the description of this application, the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In addition, in the description of this application, unless otherwise specified, the meaning of "plurality" refers to at least two.

[0113] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0114] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0115] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0116] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0117] The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc.

[0118] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0119] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. An AI digital human interaction method, characterized in that: include: When in the interactive state, listen to the user's voice input; Recognize the user's voice data and convert the user's voice data into question text; Performing semantic correction on the question text by combining the semantic model with the historical question-answer interaction data in the current interaction; Inputting the semantically corrected question text into the artificial intelligence model to obtain the answer text output by the artificial intelligence model; The reply text is converted into audio and outputted through a digital character.

2. The method according to claim 1, characterized in that The method further comprises: When in standby mode and detecting that the user's voice input contains preset wake-up keywords, the system switches to interactive mode.

3. The method according to claim 1, characterized in that: The method further comprises: When the digital character outputs audio and detects that the user's voice input contains preset interruption keywords, the audio output is stopped.

4. The method according to claim 3, characterized in that After the audio output is stopped because the user's voice input contains a preset interruption keyword, the method further includes: Outputs the preset interruption feedback reply.

5. The method according to claim 1, characterized in that The method further comprises: When in the interactive state, if the user's voice input is monitored to contain preset standby keywords, or if no question-and-answer interaction occurs within the set time, switch to the standby state.

6. The method according to claim 1, characterized in that The method further comprises: Obtain conversation scene corpora from various disciplines as training data; Training the semantic model according to the training data; The semantic model is used to perform semantic correction on the current interactive text according to the historical interactive data.

7. The method according to claim 1, characterized in that The method further comprises: After the interaction state ends, the question-answer interaction data generated in the interaction state is integrated and stored in the local knowledge base.

8. The method according to claim 7, characterized in that The method further comprises: According to the hot spots and trends of users' questions in the question-and-answer interaction data, adjust the weights and priorities of various types of data in the local knowledge base.

9. The method according to claim 1, characterized in that: The method further comprises: The digital character is generated according to the custom interaction interface and interaction logic entered by the user.

10. An AI digital-human interaction system, characterized in that: include: Processor and memory; The processor and the memory are connected via a communication bus: Wherein, the processor is used to call and execute the program stored in the memory; The memory is used to store a program, and the program is at least used to execute the AI ​​digital human interaction method described in any one of claims 1-9.

Images