Information interaction method, device, electronic device, medium and program product

Abstract

The present disclosure provides an information interaction method, apparatus, electronic device, medium, and program product applicable to the fields of cloud computing and artificial intelligence technology. The method comprises: receiving an interaction request from a client, the interaction request including voice request information from a target user to initiate real-time interaction with a virtual object; responding to the interaction request, utilizing a platform-as-a-service container deployed in the cloud to obtain textual response information in response to the voice request information; invoking an image processing server deployed in the cloud to generate video response information of the virtual object based on the textual response information; and transmitting the video response information to the client to display the response result of the interaction request to the target user.

Description

Technical Field

[0001] The present disclosure relates to the fields of cloud computing and artificial intelligence technology, and in particular to an information interaction method, device, electronic device, medium, and program product. Background Art

[0002] This section is intended to provide a background or context to the embodiments of the present disclosure that are recited in the claims. No statement herein is admitted to be prior art by virtue of its inclusion in this section.

[0003] With the rapid development and popularization of information science and computer technology, information resources are gradually becoming digital. The way people consult information is no longer limited to the traditional communication between people (users) and people (customer service staff). Using computers to synthesize lively virtual objects and supplementing them with rich, natural language, expressions, lip shapes and other natural ways of human habit to interact with users is more humane and has become an inevitable choice for consultation methods.

[0004] In the process of realizing real-time interaction between users and virtual objects, in order to achieve realistic effects, the real-time rendering of virtual objects requires powerful graphics and data processing capabilities. For example, a graphics processing unit (GPU) server is used to reconstruct and render the image of the virtual object in real time. However, GPU servers are expensive and difficult to transport, resulting in high economic and time costs for realizing human-computer interaction using virtual objects, which seriously restricts the application and promotion of virtual objects in human-computer interaction. Summary of the Invention

[0005] In view of this, in order to at least partially overcome the above-mentioned technical problems existing in the relevant technologies in the process of realizing real-time interaction between users and virtual objects, the present disclosure provides an information interaction method, device, electronic device, medium and program product that are both economical and practical as well as flexible and convenient.

[0006] In order to achieve the above-mentioned objectives, one aspect of the present disclosure provides an information interaction method, which may include: receiving an interaction request from a client, the interaction request including voice request information of a target user initiating real-time interaction with a virtual object, responding to the interaction request, utilizing a platform as a service container deployed in the cloud to obtain text reply information for responding to the voice request information, calling an image processing server deployed in the cloud to generate video response information of the virtual object based on the text reply information, and transmitting the video response information to the client to display the response result of the interaction request to the target user.

[0007] According to an embodiment of the present disclosure, the above-mentioned use of a platform as a service container deployed on the cloud to obtain text reply information for responding to the above-mentioned voice request information may include: using the platform as a service container deployed on the cloud to call an automatic speech recognition component to obtain text request information of the above-mentioned voice request information, using the platform as a service container deployed on the cloud to call a natural language processing component to obtain text response information corresponding to the above-mentioned text request information, and determining the above-mentioned text response information as text reply information for responding to the above-mentioned voice request information.

[0008] According to an embodiment of the present disclosure, the above-mentioned calling of an image processing server deployed on the cloud to generate the video response information of the above-mentioned virtual object based on the above-mentioned text reply information may include: obtaining the object model of the above-mentioned virtual object, obtaining the voice reply information corresponding to the above-mentioned text reply information, generating the expression sequence of the above-mentioned virtual object based on the above-mentioned object model and the above-mentioned voice reply information, and calling the image processing server deployed on the cloud to render the expression sequence of the above-mentioned virtual object to generate the video response information of the above-mentioned virtual object.

[0009] According to an embodiment of the present disclosure, the above-mentioned information interaction method may further include: obtaining training sample data of the above-mentioned virtual object, wherein the above-mentioned training sample data is used to characterize the morphological parameters for constructing the above-mentioned virtual object, and training the above-mentioned training sample data to pre-generate an object model of the above-mentioned virtual object.

[0010] According to an embodiment of the present disclosure, transmitting the video response information to the client may include: encapsulating the video response information using a preset transmission protocol to generate response video stream data, and transmitting the response video stream data to the client in a push streaming manner.

[0011] According to an embodiment of the present disclosure, the above-mentioned information interaction method may further include: obtaining interaction scene information corresponding to the above-mentioned interaction request, and determining a virtual object corresponding to the above-mentioned target user based on the above-mentioned interaction scene information.

[0012] In order to achieve the above-mentioned objectives, another aspect of the present disclosure provides an information interaction device, which may include: a receiving module for receiving an interaction request from a client, the interaction request including voice request information of a target user initiating real-time interaction to a virtual object, a first obtaining module for responding to the interaction request by utilizing a platform as a service container deployed in the cloud to obtain text reply information for responding to the voice request information, a generating module for calling an image processing server deployed in the cloud to generate video response information of the virtual object based on the text reply information, and a transmitting module for transmitting the video response information to the client to display the response result of the interaction request to the target user.

[0013] According to an embodiment of the present disclosure, the above-mentioned first obtaining module may include: a first obtaining sub-module, used to use the platform as a service container deployed on the cloud to call the automatic speech recognition component to obtain text request information of the above-mentioned voice request information, a second obtaining sub-module, used to use the platform as a service container deployed on the cloud to call the natural language processing component to obtain text response information corresponding to the above-mentioned text request information, and a determination module, used to determine the above-mentioned text response information as text reply information for responding to the above-mentioned voice request information.

[0014] According to an embodiment of the present disclosure, the above-mentioned generation module may include: a third acquisition sub-module for obtaining the object model of the above-mentioned virtual object, a fourth acquisition sub-module for obtaining voice response information corresponding to the above-mentioned text response information, a first generation sub-module for generating an expression sequence of the above-mentioned virtual object based on the above-mentioned object model and the above-mentioned voice response information, and a second generation sub-module for calling an image processing server deployed in the cloud to render the expression sequence of the above-mentioned virtual object to generate video response information of the above-mentioned virtual object.

[0015] According to an embodiment of the present disclosure, the above-mentioned information interaction device may further include: a fifth acquisition submodule, used to obtain training sample data of the above-mentioned virtual object, wherein the above-mentioned training sample data is used to characterize the morphological parameters for constructing the above-mentioned virtual object, and a third generation submodule, used to train the above-mentioned training sample data to pre-generate an object model of the above-mentioned virtual object.

[0016] According to an embodiment of the present disclosure, the above-mentioned transmission module may include: an encapsulation submodule, which is used to encapsulate the above-mentioned video response information using a preset transmission protocol to generate response video stream data, and a transmission submodule, which is used to transmit the above-mentioned response video stream data to the above-mentioned client in a push streaming manner.

[0017] According to an embodiment of the present disclosure, the above-mentioned information interaction device may further include: a second obtaining module for obtaining interaction scene information corresponding to the above-mentioned interaction request, and a determination module for determining the virtual object corresponding to the above-mentioned target user based on the above-mentioned interaction scene information.

[0018] In order to achieve the above objectives, another aspect of the present disclosure provides an electronic device, including: one or more processors, and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the information interaction method as described above.

[0019] In order to achieve the above objectives, another aspect of the present disclosure provides a computer-readable storage medium storing computer-executable instructions, which are used to implement the above-mentioned information interaction method when executed.

[0020] In order to achieve the above objectives, another aspect of the present disclosure provides a computer program, which includes computer-executable instructions. When the instructions are executed, they are used to implement the above information interaction method.

[0021] Unlike the solution in the related art that uses a locally deployed GPU server to render virtual objects, the information interaction method provided by the present disclosure uses an image processing server deployed in the cloud to generate video response information of the virtual object based on the text reply information used to respond to the voice request information. This can avoid using a local GPU server to render the virtual object, overcome the high price and difficulty of carrying GPU servers, which result in high economic and time costs for implementing human-computer interaction using virtual objects, and seriously restrict the application and promotion of virtual objects in human-computer interaction. Therefore, it can reduce the economic and time costs of human-computer interaction using virtual objects, and at the same time improve the rendering efficiency of virtual objects, which is conducive to the application and promotion of virtual objects in human-computer interaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become readily understood by reading the detailed description below with reference to the accompanying drawings, in which several embodiments of the present disclosure are shown by way of example and not limitation, wherein:

[0023] Figure 1 The system architecture of the information interaction method, apparatus, electronic device, medium, and program product applicable to the embodiments of the present disclosure is schematically shown;

[0024] Figure 2 Schematically illustrates application scenarios of the information interaction method, apparatus, electronic device, medium, and program product applicable to the embodiments of the present disclosure;

[0025] Figure 3 The following schematically shows a flow chart of an information interaction method according to an embodiment of the present disclosure;

[0026] Figure 4 The following schematically shows a flow chart of an information interaction method according to another embodiment of the present disclosure;

[0027] Figure 5 The following schematically shows a flow chart of an information interaction method according to another embodiment of the present disclosure;

[0028] Figure 6Schematically shows a block diagram of an information interaction device according to an embodiment of the present disclosure;

[0029] Figure 7 A schematic diagram schematically illustrates a computer-readable storage medium product suitable for implementing the information interaction method described above according to an embodiment of the present disclosure; and

[0030] Figure 8 A block diagram of an electronic device suitable for implementing the above-described information interaction method according to an embodiment of the present disclosure is schematically shown.

[0031] In the drawings, the same or corresponding reference numerals denote the same or corresponding parts.

[0032] It should be noted that the drawings are not drawn to scale and that for illustrative purposes, elements of similar structure or function are generally represented by similar reference numerals throughout the drawings. DETAILED DESCRIPTION

[0033] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present disclosure. In the detailed description below, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0034] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include" and the like used herein indicate the presence of the above-mentioned features, steps, operations and / or components, but do not exclude the presence or addition of one or more other features, steps, operations or components. All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having meanings consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0035] When expressions such as “at least one of A, B, and C, etc.” are used, they should generally be interpreted in accordance with the meaning commonly understood by those skilled in the art (e.g., “a system having at least one of A, B, and C” should include but is not limited to systems having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, and C, etc.). When expressions such as “at least one of A, B, or C, etc.” are used, they should generally be interpreted in accordance with the meaning commonly understood by those skilled in the art (e.g., “a system having at least one of A, B, or C” should include but is not limited to systems having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, and C, etc.).

[0036] Some block diagrams and / or flow charts are shown in the accompanying drawings. It should be understood that some boxes or combinations thereof in the block diagrams and / or flow charts can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable information interaction device, so that when these instructions are executed by the processor, a device for implementing the functions / operations described in these block diagrams and / or flow charts can be created. The technology of the present disclosure can be implemented in the form of hardware and / or software (including firmware, microcode, etc.). In addition, the technology of the present disclosure can take the form of a computer program product on a computer-readable storage medium storing instructions, which can be used by an instruction execution system or in combination with an instruction execution system.

[0037] In the present disclosure, virtual objects refer to virtual portraits or virtual cartoon characters constructed through computer technology, that is, digital humans. Using digital humans to replace cold robots can achieve natural interaction with users and provide users with warm services. At the same time, from a visual dimension, it can not only enrich the user's experience of use and interaction, but also better understand the user's true consulting intentions during the interaction process, and thus better provide information consulting services to customers. As a participant in human-computer interaction, digital humans require the server to perform real-time calculation and rendering of portraits. However, in related technologies, the server that performs real-time calculation and rendering of portraits is a locally deployed GPU server, which is not only expensive but also difficult to transport, resulting in a solution that uses virtual objects to achieve human-computer interaction requiring a high economic cost and time cost.

[0038] Therefore, the present disclosure provides an information interaction method, device, electronic device, medium and program product. The information interaction method may include a video response information generation process and a video response information transmission process. In the video response information generation process, an interaction request from a client is first received, and the interaction request includes voice request information for a target user to initiate real-time interaction with a virtual object. Then, in response to the interaction request, a platform as a service container deployed in the cloud is used to obtain text reply information for responding to the voice request information, and finally, an image processing server deployed in the cloud is called to generate video response information of the virtual object based on the text reply information. After completing the response information generation process, the video response information transmission process is entered, and the video response information is transmitted to the client to display the response result of the interaction request to the target user.

[0039] Since the information interaction method provided by the present disclosure utilizes a platform-as-a-service container deployed on the cloud to obtain text reply information for responding to voice request information, and utilizes image processing resources deployed on the cloud to render virtual objects to generate video response information of the virtual objects, it can at least partially overcome the high price and difficulty of carrying GPU servers, which result in high economic and time costs for implementing human-computer interaction using virtual objects, and the technical problem that seriously restricts the application and promotion of virtual objects in human-computer interaction. It can also reduce the economic and time costs of human-computer interaction using virtual objects, and at the same time improve the rendering efficiency of virtual objects, which is conducive to the application and promotion of virtual objects in human-computer interaction.

[0040] Figure 1 The system architecture 100 of the information interaction method, apparatus, electronic device, medium and program product applicable to the embodiments of the present disclosure is schematically shown. It should be noted that, Figure 1 The examples shown are merely examples of system architectures to which the embodiments of the present disclosure may be applied, to help those skilled in the art understand the technical content of the present disclosure, but do not mean that the embodiments of the present disclosure may not be used in other devices, systems, environments or scenarios.

[0041] like Figure 1 As shown, the system architecture 100 according to this embodiment may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used as a medium for providing a communication link between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0042] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, and 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only as examples).

[0043] The terminal devices 101 , 102 , and 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0044] The server 105 may be a server deployed in the cloud to provide various services, such as a backend management server (for example only) that supports websites browsed by users using the terminal devices 101, 102, and 103. The backend management server may analyze and process received user requests and other data, and feed back the processing results (e.g., web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0045] It should be noted that the information interaction method provided in the embodiments of the present disclosure can generally be executed by the server 105. Accordingly, the information interaction apparatus provided in the embodiments of the present disclosure can generally be set in the server 105. The information interaction method provided in the embodiments of the present disclosure can also be executed by a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105. Accordingly, the information interaction apparatus provided in the embodiments of the present disclosure can also be set in a server or server cluster that is different from the server 105 and can communicate with the terminal devices 101, 102, 103 and / or the server 105.

[0046] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as required.

[0047] Figure 2 The application scenarios of the information interaction method, device, electronic device, medium and program product applicable to the embodiments of the present disclosure are schematically illustrated.

[0048] like Figure 2As shown, the information interaction method, device, electronic device, medium and program product of the embodiment of the present disclosure are applicable to an application scenario 200 in which a user 210 interacts with a virtual object 220. The user 210 can initiate an interaction request, such as voice. The image processing server deployed on the cloud can generate video response information based on the text reply information of the interaction request, and feed it back to the user 210 in the form of push streaming to realize human-computer interaction between the user 210 and the virtual object 220. It should be noted that Figure 2 The character form of the virtual object shown in the figure is merely illustrative and does not constitute a specific limitation on the character image.

[0049] It should be noted that with the rapid development of 5G technology and the continuous improvement of the performance of smart terminals, digital humans used to provide new and intelligent customer services can not only reduce labor costs, but also improve service quality and efficiency, providing a good development space for the application of digital humans, enabling digital human technology to be widely used in human-computer interaction scenarios in many industries such as finance, broadcasting, education, marketing, medical care, retail, and games. The information interaction method, device, electronic device, medium, and program product provided in this disclosure can be used in human-computer interaction scenarios in the financial field, and can also be used in human-computer interaction scenarios in any field other than the financial field. Therefore, the application field of the information interaction method, device, electronic device, medium, and program product provided in this disclosure is not specifically limited.

[0050] Figure 3 The flowchart of the information interaction method according to the embodiment of the present disclosure is schematically shown.

[0051] like Figure 3 As shown, the information interaction method 300 may include operations S310 to S340.

[0052] In operation S310 , an interaction request is received from a client, where the interaction request includes voice request information for a target user to initiate real-time interaction with a virtual object.

[0053] According to an embodiment of the present disclosure, the client may be installed on Figure 1 The application (Application, APP) in any terminal device shown is used to realize human-computer interaction. For example, an intelligent customer service APP installed on a smartphone. The operator of the client, that is, the target user, acts as the initiator of the interaction request and is used to initiate an interaction request to the virtual object. The virtual object, as the receiver of the interaction request, is used to respond to the interaction request of the target user. The voice request information carried in the interaction request can be the voice information input by the target user through the client of the terminal device performing the input voice operation. In this disclosure, the sending mechanism of the interaction request and the request content of the voice request information carried in the interaction request are not specifically limited.

[0054] According to an embodiment of the present disclosure, a real-time voice audio stream or voice request information such as a recording file from a client can be obtained through a sound pickup device, such as a microphone array. Optionally, the sound pickup device can also perform pre-processing operations on the collected audio, such as echo cancellation and noise suppression, to obtain a piece of high-quality audio data.

[0055] In the present disclosure, a virtual object can be a digital human. A digital human is based on artificial intelligence technologies such as image recognition, speech recognition and synthesis, semantic understanding, and portrait modeling. It has the ability to perceive, recognize, and express the physical world. It uses electronic screens, virtual reality (VR), and other display devices as carriers. It is a virtual object that realizes human-computer interaction and can provide very warm customer service. A virtual object can be obtained by rendering a two-dimensional or three-dimensional virtual model. A two-dimensional or three-dimensional model can be custom generated, or it can be obtained by converting an image or video of a character, or it can be obtained by converting an image or video of a real person.

[0056] In operation S320 , in response to the interaction request, text reply information for responding to the voice request information is obtained using a platform as a service container deployed on the cloud.

[0057] According to an embodiment of the present disclosure, for Platform-as-a-Service (PAAS) containers, when deploying PAAS container services, it is necessary to purchase corresponding virtual machines and build corresponding networks to support communication interactions. In specific implementation, the Platform-as-a-Service container deployed in the cloud can be used to identify the language intent expressed in voice request information and obtain text reply information that matches the language intent.

[0058] In operation S330 , an image processing server deployed in the cloud is called to generate video response information of the virtual object based on the text reply information.

[0059] According to an embodiment of the present disclosure, a virtual object is driven by a text response message, and lip matching can be performed based on the text response message used to respond to a voice request message, accompanied by body movements. In a specific implementation, a two-dimensional or three-dimensional virtual model corresponding to the virtual object is stored in the cloud. In this case, the text response message can be used as driving data to adjust the two-dimensional or three-dimensional virtual model corresponding to the virtual object. Cloud-based GPU computing resources are used to adjust the two-dimensional or three-dimensional model parameters of the virtual object according to the control parameters contained in the driving data. Based on the adjusted two-dimensional or three-dimensional model parameters, a rendering engine is used to generate a response animation of the virtual object, i.e., a video response message. The generated response animation is sent to the client, and the response animation displays the virtual object's lip shape, movement, and expression. In a specific implementation, lip matching is performed based on the text response message, accompanied by body movements and expressions. The pronunciation lip shape extracts lip movement, gesture, and motion data based on the initial and final consonant pronunciation lip shape of the text. An algorithm is used to automatically synthesize the extracted data based on the human body capture animation data, enabling the digital human to interact with the user through movement, expression, and gesture.

[0060] According to an embodiment of the present disclosure, based on text reply information, driving data for enabling a virtual object to express the text reply information can be generated. It should be noted that the driving data can be deployed on a server or in the cloud, and this disclosure does not specifically limit this. Specifically, when the language intent is identified, parameters related to the language intent, i.e., entities, can be extracted from the text content. For example, the entities can be determined by system segmentation, information extraction, etc. In the data corresponding to the language intent classification, the text reply information that meets the language intent can be further determined through the entities. It should be understood by those skilled in the art that the above method is only for example, and other methods can also be used to obtain text reply information that matches the language intent, and this disclosure does not specifically limit this. The driving data may include, but is not limited to, one or more of sound driving data, expression driving data, and action driving data. It should be noted that the driving data may be pre-stored on a server or other related business server. After receiving an interaction request from a client, it can be retrieved from the server or other related business server based on the voice request information to obtain driving data that matches the voice request information. Driving data may also be generated based on voice request information, for example, by inputting the voice request information into a pre-trained deep learning model to predict driving data that matches the voice request information.

[0061] By generating action-driven data, sound-driven data, and posture-driven data through text reply information, the virtual object can not only answer interactive requests through audio and video, but also make expressions and movements that match the pronunciation, and use body movements to make corresponding demonstrations at the same time.

[0062] It should be noted that, according to actual human-computer interaction needs, different cloud computing vendors can be selected and GPU resources with different configurations can be purchased. This disclosure does not make specific limitations on this.

[0063] In operation S340 , the video response information is transmitted to the client to display the response result of the interaction request to the target user.

[0064] According to an embodiment of the present disclosure, video response information of a virtual object can be generated based on driving data, and the video response information of the virtual object can be played by controlling the video playback interface of the client to display the virtual object's response to the interaction request from the client.

[0065] Through the embodiments of the present disclosure, by using an image processing server deployed on the cloud, the rendering is submitted to the processing resources of a remote computing cluster to obtain video response information of the rendered virtual object. This can at least partially solve the technical problem that GPU servers are expensive and difficult to transport, resulting in high economic and time costs for implementing human-computer interaction using digital humans, which seriously limits the application and promotion of digital humans in the industry. Therefore, the investment of economic and time costs can be reduced, the rendering efficiency of virtual objects can be improved, and the application and promotion of digital humans in the industry can be facilitated.

[0066] As an optional embodiment, using a platform as a service container deployed on the cloud to obtain text reply information for responding to voice request information may include: using the platform as a service container deployed on the cloud to call an automatic speech recognition component to obtain text request information of the voice request information; using the platform as a service container deployed on the cloud to call a natural language processing component to obtain text response information corresponding to the text request information; and determining the text response information as text reply information for responding to the voice request information.

[0067] In the present disclosure, the text information can be extracted from the audio information based on the Automatic Speech Recognition (ASR) technology. The ASR component can perform information recognition on the audio input information to obtain the recognition result of the audio input information. In a specific implementation, the real-time voice audio stream and recording file obtained by the sound pickup device can be directly input into the ASR called by the platform as a service container deployed in the cloud for recognition, or the audio data can be obtained after pre-processing on the sound pickup device, and then the audio data can be input into the ASR called by the platform as a service container deployed in the cloud for recognition. It should be noted that the ASR component can include but is not limited to audio filtering, speech segmentation, feature extraction, acoustic model, language model, decoder and other modules.

[0068] It should be noted that other methods that can convert audio information into its corresponding text request information can be used to implement this step, and this disclosure does not list them one by one.

[0069] According to an embodiment of the present disclosure, after obtaining the text request information of the voice request information, the text request information can be processed using a pre-trained neural network model for natural language processing. The neural network model may include but is not limited to a convolutional neural network (CNN), a recurrent neural network (RNN), and a long short-term memory network (LSTM). The obtained text request information is input into the above-mentioned neural network model, and the language intent represented by the text content is classified to determine the category of the language intent expressed by the text request information. Since the text content included in the text request information may contain multiple layers of meaning, by utilizing the NLP algorithm, the real intention that the user actually wants to express can be accurately identified, thereby being able to directly feedback the text response information that the user really wants to obtain, thereby improving the user's interactive experience.

[0070] According to an embodiment of the present disclosure, after obtaining text request information for a voice request, a search can be performed within the interactive request set stored in the corpus to obtain text response information corresponding to the text request information. The interactive request set in the corpus is searched and the found text response information is fed back. During the search, priority can be given to voice request information with high user access frequency. For semantic matching, a weighted sentence similarity algorithm based on a maximum ordered sequence of keywords and / or a question similarity algorithm based on a conceptual object model can be employed, which is not specifically limited in this disclosure.

[0071] Through automatic speech recognition components and natural language processing components, corresponding text reply information can be quickly obtained based on voice request information, thereby improving interaction efficiency and accuracy.

[0072] As an optional embodiment, calling an image processing server deployed on the cloud to generate video response information of a virtual object based on text reply information may include: obtaining an object model of the virtual object; obtaining voice reply information corresponding to the text reply information; generating an expression sequence of the virtual object based on the object model and the voice reply information; and calling an image processing server deployed on the cloud to render the expression sequence of the virtual object to generate video response information of the virtual object.

[0073] According to the embodiments of the present disclosure, digital humans are created using virtual reality technology, combined with human-computer interaction, high-precision three-dimensional portrait simulation, AI, motion capture, and facial expression capture. This disclosure incorporates natural language processing (NLP) and 3D animation technology to develop 3D virtual human portraits, enabling them to communicate with users in natural language and mimic human expressions and lip movements based on emotional semantics.

[0074] According to an embodiment of the present disclosure, the core technology of the virtual object may include a modeling preparation process and a portrait generation process. The modeling preparation process may include two processes: a portrait construction process and a model training process. The portrait construction process is used to construct 2D and 3D portrait forms through imaging and animation technology. The model training process is used to generate a model based on the constructed portrait form and in combination with the corresponding data and algorithm. The portrait generation process may include two processes: model prediction and portrait rendering. The model prediction process is used to apply new data (voice) to the constructed model to generate a digital human expression sequence. The portrait rendering process is used to reconstruct the expression sequence and finally present a digital human image. It should be noted that different digital human images can be shared in model training, model prediction and portrait rendering, and each digital human image requires portrait construction.

[0075] The digital human is driven using voice response information corresponding to the text response information to generate a natural speaking video sequence, i.e., an expression sequence of the virtual object. The expression sequence can be continuous or discontinuous voice information on a timeline, such as words, phrases, or sentences with specific semantics. It can be natural speech spoken by a person, pre-recorded speech, or speech synthesized from text information using speech synthesis technology, though this disclosure is not limited to this. The current facial actuation parameters corresponding to the current voice information can be obtained based on a preset mapping relationship between the voice information and the facial actuation parameters.

[0076] The present disclosure generates video response information of virtual objects by calling an image processing server deployed in the cloud to render the expression sequence of virtual objects, which can improve the efficiency of generating response information and quickly feedback the response results to the client. At the same time, it can improve the rendering efficiency of virtual objects and display the rendering results to users in a timely manner.

[0077] As an optional embodiment, the information interaction method may further include: obtaining training sample data of the virtual object, wherein the training sample data is used to characterize morphological parameters for constructing the virtual object; and training the training sample data to pre-generate an object model of the virtual object.

[0078] According to an embodiment of the present disclosure, the acquisition of training samples can be carried out by obtaining morphological parameters for characterizing and constructing virtual objects through image acquisition equipment (such as high-definition cameras and other acquisition equipment). The morphological parameters may include but are not limited to facial information, gesture information, facial expression information, facial emotion information, mouth shape information, sound information and other image data. By acquiring images of faces and gestures, this information can be used as a reference for subsequent classification of virtual objects. In specific implementation, the virtual object can be a virtual character image based on a cartoon character image, or a virtual character image based on a real customer service staff image, which is not specifically limited in the present disclosure.

[0079] Optionally, the training sample data may include a training sample set and a test sample set. The training sample set is used to train the object model, and the test sample set is used to verify the training effect of the object model and optimize it until the training results show that the object model effect meets the requirements.

[0080] It should be noted that the object model training step is a conventional technical means in this field, so it will not be further described in detail in the embodiments of the present disclosure. Those skilled in the art can choose the model training method to pre-generate the object model of the virtual object.

[0081] The present disclosure generates an object model of a virtual object through pre-training. When the virtual object needs to be rendered in real time, the object model can be directly obtained, and then the reply voice information is synthesized into the object model of the virtual object to generate video response information, which is conducive to providing real-time response to users and improving the user's interactive experience.

[0082] As an optional embodiment, transmitting the video response information to the client may include: encapsulating the video response information using a preset transmission protocol to generate response video stream data; and transmitting the response video stream data to the client in a push streaming manner.

[0083] Taking into account the high real-time requirements of human-computer interaction, in this disclosure, the response video stream data is transmitted to the client in the form of push streaming. Push streaming refers to the process of transmitting the packaged content in the acquisition stage to the server, that is, the process of transmitting the on-site video signal to the network. Push streaming has relatively high network requirements. If the network is unstable, the effect will be affected. In order to use it for push streaming, the audio and video data must also be encapsulated using a transmission protocol and converted into streaming data. Common streaming protocol types include RTSP, RTMP (Real Time Messaging Protocol), HLS, etc. RTMP is a protocol family based on TCP. It is a network protocol designed for real-time data communication. It is mainly used for audio, video and data communication between Flash / AIR platforms and streaming media and interactive servers that support the RTMP protocol. Since the delay of RTMP transmission is usually 1-3 seconds, RTMP has also become the most commonly used streaming protocol for scenarios with very high real-time requirements. Since unstable network conditions are common in real-time interaction scenarios, a certain QoS algorithm can be used to ensure that audio and video stream data is pushed to the network end even in unstable network conditions. Specifically, caches are set up at both ends of the interaction to ensure a uniform bit rate, and finally distributed through CDN.

[0084] The present invention pushes the portrait video to the client through the push streaming method, which can quickly provide response information to the user and improve the response efficiency of human-computer interaction.

[0085] As an optional embodiment, the information interaction method may further include: obtaining interaction scene information corresponding to the interaction request; and determining a virtual object corresponding to the target user based on the interaction scene information.

[0086] According to the embodiments of the present disclosure, based on the rendering method and character image classification, digital human images in various styles such as hyperrealism and cartoon can be provided to meet the needs of different scenarios. It supports intelligent interaction with users through AI technologies such as face recognition and gesture recognition, allowing users to choose suitable virtual images according to different interaction scenarios. In this way, virtual objects can quickly connect to content service libraries in different knowledge fields and be efficiently applied to more industries. Optionally, hyperrealistic digital humans can be used to create smart front desks in banks, business halls, and service halls, interacting with users in a real and effective way, improving service quality and customer satisfaction. Optionally, cartoon-style digital humans can be used in human-computer interaction scenarios with fun interactions as the idea, such as smart guides in offline supermarkets, or smart coaches, virtual teachers, etc., to achieve the purpose of attracting customers, stimulating interest, and enhancing teaching effectiveness.

[0087] In order to further illustrate the information interaction method between users and digital humans provided by the present disclosure, the following will be combined with Figure 4 and Figure 5 The flowchart shown describes the specific implementation in detail.

[0088] Figure 4 The following schematically shows a flow chart of an information interaction method according to another embodiment of the present disclosure.

[0089] like Figure 4 As shown, the information interaction method 400 may include operations S410 to S490.

[0090] In operation S410, a user initiates an interaction request to the audio and video acquisition component 410. In operation S420, the audio and video acquisition component 410 captures the user's audio and video and sends the user request to the digital human interaction control system component 420. In operation S430, the digital human interaction control system component 420 sends the voice request to the ASR component 430. In operation S440, the ASR component 430 generates a speech recognition text and sends it to the digital human interaction control system component 420. In operation S450, the digital human interaction control system component 420 sends the text request to the NLP component 440. In operation S460, the NLP component 440 obtains the dialogue response text and sends it to the digital human interaction control system component 420. In operation S470, the digital human interaction control system component 420 sends the new response text to the digital human engine component 450 for generating a new video. In operation S480, the digital human avatar display component 460 displays the new digital human avatar in real time. In operation S490, the user views the response video.

[0091] In specific implementations, each time a user interacts with a digital human, the interaction request is first sent to the digital human interaction control system component 420 via the audio and video acquisition component 410. The interaction control module of the digital human interaction control system component 420 then coordinates various components to ultimately generate a digital human image. These components include the ASR component 430 for converting the user's voice request into text, the NLP component 440 for obtaining a text response based on the user's text request, and the digital human engine component 450 for generating a new digital human video. Finally, the digital human image is displayed in real time via the digital human image display component 460. The digital human interaction control system component 420 can be a PAAS container, and the digital human image display component 460 can be a GPU server.

[0092] This disclosure provides an information interaction method and a system construction solution for interactive digital humans based on cloud computing resources. Interactive digital human generation models are highly complex and computationally intensive, requiring high-performance GPU servers, which are also expensive to purchase. This invention proposes a solution using cloud-based GPU rendering, using high-performance cloud-based GPUs on the server side to synthesize human portraits.

[0093] Figure 5The following schematically shows a flow chart of an information interaction method according to another embodiment of the present disclosure.

[0094] like Figure 5 As shown, the information interaction method 500 involves collaboration between the caller 510, the digital human backend service platform 520, the cloud-based digital human engine GPU 530, and the video terminal 540. The information interaction method may include operations S511 to S516 between the caller 510 and the digital human backend service platform 520, operations S521 to S526 between the digital human backend service platform 520 and the cloud-based digital human engine GPU, and operations S531 to S532 between the cloud-based digital human engine GPU 530 and the video terminal 540.

[0095] In specific implementation, the real-time interactive digital human cloud rendering process of the information interaction method 500 may include a cloud initialization interface initialization process, a long connection establishment process, a video stream synthesis process, and a long connection closing process.

[0096] In the present disclosure, invoking cloud resources to implement real-time computation and rendering of virtual objects during real-time user interaction with them requires invoking a cloud-based initialization interface. During the initialization of the cloud-based initialization interface, the caller 510 first sends a startup message to the digital human backend service platform 520 (operation S511). In response to this startup message, the digital human backend service platform 520 invokes the cloud-based initialization interface of the cloud-based digital human engine GPU 530 (operation S521). If the call to the cloud-based initialization interface is successful, the cloud-based digital human engine GPU 530 returns an initialization success message to the digital human backend service platform 520 (operation S522), which in turn returns the initialization success message to the caller 510 (operation S512).

[0097] If the cloud initialization interface is successfully initialized, the persistent connection establishment process begins. During the persistent connection establishment process, the caller 510 sends a persistent connection establishment request to the digital human backend service platform 520 (operation S513). In response to the persistent connection establishment request, the digital human backend service platform 520 establishes a websocket persistent connection with the cloud digital human engine GPU 530 (operation S523). If the websocket persistent connection with the cloud digital human engine GPU 530 is successfully established, the cloud digital human engine GPU 530 returns a successful establishment message to the digital human backend service platform 520 (operation S524), and the digital human backend service platform 520 returns a successful establishment message to the caller 510 (operation S514). In the present disclosure, a persistent connection is established via websocket to receive network data sent by the user. Websocket is a protocol provided by HTML5 for full-duplex communication over a single TCP connection. Websocket simplifies data exchange between the client and the server, allowing the server to actively push data to the client. In the websocket API, the browser and server only need to complete a handshake to establish a persistent connection and enable two-way data transmission. In the websocket API, the browser and server only need to complete a handshake, and then a fast channel is formed between the browser and the server. Data can be directly transmitted between the two. The websocket protocol defined by HTML5 can better save server resources and bandwidth and enable more real-time communication. The browser sends a request to the server to establish a websocket connection through JavaScript. Once the connection is established, the client and server can exchange data directly over the TCP connection. Once the websocket connection is obtained, data can be sent to the server using the send() method and the data returned by the server can be received through the onmessage event.

[0098] If the long connection is successfully established, the video stream synthesis process begins. During the video stream synthesis process, the caller 510 sends a text message to the digital human backend service platform 520 (operation S515). The digital human backend service platform 520 sends the text message to the cloud-based digital human engine GPU 530 (operation S525). The cloud-based digital human engine GPU 530 synthesizes a text video stream based on the text message (operation S531). The cloud-based digital human engine GPU 530 then pushes the text video stream to the video terminal 540 (operation S532) so that the user can view the text video stream. It should be noted that operations S515, S525, S531, and S532 in the video stream synthesis process can be executed in a loop. That is, each time the caller 510 sends a text message to the digital human backend service platform 520 (operation S515), the digital human backend service platform 520 sends a text message to the cloud-based digital human engine GPU 530 (operation S525). The cloud-based digital human engine GPU 530 synthesizes a text video stream based on the current text information (operation S531 ), and then pushes the text audio and video stream to the video terminal 540 (operation S532 ). This cycle is repeated to complete multiple interactions between the user and the digital human.

[0099] If the video stream synthesis is successful, the persistent connection closing process begins, ending the interaction between the user and the digitized human. During the persistent connection closing process, the caller 510 first sends a persistent connection closing request to the digitized human backend service platform 520 (operation S516). The digitized human backend service platform 520 then sends a websocket connection closing request to the cloud-based digitized human engine GPU 530 (operation S526).

[0100] This disclosure utilizes an image processor deployed on a cloud computing end to perform real-time rendering of interactive digital humans, which can eliminate the dependence of real-time rendering on physical image processors. Through the cloud-based digital human system construction solution, the problem of GPU devices can be effectively solved. By using cloud-based GPU resources, the process of digital human portrait rendering can be realized with greater adaptability.

[0101] Figure 6 A block diagram of an information interaction device according to an embodiment of the present disclosure is schematically shown.

[0102] like Figure 6 As shown, the information interaction device 600 may include a receiving module 610 , a first obtaining module 620 , a generating module 630 , and a transmitting module 640 .

[0103] The receiving module 610 is used to receive an interaction request from a client, wherein the interaction request includes a voice request message for a target user to initiate a real-time interaction with a virtual object. Figure 3 The operation S310 described above will not be repeated here.

[0104] The first obtaining module 620 is used to obtain text reply information for responding to the voice request information by using the platform as a service container deployed in the cloud in response to the interaction request. Figure 3 The operation S320 described above will not be repeated here.

[0105] The generating module 630 is used to call the image processing server deployed in the cloud to generate the video response information of the virtual object based on the text reply information. Figure 3 Operation S330 described above will not be repeated here.

[0106] The transmission module 640 is used to transmit the video response information to the client to show the response result of the interactive request to the target user. Optionally, the transmission module 640 can be used to perform Figure 3 Operation S340 described above will not be repeated here.

[0107] As an optional embodiment, the aforementioned first obtaining module 620 may include: a first obtaining sub-module, used to use the platform as a service container deployed on the cloud to call the automatic speech recognition component to obtain text request information of the voice request information; a second obtaining sub-module, used to use the platform as a service container deployed on the cloud to call the natural language processing component to obtain text response information corresponding to the text request information; and a determination module, used to determine the text response information as text reply information for responding to the voice request information.

[0108] As an optional embodiment, the aforementioned generation module 630 may include: a third acquisition sub-module, used to obtain an object model of the virtual object; a fourth acquisition sub-module, used to obtain voice response information corresponding to the text response information; a first generation sub-module, used to generate an expression sequence of the virtual object based on the object model and the voice response information; and a second generation sub-module, used to call an image processing server deployed in the cloud to render the expression sequence of the virtual object to generate video response information of the virtual object.

[0109] As an optional embodiment, the aforementioned information interaction device may also include: a fifth acquisition submodule, used to obtain training sample data of the virtual object, wherein the training sample data is used to characterize the morphological parameters of the constructed virtual object; and a third generation submodule, used to train the training sample data to pre-generate an object model of the virtual object.

[0110] As an optional embodiment, the aforementioned transmission module 640 may include: an encapsulation submodule for encapsulating the video response information using a preset transmission protocol to generate response video stream data; and a transmission submodule for transmitting the response video stream data to the client in a push streaming manner.

[0111] As an optional embodiment, the aforementioned information interaction device may further include: a second obtaining module for obtaining interaction scene information corresponding to the interaction request; and a determining module for determining a virtual object corresponding to the target user based on the interaction scene information.

[0112] It should be noted that the implementation methods, technical problems solved, functions implemented, and technical effects achieved of each module in some embodiments of the information interaction device are the same or similar to the implementation methods, technical problems solved, functions implemented, and technical effects achieved of each corresponding step in some embodiments of the information interaction method, and will not be repeated here.

[0113] According to the embodiments of the present invention, any one of the modules, sub-modules, units, and sub-units, or at least part of the functions of any one of them can be implemented in one module. According to the embodiments of the present invention, any one or more of the modules, sub-modules, units, and sub-units can be split into multiple modules for implementation. According to the embodiments of the present invention, any one or more of the modules, sub-modules, units, and sub-units can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FNGA), a programmable logic array (NLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware of any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation modes of software, hardware, and firmware, or in an appropriate combination of any of them. Alternatively, according to the embodiments of the present invention, one or more of the modules, sub-modules, units, and sub-units can be at least partially implemented as a computer program module, which can perform the corresponding function when the computer program module is run.

[0114] For example, the receiving module, the first obtaining module, the generating module, the transmitting module, the first obtaining submodule, the second obtaining submodule, the determining module, the third obtaining submodule, the fourth obtaining submodule, the first generating submodule, the second generating submodule, the fifth obtaining submodule, the third generating submodule, the encapsulating submodule, the transmitting submodule, the second obtaining module, and the determining module can be combined into one module for implementation, or any one of the modules can be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules can be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the receiving module, the first obtaining module, the generating module, the transmitting module, the first obtaining sub-module, the second obtaining sub-module, the determining module, the third obtaining sub-module, the fourth obtaining sub-module, the first generating sub-module, the second generating sub-module, the fifth obtaining sub-module, the third generating sub-module, the encapsulating sub-module, the transmitting sub-module, the second obtaining module and the determining module may be at least partially implemented as a hardware circuit, such as a field programmable gate array (FNGA), a programmable logic array (NLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or may be implemented by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or may be implemented in any one of the three implementation methods of software, hardware and firmware, or in a suitable combination of any of them. Alternatively, at least one of the receiving module, the first obtaining module, the generating module, the transmitting module, the first obtaining sub-module, the second obtaining sub-module, the determining module, the third obtaining sub-module, the fourth obtaining sub-module, the first generating sub-module, the second generating sub-module, the fifth obtaining sub-module, the third generating sub-module, the encapsulating sub-module, the transmitting sub-module, the second obtaining module and the determining module can be at least partially implemented as a computer program module, which can perform corresponding functions when the computer program module is run.

[0115] Figure 7 A schematic diagram of a computer-readable storage medium product suitable for implementing the information interaction method described above according to an embodiment of the present disclosure is schematically shown.

[0116] In some possible implementations, various aspects of the present invention may also be implemented in the form of a program product, which includes program code. When the program product is run on a device, the program code is used to enable the device to perform the aforementioned operations (or steps) in the information interaction method according to various exemplary embodiments of the present invention described in the "Exemplary Method" section above. For example, the electronic device may perform the following operations: Figure 3 Operations S310 to S340 shown in FIG. Figure 4 Operations S410 to S490 shown in FIG. Figure 5Operations S511 to S516, operations S521 to S526, and operations S531 to S532 shown in FIG.

[0117] The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, system or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (ENROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0118] like Figure 7 As shown, a program product 700 for information interaction according to an embodiment of the present invention is described. The program product 700 may be a portable compact disk read-only memory (CD-ROM) and include program code, and may be run on a device such as a personal computer. However, the program product of the present invention is not limited thereto. In this document, a readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device.

[0119] A readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries readable program code. Such propagated data signals may take a variety of forms, including, but not limited to, electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium that can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, system, or device. The program code contained on the readable medium may be transmitted using any suitable medium, including, but not limited to, wireless, wired, optical cable, RF, etc., or any suitable combination thereof.

[0120] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, and the like, as well as conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device via any type of network, including a local area network (LAA) or a wide area network (WAA), or may be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0121] Figure 8 A block diagram of an electronic device suitable for implementing the above-described information interaction method according to an embodiment of the present disclosure is schematically shown. Figure 8 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0122] like Figure 8 As shown, the electronic device 800 according to an embodiment of the present disclosure includes a processor 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage part 808 into a random access memory (RAM) 803. The processor 801 may, for example, include a general-purpose microprocessor (e.g., a CNU), an instruction set processor and / or a related chipset and / or a dedicated microprocessor (e.g., an application-specific integrated circuit (ASIC)), and the like. The processor 801 may also include an onboard memory for caching purposes. The processor 801 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0123] In RAM 803, various programs and data required for the operation of electronic device 800 are stored. Processor 801, ROM 802 and RAM 803 are connected to each other via bus 804. Processor 801 executes various operations of the method flow according to the embodiment of the present disclosure by executing the programs in ROM 802 and / or RAM 803. It should be noted that the programs can also be stored in one or more memories other than ROM 802 and RAM 803. Processor 801 can also execute the programs stored in the one or more memories. Figure 3 Operations S310 to S340 shown in FIG. Figure 4 Operations S410 to S490 shown in FIG. Figure 5Operations S511 to S516, operations S521 to S526, and operations S531 to S532 shown in FIG.

[0124] According to an embodiment of the present disclosure, the electronic device 800 may further include an input / output (I / O) interface 805, which is also connected to the bus 804. The system 800 may also include one or more of the following components connected to the I / O interface 805: an input portion 806 including a keyboard, a mouse, etc.; an output portion 807 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker; a storage portion 808 including a hard disk; and a communication portion 809 including a network interface card such as an LAA card or a modem. The communication portion 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the I / O interface 805 as needed. A removable medium 811, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 810 as needed so that a computer program read therefrom can be installed into the storage portion 808 as needed.

[0125] According to an embodiment of the present disclosure, the method flow according to an embodiment of the present disclosure can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 809, and / or installed from the removable medium 811. When the computer program is executed by the processor 801, the above-mentioned functions defined in the system of the embodiment of the present disclosure are executed. According to an embodiment of the present disclosure, the system, device, apparatus, module, unit, etc. described above can be implemented by a computer program module.

[0126] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the information interaction method according to the embodiment of the present disclosure is implemented, including the following: Figure 3 Operations S310 to S340 shown in FIG. Figure 4 Operations S410 to S490 shown in FIG. Figure 5 Operations S511 to S516, operations S521 to S526, and operations S531 to S532 shown in FIG.

[0127] According to an embodiment of the present disclosure, a computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (ENROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, apparatus or device. For example, according to an embodiment of the present disclosure, a computer-readable storage medium may include the ROM 802 and / or RAM 803 described above and / or one or more memories other than ROM 802 and RAM 803.

[0128] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the above-mentioned module, program segment, or a part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0129] Those skilled in the art will appreciate that various combinations and / or combinations of features described in the various embodiments and / or claims of this disclosure may be made, even if such combinations or combinations are not explicitly described in this disclosure. In particular, various combinations and / or combinations of features described in the various embodiments and / or claims of this disclosure may be made, without departing from the spirit and teachings of this disclosure. All such combinations and / or combinations fall within the scope of this disclosure.

[0130] The embodiments of the present disclosure are described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although each embodiment has been described separately above, this does not mean that the measures in each embodiment cannot be used in combination to advantage. The scope of the present disclosure is defined by the appended claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. An information interaction method, comprising: Receiving an interaction request from a client, wherein the interaction request includes voice request information for a target user to initiate real-time interaction with a virtual object; In response to the interaction request, using a platform-as-a-service container deployed in the cloud to call an automatic speech recognition component and a natural language processing component to obtain text reply information for responding to the voice request information; Invoke an image processing server deployed in the cloud to generate video response information of the virtual object based on the text reply information, The calling of an image processing server deployed in the cloud to generate video response information of the virtual object based on the text reply information includes: obtaining an object model of the virtual object; Obtaining voice response information corresponding to the text response information; generating an expression sequence of the virtual object based on the object model and the voice response information, the expression sequence representing a video sequence corresponding to the voice response information; and Invoking an image processing server deployed in the cloud to render the expression sequence of the virtual object to generate video response information of the virtual object; and The video response information is transmitted to the client to display the response result of the interaction request to the target user.

2. The method according to claim 1, wherein The method of using a platform-as-a-service container deployed on the cloud to call an automatic speech recognition component and a natural language processing component to obtain text reply information in response to the voice request information includes: Using a platform-as-a-service container deployed in the cloud to call an automatic speech recognition component to obtain text request information of the voice request information; Using the platform-as-a-service container deployed on the cloud to call a natural language processing component to obtain text response information corresponding to the text request information; and The text response information is determined as text reply information for responding to the voice request information.

3. The method according to claim 1, wherein The method further comprises: Obtaining training sample data of the virtual object, wherein the training sample data is used to characterize morphological parameters for constructing the virtual object; and The training sample data is trained to pre-generate an object model of the virtual object.

4. The method according to claim 1, wherein The transmitting the video response information to the client comprises: Encapsulating the video response information using a preset transmission protocol to generate response video stream data; and The response video stream data is transmitted to the client in a push streaming manner.

5. The method according to claim 1, wherein The method further comprises: Obtaining interaction scenario information corresponding to the interaction request; and Based on the interaction scene information, a virtual object corresponding to the target user is determined.

6. An information interaction device, comprising: A receiving module, configured to receive an interaction request from a client, wherein the interaction request includes voice request information for a target user to initiate real-time interaction with a virtual object; A first obtaining module is configured to, in response to the interaction request, utilize a platform-as-a-service container deployed in the cloud to call an automatic speech recognition component and a natural language processing component to obtain textual response information in response to the voice request information; A generation module is used to call an image processing server deployed in the cloud to generate video response information of the virtual object based on the text reply information, Wherein, the generation module includes: A third obtaining submodule is used to obtain an object model of the virtual object; A fourth obtaining submodule, configured to obtain voice response information corresponding to the text response information; A first generating submodule is configured to generate an expression sequence of the virtual object based on the object model and the voice response information, wherein the expression sequence represents a video sequence corresponding to the voice response information; and A second generating submodule is configured to call an image processing server deployed in the cloud to render the expression sequence of the virtual object to generate video response information of the virtual object; and A transmission module is used to transmit the video response information to the client to show the response result of the interaction request to the target user.

7. An electronic device comprising: one or more processors; as well as a memory for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors are enabled to perform the method according to any one of claims 1 to 5. 8 . A computer-readable storage medium having computer-executable instructions stored thereon, which, when executed, cause a processor to perform the method according to claim 1 .

9. A computer program product comprising a computer program, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 5 is performed.

Images