Virtual digital human interaction method and device, equipment and storage medium

Abstract

The invention provides a virtual digital human interaction method and device, equipment and a storage medium. Relates to the technical field of artificial intelligence. The method comprises the steps that service processing information is obtained according to user interaction information, the user interaction information represents an interaction instruction input when a user handles user to-be-handled services through a virtual digital person, and the service processing information represents to-be-handled service characteristics corresponding to the user to-be-handled services; target business information is obtained according to the business processing information and a business information database, the business information database is used for storing business information corresponding to at least one user to-be-handled business and a corresponding business identifier, and the target business information represents business information determined based on to-be-handled business characteristics; and according to the target service information, digital human interaction information is generated, and the digital human interaction information is used for indicating the virtual digital human to feed back the service information corresponding to the to-be-handled service of the user to the user, so that accurate service information is provided for the user, and the problem of poor user experience is solved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a virtual digital human interaction method, device, equipment and storage medium. Background Art

[0002] With the development of artificial intelligence technology, digital customer service has been widely used in various applications (APPs) to interact with users and answer their questions.

[0003] In the prior art, users can input questions they want to ask through an application, and then digital customer service matches answers from a preset database based on keywords corresponding to the questions input by the user to answer the questions asked by the user.

[0004] Regarding the solutions of the prior art, there is a problem of poor user experience when digital customer service answers users' questions. Summary of the invention

[0005] The present application provides a virtual digital human interaction method, device, equipment and storage medium, which are used to solve the problem of poor user experience when digital customer service answers user questions in the prior art.

[0006] In a first aspect, the present application provides a virtual digital human interaction method, comprising: obtaining business processing information based on user interaction information, the user interaction information representing the interaction instructions input by the user when handling the user's pending business through the virtual digital human, and the business processing information representing the pending business characteristics corresponding to the user's pending business; obtaining target business information based on the business processing information and a business information database, the business information database being used to store business information and corresponding business identifiers corresponding to at least one user's pending business, the target business information representing business information determined based on the pending business characteristics; generating digital human interaction information based on the target business information, the digital human interaction information being used to instruct the virtual digital human to feed back the business information corresponding to the user's pending business to the user.

[0007] In one possible design, the user interaction information includes at least one of the following interaction information: text information, audio information or image information, and obtaining business processing information based on the user interaction information includes: inputting the user interaction information into a pre-trained multimodal large model to obtain interaction instruction features, the interaction instruction features including text instruction features, audio instruction features or image instruction features, the text instruction features corresponding to the text information, the audio instruction features corresponding to the audio information, and the image instruction features corresponding to the image information; obtaining the business processing information based on the interaction instruction features.

[0008] In one possible design, the image information includes user limb image information, and the user limb image information is used to indicate the user's limb movements within a preset time interval, and the interaction instruction features include limb interaction instruction features; the user interaction information is input into a pre-trained multimodal large model to obtain the interaction instruction features, including: obtaining a limb video frame based on the user limb image information, the limb video frame including at least two sub-limb video frames, and the sub-limb video frames in the limb video frame are arranged based on the action timing features corresponding to the limb movements; the limb video frame is input into a pre-trained multimodal large model to obtain the limb interaction instruction features, and the limb interaction instruction features represent image instruction features determined based on the action timing features.

[0009] In a possible design, the image information includes user expression image information, the audio information includes user voice feature information, and the method further includes: obtaining user emotion information based on the user expression image information and / or the user voice feature information, wherein the user emotion information represents the urgency of the user pending business handled by the user; and generating digital human interaction information based on the target business information includes: generating the digital human interaction information based on the user emotion information and the target business information.

[0010] In a possible design, the method also includes: generating digital human configuration information based on user login information and the user interaction information, the user login information representing the user identity characteristics when the user logs in to the business system corresponding to the user's pending business, and the digital human configuration information representing the personalized configuration characteristics of the virtual digital human; generating target interaction information based on the digital human configuration information and the digital human interaction information, the target interaction information representing the interaction characteristics when the virtual digital human interacts with the user.

[0011] In a possible design, the business information database includes a first database and a second database, the first database is used to indicate a business information database for high-frequency access services, and the second database is used to indicate a business information database for low-frequency access services; the target business information is obtained based on the business processing information and the business information database, including: determining contracted business information based on user login information, the contracted business information is used to indicate the high-frequency access services and / or low-frequency access services signed by the user; determining contracted business ratio information based on the contracted business information, the contracted business ratio information characterizing the proportional relationship between the number of high-frequency access services signed by the user and the number of low-frequency access services signed by the user; the target business information is obtained based on the business processing information, the contracted business ratio information, the first database and the second database.

[0012] In a possible design, the target business information is obtained according to the business processing information, the contracted business ratio information, the first database and the second database, including: if the ratio corresponding to the contracted business ratio information is greater than a preset ratio, then according to the business processing information, the first database and the second database are accessed in sequence to obtain the target business information; or, if the ratio corresponding to the contracted business ratio information is less than or equal to the preset ratio, then according to the business processing information, the second database and the first database are accessed in sequence to obtain the target business information.

[0013] In a possible design, generating digital human interaction information according to the target business information includes: obtaining full-version business information and simplified-version business information according to the target business information, wherein the full-version business information is used to indicate the complete business information corresponding to the user's pending business, and the simplified-version business information is used to indicate the simplified business information corresponding to the user's pending business, and the simplified-version business information includes an identification code of the full-version business information, and the identification code is used to provide the full-version business information to the user when recognized by the user's terminal device; generating digital human interaction information according to the full-version business information and the simplified-version business information.

[0014] In a second aspect, the present application provides a virtual digital human interaction device, comprising:

[0015] A processing module, used to obtain business processing information according to user interaction information, wherein the user interaction information represents the interaction instructions input by the user when handling the user's pending business through a virtual digital person, and the business processing information represents the pending business characteristics corresponding to the user's pending business;

[0016] A determination module, configured to obtain target business information according to the business processing information and a business information database, wherein the business information database is configured to store business information corresponding to at least one pending business of a user and a corresponding business identifier, wherein the target business information represents business information determined based on the characteristics of the pending business;

[0017] A generation module is used to generate digital human interaction information according to the target business information, wherein the digital human interaction information is used to instruct the virtual digital human to feed back business information corresponding to the user's pending business to the user.

[0018] In one possible design, the user interaction information includes at least one of the following interaction information: text information, audio information or image information. When the processing module obtains the business processing information based on the user interaction information, it is specifically used to: input the user interaction information into a pre-trained multimodal large model to obtain interaction instruction features, the interaction instruction features including text instruction features, audio instruction features or image instruction features, the text instruction features corresponding to the text information, the audio instruction features corresponding to the audio information, and the image instruction features corresponding to the image information; obtain the business processing information based on the interaction instruction features.

[0019] In one possible design, the image information includes user limb image information, and the user limb image information is used to indicate the user's limb movements within a preset time interval, and the interaction instruction features include limb interaction instruction features; when the processing module inputs the user interaction information into a pre-trained multimodal large model to obtain the interaction instruction features, it is specifically used to: obtain a limb video frame based on the user limb image information, and the limb video frame includes at least two sub-limb video frames, and the sub-limb video frames in the limb video frame are arranged based on the action timing features corresponding to the limb movements; input the limb video frame into the pre-trained multimodal large model to obtain the limb interaction instruction features, and the limb interaction instruction features represent the image instruction features determined based on the action timing features.

[0020] In a possible design, the image information includes user expression image information, the audio information includes user voice feature information, and the generation module is further used to: obtain user emotion information based on the user expression image information and / or user voice feature information, and the user emotion information represents the urgency of the user pending business handled by the user; when the generation module generates the digital human interaction information based on the target business information, it is specifically used to: generate the digital human interaction information based on the user emotion information and the target business information.

[0021] In a possible design, the generation module is also used to: generate digital human configuration information based on user login information and the user interaction information, the user login information represents the user identity characteristics when the user logs in to the business system corresponding to the user's pending business, and the digital human configuration information represents the personalized configuration characteristics of the virtual digital human; generate target interaction information based on the digital human configuration information and the digital human interaction information, and the target interaction information represents the interaction characteristics when the virtual digital human interacts with the user.

[0022] In a possible design, the business information database includes a first database and a second database, the first database is used to indicate a business information database for high-frequency access services, and the second database is used to indicate a business information database for low-frequency access services; when the determination module obtains the target business information based on the business processing information and the business information database, it is specifically used to: determine the contracted business information based on the user login information, and the contracted business information is used to indicate the high-frequency access services and / or low-frequency access services signed by the user; determine the contracted business ratio information based on the contracted business information, and the contracted business ratio information represents the proportional relationship between the number of high-frequency access services signed by the user and the number of low-frequency access services signed by the user; obtain the target business information based on the business processing information, the contracted business ratio information, the first database and the second database.

[0023] In a possible design, when the determination module obtains the target business information based on the business processing information, the contracted business ratio information, the first database and the second database, it is specifically used to: if the ratio corresponding to the contracted business ratio information is greater than a preset ratio, then according to the business processing information, the first database and the second database are accessed in sequence to obtain the target business information; or, if the ratio corresponding to the contracted business ratio information is less than or equal to the preset ratio, then according to the business processing information, the second database and the first database are accessed in sequence to obtain the target business information.

[0024] In a possible design, when the generation module generates digital human interaction information according to the target business information, it is specifically used to: obtain full-version business information and simplified business information according to the target business information, wherein the full-version business information is used to indicate the complete business information corresponding to the user's pending business, and the simplified business information is used to indicate the simplified business information corresponding to the user's pending business, and the simplified business information includes an identification code of the full-version business information, and the identification code is used to provide the full-version business information to the user when it is recognized by the user's terminal device; generate digital human interaction information according to the full-version business information and the simplified business information.

[0025] In a third aspect, an embodiment of the present application provides an electronic device comprising: at least one processor and a memory; the memory stores computer execution instructions; the at least one processor executes the computer execution instructions stored in the memory, so that the at least one processor executes the virtual digital human interaction method described in the first aspect and various possible designs of the first aspect.

[0026] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer execution instructions are stored. When a processor executes the computer execution instructions, the virtual digital human interaction method described in the first aspect and various possible designs of the first aspect is implemented.

[0027] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program. When the computer program is executed by a processor, it implements the virtual digital human interaction method described in the first aspect and various possible designs of the first aspect.

[0028] The virtual digital human interaction method, device, equipment and storage medium provided by the present application obtain business processing information according to user interaction information, wherein the user interaction information represents the interaction instruction input by the user when handling the user's pending business through the virtual digital human, and the business processing information represents the pending business characteristics corresponding to the user's pending business; according to the business processing information and the business information database, the target business information is obtained, and the business information database is used to store the business information corresponding to at least one user's pending business and the corresponding business identifier, and the target business information represents the business information determined based on the pending business characteristics; according to the target business information, digital human interaction information is generated, and the digital human interaction information is used to instruct the virtual digital human to feedback the business information corresponding to the user's pending business to the user. By performing semantic analysis and processing on the interaction instruction input by the user when handling the user's pending business through the virtual digital human, the pending business characteristics corresponding to the user's pending business are determined, and then the target business information is determined in the business information database based on the pending business characteristics, and then by performing data processing on the target business information, the virtual digital human is realized to feedback the business information to the user, that is, to provide the user with accurate business information, and solve the problem of poor user experience when the digital customer service answers the user's questions. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0030] Figure 1 An application scenario diagram of the virtual digital human interaction method provided in the embodiment of the present application;

[0031] Figure 2 A flowchart of a virtual digital human interaction method provided by an embodiment of the present application;

[0032] Figure 3 for Figure 2 A schematic diagram of a specific implementation process of step S101 in the illustrated embodiment;

[0033] Figure 4 for Figure 3 A schematic diagram of a specific implementation process of step S1011 in the illustrated embodiment;

[0034] Figure 5 A schematic diagram of a process for obtaining body interaction instruction features based on user body image information provided by the present application;

[0035] Figure 6 A schematic diagram of a process for determining target business information provided for this application;

[0036] Figure 7A schematic diagram of a scenario in which a virtual digital person provides feedback of business information to a user provided in this application;

[0037] Figure 8 A flowchart of a virtual digital human interaction method provided by another embodiment of the present application;

[0038] Fig. 9 A schematic diagram of the structure of a virtual digital human interaction device provided in one embodiment of the present application;

[0039] Fig.10 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0040] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0041] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0042] In the technical solution of this application, the collection, storage, use, processing, transmission, provision and disclosure of information such as financial data or user data involved shall comply with the provisions of relevant laws and regulations and shall not violate public order and good morals.

[0043] It should be noted that in the embodiments of the present application, certain software, components, models and other existing solutions in the industry may be mentioned, and they should be regarded as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of the present application, but it does not mean that the applicant has or will necessarily use the solution.

[0044] The application scenarios of the embodiments of the present application are explained below:

[0045] Figure 1 FIG. 1 is an application scenario diagram of the virtual digital human interaction method provided in the embodiment of the present application. The virtual digital human interaction method provided in the embodiment of the present application can be applied to the scenario of answering user questions through the virtual digital human. For example, Figure 1As shown, the execution subject of the method provided in the embodiment of the present application may be an electronic control unit, a terminal device or a server. The terminal device is used as the execution subject for explanation. For example, user user_0 inputs a consulting question quest_1 about business A through the terminal device. The terminal device parses and processes the consulting question quest_1 based on the method provided in the present application to obtain the question features corresponding to the consulting question quest_1, and then determines the corresponding target business information, and the virtual digital person feeds back the target business information to the user.

[0046] Currently, after the user inputs the question to be consulted through the application of the terminal device, the digital customer service can only extract keywords from the question input by the user, and then match the keywords in the preset database according to the extracted keywords to obtain the corresponding answer. If the keywords do not match, even if the corresponding answer is saved in the database, the digital customer service still cannot determine the answer. For example, the question input by the user is "Please introduce me to business task_1", and the key field saved in the preset database is "Business details of business task_1", and the corresponding business introduction information answer_1 is stored. Although the answer pointed to by the question input by the user is the business introduction information answer_1, because the keyword "Please introduce me to" is inconsistent with the keyword "business details", the digital customer service still cannot feedback the business introduction information answer_1 to the user, resulting in a poor user experience when the digital customer service answers the user's question.

[0047] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0048] Figure 2 A flowchart of a virtual digital human interaction method provided by an embodiment of the present application, such as Figure 2 As shown, the execution subject of the virtual digital human interaction method provided in this embodiment may be an electronic control unit, a terminal device or a server. By way of example, this embodiment uses a terminal device as the execution subject of the method of this embodiment for explanation. The virtual digital human interaction method provided in this embodiment includes the following steps:

[0049] Step S101, obtaining business processing information according to user interaction information, wherein the user interaction information represents the interaction instructions input by the user when handling the user's pending business through a virtual digital person, and the business processing information represents the pending business characteristics corresponding to the user's pending business.

[0050] Exemplarily, user interaction information is used to indicate the interaction instructions input by the user when handling user pending services through a virtual digital person, such as text information input by the user through a text input box; business processing information is used to indicate the user pending services determined based on the interaction instructions input by the user and the corresponding pending business features, wherein "handling user pending services" includes users consulting business issues and signing business contracts.

[0051] Specifically, for example, in response to the text information input by the user, the terminal device calls the large language model to process the text information, and based on the semantic analysis, the user's pending business corresponding to the text information and the corresponding pending business features can be determined. In a possible implementation, after receiving the user interaction information input by the user, for example, after receiving the text information, the terminal device inputs the text information into the large language model, and the large language model processes the text information based on the text encoder to obtain the text information features, and then the large language model performs semantic analysis based on the text information features to determine the user's pending business and the corresponding pending business features; for example, if the text information input by the user is "Please introduce business task_1 to me", the large language model determines that the user's pending business is business task_1 based on "Please introduce business task_1 to me", and the entries corresponding to the pending business features are: "Business introduction of business task_1", "Business content of business task_1", "Business details of business task_1", that is, the pending business features are determined, that is, the business processing information is obtained.

[0052] Furthermore, the user interaction information includes at least one of the following interaction information: text information, audio information or image information, Figure 3 for Figure 2 The schematic diagram of the specific implementation process of step S101 in the embodiment shown is as follows: Figure 3 As shown, the specific implementation process of step S101 includes:

[0053] Step S1011, input the user interaction information into the pre-trained multimodal large model to obtain interaction instruction features, the interaction instruction features include text instruction features, audio instruction features or image instruction features, the text instruction features correspond to text information, the audio instruction features correspond to audio information, and the image instruction features correspond to image information.

[0054] Step S1012, obtaining business processing information according to the interaction instruction characteristics.

[0055] Exemplarily, the user interaction information includes at least one of the following interaction information: text information, audio information or image information. After the terminal device inputs the user interaction information into the pre-trained multimodal large model, the pre-trained multimodal large model processes the text information through a text encoder to generate text instruction features, the pre-trained multimodal large model processes the audio information through an audio encoder to generate audio instruction features, and the pre-trained multimodal large model processes the image information through an image encoder to generate image instruction features, that is, based on the text instruction features, and / or audio instruction features, and / or image instruction features, the interaction instruction features are obtained; further, in a possible implementation method, if the interaction instruction features include two or more instruction features, for example, the interaction instruction features include text instruction features and image instruction features, the pre-trained multimodal large model enhances the text instruction features based on the image instruction features, and then obtains the enhanced text instruction features, and then based on the enhanced text instruction features, obtains the to-do business features corresponding to the user's to-do business, that is, obtains the business processing information.

[0056] In the steps of this embodiment, the user interaction information is processed by a pre-trained multimodal large model to obtain interaction instruction features. Furthermore, if the interaction instruction features include two or more instruction features, the pre-trained multimodal large model uses one interaction instruction feature to enhance the features of another interaction instruction feature, thereby generating enhanced interaction instruction features, and based on the enhanced interaction instruction features, determining business processing information, providing accurate data support for determining corresponding target business information in subsequent steps, and thereby realizing accurate business information corresponding to the user's pending business being fed back to the user, thereby improving user experience.

[0057] Further, in another possible implementation, the image information includes user body image information, the user body image information is used to indicate the user's body movements within a preset time interval, and the interaction instruction feature includes a body interaction instruction feature; Figure 4 for Figure 3 The schematic diagram of the specific implementation process of step S1011 in the embodiment shown is as follows: Figure 4 As shown, the specific implementation process of step S1011 includes:

[0058] Step S10111, obtaining a limb video frame according to the user limb image information, wherein the limb video frame includes at least two sub-limb video frames, and the sub-limb video frames in the limb video frame are arranged based on the action timing characteristics corresponding to the limb action.

[0059] Step S10112, input the limb video frame into the pre-trained multimodal large model to obtain limb interaction command features, where the limb interaction command features represent image command features determined based on action timing features.

[0060] Exemplarily, the action timing feature is used to indicate the sequence of the user's limb movements in the user's limb image information, and then the terminal device extracts the video frame of the user's limb image information to obtain a limb video frame, wherein the limb video frame includes at least two sub-limb video frames, and the above-mentioned sub-limb video frames are arranged based on the action timing feature corresponding to the limb movement. Furthermore, the pre-trained multimodal large model can obtain the corresponding limb interaction instruction feature by encoding and parsing the limb video frame based on the action timing feature, wherein the limb interaction instruction feature also has the action timing feature.

[0061] In a possible implementation, the user's body image information is a 10-second video of the user's body movements. In this video, from the 0th second to the 2nd second, the user performs body movement moves_1, from the 2nd second to the 3rd second, the user does not perform body movement, from the 3rd second to the 6th second, the user performs body movement moves_2, from the 6th second to the 8th second, the user does not perform body movement, and from the 8th second to the 10th second, the user performs body movement moves_3. When the duration of no body movement is less than or equal to 1 second, the action pause time is not taken as the value. The basis for dividing the time interval is to merge the body movements moves_1 and moves_2 between the 0th second and the 6th second into a continuous body movement moves_0. When the duration of no body movement is greater than 1 second, the pause time of the movement is used as the basis for dividing the time interval, that is, the body movement moves_3 is determined as a continuous body movement and is not merged with the body movement moves_2 (or the body movement moves_0). That is, the preset time interval is divided and determined based on the length of the pause time of the body movement.

[0062] Further, Figure 5 A schematic diagram of a process for obtaining body interaction instruction features based on user body image information provided by this application, such as Figure 5As shown, the terminal device extracts video frames from the 10-second video of the limb movement every 500 milliseconds to obtain a limb video frame composed of 20 sub-limb video frames, wherein the 20 sub-limb video frames are arranged based on the action timing features, wherein the 1st to 12th sub-limb video frames correspond to the continuous limb movement moves_0, and the 17th to 20th sub-limb video frames correspond to the continuous limb movement moves_3; then the pre-trained multimodal large model encodes and parses the 20 sub-limb video frames to obtain the limb interaction instruction feature feat_limb_0 corresponding to the continuous limb movement moves_0 and the limb interaction instruction feature feat_limb_3 corresponding to the continuous limb movement moves_3, and then based on the limb interaction instruction feature feat_limb_0 and the limb interaction instruction feature feat_limb_3, the limb interaction instruction feature feat_limb_g corresponding to the 10-second video of the limb movement (user limb image information) is obtained.

[0063] In the steps of this embodiment, by extracting video frames of the user's body movements within a preset time interval, body video frames arranged based on the timing characteristics of the movements are obtained, and then decoding and parsing are performed based on the pre-trained multimodal large model to obtain the body interaction instruction characteristics corresponding to the user's body movements, which provides data support for the subsequent virtual digital human to feedback business information corresponding to the user's pending business to special groups (for example, the deaf-mute people). It not only establishes a communication channel between special groups (for example, the deaf-mute people) and business service personnel who do not understand sign language, that is, business service personnel who do not understand sign language can communicate with special groups (for example, the deaf-mute people) through virtual digital humans, and then complete the answer and / or processing of the business, but also enables special groups (for example, the deaf-mute people) to directly handle the user's pending business through virtual digital humans.

[0064] Step S102, obtaining target business information according to the business processing information and the business information database, the business information database is used to store business information corresponding to at least one user's pending business and a corresponding business identifier, and the target business information represents business information determined based on the characteristics of the pending business.

[0065] Exemplarily, the business information database stores business information and corresponding business identifiers corresponding to at least one pending business of the user, and then based on the pending business characteristics of the pending business of the user corresponding to the business processing information, matching can be performed in the business information database to determine the corresponding target business information. Specifically, for example, the business processing information includes the business identifier and pending business characteristics of the pending business of the user, and then, according to the business identifier corresponding to the business processing information, the matching target business is determined from the business information database, and then the business information corresponding to the target business is determined, and then based on the pending business characteristics, the corresponding target business information is determined from the business information of the target business.

[0066] In one possible implementation, Figure 6 A schematic diagram of a process for determining target business information provided for this application, such as Figure 6As shown, the business identifier corresponding to the business processing information deal_1 is label_1, the pending business features are feat_1_1, feat_1_2, and feat_1_3, and the business information database includes business A and the corresponding business label task_1, business information info_1_1, business information info_1_2, and business information info_1_3, and business B and the corresponding business label task_2, business information info_2_1, business information info_2_2, business information info_2_3, and business information info_2_4; further, the terminal device matches the business identifier label_1 with the business label task_2, and further determines business B as the user pending business corresponding to the business processing information deal_1; further, the terminal device matches the pending business features feat_1_1, feat_1_2, and feat_1_3 with the business information info_2_1, business information info_2_2, business information info_2_3, and business information info_2_4 corresponding to business B. info_2_4 is matched to obtain the corresponding target business information; for example, if the business information info_2_2 matches with feat_1_3, the business information info_2_2 is determined as the target business information; specifically, the pending business feature feat_1_1 is: "Business introduction of business task_1", the pending business feature feat_1_2 is: "Business content of business task_1", the pending business feature feat_1_3 is: "Business details of business task_1", the business information info_2_1 is: "Processing procedures of business B", the business information info_2_2 is: "Specific business details of business B", the business information info_2_3 is: "Related businesses of business B", and the business information info_2_4 is: "Service area of ​​business B"; based on the matching of the business information info_2_2 with the pending business feature feat_1_2, that is, the keyword "business details" is matched, the business information info_2_2 is determined as the target business information.

[0067] Step S103, generating digital human interaction information according to the target business information, where the digital human interaction information is used to instruct the virtual digital human to feed back business information corresponding to the user's pending business to the user.

[0068] Exemplarily, the digital human interaction information is used to instruct the virtual digital human to feedback the business information corresponding to the user's pending business to the user, and the digital human interaction information includes text interaction information, audio interaction information and image interaction information, wherein the image interaction information includes picture interaction information and video interaction information; further, when the virtual digital human feedbacks the business information corresponding to the user's pending business to the user, the feedback form adopted includes text form, audio form and image form (picture and / or video), so that the user can obtain the business information corresponding to the user's pending business through the screen display end and / or audio output end of the terminal device. Specifically, for example, the terminal device generates target information in an output data format that can be called by the virtual digital human based on the determined target business information, and the output data format includes a text output data format, an audio output data format and an image data format, and the target information is the digital human interaction information.

[0069] In one possible implementation, full-version business information and simplified-version business information are obtained based on the target business information, wherein the full-version business information is used to indicate the complete business information corresponding to the user's pending business, and the simplified-version business information is used to indicate the simplified business information corresponding to the user's pending business, and the simplified-version business information includes an identification code of the full-version business information, and the identification code is used to provide the full-version business information to the user when it is recognized by the user's terminal device; digital human interaction information is generated based on the full-version business information and the simplified-version business information. Business information info_2_2: "Specific business details of business B" is processed by the terminal device as the target business information. Specifically, for example, the number of text characters in business information info_2_2 is 100. The terminal device generates corresponding text interaction information, audio interaction information and image interaction information by identifying paragraphs and numbers therein, wherein the text interaction information is a text introduction of the complete business details corresponding to the 100 text characters, the audio interaction information is an audio introduction of the complete business details generated based on the 100 text characters, and the image interaction information is a picture introduction generated by simplifying the 100 text characters based on paragraphs and numbers, that is, the text interaction information and the audio interaction information correspond to the full version of the business information, and the image interaction information corresponds to the simplified version of the business information; Figure 7 A schematic diagram of a scenario in which a virtual digital person provides feedback of business information to a user, such as Figure 7As shown, on the upper left side of the virtual digital human, a complete text introduction of "Specific business details of business B" is displayed, and on the upper right side of the virtual digital human, a simplified picture introduction of "Specific business details of business B" is displayed, wherein the picture can also be saved and shared by the user, and the terminal device loads the audio interaction information to enable the virtual digital human to introduce the specific business details of business B to the user in audio format, that is, to provide accurate business information to the user, wherein the audio duration converted from 100 text characters is, for example, 25 seconds; further, the picture also includes an identification code (for example, a bar code, a QR code). After the user saves the picture, without using the virtual digital human, the user can still use the terminal device to scan the QR code, so that the display interface of the terminal device displays the complete text introduction and / or audio introduction of "Specific business details of business B".

[0070] In this embodiment, business processing information is obtained according to user interaction information, the user interaction information represents the interaction instructions input by the user when handling the user's pending business through the virtual digital person, and the business processing information represents the pending business characteristics corresponding to the user's pending business; according to the business processing information and the business information database, the target business information is obtained, the business information database is used to store at least one business information corresponding to the user's pending business and the corresponding business identifier, and the target business information represents the business information determined based on the pending business characteristics; according to the target business information, digital human interaction information is generated, and the digital human interaction information is used to instruct the virtual digital person to feedback the business information corresponding to the user's pending business to the user. By performing semantic analysis and processing on the interaction instructions input by the user when handling the user's pending business through the virtual digital person, the pending business characteristics corresponding to the user's pending business are determined, and then the target business information is determined in the business information database based on the pending business characteristics, and then by performing data processing on the target business information, the virtual digital person is realized to feedback the business information to the user, that is, to provide the user with accurate business information, solving the problem of poor user experience when the digital customer service answers the user's questions.

[0071] Furthermore, the virtual digital human can also identify the urgency of the pending business that the user is handling (consulting) to provide feedback to the user on corresponding business information; specifically, the image information includes user expression image information, and the audio information includes user voice feature information. The user emotion information is obtained based on the user expression image information and / or user voice feature information, and the user emotion information represents the urgency of the pending business that the user is handling (consulting); digital human interaction information is generated based on the user emotion information and the target business information.

[0072] In a possible implementation, the user expression image information is used to indicate the user's facial expression, such as smile, tension, anger, and the user voice feature information is used to indicate the volume, pitch and / or speed of the user's speech; after the terminal device receives the text information, and / or audio information, and / or image information input by the user, the corresponding user expression image information is extracted from the image information, and the corresponding user voice feature information is extracted from the audio information; then, by quantifying the user expression image information and / or the user voice feature information, a user emotion evaluation score is calculated based on a weighted sum method, that is, the urgency of the user's pending business handled (consulted) by the user is obtained, that is, the user emotion information is obtained. For example, the user emotion evaluation score score_0 is calculated by formula (1),

[0073]

[0074] Among them, p_1 is the weighting coefficient corresponding to the user's facial expression image information, face_i is the score corresponding to different facial expressions of the user, p_2 is the weighting coefficient corresponding to the user's voice feature information, p_3 is the weighting coefficient corresponding to the volume of the user's speech, vol_j is the score corresponding to the volume of the user's speech, p_4 is the weighting coefficient corresponding to the pitch of the user's speech, ton_j is the score corresponding to the tone of the user's speech, p_5 is the weighting coefficient corresponding to the speed of the user's speech, spe_j is the score corresponding to the tone of the user's speech, and the sum of p_3, p_4 and p_5 is 1;

[0075] Exemplarily, the numerical range corresponding to the quantization of the user's expression image information and / or the user's voice feature information is 1 to 10. The higher the value of the user's emotion evaluation score score_0, the higher the urgency of the user's pending business handled (consulted) by the corresponding user. Correspondingly, for the value of face_i, if the user's expression is smiling, the value of face_i is, for example, 3; if the user's expression is nervous, the value of face_i is, for example, 6; if the user's expression is angry or impatient, The value of face_i is, for example, 10; for the value of vol_j, the lower the volume of the user's speech, the lower the value, the lowest being 1, and the louder the volume of the user's speech, the higher the value, the highest being 10; for the value of ton_j, the lower the pitch of the user's speech, the lower the value, the lowest being 1, and the higher the pitch of the user's speech, the higher the value, the highest being 10; for the value of spe_j, the slower the user's speaking speed, the lower the value, the lowest being 1, and the faster the user's speaking speed, the higher the value, the highest being 10.

[0076] Furthermore, after obtaining the user emotion information corresponding to the user emotion evaluation score, the terminal device generates digital human interaction information in combination with the target business information; specifically, for example, the urgency corresponding to the user emotion information is determined in a quantitative form, for example, the user emotion evaluation score is equal to the urgency quantization value of the urgency, then when the urgency quantization value is less than or equal to a first preset value, the urgency is low, when the urgency quantization value is greater than the first preset value and less than or equal to a second preset value, the urgency is high, when the urgency quantization value is greater than the second preset value, the urgency is high; and then after the terminal device determines the target business information, if the urgency corresponding to the user emotion information is low, the virtual digital human generates digital human interaction information based on the target business information. The target business information will be used to feedback the business information corresponding to the user's pending business to the user; if the urgency corresponding to the user's emotional information is high, the virtual digital human will feedback the business information corresponding to the user's pending business to the user based on the target business information. At the same time, the virtual digital human will mark the user as a concerned user and prompt the business service personnel to prepare to intervene and provide manual assistance / or manual service to the user; if the urgency corresponding to the user's emotional information is high, the virtual digital human will feedback the business information corresponding to the user's pending business to the user based on the target business information. At the same time, the virtual digital human will inform the user that the business service personnel will provide you with manual assistance / or manual service, and the virtual digital human will prompt the business service personnel to provide manual assistance / or manual service to the user in a timely manner.

[0077] In the steps of this embodiment, the corresponding user emotional information is determined by identifying the user's facial expression image information and the user's voice feature information, and then the urgency of the user's pending business (consultation) is obtained; and according to the different urgency levels, the virtual digital person provides different feedback forms to the user, and promptly prompts the corresponding business service personnel to intervene in time to provide the user with manual assistance / or manual service, thereby solving the problem of poor user experience when digital customer service answers user questions.

[0078] Furthermore, the virtual digital human can also adaptively adjust the feedback form when feeding back business information to users according to different users; specifically, based on the user login information and user interaction information, generate the digital human configuration information, the user login information represents the user identity characteristics when the user logs in to the business system corresponding to the user's pending business, and the digital human configuration information represents the personalized configuration characteristics of the virtual digital human; based on the digital human configuration information and the digital human interaction information, generate the target interaction information, the target interaction information represents the interaction characteristics when the virtual digital human interacts with the user.

[0079] In a possible implementation form, when a user interacts with a virtual digital human, the terminal device obtains the user identity characteristics corresponding to the user login information, and then generates personalized configuration characteristics for the virtual digital human based on the user identity characteristics and the user interaction information, that is, generates digital human configuration information; specifically, for example, the user identity characteristics corresponding to the user login information are: the user's residence area is area A, the user's age is 60 years old, and the audio information corresponding to the user interaction information is the regional dialect of area A, then the personalized configuration characteristics of the virtual digital human generated by the terminal device include: switching the clothing image of the virtual digital human to the regional clothing of area A; The terminal device updates the digital human interaction information and the virtual digital human according to the digital human configuration information, and obtains the target interaction information and the updated virtual digital human, wherein the clothing image of the updated virtual digital human is the regional clothing image of region A, and the updated virtual digital human interacts with the user with the target interaction information (the form corresponding to the above-mentioned personalized configuration features) and feeds back the corresponding business information to the user.

[0080] In the steps of this embodiment, based on the user identity characteristics when the user logs into the business system corresponding to the user's to-do business and the user interaction information between the user and the virtual digital person, a personalized configuration feature associated with the user and for the virtual digital person is generated, and then the virtual digital person is updated to enable the updated virtual digital person to interact with the user and feedback business information based on the personalized configuration features, thereby improving the user's interactive experience and solving the problem of poor user experience.

[0081] Figure 8 A flowchart of a virtual digital human interaction method provided by another embodiment of the present application is shown in FIG. Figure 8 As shown, the virtual digital human interaction method provided in this embodiment is Figure 2 On the basis of the virtual digital human interaction method provided in the illustrated embodiment, step S102 is further refined, the business information database includes a first database and a second database, the first database is used to indicate a business information database of high-frequency access services, and the second database is used to indicate a business information database of low-frequency access services. The virtual digital human interaction method provided in this embodiment includes the following steps:

[0082] Step S201, obtaining business processing information according to user interaction information, wherein the user interaction information represents the interaction instructions input by the user when handling the user's pending business through a virtual digital person, and the business processing information represents the pending business characteristics corresponding to the user's pending business.

[0083] Step S202: determine the contracted service information according to the user login information, where the contracted service information is used to indicate the high-frequency access services and / or low-frequency access services contracted by the user.

[0084] Step S203, determining the contracted service ratio information according to the contracted service information, wherein the contracted service ratio information represents the ratio between the number of high-frequency access services contracted by the user and the number of low-frequency access services contracted by the user.

[0085] Exemplarily, the user login information is used to indicate the user identity characteristics when the user logs into the business system corresponding to the user's pending business, and then based on the user identity characteristics, the business signed by the user and the type of the business signed by the user can be determined, that is, whether the business signed by the user is a high-frequency access business or a low-frequency access business, that is, the contracted business information is obtained; and then according to the type of the business signed by the user, the contracted business is divided into two categories: high-frequency access business and low-frequency access business, and the corresponding number of businesses is counted, that is, the number of high-frequency access businesses signed by the user and the number of low-frequency access businesses signed by the user are obtained, and then the proportion information of the contracted business is obtained.

[0086] In one possible implementation, the terminal device determines the services signed by the user based on the user identity characteristics corresponding to the user login information: service busi_1, service busi_2, service busi_3, service busi_4, and service busi_5, and then classifies service busi_1, service busi_2, service busi_3, service busi_4, and service busi_5 based on the services included in the high-frequency access services and the low-frequency access services. For example, busi_1, service busi_2, and service busi_4 belong to the high-frequency access services, and service busi_3 and service busi_5 belong to the low-frequency access services. Then, the number of high-frequency access services signed by the user is 3, and the number of low-frequency access services signed by the user is 2. Therefore, the proportion relationship corresponding to the contracted service ratio information is 3:2, which is converted into a percentage ratio of 150%.

[0087] Step S204, obtaining target business information according to the business processing information, the contracted business ratio information, the first database and the second database.

[0088] Exemplarily, based on the contracted service ratio information, that is, whether the service contracted by the user tends to be a high-frequency access service or a low-frequency access service, it is determined whether the terminal device first accesses the first database or the second database to determine the target service information corresponding to the service processing information. Specifically, if the ratio corresponding to the contracted service ratio information is greater than the preset ratio, the first database and the second database are accessed in sequence according to the service processing information to obtain the target service information; or, if the ratio corresponding to the contracted service ratio information is less than or equal to the preset ratio, the second database and the first database are accessed in sequence according to the service processing information to obtain the target service information.

[0089] In a possible implementation, the preset ratio is 120%, the first database is used to indicate the service information database of high-frequency access services, and the second database is used to indicate the service information database of low-frequency access services; if the proportion relationship corresponding to the contracted service ratio information is 3:2, it is converted into a percentage ratio of 150%, which is greater than the preset ratio of 120%, indicating that most of the services signed by the user are high-frequency access services, and the possibility that the user's pending services corresponding to the service processing information are high-frequency access services is greater than the possibility of being low-frequency access services, so the first database and the second database are accessed in turn to obtain the target service information; if the proportion relationship corresponding to the contracted service ratio information is 7:6, it is converted into a percentage ratio of 116.7%, which is less than the preset ratio of 120%, indicating that compared with other users, the low-frequency access services signed by the user are also maintained at a relatively high number, and the possibility that the user's pending services corresponding to the service processing information are low-frequency access services is greater than the possibility of being high-frequency access services, so the second database and the first database are accessed in turn to obtain the target service information.

[0090] In this embodiment, by analyzing and classifying the types of services signed by the user, the number of high-frequency access services signed by the user, the number of low-frequency access services signed by the user, and the ratio therebetween, i.e., the contracted service ratio information, are obtained. Then, based on the contracted service ratio information, the order of accessing the first database and the second database is determined to improve the efficiency of obtaining the target service information, i.e., improve the feedback efficiency when feeding back relevant service information to the user in subsequent steps, shorten the user's waiting time, and improve the user experience.

[0091] Furthermore, after step S202 and before step S203, it also includes: if the user's pending business corresponding to the business processing information is a business that the user has signed a contract for, the terminal device directly accesses the database to which the business belongs to obtain the corresponding target business information, and then directly executes step S205; specifically, for example, if the user's pending business corresponding to the business processing information is the business busi_5 that the user has signed a contract for, and business busi_5 is a low-frequency access business, the terminal device matches the business information in the second database according to the business processing information, and then obtains the corresponding target business information.

[0092] Step S205: Generate digital human interaction information according to the target business information. The digital human interaction information is used to instruct the virtual digital human to feed back business information corresponding to the user's pending business to the user.

[0093] In this embodiment, the implementation method of step S201 and step S205 is the same as that of the present application. Figure 2 The implementation methods of step S101 and step S103 in the illustrated embodiment are the same and will not be described in detail here.

[0094] It can be understood that the user pending services mentioned in the embodiments of the present application can be business consultation and processing corresponding to the services that the user has signed a contract for, or can be business consultation and processing corresponding to the services that the user has not signed a contract for.

[0095] Fig. 9 A schematic diagram of the structure of a virtual digital human interaction device provided in one embodiment of the present application is shown in FIG. Fig. 9 As shown, the virtual digital human interaction device 3 provided in this embodiment includes:

[0096] The processing module 31 is used to obtain business processing information according to the user interaction information, wherein the user interaction information represents the interaction instructions input by the user when the user handles the user's pending business through the virtual digital person, and the business processing information represents the pending business characteristics corresponding to the user's pending business;

[0097] A determination module 32, configured to obtain target business information according to the business processing information and a business information database, wherein the business information database is configured to store business information corresponding to at least one pending business of a user and a corresponding business identifier, wherein the target business information represents business information determined based on the characteristics of the pending business;

[0098] The generating module 33 is used to generate digital human interaction information according to the target business information, and the digital human interaction information is used to instruct the virtual digital human to feed back the business information corresponding to the user's pending business to the user.

[0099] In one possible design, the user interaction information includes at least one of the following interaction information: text information, audio information or image information. When the processing module 31 obtains the business processing information based on the user interaction information, it is specifically used to: input the user interaction information into a pre-trained multimodal large model to obtain interaction instruction features, the interaction instruction features include text instruction features, audio instruction features or image instruction features, the text instruction features correspond to the text information, the audio instruction features correspond to the audio information, and the image instruction features correspond to the image information; obtain the business processing information based on the interaction instruction features.

[0100] In one possible design, the image information includes user limb image information, and the user limb image information is used to indicate the user's limb movements within a preset time interval, and the interaction instruction features include limb interaction instruction features; when the processing module 31 inputs the user interaction information into a pre-trained multimodal large model to obtain the interaction instruction features, it is specifically used to: obtain a limb video frame based on the user limb image information, the limb video frame includes at least two sub-limb video frames, and the sub-limb video frames in the limb video frame are arranged based on the action timing features corresponding to the limb movements; input the limb video frame into the pre-trained multimodal large model to obtain the limb interaction instruction features, and the limb interaction instruction features represent the image instruction features determined based on the action timing features.

[0101] In a possible design, the image information includes user expression image information, the audio information includes user voice feature information, and the generation module 33 is further used to: obtain user emotion information based on the user expression image information and / or user voice feature information, and the user emotion information represents the urgency of the user pending business handled by the user; when the generation module 33 generates digital human interaction information based on the target business information, it is specifically used to: generate digital human interaction information based on the user emotion information and the target business information.

[0102] In a possible design, the generation module 33 is also used to: generate digital human configuration information based on user login information and user interaction information, the user login information represents the user identity characteristics when the user logs in to the business system corresponding to the user's pending business, and the digital human configuration information represents the personalized configuration characteristics of the virtual digital human; generate target interaction information based on the digital human configuration information and the digital human interaction information, and the target interaction information represents the interaction characteristics when the virtual digital human interacts with the user.

[0103] In one possible design, the business information database includes a first database and a second database, the first database is used to indicate a business information database for high-frequency access services, and the second database is used to indicate a business information database for low-frequency access services; when the determination module 32 obtains the target business information based on the business processing information and the business information database, it is specifically used to: determine the contracted business information based on the user login information, the contracted business information is used to indicate the high-frequency access services and / or low-frequency access services signed by the user; determine the contracted business ratio information based on the contracted business information, the contracted business ratio information represents the proportional relationship between the number of high-frequency access services signed by the user and the number of low-frequency access services signed by the user; obtain the target business information based on the business processing information, the contracted business ratio information, the first database and the second database.

[0104] In one possible design, when the determination module 32 obtains the target business information based on the business processing information, the contracted business ratio information, the first database and the second database, it is specifically used to: if the ratio corresponding to the contracted business ratio information is greater than the preset ratio, then according to the business processing information, the first database and the second database are accessed in sequence to obtain the target business information; or, if the ratio corresponding to the contracted business ratio information is less than or equal to the preset ratio, then according to the business processing information, the second database and the first database are accessed in sequence to obtain the target business information.

[0105] In a possible design, when the generation module 33 generates digital human interaction information according to the target business information, it is specifically used to: obtain full-version business information and simplified business information according to the target business information, wherein the full-version business information is used to indicate the complete business information corresponding to the user's pending business, and the simplified business information is used to indicate the simplified business information corresponding to the user's pending business, and the simplified business information includes an identification code of the full-version business information, and the identification code is used to provide the full-version business information to the user when it is recognized by the user's terminal device; generate digital human interaction information according to the full-version business information and the simplified business information.

[0106] The processing module 31, the determining module 32 and the generating module 33 are connected in sequence. The virtual digital human interaction device 3 provided in this embodiment can execute the following steps: Figure 2-Figure 8 The technical solutions of any of the method embodiments shown have similar implementation principles and technical effects, which will not be described in detail here.

[0107] The virtual digital human interaction device provided in the embodiment of the present application can be used to execute the technical solution of the virtual digital human interaction method in the above embodiment. Its implementation principle and technical effect are similar and will not be repeated here.

[0108] It should be noted that it should be understood that the division of the various modules of the above device is only a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. And these modules can all be implemented in the form of software called by processing elements; they can also be all implemented in the form of hardware; some modules can also be implemented in the form of software called by processing elements, and some modules can be implemented in the form of hardware. For example, the processing module 31 can be a separately established processing element, or it can be integrated in a chip of the above device. In addition, it can also be stored in the memory of the above device in the form of program code, and called and executed by a processing element of the above device. The function of the above processing module 31. The implementation of other modules is similar. In addition, these modules can be fully or partially integrated together, or they can be implemented independently. The processing element here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each module above can be completed by an integrated logic circuit of hardware in the processor element or instructions in the form of software.

[0109] Fig.10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Fig.10 As shown, the electronic device may include: a transceiver 121 , a processor 122 , and a memory 123 .

[0110] The processor 122 executes the computer execution instructions stored in the memory, so that the processor 122 executes the scheme in the above embodiment. The processor 122 can be a general-purpose processor, including a central processing unit CPU, a network processor (network processor, NP), etc.; it can also be a digital signal processor DSP, an application-specific integrated circuit ASIC, a field programmable gate array FPGA or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.

[0111] The memory 123 is connected to the processor 122 via a system bus and completes communication between them. The memory 123 is used to store computer program instructions.

[0112] The transceiver 121 may be used to obtain tasks to be executed and configuration information of the tasks to be executed.

[0113] The system bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The system bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus. The transceiver is used to realize the communication between the database access device and other computers (such as clients, read-write libraries, and read-only libraries). The memory may include random access memory (RAM) and may also include non-volatile memory.

[0114] The electronic device provided in the embodiment of the present application may be the terminal device of the above embodiment.

[0115] The embodiment of the present application also provides a chip for running instructions, which is used to execute the technical solution of the virtual digital human interaction method in the above embodiment.

[0116] The embodiment of the present application also provides a computer-readable storage medium, in which computer instructions are stored. When the computer instructions are executed on a computer, the computer executes the technical solution of the virtual digital human interaction method of the above embodiment.

[0117] An embodiment of the present application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium. When at least one processor executes the computer program, the technical solution of the virtual digital human interaction method in the above embodiment can be implemented.

[0118] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules is only a logical function division, and there may be other division methods in actual implementation, such as multiple modules can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0119] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to implement the solution of this embodiment.

[0120] In addition, each functional module in each embodiment of the present application can be integrated into one processing unit, or each module can exist physically separately, or two or more modules can be integrated into one unit. The above-mentioned module-composed unit can be implemented in the form of hardware or in the form of hardware plus software functional units.

[0121] The above-mentioned integrated module implemented in the form of a software function module can be stored in a computer-readable storage medium. The above-mentioned software function module is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform some steps of the methods of various embodiments of the present application.

[0122] It should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the invention may be directly implemented as being executed by a hardware processor, or may be implemented by a combination of hardware and software modules in the processor.

[0123] The memory may include a high-speed RAM memory, and may also include a non-volatile storage NVM, such as at least one disk memory, and may also be a USB flash drive, a mobile hard disk, a read-only memory, a magnetic disk or an optical disk, etc.

[0124] The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of the present application is not limited to only one bus or one type of bus.

[0125] The above storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The storage medium can be any available medium that can be accessed by a general or special purpose computer.

[0126] An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the storage medium can also exist as discrete components in an electronic control unit or a main control device.

[0127] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, disk or optical disk and other media that can store program codes.

[0128] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A virtual digital human interaction method, characterized in that: include: Obtaining business processing information according to the user interaction information, wherein the user interaction information represents the interaction instructions input by the user when handling the user's pending business through the virtual digital person, and the business processing information represents the pending business characteristics corresponding to the user's pending business; Obtain target business information according to the business processing information and a business information database, wherein the business information database is used to store business information corresponding to at least one pending business of a user and a corresponding business identifier, and the target business information represents business information determined based on the characteristics of the pending business; According to the target business information, digital human interaction information is generated, and the digital human interaction information is used to instruct the virtual digital human to feed back business information corresponding to the user's pending business to the user.

2. The method according to claim 1, characterized in that The user interaction information includes at least one of the following interaction information: text information, audio information or image information, and obtaining the service processing information according to the user interaction information includes: Inputting the user interaction information into a pre-trained multimodal large model to obtain an interaction instruction feature, wherein the interaction instruction feature includes a text instruction feature, an audio instruction feature or an image instruction feature, wherein the text instruction feature corresponds to the text information, the audio instruction feature corresponds to the audio information, and the image instruction feature corresponds to the image information; The business processing information is obtained according to the interaction instruction characteristics.

3. The method according to claim 2, characterized in that The image information includes user limb image information, the user limb image information is used to indicate the user's limb movements within a preset time interval, and the interaction instruction feature includes a limb interaction instruction feature; The step of inputting the user interaction information into a pre-trained multimodal large model to obtain interaction instruction features includes: Obtaining a limb video frame according to the user limb image information, wherein the limb video frame includes at least two sub-limb video frames, and the sub-limb video frames in the limb video frame are arranged based on the action timing characteristics corresponding to the limb action; The limb video frame is input into a pre-trained multimodal large model to obtain the limb interaction instruction feature, wherein the limb interaction instruction feature represents the image instruction feature determined based on the action timing feature.

4. The method according to claim 2, characterized in that: The image information includes user expression image information, the audio information includes user voice feature information, and the method further includes: Obtaining user emotion information according to user expression image information and / or user voice feature information, wherein the user emotion information represents the urgency of the pending user service handled by the user; The step of generating digital human interaction information according to the target business information includes: The digital human interaction information is generated according to the user emotion information and the target business information.

5. The method according to claim 1, characterized in that The method further comprises: Generate digital human configuration information according to the user login information and the user interaction information, wherein the user login information represents the user identity characteristics when the user logs into the business system corresponding to the user's pending business, and the digital human configuration information represents the personalized configuration characteristics of the virtual digital human; Target interaction information is generated according to the digital human configuration information and the digital human interaction information, wherein the target interaction information represents interaction characteristics when the virtual digital human interacts with the user.

6. The method according to claim 1, characterized in that The service information database includes a first database and a second database, the first database is used to indicate a service information database of high-frequency access services, and the second database is used to indicate a service information database of low-frequency access services; The step of obtaining target business information according to the business processing information and the business information database includes: Determine contracted service information according to user login information, where the contracted service information is used to indicate the high-frequency access service and / or low-frequency access service contracted by the user; Determine contracted service ratio information according to the contracted service information, wherein the contracted service ratio information represents a ratio between the number of high-frequency access services contracted by the user and the number of low-frequency access services contracted by the user; The target business information is obtained according to the business processing information, the contracted business proportion information, the first database and the second database.

7. The method according to claim 6, characterized in that The obtaining the target service information according to the service processing information, the contracted service ratio information, the first database and the second database includes: If the ratio corresponding to the contracted business ratio information is greater than a preset ratio, accessing the first database and the second database in sequence according to the business processing information to obtain the target business information; Or, if the ratio corresponding to the contracted business proportion information is less than or equal to the preset ratio, the second database and the first database are accessed in sequence according to the business processing information to obtain the target business information.

8. The method according to claim 6, characterized in that The step of generating digital human interaction information according to the target business information includes: According to the target service information, full-version service information and simplified-version service information are obtained, wherein the full-version service information is used to indicate the complete service information corresponding to the user's pending service, and the simplified-version service information is used to indicate the simplified service information corresponding to the user's pending service, and the simplified-version service information includes an identification code of the full-version service information, and the identification code is used to provide the full-version service information to the user when it is recognized by the user's terminal device; Digital human interaction information is generated based on the full version of business information and the simplified version of business information.

9. A virtual digital human interaction device, characterized in that: include: A processing module, used to obtain business processing information according to user interaction information, wherein the user interaction information represents the interaction instructions input by the user when handling the user's pending business through a virtual digital person, and the business processing information represents the pending business characteristics corresponding to the user's pending business; A determination module, configured to obtain target business information according to the business processing information and a business information database, wherein the business information database is configured to store business information corresponding to at least one pending business of a user and a corresponding business identifier, wherein the target business information represents business information determined based on the characteristics of the pending business; A generation module is used to generate digital human interaction information according to the target business information, wherein the digital human interaction information is used to instruct the virtual digital human to feed back business information corresponding to the user's pending business to the user.

10. An electronic device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 7.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 8 when executed by a processor.

12. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 1 to 8 when being executed by a processor.

Images