Digital human and web interaction method and system

Abstract

The invention relates to an interaction method and system of a digital human and a web, and belongs to the technical field of computers.The interaction method of the digital human and the web comprises the steps that voice signals in the interaction process of the digital human and a user are obtained in real time, the voice signals comprise instruction information of the user, the voice signals are recognized, answer texts are obtained, and the answer texts are sent to the user; the answer text is analyzed, the calling instruction is obtained, the web is called based on the calling instruction, interactive operation of the digital human and the web is achieved, mouse operation is not needed, a convenient man-machine interaction mode is achieved, user experience is enhanced, and interactivity is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method and system for interaction between a digital human and a web. Background Art

[0002] The web (World Wide Web) is a global, dynamically interactive, cross-platform distributed graphic information system based on hypertext and HTTP. It is a network service built on the Internet that provides a graphical, easy-to-access, intuitive interface for viewers to search and browse information on the Internet.

[0003] In the process of controlling the web system, the mouse is usually used to click the menu or button on the page to control the page jump or call the remote interface. This method is relatively traditional and mechanical, and the user experience is poor. Summary of the invention

[0004] In view of this, it is necessary to provide a method and system for interaction between a digital human and the web to solve the technical problem of poor user experience in operating web pages.

[0005] In order to solve the above problems, the present invention provides a method for interaction between a digital human and a web, comprising: Acquire in real time the voice signal during the interaction between the digital human and the user, wherein the voice signal includes the user's instruction information; The voice signal is recognized to obtain an answer text, the answer text is parsed to obtain a calling instruction, and the web is called based on the calling instruction to realize the interactive operation between the digital human and the web.

[0006] In a possible implementation, the digital human includes a digital human client and a digital human server; the real-time acquisition of the voice signal during the interaction between the digital human and the user includes: The voice signal in the process of interaction between the user and the Digital Human client is obtained, and the voice signal is transparently transmitted to the Digital Human server through the streaming media protocol, wherein the streaming media protocol at least includes RTMP.

[0007] In a possible implementation, the recognizing the voice signal to obtain the answer text includes: The digital human server calls ASR to perform text recognition on the voice signal to obtain the question text; Based on the question text, the digital human server calls the knowledge base to configure the answer to the question text to obtain the answer text.

[0008] In a possible implementation, the answer text includes an instruction set that is set in a specific format in a configurable form, wherein the instruction set includes interaction instructions and parameters.

[0009] In a possible implementation, the calling instruction includes a first instruction, a second instruction, a third instruction, and a fourth instruction, and the web includes a web end and a web service end; calling the web based on the calling instruction to implement the interactive operation between the digital human and the web includes: Based on the first instruction, the web server is called, and after obtaining the query data through the web server, the query data is sent to the digital human client, and the query data is broadcasted in real time through the digital human client; Calling a web server based on the second instruction, and controlling the IoT device through the web server to achieve device linkage; Calling the digital human client based on the third instruction, parsing the third instruction through the digital human client, and sending the parsed third instruction to the web end to realize the automated operation of the web page; Based on the fourth instruction, the web server, the digital human client and the web client are called to realize the real-time broadcast of the digital human client and the automatic operation of the web page.

[0010] In a possible implementation, the calling of the web server based on the first instruction, after obtaining the query data through the web server, sending the query data to the digital human client, and broadcasting the query data in real time through the digital human client, includes: Obtaining a remote interface query data identifier, an interface address, a request type, and parameters based on the first instruction, wherein the remote interface query data identifier is QUERY_INTERFACE; Based on the remote interface query data identifier, interface address, request type and parameters, the digital human server performs an RPC call to call the web service end, obtains the query data corresponding to the interface address through the web service end, assembles the query data with the text of the answer text, obtains the interaction data, and sends the interaction data to the digital human client; The interactive data is reported in real time through the digital human client.

[0011] In a possible implementation, calling a web server based on the second instruction and controlling an IOT device through the web server to achieve device linkage includes: Based on the second instruction, obtaining a remote interface call identifier, an interface address, a request type, and parameters, wherein the remote interface call identifier is CALL_INTERFACE and the request type is POST; Based on the remote interface call identifier, interface address, request type and parameters, an RPC call is made through the digital human server to call the web server, and the IOT device is operated through the web server to complete device linkage.

[0012] In a possible implementation, the calling of the digital human client based on the third instruction, parsing the third instruction by the digital human client, and sending the parsed third instruction to the web end to realize the automated operation of the web page includes: Sending the third instruction to the digital human client through the digital human server, and calculating the time interval for sending the instructions; The third instruction is parsed by the digital human client, and the parsed third instruction is sent to the web end, wherein the parsed third instruction at least includes a page jump mark, a page mode highlight mark and a page address, and the digital human client is connected to the web end via a websocket; Based on the page jump identifier, the page mode highlight identifier, the page address and the time interval, the web terminal jumps to the web page corresponding to the page address and highlights the web page.

[0013] In a possible implementation, the calling of the web server, the digital human client and the web terminal based on the fourth instruction to realize the real-time broadcast of the digital human client and the automatic operation of the web page includes: Based on the fourth instruction, obtain the remote interface query data identifier, interface address, request type, page jump identifier, page mode highlight identifier, page address and parameters; Based on the remote interface query data identifier, interface address, request type, page jump identifier, page mode highlight identifier, page address and parameters, the web server, digital human client and web terminal are called respectively to realize the real-time broadcast of the digital human client and the automatic operation of the web page.

[0014] On the other hand, the present invention also provides a digital human and web interaction system, comprising: A voice signal acquisition module, used to acquire the voice signal in the process of interaction between the digital human and the user in real time, wherein the voice signal includes the user's instruction information; The interactive module is used to recognize the voice signal, obtain the answer text, parse the answer text, obtain the calling instruction, call the web based on the calling instruction, and realize the interactive operation between the digital human and the web.

[0015] The beneficial effects of the present invention are: obtaining voice signals in the process of interaction between the digital human and the user in real time; in the process of dialogue between the user and the digital human, the digital human issues commands without mouse operation, recognizes voice signals, obtains answer files, parses answer texts, obtains calling instructions, calls the web based on the calling instructions, and realizes interactive operations between the digital human and the web; through the interaction between the digital human and the web, the actions of the digital human can be realized; the web data is dynamically queried and reported in real time by the digital human server calling the web server, thus realizing a convenient human-computer interaction method, enhancing user experience and improving interactivity. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A flowchart of an embodiment of the method for interaction between a digital human and a web provided by the present invention; Figure 2 A flowchart of the digital human and web interaction operation of the digital human and web interaction method provided by the present invention; Figure 3 This is a structural schematic diagram of an embodiment of the digital human interaction system provided by the present invention. DETAILED DESCRIPTION

[0017] The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not used to limit the scope of the present invention.

[0018] A specific embodiment of the present invention discloses a method for interaction between a digital human and a web. Figure 1 is a flowchart of the method for interaction between a digital human and a web provided by an embodiment of the present invention. Figure 1 , the interaction methods between digital humans and the web include: S101, acquiring in real time a voice signal during the interaction between the digital human and the user, wherein the voice signal includes the user's instruction information; S102, recognizing the voice signal to obtain an answer text, parsing the answer text to obtain a calling instruction, calling the web based on the calling instruction, and realizing the interactive operation between the digital human and the web.

[0019] Among them, in the web system, by introducing digital humans (also known as "AI digital humans", "3D digital humans", "virtual images", "virtual idols", "3D digital virtual humans", and "virtual digital humans") to interact with the web system, digital humans include digital human clients and digital human servers, and the web includes web terminals and web servers. The digital human client is a Windows client application, and a graphical interface can be written in languages ​​such as C++ or Python, and runs on the Windows operating system; users interact with the digital human client in real time. During the interaction process, the user's instructions are issued through the digital human client to realize the interactive operation between the digital human and the web, and the interaction results are fed back to the user through the digital human client.

[0020] Compared with the prior art, the method for interacting between a digital human and a web provided in this embodiment can obtain voice signals in real time during the interaction between the digital human and the user, wherein the voice signals include the user's command information. The user can interact with the digital human without mouse operation, and can issue commands in the form of voice signals, recognize the voice signals, obtain answer texts, parse the answer texts, obtain calling commands, and call the web based on the calling commands to realize the interactive operation between the digital human and the web. The digital human client, web server and web end are called through the calling commands. Through the interactive operation between the digital human and the web, a more convenient human-computer interaction method is realized, which enhances the user experience and improves the interactivity.

[0021] In some embodiments, in step S101, the digital human client is installed. After the digital human client is running, a local webSocket server is set up, and the webSocket server processes the connection and messages of the web client; the digital human client connects to the web client via webSocket. After the digital human client is running, the user interacts with the digital human client by voice, and the voice signal in the process of interaction between the user and the digital human client is obtained in real time, wherein the voice signal includes the instruction information issued by the user. The digital human client and the digital human server communicate with each other using a streaming media protocol. After obtaining the voice signal, the digital human client transparently transmits the voice signal to the digital human server through the streaming media protocol, wherein the streaming media protocol at least includes RTMP.

[0022] In some embodiments, in step S102, after the digital human server obtains the voice signal, it recognizes the voice signal and obtains the answer text. First, the digital human server performs text recognition on the voice signal through ASR to obtain the question text, that is, converts the voice signal into a text signal for easy recognition. Secondly, based on the question text, the digital human server calls the knowledge base to configure the answer to the question text and obtains the answer file. The answer text includes an instruction set set in a specific format in a configured form. The instruction set includes interactive instructions and parameters. In the process of voice interaction with the digital human, the digital human's answer words come from the search results of the knowledge base. In the answer text configured in the knowledge base, the instruction set is set in a specific format (such as json format) in a configured form, and the relevant interactive instructions and parameters are placed in the instruction set. After obtaining the answer text, the instruction set module of the digital human server parses the answer file to obtain the calling instruction, which includes the first instruction, the second instruction, the third instruction and the fourth instruction. The web is called based on the calling instruction to realize the interactive operation between the digital human and the web.

[0023] In some embodiments, see Figure 2 , based on the calling instructions, the interactive operation between the digital human and the web is realized, including: S201, calling the web server based on the first instruction, after obtaining the query data through the web server, sending the query data to the Digital Human client, and broadcasting the query data in real time through the Digital Human client; S202, calling the web server based on the second instruction, and controlling the IOT device through the web server to achieve device linkage; S203, calling the Digital Human client based on the third instruction, parsing the third instruction through the Digital Human client, and sending the parsed third instruction to the web end to realize the automated operation of the web page; S204: Based on the fourth instruction, the web server, the digital human client and the web client are called to realize the real-time broadcast of the digital human client and the automatic operation of the web page.

[0024] In some embodiments, in step S201, based on the first instruction, a remote interface query data identifier, an interface address, a request type and parameters are obtained, wherein the remote interface query data identifier is QUERY_INTERFACE, based on the remote interface query data identifier, the interface address, the request type and parameters, an RPC call is made through the Digital Human server to call the web server, after the query data corresponding to the interface address is obtained through the web server, the query data is assembled with the text of the answer text to obtain the interaction data, and the interaction data is sent to the Digital Human client, and the interaction data is broadcasted in real time through the Digital Human client; When the voice signal received by the Digihuman client is: "Please help me query the attendance number of today's project id %s", the answer text is configured through the knowledge base: "OK, the attendance number of today's project id %s is queried for you: [{"cmd": "CUSTOM_EVENT", "operation": "QUERY_INTERFACE", "location": {"url": "http: / / www.xxx.com / project / attendance / work_people", "function": "GET", "project_id=%s", "result": "data.people_number"}}]", where %s is a variable, which is filled in according to the user's question voice recognition, cmd=CUSTOM_EVENT represents the user-defined command, QU ERY_INTERFACE is the remote interface query data identifier, the request type is GAT, the request address is the address of url, result indicates the returned result, and the result is the value of the people_number field in data. The answer file is parsed through the digital human server. When QUERY_INTERFACE exists in the instruction, the digital human server makes an RPC call through the RPC module, that is, calls the web server through RPC, queries the data corresponding to the address url through the web server, that is, obtains the query data corresponding to the interface address, and after obtaining the query data, returns the query data to the digital human server, and splices the query data with the answer file through the digital human server, that is, inserts the query data into the %s variable to obtain the interaction data. The digital human server sends the interaction data to the digital human client, and broadcasts the interaction data in real time through the digital human client to complete the interaction between the digital human client and the user.

[0025] Through configuration, voice interaction with the digital human can be achieved. The digital human server dynamically queries the real-time data of the business system, assembles the data returned by the interface with the text, dynamically broadcasts the data of the business system, and realizes various scenarios such as automated explanation of the web system.

[0026] In some embodiments, in step S202, the web server is called based on the second instruction, and the IOT device is controlled through the web server to achieve device linkage, specifically: based on the second instruction, a remote interface call identifier, an interface address, a request type, and parameters are obtained, wherein the remote interface call identifier is CALL_INTERFACE, and the request type is POST; based on the remote interface call identifier, the interface address, the request type, and the parameters, an RPC call is made through the digital human server to call the web server, and the IOT device is operated through the web server to complete device linkage; When the voice signal received by the Digital Human client is: "Please turn on the emergency rescue mode", the question is configured through the knowledge base, and the answer text is: "Received, the emergency rescue mode is being turned on for you [{"cmd":"CUSTOM_EVENT","operation":"CALL_INTERFACE","location":{"url":"http: / / www.xxx.com / project / emergency / rescue","function":"POST","project_id":658},"result":"data.people_number"}}]" Among them, CALL_INTERFACE is the remote interface call identifier, POST is the request type, the parameter to be transmitted by POST is 658, and the returned result is people_number in data. The Digital Human server parses the answer file. When CALL_INTERFACE exists in the instruction, the Digital Human server performs an RPC call through the RPC module, that is, calls the web server through RPC. According to the request type, the web server operates the IOT device corresponding to the id number 658 to perform device linkage.

[0027] By calling the server of the business system through the digital human server, IOT devices can be operated and equipment linkage can be achieved to achieve a more convenient form of human-computer interaction.

[0028] In some embodiments, in step S203, the digital human client is called based on the third instruction, the third instruction is parsed by the digital human client, and the parsed third instruction is sent to the web end to realize the automatic operation of the web page, including: the digital human server sends the third instruction to the digital human client, and calculates the time interval for sending the instruction, the digital human client parses the third instruction, and sends the parsed third instruction to the web end, wherein the parsed instruction at least includes a page jump mark, a page mode highlight mark and a page address, based on the page jump mark, the page mode highlight mark, the page address and the time interval, the web end jumps to the web page corresponding to the page address, and highlights the web page; When the voice signal received by the Digihuman client is: "Introduce the business management function", the question is configured through the knowledge base, and the answer text is: "Jumping to the business management page for you [{"cmd":"CUSTOM_EVENT","operation":"JUMP_PAGE","location":". / company / company_manage"}, {"cmd":"CUSTOM_EVENT","operation":"HIGH_LIGHT","location":"title_div"}]" Among them, JUMP_PAGE is the page jump mark, CUSTOM_EVENT is the page mode highlight mark, the page address is company / company_manage, title_div is the specific module identifier to be highlighted, and the entire instruction format is a json string in array format. Multiple instruction sets can be configured. The Digihuman server parses the answer file. When JUMP_PAGE and HIGH_LIGHT exist in the instruction, the Digihuman server sends the answer text to the Digihuman client and calculates the time interval for sending the instruction. The time interval for sending the instruction is calculated as follows: according to the position of the instruction in the text, the number of characters in the preceding text is calculated and divided by the current speaking speed of the Digihuman, and the time interval for sending the instruction to the Digihuman client can be obtained. For example, if there are 13 words in front of the instruction and the current speaking speed is 70 words / min, then the time interval for sending the instruction is 13 / 70*60=11 seconds. After receiving the answer text, the Digihuman client parses the answer text and obtains the parsed instructions, namely the jump instruction and the highlight instruction, and sends the parsed instructions to the web end. The web end jumps to the web page corresponding to the page address according to the jump instruction, the highlight instruction and the time interval, and automatically highlights a module on the page.

[0029] In some embodiments, in step S204, the web server, the digital human client and the web terminal are called based on the fourth instruction to realize the real-time broadcast of the digital human client and the automatic operation of the web page. The remote interface query data identifier, interface address, request type, page jump identifier, page mode highlight identifier, page address and parameters are obtained based on the fourth instruction, that is, the remote interface query data identifier, interface address, request type, page jump identifier, page mode highlight identifier, page address and parameters are obtained through the analysis of the answer text by the digital human server. Based on the remote interface query data identifier, interface address, request type, page jump identifier, page mode highlight identifier, page address and parameters, the web server, the digital human client and the web terminal are called respectively to realize the real-time broadcast of the digital human client and the automatic operation of the web page. The web server is called by the digital human server to obtain the query data, and the query data is sent to the digital human client. The digital human client performs language broadcast. At the same time, the web terminal is called by the digital human client, and the web terminal performs page jump and highlight display, so that the real-time broadcast of the digital human and the page jump and height display are carried out simultaneously.

[0030] The instruction set parsing module of the digital human server dynamically parses the instruction information called by RPC (remote procedure call) and calls the remote interface (such as http) in real time. It can not only query the background data of the web server in real time, but also control the remote system and equipment. The digital human client transmits the instruction information through the digital human server. The digital human client can broadcast voice information in real time and call the web end to jump and highlight the page. The digital human interacts with the user in real time and provides personalized services through voice, text and visual feedback to improve user participation and satisfaction. It can also realize emotional communication. The virtual digital human can express emotions through facial expressions, intonation and body language, making users feel more humane communication and establish emotional connection.

[0031] In order to better implement the digital human and web interaction method in the embodiment of the present invention, based on the digital human and web interaction method, correspondingly, Figure 3 As shown, the embodiment of the present invention further provides a digital human interaction system, the digital human interaction system 300 includes: a voice signal acquisition module 301 and an interaction module 302; The voice signal acquisition module 301 is used to acquire the voice signal in the process of interaction between the digital human and the user in real time, wherein the voice signal includes the user's instruction information; The interactive module 302 is used to recognize the voice signal, obtain the answer text, parse the answer text, obtain the calling instruction, call the web based on the calling instruction, and realize the interactive operation between the digital human and the web.

[0032] In summary, the method and system for digital human-web interaction provided by the present invention can obtain voice signals in real time during the interaction between the digital human and the user, wherein the voice signals include the user's instruction information; recognize the voice signals to obtain answer texts, parse the answer texts to obtain calling instructions, and call the web based on the calling instructions to realize the interactive operation between the digital human and the web without the need for mouse operation, thereby realizing a convenient human-computer interaction mode, enhancing the user experience, and improving interactivity.

[0033] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention.

Claims

1. A method for interaction between digital humans and the web, It is characterized in that include: Acquire in real time the voice signal during the interaction between the digital human and the user, wherein the voice signal includes the user's instruction information; The voice signal is recognized to obtain an answer text, the answer text is parsed to obtain a calling instruction, and the web is called based on the calling instruction to realize the interactive operation between the digital human and the web.

2. The method for interaction between a digital human and a web according to claim 1, It is characterized in that The digital human includes a digital human client and a digital human server; Real-time acquisition of voice signals during the interaction between digital humans and users, including: The voice signal in the process of interaction between the user and the Digital Human client is acquired in real time, and the voice signal is transparently transmitted to the Digital Human server through the streaming media protocol, wherein the streaming media protocol at least includes RTMP.

3. The method for interaction between a digital human and a web according to claim 1, It is characterized in that The step of recognizing the voice signal to obtain the answer text includes: The digital human server calls ASR to perform text recognition on the voice signal to obtain the question text; Based on the question text, the digital human server calls the knowledge base to configure the answer to the question text to obtain the answer text.

4. The method for interaction between a digital human and a web according to claim 3, It is characterized in that The answer text includes an instruction set in a configurable form and in a specific format, wherein the instruction set includes interaction instructions and parameters.

5. The method for interaction between a digital human and a web according to claim 4, It is characterized in that The calling instruction includes a first instruction, a second instruction, a third instruction and a fourth instruction, and the web includes a web end and a web service end; calling the web based on the calling instruction to realize the interactive operation between the digital human and the web includes: Based on the first instruction, the web server is called, and after obtaining the query data through the web server, the query data is sent to the digital human client, and the query data is broadcasted in real time through the digital human client; Calling a web server based on the second instruction, and controlling the IoT device through the web server to achieve device linkage; Calling the digital human client based on the third instruction, parsing the third instruction through the digital human client, and sending the parsed third instruction to the web end to realize the automated operation of the web page; Based on the fourth instruction, the web server, the digital human client and the web client are called to realize the real-time broadcast of the digital human client and the automatic operation of the web page.

6. The method for interaction between a digital human and a web according to claim 5, It is characterized in that The calling of the web server based on the first instruction, after obtaining the query data through the web server, sending the query data to the digital human client, and broadcasting the query data in real time through the digital human client, comprises: Obtaining a remote interface query data identifier, an interface address, a request type, and parameters based on the first instruction, wherein the remote interface query data identifier is QUERY_INTERFACE; Based on the remote interface query data identifier, interface address, request type and parameters, the digital human server performs an RPC call to call the web service end, obtains the query data corresponding to the interface address through the web service end, assembles the query data with the text of the answer text, obtains the interaction data, and sends the interaction data to the digital human client; The interactive data is reported in real time through the digital human client.

7. The method for interaction between a digital human and a web according to claim 5, It is characterized in that The calling of the web server based on the second instruction and controlling the IOT device through the web server to achieve device linkage includes: Based on the second instruction, obtaining a remote interface call identifier, an interface address, a request type, and parameters, wherein the remote interface call identifier is CALL_INTERFACE and the request type is POST; Based on the remote interface call identifier, interface address, request type and parameters, an RPC call is made through the digital human server to call the web server, and the IOT device is operated through the web server to complete device linkage.

8. The method for interaction between a digital human and a web according to claim 5, It is characterized in that The calling of the digital human client based on the third instruction, parsing the third instruction by the digital human client, and sending the parsed third instruction to the web end to realize the automatic operation of the web page includes: Sending the third instruction to the digital human client through the digital human server, and calculating the time interval for sending the instructions; The third instruction is parsed by the digital human client, and the parsed third instruction is sent to the web end, wherein the parsed third instruction at least includes a page jump mark, a page mode highlight mark and a page address, and the digital human client is connected to the web end via a websocket; Based on the page jump identifier, the page mode highlight identifier, the page address and the time interval, the web terminal jumps to the web page corresponding to the page address and highlights the web page.

9. The method for interaction between a digital human and a web according to claim 5, It is characterized in that The calling of the web server, the digital human client and the web terminal based on the fourth instruction to realize the real-time broadcast of the digital human client and the automatic operation of the web page includes: Based on the fourth instruction, obtain the remote interface query data identifier, interface address, request type, page jump identifier, page mode highlight identifier, page address and parameters; Based on the remote interface query data identifier, interface address, request type, page jump identifier, page mode highlight identifier, page address and parameters, the web server, digital human client and web terminal are called respectively to realize the real-time broadcast of the digital human client and the automatic operation of the web page.

10. A digital human and web interactive system, It is characterized in that include: A voice signal acquisition module, used to acquire the voice signal in the process of interaction between the digital human and the user in real time, wherein the voice signal includes the user's instruction information; The interactive module is used to recognize the voice signal, obtain the answer text, parse the answer text, obtain the calling instruction, call the web based on the calling instruction, and realize the interactive operation between the digital human and the web.

Images