Interaction method, device, electronic device and storage medium of virtual image in live broadcast room

Abstract

The present application relates to the field of live broadcasting, and provides an interactive method, device, equipment and storage medium for a virtual image in a live broadcast room. The present application can realize the interaction between a user and a three-dimensional virtual image in a live broadcast room based on voice input, and improve the convenience of interaction between the user in the live broadcast room and the three-dimensional virtual image in the live broadcast room. The method includes: receiving the interactive voice input by the target user to the three-dimensional virtual image in the live broadcast room, and then obtaining a target action script matching the interactive voice in a script database according to the interactive voice, and then obtaining a material resource corresponding to the target action script in a material resource library according to the target action script, and finally playing the response action of the three-dimensional virtual image in the live broadcast room according to the target action script and the material resource.

Description

Technical Field

[0001] The present application relates to the technical field of network live broadcasting, and in particular to an interactive method, device, electronic device and computer-readable storage medium for a virtual image in a live broadcast room. Background Art

[0002] With the development of online live broadcast technology, there are more and more ways for anchors and audiences to interact in the live broadcast room, and three-dimensional virtual images have also been applied to live broadcast scenes.

[0003] With current technology, live broadcast rooms can display three-dimensional virtual images in the form of people, animals, etc. Under the interactive operations of the anchor and the audience, the three-dimensional virtual images can respond. The interactive solutions provided by current technology require users to perform relatively cumbersome interactive operations, and there is a technical problem of poor convenience in interacting with the three-dimensional virtual images in the live broadcast room. Summary of the invention

[0004] Based on this, it is necessary to provide an interactive method, device, electronic device and computer-readable storage medium for a virtual image in a live broadcast room in response to the above-mentioned technical problems.

[0005] In a first aspect, the present application provides a method for interacting with a virtual image in a live broadcast room. The method comprises:

[0006] Receiving interactive voice input from a target user to a three-dimensional virtual image in a live broadcast room;

[0007] According to the interactive speech, obtaining a target action script matching the interactive speech in a script database;

[0008] According to the target action script, obtaining the material resources corresponding to the target action script in the material resource library;

[0009] According to the target action script and material resources, the response action of the three-dimensional virtual image is played.

[0010] In a second aspect, the present application provides an interactive device for a virtual image in a live broadcast room. The device comprises:

[0011] A voice receiving module, used to receive the interactive voice input by the target user to the three-dimensional virtual image in the live broadcast room;

[0012] A script acquisition module, used for acquiring a target action script matched by the interactive speech in a script database according to the interactive speech;

[0013] A resource acquisition module, used to acquire the material resources corresponding to the target action script in the material resource library according to the target action script;

[0014] The action playing module is used to play the response action of the three-dimensional virtual image according to the target action script and material resources.

[0015] In a third aspect, the present application provides an electronic device. The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0016] Receive interactive voice input by the target user to the three-dimensional virtual image in the live broadcast room; based on the interactive voice, obtain the target action script matching the interactive voice in the script database; based on the target action script, obtain the material resources corresponding to the target action script in the material resource library; based on the target action script and the material resources, play the response action of the three-dimensional virtual image.

[0017] In a fourth aspect, the present application provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0018] Receive interactive voice input by the target user to the three-dimensional virtual image in the live broadcast room; based on the interactive voice, obtain the target action script matching the interactive voice in the script database; based on the target action script, obtain the material resources corresponding to the target action script in the material resource library; based on the target action script and the material resources, play the response action of the three-dimensional virtual image.

[0019] The interactive method, device, equipment and storage medium of the above-mentioned live broadcast room virtual image receive the interactive voice input by the target user to the three-dimensional virtual image in the live broadcast room, and then obtain the target action script matching the interactive voice in the script database according to the interactive voice, and then obtain the material resources corresponding to the target action script in the material resource library according to the target action script, and finally play the response action of the three-dimensional virtual image in the live broadcast room according to the target action script and the material resources. This scheme can realize the interaction between the user and the three-dimensional virtual image in the live broadcast room based on voice input, improve the convenience of interaction between the user in the live broadcast room and the three-dimensional virtual image in the live broadcast room, and can be applied to the interactive processing between the user and the three-dimensional virtual image in the metaverse live broadcast scene, making the interaction between the user and the three-dimensional virtual image in the metaverse live broadcast more convenient, and also enriching the interaction mode between the user and the three-dimensional virtual image in the metaverse live broadcast. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is an application scenario diagram of the interactive method of the live broadcast room virtual image in the embodiment of the present application;

[0021] Figure 2 A schematic diagram of the flow of the interactive method of the live broadcast room virtual image in the embodiment of the present application;

[0022] Figure 3 A flowchart of an interactive method for a live broadcast room virtual image in a specific example of this application;

[0023] Figure 4 A flowchart of the steps of identifying a target user in an embodiment of the present application;

[0024] Figure 5 This is a structural block diagram of an interactive device for a live broadcast room virtual image in an embodiment of the present application;

[0025] Figure 6 Schematic diagram of the internal structure of an electronic device in an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0027] The interactive method of the live broadcast room virtual image provided in the embodiment of the present application can be applied to Figure 1 In the application scenario shown, the application scenario may include a terminal and a server, wherein the terminal may specifically include a host terminal and multiple audience terminals (such as audience terminal 1, audience terminal 2, etc.) of a live broadcast room, and the host terminal and multiple audience terminals may communicate with the server through the Internet respectively, and the server provides live broadcast related services for the host terminal and multiple audience terminals of the live broadcast room. Among them, the terminal may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers and portable wearable devices, and the portable wearable devices may be head-mounted devices, etc.; the server may be implemented by an independent server or a server cluster composed of multiple servers. Among them, the application scenario may be a metaverse live broadcast scene, and the server may create an independent metaverse scene for the host and audience of the live broadcast room, build a virtual space parallel to the real world, and use scientific and technological means to link and create a virtual world that maps and interacts with the real world, and various three-dimensional virtual images such as people and animals may be displayed in the virtual space, and the host and the user may interact freely with the three-dimensional virtual image in the virtual space. Specifically, users in the live broadcast room can complete human-computer interaction through terminals such as mobile phones and AR glasses, and realize basic control operations, such as gesture movement switching, choosing to exit, etc. The terminal can receive and display data sent by the server such as live video streams, three-dimensional virtual images, etc., display information for the metaverse scene, and provide metaverse information to users.

[0028] Among them, for the metaverse live broadcast scene, the anchor or the performing guest can start broadcasting on the green screen studio. The stage of the green screen studio will be placed with photographic lights, microphones, and cameras. The live video shot by the camera placed on the stage can push the video stream to the on-site matte server in real time. The matte server can separate the characters from the green screen background and cut out the characters. In the existing technology, the human portrait can be cut out by software and hardware. The processed character video stream can be used for VR panoramic synthesis. The characters need to be positioned at the appropriate coordinate position of the pre-prepared scene background. This scene can be a two-dimensional background or a three-dimensional background scene. When the audience plays through the VR device, the video in the VR live broadcast is mainly a panoramic video, which can include 180-degree and 360-degree videos. When the corresponding degree video is received, the video live stream will be rendered on the inner surface of a transparent hemisphere (180 degrees) or a sphere (360 degrees) or a 1 / 4 sphere, and a virtual camera will be placed at a suitable position for screen display. In the metaverse live broadcast scene, a virtual image of the live broadcast room can also be displayed for users to interact with, achieving a virtual and real combination effect.

[0029] In the metaverse live broadcast scenario including but not limited to, based on the interactive method of the virtual image in the live broadcast room provided in this application, the terminal can display the three-dimensional virtual image in the live broadcast room, and the user can input interactive voice on the terminal to make the three-dimensional virtual image respond, thereby realizing convenient interaction with the three-dimensional virtual image. It can be applied to the interactive processing between the user and the three-dimensional virtual image in the metaverse live broadcast scenario, making the interaction between the user and the three-dimensional virtual image in the metaverse live broadcast more convenient, and enriching the interaction method between the user and the three-dimensional virtual image in the metaverse live broadcast.

[0030] The following is based on Figure 1 The application environment shown, combined with various embodiments and corresponding drawings, illustrates the interaction method of the live broadcast room virtual image of the present application.

[0031] In one embodiment, Figure 2 As shown, a method for interacting with a live broadcast room virtual image is provided, and the method can be performed as follows Figure 1 The terminal execution shown in includes the following steps:

[0032] Step S201, receiving interactive voice input by the target user to the three-dimensional virtual image in the live broadcast room.

[0033] In this step, the target user refers to one or more specific users in the live broadcast room, which can be anchors or viewers. Specifically, anchors or viewers with preset permissions can be determined as target users. Exemplarily, preset permissions can include identity permissions, level permissions, etc. In actual applications, for identity permissions, the anchors in the anchor room can be determined as target users, and for level permissions, viewers who reach the preset audience level in the live broadcast room can be determined as target users. The setting and improvement of audience levels can be set according to the live broadcast business rules, and are only cited here as examples for explaining target users. Furthermore, if the local user is determined as a target user, the terminal can receive interactive voice input by the target user to the three-dimensional virtual image in the live broadcast room. The three-dimensional virtual image can specifically be a three-dimensional virtual image of various forms such as people and animals displayed in the live broadcast room. The interactive voice refers to the voice used by the user to interact with the three-dimensional virtual image. It can be in the form of a sentence or in the form of words. Based on the interactive voice, the three-dimensional virtual image may make corresponding response actions. Therefore, the interactive voice usually includes the meaning of requiring or requesting the three-dimensional virtual image to make certain response actions. For example, "Please sit down" includes the meaning of requesting the three-dimensional virtual image to sit down, and "Can you dance for everyone?" includes the meaning of requesting the three-dimensional virtual image to dance. Therefore, "Please sit down" and "Can you dance for everyone?" can be input into the terminal by the user as interactive voice.

[0034] Step S202: Obtain a target action script that matches the interactive speech in a script database according to the interactive speech.

[0035] Step S203: acquiring material resources corresponding to the target action script in the material resource library according to the target action script.

[0036] For steps S202 and S203, the script database and the resource library can be configured first. Among them, for the three-dimensional virtual image, the three-dimensional virtual images of various forms such as people and animals can be designed by designers according to certain specifications. The standardized design can maintain the reusability of the action script. When these three-dimensional virtual images make certain actions, they will be accompanied by the movement of each bone joint of the entire three-dimensional virtual image. The bone joints will mark the fineness of their movement. Generally, the higher the fineness, the more fine the number of bone joints and the number of faces will be designed. In addition, when the three-dimensional virtual image makes certain actions, it will also be accompanied by some sound, picture and other material resources. For sound material resources, such as background music, expression music, etc., for picture material resources, such as clothing and props of the three-dimensional virtual image. These material resources need to be pre-made and recorded to form a resource library. The various material resources contained in the resource library can be associated with the action script respectively, so as to achieve the purpose of obtaining an action script from which the corresponding material resources can be obtained.

[0037] Specifically, for action scripts, different three-dimensional virtual images will have some specific actions, and when performing specific actions, they need to be matched with some specific sounds, pictures and other material resources to perform. These specific actions and material resources can be controlled and bound through code scripts. For example, the three-dimensional virtual image has actions such as walking, standing, raising hands, singing, and dancing. It is necessary to write corresponding code scripts according to the skeletal joints and required material resources of the model of the three-dimensional virtual image in the model production of the three-dimensional virtual image, and save them to obtain action scripts for actions such as walking, standing, raising hands, singing, and dancing, and set the identification of each action script and its description information. The description information can specifically include information such as the trigger instructions corresponding to the action script. In actual applications, if the models of other three-dimensional virtual images are also designed according to the same specifications, their corresponding action scripts can be reused to a certain extent to improve the production efficiency of action scripts. Therefore, many action scripts can be obtained in the production of different three-dimensional virtual images. As mentioned earlier, in order to facilitate the subsequent use, reuse and management of these action scripts, a unique identifier can be set for each action script, and the name of the action script can be defined (which can be used to describe the action meaning of the action script, such as walking, standing, raising hands, singing, dancing, etc.). In addition, each action script can be set with its corresponding three-dimensional virtual image, corresponding live broadcast scene, corresponding image attributes, corresponding time and other labels for further classification and storage to form a script database.

[0038] Based on this, in step S202, after the terminal obtains the interactive voice input by the target user, the interactive voice can be recognized as the corresponding text using the existing voice recognition model. The text is called interactive voice text. For example, the voice form of "please sing" can be recognized as the text form of "please sing" to obtain the interactive voice text. Then the terminal can send the interactive voice text to the server, and the server matches it in the script database according to the interactive voice text. The matching can be performed by fuzzy matching or exact matching. The action script that matches the interactive voice text in the script database is obtained as the target action script. As an example, for the interactive voice text "please sing", the server can match the interactive voice text with the names of each action script in the script database, match the action script named singing as the target action script and return it to the terminal. In a specific implementation, a matching threshold can be pre-defined. When the matching reaches the matching threshold, the server can consider the matching successful and send the matching target action text to the terminal. If the matching does not reach the matching threshold, the server can consider the matching failed. At this time, the server does not send the action text but prompts the target user through the terminal that the matching failed. In a specific implementation, further, during the early cold start process of the action script, the number of deposited action scripts may be relatively small. In order to improve the efficiency and effectiveness of search matching, the target user can be prompted through the terminal with the name or instruction corresponding to the currently available action script, so that the target user can input the corresponding interactive voice for interaction. When more action scripts are deposited in the later stage, the prompting link can be omitted.

[0039] For step S202, in some embodiments, step S202 may further include: determining the live broadcast scene in which the three-dimensional virtual image is located and the image attributes of the three-dimensional virtual image; according to the live broadcast scene, image attributes and interactive voice, obtaining the target action script matching the live broadcast scene, image attributes and interactive voice in the script database. In this embodiment, the three-dimensional virtual image displayed in the live broadcast room can be in a variety of live broadcast scenes, and the live broadcast scene refers to the virtual scene arranged in the live broadcast room, which can be an outdoor scene or an indoor scene, such as a booth, a stage, etc. The three-dimensional virtual image can have its own image attributes, such as personality characteristics, singing level, etc. In this embodiment, according to the live broadcast scene, image attributes and interactive voice in which the three-dimensional virtual image is located, the target action script matching it in the script database is obtained, so that the response action of the three-dimensional virtual image based on the target action script can be more vivid and more adapted to the current scene and image. Specifically, labels in terms of corresponding live broadcast scenes, image attributes, etc. can be set for each action specimen in the script database. The server can then match the corresponding target action script in the script database according to the live broadcast scene, image attributes and interactive voice of the three-dimensional virtual image. In this way, for the same interactive voice, in different live broadcast scenes and with different image attributes, the three-dimensional virtual image can make more vivid and adaptive response actions, enriching the interactive performance of the three-dimensional virtual image.

[0040] In step S203, the terminal can receive the target action script sent by the server, and then download the material resources corresponding to the target action script from the material resource library according to the target action script. Specifically, the terminal can download the material resources corresponding to the identifier of the target action script from the material resource library according to the identifier of the target action script.

[0041] For step S203, in one embodiment, step S203 may further include: obtaining the identifier of the target action script, obtaining the identifier of the three-dimensional virtual image; if the identifier of the target action script matches the identifier of the three-dimensional virtual image, then obtaining the material resources corresponding to the target action script in the material resource library. Specifically, the target action script issued by the server may not necessarily be adapted to the three-dimensional virtual image currently displayed in the live broadcast room, such as the bone structure is not compatible, etc. To prevent control errors, after receiving the target action script, the terminal matches the identifier of the target action script with the identifier of the three-dimensional virtual image. As mentioned above, when the action script is produced, the three-dimensional virtual image adapted to it will be recorded accordingly. The three-dimensional virtual image that the target action script can adapt to can be recorded by setting the identifier correspondence relationship. Therefore, it can be determined whether the identifier of the target action script matches the identifier of the three-dimensional virtual image based on the identifier correspondence relationship. If so, the terminal can download the corresponding material resources, such as music, pictures and other material resources, from the material resource library according to the identifier of the target action script. Otherwise, the terminal can not process the three-dimensional virtual image, and the terminal can feedback the information of the mismatched identifier to the server to form an alarm and subsequent problem repair.

[0042] Step S204, playing the response action of the three-dimensional virtual image according to the target action script and the material resources.

[0043] In step S204, the terminal can bind the target action script to the three-dimensional virtual image, so that the three-dimensional virtual image can present corresponding actions according to the target action script and the material resources. For example, the three-dimensional virtual image can respond to the action of walking according to the issued target action script. The target action script can specify which bone of the three-dimensional virtual image moves, in which direction to move, where the coordinates of the movement are, how long the movement takes, and so on.

[0044] The interactive method of the live broadcast room virtual image of this embodiment receives the interactive voice input by the target user to the three-dimensional virtual image in the live broadcast room, and then obtains the target action script matching the interactive voice in the script database according to the interactive voice, and then obtains the material resources corresponding to the target action script in the material resource library according to the target action script, and finally plays the response action of the three-dimensional virtual image in the live broadcast room according to the target action script and the material resources. This scheme can realize the interaction between the user and the three-dimensional virtual image in the live broadcast room based on voice input, improve the convenience of interaction between the user in the live broadcast room and the three-dimensional virtual image in the live broadcast room, and can be applied to the interactive processing between the user and the three-dimensional virtual image in the metaverse live broadcast scene, making the interaction between the user and the three-dimensional virtual image in the metaverse live broadcast more convenient, and also enriching the interaction mode between the user and the three-dimensional virtual image in the metaverse live broadcast.

[0045] As a specific example, the interactive method of the live broadcast room virtual image of the present application may specifically include the following: Figure 3 , wherein the production of the three-dimensional virtual image and material resources, the production of action scripts, the management of action scripts and resource materials, the voice input, the matching of action scripts and material resources, and the action of the three-dimensional virtual image can refer to the corresponding contents disclosed in the above embodiments. Then, in the action synchronization, if the picture of the response action of the three-dimensional virtual image needs to be synchronized with the live broadcast picture in real time, a push-stream synchronization solution can be adopted. For example, the host is conducting a lottery activity through the three-dimensional virtual image, and the three-dimensional virtual image is communicating with the host or the audience. At this time, the response action of the three-dimensional virtual image needs to be synchronized with the audio and video of the live broadcast in a relatively real-time manner. The picture or audio containing the response action of the three-dimensional virtual image can be mixed into the original live broadcast stream and pushed to each audience terminal, so that the audience can see that the action of the three-dimensional virtual image is synchronized with the audio and video in real time at the audience terminal; if the response action of the three-dimensional virtual image is not required to be synchronized with the live audio and video picture in real time, the server can send the target action script at the same time, and each terminal can play the response action separately, that is, when each terminal receives the target action script, it plays the response action of the three-dimensional virtual image in combination with the material resources. In the result reporting, when the playback is successful, the terminal can report to the server the information that the three-dimensional virtual image plays the target action script successfully. If the playback fails, it reports the information that the three-dimensional virtual image plays the target action script unsuccessfully, so as to facilitate the formation of an alarm and subsequent problem repair.

[0046] In some embodiments, Figure 4 As shown, before step S201, i.e. receiving the interactive voice inputted by the target user to the three-dimensional virtual image in the live broadcast room, the following steps are also included:

[0047] Step S401: If the local user has preset authority, a preset test text for interactive voice test is displayed.

[0048] Step S402: Acquire the test voice input by the local user.

[0049] Step S403, identifying the test speech text corresponding to the test speech.

[0050] Step S404: If the test voice text matches the preset test text, the local user is set as the target user.

[0051] In this embodiment, in addition to the preset permissions as described in the above embodiment, the local user needs to be identified as a target user through voice input verification. In this way, it is possible to test whether the user's terminal device has good voice input performance and avoid wasting search and matching resources on the server side. Specifically, in step S401, if the local user has preset permissions, such as the local user is the anchor of the live broadcast room, is a viewer who has reached a certain audience level in the live broadcast room, etc., then the terminal can display a preset test text for the interactive voice test for the local user alone. The preset test text can be a pre-set sentence related to the interaction with the three-dimensional virtual image, such as "Please dance for everyone", etc. Then the terminal can prompt the local user to read the preset test text and input the corresponding voice, which is called the test voice. Then in step S402, the terminal obtains the test voice input by the local user, and in step S403, the text corresponding to the test voice can be recognized through the voice recognition model. The corresponding text is called the test voice text. Then in step S404, the terminal can match the test voice text with the preset test text. When the test voice text is consistent with the preset test text, the terminal can determine that the test voice text matches the preset test text, and at this time, the local user is set as the target user.

[0052] In one embodiment, displaying the preset test text for interactive voice testing in step S401 may include: obtaining a preset test text for interactive voice testing, the preset test text comprising a plurality of test text segments in sequence; displaying the plurality of test text segments and the sorting information of the plurality of test text segments in the preset test text in different viewing perspectives of the local user on the live broadcast room, so that the local user can read and input the corresponding test voice in sequence.

[0053] In this embodiment, the preset test text obtained by the terminal includes multiple test text segments in sequence. The multiple test text segments can be pre-segmented from the preset test text. For example, "Please dance for everyone" can be pre-segmented into two test text segments, "Please dance for everyone" and "Dance". The two test text segments also have corresponding sorting information, that is, their respective sorting information in the preset test text. For example, the sorting information of "Please dance for everyone" can be set to 1 to represent the beginning, and the sorting information of "Dance" can be set to N to represent the end, etc. Therefore, when the terminal displays the multiple test text segments, it displays each test text segment together with its corresponding sorting information, such as displaying "please give it to everyone" together with 1, and displaying "dance" together with N, so as to indicate the sorting information corresponding to the test text segment. Specifically, in scenarios such as VR live broadcast and metaverse live broadcast, the user can have multiple viewing perspectives of the live broadcast room on the terminal, and the user can change the viewing perspective of the live broadcast room by rotating the mobile phone or VR glasses. For example, by wearing VR glasses, the viewing perspective can be changed from the viewing perspective of the stage in front of the live broadcast room to the viewing perspective of the left side of the stage in the live broadcast room, etc. Based on this, the terminal can display the multiple test text segments and their respective sorting information in different viewing perspectives of the local user on the live broadcast room, such as displaying "please give it to everyone" and its sorting information in the viewing perspective in front of the stage, and displaying "dance" and its sorting information in the viewing perspective on the left side of the stage, etc., prompting the local user to find and read each test text segment in sequence by changing the viewing perspective and input the corresponding test voice. Through the scheme of this embodiment, it is possible to combine the interactive verification methods in scenarios such as VR live broadcast and metaverse live broadcast, so that users of terminals with certain viewing perspective switching conditions and performance can be identified as target users. It can provide the ability to identify target users for various interactive modes related to interaction with three-dimensional virtual images in scenarios such as VR live broadcast and metaverse live broadcast.

[0054] Based on the solution of the above embodiment, further, in one embodiment, after the method of the present application sets the local user as the target user, it may also include:

[0055] Obtain a preset interactive text for a three-dimensional virtual image sent by a server, wherein the preset interactive text includes a plurality of interactive text segments in sequence; display the plurality of interactive text segments and the sorting information of each of the plurality of interactive text segments in the preset interactive text in different viewing angles of a target user for a live broadcast room, so that the target user can read and input corresponding interactive voices in sequence.

[0056] This embodiment is a solution for the stage of actual interaction with the three-dimensional virtual image after the local user is determined as the target user based on the solution of the above embodiment. Some interactive texts for the three-dimensional virtual image can be predefined in the server, which are called preset interactive texts. For example, corresponding interactive texts can be set for specific festivals such as the Spring Festival. The interactive text can enable the three-dimensional virtual image to perform specific response actions to express the celebration of the Spring Festival. The preset interactive text can be pre-divided into multiple interactive text segments in sequence, and there can be multiple target users in the live broadcast room. The server can send the preset interactive text containing multiple interactive text segments in sequence to the terminal of each target user in the live broadcast room. After receiving the preset interactive text, the terminal obtains multiple interactive text segments and their respective sorting information in the preset interactive text, and then displays the multiple interactive text segments and their respective sorting information in the target user's different viewing perspectives of the live broadcast room, so that the target user can read and input the corresponding interactive voice in sequence.

[0057] Based on this, further, in step S202 of the method of the present application, obtaining a target action script matching the interactive speech in a script database according to the interactive speech specifically includes:

[0058] Identify the interactive voice text corresponding to the interactive voice, and send the interactive voice text to the server; receive the target action script returned by the server.

[0059] Among them, after receiving the interactive voice input by the target user, the terminal recognizes the interactive voice text corresponding to the interactive voice, and then sends the interactive voice text to the server. The server can match the interactive voice text with the preset interactive text. If the interactive voice text matches the preset interactive text, the target action script corresponding to the preset interactive text is obtained from the script database and returned to the terminal, so that the terminal receives the target action script and plays the response action of the three-dimensional virtual image. In actual scenarios, there are multiple target users in the live broadcast room, so the server may receive multiple interactive voice texts. When the server successfully matches the first interactive voice text, it can stop matching the interactive voice texts from other terminals with the preset interactive text, and instruct the terminals of each target user to cancel the display of the preset interactive text, and can send the target action script to each terminal to play the response action of the three-dimensional virtual image. The solution of this embodiment can enrich the interactive modes related to the interaction with three-dimensional virtual images in scenarios such as VR live broadcast and metaverse live broadcast. Specifically, it can allow multiple target users in the live broadcast room to quickly find and read the interactive text segments distributed in multiple viewing angles in the live broadcast room to form a complete interactive voice, so that when the corresponding interactive voice text successfully matches the preset interactive text for the first time, the three-dimensional virtual image can make a specific response action. The three-dimensional virtual image can also further broadcast the user nickname and other identification information of the target user who successfully matches it through the response action, thereby achieving the effect of enriching the interactive mode.

[0060] In some other embodiments, displaying the preset test text for the interactive voice test in step S401 may include: obtaining a preset test text for the interactive voice test, the preset test text containing preset test keywords; displaying the preset test text and highlighting the preset test keywords contained in the preset test text.

[0061] In this embodiment, the preset test text obtained by the terminal includes a preset test keyword. For example, "dance" in the preset test text "please dance for everyone" can be set as the preset test keyword. The terminal can then display the preset test text in the live broadcast room for the local user and highlight the preset test keyword contained therein, and can also prompt the user to read the preset test keyword at a relatively high volume during the process of reading the preset test text. Here, relative refers to the volume relative to the volume when reading other words in the preset test text, such as reading "please dance for everyone" with a volume X, and reading "dance" with a volume Y greater than the volume X.

[0062] Thus, the terminal can receive and obtain the test voice input by the local user only for the preset test text, and recognize the test voice text corresponding to the test voice. After recognizing the test voice text corresponding to the test voice, it also includes: obtaining the test voice text segment corresponding to the high-volume voice segment in the test voice in the test voice text. Among them, the high-volume voice segment in the test voice refers to a voice segment with a relatively high volume when the local user reads and inputs the test voice. The meaning of the relatively high volume can refer to the content described above. Specifically, the high-volume voice segment in the test voice can be extracted according to the volume information of the test voice, and then a text segment corresponding to the high-volume voice segment can be obtained in the recognized test voice text. This text segment is recorded as the test voice text segment. For example, if the local user reads "Dancing" in the process of reading "Please dance for everyone" with a relatively high volume, then "Dancing" is the test voice text segment.

[0063] Based on this, if the test voice text matches the preset test text in step S404 of the method of the present application, the local user is set as the target user, specifically including: if the test voice text matches the preset test text, and the test voice text segment matches the preset test keyword, the local user is set as the target user. In this embodiment, the test voice text needs to match the preset test text, and the test voice text segment needs to match the preset test keyword before the local user is identified as the target user, so that the local user with clear voice input and clear instruction expression can be screened out more accurately as the target user, avoiding waste of resources in the search and matching of the server.

[0064] Based on the solution of the above embodiment, further, in one embodiment, step S202 in the method of the present application is to obtain a target action script matching the interactive speech in a script database according to the interactive speech, which specifically includes:

[0065] Identify the interactive speech text corresponding to the interactive speech, obtain the interactive speech text segment corresponding to the high-volume speech segment in the interactive speech; send the interactive speech text segment to the server; and receive the target action script returned by the server.

[0066] In this embodiment, the target user can autonomously input interactive voice through the terminal, such as the target user autonomously inputs the interactive voice with the content of "Can you dance for us?" In the process of interactive voice input, the target user can read "dance" at a relatively high volume to complete the input of the interactive voice. Then, the terminal can identify the interactive voice text corresponding to the interactive voice, and obtain the text form of "Can you dance for us?" The terminal can also obtain the high-volume voice segment according to the volume information of the interactive voice, and then obtain the text segment "dance" corresponding to the high-volume voice segment in the interactive voice text, which is recorded as the interactive voice text segment. Then, the terminal can send the interactive voice text segment to the server, and the server can match the interactive voice text segment in the script database. Specifically, each action script in the script database can correspond to a preset keyword, and the preset keyword can be the name of the action script, etc. The server can match the interactive voice text segment with these preset keywords. For example, when the preset keyword "dance" is matched, the server can determine that the interactive voice text segment matches the preset keyword. In this case, the server can obtain the action script corresponding to the preset keyword from the script database as the target action script and return it to the terminal, and the terminal receives the target action script returned by the server. The solution of this embodiment allows users to send interactive voice to the three-dimensional virtual image in a relatively natural and plain rather than imperative sentence to make it respond, so that users in the live broadcast room can behave naturally even during the process of voice interaction with the three-dimensional virtual image, thereby improving the interactive experience.

[0067] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0068] Based on the same inventive concept, the embodiment of the present application also provides a device for interacting with a live broadcast virtual image in a live broadcast room for implementing the interactive method for the live broadcast virtual image involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the embodiments of the interactive device for one or more live broadcast virtual images provided below can be referred to the limitations of the interactive method for the live broadcast virtual image in the above text, and will not be repeated here.

[0069] In one embodiment, Figure 5 As shown, a device for interacting with a virtual image in a live broadcast room is provided, and the device 500 may include:

[0070] The voice receiving module 501 is used to receive the interactive voice input by the target user to the three-dimensional virtual image in the live broadcast room;

[0071] A script acquisition module 502, configured to acquire a target action script matching the interactive speech in a script database according to the interactive speech;

[0072] The resource acquisition module 503 is used to acquire the material resources corresponding to the target action script in the material resource library according to the target action script;

[0073] The action playing module 504 is used to play the response action of the three-dimensional virtual image according to the target action script and material resources.

[0074] In one embodiment, the device 500 may also include: a user identification module, which is used to display a preset test text for interactive voice testing if the local user has preset permissions; obtain the test voice input by the local user; identify the test voice text corresponding to the test voice; if the test voice text matches the preset test text, set the local user as the target user.

[0075] In one embodiment, a user identification module is used to obtain a preset test text for an interactive voice test; the preset test text includes a plurality of test text segments in sequence; the plurality of test text segments and the sorting information of each of the plurality of test text segments in the preset test text are displayed in different viewing angles of the local user on the live broadcast room, so that the local user can read and input the test voice in sequence.

[0076] In one embodiment, the voice receiving module 501 is also used to obtain a preset interactive text for the three-dimensional virtual image sent by the server; the preset interactive text includes multiple interactive text segments in sequence; the multiple interactive text segments and the sorting information of the multiple interactive text segments in the preset interactive text are displayed in the different viewing angles of the target user on the live broadcast room, so that the target user can read and input the interactive voice in sequence; the script acquisition module 502 is used to identify the interactive voice text corresponding to the interactive voice, and send the interactive voice text to the server; receive the target action script returned by the server; wherein the target action script is the target action script corresponding to the preset interactive text obtained and returned by the server from the script database when the interactive voice text matches the preset interactive text.

[0077] In one embodiment, a user identification module is used to obtain a preset test text for an interactive voice test; the preset test text includes a preset test keyword; the preset test text is displayed and the preset test keyword included in the preset test text is highlighted; a test voice text segment in the test voice text corresponding to a high-volume voice segment in the test voice is obtained; if the test voice text matches the preset test text, and the test voice text segment matches the preset test keyword, the local user is set as a target user.

[0078] In one embodiment, the script acquisition module 502 is used to identify the interactive speech text corresponding to the interactive speech, obtain the interactive speech text segment corresponding to the high-volume speech segment in the interactive speech; send the interactive speech text segment to the server; receive the target action script returned by the server; the target action script is the target action script corresponding to the preset keyword obtained and returned by the server from the script database when the interactive speech text segment matches the preset keyword.

[0079] In one embodiment, the resource acquisition module 503 is used to obtain the identifier of the target action script and the identifier of the three-dimensional virtual image; if the identifier of the target action script matches the identifier of the three-dimensional virtual image, the material resource corresponding to the target action script in the material resource library is obtained.

[0080] In one embodiment, the script acquisition module 502 is used to determine the live broadcast scene in which the three-dimensional virtual image is located and the image attributes of the three-dimensional virtual image; based on the live broadcast scene, image attributes and interactive voice, obtain the target action script that matches the live broadcast scene, image attributes and interactive voice in the script database.

[0081] Each module in the interactive device of the above-mentioned live broadcast virtual image can be implemented in whole or in part by software, hardware and their combination. The above-mentioned modules can be embedded in or independent of the processor in the electronic device in the form of hardware, or can be stored in the memory of the electronic device in the form of software, so that the processor can call and execute the corresponding operations of the above modules.

[0082] In one embodiment, an electronic device is provided. The electronic device may be a terminal, and its internal structure diagram may be as follows: Figure 6As shown. The electronic device includes a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the electronic device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a method for interacting with a virtual image in a live broadcast room is implemented. The display screen of the electronic device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic device can be a touch layer covered on the display screen, or a button, trackball or touchpad set on the housing of the electronic device, or an external keyboard, touchpad or mouse.

[0083] Those skilled in the art will understand that Figure 6 The structure shown in the figure is merely a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the electronic device to which the scheme of the present application is applied. The specific electronic device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0084] In one embodiment, an electronic device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0085] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0086] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0087] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0088] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0089] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for interacting with a virtual image in a live broadcast room, characterized in that: The method comprises: Acquire a preset test text for interactive voice testing, display the sorting information of each test text segment in the preset test text in different viewing angles of the local user on the live broadcast room, determine the local user whose input test voice matches the preset test text as the target user, and receive the interactive voice input by the target user to the three-dimensional virtual image in the live broadcast room; According to the interactive speech, obtaining a target action script matching the interactive speech in a script database; According to the target action script, obtaining the material resources corresponding to the target action script in the material resource library; According to the target action script and material resources, the response action of the three-dimensional virtual image is played.

2. The method according to claim 1, characterized in that Before receiving the interactive voice input by the target user to the three-dimensional virtual image in the live broadcast room, the method further includes: If the local user has the preset permission, the preset test text for interactive voice test is displayed; Acquire the test voice input by the local user; Identify a test voice text corresponding to the test voice; If the test voice text matches the preset test text, the local user is set as the target user.

3. The method according to claim 2, characterized in that The displaying the sorting information of each test text segment in the preset test text in different viewing angles of the local user on the live broadcast room includes: The preset test text includes a plurality of test text segments in sequence; The multiple test text segments and the sorting information of each of the multiple test text segments in the preset test text are displayed in different viewing angles of the local user on the live broadcast room, so that the local user can read and input the test voice in sequence.

4. The method according to claim 3, characterized in that After setting the local user as the target user, the method further includes: Acquire a preset interactive text for the three-dimensional virtual image sent by a server; the preset interactive text sequentially includes a plurality of interactive text segments; Displaying the plurality of interactive text segments and the sorting information of each of the plurality of interactive text segments in the preset interactive text in different viewing angles of the target user on the live broadcast room, so that the target user can read and input the interactive voice in sequence; The step of obtaining a target action script matched by the interactive voice in a script database according to the interactive voice comprises: Identify the interactive voice text corresponding to the interactive voice, and send the interactive voice text to the server; Receive the target action script returned by the server; wherein the target action script is the target action script corresponding to the preset interactive text obtained and returned from the script database by the server when the interactive voice text matches the preset interactive text.

5. The method according to claim 2, characterized in that: The display of the preset test text for the interactive voice test includes: Acquire a preset test text for interactive speech testing; the preset test text includes preset test keywords; Displaying the preset test text and highlighting the preset test keywords contained in the preset test text; After the test voice text corresponding to the test voice is identified, the method further includes: Acquire a test speech text segment in the test speech text corresponding to a high-volume speech segment in the test speech; If the test voice text matches the preset test text, setting the local user as a target user includes: If the test voice text matches the preset test text, and the test voice text segment matches the preset test keyword, the local user is set as a target user.

6. The method according to claim 5, characterized in that The step of obtaining a target action script matched by the interactive voice in a script database according to the interactive voice comprises: Identify the interactive speech text corresponding to the interactive speech, and obtain the interactive speech text segment corresponding to the high-volume speech segment in the interactive speech; Sending the interactive voice text segment to a server; Receive the target action script returned by the server; the target action script is the target action script corresponding to the preset keyword obtained and returned by the server from the script database when the interactive voice text segment matches the preset keyword.

7. The method according to any one of claims 1 to 6, characterized in that: The step of acquiring the material resource corresponding to the target action script in the material resource library according to the target action script includes: Obtaining an identifier of the target action script and an identifier of the three-dimensional virtual image; If the identifier of the target action script matches the identifier of the three-dimensional virtual image, the material resource corresponding to the target action script in the material resource library is obtained.

8. The method according to any one of claims 1 to 6, characterized in that: The step of obtaining a target action script matched by the interactive voice in a script database according to the interactive voice comprises: Determining a live broadcast scene in which the three-dimensional virtual image is located and image attributes of the three-dimensional virtual image; According to the live broadcast scene, image attributes and interactive voice, a target action script matching the live broadcast scene, image attributes and interactive voice is obtained in the script database.

9. An interactive device for a live broadcast room virtual image, characterized in that: The device comprises: A voice receiving module is used to obtain a preset test text for interactive voice testing, display the sorting information of each test text segment in the preset test text in different viewing angles of the local user on the live broadcast room, determine the local user whose input test voice matches the preset test text as a target user, and receive the interactive voice input by the target user to the three-dimensional virtual image in the live broadcast room; A script acquisition module, used for acquiring a target action script matched by the interactive speech in a script database according to the interactive speech; A resource acquisition module, used to acquire the material resources corresponding to the target action script in the material resource library according to the target action script; The action playing module is used to play the response action of the three-dimensional virtual image according to the target action script and material resources.

10. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images