Virtual character control method, device, computer equipment and storage medium

Abstract

This application relates to a virtual character control method, apparatus, computer equipment, virtual character interaction system, and storage medium. The method includes: in response to receiving a user interaction screen output by an image acquisition device, performing element decomposition processing on the user interaction screen to obtain a real screen element set; performing element identification and matching on the real screen element set based on a preset virtual screen element database to obtain a virtual screen element set; generating virtual character control instructions based on the virtual screen element set, and controlling the virtual character to perform interactive actions based on the virtual character control instructions. This method can enrich the interactive content between the virtual character and the user.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method and apparatus for controlling a virtual character, a computer device, a virtual character interaction system, and a storage medium. Background Art

[0002] With the development of artificial intelligence (AI) technology, virtual characters in the virtual world have appeared in different application scenarios such as games, advertisements, endorsements, performances or live broadcasts. For example, virtual anchors in interactive games, intelligent triage interactions on large screens in hospitals, interactive virtual front desks for enterprises, and intelligent interactive images in exhibitions.

[0003] However, in the prior art, the interactive content between the virtual character and the user is relatively simple, resulting in a lack of in-depth interaction between the user and the virtual character. Summary of the Invention

[0004] Based on this, it is necessary to provide a virtual character control method, device, computer equipment, virtual character interaction system and storage medium that can enrich the interactive content between the virtual character and the user in order to address the above technical problems.

[0005] In a first aspect, a method for controlling a virtual character is provided, the method comprising:

[0006] In response to receiving the user interaction picture output by the image acquisition device, performing element decomposition processing on the user interaction picture to obtain a real picture element set;

[0007] Based on a preset virtual screen element database, element recognition and matching are performed on the real screen element set to obtain a virtual screen element set;

[0008] Generate a virtual character control instruction according to the virtual screen element set, and control the virtual character to perform interactive actions according to the virtual character control instruction.

[0009] In one embodiment, the virtual screen element database includes at least one virtual screen element sub-database; based on the preset virtual screen element database, element recognition and matching are performed on the real screen element set to obtain the virtual screen element set, including: category recognition of each element in the real screen element set; based on the virtual screen element sub-database corresponding to the category recognition result of each element, element recognition and matching are performed on each element to obtain the virtual screen element set.

[0010] In one embodiment, the virtual screen element sub-database includes a virtual interactive action sub-database; performing category identification on each element in the real screen element set includes identifying the user's real action elements from the real screen element set; performing element identification and matching on each element based on the virtual screen element sub-database corresponding to the category identification result of each element, and obtaining the virtual screen element set includes: performing element identification and matching on the user's real action elements based on the virtual interactive action sub-database to obtain virtual interactive action elements; the virtual screen element set includes virtual interactive action elements; wherein the user's real action elements are used to represent the head movements and / or body movements displayed by the user in the user interactive screen; the virtual interactive action elements are used to represent the head movements and / or body movements required for the virtual character to complete the interactive action.

[0011] In one embodiment, the virtual screen element sub-database includes a virtual interactive props sub-database; performing category identification on each element in the real screen element set includes identifying the user's real prop elements from the real screen element set; performing element identification and matching on each element based on the virtual screen element sub-database corresponding to the category identification result of each element, and obtaining the virtual screen element set includes: identifying the user's real prop elements to obtain the prop type, prop identifier and prop color; matching the corresponding interactive prop frame in the virtual interactive prop sub-database according to the prop type; rendering the interactive prop frame according to the prop identifier and prop color to obtain the virtual interactive prop element; the virtual screen element set also includes virtual interactive prop elements; wherein the user's real prop elements are used to represent the props used by the user in the user interactive screen; the virtual interactive prop elements are used to represent the props required for the virtual character to complete the interactive action.

[0012] In one embodiment, the virtual screen element sub-database includes a virtual interactive expression sub-database; performing category identification on each element in the real screen element set includes identifying user facial expression elements from the real screen element set; performing element identification and matching on each element based on the virtual screen element sub-database corresponding to the category identification result of each element, and obtaining a virtual screen element set includes: performing element identification and matching on user facial expression elements based on the virtual interactive expression sub-database to obtain virtual interactive expression elements; the virtual screen element set also includes virtual interactive expression elements; wherein, the user facial elements are used to represent the facial expressions displayed by the user in the user interactive screen; the virtual interactive expression elements are used to represent the facial expressions required for the virtual character to complete the interactive action.

[0013] In one embodiment, the above method also includes: in response to receiving user interaction audio output by the audio acquisition device, identifying and processing the user interaction audio to obtain a real audio keyword set; correcting each element in the virtual screen element set according to the real audio keyword set to obtain a corrected virtual screen element set; generating a virtual character control instruction according to the virtual screen element set, and controlling the virtual character to perform interactive actions according to the virtual character control instruction, including: generating a virtual character control instruction according to the corrected virtual screen element set, and controlling the virtual character to perform interactive actions according to the virtual character control instruction.

[0014] In one embodiment, the real screen element set also includes the user's real background elements; the above method also includes: generating background rendering instructions based on the user's real background elements, and rendering the background where the virtual character is located according to the background rendering instructions; wherein the user's real background elements are used to represent the environmental background of the user in the user interaction screen.

[0015] In one embodiment, the virtual character control instruction includes a first virtual character control instruction and a second virtual character control instruction; the virtual character control instruction is generated according to the virtual screen element set, and the virtual character is controlled to perform interactive actions according to the virtual character control instruction, including: in response to selecting an interface display mode, the first virtual character control instruction is generated according to the virtual screen element set, and the virtual character displayed on the user interface is controlled to perform interactive actions according to the first virtual character control instruction; in response to selecting a holographic projection mode, the second virtual character control instruction is generated according to the virtual screen element set, and the virtual character presented by the holographic projection device is controlled to perform interactive actions according to the second virtual character control instruction.

[0016] In a second aspect, a device for controlling a virtual character is provided, the device comprising a picture disassembly module, a recognition and matching module, and a character control module.

[0017] Among them, the picture disassembly module is used to respond to the user interaction picture output by the image acquisition device, perform element disassembly processing on the user interaction picture, and obtain a real picture element set; the recognition and matching module is used to perform element recognition and matching on the real picture element set based on a preset virtual picture element database, and obtain a virtual picture element set; the character control module is used to generate virtual character control instructions according to the virtual picture element set, and control the virtual character to perform interactive actions according to the virtual character control instructions.

[0018] In a third aspect, a computer device is provided. The storage device includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, the steps of any method in the above method embodiments are implemented.

[0019] In a fourth aspect, a virtual character interaction system is provided, the virtual character interaction system comprising an image acquisition device and a computer device according to any one of the above device embodiments.

[0020] The image acquisition device is electrically connected to the computer device for acquiring and outputting user interaction images.

[0021] In a fifth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of any method in the above method embodiments are implemented.

[0022] The above-mentioned virtual character control method, device, computer equipment, virtual character interaction system and storage medium, in response to receiving the user interaction screen output by the image acquisition device, perform element decomposition processing on the user interaction screen to obtain a real screen element set; then, based on a preset virtual screen element database, perform element identification and matching on the real screen element set to obtain a virtual screen element set; then, generate virtual character control instructions according to the virtual screen element set, and control the virtual character to perform interactive actions according to the virtual character control instructions, which can enrich the interactive content between the virtual character and the user, improve the interactive flexibility and authenticity of the virtual character, make the virtual character demonstration more vivid and natural, and enhance the in-depth interaction between the user and the virtual character. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 A diagram showing an application environment of a method for controlling a virtual character in one embodiment;

[0024] Figure 2 is a first flow chart of a method for controlling a virtual character in one embodiment;

[0025] Figure 3 A flowchart illustrating the steps of performing element recognition and matching on a real screen element set based on a preset virtual screen element database to obtain a virtual screen element set in one embodiment;

[0026] Figure 4 1. A flowchart of steps for performing element identification and matching on each element based on the virtual screen element sub-database corresponding to the category identification result of each element to obtain a virtual screen element set in one embodiment;

[0027] Figure 5 A second flow chart of a method for controlling a virtual character in one embodiment;

[0028] Figure 6 is a third flow chart of a method for controlling a virtual character in one embodiment;

[0029] Figure 7is a fourth flow chart of a method for controlling a virtual character in one embodiment;

[0030] Figure 8 is a structural block diagram of a control device for a virtual character in one embodiment;

[0031] Figure 9 is a diagram of the internal structure of a computer device in one embodiment;

[0032] Figure 10 is a schematic diagram of a first internal structure of a virtual character interaction system in one embodiment;

[0033] Figure 11 Schematic diagram of the second internal structure of the virtual character interaction system in one embodiment. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0035] To facilitate understanding of the present application, the present application will be described more fully below with reference to the accompanying drawings. The accompanying drawings provide embodiments of the present application. However, the present application may be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to make the disclosure of the present application more thorough and comprehensive.

[0036] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application pertains. The terms used herein in the specification of this application are for the purpose of describing specific embodiments only and are not intended to limit this application.

[0037] It will be understood that the terms "first," "second," etc., used herein may be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish a first element from another element. For example, a first resistor may be referred to as a second resistor, and similarly, a second resistor may be referred to as a first resistor without departing from the scope of this application. The first resistor and the second resistor are both resistors, but they are not the same resistor.

[0038] It can be understood that the “connection” in the following embodiments should be understood as “electrical connection”, “communication connection”, etc. if there is transmission of electrical signals or data between the connected circuits, modules, units, etc.

[0039] When used herein, the singular forms "a", "an", and "the" may also include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the terms "include / comprise" or "have" and the like specify the presence of stated features, integers, steps, operations, components, parts, or combinations thereof, but do not preclude the possibility of the presence or addition of one or more other features, integers, steps, operations, components, parts, or combinations thereof.

[0040] The virtual character control method provided in this application can be applied to Figure 1 In the application environment shown, the terminal 102 communicates with the server 104 via a network. The terminal 102 may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, and portable wearable devices, and the server 104 may be implemented as an independent server or a server cluster consisting of multiple servers.

[0041] First, as Figure 2 As shown, a method for controlling a virtual character is provided. Figure 1 The server in is used as an example to illustrate, including the following steps 202 to 206.

[0042] Step 202 : in response to receiving the user interaction picture output by the image acquisition device, performing element decomposition processing on the user interaction picture to obtain a real picture element set.

[0043] The user interaction screen refers to the screen used by the user to interact with the virtual character, and may include, but is not limited to, user game interaction screens, user advertising interaction screens, user performance interaction screens, or user online live broadcast screens. The real screen element set may include, but is not limited to, user real action elements, user real prop elements, user facial expression elements, and / or user real background elements.

[0044] In one specific example, the user's real action element is used to represent the user's head movements and / or body movements in the user interaction screen, the user's real prop element is used to represent the props used by the user in the user interaction screen, the user's face element is used to represent the user's facial expressions in the user interaction screen, and the user's real background element is used to represent the user's environmental background in the user interaction screen. The above are only specific examples. In actual applications, they can be flexibly configured according to user needs and are not limited here.

[0045] It is understood that the user interaction screen can be captured and output by configuring an image capture device electrically connected to the server. The server can receive the user interaction screen output by the image capture device and perform element decomposition processing on the user interaction screen to obtain a real screen element set.

[0046] Step 204 : Based on a preset virtual screen element database, perform element recognition and matching on the real screen element set to obtain a virtual screen element set.

[0047] The server is pre-configured with a preset virtual screen element database, which may include, but is not limited to, at least one virtual screen element sub-database. Based on the preset virtual screen element database, the server can perform element recognition and matching on a real screen element set to obtain a virtual screen element set. Furthermore, recognition and matching can be implemented using, but is not limited to, a recognition and matching algorithm or a pre-trained recognition and matching neural network model.

[0048] In a specific example, the virtual screen element sub-database may include, but is not limited to, a virtual interactive action sub-database, a virtual interactive props sub-database, and / or a virtual interactive expression sub-database. The virtual interactive action sub-database is used to store various virtual interactive action elements, which are used to represent the head movements and / or body movements required for the virtual character to complete the interactive action. The virtual interactive props sub-database is used to store various virtual interactive prop elements, which are used to represent the props required for the virtual character to complete the interactive action. The virtual interactive expression sub-database is used to store various virtual interactive expression elements, which are used to represent the facial expressions required for the virtual character to complete the interactive action. The above are only specific examples, and in actual applications, they are flexibly set according to user needs and are not limited here.

[0049] In one embodiment, Figure 3 As shown, the virtual screen element database includes at least one virtual screen element sub-database. Based on the preset virtual screen element database, element recognition and matching are performed on the real screen element set to obtain the virtual screen element set, including steps 301 and 302.

[0050] Step 301, classifying each element in the real picture element set;

[0051] Step 302 : performing element recognition and matching on each element based on the virtual screen element sub-database corresponding to the category recognition result of each element to obtain a virtual screen element set.

[0052] Among them, the server can perform category identification on each element in the real picture element set and obtain the category identification result of each element; then, based on the virtual picture element sub-database corresponding to the category identification result of each element, it can perform element identification and matching on each element to obtain a virtual picture element set.

[0053] In this embodiment, each element in the real screen element set is classified; then, based on the virtual screen element sub-database corresponding to the category identification result of each element, each element is identified and matched to obtain a virtual screen element set, thereby improving the richness of the virtual screen element set, and indirectly enriching the interactive content between the virtual character and the user, and strengthening the depth of interaction between the user and the virtual character.

[0054] Step 206: Generate a virtual character control instruction based on the virtual screen element set, and control the virtual character to perform interactive actions according to the virtual character control instruction.

[0055] Specifically, the server generates virtual character control instructions based on the virtual screen element set, and can then control the virtual character to perform interactive actions according to the virtual character control instructions. In one specific example, the interactive actions between virtual characters can be displayed through a user interface displayed on a display device, or can be presented in real space using a holographic projection device. The above are only specific examples, and in actual applications, flexible settings can be set based on user needs and are not limited here.

[0056] Based on this, the above-mentioned virtual character control method, in response to receiving the user interaction screen output by the image acquisition device, performs element decomposition processing on the user interaction screen to obtain a real screen element set; then, based on a preset virtual screen element database, performs element identification and matching on the real screen element set to obtain a virtual screen element set; then, generates a virtual character control instruction according to the virtual screen element set, and then controls the virtual character to perform interactive actions according to the virtual character control instruction, thereby enriching the interactive content between the virtual character and the user, improving the interactive flexibility and authenticity of the virtual character, making the virtual character demonstration more vivid and natural, and strengthening the in-depth interaction between the user and the virtual character.

[0057] In one embodiment, the virtual screen element sub-database includes a virtual interactive action sub-database, wherein classifying each element in the real screen element set includes: identifying user real action elements from the real screen element set.

[0058] Specifically, the real screen element set may include, but is not limited to, user real action elements, user real prop elements, user facial expression elements, and / or user real background elements. The server may use, but is not limited to, a category recognition algorithm to identify the user real action elements from the real screen element set during the process of performing category recognition on each element in the real screen element set.

[0059] Based on the virtual screen element sub-database corresponding to the category recognition result of each element, element recognition and matching are performed on each element to obtain a virtual screen element set, including: based on the virtual interactive action sub-database, element recognition and matching are performed on the user's real action elements to obtain virtual interactive action elements.

[0060] Among them, the virtual screen element set includes virtual interactive action elements, and the user real action elements are used to represent the head movements and / or body movements displayed by the user in the user interactive screen; the virtual interactive action elements are used to represent the head movements and / or body movements required for the virtual character to complete the interactive action.

[0061] Specifically, in the process of obtaining a virtual screen element set, the server performs element recognition and matching on each element based on the virtual screen element sub-database corresponding to the category recognition result of each element. At the same time, the server can perform element recognition and matching on the user's real action elements identified from the real screen element set based on the virtual interactive action sub-database, thereby obtaining the virtual interactive action elements in the virtual screen element set.

[0062] In a specific example, the user's real action element can be, but is not limited to, a flower-giving action element, a medal-distributing action element, a dancing action element, or a hand-raising and cheering action element. The server identifies the user's real action element as a flower-giving action element from the real screen element set. Then, based on the virtual interactive action sub-database, the server can perform element recognition and matching on the flower-giving action element, and the resulting virtual interactive action element is a flower-receiving interactive element. The flower-receiving interactive element is stored in the virtual screen element set, and the virtual character completes the flower-receiving interactive action when displaying it. The above is only a specific example. In actual application, it can be flexibly set according to user needs and is not limited here.

[0063] In this embodiment, the user's real action elements are identified from the real picture element set; then, based on the virtual interactive action sub-database, the user's real action elements are identified and matched to obtain virtual interactive action elements, thereby achieving accurate matching of the user's real action elements in the real picture element set with the corresponding virtual interactive action elements, improving the interactive flexibility and authenticity of the virtual character, making the virtual character demonstration more vivid and natural, and strengthening the in-depth interaction between the user and the virtual character.

[0064] In one embodiment, the virtual screen element sub-database includes a virtual interactive prop sub-database, wherein the class identification of each element in the real screen element set includes identifying the user's real prop element from the real screen element set.

[0065] Specifically, the real screen element set may include, but is not limited to, user real action elements, user real prop elements, user facial expression elements, and / or user real background elements. The server may, but is not limited to, use a category recognition algorithm to identify the user real prop elements from the real screen element set during the process of performing category recognition on each element in the real screen element set.

[0066] like Figure 4 As shown, based on the virtual screen element sub-database corresponding to the category recognition result of each element, element recognition and matching are performed on each element to obtain a virtual screen element set, including steps 401 to 403.

[0067] Step 401: Identify and process the user's real prop elements to obtain the prop type, prop identifier, and prop color.

[0068] Step 402: Match the corresponding interactive prop framework in the virtual interactive prop sub-database according to the prop type.

[0069] Step 403 : Render the interactive prop frame according to the prop identifier and the prop color to obtain a virtual interactive prop element.

[0070] The virtual screen element set also includes virtual interactive prop elements; user real prop elements are used to represent props used by users in the user interactive screen; and virtual interactive prop elements are used to represent props required for virtual characters to complete interactive actions. The server identifies and matches each element based on the virtual screen element sub-database corresponding to the category identification results of each element to obtain the virtual screen element set. The server can then identify and process the user real prop elements to obtain the prop type, prop identifier, and prop color. Then, based on the prop type, the server matches the corresponding interactive prop frame in the virtual interactive prop sub-database. Next, the interactive prop frame is rendered based on the prop identifier and prop color to obtain the virtual interactive prop elements in the virtual screen element set.

[0071] In a specific example, the user's real prop element can be, but is not limited to, a real flower element, a real medal element, and a real water cup element. The server identifies the user's real prop element as a real flower element from the real screen element set; then, the real flower element is identified and processed to obtain the prop type of the real flower element as a bouquet, the prop identifier of the real flower element as the Fresh Story brand, and the prop color of the real flower element as crimson; then, based on the prop type of the real flower element as a bouquet, the corresponding interactive prop frame can be matched in the virtual interactive prop sub-database as a virtual bouquet frame; finally, the prop identifier of the Fresh Story brand, i.e., the real flower element, and the crimson color, i.e., the prop color of the real flower element, are rendered into the above-mentioned virtual bouquet frame, thereby obtaining the corresponding virtual interactive prop element, which can be displayed in the hands of the virtual character, thereby improving the interactive flexibility and authenticity of the virtual character, making the virtual character presentation more vivid and natural, and strengthening the deep interaction between the user and the virtual character. The above is only a specific example. In actual application, it can be flexibly set according to user needs and is not limited here.

[0072] In this embodiment, the user's real prop elements are identified and processed to obtain the prop type, prop identifier and prop color; then, according to the prop type, the corresponding interactive prop framework is matched in the virtual interactive prop sub-database; then, according to the prop identifier and prop color, the interactive prop framework is rendered to obtain the virtual interactive prop elements, thereby improving the interactive flexibility and authenticity of the virtual character, making the virtual character demonstration more vivid and natural, and strengthening the in-depth interaction between the user and the virtual character.

[0073] In one embodiment, the virtual screen element sub-database includes a virtual interactive expression sub-database, wherein the class identification of each element in the real screen element set includes: identifying user facial expression elements from the real screen element set.

[0074] Specifically, the real picture element set may include, but is not limited to, user real action elements, user real prop elements, user facial expression elements, and / or user real background elements. The server may use, but is not limited to, a category recognition algorithm to identify the user facial expression element from the real picture element set during the process of performing category recognition on each element in the real picture element set.

[0075] Based on the virtual screen element sub-database corresponding to the category recognition result of each element, element recognition and matching are performed on each element to obtain a virtual screen element set, including: based on the virtual interactive expression sub-database, element recognition and matching are performed on the user's facial expression elements to obtain virtual interactive expression elements.

[0076] Among them, the virtual screen element set also includes virtual interactive expression elements; among them, the user face element is used to represent the facial expression displayed by the user in the user interactive screen; the virtual interactive expression element is used to represent the facial expression required for the virtual character to complete the interactive action.

[0077] Specifically, in the process of obtaining a virtual screen element set, the server performs element recognition and matching on each element based on the virtual screen element sub-database corresponding to the category recognition result of each element. At the same time, the server can perform element recognition and matching on the user's facial expression elements based on the virtual interactive expression sub-database to obtain the virtual interactive expression elements in the virtual screen element set.

[0078] In a specific example, the user's facial expression elements can be determined based on the state of the eyes, mouth and / or eyebrows in the user interaction screen. User facial expression elements may include happy expression elements, sad expression elements and surprised expression elements, etc. The server identifies the user's facial expression elements as happy expression elements from the real screen element set; then, based on the virtual interactive expression sub-database, the happy expression elements, that is, the user's facial expression elements, can be identified and matched to obtain the virtual interactive expression elements as happy interactive expression elements, so that the virtual character can show a happy interactive expression, thereby improving the interactive flexibility and authenticity of the virtual character, making the virtual character demonstration more vivid and natural, and strengthening the deep interaction between the user and the virtual character. The above is only a specific example, and it is flexibly set according to user needs in actual applications and is not limited here.

[0079] In this embodiment, by identifying the user's facial expression elements from the real picture element set; and then performing element recognition and matching on each element to obtain the virtual picture element set, the user's facial expression elements can be recognized and matched based on the virtual interactive expression sub-database to obtain the virtual interactive expression elements in the virtual picture element set, thereby improving the flexibility and authenticity of the virtual character's expression interaction, making the expression demonstrated by the virtual character more vivid and natural, and strengthening the in-depth interaction between the user and the virtual character.

[0080] In one embodiment, Figure 5 As shown, the above method also includes step 501 and step 502.

[0081] Step 501: In response to receiving user interaction audio output by an audio acquisition device, the user interaction audio is recognized and processed to obtain a real audio keyword set.

[0082] Step 502 : Correct each element in the virtual picture element set according to the real audio keyword set to obtain a corrected virtual picture element set.

[0083] It is understood that user interaction audio can be collected and output by configuring an audio collection device electrically connected to the server. The server can, upon receiving the user interaction audio output by the audio collection device, perform recognition processing on the user interaction audio to obtain a real audio keyword set; then, based on the real audio keyword set, correct each element in the virtual screen element set to obtain a corrected virtual screen element set, thereby avoiding matching errors or errors in the virtual screen element set due to obstructions in the user interaction screen, recognition errors in the user interaction screen, or garbled characters in the user interaction screen during the control of the virtual character.

[0084] Generating a virtual character control instruction according to the virtual screen element set, and controlling the virtual character to perform interactive actions according to the virtual character control instruction, includes step 503 .

[0085] Step 503 : Generate a virtual character control instruction according to the corrected virtual screen element set, and control the virtual character to perform interactive actions according to the virtual character control instruction.

[0086] Specifically, the server can generate virtual character control instructions based on the corrected set of virtual screen elements, and control the virtual character to perform interactive actions according to the virtual character control instructions. In one specific example, the interactive actions performed by the virtual characters can be displayed through a user interface displayed by a display device, or can be presented in real space using a holographic projection device. The above are only specific examples, and in actual applications, flexible settings can be set based on user needs and are not limited here.

[0087] In one specific example, a user displays a real flower element on an image capture device. The server may, due to an obstruction in the user interaction screen, recognition errors in the user interaction screen, or garbled text in the user interaction screen, determine that the user's real prop element in the real screen element set is a real ice cream element after performing category recognition on the elements in the real screen element set. In this case, the user speaks the user interaction audio "Give you a flower!" in front of the audio capture device. The server can then process the user interaction audio based on the recognition and obtain a real audio keyword set including the keyword "flower." Next, the server can correct the virtual screen element set including the real ice cream element based on the real audio keyword set including the keyword "flower," obtaining a corrected virtual screen element set including the real flower element. This avoids matching errors or inaccuracies in the virtual screen element set caused by obstructions in the user interaction screen, recognition errors in the user interaction screen, or garbled text in the user interaction screen during the control of the virtual character. The above are only specific examples. In actual applications, the server can flexibly configure the settings based on user needs and is not limited here.

[0088] In this embodiment, in response to receiving the user interaction audio output by the audio acquisition device, the user interaction audio is recognized and processed to obtain a real audio keyword set; then, each element in the virtual screen element set is corrected according to the real audio keyword set to obtain a corrected virtual screen element set; and, a virtual character control instruction is generated according to the corrected virtual screen element set, and the virtual character is controlled to perform interactive actions according to the virtual character control instruction, thereby avoiding matching errors or errors in the virtual screen element set due to obstructions in the user interaction screen, recognition errors in the user interaction screen, or garbled characters in the user interaction screen during the control process of the virtual character, thereby ensuring the accuracy of the virtual screen element set, further improving the authenticity and accuracy of the virtual character, making the virtual character demonstration more realistic, vivid and natural, and strengthening the in-depth interaction between the user and the virtual character.

[0089] In one embodiment, Figure 6 As shown, the real picture element set also includes the user's real background element. The above method further includes step 601.

[0090] Step 601: Generate a background rendering instruction according to the user's real background elements, and render the background where the virtual character is located according to the background rendering instruction.

[0091] The user's real background elements are used to represent the user's surrounding environment in the user interaction screen. The server can automatically generate background rendering instructions based on the user's real background elements and render the background of the virtual character according to the background rendering instructions, thereby improving the realism of the virtual character's background and enhancing the depth of interaction between the user and the virtual character.

[0092] In one specific example, the user interaction screen is decomposed into elements, and the user's real background element within the real screen element set is obtained as the user's office background element. A background rendering instruction is then generated based on the user's office background element, and the background of the virtual character is rendered according to the background rendering instruction, thereby making the background of the virtual character displayed in the user interface more realistic. The above is only a specific example. In actual applications, the settings can be flexibly adjusted according to user needs and are not limited here.

[0093] In this embodiment, by generating background rendering instructions based on the user's real background elements and rendering the background of the virtual character according to the background rendering instructions, the authenticity of the background of the virtual character is improved, and the depth of interaction between the user and the virtual character is enhanced.

[0094] In one embodiment, Figure 7As shown, the virtual character control instruction includes a first virtual character control instruction and a second virtual character control instruction; generating the virtual character control instruction according to the virtual screen element set, and controlling the virtual character to perform interactive actions according to the virtual character control instruction includes: steps 701 and 702.

[0095] Step 701 : In response to selecting an interface display mode, generating a first virtual character control instruction according to a virtual screen element set, and controlling a virtual character displayed on a user interface to perform an interactive action according to the first virtual character control instruction.

[0096] Step 702 : In response to selecting the holographic projection mode, generating a second virtual character control instruction according to the virtual screen element set, and controlling the virtual character presented by the holographic projection device to perform interactive actions according to the second virtual character control instruction.

[0097] The display modes of the virtual character may include, but are not limited to, an interface display mode and a holographic projection mode. The server may determine whether the user has selected the interface display mode or the holographic projection mode based on the user mode selection instruction. It is understood that if the user selects the interface display mode, the virtual character and the interactive actions performed by the virtual character are displayed on the user interface of the display device. If the user selects the holographic projection mode, the virtual character and the interactive actions performed by the virtual character are displayed in real space via the holographic projection device.

[0098] Specifically, when the interface display mode is selected, a character control instruction can be generated based on the virtual screen element set, thereby controlling the virtual character displayed on the user interface to perform interactive actions according to the first virtual character control instruction; that is, the display device can display the virtual character performing interactive actions on the user interface according to the control of the first virtual character control instruction. Furthermore, when the holographic projection mode is selected, the server can generate a second virtual character control instruction based on the virtual screen element set, and control the virtual character presented by the holographic projection device to perform interactive actions according to the second virtual character control instruction; that is, the holographic projection device can display the holographic virtual character performing interactive actions according to the control of the second virtual character control instruction.

[0099] In this embodiment, in response to selecting an interface display mode, a character control instruction can be generated according to a virtual screen element set, so that the virtual character displayed by the user interface can be controlled to perform interactive actions according to the first virtual character control instruction; and, in response to selecting a holographic projection mode, a second virtual character control instruction is generated according to the virtual screen element set, and the holographic virtual character presented by the holographic projection device is controlled to perform interactive actions according to the second virtual character control instruction, thereby improving the interactive flexibility and authenticity of the virtual character, making the virtual character creation more vivid and interesting, and enhancing the user experience.

[0100] It should be understood that although Figure 2-7 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. In addition, Figure 2-7 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0101] Second, as Figure 8 As shown, a device for controlling a virtual character is provided, which includes a picture disassembly module 801, a recognition and matching module 802 and a character control module 803.

[0102] Among them, the picture disassembly module 801 is used to respond to the user interaction picture output by the image acquisition device, perform element disassembly processing on the user interaction picture, and obtain a real picture element set; the recognition and matching module 802 is used to perform element recognition and matching on the real picture element set based on a preset virtual picture element database, and obtain a virtual picture element set; the character control module 803 is used to generate virtual character control instructions according to the virtual picture element set, and control the virtual character to perform interactive actions according to the virtual character control instructions.

[0103] In one embodiment, the virtual screen element database includes at least one virtual screen element sub-database; the identification and matching module 802 includes a category identification unit and an element identification and matching unit.

[0104] Among them, the category identification unit is used to perform category identification on each element in the real picture element set; the element identification and matching unit is used to perform element identification and matching on each element based on the virtual picture element sub-database corresponding to the category identification result of each element to obtain a virtual picture element set.

[0105] In one embodiment, the virtual screen element sub-database includes a virtual interactive action sub-database; and the category identification unit includes a first category identification sub-unit. The first category identification sub-unit is used to identify user real action elements from a set of real screen elements.

[0106] Among them, the element recognition and matching unit includes a first element recognition and matching sub-unit; the first element recognition and matching sub-unit is used to perform element recognition and matching on the user's real action elements based on the virtual interactive action sub-database to obtain virtual interactive action elements; the virtual screen element set includes virtual interactive action elements; among them, the user's real action elements are used to represent the head movements and / or body movements displayed by the user in the user interactive screen; the virtual interactive action elements are used to represent the head movements and / or body movements required for the virtual character to complete the interactive action.

[0107] In one embodiment, the virtual screen element sub-database includes a virtual interactive prop sub-database, and the category identification unit includes a second category identification sub-unit. The second category identification sub-unit is used to identify the user's real prop elements from the real screen element set.

[0108] Among them, the element recognition and matching unit includes a second element recognition and matching sub-unit; the second element recognition and matching sub-unit is used to identify and process the user's real prop elements to obtain the prop type, prop identifier and prop color; the second element recognition and matching sub-unit is used to match the corresponding interactive prop framework in the virtual interactive prop sub-database according to the prop type; the second element recognition and matching sub-unit is used to render the interactive prop framework according to the prop identifier and prop color to obtain virtual interactive prop elements; the virtual screen element set also includes virtual interactive prop elements; among them, the user's real prop elements are used to represent the props used by the user in the user interactive screen; the virtual interactive prop elements are used to represent the props required for the virtual character to complete the interactive action.

[0109] In one embodiment, the virtual screen element sub-database includes a virtual interactive expression sub-database; and the category recognition unit includes a third category recognition sub-unit. The third category recognition sub-unit is used to recognize user facial expression elements from the real screen element set.

[0110] The element recognition and matching unit includes a third element recognition and matching subunit, which is used to perform element recognition and matching on the user's facial expression elements based on the virtual interactive expression sub-database to obtain virtual interactive expression elements.

[0111] Among them, the virtual screen element set also includes virtual interactive expression elements; user facial elements are used to represent the facial expressions displayed by users in the user interactive screen; virtual interactive expression elements are used to represent the facial expressions required for virtual characters to complete interactive actions.

[0112] In one embodiment, the virtual character control device further includes an audio recognition module and a correction module. The character control module 803 includes a first character control unit. The audio recognition module is configured to, in response to receiving user interaction audio output by an audio acquisition device, identify and process the user interaction audio to obtain a real audio keyword set; the correction module is configured to correct each element in the virtual screen element set based on the real audio keyword set to obtain a corrected virtual screen element set; and the first character control unit is configured to generate virtual character control instructions based on the corrected virtual screen element set and control the virtual character to perform interactive actions based on the virtual character control instructions.

[0113] In one embodiment, the real picture element set also includes the user's real background elements; the control device of the virtual character also includes a background rendering module.

[0114] Among them, the background rendering module is used to generate background rendering instructions based on the user's real background elements, and render the background where the virtual character is located according to the background rendering instructions; among them, the user's real background elements are used to represent the environmental background of the user in the user interaction screen.

[0115] In one embodiment, the virtual character control instructions include a first virtual character control instruction and a second virtual character control instruction; the character control module 803 includes a second character control unit. The second character control unit is configured to generate the first virtual character control instruction based on a virtual screen element set in response to selecting an interface display mode, and control the virtual character displayed on the user interface to perform interactive actions based on the first virtual character control instruction; the second character control unit is further configured to generate the second virtual character control instruction based on the virtual screen element set in response to selecting a holographic projection mode, and control the virtual character presented by the holographic projection device to perform interactive actions based on the second virtual character control instruction.

[0116] The specific definitions of the virtual character control device can be found in the definitions of the virtual character control method above and will not be repeated here. Each module in the aforementioned virtual character control device can be implemented in whole or in part through software, hardware, or a combination thereof. Each of these modules can be embedded in or independent of a processor in a computer device in hardware form, or can be stored in a computer device memory in software form, so that the processor can call and execute the corresponding operations of each of these modules.

[0117] In one embodiment, a computer device 9000 is provided. The computer device 9000 may be a server, and its internal structure diagram may be as follows: Figure 9As shown. The computer device 9000 includes a processor, a memory, a network interface and a database connected via a system bus. The processor of the computer device 9000 is used to provide computing and control capabilities. The memory of the computer device 9000 includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device 9000 is used to store real screen element set data and virtual screen element set data. The network interface of the computer device 9000 is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a method for controlling a virtual character is implemented.

[0118] Those skilled in the art will understand that Figure 9 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present application, and does not constitute a limitation on the computer device 9000 to which the solution of the present application is applied. The specific computer device 9000 may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0119] Thirdly, as Figure 9 As shown, a computer device 9000 is provided. The storage device includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, the steps of any method in the above method embodiments are implemented.

[0120] Fourthly, Figure 10 As shown, a virtual character interaction system is provided, which includes an image acquisition device 1001 and a computer device 9000 of any one of the above device embodiments.

[0121] The image acquisition device 1001 is electrically connected to the computer device 9000 and is used to acquire and output the user interaction screen. In one embodiment, the image acquisition device 1001 can be, but is not limited to, a camera.

[0122] In this embodiment, the virtual character interaction system can enrich the interactive content between the virtual character and the user, improve the interactive flexibility and authenticity of the virtual character, make the virtual character presentation more vivid and natural, and strengthen the in-depth interaction between the user and the virtual character.

[0123] In one embodiment, Figure 11 As shown, the virtual character interaction system further includes a display device 1002 , an audio acquisition device 1003 and a holographic projection device 1004 .

[0124] The display device 1002 is electrically connected to the computer device 9000 and is used to display a user interface and display the virtual character performing interactive actions on the user interface according to the control instructions of the virtual character. In one embodiment, the display device can be, but is not limited to, an LED display screen.

[0125] The audio collection device 1003 is electrically connected to the computer device 9000 and is used to collect and output user interaction audio. In one embodiment, the audio collection device 1003 can be, but is not limited to, a microphone.

[0126] The holographic projection device 1004 is electrically connected to the computer device 9000 and is configured to perform interactive actions on the holographic virtual character presented under the control of the second virtual character control instruction.

[0127] In this embodiment, by configuring a virtual character interaction system with an audio acquisition device 1003 and a holographic projection device 1004, the interactive flexibility and authenticity of the virtual character can be improved, so that the virtual character creation is more vivid, more interesting, and the user experience is enhanced.

[0128] In a fifth aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of any method in the above method embodiments are implemented.

[0129] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0130] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0131] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A method for controlling a virtual character, characterized in that: The method comprises: In response to receiving a user interaction screen output by an image acquisition device, performing element decomposition processing on the user interaction screen to obtain a real screen element set; the user interaction screen includes a user game interaction screen, a user advertising interaction screen, a user performance interaction screen, or a user network live broadcast screen; the real screen element set includes user real action elements, user real prop elements, user facial expression elements, and / or user real background elements; Based on a preset virtual screen element database, element recognition and matching are performed on the real screen element set to obtain a virtual screen element set; In response to receiving the user interaction audio output by the audio acquisition device, performing recognition processing on the user interaction audio to obtain a real audio keyword set; Correcting each element in the virtual screen element set according to the real audio keyword set to obtain a corrected virtual screen element set; Generate virtual character control instructions according to the virtual screen element set, and control the virtual character to perform interactive actions according to the virtual character control instructions; wherein, generating virtual character control instructions according to the virtual screen element set, and controlling the virtual character to perform interactive actions according to the virtual character control instructions, includes: generating the virtual character control instructions according to the corrected virtual screen element set, and controlling the virtual character to perform interactive actions according to the virtual character control instructions.

2. The method according to claim 1, characterized in that The virtual screen element database includes at least one virtual screen element sub-database; the real screen element set is subjected to element recognition and matching based on the preset virtual screen element database to obtain a virtual screen element set, including: Performing category recognition on each element in the real picture element set; Based on the virtual screen element sub-database corresponding to the category recognition result of each element, element recognition and matching are performed on each element to obtain the virtual screen element set.

3. The method according to claim 2, characterized in that The virtual screen element sub-database includes a virtual interactive action sub-database; the classifying of each element in the real screen element set includes identifying user real action elements from the real screen element set; The performing element identification and matching on each of the elements based on the virtual screen element sub-database corresponding to the category identification result of each of the elements to obtain the virtual screen element set includes: Based on the virtual interactive action sub-database, element recognition and matching are performed on the user's real action elements to obtain the virtual interactive action elements; wherein, the virtual screen element set includes the virtual interactive action elements; the user's real action elements are used to represent the head movements and / or body movements displayed by the user in the user interactive screen; the virtual interactive action elements are used to represent the head movements and / or body movements required for the virtual character to complete the interactive action.

4. The method according to claim 2, characterized in that The virtual screen element sub-database includes a virtual interactive prop sub-database; the classifying of each element in the real screen element set includes identifying a user's real prop element from the real screen element set; The performing element identification and matching on each of the elements based on the virtual screen element sub-database corresponding to the category identification result of each of the elements to obtain the virtual screen element set includes: Identify the user's real prop elements to obtain the prop type, prop identifier, and prop color; According to the prop type, a corresponding interactive prop framework is matched in the virtual interactive prop sub-database; The interactive prop frame is rendered according to the prop identifier and the prop color to obtain the virtual interactive prop element; the virtual screen element set also includes the virtual interactive prop element; wherein, the user real prop element is used to represent the prop used by the user in the user interactive screen; the virtual interactive prop element is used to represent the prop required for the virtual character to complete the interactive action.

5. The method according to claim 2, characterized in that The virtual screen element sub-database includes a virtual interactive expression sub-database; the class identification of each element in the real screen element set includes identifying user facial expression elements from the real screen element set; The performing element identification and matching on each of the elements based on the virtual screen element sub-database corresponding to the category identification result of each of the elements to obtain the virtual screen element set includes: Based on the virtual interactive expression sub-database, element recognition and matching are performed on the user facial expression elements to obtain the virtual interactive expression elements; the virtual screen element set also includes the virtual interactive expression elements; wherein, the user facial elements are used to represent the facial expressions displayed by the user in the user interactive screen; the virtual interactive expression elements are used to represent the facial expressions required for the virtual character to complete the interactive action.

6. The method according to claim 1, characterized in that The real picture element set also includes a user's real background element; the method further includes: A background rendering instruction is generated according to the user's real background element, and the background where the virtual character is located is rendered according to the background rendering instruction; wherein the user's real background element is used to represent the environmental background where the user is located in the user interaction screen.

7. The method according to claim 1, characterized in that The virtual character control instruction includes a first virtual character control instruction and a second virtual character control instruction; Generating a virtual character control instruction according to the virtual screen element set, and controlling the virtual character to perform interactive actions according to the virtual character control instruction, includes: In response to selecting the interface display mode, generating the first virtual character control instruction according to the virtual screen element set, and controlling the virtual character displayed on the user interface to perform an interactive action according to the first virtual character control instruction; In response to selecting the holographic projection mode, the second virtual character control instruction is generated according to the virtual screen element set, and the virtual character presented by the holographic projection device is controlled to perform interactive actions according to the second virtual character control instruction.

8. A control device for a virtual character, characterized in that: The device comprises: a screen decomposition module configured to, in response to receiving a user interaction screen output by an image acquisition device, perform element decomposition processing on the user interaction screen to obtain a real screen element set; the user interaction screen includes a user game interaction screen, a user advertising interaction screen, a user performance interaction screen, or a user network live broadcast screen; the real screen element set includes user real action elements, user real prop elements, user facial expression elements, and / or user real background elements; an identification and matching module, configured to perform element identification and matching on the real picture element set based on a preset virtual picture element database to obtain a virtual picture element set; an audio recognition module, configured to, in response to receiving user interaction audio output by an audio acquisition device, perform recognition processing on the user interaction audio to obtain a real audio keyword set; a correction module, configured to perform correction processing on each element in the virtual picture element set according to the real audio keyword set to obtain the corrected virtual picture element set; A character control module is used to generate virtual character control instructions based on the virtual screen element set, and control the virtual character to perform interactive actions according to the virtual character control instructions; the character control module includes a first character control unit; the first character control unit is used to generate the virtual character control instructions based on the corrected virtual screen element set, and control the virtual character to perform interactive actions according to the virtual character control instructions.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A virtual character interaction system, characterized in that: The system comprises an image acquisition device and a computer device according to claim 9; The image acquisition device is electrically connected to the computer device and is used to acquire and output the user interaction screen.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images