Actor Virtual Performance System Based on Digital Twin Simulation and Its Application Method

Abstract

The present application relates to an actor virtual performance system based on digital twin simulation and its application method. By using digital twin technology, a virtual reality scene of the actor is constructed, and virtual algorithm interaction is carried out between the constructed virtual actor and the physical actor. The action data displayed by the actor through the wearable device is collected and subjected to simulation analysis, so as to achieve synchronous actions in virtual reality. It can enable a large number of actors to conduct online acting training and learning, thereby improving their acting skills, overcoming the actor dilemmas in the existing technology, enabling actors to improve their acting skills by themselves, and greatly saving the learning cost. By creating a highly realistic digital twin model, the real performance of the actor can be accurately simulated. Through algorithms, real-time interaction between the virtual actor and the physical actor is achieved, so as to achieve the effect of synchronous actions in virtual reality. Through simulation analysis, the accuracy of the digital twin model is improved, providing more accurate data support for future virtual reality applications.

Description

Technical Field

[0001] The present disclosure relates to the field of virtual reality technology, and particularly to an actor virtual performance system based on digital twin simulation, an application method thereof, and an electronic device. Background Art

[0002] Existing actors face various performance dilemmas:

[0003] Firstly, actors need to constantly challenge themselves to adapt to different types of roles and different scripts. This requires actors to have high acting skills and flexible adaptability to meet various different performance requirements.

[0004] Secondly, actors need to closely cooperate with directors, producers, and other actors to ensure the quality and effect of the performance. This requires actors to have high communication skills and teamwork spirit to handle various different opinions and challenges.

[0005] In addition, actors also need to face the pressure of market competition. In the film and television industry, the competition among actors is very fierce. Actors need to continuously improve their acting skills and popularity to obtain more opportunities and recognition.

[0006] Finally, actors also need to face self-challenges. Actors need to constantly explore their acting styles and potential to break through their limitations and create more outstanding performances.

[0007] Therefore, existing actors face various performance dilemmas and need to continuously improve their acting skills, communication skills, teamwork spirit, and competition awareness to cope with various different challenges and pressures. If actors want to improve their acting skills, they can only play small roles or take on marginal characters, and they cannot get in-depth guidance to improve their acting skills. Therefore, their acting learning is limited, and there are not many opportunities in reality for a large number of actors to use, which is even more difficult for these actors. Moreover, actors in dramas need to perform in different scenes, which requires a large amount of manpower and material resources for replacement.

[0008] With the rise of digital twin technology, through mathematical models and real-time data, the physical objects, systems, or processes in the real world are synchronously and interactively represented digitally in the virtual world in real time, which can solve the above-mentioned technical difficulties in actor performance learning.

[0009] Digital twin technology can build a digital twin network platform by creating a virtual mirror image of physical network facilities. Through real-time interaction and mutual influence between the physical network and the twin network, the digital twin network platform can help the network achieve low-cost trial and error, intelligent decision-making, and high-efficiency innovation. The research and application of digital twin networks are still in their infancy in the industry and academia.

[0010] A digital twin network is a network system with physical network entities and virtual twins that can interact and map in real time. In this system, various network management and applications can utilize the network virtual twins constructed by digital twin technology to efficiently analyze, diagnose, simulate, and control the physical network based on data and models.

[0011] Based on the digital twin network, a virtual actor performance system can be created, enabling actors to perform simulations through virtual characters and plots. Therefore, in this technical solution, this solution plans to adopt digital twin technology to construct a virtual reality scene for actors, achieve virtual algorithm interaction between virtual actors and physical actors, so as to achieve the effect of virtual reality synchronous actions, and provide a platform for mass actors to train and learn acting skills. Summary of the Invention

[0012] To solve the above problems, this application proposes an actor virtual performance system based on digital twin simulation, its application method, and an electronic device.

[0013] On one hand, this application proposes an actor virtual performance system based on digital twin simulation, including:

[0014] A motion capture device for collecting the first action data and the second action data of the actor;

[0015] A digital twin Internet of Things platform for constructing a virtual actor model matching the actor entity according to the first action data in the virtual simulation system; and, converting the second action data into the actions of the virtual actor model through an AI algorithm, performing real-time simulation on the virtual actor model, and generating corresponding action simulation data;

[0016] A VR device for visually displaying the action simulation data and realizing the action data interaction between the actor and the digital twin Internet of Things platform;

[0017] A background server for setting corresponding scripts for the actor and analyzing and generating scene levels in different scripts through an AI algorithm; and, scheduling the motion capture device, the digital twin Internet of Things platform, and the VR device according to a preset performance strategy to execute the corresponding scene levels;

[0018] A database for storing scene levels in different scripts;

[0019] Both the database and the digital twin Internet of Things platform are deployed on the background server;

[0020] The motion capture device, the digital twin Internet of Things platform, the VR device, and the database are respectively communicatively connected to the background server.

[0021] As an optional implementation of this application, optionally, the background server is further used for:

[0022] Register and save the actor's identity ID;

[0023] Receive the actor's script selection information, and based on the selection information, retrieve the corresponding scene level from the database;

[0024] Bind the scene level to the actor's identity ID and send it to the VR device.

[0025] As an optional implementation of this application, optionally, the VR device is further configured to:

[0026] Display the scene level, interact with the actor according to the script plot on the scene level, and play the corresponding plot voice;

[0027] After the actor responds to the plot voice, collect the voice information of the actor answering the plot and upload it to the background server.

[0028] As an optional implementation of this application, optionally, the motion capture device is further configured to:

[0029] After the actor responds to the plot voice, collect the second action data of the actor answering the plot and upload it to the background server.

[0030] As an optional implementation of this application, optionally, the background server is further configured to:

[0031] Record and save the voice information and the second action data in real time to the database;

[0032] Bind the voice information and the second action data respectively under the actor's identity ID;

[0033] Synchronize the second action data to the digital twin IoT platform in a real-time incremental synchronization manner.

[0034] As an optional implementation of this application, optionally, the background server is further configured to:

[0035] Monitor the event execution process of the scene level;

[0036] Judge whether the event execution process of the current plot triggers an event branch preset in the scene level:

[0037] If triggered, identify the semantic information of the actor from the previous voice information, select the plot branch that best matches the semantic information from the event branch, and switch to execute the current plot branch;

[0038] Otherwise, give up.

[0039] As an optional implementation of this application, optionally, the background server is further configured to:

[0040] Monitor whether the trigger timing of the preset plot branch in the scenario level is reached:

[0041] If so, switch to execute the plot branch at the current time;

[0042] Otherwise, give up.

[0043] As an optional implementation solution of the present application, optionally, the background server is further configured to:

[0044] Export the action simulation data generated by executing this scenario level from the digital twin Internet of Things platform, and bind it under the identity ID of the actor;

[0045] Divide the action simulation data according to different plot nodes of the scenario level to obtain action simulation data of several different plot nodes, and play them in order;

[0046] Score the action simulation data of the actor at different plot nodes in the scenario level according to the preset scoring mechanism, and bind the score under the identity ID of the actor.

[0047] On the other hand, the present application proposes an application method for an actor virtual performance system based on digital twin simulation, including the following steps:

[0048] The actor logs in to the background server, activates the motion capture device and the VR device, and establishes communication with the background server;

[0049] Make corresponding selection information for the script from the database. The background server retrieves the corresponding scenario level from the database according to the selection information, binds the scenario level with the identity ID of the actor, and sends it to the VR device;

[0050] Display the scenario level through the VR device, interact with the actor according to the script plot on the scenario level, and play the corresponding plot voice;

[0051] The actor responds to the plot voice, collects the second action data of the actor answering the plot by the motion capture device and uploads it to the background server, and the background server synchronizes the second action data to the digital twin Internet of Things platform in a real-time incremental synchronization manner;

[0052] The digital twin Internet of Things platform converts the second action data into the actions of the virtual actor model through the AI algorithm, performs real-time simulation on the virtual actor model, generates the corresponding action simulation data, and saves it.

[0053] On the other hand, the present application also proposes an electronic device, including:

[0054] A processor;

[0055] A memory for storing processor-executable instructions;

[0056] Wherein, when the processor is configured to execute the executable instructions, the described application method is implemented.

[0057] Technical effects of the present invention:

[0058] By adopting digital twin technology, this application constructs a virtual reality scene for actors. Through virtual algorithm interaction between the constructed virtual actors and physical actors, the action data displayed by the actors collected through wearable devices is subjected to simulation analysis, and then the virtual actor tasks constructed by the system are performed, so as to achieve synchronous actions in virtual reality. It enables a large number of actors to conduct online acting skills training and learning, thereby improving their acting skills, overcoming the dilemmas of actors in the prior art, enabling actors to improve their acting skills by themselves, and greatly saving learning costs.

[0059] By creating a highly realistic digital twin model, this application can accurately simulate the real performance of actors. Through algorithms, real-time interaction between virtual actors and physical actors is achieved, resulting in the effect of synchronous actions in virtual reality. Through simulation analysis, the accuracy of the digital twin model is improved, providing more accurate data support for future virtual reality applications.

[0060] According to the following detailed description of the exemplary embodiments with reference to the accompanying drawings, other features and aspects of the present disclosure will become clear. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] The accompanying drawings, which are included in and constitute a part of this specification, illustrate the exemplary embodiments, features, and aspects of the present disclosure together with the specification, and are used to explain the principles of the present disclosure.

[0062] Figure 1 Shown is a schematic diagram of the application system of the present invention;

[0063] Figure 2 Shown is a schematic diagram of the architecture of a digital twin Internet of Things platform adopted by the present invention;

[0064] Figure 3 Shown is a schematic diagram of the flow of the method of the present invention;

[0065] Figure 4 Shown is an application schematic diagram of the electronic device of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0066] The following will detail various exemplary embodiments, features, and aspects of the present disclosure with reference to the accompanying drawings. The same reference numerals in the drawings denote elements having the same or similar functions. Although various aspects of the embodiments are shown in the drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0067] As used herein, the term "exemplary" means "serving as an example, embodiment, or illustration". Any embodiment described as "exemplary" herein need not be construed as superior or better than other embodiments.

[0068] In addition, for a better illustration of the present disclosure, numerous specific details are given in the following detailed description. Those skilled in the art should understand that the present disclosure can be implemented without some of these specific details. In some instances, well-known means, elements, and circuits have not been described in detail so as to highlight the gist of the present disclosure.

[0069] Embodiment 1

[0070] As Figure 1 shown, on the one hand, the present application proposes an actor virtual performance system based on digital twin simulation, including:

[0071] A motion capture device for collecting the first action data and the second action data of the actor;

[0072] A digital twin Internet of Things platform for constructing a virtual actor model matching the actor entity in a virtual simulation system according to the first action data; and converting the second action data into the actions of the virtual actor model through an AI algorithm, performing real-time simulation on the virtual actor model, and generating corresponding action simulation data;

[0073] A VR device for visually displaying the action simulation data and realizing the action data interaction between the actor and the digital twin Internet of Things platform;

[0074] A background server for setting a corresponding script for the actor and analyzing and generating scene levels in different scripts through an AI algorithm; and scheduling the motion capture device, the digital twin Internet of Things platform, and the VR device according to a preset performance strategy to execute the corresponding scene levels;

[0075] A database for storing scene levels in different scripts;

[0076] Both the database and the digital twin Internet of Things platform are deployed on the background server;

[0077] The motion capture device, the digital twin Internet of Things platform, the VR device, and the database are respectively communicatively connected to the background server.

[0078] The actor simulation system of this solution enables the actor to perform acting training according to a predetermined script in a virtual interaction manner.

[0079] This solution requires collecting the physical actor's action data, including movements, expressions, sounds, etc., and using this data to create a digital twin model in a virtual environment. This model will simulate the actor's real performance as accurately as possible.

[0080] After the digital twin model is established, this solution will collect the action data of the physical actor through the wearable device, and then convert this data into the action of the virtual actor through the virtual algorithm (the data processing algorithm of the virtual simulation system used on some digital twin IoT platforms currently depends on the digital twin IoT platform selected by the user, so this embodiment will not be repeated or limited). This process requires real-time data transmission and processing to ensure that the actions of the virtual actor are synchronized with the physical actor.

[0081] After the virtual actor completes the action performance, the program will conduct simulation analysis. By comparing the action data of the virtual actor and the real actor, the program can evaluate the accuracy of the digital twin model. This will help the program improve the model and enhance its ability to simulate the real world.

[0082] The main implementation methods and functions of this solution are:

[0083] On the backend server, the administrator can set corresponding scripts for the actors and analyze them through AI algorithms (text recognition, keyword recognition, etc.) to generate different scene levels based on the plot nodes. Corresponding plot tasks are arranged in each scene level. The plot tasks set for the actors are executed through the backend control and application of motion capture equipment, digital twin Internet platform and VR equipment.

[0084] The VR device will specifically execute the plot dialogue in each scene level. After the actor wears the VR device, the plot level is displayed through the VR device, and the actor interacts with the plot dialogue task and performs actions in response to the plot task. At the same time, the motion capture device collects the actor's action data and feeds it back to the background, which synchronizes the action data to the digital twin Internet platform in real time. The platform performs virtual algorithm processing, calculates the action data into corresponding simulation action parameters, simulates and controls the character actions of the virtual actor model, and performs performance simulation.

[0085] The platform can record the actors' motion simulation data in real time, and can perform motion simulation performances based on the platform's simulation analysis function. After the subsequent performance, the actors can view their own motion simulation data through the background server and display it visually.

[0086] Through the action simulation of the virtual actor model, the virtual performance is carried out, allowing the actors to judge and improve their acting skills based on the performance movements of the virtual simulation actions.

[0087] Administrators can also retrieve the corresponding action simulation data based on each actor's identity ID, and score the actor's virtual performance actions at each node according to the plot nodes, comprehensively evaluate the actor's performance skill score, and conduct online remote judging.

[0088] Therefore, the virtual actor system using this solution can enable a large number of actors to conduct online acting training and learning, so as to improve their acting skills, overcome the actor dilemma of existing technology, allow actors to improve their acting skills by themselves, and greatly save learning costs.

[0089] In this embodiment, the choice of the digital twin IoT platform is determined by the user.

[0090] like Figure 2 The digital twin IoT platform shown in the figure has multi-level, multi-physics, and multi-scale system model functions. It can also establish multi-field performance models based on finite elements, build high-fidelity mechanism models through "system + professional" model fusion, and realize high-precision hybrid models of mechanism-data fusion through mechanism models, experimental data, and machine learning algorithms. At the tool level, the mechanism-data model fusion is realized through data proxy, model reduction, and FMI interface methods, achieving a balance between high efficiency and high-precision simulation of digital twins.

[0091] The digital twin IoT platform can provide storage management technologies and mechanisms for the entire set of virtual and real data based on a single or multiple data sources, realize systematic storage and management of models, algorithms, and design / simulation / test / operation and maintenance data throughout the life cycle of the digital twin, achieve effective cross-task and cross-stage management and control, maintain the associations and changes between data, and ensure data consistency.

[0092] By unifying and standardizing model metadata and feature data expressions, we can achieve unified access and management of heterogeneous models and data, as well as the extraction, conversion and loading of multi-source data. On the basis of basic transmission security, we fully consider the data ownership requirements and ensure data security and confidentiality.

[0093] The digital twin IoT platform can realize data interaction with the background, motion capture equipment, host computer software, etc. through the communication method of data interface / bus.

[0094] Therefore, in this embodiment, the motion capture device, digital twin Internet of Things platform, VR device and database are respectively communicated with the background server. The communication connection method is not limited in this embodiment and can be used in combination with the existing digital twin platform architecture.

[0095] Motion capture equipment and VR equipment are not limited in this embodiment.

[0096] In this solution, for the digital twin Internet of Things platform, in the virtual simulation system, a virtual actor model matching the actor entity is constructed according to the first action data. Specifically,

[0097] On the digital twin Internet platform, virtual simulation models of corresponding entities can be constructed.

[0098] In this solution, the virtual simulation model is a virtual actor model constructed according to the first action data of the actor. On the platform, virtual simulation modeling can be carried out based on the first action data generated by the actor's actions under specific guidance and environment collected by the motion capture device. The virtual actor model is constructed on the platform, and the virtual actor model is established and optimized in combination with the first action data.

[0099] Specifically, the construction scheme of the virtual simulation model on the platform can be understood in combination with the existing digital twin technology.

[0100] In digital twin technology, a corresponding data mapping relationship needs to be constructed between the constructed virtual actor model and the corresponding actor entity. The pose models of the corresponding action nodes are associated. Subsequently, simulation calculations will be carried out through the action data generated by the actor entity, and the parts of the corresponding virtual model will be synchronously controlled to perform simulation actions. The action mapping relationship between the virtual actor model and the actor entity will be configured with the corresponding mapping relationship after the virtual model is constructed on the platform.

[0101] Digital twin mainly aims to create a virtual body or digital model equivalent to the physical entity. The virtual body can perform simulation analysis on the physical entity, monitor the operating state of the physical entity according to the real-time feedback information of the physical entity's operation, and improve the simulation analysis algorithm of the virtual body based on the collected operating data of the physical entity, so as to provide more accurate decisions for the subsequent operation and improvement of the physical entity's physical products.

[0102] In digital twin technology, the interaction control principle between the virtual model and the real model mainly depends on real-time data collection, modeling, simulation, and analysis.

[0103] First of all, through various sensors and data collection devices, digital twin technology can obtain various data information of the actual system in real time, including system status, behavior, performance, etc. After being processed, these data are used to establish corresponding mathematical models.

[0104] Then, digital twin technology realizes real-time monitoring and prediction of the system status by synchronously updating the mathematical model with the actual system. This synchronous update mechanism enables the virtual model to accurately reflect the status and behavior of the actual system, thus realizing accurate simulation and prediction of the actual system.

[0105] In terms of interactive control, digital twin technology can perform virtual simulation analysis on virtual models through computer programs. This virtual simulation analysis can predict the future state and behavior of the system based on the state and behavior of the actual system, thus providing important reference information for decision-makers.

[0106] In addition, digital twin technology can also optimize the design scheme or adjust the actual system parameters according to the virtual simulation results. This optimization and adjustment can be achieved through computer programs, further improving the performance of the actual system.

[0107] Generally speaking, the principle of interactive control in digital twin technology is to achieve the synchronous update and interactive control of virtual models and real models through real-time data collection, modeling, simulation, and analysis. This technology can be widely applied in various fields, including industrial manufacturing, urban planning, traffic management, medical health, etc.

[0108] Using digital twin technology, a virtual reality scene of the actor is constructed, and virtual algorithm interaction is carried out between the constructed virtual actor and the physical actor. The action data displayed by the actor through the wearable device is collected and subjected to simulation analysis, and then the virtual actor tasks constructed by the system are performed, so as to achieve synchronous actions in virtual reality.

[0109] The system provides the user with the basis for scene and script ideas, and the scene can be intelligently constructed through AI algorithms.

[0110] Next, the performance process of the script will be further described in combination with the functions of the background.

[0111] As an optional implementation scheme of this application, optionally, the background server is further used for:

[0112] Registering and saving the identity ID of the actor;

[0113] Receiving the actor's selection information for the script, and retrieving the corresponding scene level from the database according to the selection information;

[0114] Binding the scene level with the actor's identity ID and sending it to the VR device.

[0115] The background server processes the script through the actor management module:

[0116] 1. The background administrator logs in to the background and can import the conceived script into the system in advance. The system analyzes the scene descriptions (time, location, characters, atmosphere, events) in the script through AI algorithms;

[0117] 2. According to the time, location attributes (type, size, atmosphere) and character image (character name, gender, height, weight), the AI ​​algorithm will filter out the appropriate plot scenes from the database (set the corresponding plot scenes in advance according to the plot, or use AI to generate the corresponding plot environment; after entering the current scene level into the platform, the platform's simulation system will directly call up the corresponding plot environment (such as a virtual palace) based on the script selection (script attributes such as a palace plot), and load the virtual actor into the plot environment. The simulation system can be used to adjust the environment later, such as lighting, special effects, and screen color), and load and combine into scene levels in different scripts;

[0118] 3. According to the description of the atmosphere, the scene art effects are automatically adjusted (lighting, special effects, screen colors, character positions and postures, etc. These script parameters will be automatically read by the system, and the script parameters can be matched by the administrator according to the script. A set of default parameters can be pre-set by the system. After the virtual actor is imported into the plot environment, the system can directly put the virtual actor in the corresponding environment position and trigger the corresponding action);

[0119] 4. Analyze event categories, event branches and triggering timing through event description;

[0120] 5. Users wearing motion capture equipment and VR equipment can enter the system immersively, choose the role they want to play in the script and start performing;

[0121] 6. Plot development can be recorded and played into the system via audio;

[0122] 7. When an event triggers an event branch or the plot development timing reaches the corresponding trigger timing, the system uses voice recognition to identify the event branch that best matches the user's words and best meets the user's expectations;

[0123] 8. After the performance, users can replay the parts of the performance that they are not satisfied with.

[0124] As an optional implementation scheme of the present application, optionally, the VR device is further used for:

[0125] Display the scene level, interact with the actors according to the plot of the script on the scene level, and play the corresponding plot voice;

[0126] After the actor responds to the plot voice, the voice information of the actor's answer to the plot is collected and uploaded to the backend server.

[0127] After the actor logs in to the background server, they can select a script and choose the corresponding script from different performance styles for performance training. They can input the corresponding script selection information to the background, and the background will retrieve the corresponding scene level from the database according to the selection information. For example, if the actor needs to perform a script training in a palace, they can select the corresponding scene level according to the type of the script or other name information, etc.

[0128] After the selection, the background executes the scene level, creates the corresponding scheduling task, and schedules the VR device, motion capture device, and platform to prepare for executing the scene level.

[0129] The VR device has a voice interaction function and sends a voice prompt to the actor to prepare for entering the training of the current scene level.

[0130] When reading the script of the current scene level, the VR device can send the corresponding script voice information to the actor according to the script plot on the scene level and play the corresponding plot voice. After hearing the plot voice, the actor makes corresponding actions and can give feedback. The voice information answering the plot is collected by the VR device, marked with a time tag, and uploaded to the background server, which is recorded and saved in the database by the background.

[0131] As an optional implementation of this application, optionally, the motion capture device is also used for:

[0132] After the actor responds to the plot voice, collect the second action data of the actor answering the plot and upload it to the background server.

[0133] At the same time, the motion capture device will collect the data of the actions made by the actor, upload the collected second action data to the background server, which is saved by the background server and synchronized to the platform at the same time. The digital twin Internet platform reads and analyzes the second action data and performs virtual simulation control of the corresponding virtual actor model according to the analyzed second action data, controlling the virtual actor model to make corresponding virtual simulation actions. The platform records and saves the corresponding virtual simulation data, so as to realize the virtual interaction control between the actor entity and the virtual actor model.

[0134] As an optional implementation of this application, optionally, the background server is also used for:

[0135] Real-time record and save the voice information and the second action data to the database;

[0136] Bind the voice information and the second action data to the actor's identity ID respectively;

[0137] Synchronize the second action data to the digital twin Internet of Things platform in a real-time incremental synchronization manner.

[0138] For real-time incremental synchronization of data, it can be understood by referring to existing different data increment methods.

[0139] As an optional implementation of this application, optionally, the background server is further used for:

[0140] Monitoring the event execution process of the scenario level;

[0141] Judging whether the event execution process of the current plot triggers an event branch preset in the scenario level:

[0142] If triggered, identifying the semantic information of the actor from the previous voice information, selecting the plot branch that best matches the semantic information from the event branch, and switching to execute the current plot branch;

[0143] Otherwise, give up.

[0144] The script process module will monitor the actor's performance process according to the current dialogue interaction data. It can also judge the process through speech recognition.

[0145] As an optional implementation of this application, optionally, the background server is further used for:

[0146] Monitoring whether the triggering time of the preset plot branch in the scenario level is reached:

[0147] If so, switch to execute the plot branch at the current time;

[0148] Otherwise, give up.

[0149] The event execution process is monitored by the background, and the administrator can participate in the monitoring.

[0150] The event branch, that is, a small performance script that needs to be switched, can switch to the corresponding scene for performance, and at the same time switch the corresponding script plot for performance control.

[0151] The triggering time of the preset plot branch, that is, when the current performance reaches a certain moment, the background can switch to other plot tasks at regular intervals for scene switching.

[0152] The triggering time of the preset plot branch and the plot branch are specifically set by the administrator.

[0153] As an optional implementation of this application, optionally, the background server is further used for:

[0154] Exporting the action simulation data generated by executing this scenario level from the digital twin Internet of Things platform and binding it under the identity ID of the actor;

[0155] Divide the action simulation data according to different plot nodes of the scenario levels to obtain the action simulation data of several different plot nodes and play them in order;

[0156] According to the preset scoring mechanism, score the action simulation data of the actor at different plot nodes in the scenario level and bind the scores to the identity ID of the actor.

[0157] Each scenario level can be divided into several nodes according to the plot. For example, according to the time nodes of each plot switch, the scenario level is automatically divided into several plot nodes. Divide the actor's performance data into several nodes of the performance training process according to the plot nodes, and manage and score them respectively, so as to effectively improve the actor's acting skills learning ability at each stage.

[0158] The scoring mechanism is specifically set by the administrator.

[0159] The data management module, for example, uses the API data port to implement the forwarding of action data, etc.

[0160] Therefore, through the creation of a highly realistic digital twin model, this application can accurately simulate the real performance of the actor. Through the algorithm, real-time interaction between the virtual actor and the physical actor is achieved, achieving the effect of synchronous actions in virtual reality. Through simulation analysis, the accuracy of the digital twin model is improved, providing more accurate data support for future virtual reality applications.

[0161] Obviously, those skilled in the art should understand that to implement all or part of the processes in the above embodiments, it can be completed by instructing relevant hardware through a computer program. The program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the above control embodiments. Those skilled in the art can understand that to implement all or part of the processes in the above embodiments, it can be completed by instructing relevant hardware through a computer program. The program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the above control embodiments. Among them, the storage medium can be a magnetic disk, an optical disc, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk (Hard Disk Drive, abbreviated as HDD) or a solid-state drive (Solid-State Drive, SSD), etc.; the storage medium can also include a combination of the above types of memories.

[0162] Embodiment 2

[0163] Such as Figure 3Based on the implementation principle of Embodiment 1, on the other hand, the present application proposes an application method for an actor virtual performance system based on digital twin simulation, including the following steps:

[0164] S1. The actor logs in to the background server, activates the motion capture device and the VR device, and establishes communication with the background server.

[0165] S2. Make corresponding selection information for the script from the database. The background server retrieves the corresponding scene level from the database according to the selection information, binds the scene level with the actor's identity ID, and sends it to the VR device.

[0166] S3. Display the scene level through the VR device, interact with the actor according to the script plot on the scene level, and play the corresponding plot voice.

[0167] S4. The actor responds to the plot voice, collects the second action data of the actor's answer to the plot through the motion capture device and uploads it to the background server, and the background server synchronizes the second action data to the digital twin Internet of Things platform in a real-time incremental synchronization manner.

[0168] S5. The digital twin Internet of Things platform converts the second action data into the actions of the virtual actor model through the AI algorithm, performs real-time simulation on the virtual actor model, generates the corresponding action simulation data, and saves it.

[0169] For each step of the above method, please refer to Embodiment 1 for understanding, and details will not be elaborated in this embodiment.

[0170] For the interaction functions between each entity, please refer to Embodiment 1 for further understanding.

[0171] Each module or step of the present invention described above can be implemented by a general computing system. They can be concentrated on a single computing system or distributed on a network composed of multiple computing systems. Optionally, they can be implemented by program codes executable by the computing system, so that they can be stored in the storage system and executed by the computing system, or they can be separately made into individual integrated circuit modules, or multiple modules or steps of them can be made into a single integrated circuit module to be implemented. In this way, the present invention is not limited to any specific combination of hardware and software.

[0172] Embodiment 3

[0173] As Figure 4 shown, further, on the other hand, the present application also proposes an electronic device, including:

[0174] A processor;

[0175] A memory for storing processor-executable instructions;

[0176] Wherein, an application method implemented when the processor is configured to execute the executable instructions.

[0177] An electronic device according to an embodiment of the present disclosure includes a processor and a memory for storing processor-executable instructions. Wherein, the processor is configured to implement the application method described in Embodiment 2 when executing the executable instructions.

[0178] Here, it should be noted that the number of processors can be one or more. At the same time, in the electronic device according to the embodiment of the present disclosure, an input system and an output system may also be included. Wherein, the processor, the memory, the input system and the output system may be connected through a bus or in other ways, which is not specifically limited herein.

[0179] The memory, as a computer-readable storage medium, can be used to store software programs, computer-executable programs and various modules, such as: programs or modules corresponding to the application method of the embodiment of the present disclosure. The processor executes various functional applications and data processing of the electronic device by running the software programs or modules stored in the memory.

[0180] The input system can be used to receive input numbers or signals. Wherein, the signal can be a key signal related to the user settings and function control of the device / terminal / server. The output system may include a display device such as a display screen.

[0181] The above has described the embodiments of the present disclosure. The above description is exemplary and not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations are obvious to those of ordinary skill in the art in the technical field without departing from the scope and spirit of the described embodiments. The choice of terms used herein is intended to best explain the principles of the embodiments, practical applications or improvements to the technology in the market, or to enable other ordinary skill in the art in the technical field to understand the embodiments disclosed herein.

Claims

1. Actor virtual performance system based on digital twin simulation, characterized by: include: A motion capture device, used for collecting first motion data and second motion data of an actor; A digital twin IoT platform is used to construct a virtual actor model matching the actor entity according to the first action data in a virtual simulation system; and to convert the second action data into an action of the virtual actor model through an AI algorithm, perform real-time simulation on the virtual actor model, and generate corresponding action simulation data; VR equipment is used to visualize motion simulation data and realize the interaction of motion data between actors and the digital twin IoT platform; The backend server is used to set corresponding scripts for actors, and analyze and generate scene levels in different scripts through AI algorithms; and dispatch motion capture equipment, digital twin IoT platform and VR equipment according to the preset performance strategy to execute the corresponding scene levels; the backend server processes the script through the actor management module: The backstage administrator logs in to the backstage and imports the conceived script into the system in advance. The system uses AI algorithms to analyze the scene descriptions in the script. According to the time, location and character image, the AI ​​algorithm selects the appropriate plot scenes from the database and loads them into scene levels in different scripts. Automatically adjust the scene art effects according to the atmosphere description; Through event description, analyze the event category, event branch and triggering time; Users wear motion capture equipment and VR devices to enter the system immersively, choose the role they want to play in the script and start performing; Plot developments are recorded via audio and played into the system; When an event triggers an event branch or the plot development timing reaches the corresponding trigger timing, the system uses voice recognition to identify the event branch that best matches the user's words and best meets the user's expectations; At the end of the performance, the user will replay the parts of the performance that he was not satisfied with; Database, used to store scenes and levels in different scripts; The database and digital twin IoT platform are deployed on the backend server; The motion capture equipment, digital twin IoT platform, VR equipment and database are connected to the backend server respectively; The backend server is also used to export the action simulation data generated by executing this scene level from the digital twin IoT platform and bind it to the actor’s identity ID; According to different plot nodes of scene levels, the action simulation data is divided to obtain action simulation data of several different plot nodes and play them in order; According to the preset scoring mechanism, the actor's action simulation data at different plot nodes in the scene level is scored, and the score is bound to the actor's identity ID.

2. The actor virtual performance system based on digital twin simulation according to claim 1 is characterized in that: Backend servers are also used for: Register and save the actor's identity ID; Receive the actor's selection information for the script, and retrieve the corresponding scene level from the database based on the selection information; Bind the scene level with the actor’s identity ID and send it to the VR device.

3. The actor virtual performance system based on digital twin simulation according to claim 2 is characterized in that: VR devices are also used for: Display the scene level, interact with the actors according to the plot of the script on the scene level, and play the corresponding plot voice; After the actor responds to the plot voice, the voice information of the actor's answer to the plot is collected and uploaded to the backend server.

4. The actor virtual performance system based on digital twin simulation according to claim 3 is characterized in that: Motion capture equipment is also used for: After the actor responds to the plot voice, the second action data of the actor's answer to the plot is collected and uploaded to the backend server.

5. The actor virtual performance system based on digital twin simulation according to claim 4 is characterized in that: Backend servers are also used for: Record and save voice information and second action data to the database in real time; Bind the voice information and the second action data to the actor's identity ID respectively; The second action data is synchronized to the digital twin IoT platform using real-time incremental synchronization.

6. The actor virtual performance system based on digital twin simulation according to claim 5 is characterized in that: Backend servers are also used for: Monitor the event execution process of the scene level; Determine the event execution process of the current plot and whether it triggers the event branch preset in the scene level: If triggered, the actor's semantic information is identified from the previous voice information, and the plot branch that best matches the semantic information is selected from the event branch, and the current plot branch is switched to execute; Otherwise give up.

7. The actor virtual performance system based on digital twin simulation according to claim 5 is characterized in that: Backend servers are also used for: Monitor whether the triggering time of the preset plot branch in the scene level is reached: If yes, switch to execute the plot branch of the current time; Otherwise give up.

8. The application method of the actor virtual performance system based on digital twin simulation according to any one of claims 1 to 7, characterized in that: The steps include: The actor logs in to the backend server, activates the motion capture device and VR device, and establishes communication with the backend server; The script is selected from the database. The backend server retrieves the corresponding scene level from the database based on the selection information, binds the scene level to the actor's identity ID, and sends it to the VR device. Use VR devices to display scene levels, interact with actors according to the plot of the script on the scene level, and play the corresponding plot voice; The actor responds to the plot voice, and the motion capture device collects the actor's second action data of answering the plot and uploads it to the backend server. The backend server uses real-time incremental synchronization to synchronize the second action data to the digital twin IoT platform; The digital twin IoT platform converts the second action data into the action of the virtual actor model through the AI ​​algorithm, simulates the virtual actor model in real time, generates and saves the corresponding action simulation data.

9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; Wherein, the processor is configured to implement the application method described in claim 8 when executing executable instructions.

Images