Interaction control method and device based on meta universe, and storage medium

Abstract

The invention provides an interaction control method and device based on meta universe, and a storage medium. The method comprises the steps of performing data monitoring on a target entity, and dynamically obtaining data information of the target entity; constructing a digital twin model according to the data information of the target entity; the data processing model constructs a target virtual model in the meta universe according to the digital twinborn model; in response to a target instruction triggered by a user behavior to a target virtual model in the meta universe, enabling the data processing model to control the target entity; and changing the target virtual model in the meta universe according to the data information of the target entity. According to the invention, deep interaction with the physical world based on the meta universe is realized.

Description

Technical Field

[0001] The technical field of this application is the field of cloud universe, and specifically relates to an interactive control method, device, and storage medium based on the metaverse. Background Art

[0002] Currently, the metaverse is mostly used for users to have immersive virtual experiences and has no impact on the physical world. This shows that the development and application of the metaverse is at a low level.

[0003] Based on the highly immersive experience of the metaverse, if the metaverse can interact with the physical world, users can experience the target entities in the physical world in the metaverse, and then better manipulate the target entities in the physical world, increasing the user's open experience and reducing the trial-and-error cost of manipulating the target entities.

[0004] Therefore, how to achieve deep interaction between the metaverse and the physical world is a technical problem that needs to be solved urgently. Summary of the invention

[0005] The present application aims to solve the technical problem of how to efficiently and conveniently obtain a processing path for a workpiece.

[0006] According to one aspect of an embodiment of the present application, the present application provides an interactive control method based on a metaverse, the method comprising:

[0007] Monitor the data of the target entity and dynamically obtain the data information of the target entity;

[0008] Building a digital twin model according to the data information of the target entity;

[0009] The data processing model constructs a target virtual model in the metaverse according to the digital twin model;

[0010] In response to a target instruction triggered by a user behavior on a target virtual model in the metaverse, the data processing model manipulates the target entity;

[0011] According to the data information of the target entity, the target virtual model is changed in the metaverse.

[0012] According to one aspect of an embodiment of the present application, in response to a target instruction triggered by a user behavior on a target virtual model in the metaverse, causing the data processing model to manipulate the target entity includes:

[0013] The data processing model provides a plurality of first decision suggestions to the user in the metaverse according to the target instruction and the data information of the target entity;

[0014] In response to the user behavior, selecting at least one first decision suggestion from a plurality of first decision suggestions as a first instruction;

[0015] The data processing model demonstrates in the metaverse a process in which the target virtual model executes the first instruction according to the first instruction and the data information of the target entity;

[0016] generating a second instruction in response to feedback formed by the user's behavior;

[0017] According to the second instruction, the data processing model is enabled to manipulate the target entity.

[0018] According to one aspect of an embodiment of the present application, according to the second instruction, causing the data processing model to manipulate the target entity includes:

[0019] Controlling the digital twin model to execute a second instruction, and obtaining execution data of the digital twin model executing the second instruction;

[0020] The data processing model generates a plurality of second decision suggestions in the metaverse according to the execution data of the digital twin model;

[0021] In response to the user behavior, selecting at least one second decision suggestion from a plurality of the second decision suggestions as a third instruction;

[0022] The data processing model controls the target virtual model to demonstrate the execution process of the third instruction in the metaverse;

[0023] If positive feedback is obtained in response to the user behavior, the data processing model is caused to manipulate the target entity according to the third instruction.

[0024] According to one aspect of an embodiment of the present application, after the data processing model controls the target virtual model to demonstrate the execution process of the third instruction in the metaverse, the method further includes:

[0025] If negative feedback is obtained in response to the user behavior, the data processing model generates a plurality of third decision suggestions in the metaverse according to the negative feedback;

[0026] In response to the user behavior, select at least one of the third decision suggestions from the plurality of third decision suggestions as a fourth instruction;

[0027] The data processing model is caused to manipulate the target entity according to the fourth instruction.

[0028] According to one aspect of an embodiment of the present application, in response to a target instruction triggered by a user behavior on a target virtual model in the metaverse, causing the data processing model to manipulate the target entity, further comprising:

[0029] If the data processing model finds an emergency event in the data information of the target entity, a plurality of fourth decision suggestions are generated in the metaverse according to the emergency event;

[0030] In response to user behavior, at least one fourth decision suggestion is selected from the plurality of fourth decision suggestions as a fifth instruction, so that the data processing model controls the target entity to execute the fifth instruction.

[0031] According to one aspect of an embodiment of the present application, when the data processing model finds an emergency event in the data information of the target entity, before generating a plurality of fourth decision suggestions in the metaverse according to the emergency event, the method further includes:

[0032] Calculate the operation status of the digital twin model in a set period after the current moment according to the data information of the target entity at the current moment;

[0033] If a set event occurs in the running state of the digital twin model in the set time period after the current moment, it is considered that there is an emergency event in the data information of the target entity.

[0034] According to one aspect of the embodiment of the present application, the method further includes:

[0035] According to the user's selection record of decision suggestions and the generation scenario of the decision suggestions, the decision suggestion generation model provided in the data processing model is trained so that the decision suggestions generated by the data processing model are more in line with the user's operating habits.

[0036] According to one aspect of an embodiment of the present application, the present application provides an interactive control device based on a metaverse, the device comprising:

[0037] A digital twin module is configured to obtain data information of a target entity and establish a digital twin model;

[0038] The metaverse module is configured to construct a corresponding metaverse scenario based on the data information of the target entity and the digital twin model;

[0039] The data analysis module is configured to install a data processing model to process data information generated from the metaverse, digital twin model, and target entity, and generate corresponding decision recommendations.

[0040] According to one aspect of an embodiment of the present application, the present application provides a readable storage medium having a readable program / instruction stored thereon. When the readable program / instruction is executed by a processor, any of the methods described above is implemented.

[0041] According to one aspect of an embodiment of the present application, the present application provides a program product, including a readable program / instruction, and when the readable program / instruction is executed by a processor, the method as described in any one of the above items is implemented.

[0042] This application dynamically obtains the data information of the target entity by monitoring the data of the target entity; constructs a digital twin model according to the data information of the target entity; the data processing model constructs a target virtual model in the metaverse according to the digital twin model; responds to the target instruction triggered by the user behavior on the target virtual model in the metaverse, so that the data processing model manipulates the target entity; according to the data information of the target entity, the target virtual model is changed in the metaverse. A high-precision digital twin model is constructed through the data information of the target entity, and the data information is updated in real time to reflect the real state changes of the target entity. The data processing model receives the digital twin model data and the instructions generated by the user behavior, and deeply analyzes and understands the data. Based on the processing results of the data processing model, the metaverse constructs a target virtual model corresponding to the digital twin model in the metaverse space. The target entity receives the instructions processed by the digital processing model and runs according to the instructions. Users can interact immersively with the target virtual model in the metaverse, and then interact with the digital twin model, thereby realizing remote control of the target entity and exploring the potential application scenarios of the target entity in the virtual metaverse.

[0043] Other features and advantages of the present application will become apparent from the following detailed description, or may be learned in part by the practice of the present application.

[0044] It should be understood that the foregoing general description and the following detailed description are exemplary only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The above and other objects, features and advantages of the present application will become more apparent by describing in detail exemplary embodiments thereof with reference to the attached drawings.

[0046] Figure 1 A flow chart of an interactive control method based on a metaverse according to an embodiment of the present application is shown.

[0047] Figure 2 A flow chart is shown of a method for triggering a target instruction on a target virtual model in the metaverse in response to user behavior according to an embodiment of the present application, so that a data processing model can manipulate a target entity.

[0048] Figure 3 A flow chart is shown for enabling the data processing model to manipulate the target entity according to the second instruction.

[0049] Figure 4A flowchart of modifying the third instruction according to an embodiment of the present application is shown.

[0050] Figure 5 A flowchart is shown of another embodiment of the present application, which responds to a user behavior to trigger a target instruction on a target virtual model in the metaverse, so that the data processing model manipulates the target entity.

[0051] Figure 6 A flow chart for determining an emergency event according to an embodiment of the present application is shown.

[0052] Figure 7 A schematic diagram of an interactive control device based on a metaverse according to an embodiment of the present application is shown.

[0053] Figure 8 A hardware structure diagram for implementing an interactive control method based on a metaverse according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0054] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these example embodiments are provided so that the description of the present application will be more comprehensive and complete and the concepts of the example embodiments will be fully conveyed to those skilled in the art. The accompanying drawings are only schematic illustrations of the present application and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and thus their repeated description will be omitted.

[0055] In addition, the described features, structures or characteristics may be combined in one or more example embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the example embodiments of the present application. However, those skilled in the art will appreciate that the technical solution of the present application may be practiced while omitting one or more of the specific details, or other methods, components, steps, etc. may be adopted. In other cases, known structures, methods, implementations or operations are not shown or described in detail to avoid obscuring the present application and making the various aspects of the present application obscure.

[0056] Some of the blocks shown in the accompanying drawings are functional entities that do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0057] See also Figure 1 , Figure 1A flowchart of an interactive control method based on a metaverse according to an embodiment of the present application is shown. The present application embodiment provides a method for interactive control based on a metaverse, including:

[0058] Step S110, performing data monitoring on the target entity and dynamically acquiring data information of the target entity;

[0059] Step S120, constructing a digital twin model according to the data information of the target entity;

[0060] Step S130, the data processing model constructs a target virtual model in the metaverse according to the digital twin model;

[0061] Step S140, responding to the target instruction triggered by the user behavior on the target virtual model in the metaverse, so that the data processing model manipulates the target entity;

[0062] Step S150, changing the target virtual model in the metaverse according to the data information of the target entity.

[0063] The above five steps are described in detail below.

[0064] In step S110, data monitoring is performed on the target entity to obtain data information of the target entity. First of all, it should be made clear that the data information of the target entity includes but is not limited to the appearance structure, operating parameters, and environmental information. For different target entities, the data information of the target entity has different content or emphasis. For example, if the target entity is a mechanical device or a hardware structure, the data information of the target entity in this case focuses on the appearance structure and operating parameters. If the target entity is a data processing device, such as a signal base station, the data information of the target entity in this case focuses on the operating parameters. If the target entity is a sports device, such as a car, the data information of the target entity in this case focuses on the operating parameters and environmental information of the sports device, supplemented by the appearance structure.

[0065] It should be further explained that data monitoring of the target entity can be performed by setting up a sensing device or a monitoring device (for obtaining the appearance structure and environmental information of the target entity) and a data monitoring device (for obtaining the operating parameters of the target entity).

[0066] In step S120, a digital twin model is constructed based on the data information of the target entity. A digital twin (Digital Twin) is a virtual, digital entity, which can also be called digital bionics. It corresponds to the target entity in the physical world. The target entity in the physical world is modeled and simulated to achieve real-time monitoring, prediction and optimization of the physical world.

[0067] The digital twin model enables monitoring and control of the target entity. For example, the digital twin can monitor the operating status and / or environmental information of the target entity in real time, predict equipment failures and maintenance needs, and achieve remote control and optimization of the equipment.

[0068] If the data information of the target entity includes environmental information, a digital twin model of the corresponding environmental data will also be constructed.

[0069] In step S130, the data processing model constructs a target virtual model in the metaverse by acquiring information from the digital twin model.

[0070] In the embodiment of the present application, compared with the data information of the target entity, the information of the digital twin model has been processed by the digital twin technology, and has been modeled and is relatively digital. Because the data processing model directly establishes the target virtual model in the metaverse based on the information of the digital twin model. It is more efficient and convenient, and avoids the waste of computing resources.

[0071] In step S140, in response to the target instruction triggered by the user behavior on the target virtual model in the metaverse, the data processing model manipulates the target entity according to the target instruction triggered in the metaverse, and maps the target instruction triggered by the user on the target virtual model in the metaverse to the target entity in the physical world. In other words, the interaction of the user on the target virtual model in the metaverse is mapped to the target entity in the physical world. The interaction between the metaverse and the target entity in the physical world is realized.

[0072] In some embodiments, an interactive panel is constructed in the metaverse to facilitate the manipulation of the target virtual model, or to directly form target instructions based on the user's operating behavior to interact or manipulate the physical world.

[0073] In some embodiments, a user can enter the metaverse through a terminal device, manipulate the target virtual model, and then generate corresponding target instructions based on the user's manipulation behavior or changes in the structure or operating status of the target virtual model.

[0074] In some implementations, users can also issue target instructions directly in the metaverse without going through the target virtual model.

[0075] See also Figure 2 , Figure 2 A flowchart is shown of a method for causing a data processing model to manipulate a target entity in response to a target instruction triggered by a user behavior on a target virtual model in the metaverse according to an embodiment of the present application. The present application embodiment provides a step S140 of causing a data processing model to manipulate a target entity in response to a target instruction triggered by a user behavior on a target virtual model in the metaverse, including:

[0076] Step S141a, the data processing model provides a plurality of first decision suggestions to the user in the metaverse according to the target instruction and the data information of the target entity;

[0077] Step S142a, selecting at least one first decision suggestion from a plurality of first decision suggestions in response to a user behavior as a first instruction;

[0078] Step S143a, the data processing model demonstrates the process of the target virtual model executing the first instruction in the metaverse according to the first instruction and the data information of the target entity;

[0079] Step S144a, generating a second instruction in response to feedback formed by the user's behavior;

[0080] Step S145a, according to the second instruction, the data processing model manipulates the target entity.

[0081] The above five steps are described in detail below.

[0082] In step S141a, the data processing model performs a deep analysis of the target instruction based on the content of the target instruction and the data information of the target entity, and then can predict the result after the target entity executes the target instruction, and then generate multiple first decision suggestions based on the prediction results. It should be clear that each time the data processing model performs a deep analysis of the target instruction, it needs to obtain the latest data information of the target entity.

[0083] For example, the data processing model predicts, based on the content of the target instruction and the data information of the target entity, that if the target instruction is executed, a parameter of the target entity will be abnormal. For the abnormal parameter, the data processing model will make different degrees of changes to the target instruction, generate several first decision suggestions, and display them in the metaverse for the user to choose, so as to avoid the predicted data anomaly in the target entity.

[0084] In step S142a, at least one first decision suggestion is selected as a first instruction from among multiple first decision suggestions in response to the user behavior.

[0085] In step S143a, the data processing model demonstrates the process of executing the first instruction of the target virtual model in the metaverse according to the first instruction and the data information of the target entity. This process is equivalent to allowing the user to preview the process or result of executing the first instruction in advance. And obtain the user's feedback to determine whether the first instruction needs to be further executed or needs to be further modified.

[0086] It should be clear that in the embodiment of the present application, "demonstration" is only a preview and does not cause permanent changes to the target virtual model. After the "demonstration" is completed, the target virtual model will automatically return to the "before demonstration" state.

[0087] In step S144a, a second instruction is generated in response to the feedback formed by the user behavior. If the user behavior indicates positive feedback, the first instruction is used as the second instruction. If the user behavior indicates negative feedback, the data processing model adjusts the first instruction according to the feedback to generate a second instruction.

[0088] In step S145a, in response to the second instruction, the data processing model manipulates the target entity.

[0089] In the embodiment of the present application, the user interacts with the data processing model multiple times based on the metaverse, and analyzes the target instruction based on the future running state of the target entity predicted by the data processing model to avoid some accidents when the target entity is running. By increasing the number of interactions between the data processing model and the user, the fault tolerance of the interaction between the user and the target entity is greatly increased.

[0090] See also Figure 3 , Figure 3 The flowchart of making the data processing model manipulate the target entity according to the second instruction is shown. The embodiment of the present application provides a step S145 of making the data processing model manipulate the target entity according to the second instruction, including:

[0091] Step S201, controlling the digital twin model to execute the second instruction, and obtaining execution data of the digital twin model executing the second instruction;

[0092] Step S202, the data processing model generates a plurality of second decision suggestions in the metaverse according to the execution data of the digital twin model;

[0093] Step S203, selecting at least one second decision suggestion from a plurality of second decision suggestions as a third instruction in response to the user behavior;

[0094] Step S204, the data processing model controls the target virtual model to demonstrate the execution process of the third instruction in the metaverse;

[0095] Step S205: If positive feedback is obtained in response to the user behavior, the data processing model is enabled to manipulate the target entity according to the third instruction.

[0096] The above five steps are described in detail below.

[0097] In step S201, after determining the second instruction, the data processing model inputs the second instruction into the digital twin model for operation to simulate the state of the target entity when the second instruction is executed. Since the digital twin model is a digital bionic body of the target entity, the digital twin model is more capable of reflecting the state of the target entity. For this reason, the trial operation of the second instruction in the digital twin model can better simulate the state of the target entity when the second instruction is executed.

[0098] The execution data of the digital twin model when running the second instruction is recorded, and the data analysis model analyzes the execution data, such as calculating how to change the second instruction to achieve better running effect or running speed, or avoiding the content in the second instruction that causes the digital twin model to produce abnormalities. In this way, the optimization plan or possible problems when the target entity runs the second instruction are predicted, and the content of the second instruction is optimized to achieve better results when the target entity runs.

[0099] It should be made clear that the digital twin model is updated in real time based on the data information of the target entity. When the second instruction is executed, the digital twin model has been updated based on the latest data information of the target entity.

[0100] In step S202, the data analysis model generates a plurality of second decision suggestions in the metaverse by analyzing the execution data for the user to select.

[0101] In step S203, in response to the user behavior, at least one second decision suggestion is selected from multiple second decision suggestions as a third instruction.

[0102] In step S204, the data processing model controls the target virtual model to demonstrate the execution process of the third instruction in the metaverse, and previews the execution process of the third instruction of the target entity.

[0103] It should be clear that in the embodiment of the present application, "demonstration" is only a preview and does not cause permanent changes to the target virtual model. After the "demonstration" is completed, the target virtual model will automatically return to the "before demonstration" state.

[0104] In step S205, if positive feedback is obtained in response to the user behavior, the data processing model is caused to manipulate the target entity according to the third instruction.

[0105] In the embodiment of the present application, for the target instruction issued by the user, the data processing model is first adjusted according to the inference result, and then adjusted according to the execution data of the digital twin model. Through the two adjustments, the user's target instruction is modified multiple times to obtain the third instruction. While ensuring that the user interacts with the target entity through the metaverse, the technical effect of accurately controlling the target entity based on the user's target instruction is achieved.

[0106] See also Figure 4 , Figure 4 A flowchart of modifying a third instruction according to an embodiment of the present application is shown. The embodiment of the present application provides steps of modifying a third instruction, including:

[0107] Step S301, if negative feedback is obtained in response to the user behavior, the data processing model generates a plurality of third decision suggestions in the metaverse according to the negative feedback;

[0108] Step S302, selecting at least one third decision suggestion from a plurality of third decision suggestions in response to a user behavior as a fourth instruction;

[0109] Step S303, enabling the data processing model to manipulate the target entity according to the fourth instruction.

[0110] The above three steps are described in detail below.

[0111] In step S301, after the data processing model controls the target virtual model to demonstrate the third instruction execution process in the metaverse, if negative feedback is obtained in response to the user behavior, the data processing model generates multiple third decision suggestions in the metaverse based on the user's negative feedback to resolve the user's negative feedback to varying degrees.

[0112] In step S302, in response to the user behavior, at least one third decision suggestion is selected from a plurality of third decision suggestions as a fourth instruction;

[0113] In step S303, the data processing model is enabled to manipulate the target entity according to the fourth instruction.

[0114] In the embodiment of the present application, based on multiple interactions between the metaverse and the user, the instructions executed by the target entity are all multiply polished and considered, so that the operation effect of the target entity is better.

[0115] See also Figure 5 , Figure 5 A flowchart is shown of another embodiment of the present application, in response to a target instruction triggered by a user behavior on a target virtual model in the metaverse, so that the data processing model manipulates the target entity. The present application embodiment provides a step S140 of causing the data processing model to manipulate the target entity in response to a target instruction triggered by a user behavior on a target virtual model in the metaverse, including:

[0116] Step S141b, if the data processing model finds an emergency event in the data information of the target entity, a plurality of fourth decision suggestions are generated in the metaverse according to the emergency event;

[0117] Step S142b, in response to the user behavior, select at least one fourth decision suggestion from multiple fourth decision suggestions as the fifth instruction, so that the data processing model controls the target entity to execute the fifth instruction.

[0118] The above two steps are described in detail below.

[0119] In step S141b, since the data processing model obtains the data information of the target entity in real time through the digital twin model and can deduce the target entity for a period of time in the future, when the target entity is about to fail or have an accident, the data processing model can predict and know in advance the failure or accident that will occur in the target entity.

[0120] In some embodiments, the data processing model discovers an emergency event through the data information of the target entity, that is, the data processing model predicts that the target entity will have a failure or accident based on the data information of the target entity, and then generates multiple fourth decision suggestions to avoid the upcoming accident or failure from different angles.

[0121] See also Figure 6 , Figure 6 A flow chart of determining an emergency event according to an embodiment of the present application is shown. The present application embodiment provides steps for determining an emergency event, including:

[0122] Step S401, calculating the operating state of the digital twin model in a set period after the current moment according to the data information of the target entity at the current moment;

[0123] Step S402: If a set event occurs in the running state of the digital twin model in the set time period after the current moment, it is considered that there is an emergency event in the data information of the target entity.

[0124] The above two steps are described in detail below.

[0125] In step S401, the data information of the target entity at the current moment is obtained, the data information of the data processing model at the current moment is input into the digital twin model, and the operation of the digital twin model is accelerated to predict the operation process of the target entity in the set time period after the current moment.

[0126] In step S402, if a set event occurs in the operation state of the digital twin model in the set time period after the current moment, it is considered that an emergency event exists in the data information of the target entity.

[0127] In step S142b, in response to the user behavior, at least one fourth decision suggestion is selected from multiple fourth decision suggestions as the sixth instruction, so that the data processing model controls the target entity to execute the fifth instruction, so that the target entity can avoid the accident or failure that comes in advance.

[0128] For example, a moving vehicle is taken as the target entity, and the data information of the target entity includes the driving environment and operating parameters of the vehicle. When an obstacle appears in front of the vehicle, it is known by the data processing model. After analysis, it is known that if the vehicle continues to operate according to the current operating parameters, a traffic accident will occur. The fourth decision suggestion of deceleration, parking, lane change, etc. is proposed.

[0129] It should be noted that if no user feedback can be received to determine the fifth instruction, then after the set time, the fourth decision suggestion with the highest priority will be used as the fifth instruction.

[0130] In this way, impending accidents or failures of the target entity can be automatically avoided in advance.

[0131] In step S150, the user's manipulation of the target virtual model in the metaverse is only a demonstration. The structure or operating status information of the target virtual model will be restored to the state before the demonstration after the demonstration is completed, that is, the structure or operating status of the target virtual model depends on the data information of the target entity. Therefore, the data processing model needs to manipulate the target entity and update the changes of the target entity to the digital twin model and the target virtual model according to the changes of the target entity data information.

[0132] In some embodiments, the first decision suggestion, the second decision suggestion, the third decision suggestion, and the fourth decision suggestion are referred to as decision suggestions below. According to the user's selection record of the decision suggestion and the generation scenario of the decision suggestion, the decision suggestion generation model set in the data processing model is trained to make the decision suggestion generated by the data processing model more in line with the user's operating habits.

[0133] In some embodiments, data monitoring is performed on the target entity, data information of the target entity is dynamically obtained, and a digital twin model is constructed according to the data information of the target entity. The data processing model constructs a target virtual model in the metaverse according to the digital twin model, and the data processing model provides multiple first decision suggestions to the user in the metaverse according to the target instruction and the data information of the target entity. In response to user behavior, at least one first decision suggestion is selected as the first instruction from multiple first decision suggestions, and the data processing model demonstrates the process of the target virtual model executing the first instruction in the metaverse according to the first instruction and the data information of the target entity. In response to the feedback formed by the user behavior, a second instruction is generated, the digital twin model is controlled to execute the second instruction, and the execution data of the digital twin model executing the second instruction is obtained. The data processing model generates multiple second decision suggestions in the metaverse according to the execution data of the digital twin model, and selects at least one second decision suggestion as the third instruction from multiple second decision suggestions in response to the user behavior. The data processing model controls the target virtual model to demonstrate the execution process of the third instruction in the metaverse, and if positive feedback is obtained in response to the user behavior, the data processing model is caused to manipulate the target entity according to the third instruction. If negative feedback is obtained in response to user behavior, the data processing model generates multiple third decision suggestions in the metaverse based on the negative feedback. If negative feedback is obtained in response to user behavior, the data processing model generates multiple third decision suggestions in the metaverse based on the negative feedback, so that the data processing model manipulates the target entity according to the fourth instruction, and changes the target virtual model in the metaverse according to the data information of the target entity.

[0134] In another embodiment, based on the data information of the target entity at the current moment, the running state of the digital twin model in a set period after the current moment is calculated. If a set event occurs in the running state of the digital twin model in the set period after the current moment, it is considered that there is an emergency event in the data information of the target entity. If the data processing model finds an emergency event in the data information of the target entity, multiple fourth decision suggestions are generated in the metaverse according to the emergency event, and at least one fourth decision suggestion is selected as the fifth instruction from the multiple fourth decision suggestions in response to user behavior, so that the data processing model controls the target entity to execute the fifth instruction.

[0135] See also Figure 7 , Figure 7 A schematic diagram of an interactive control device based on a metaverse according to an embodiment of the present application is shown. The interactive control device based on a metaverse includes:

[0136] A digital twin module is configured to obtain data information of a target entity and establish a digital twin model;

[0137] The metaverse module is configured to construct a corresponding metaverse scenario based on the data information of the target entity and the digital twin model;

[0138] The data analysis module is configured to install a data processing model to process data information generated from the metaverse, digital twin model, and target entity, and generate corresponding decision recommendations.

[0139] Among them, the metaverse module interacts with the user in a two-way manner, the data analysis module interacts with the metaverse module in a two-way manner, the data analysis module interacts with the digital twin module in a two-way manner, and the digital twin module interacts with the target entity in a two-way manner.

[0140] In this application, the digital twin module can comprehensively collect various data such as the appearance structure, operating parameters, environmental information, etc. of the target entity, and build a high-precision digital twin model. It can also accurately and timely reflect the real state changes of the target entity in real time, which enables subsequent interactions to be based on the latest and accurate data information of the target entity, which is different from some existing technologies that may have untimely data updates or incomplete data collection, thus affecting the accuracy of interaction.

[0141] The data processing model uses the powerful capabilities of the big model to deeply analyze and understand the received digital twin model data and user interaction instructions. It can not only predict the future operation trend of the target entity and help users make corresponding plans and decisions in advance, but also generate personalized interaction strategies based on different user needs. Compared with the existing technology, which often only conducts conventional and general data analysis and lacks deep mining and personalized customization capabilities, it adds higher intelligence and pertinence to the interaction.

[0142] Based on the processing results of the large model, Metaverse constructs corresponding virtual scenes and interactive interfaces in the Metaverse space, allowing users to interact immersively with the digital twin model through terminals such as virtual reality devices. This interactive method breaks through the limitations of traditional interactions that are limited to flat displays or simple three-dimensional visualizations, and realizes innovative interactive applications such as remote control of virtual target entities and exploration of their potential application scenarios, providing users with a more immersive and richer interactive experience, greatly expanding the depth and breadth of interaction, which is an advantage that most existing interactive technologies cannot achieve.

[0143] The entire intelligent interaction system organically integrates the digital twin module, data processing model and metaverse, forming an integrated process from target entity data collection, intelligent analysis to metaverse immersive interaction. The modules cooperate and work together to avoid the possible separation of various links in the existing technology, effectively improve the overall efficiency and coherence of the interaction, and create a more smooth, efficient and fully functional interaction solution for users.

[0144] The interactive control method based on the metaverse in the embodiment of the present application can be performed by Figure 8 The hardware device shown is implemented. Figure 8To describe the hardware device according to the embodiment of the present application. Figure 8 The hardware device shown is only an example and should not bring any limitation to the functions and scope of application of the embodiments of the present application.

[0145] like Figure 8 As shown, the hardware device may be in the form of a general computing device. The components of the hardware device may include, but are not limited to: at least one processing unit 810, at least one storage unit 820, and a bus 830 connecting different system components (including the storage unit 820 and the processing unit 810).

[0146] The storage unit stores program codes, which can be executed by the processing unit 810, so that the processing unit 810 performs the steps according to various exemplary embodiments of the present invention described in the description of the exemplary method described above in this specification. For example, the processing unit 810 can perform the following steps: Figure 1 The steps shown in .

[0147] The storage unit 820 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 8201 and / or a cache memory unit 8202 , and may further include a read-only memory unit (ROM) 8203 .

[0148] The storage unit 820 may also include a program / utility 8204 having a set (at least one) of program modules 8205, such program modules 8205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0149] Bus 830 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0150] The hardware device may also communicate with one or more external devices 700 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may communicate with one or more devices that enable a user to interact with the hardware device, and / or may communicate with any device that enables the hardware device to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed through an input / output (I / O) interface 850. Furthermore, the hardware device may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 860. As shown, the network adapter 860 communicates with other modules of the hydrogen refueling device through a bus 830. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the hardware device, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0151] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the implementation methods of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the implementation methods of the present application.

[0152] In an exemplary embodiment of the present application, a computer program medium is further provided, on which computer-readable instructions are stored. When the computer-readable instructions are executed by a processor of a computer, the computer is caused to execute the method described in the above method embodiment.

[0153] According to one embodiment of the present application, a program product for implementing the method in the above method embodiment is also provided, which can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited to this. In this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, an apparatus or a device.

[0154] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0155] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0156] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0157] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0158] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0159] In addition, although the steps of the method in the present application are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.

[0160] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the implementation methods of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the implementation methods of the present application.

[0161] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the inventions applied for herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that the present application does not apply for. The specification and embodiments are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the appended claims.

Claims

1. An interactive control method based on the metaverse, characterized in that: The method comprises: Monitor the data of the target entity and dynamically obtain the data information of the target entity; Building a digital twin model based on the data information of the target entity; The data processing model constructs a target virtual model in the metaverse according to the digital twin model; In response to a target instruction triggered by a user behavior on a target virtual model in the metaverse, the data processing model manipulates the target entity; According to the data information of the target entity, the target virtual model is changed in the metaverse.

2. The method according to claim 1, characterized in that Responding to a target instruction triggered by a user behavior on a target virtual model in the metaverse, causing the data processing model to manipulate the target entity, including: The data processing model provides a plurality of first decision suggestions to the user in the metaverse according to the target instruction and the data information of the target entity; In response to the user behavior, selecting at least one first decision suggestion from a plurality of first decision suggestions as a first instruction; The data processing model demonstrates in the metaverse a process in which the target virtual model executes the first instruction according to the first instruction and the data information of the target entity; generating a second instruction in response to feedback formed by the user's behavior; According to the second instruction, the data processing model is enabled to manipulate the target entity.

3. The method according to claim 2, characterized in that According to the second instruction, causing the data processing model to manipulate the target entity includes: Controlling the digital twin model to execute a second instruction, and obtaining execution data of the digital twin model executing the second instruction; The data processing model generates a plurality of second decision suggestions in the metaverse according to the execution data of the digital twin model; In response to the user behavior, selecting at least one second decision suggestion from a plurality of the second decision suggestions as a third instruction; The data processing model controls the target virtual model to demonstrate the execution process of the third instruction in the metaverse; If positive feedback is obtained in response to the user behavior, the data processing model is caused to manipulate the target entity according to the third instruction.

4. The method according to claim 3, characterized in that After the data processing model controls the target virtual model to demonstrate the execution process of the third instruction in the metaverse, the method further includes: If negative feedback is obtained in response to the user behavior, the data processing model generates a plurality of third decision suggestions in the metaverse according to the negative feedback; In response to the user behavior, select at least one of the third decision suggestions from the plurality of third decision suggestions as a fourth instruction; The data processing model is caused to manipulate the target entity according to the fourth instruction.

5. The method according to claim 1, characterized in that In response to a target instruction triggered by a user behavior on a target virtual model in the metaverse, causing the data processing model to manipulate the target entity, the method further includes: If the data processing model finds an emergency event in the data information of the target entity, a plurality of fourth decision suggestions are generated in the metaverse according to the emergency event; In response to user behavior, at least one fourth decision suggestion is selected from the plurality of fourth decision suggestions as a fifth instruction, so that the data processing model controls the target entity to execute the fifth instruction.

6. The method according to claim 5, characterized in that When the data processing model finds an emergency event in the data information of the target entity, before generating a plurality of fourth decision suggestions in the metaverse according to the emergency event, the method further includes: Calculate the operation status of the digital twin model in a set period after the current moment according to the data information of the target entity at the current moment; If a set event occurs in the running state of the digital twin model in the set time period after the current moment, it is considered that there is an emergency event in the data information of the target entity.

7. The method according to claim 1, characterized in that The method further comprises: According to the user's selection record of decision suggestions and the generation scenario of the decision suggestions, the decision suggestion generation model provided in the data processing model is trained so that the decision suggestions generated by the data processing model are more in line with the user's operating habits.

8. An interactive control device based on the metaverse, characterized in that: The device comprises: A digital twin module is configured to obtain data information of a target entity and establish a digital twin model; The metaverse module is configured to construct a corresponding metaverse scenario based on the data information of the target entity and the digital twin model; The data analysis module is configured to install a data processing model to process data information generated from the metaverse, digital twin model, and target entity, and generate corresponding decision recommendations.

9. A readable storage medium, characterized in that: A readable program / instruction is stored thereon, and when the readable program / instruction is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. A program product, comprising a readable program / instruction, characterized in that: When the readable program / instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images