Implementation method and device of twin multi-modal representation

Abstract

The application relates to a method and device for realizing twin body multi-modal representation, and the method comprises the following steps: improving a model node of an entity model into a twin body object; generating corresponding rendering data according to business requirements; driving business data by accessing the rendering data; and constructing display logic for rendering the model node according to business requirements. The application solves the technical problem of how to realize efficient fusion and display of multi-modal information of a digital twin body in different business scenarios, and realizes intelligent visual innovation of "one twin and multiple modes".

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, and in particular to a method and device for realizing multi-modal representation of a twin. BACKGROUND

[0002] In actual business scenarios, the same twin may generate different modal representation requirements due to different business needs. For example, in a storage station scenario, the same storage cabinet needs to be represented in the form of a temperature field under temperature management business, in the form of a solid model under asset management business, and in the form of an energy flow under power management business.

[0003] Currently, the mainstream processing method for multi-modal representation of a twin is to maintain a solid model and process it by transforming model maps, colors, or adding a top card according to business. Although the above processing methods can realize multi-modal representation of a twin to some extent, there are still great deficiencies. First, modifying model maps and colors cannot effectively represent business information. Although the method of adding a top card and attaching text to the top card can display business data, it cannot significantly express the particularity of a twin in different modalities. Second, whether modifying textures and colors or adding a top card will destroy the unity of the scene style. SUMMARY

[0004] The technical problem to be solved by the present application is to provide a method and device for realizing multi-modal representation of a twin to solve the deficiencies of the prior art and realize intelligent visual display of multiple different business scenarios.

[0005] The technical solution of the present application to solve the above technical problem is as follows:

[0006] In a first aspect, a method for realizing multi-modal representation of a twin is provided, comprising:

[0007] promoting a model node of a solid model to a twin object;

[0008] generating corresponding rendering data according to business needs;

[0009] driving the rendering data by inputting business data;

[0010] constructing display logic for rendering the model node according to business needs.

[0011] Optionally, promoting the model node of the solid model to the twin object comprises:

[0012] creating a sub-twin object, and setting a parent object of the sub-twin object as the twin object;

[0013] According to the model node name of the entity model, find the corresponding mesh information in the twin body object; split the mesh information from the twin body object as a rendering node of the sub-twin body object;

[0014] Sort the twin body object to remove the remaining part of the mesh information, and take it as a rendering node of the twin body object.

[0015] Optionally, the generating of the corresponding rendering data according to the business requirement comprises:

[0016] According to the business requirement, the rendering data of the twin body object and the rendering data of the sub-twin body object are respectively created;

[0017] For each rendering data, a root node is established, and other data is taken as a child node of the node, so that the rendering tree of the twin body object and the sub-twin body object for the business requirement is obtained;

[0018] Mount the rendering tree.

[0019] Optionally, the mounting of the rendering tree comprises:

[0020] A rendering root node is respectively created for the twin body object and the sub-twin body object;

[0021] The root nodes of the rendering trees of the twin body object and the sub-twin body object are respectively mounted to the respective rendering root nodes as child nodes and named;

[0022] The rendering data of the twin body object and the rendering data of the sub-twin body object are mounted to the rendering root nodes and named.

[0023] Optionally, the driving of the rendering data by the business data comprises:

[0024] When the business data changes cause the data to undergo a simple transformation, the matrix information of the child node is updated;

[0025] When the business data changes cause the rendering data to undergo a non-simple transformation, a new rendering tree is generated to replace the mounted node.

[0026] Optionally, the simple transformation comprises translation, rotation, and scaling.

[0027] Optionally, the constructing of the display logic for rendering the model node according to the business requirement comprises:

[0028] For the current mode, the nodes under the rendering root node are set to be displayed or hidden;

[0029] Before the level switching of the twin body object, the current mode of the twin body object is recorded, and after the level switching is completed, the mode information is updated.

[0030] In a second aspect, an implementation device for twin multi-modal representation is provided, comprising:

[0031] a model node promotion module configured to promote a model node of an entity model to a twin object;

[0032] a rendering data generation module configured to generate corresponding rendering data according to a business requirement;

[0033] a business data driving module configured to drive the rendering data by accessing business data;

[0034] a display logic processing module configured to construct a display logic for rendering the model node according to the business requirement.

[0035] In a third aspect, a computer storage medium is provided, which stores a computer program, and the program is executed by a processor to implement the steps of the implementation method for twin multi-modal representation according to any one of the first aspect.

[0036] In a fourth aspect, a computer program product is provided, which comprises a computer program or instructions, and the computer program or instructions are executed by a processor to implement the steps of the implementation method for twin multi-modal representation according to the first aspect.

[0037] Advantages

[0038] The present application is aimed at different business requirements, based on the physical structure of the twin object, by dynamically constructing the custom rendering data, and adding it to the rendering layer, and constructing the display logic of the node rendering according to the business requirement, and performing display management based on the business requirement, and finally realizing the multi-modal representation of the twin. The technical problem of how to realize efficient fusion and display of multi-modal information of digital twin in different business scenarios is solved, and the intelligent visualization innovation of "one twin and multi-modal" is realized.

[0039] The advantages of the additional aspects of the present application will be partially given in the following description, partially will become obvious from the following description, or will be understood by the practice of the present application. BRIEF DESCRIPTION OF DRAWINGS

[0040] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments of the present application or the prior art description. Obviously, the drawings described below are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0041] Figure 1 A flowchart of the implementation method for twin multi-modal representation described in the embodiments of the present application;

[0042] Figure 2Fig. 1 is a schematic diagram of one energy storage cabinet model example in the embodiments of the present application;

[0043] Figure 3 Fig. 2 is a schematic diagram of one display energy storage cabinet power example in the embodiments of the present application;

[0044] Figure 4 Fig. 3 is a principle block diagram of the implementation device of the twin multi-modal representation described in the embodiments of the present application. DETAILED DESCRIPTION

[0045] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should fall within the scope of protection of the present application.

[0046] The implementation principle of the twin multi-modal representation is to create custom rendering data based on the physical structure of the twin based on business requirements, dynamically display in combination with business data, add to the rendering layer, and rely on the show and hide logic based on business requirements, in combination with the unified management of entity models, to realize the multi-modal representation of the twin.

[0047] Embodiment one

[0048] As shown in Fig. 1, a method for implementing twin multi-modal representation includes the following steps: Figure 1

[0049] Step S1: promoting the model node of the entity model to be a twin object;

[0050] In the physical world, a twin object often includes several components, so although the entity model of the twin object is usually a complete model, the entity model will be composed of different model nodes. Since rendering based on business logic usually requires special display and separate show and hide of the components, the model node of the entity model needs to be promoted to a twin object.

[0051] The model node promotion process is as follows:

[0052] Step S1.1: creating a child twin object, and setting the parent object of the child twin object to be the twin object;

[0053] Specifically, a twin object subObj is created; the attribute information, style information, and matrix information of subObj are consistent with those of the twin object; and subObj sets the parent object to be the twin object.

[0054] ​Step S1.2: According to the model node name of the entity model, find the corresponding mesh information in the twin object; separate the mesh information from the twin object as the rendering node of the sub-twin object;

[0055] Specifically, according to the model node name of the entity model, find the corresponding mesh information mesh in the twin object; separate the mesh from the object model as the rendering node of subObj,

[0056] Step S1.3: Arrange the remaining part of the twin object after removing the mesh information as the rendering node of the twin object.

[0057] Specifically, arrange the remaining part of the twin object after removing the mesh as the rendering node of the twin object.

[0058] As shown in Figure 2 The model data given by the energy storage cabinet object is a complete energy storage cabinet model, which includes several battery clusters. Due to specific business requirements, different battery clusters need different display performances, so it is necessary to promote the battery cluster object to a sub-object of the energy storage cabinet object, and assign the rendering data of the battery clusters in the energy storage cabinet to the corresponding sub-objects respectively, and leave the remaining rendering data to the energy storage cabinet object.

[0059] Step S2: Generate corresponding rendering data according to business requirements;

[0060] Under different business requirements, the twin object is in different modalities and has different forms of rendering requirements. Specifically, the data required for rendering under the current business requirement is organized according to the business requirement.

[0061] The specific implementation process includes the following steps:

[0062] Step S2.1: According to the business requirement, create the rendering data of the twin object and the rendering data of the sub-twin object respectively;

[0063] Under the specified business requirement, the twin object and the sub-twin object promoted by it both have special display needs and independent display logic, so it is necessary to divide the generated rendering data to belong to the twin object or the sub-twin object.

[0064] Step S2.2: For each rendering data, establish a root node, and take other data as the child node of the node, to obtain the rendering tree of the twin object and the sub-twin object for the business requirement;

[0065] According to the business requirement, rendering data of the twin object and rendering data of the sub-twin object are created respectively. It should be noted that, due to the complexity of the business requirement, the rendering data to be created may need multiple models, top cards and text information to be jointly constructed, which requires a root node to be established for each rendering data, and other data to be taken as a child node of the node. Thus, the rendering tree of the twin object and the sub-twin object for the business requirement is finally obtained.

[0066] Step S2.3: mounting the rendering tree;

[0067] Specifically, the twin object and the sub-twin object obtain the corresponding rendering tree under the current requirement in the previous step; this step is to mount the rendering tree. Since the twin object and the sub-twin object are processed in the same way, the following will not be distinguished.

[0068] The mounting process is as follows:

[0069] Step S2.3.1: creating a rendering root node for the twin object and the sub-twin object respectively;

[0070] Specifically, a rendering root node root is created for the object;

[0071] Step S2.3.2: mounting the root node of the rendering tree of the twin object and the sub-twin object as a child node to the respective rendering root node and naming;

[0072] Specifically, the root node of the rendering tree is mounted as a child node to root and named.

[0073] Step S2.3.3: mounting the rendering data of the twin object and the rendering data of the sub-twin object to the rendering root node and naming.

[0074] Specifically, after the final rendering data of the twin object to be rendered is pointed to the remaining mesh of the twin object model except the promotion part, it is mounted to root and named.

[0075] Specific examples are explained as follows:

[0076] As Figure 3As shown, the energy storage cabinet electric quantity display needs to display the energy storage cabinet transparent shell, the energy storage cabinet business data, and the battery cluster electric quantity box information. First, the rendering is divided into rendering, wherein the transparent shell and the business data belong to the display information of the energy storage cabinet body, and the battery cluster box belongs to the display information of the battery cluster. Then, a rendering tree is drawn, taking the energy storage cabinet body as an example, a root node is first created, then a transparent box object and a top card object are created respectively, and both are added to the root node to obtain a rendering tree. Finally, the rendering tree is mounted, a rendering root node root is created on the energy storage cabinet object, and the rendering data of the energy storage cabinet body and the rendering data of the rendering tree are respectively named and added to the root.

[0077] Step S3: driving the rendering data by accessing the business data;

[0078] In a specified business scenario, the twin object data usually needs to be dynamically displayed, so the rendering data also needs to be driven based on the business data. However, since the rendering data has been added to the rendering layer, it is not convenient to respond dynamically, so we handle it in two cases. Specifically, it includes the following steps:

[0079] Step S3.1: when the rendering data changes after the business data changes, update the matrix information corresponding to the child node;

[0080] Under the rendering root node of the twin object, there is corresponding matrix information for different child nodes, which can assist the child node in transformation; simple transformation means only translation, rotation and scaling.

[0081] Step S3.2: when the rendering data changes after the business data changes, re-generate a rendering tree to replace the mounted node.

[0082] After steps S1-S3, the rendering preparation work of the twin object based on the specified business requirement is completed, and further, we need to handle the display processing logic under the specific business requirement.

[0083] Step S4: according to the business requirement, build the display logic of the rendering model node. Including:

[0084] Step S4.1: for the current modality, set the nodes under the rendering root node to be visible or invisible;

[0085] Specifically, this step belongs to multi-modality switching, since the twin body mounts the rendering tree, the rendering data nodes of various modalities under the root have been named, so when multi-modality switching is performed, only the nodes under the root need to be set to be visible or invisible for the current modality, that is, only the current modality nodes are displayed.

[0086] Step S4.2: Record the current modal of the twin object before the level switching of the twin object, and update the modal information after the completion of the level switching.

[0087] Since the level switching will also set the visibility of the twin object, it will interfere with the multi-modal of the twin object, so it is necessary to record the current modal of the twin object before the level switching, and update the modal information after the completion of the level switching. At the same time, it needs to be noted that the above processing is completed in the same frame, which can avoid the flicker problem.

[0088] Through specific examples, it is shown that the method dynamically creates a twin rendering node according to the business logic, and performs display management, thereby realizing the intelligent visualization innovation of the multi-modal representation of the twin.

[0089] Embodiment Two

[0090] The present application provides a kind of twin multi-modal representation implementation device, as shown in Figure 4 It includes:

[0091] Model node promotion module is used to promote the model node of entity model to twin object;

[0092] Rendering data generation module is used to generate corresponding rendering data according to business requirements;

[0093] Business data driving module is used to drive by accessing business data to the rendering data;

[0094] Display logic processing module is used to construct the display logic of rendering the model node according to business requirements.

[0095] The twin multi-modal representation implementation device provided in the embodiment is realized by the twin multi-modal representation implementation method provided in embodiment one, which has the same technical features as the twin multi-modal representation implementation method provided in embodiment one, so it can also solve the same technical problems and achieve the same technical effects.

[0096] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described system, device and unit can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

[0097] Embodiment Three

[0098] The present application provides a computer storage medium, which stores a computer program, and the program is executed by a processor to realize the steps of the twin multi-modal representation implementation method in any one of embodiment one.

[0099] It can be understood that the storage medium includes: flash memory, hard disk, multimedia card, card type memory (for example, SD (Secure Digital Memory Card) or DX (Memory Data Register, MDR) memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, server, APP (Application) application market and various media capable of storing program check codes.

[0100] Embodiment Four

[0101] The present application provides a computer program product, comprising a computer program or instructions, which, when executed by a processor, implements the steps in the implementation method of the twin multi-modal representation of the first aspect.

[0102] Based on such understanding, all or part of the processes in the above-mentioned embodiment methods can also be completed by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of each method embodiment described above can be implemented. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or some intermediate forms, etc. The computer-readable medium can include any entity or system, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal, and software distribution medium, etc. that can carry the computer program code. It should be noted that the contents included in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electric carrier signals and telecommunication signals.

[0103] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements for some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

[0104] The above merely provides the specific implementation of the present application, but the protection scope of the present application is not limited to this. Any person skilled in the art can easily think of various equivalent modifications or replacements within the technical range disclosed by the present application, and these modifications or replacements should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for realizing twin multi-modal representation, which is based on the physical structure of the twin, creates customized rendering data based on business needs, and dynamically displays in combination with business data, characterized in that, The method comprises the steps of: The model node of the entity model is promoted to a twin body object; According to the business requirements, corresponding rendering data is generated; The rendering data is driven by accessing the business data; According to the business requirements, the display logic of rendering the model node is constructed; The model node of the entity model is promoted to a twin body object, comprising: A child twin body object is created, and the parent object of the child twin body object is set as the twin body object; According to the model node name of the entity model, corresponding mesh information in the twin body object is found; the mesh information is segmented from the twin body object as a rendering node of the child twin body object; The remaining part of the twin body object after removing the mesh information is arranged to be a rendering node of the twin body object; According to the business requirements, corresponding rendering data is generated; According to the business requirements, rendering data of the twin body object and rendering data of the child twin body object are respectively created; For each rendering data, a root node is established, and other data is taken as a child node of the node, so that a rendering tree of the twin body object and the child twin body object for the business requirements is obtained; The rendering tree is mounted; The rendering tree is mounted, comprising: A rendering root node is respectively created for the twin body object and the child twin body object; The root nodes of the rendering trees of the twin body object and the child twin body object are respectively mounted to the respective rendering root nodes and named; The rendering data of the twin body object and the rendering data of the child twin body object are mounted to the rendering root nodes and named; When the business data changes cause simple transformation of the data, the matrix information of the child node is updated; When the business data changes cause non-simple transformation of the rendering data, a rendering tree is re-generated to replace the mounted node. The simple transformation comprises translation, rotation and scaling.

2. The method of twin-multiple modal characterization implementation according to claim 1, wherein, According to the business requirements, the display logic of rendering the model node is constructed, comprising:

3. The method of twin-multiple modal characterization implementation according to claim 1, wherein, For the current mode, the nodes under the rendering root node are set to be visible or invisible; Before the twin body object is switched in the hierarchy, the current mode of the twin body object is recorded, and after the hierarchy switching is completed, the mode information is updated. The method comprises the steps of:

4. A device for realizing twin multi-modal characterization, which is based on the physical structure of the twin, creates customized rendering data based on business requirements, and dynamically displays in combination with business data. A model node promotion module is configured to promote the model node of the entity model to a twin body object; A rendering data generation module is configured to generate corresponding rendering data according to business requirements; A business data driving module is configured to drive the rendering data by accessing the business data; A display logic processing module is configured to construct the display logic of rendering the model node according to the business requirements; The model node of the entity model is promoted to a twin body object, comprising: A child twin body object is created, and the parent object of the child twin body object is set as the twin body object; According to the model node name of the entity model, corresponding mesh information in the twin body object is found; the mesh information is segmented from the twin body object as a rendering node of the child twin body object; The remaining part of the twin body object after removing the mesh information is arranged to be a rendering node of the twin body object; According to the business requirements, corresponding rendering data is generated; According to the business requirements, rendering data of the twin body object and rendering data of the child twin body object are respectively created; ​ A root node is established for each rendering data, and other data is taken as a child node of the node, so as to obtain a rendering tree of the twin object and the child twin object for the business requirement; Mounting the rendering tree; The mounting rendering tree comprises: A rendering root node is respectively created for the twin object and the child twin object; The root nodes of the rendering trees of the twin object and the child twin object are mounted as child nodes to the respective rendering root nodes and named; The rendering data of the twin object and the rendering data of the child twin object are mounted on the rendering root nodes and named; The rendering data is driven by the business data, comprising: When the business data changes cause simple transformation of the data, the matrix information corresponding to the child node is updated; When the business data changes cause non-simple transformation of the rendering data, a rendering tree is regenerated to replace the mounted node.

5. A computer readable storage medium, characterized in that, The computer readable storage medium stores one or more programs, which can be executed by one or more processors to implement the steps in the implementation method of the twin multi-modal representation according to any one of claims 1-3.

6. A computer program product comprising computer programs or instructions, characterized in that, The computer program or instruction is executed by the processor to implement the steps in the implementation method of the twin multi-modal representation according to any one of claims 1-3.

Images