A digital twin loading method and related device

Abstract

The embodiment of the present application discloses a method for loading a digital twin and a related device. The method includes: receiving an input digital twin model, the digital twin model including skeleton data; constructing a digital twin skeleton based on the skeleton data; the digital twin skeleton includes a sub-skeleton corresponding to each digital twin of the smallest granularity; constructing a virtual camera outside the digital twin skeleton; emitting rays to a center point based on the virtual camera, and marking the hierarchy of the sub-skeleton to which each end face belongs according to the time sequence of different end faces of the digital twin contacted by the ray; normalizing the marking results to determine the hierarchy of each sub-skeleton; generating a configuration file for indicating the loading order of the digital twin skeleton based on the hierarchy of each sub-skeleton; and loading each sub-skeleton in sequence based on the configuration file. Through this method, the loading speed of the digital twin can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a loading method and related devices for a digital twin. Background Art

[0002] A digital twin (DT) is a simulation process that integrates multiple disciplines, multiple physical quantities, multiple scales, and multiple probabilities, leveraging data such as physical models, sensor updates, and operational history. Based on digital twin technology, a physical device can be mapped into a virtual space, creating a corresponding virtual model (or digital model), which can be called a "digital twin." A digital twin is a virtual representation of the real world, encompassing physical objects, processes, relationships, and behaviors. It can reflect the entire lifecycle of its corresponding physical device and simulate its behavior using real-time data transmitted from sensors on the physical device and historical operational data from the physical device.

[0003] Technicians in this field can use digital twins to simulate and analyze physical devices in the real world, providing auxiliary information for the subsequent operation and improvement of the physical devices. For example, digital twins can be used for the health maintenance of aerospace aircraft. First, a virtual model of the actual aircraft is created in the digital space, and it is synchronized with the actual aircraft status through real-time updates of sensor data. In this way, after each flight, technicians can promptly analyze and assess whether the aircraft needs maintenance and whether it can withstand the next mission load based on the current situation and past loads.

[0004] The digital twin skeleton refers to the geometry of the digital twin. Generally, when loading a digital twin, if the digital twin consists of multiple digital twins of the smallest granularity, the skeletons of each digital twin of the smallest granularity are requested and loaded simultaneously. However, due to the high precision and large file size of the digital twin skeleton, this approach can result in slow loading of the digital twin. Summary of the Invention

[0005] The embodiments of the present application provide a digital twin loading method and related devices, which can improve the loading speed of the digital twin.

[0006] In a first aspect, an embodiment of the present application provides a method for loading a digital twin, the method comprising: receiving an input digital twin model, the digital twin model including skeleton data, the digital twin model being composed of at least two digital twins of minimum granularity; constructing a digital twin skeleton based on the skeleton data, the digital twin skeleton including sub-skeletons corresponding to each digital twin of minimum granularity; determining the hierarchy of each sub-skeleton; generating a configuration file for the digital twin model, the configuration file including information indicating the loading order of the digital twin skeleton, the loading order of the digital twin skeleton being determined by the hierarchy corresponding to each sub-skeleton; and loading each sub-skeleton in sequence based on the configuration file. This method can improve the loading speed of the digital twin.

[0007] In combination with the first aspect, in a possible implementation method, the hierarchy of each sub-skeleton is determined, including: constructing a virtual camera outside the digital twin skeleton, where the distance between the virtual camera and the center point of the digital twin skeleton is a multiple of the longest distance from the center point to each end face of the digital twin skeleton; emitting rays to the center point based on the virtual camera, and marking the hierarchy of the sub-skeleton to which each end face belongs according to the time sequence in which the rays contact the different end faces of the digital twin skeleton; and normalizing the marking results to determine the hierarchy of each sub-skeleton.

[0008] In combination with the first aspect, in one possible implementation, constructing a virtual camera outside the digital twin skeleton includes: constructing a sphere with a radius that is a multiple of the longest distance from the center point to each end face of the digital twin skeleton, and the center point as the center of the sphere; and constructing a virtual camera in the projection area of ​​each end face of the digital twin skeleton on the spherical surface of the sphere.

[0009] In conjunction with the first aspect, in one possible implementation, the annotation results include at least one level of annotation for each sub-skeleton; and the annotation results are normalized, including: when a sub-skeleton has two or more levels, determining the level of a sub-skeleton as the outermost level. This normalization process can avoid the problem of a sub-skeleton having multiple levels of annotation, which may cause a loading order disorder.

[0010] In conjunction with the first aspect, in one possible implementation, each sub-skeleton is loaded sequentially based on the configuration file, including: loading from the outermost digital twin skeleton to the innermost digital twin skeleton; sub-skeletons marked as the same level are redefined as sub-skeletons of the same level. This loading order setting allows the digital twin to load the skeleton from the outside to the inside, which can speed up the loading of the outer skeleton of the digital twin, allowing users to view the loaded digital twin more quickly.

[0011] In combination with the first aspect, in a possible implementation, the digital twin model also includes texture map data; the method also includes: generating a texture map based on the texture map data, wherein the texture map has a corresponding relationship with the sub-skeleton corresponding to the smallest granularity of the digital twin; compressing each texture map to generate an image placeholder corresponding to each texture map; generating a configuration file for the digital twin model, the configuration file including a data file for each image placeholder, and the configuration file is also used to generate: when loading each sub-skeleton, an image of the surface of the sub-skeleton. The image placeholder can retain part of the image information of the original texture map, and can significantly reduce the amount of image data, which can speed up the loading speed of the image of the surface of the sub-skeleton, thereby effectively avoiding the "white mold phenomenon" and improving the user's perceptual experience.

[0012] In combination with the first aspect, in one possible implementation, the method further includes: generating a blurred animation based on each texture map and each image placeholder, the blurred animation being used to display the animation effect of loading from each image placeholder to each texture map; the configuration file also includes a data file for the blurred animation; when loading each sub-skeleton in sequence, the method further includes: based on the configuration file, loading and displaying the image placeholder corresponding to the sub-skeleton at the same level, loading and displaying the blurred animation corresponding to the sub-skeleton at the same level, and replacing the image placeholder corresponding to the sub-skeleton at the same level with the texture map. The blurred animation can display the animation effect of loading from the image placeholder to the texture map. In other words, in terms of visual perception, the user can see that the texture image gradually becomes clear from blurred.

[0013] In conjunction with the first aspect, in one possible implementation, compressing each texture map to generate an image placeholder corresponding to each texture map includes: performing lossless compression on each texture map to generate a compressed image; and performing lossy compression on the compressed image to generate an image placeholder corresponding to each texture map. This two-round compression of the texture maps can significantly reduce the image data size while retaining some of the original texture map's image information.

[0014] In a second aspect, an embodiment of the present application provides another method for loading a digital twin. The method includes: receiving an input digital twin model, the digital twin model including skeleton data and texture map data, and the digital twin model is composed of at least two digital twins of minimum granularity; obtaining a configuration file of the digital twin model, the configuration file including information indicating the loading order of the digital twin skeleton of the digital twin model, and a data file of a picture placeholder and a fuzzy animation corresponding to the texture map of the digital twin model; wherein the digital twin skeleton includes a sub-skeleton corresponding to each digital twin of minimum granularity, the sub-skeleton has a corresponding relationship with the texture map, and the loading order of the digital twin skeleton is from the outermost sub-skeleton to the innermost sub-skeleton; according to the skeleton data, texture map data and configuration file, the following operations are performed in sequence according to the loading order of the digital twin skeleton: loading and displaying the sub-skeletons of the same level and the picture placeholders corresponding to the sub-skeletons of the same level; loading and displaying the fuzzy animation corresponding to the sub-skeletons of the same level, and replacing the picture placeholders corresponding to the sub-skeletons of the same level with texture maps. This method can improve the loading speed of digital twins.

[0015] In a third aspect, an embodiment of the present application provides a loading device for a digital twin, which includes a communication unit, a processing unit, and a loading unit. The communication unit is configured to receive an input digital twin model, the digital twin model including skeleton data, and the digital twin model is composed of at least two digital twins of minimum granularity; the processing unit is configured to perform the following operations: construct a digital twin skeleton based on the skeleton data; the digital twin skeleton includes a sub-skeleton corresponding to each digital twin of minimum granularity; determine the hierarchy of each sub-skeleton; generate a configuration file for the digital twin model, the configuration file including information indicating the loading order of the digital twin skeleton, the loading order of the digital twin skeleton being determined by the hierarchy corresponding to each sub-skeleton; and the loading unit is configured to sequentially load each sub-skeleton based on the configuration file.

[0016] In one possible implementation, the processing unit is further configured to: construct a virtual camera outside the digital twin skeleton, wherein the distance between the virtual camera and the center point of the digital twin skeleton is a multiple of the longest distance between the center point and each end face of the digital twin skeleton; emit rays toward the center point based on the virtual camera, and mark the sub-skeleton level to which each end face belongs according to the time sequence of the different end faces of the digital twin skeleton contacted by the rays; and normalize the marking results.

[0017] In one possible implementation, when the processing unit is used to construct a virtual camera outside the digital twin skeleton, it specifically includes: constructing a sphere with a radius that is a multiple of the longest distance from the center point to each end face of the digital twin skeleton, and the center point as the center of the sphere; constructing a virtual camera per unit area of ​​the spherical surface of the sphere.

[0018] In one possible implementation, when the processing unit is used to construct a virtual camera outside the digital twin skeleton, it specifically includes: constructing a sphere with a radius that is a multiple of the longest distance from the center point to each end face of the digital twin skeleton, and the center point as the center of the sphere; constructing a virtual camera in the projection area of ​​each end face of the digital twin skeleton on the spherical surface of the sphere.

[0019] In one possible implementation, the annotation result includes at least one level at which each sub-skeleton is annotated; when the processing unit is used to normalize the annotation result, it specifically includes: when a sub-skeleton has two or more levels, the level of a sub-skeleton is determined as the outermost level.

[0020] In one possible implementation, the loading unit is used to load each sub-skeleton in sequence based on the configuration file, specifically including: loading from the outermost digital twin skeleton to the innermost digital twin skeleton in sequence; and re-determining the sub-skeletons marked as the same level as the sub-skeletons of the same level.

[0021] In one possible implementation, the digital twin model also includes texture map data; the processing unit is also used to: generate a texture map based on the texture map data, wherein the texture map has a corresponding relationship with the sub-skeleton corresponding to the smallest granularity of the digital twin; compress each texture map to generate an image placeholder corresponding to each texture map; generate a configuration file for the digital twin model, the configuration file includes a data file for each image placeholder, and the configuration file is also used to generate: an image of the surface of the sub-skeleton when loading each sub-skeleton.

[0022] In one possible implementation, the processing unit is further used to: generate a blurred animation based on each texture map and each image placeholder, the blurred animation being used to display the animation effect loaded from each image placeholder to each texture map; the configuration file also includes a data file for the blurred animation; when the loading unit is used to load each sub-skeleton in sequence, it is also used to: based on the configuration file, load and display the image placeholder corresponding to the sub-skeletons at the same level, load and display the blurred animation corresponding to the sub-skeletons at the same level, and replace the image placeholder corresponding to the sub-skeletons at the same level with the texture map.

[0023] In one possible implementation, the processing unit is used to compress each texture map and generate an image placeholder corresponding to each texture map, specifically including: performing lossless compression on each texture map to generate a compressed image; performing lossy compression on the compressed image to generate an image placeholder corresponding to each texture map.

[0024] In a fourth aspect, an embodiment of the present application provides a loading device for a digital twin, which includes a communication unit, a processing unit, and a loading unit. Among them, the communication unit is used to receive the input digital twin model, the digital twin model includes skeleton data and texture map data, and the digital twin model is composed of at least two digital twins with the smallest granularity; the processing unit is used to obtain the configuration file of the digital twin model, the configuration file includes information indicating the loading order of the digital twin skeleton of the digital twin model, and the data file of the image placeholder and blurred animation corresponding to the texture map of the digital twin model; wherein the digital twin skeleton includes a sub-skeleton corresponding to each digital twin with the smallest granularity, the sub-skeleton has a corresponding relationship with the texture map, and the loading order of the digital twin skeleton is from the outermost sub-skeleton to the innermost sub-skeleton; the loading unit is used to perform the following operations in sequence according to the skeleton data, texture map data and configuration file in the loading order of the digital twin skeleton: load and display the sub-skeletons of the same level and the image placeholders corresponding to the sub-skeletons of the same level; load and display the blurred animation corresponding to the sub-skeletons of the same level, and replace the image placeholders corresponding to the sub-skeletons of the same level with texture maps.

[0025] In a fifth aspect, an embodiment of the present application provides a server, which is used to execute the methods executed in the above aspects.

[0026] In a sixth aspect, a computer program product is provided, which includes: a computer program code, and when the computer program code is run, the methods performed by the above aspects are executed.

[0027] In a seventh aspect, the present application provides a computer-readable storage medium, which stores a computer program or instruction. When the computer program or instruction is executed by a communication device, the method of executing the above aspects is implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 is a schematic diagram of a digital twin and a digital twin skeleton provided in an embodiment of the present application;

[0029] Figure 2 is a schematic diagram of a digital twin and texture map provided in an embodiment of the present application;

[0030] Figure 3 Schematic diagram of the software structure of a digital twin loading platform provided in an embodiment of the present application;

[0031] Figure 4 This is a flow chart of a method for loading a digital twin provided in an embodiment of the present application;

[0032] Figure 5 This is a flow chart of generating a configuration file provided by an embodiment of the present application;

[0033] Figure 6 is a schematic diagram of an environmental ray scanning provided by an embodiment of the present application;

[0034] Figure 7 is a schematic diagram of another environmental ray scanning provided by an embodiment of the present application;

[0035] Figure 8 is a schematic diagram of another environmental ray scanning provided by an embodiment of the present application;

[0036] Figure 9 This is another flow chart for generating a configuration file provided in an embodiment of the present application;

[0037] Figure 10 This is a flowchart of assembling a digital twin provided in an embodiment of the present application;

[0038] Figure 11 is a schematic diagram of a digital twin provided in an embodiment of the present application;

[0039] Figure 12 This is a schematic diagram of a digital twin loading process provided in an embodiment of the present application;

[0040] Figure 13 This is a flow chart of a method for loading a digital twin provided in an embodiment of the present application;

[0041] Figure 14 This is a flowchart of another method for loading a digital twin provided in an embodiment of the present application;

[0042] Figure 15 It is a structural schematic diagram of a loading device of a digital twin provided in an embodiment of the present application. DETAILED DESCRIPTION

[0043] The specific embodiments of the present application are further described in detail below with reference to the accompanying drawings.

[0044] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the specification and appended claims of this application, the singular expressions "a", "an", "said", "above", "the" and "this" are intended to also include plural expressions unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in this application refers to and includes any or all possible combinations of one or more listed items. The term "plurality" used in this application refers to two or more.

[0045] It should be noted that the terms "first", "second", "third", etc. in the specification and claims of the present application and in the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way are interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the term "comprises" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0046] First, before further explaining the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained to facilitate understanding by those skilled in the art. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations:

[0047] 1. Digital Twin

[0048] A digital twin is a simulation process that integrates multiple disciplines, multiple physical quantities, multiple scales, and multiple probabilities, leveraging data such as physical models, sensor updates, and operational history. Based on digital twin technology, a physical device can be mapped into a virtual space, creating a corresponding virtual model (or digital model) of the physical device. This virtual model can be called a "digital twin." A digital twin is a virtual representation of the real world, encompassing physical objects, processes, relationships, and behaviors. It can reflect the entire lifecycle of its corresponding physical device and simulate its behavior using real-time data transmitted from sensors on the physical device and historical data from the physical device's operation.

[0049] Technicians in this field can use digital twins to simulate and analyze physical devices in the real world, providing auxiliary information for the subsequent operation and improvement of the physical devices. For example, digital twins can be used for the health maintenance of aerospace aircraft. First, a virtual model of the actual aircraft is created in the digital space, and it is synchronized with the actual aircraft status through real-time updates of sensor data. In this way, after each flight, technicians can promptly analyze and assess whether the aircraft needs maintenance and whether it can withstand the next mission load based on the current situation and past loads.

[0050] 2. Digital Twin System

[0051] Some interrelated or interacting digital twins constitute a digital twin system. The digital twin system includes digital twins as well as background data, operations, algorithms, networks and other information. For example, in the digital space, the server's motherboard, core processor, memory and input and output devices, etc. can constitute an entire server. The server can be regarded as a digital twin system, and the motherboard (or other components) can be regarded as a digital twin. In another example, the various devices in the computer room constitute an entire computer room. The computer room can be regarded as a digital twin system, and one of the devices can be regarded as a digital twin.

[0052] 3. Digital Twin Skeleton

[0053] The digital twin skeleton refers to the geometry of the digital twin. A digital twin of the smallest granularity corresponds to a geometry, and a digital twin can be composed of one or more digital twins of the smallest granularity. When a digital twin is composed of multiple digital twins of the smallest granularity, the skeleton of each digital twin of the smallest granularity can be considered a sub-skeleton. That is, each digital twin of the smallest granularity corresponds to a sub-skeleton, and multiple sub-skeletons together constitute the digital twin skeleton.

[0054] The digital twin of the minimum granularity can be understood as a separate digital twin that cannot be separated by the user. It should be noted that the "minimum granularity" does not represent a limit on the size or data volume of the digital twin, but is related to the settings at the time of creation. In one example, when creating a digital twin of an aircraft, each component device in the aircraft can be set as a digital twin of the minimum granularity. In another example, when creating a digital twin of an electronic device, each component module in the electronic device can be set as a digital twin of the minimum granularity.

[0055] In geometry, a finite shape enclosed by several geometric faces (planes or curved surfaces) is called a geometric body, the faces enclosing the geometric body are called interfaces or surfaces of the geometric body, the intersection lines of different interfaces are called edges of the geometric body, and the intersection points of different edges are called vertices of the geometric body. A geometric body can also be regarded as a finite space area divided by several geometric faces in space.

[0056] For example, see Figure 1 , is a schematic diagram of a digital twin and a digital twin skeleton provided in an embodiment of the present application. Figure 1 The digital twin illustrated in the figure is a digital twin with the smallest granularity. The skeleton of the digital twin is a cuboid, which is surrounded by plane ABCD, plane DCFE, plane EFGH, plane ADEH, plane BCFG and plane ABGH. The faces surrounding the cuboid can be called interfaces, end faces or surfaces of the cuboid.

[0057] 4. Material

[0058] Materials are the surface properties of rendered geometry, including color, glossiness, texture, smoothness, roughness, transparency, refractive index, and so on. For example, a glass material has bright highlights, is transparent, and has strong reflection and refraction properties, while a stone material has a rough surface, weak reflection, and no refraction properties. Materials can give digital twins realistic or unique visual effects. A material can reference one or more textures.

[0059] 5. Texture

[0060] Textures describe the appearance of geometric surfaces, reflecting properties such as pattern, texture, and color. Examples include wood, marble, and metal. Textures are images wrapped around the surface of geometric objects. They can be loaded from files, generated on the canvas, or rendered into the scene.

[0061] 6. Digital Twin Texture Mapping

[0062] The digital twin texture map is used to reflect the texture of the digital twin surface. A texture map can be expressed as a UV map file. Among them, U and V refer to the horizontal axis and vertical axis of the two-dimensional space (similar to the X axis and Y axis in the plane rectangular coordinate system). The UV map file contains the coordinate information of each point on the image. The coordinate information of each point corresponds to the point on the three-dimensional model. Based on the UV map file, it can be determined where a point on the texture image should be placed on the surface of the digital twin skeleton. For example, see Figure 2, is a schematic diagram of a digital twin and texture map provided in an embodiment of the present application. The texture map is displayed on the surface of the digital twin skeleton, and the points on the texture map correspond to the points on the surface of the digital twin skeleton. For example, vertex A-vertex H of the digital twin skeleton corresponds to point A-point H in the texture map.

[0063] The texture map has a corresponding relationship with the sub-skeleton corresponding to the digital twin of the smallest granularity. The corresponding relationship can include one-to-one, one-to-many, and many-to-one possibilities. Optionally, one texture map can correspond to one sub-skeleton, that is, the surface of a sub-skeleton displays one texture map; optionally, one texture map can correspond to multiple sub-skeletons, that is, the surfaces of multiple sub-skeletons can display the same texture map; optionally, multiple texture maps can correspond to one sub-skeleton, that is, the surface of a sub-skeleton displays multiple texture maps. In an embodiment of the present application, the digital twin model includes skeleton data and texture map data. There is a corresponding relationship between the skeleton data and the texture map data, and based on the corresponding relationship, it can be clarified which texture maps are displayed on the surface of each digital twin of the smallest granularity. Exemplarily, sub-skeleton 1 corresponds to texture map file 1, skeleton 2 corresponds to texture map file 1, sub-skeleton 3 corresponds to texture map file 2, and so on.

[0064] 7. White mold phenomenon

[0065] The white model phenomenon refers to the phenomenon that when the client network is poor, the digital twin skeleton is loaded first, but the texture map is not loaded, causing the surface of the digital twin to appear white.

[0066] 8. Web graphics library (WebGL)

[0067] WebGL is a technology that renders interactive two-dimensional (2D) and three-dimensional (3D) graphics in any compatible web browser without the use of plugins. WebGL is integrated into the browser's web page standards. Based on WebGL, you can use the canvas tag on a web page to call on the graphics processing unit (GPU) resources to render 2D or 3D interactive effects. WebGL elements can be added to other hypertext markup language (HTML) elements and blended with other parts of the web page or its background. WebGL programs consist of handlers written in JavaScript and shader code written in the OpenGL Shading Language (GLSL), a language similar to C or C++, that executes on the computer's graphics processing unit (GPU). The WebGL technical standard eliminates the need to develop dedicated web rendering plugins and can be used to create website pages with complex 3D structures.

[0068] 9. Web 3-dimension (Web3D)

[0069] Web3D refers to various methods of displaying three-dimensional computer graphics through a web browser. In the past, Web3D mainly displayed 3D models by installing browser plug-ins. Today, the industry's WebGL-based programs no longer require browser plug-ins.

[0070] 10.Three.js

[0071] Three.js is a cross-browser Web3D engine that uses a JavaScript library or application programming interface (API) to create and display animated 3D computer graphics in a web browser. Three.js allows you to create GPU-accelerated 3D animated elements in web pages using JavaScript, rather than using specific browser plugins.

[0072] 11.3D Model

[0073] A 3D model is a polygonal representation of an object, typically displayed on a computer or other video device. The displayed object can be real-world or imaginary. Anything that exists in the physical world can be represented by a 3D model.

[0074] The process of loading the digital twin requires loading the digital twin skeleton and texture map. Generally, during the loading process, if the digital twin is composed of multiple digital twins of minimum granularity, the skeletons of the digital twins of each minimum granularity will be requested and loaded at the same time. However, due to the high precision of the digital twin skeleton and the large file size, this method will lead to the problem of slow loading speed of the digital twin. In addition, in order to achieve a more realistic display effect, the data volume of the texture map of the digital twin is also relatively large, which will take a long time to transmit, and it is easy to have a "white mold phenomenon" with only a model skeleton but no texture map. In view of this, a loading method for the digital twin of the embodiment of the present application is proposed.

[0075] See Figure 3 , is a software structure diagram of a digital twin loading platform provided in an embodiment of the present application. The digital twin loading platform (to simplify the description, it is also referred to as the "loading platform" in the subsequent content) includes a communication module, a skeleton scanning module, a texture mapping processing module, an application assembly module, and a display module. The digital twin loading platform can be a server or an electronic device.

[0076] Among them, the communication module is used to receive the digital twin model input by the user. The digital twin model is a model of the digital twin built by the user based on the needs. The embodiment of the present application does not limit the way the user builds the digital twin model and the format, size, etc. of the digital twin model. In the embodiment of the present application, the digital twin model is composed of two or more digital twins of the smallest granularity. The digital twin model contains skeleton data and texture map data. There is a corresponding relationship between the skeleton data and the texture map data. Based on this association relationship, it can be clearly determined which texture maps are displayed on the surface of each digital twin of the smallest granularity.

[0077] The skeleton scanning module is used to process the skeleton data of the digital twin model. It can divide the skeleton of the digital twin model into multiple levels according to the internal and external relationship of the position. Based on the determined levels, the loading order of the digital twin skeleton during the subsequent loading process can be determined. For example, the loading platform loads the digital twin skeleton from the outer layer to the inner layer. The skeleton scanning module generates the determined loading order of the digital twin skeleton into the configuration file of the digital twin model.

[0078] The texture map processing module is used to process the texture map data of the digital twin model, and can generate its corresponding image placeholder based on the texture map of the digital twin model. The image placeholder is an image generated after lossy compression of the texture map, and has a corresponding relationship (or called an association relationship) with the texture map. The texture map processing module is also used to generate a blurred animation based on the texture map and the image placeholder corresponding to the texture map. The blurred animation shows the animation effect of loading from the image placeholder to the texture map. In other words, in terms of visual perception, through the blurred animation, the user can see that the texture image gradually becomes clear from blurred. The texture map processing module writes the generated image placeholder and blurred animation into the configuration file of the digital twin model.

[0079] The application assembly module is used to assemble the digital twin based on the digital twin's skeleton data, texture mapping data, and configuration files. The configuration files include the loading order of the digital twin skeleton generated by the skeleton scanning module, as well as the image placeholder and blur animation data files generated by the texture mapping module.

[0080] The display module is used to display the digital twin generated by the application assembly module. It should be noted that during the process of applying the assembly module to generate the digital twin, the display module also displays the generated digital twin.

[0081] It should also be noted that Figure 3 The software architecture of the digital twin loading platform in the example is only an example. In actual applications, it may include more or fewer modules, or the function of a module may be provided by multiple modules. The embodiments of this application do not limit the division method of the modules.

[0082] See Figure 4 , is a flow chart of a method for loading a digital twin provided in an embodiment of the present application, which can be based on Figure 3 The digital twin loading platform shown is used to achieve this. The digital twin targeted by this method is composed of two or more digital twins of the smallest granularity. The method includes but is not limited to the following steps.

[0083] Step 401: Receive a digital twin model input by a user.

[0084] This step 401 can be executed by the communication module of the loading platform. The introduction of the digital twin model can refer to the above description and will not be repeated here.

[0085] Step 402: Determine the loading order of the digital twin skeleton according to the skeleton data of the digital twin model, and generate a configuration file of the digital twin model based on the loading order of the digital twin skeleton.

[0086] This step 402 can be performed by the skeleton scanning module of the loading platform. Optionally, the loading platform executes step 402 by calling a service API, which has the function of executing step 402. The execution of step 402 can refer to Figure 5 The method flow chart shown in FIG. The method includes but is not limited to the following steps.

[0087] Step 501: Read the skeleton data of the digital twin model, and construct a digital twin skeleton based on the skeleton data.

[0088] Step 502: Use environmental rays to scan the digital twin skeleton and mark the levels of the digital twin in the digital twin skeleton.

[0089] First, a virtual camera is set outside the digital twin skeleton. The distance between the virtual camera and the origin of the skeleton is a multiple of the longest distance from the origin of the skeleton to each end face of the skeleton (for example, twice, or other values ​​such as 1.5 times, three times, etc.). The origin of the skeleton can be set when the user builds the digital twin model. For example, the skeleton is Figure 6 In the case of the rectangular parallelepiped shown in , the origin O of the skeleton can be the geometric center of the skeleton.

[0090] In the embodiments of the present application, the end face of a skeleton is understood to be the outermost face of the skeleton. An end face can be a flat surface, a curved surface, or other irregular surface (e.g., a surface with holes, an uneven surface, etc.). The distance from the origin to the end face is the distance from the origin to each point on the end face. It should be noted that the distance from the origin to the end face in the embodiments of the present application is defined differently from the distance from a point to a plane (defined as the minimum distance from a point in space to a point in a plane).

[0091] The longest distance from the skeleton's origin to its end face can be read from the skeleton data. Alternatively, several rays can be emitted from the skeleton's origin toward the outside of the skeleton, with the distance between the point where the ray contacts the skeleton's end face and the origin being the distance from the origin to the end face. From the multiple distances determined by these rays, the largest distance is selected as the longest distance from the skeleton's origin to the end face. In particular, if there are holes in the skeleton, some rays may not be able to contact the skeleton's end face. In this case, the distance from the origin to the end face cannot be determined for these rays, or the distance is infinite. The data for these rays is discarded when selecting the longest distance.

[0092] by Figure 6For example, the skeleton is composed of two cuboids (a first cuboid and a second cuboid). From the perspective of the internal and external relationship of the skeleton, the first cuboid is located outside the second cuboid. The end faces of the skeleton are the faces of the first cuboid. From the geometric structure of the cuboid, it can be seen that the longest distance from the origin O of the skeleton to the end faces of the skeleton should be the distance from the origin O to the vertex of the first cuboid (for example, point A) (for example, the length of the line segment OA). If the distance between the virtual camera and the origin O is set to twice the longest distance from the origin O to the end faces of the skeleton, then the distance between the virtual camera and the origin of the skeleton is twice the length of the line segment OA.

[0093] In a possible implementation, the position of the virtual camera can correspond to each end face of the skeleton, and one virtual camera has a corresponding relationship with one end face. Figure 6 , the origin (center point) O of the skeleton is connected to one end face, that is, the geometric center point V of the plane ABCD, to form a connecting line, and the connecting line is extended from the origin O to the point V until the length of the connecting line is equal to twice the distance from the origin to the point V. The position of the endpoint can be determined as the position of the virtual camera, which can be referred to Figure 6 In the same way, the virtual cameras corresponding to the other end faces of the skeleton can be determined, which can be referred to Figure 6 The virtual cameras W2-W6 in . It should be noted that Figure 6 The setting method of W1-W4 is only a schematic. In fact, on the sphere formed by W1-W4, W1 can be set at any position on the projection of the ABCD surface on the sphere. The positions of other cameras are the same as W1 and will not be repeated here.

[0094] In another possible implementation, for situations where the end face structure of the skeleton is more complex (for example, the end face is less smooth, uneven, or the end face is not composed of a plane or a simple curved surface), or the internal structure of the skeleton is more complex (for example, the digital twin model contains a large number of digital twins of the minimum granularity), more virtual cameras can be set. For example, multiple virtual cameras can form a sphere containing the skeleton, and the center of the sphere is the origin of the skeleton. A virtual camera can be regarded as a point on the sphere, and the number of virtual cameras is not limited. Optionally, in one implementation, a virtual sphere can be constructed with the origin of the skeleton as the center of the sphere and the multiple of the longest distance from the origin of the skeleton to the end face of the skeleton as the radius. On the spherical surface of the virtual sphere, a virtual camera is set up per unit area. The selection of the unit area is not limited. The smaller the unit area, the more virtual cameras there are. Optionally, when the unit area is small enough, each point on the sphere can be regarded as set as a virtual camera.

[0095] Next, each virtual camera emits a ray toward the origin of the skeleton for scanning, and marks the level of the digital twin to which the interface of the geometric body belongs (which can also be understood as marking the digital twin skeleton to which the interface of the geometric body belongs) in the time sequence of the interfaces of the geometric body that the ray contacts. It should be noted that the digital twin targeted by this marking is the digital twin with the smallest granularity, that is, the level of the digital twin with the smallest granularity is marked (which can also be understood as marking the level of the sub-skeleton corresponding to the digital twin with the smallest granularity). This level can reflect the internal and external structure of the digital twin with the smallest granularity in the entire digital twin. If a digital twin with the smallest granularity is the "first level", it means that the digital twin with the smallest granularity is located at the outermost layer of the entire digital twin. If a digital twin with the smallest granularity is the "second level", it means that the digital twin with the smallest granularity is located inside the entire digital twin and adjacent to the outermost layer.

[0096] For example, if a ray contacts three interfaces in succession, the digital twin to which the first interface is contacted is labeled as "first level" (it can also be labeled as "1", "frist", etc., and this application does not limit the labeling method), the digital twin to which the second interface is contacted is labeled as "second level", and the digital twin to which the third interface is contacted is labeled as "third level". Optionally, a maximum number of levels can be preset, for example five levels. For a ray, the digital twin with the same minimum granularity is labeled with the level only once.

[0097] For example, see Figure 6 , is a schematic diagram of an environmental ray scanning provided by an embodiment of the present application. Figure 6 In the figure, the skeleton of the digital twin is illustrated as two cuboids, and a total of 6 virtual cameras, namely W1-W6, are set up outside the skeleton. The distance between the virtual camera and the origin of the digital twin skeleton is a multiple of the longest distance from the origin of the skeleton to the interface of the skeleton, and each virtual camera emits a ray toward the origin. Taking virtual camera W1 as an example, virtual camera W1 emits a ray toward the origin O. The ray starts from W1 and first touches the interface ABCD. The digital twin to which the interface ABCD belongs, that is, the first cuboid, is marked as "first level". After that, it touches the interface IKLJ and the digital twin to which the interface IKLJ belongs, that is, the second cuboid, is marked as "second level". Next, the ray reaches the origin O, and the marking of the ray ends. The marking method of other virtual cameras can refer to this method and will not be repeated here.

[0098] Step 503: normalize the annotation results obtained by scanning.

[0099] The meaning of normalization is that when a digital twin has multiple hierarchical annotations at the same time, the hierarchical annotation of the digital twin is determined as the hierarchical annotation of the upper layer (also known as the top layer, the highest layer, the smallest layer), and the hierarchical annotation of the lower layer is discarded. For example, the hierarchical annotations of a digital twin are "second level" and "third level", then the hierarchical annotation of the digital twin is determined as "second level". The reason why a digital twin has multiple hierarchical annotations at the same time is that the relative positions of the same digital twin and different cameras are different, and the hierarchical annotations of the same digital twin by different cameras may be different. By means of normalization, the problem of disordered loading order caused by the existence of multiple hierarchical annotations for a digital twin can be avoided.

[0100] For example, see Figure 7 , Figure 7 This is a schematic diagram of another environmental ray scanning provided in an embodiment of the present application. Figure 7 exist Figure 6 On the basis of , a digital twin with the smallest granularity, namely the third cuboid, is added to the skeleton. For virtual camera W2, virtual camera W2 emits a ray toward the origin O. Starting from W2, the ray first contacts the interface RUZW and labels the digital twin to which interface RUZW belongs, namely the third cuboid, as "first level". Then, it contacts the interface STYX. The digital twin to which interface STYX belongs is the third cuboid. Since the third cuboid has been labeled by the ray, no further labeling is performed. Then, it contacts the interface ADEH and labels the digital twin to which interface ADEH belongs, namely the first cuboid, as "second level". Then, it contacts the interface IJMP and labels the digital twin to which interface IJMP belongs, namely the second cuboid, as "third level". Next, the ray reaches the origin O, and the labeling of the ray ends. In addition, in addition to virtual camera W2, other virtual cameras will also perform ray scanning. According to the same labeling rules, other virtual cameras all label the first cuboid as "first level" and the second cuboid as "second level".

[0101] All virtual cameras have been labeled, and the results are as follows: the first cuboid has two hierarchical labels, "First Level" and "Second Level," the second cuboid has two hierarchical labels, "Second Level" and "Third Level," and the third cuboid has one hierarchical label, "First Level." Following the normalization method of retaining the highest-level label for a digital twin with the smallest granularity, the final hierarchical labels are: "First Level" for the first and third cuboids, and "Second Level" for the second cuboid. This normalization method avoids the problem of multiple hierarchical labels on a single digital twin, which can lead to a disordered loading order.

[0102] Optionally, to avoid the impact of holes that may exist in the digital twin, one possible implementation involves calculating the average distance from all virtual cameras to each level on the projection of each face onto the sphere formed by the virtual cameras. If the relative distance from a virtual camera to a level differs significantly from the average distance from all virtual cameras to that level, the relative distance is treated as an outlier, and the level marked with the virtual camera is downgraded by one level. For example, if the level is "Second Level," the "Second Level" is modified to "Third Level."

[0103] The relative distance from a virtual camera to a level is the distance between the location of the virtual camera and the contact point of the interface of the digital twin of the level that the ray of the virtual camera first contacts. Figure 7 For example, if the ray from virtual camera W1 first contacts the interface of the digital twin at the first level at point V1, the relative distance from virtual camera W1 to the first level is the length of line segment W1V1. If the ray from virtual camera W1 first contacts the interface of the digital twin at the second level at point V2, the relative distance from virtual camera W1 to the first level is the length of line segment W1V2. Using this method of determining relative distances, the relative distance from each virtual camera to that level can be determined, thereby calculating the average distance.

[0104] The following lists the scenarios where digital twins have "holes", see Figure 8 , Figure 8 This is a schematic diagram of another environmental ray scanning provided in an embodiment of the present application. Figure 8 The first interface is the interface of a digital twin, and there is a hole in the first interface. The second interface is the interface of another digital twin. Figure 8 Virtual cameras W7-W10 are also illustrated. In actual scenes, there may be other virtual cameras. For ease of introduction, these four virtual cameras are used as an example. The average distance is calculated based on the relative distances between the four virtual cameras.

[0105] For the ray emitted by virtual camera W7, there is a contact point a with the first interface and a contact point b with the second interface. Therefore, the level of the digital twin to which the first interface belongs is labeled "first level", and the level of the digital twin to which the second interface belongs is labeled "second level". The relative distance between virtual camera W7 and the first level is the length of line segment W7a, and the relative distance between virtual camera W7 and the second level is the length of line segment W7b. For the ray emitted by virtual camera W8, there is a contact point c with the first interface and a contact point d with the second interface. Therefore, the level of the digital twin to which the first interface belongs is labeled "first level", and the level of the digital twin to which the second interface belongs is labeled "second level". The relative distance between virtual camera W8 and the first level is the length of line segment W8c, and the relative distance between virtual camera W8 and the second level is the length of line segment W8d. For the ray emitted by virtual camera W9, due to the influence of the "hole", there is no contact point with the first interface, but there is a contact point f with the second interface. Therefore, the level of the digital twin to which the second interface belongs is marked as "first level", and the relative distance between virtual camera W9 and the first level is the length of line segment W9f. For the ray emitted by virtual camera W10, there is a contact point e with the first interface and a contact point g with the second interface. Therefore, the level of the digital twin to which the first interface belongs is marked as "first level", and the level of the digital twin to which the second interface belongs is marked as "second level". The relative distance between virtual camera W10 and the first level is the length of line segment W10e, and the relative distance between virtual camera W10 and the second level is the length of line segment W10g.

[0106] Calculate the average distance from all virtual cameras to the first level, that is, the average of the lengths of line segment W7a, line segment W8c, line segment W9f and line segment W10e. Due to the influence of the holes, the difference between the length of line segment W9f and the average distance is larger than the difference between the length of other line segments and the average distance. Therefore, the relative distance of "line segment W9f" is taken as an outlier, and the "first level" marked by the virtual camera W9 is lowered by one level, that is, the "first level" is changed to "second level". After the modification, the annotation result of the virtual camera W9 for the second interface is the same as that of other virtual cameras, avoiding the problem of incorrect hierarchical annotation caused by holes. Optionally, in another possible implementation method, the difference between each relative distance and other relative distances can be calculated separately. When the difference is greater than a preset value, the relative distance corresponding to the difference is taken as an outlier. Figure 8 For example, the difference between line segment W7a and line segment W8c and line segment W10e is small and smaller than the preset value; the difference between line segment W7a and line segment W9f is large and larger than the preset value, so line segment W9f is regarded as an abnormal value.

[0107] Optionally, in another possible implementation, multiple distance ranges may be preset. For example, the first distance range is (0, a], the second distance range is (a, b], the third distance range is (b, c], the fourth distance range is (c, d], and the fifth distance range is (d, e]. Optionally, the values ​​of the dividing points of each distance range (i.e., a, b, c, d, e) may be determined based on the longest distance from the origin of the skeleton to the end face of the skeleton and the number of distance ranges. For example, the longest distance from the end face of the skeleton is If L is the distance range, and the number of distance ranges is 5, then a = 1 / 5L, b = 2 / 5L, c = 3 / 5L, d = 4 / 5L, and e = L. Count the number of relative distances that fall within each distance range, and consider the distance range with the largest number of distances as normal values. Consider the relative distances that fall within the other distance ranges as outliers, and perform a descending labeling operation. Alternatively, consider the distance range with the smallest number of distances as an outlier, and consider the relative distances that fall within this outlier as outliers, and perform a descending labeling operation.

[0108] Step 504: Determine the loading order of the digital twin skeleton according to the normalized annotation results.

[0109] Based on the normalized annotation results, the hierarchy of each digital twin in the digital twin skeleton can be determined, and the loading order of the digital twin skeleton can be determined based on the hierarchy. This means that the digital twin is loaded from the first level to the highest level; from a positional perspective, the digital twin skeleton is loaded from the outside to the inside. This loading order setting allows the digital twin skeleton to be loaded from the outside to the inside, which can speed up the loading of the outer skeleton of the digital twin, allowing users to view the loaded digital twin more quickly.

[0110] Step 505: Write the loading order of the digital twin skeleton into the configuration file of the digital twin model.

[0111] In this configuration file, the digital twin skeletons marked as the smallest granularity of the same level constitute the digital twin skeletons of the same level.

[0112] Step 403: Process the texture map data of the digital twin model to generate an image placeholder and a blurred animation corresponding to the texture map, and write the data files of the image placeholder and the blurred animation into the configuration file of the digital twin model.

[0113] This step 403 can be performed by the texture mapping processing module of the loading platform. The execution of this step 403 can refer to Figure 9 The method flow chart shown in FIG. The method includes but is not limited to the following steps.

[0114] Step 901: Read the texture map data of the digital twin model, and generate a texture map based on the texture map data.

[0115] Step 902: Use a lossless compression tool to perform a first round of compression on the texture map.

[0116] Lossless compression is a compression method that uses statistical redundancy of data. Lossless compression can reduce the amount of texture map data, and the original data can be completely restored from the lossless compressed data without any distortion. The embodiments of this application do not limit the tools / methods of lossless compression.

[0117] Step 903: Perform a second round of lossy extreme compression on the texture map after the first round of compression to generate an image placeholder corresponding to the texture map.

[0118] This image placeholder is an image generated after lossy compression of the texture map, and has a corresponding relationship with the texture map. The second round of lossy extreme compression can significantly reduce the image data volume while retaining some of the image information of the original texture map.

[0119] Step 904: Generate a blur animation based on the texture map and the image placeholder corresponding to the texture map.

[0120] The blurred animation shows the animation effect of loading from the image placeholder to the texture map. In other words, in terms of visual perception, the user can see that the texture image gradually becomes clear from blurred. Optionally, the duration of the blurred animation can be determined based on the data size ratio of the texture map and the image placeholder. Exemplarily, the larger the data size ratio of the texture map to the image placeholder, the longer the duration of the blurred animation. For example, if the data size ratio of the texture map to the image placeholder is 1000, the duration of the blurred animation can be 2 seconds; if the data size ratio of the texture map to the image placeholder is 10000, the duration of the blurred animation can be 8 seconds.

[0121] Step 905: Write the image placeholder and blur animation data file corresponding to the texture map into the configuration file of the digital twin model. If there are multiple texture maps, steps 902 to 904 are performed for each texture map.

[0122] Step 404: Assemble the digital twin according to the skeleton data, texture map data, and configuration file of the digital twin.

[0123] Among them, the configuration file includes the loading order of the digital twin skeleton, as well as the image placeholders and blur animations corresponding to the texture maps.

[0124] This step 404 can be performed by the application assembly module of the loading platform. The execution of this step 404 can refer to Figure 10The method flow chart shown in FIG. The method includes but is not limited to the following steps.

[0125] Step 1001: Read the configuration file.

[0126] Step 1002: Call the Three.js engine. By calling the Three.js engine, three-dimensional computer graphics of the digital twin can be created and displayed in a web browser.

[0127] Step 1003: Load the first-level digital twin skeleton and the image placeholder corresponding to the first-level digital twin skeleton.

[0128] Among them, the digital twin skeleton of the first level is the skeleton of the outermost layer of the digital twin, and the image placeholder corresponding to the digital twin skeleton of the first level can reflect the image information of part of the texture map of the digital twin skeleton of the first level. The skeletons of the digital twins with the smallest granularity marked as "first level" together constitute the digital twin skeleton of the first level. By executing step 8805, the outermost skeleton of the digital twin and the corresponding image placeholder of the skeleton can be loaded preferentially during the entire loading process, thereby improving the loading speed of the digital twin and allowing users to view the loaded digital twin faster. Because the data volume of the image placeholder is small and the loading speed is fast, the "white mold phenomenon" can be effectively avoided, thereby improving the user's perceptual experience.

[0129] Step 1004: Load the blur animation corresponding to the first-level digital twin skeleton, and replace the image placeholder corresponding to the first-level digital twin skeleton with a texture map.

[0130] By executing step 1004, the loaded blurred animation is displayed on the first-level digital twin skeleton, and the user can view the animation effect loaded from the image placeholder to the texture map. Finally, the texture map corresponding to the skeleton is displayed on the first-level digital twin skeleton.

[0131] Step 1005: sequentially load digital twin skeletons of other levels lower than the first level and their corresponding image placeholders.

[0132] Optionally, step 1005 may be performed while replacing the image placeholder corresponding to the first-level digital twin skeleton with a texture map.

[0133] Step 1006: Load the blurred animations corresponding to the digital twin skeletons of other levels lower than the first level in sequence, and replace the corresponding image placeholders with texture maps.

[0134] Step 1005 and step 1006 are executed in sequence. For example, after executing step 1004, the loading platform loads the digital twin skeleton of the second level and its corresponding image placeholder, and then loads the fuzzy animation corresponding to the digital twin skeleton of the second level, and replaces its corresponding image placeholder with a texture map. The skeletons of the digital twins with the smallest granularity marked as "second level" together constitute the digital twin skeleton of the second level. Afterwards, the loading platform loads the digital twin skeleton of the third level and its corresponding image placeholder, and then loads the fuzzy animation corresponding to the digital twin skeleton of the third level, and replaces its corresponding image placeholder with a texture map. The skeletons of the digital twins with the smallest granularity marked as "third level" together constitute the digital twin skeleton of the third level. According to this rule, the digital twin skeletons of other levels are loaded in sequence until the digital twin skeletons of all levels are loaded.

[0135] Optionally, since digital twin skeletons at other levels lower than the first level are generally invisible to users, the loading order of digital twin skeletons at other levels may not be constrained during the loading process.

[0136] Optionally, the method of loading the digital twin skeleton in layers in the embodiment of the present application and loading from the image placeholder to the texture map can be called progressive loading.

[0137] Step 405: Display the generated digital twin.

[0138] Step 405 can be performed by the display module of the loading platform. It should be noted that step 405 is performed simultaneously with step 404, meaning that the generated digital twin is displayed simultaneously during the digital twin generation process. Ultimately, the loading platform can display the final generated digital twin on the display interface.

[0139] For example, see Figure 11 , is a schematic diagram of a digital twin provided in an embodiment of the present application. Figure 11 The digital twin shown in the figure is a server.

[0140] In another example, Figure 6 A description of the loading process based on the digital twin described in . Figure 12 , is a schematic diagram of a digital twin loading process provided in an embodiment of the present application.

[0141] First, load the first-level digital twin skeleton, that is, load the first cuboid; and load the image placeholder corresponding to the first-level digital twin skeleton, that is, load the image placeholder corresponding to the texture map on the surface of the first cuboid. The display effect corresponding to this step can be referred to Figure 12 (1) shown.

[0142] Next, load the blur animation corresponding to the first-level digital twin skeleton and replace the image placeholder corresponding to the first-level digital twin skeleton with a texture map, that is, gradually change from displaying the image placeholder to displaying the texture map on the surface of the first cuboid. The final display effect corresponding to this step can be referred to Figure 12 (2) shown.

[0143] After that, load the second-level digital twin skeleton, that is, load the second cuboid; and load the image placeholder corresponding to the second-level digital twin skeleton, that is, load the image placeholder corresponding to the texture map on the surface of the second cuboid. The display effect corresponding to this step can be referred to Figure 12 (3) shown.

[0144] Finally, load the blur animation corresponding to the second-level digital twin skeleton and replace the image placeholder corresponding to the second-level digital twin skeleton with a texture map, that is, gradually change from displaying the image placeholder to displaying the texture map on the surface of the second cuboid. The final display effect corresponding to this step can be referred to Figure 12 (4) As shown. At this point, the digital twin is loaded.

[0145] Since the digital twin corresponding to the second cuboid is obscured from the user, the user generally does not perceive the loading process of the digital twin corresponding to the second cuboid. Figure 12 (2) When the user perceives that the digital twin has been successfully loaded. In addition, due to the hierarchical loading method, the amount of data initially loaded in this application is reduced compared to full loading, which can make the first-level digital twin load out faster, and the user-perceived loading speed of the digital twin will also be improved. In summary, the loading speed of the digital twin can be improved through the methods of the embodiments of the present application.

[0146] In addition, based on the method of the embodiment of the present application, the corresponding web application can be generated in real time according to the digital twin model input by the user, which greatly reduces the difficulty of developing the digital twin. After the user has developed the digital twin model, the digital twin model can be uploaded to the loading platform that provides cloud services. Through the loading platform, the corresponding web application can be automatically generated according to the digital twin model, which facilitates the secondary development of the web application or embedding it into other digital twins, reducing the development difficulty. On the other hand, the loading platform can standardize the generation method of the digital twin web application, which is conducive to creating a series of progressive digital twin applications.

[0147] See Figure 13, is a flow chart of a method for loading a digital twin provided in an embodiment of the present application. This method can be based on Figure 3 The digital twin loading platform shown is used to achieve this. The method includes but is not limited to the following steps.

[0148] Step 1301: Receive an input digital twin model, where the digital twin model includes skeleton data and is composed of at least two digital twins of minimum granularity.

[0149] Step 1302: construct a digital twin skeleton based on the skeleton data; the digital twin skeleton includes a sub-skeleton corresponding to each digital twin of the smallest granularity.

[0150] Optionally, the center points of each sub-skeleton are coincident. For example, the execution method of step 1302 can refer to Figure 5 The center point of the sub-skeleton can be referred to as Figure 5 The origin O of the skeleton introduced in .

[0151] Step 1303: construct a virtual camera outside the digital twin skeleton, where the distance between the virtual camera and the center point of the digital twin skeleton is a multiple of the longest distance from the center point to each end face of the digital twin skeleton.

[0152] Step 1304: Radiate rays toward the center point based on the virtual camera, and mark the level of the sub-skeleton to which each end face belongs according to the time sequence of the different end faces of the digital twin skeleton that the ray contacts.

[0153] For example, the execution method of step 1303 and step 1304 can refer to Figure 5 5. Introduction of step 502 in FIG.

[0154] Step 1305: Normalize the annotation results to determine the level of each sub-skeleton.

[0155] For example, the execution method of step 1305 can refer to Figure 5 Introduction to step 503 and step 504 in.

[0156] Step 1306: Generate a configuration file for the digital twin model. The configuration file includes information indicating the loading order of the digital twin skeleton. The loading order of the digital twin skeleton is determined by the level corresponding to each sub-skeleton.

[0157] For example, the execution method of step 1306 can refer to Figure 5 505 in the introduction.

[0158] Step 1307: Based on the configuration file, load each sub-skeleton in sequence.

[0159] For example, the execution method of step 1307 can refer to Figure 10 An introduction to the corresponding embodiment.

[0160] In one possible implementation, constructing a virtual camera outside the digital twin skeleton includes: constructing a sphere with a radius equal to a multiple of the longest distance from the center point to each end face of the digital twin skeleton, and the center point as the center of the sphere; constructing a virtual camera per unit area of ​​the spherical surface of the sphere. For example, this method of constructing a virtual camera can refer to Figure 5 5. Introduction of step 502 in FIG.

[0161] In one possible implementation, constructing a virtual camera outside the digital twin skeleton includes: constructing a sphere with a radius equal to a multiple of the longest distance from the center point to each end face of the digital twin skeleton, and the center point as the center of the sphere; constructing a virtual camera in the projection area of ​​each end face of the digital twin skeleton on the spherical surface of the sphere. For example, this method of constructing a virtual camera can refer to Figure 5 5. Introduction of step 502 in FIG.

[0162] In a possible implementation, the annotation result includes at least one level of each sub-skeleton being annotated; the annotation result is normalized, including: when a sub-skeleton has two or more levels, the level of a sub-skeleton is determined to be the outermost level. For example, the normalization method can refer to Figure 5 503 in the introduction.

[0163] In one possible implementation, based on the configuration file, each sub-skeleton is loaded in sequence, including: loading from the outermost digital twin skeleton to the innermost digital twin skeleton in sequence; and sub-skeletons marked as the same level are re-determined as sub-skeletons of the same level.

[0164] In one possible implementation, the digital twin model also includes texture map data; the method further includes: generating a texture map based on the texture map data, wherein the texture map has a corresponding relationship with the sub-skeleton corresponding to the smallest granularity of the digital twin; compressing each texture map to generate an image placeholder corresponding to each texture map; generating a configuration file for the digital twin model, the configuration file including the data file of each image placeholder, and the configuration file is also used to generate: when loading each sub-skeleton, the image of the surface of the sub-skeleton. This implementation can refer to Figure 9 An introduction to the corresponding embodiments.

[0165] In a possible implementation, the method further includes: generating a blur animation according to each texture map and each image placeholder, the blur animation being used to display the animation effect of loading from each image placeholder to each texture map; and the configuration file also includes a data file of the blur animation. For example, the implementation of this step can refer to Figure 9 When loading each sub-skeleton in sequence, the method further includes: based on the configuration file, loading and displaying the image placeholders corresponding to the sub-skeletons at the same level, loading and displaying the blurred animation corresponding to the sub-skeletons at the same level, and replacing the image placeholders corresponding to the sub-skeletons at the same level with texture maps. For example, the execution method of this step can be referred to Figure 10 An introduction to the corresponding embodiment.

[0166] In a possible implementation, each texture map is compressed to generate a picture placeholder corresponding to each texture map, including: performing lossless compression on each texture map to generate a compressed image; performing lossy compression on the compressed image to generate a picture placeholder corresponding to each texture map. Figure 9 902 and 903 in the introduction.

[0167] See Figure 14 , is a flow chart of a method for loading a digital twin provided in an embodiment of the present application. This method can be based on Figure 3 The digital twin loading platform shown is used to achieve this. The method includes but is not limited to the following steps.

[0168] Step 1401: Receive an input digital twin model, where the digital twin model includes skeleton data and texture map data. The digital twin model is composed of at least two digital twins of minimum granularity.

[0169] For example, the execution method of step 1401 can refer to Figure 4 401 in the introduction.

[0170] Step 1402: Obtain a configuration file of the digital twin model. The configuration file includes information indicating the loading order of the digital twin skeleton of the digital twin model, as well as image placeholders and blur animation data files corresponding to the texture map of the digital twin model.

[0171] The digital twin skeleton includes a sub-skeleton corresponding to each digital twin of the smallest granularity. The sub-skeleton has a corresponding relationship with the texture map. The loading order of the digital twin skeleton is from the outermost sub-skeleton to the innermost sub-skeleton. For example, the execution method of step 1402 can refer to Figure 4Optionally, the configuration file may be generated by the digital twin loading platform based on the input digital twin model, or may be a pre-generated configuration file.

[0172] Step 1403: Based on the skeleton data, texture map data, and configuration file, perform the following operations in the order in which the digital twin skeleton is loaded: load and display the sub-skeletons of the same level and the image placeholders corresponding to the sub-skeletons of the same level; load and display the blurred animation corresponding to the sub-skeletons of the same level, and replace the image placeholders corresponding to the sub-skeletons of the same level with texture maps.

[0173] For example, the execution method of step 1403 can refer to Figure 4 The introduction of step 404 and step 405 in FIG.

[0174] It is understandable that in order to realize the functions in the above embodiments, the loading device of the digital twin (i.e., the digital twin loading platform) includes hardware structures and / or software modules corresponding to the execution of each function. It should be easy for those skilled in the art to realize that, in combination with the units and method steps of each example described in the embodiments disclosed in this application, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application scenario and design constraints of the technical solution.

[0175] See Figure 15 , is a schematic diagram of the structure of a digital twin loading device provided in an embodiment of the present application. The digital twin loading device can be used to implement the functions of the digital twin loading platform in the above method embodiment, thereby also achieving the beneficial effects of the above method embodiment.

[0176] like Figure 15 As shown, the loading device 150 of the digital twin includes a communication unit 1501, a processing unit 1502 and a loading unit 1503. For example, the communication unit 1501 can refer to Figure 3 For the introduction of the communication module in the processing unit 1502, please refer to Figure 3 The skeleton scanning module and texture mapping processing module in the introduction, the loading unit 1503 can refer to Figure 3 Introduction to the application assembly module and display module.

[0177] In some embodiments:

[0178] The communication unit 1501 is used to receive an input digital twin model, where the digital twin model includes skeleton data and is composed of at least two digital twins of minimum granularity.

[0179] The processing unit 1502 is used to perform the following operations: construct a digital twin skeleton based on skeleton data; the digital twin skeleton includes a sub-skeleton corresponding to each digital twin of the smallest granularity; construct a virtual camera outside the digital twin skeleton, and the distance between the virtual camera and the center point of the digital twin skeleton is a multiple of the longest distance from the center point to each end face of the digital twin skeleton; based on the virtual camera, rays are emitted to the center point, and the hierarchy of the sub-skeleton to which each end face belongs is marked according to the time sequence of the different end faces of the digital twin skeleton contacted by the rays; the marking results are normalized to determine the hierarchy of each sub-skeleton; a configuration file of the digital twin model is generated, and the configuration file includes information indicating the loading order of the digital twin skeleton, and the loading order of the digital twin skeleton is determined by the hierarchy corresponding to each sub-skeleton.

[0180] The loading unit 1503 is used to load each sub-skeleton in sequence based on the configuration file.

[0181] In one possible implementation, when the processing unit 1502 is used to construct a virtual camera outside the digital twin skeleton, it specifically includes: constructing a sphere with a radius that is a multiple of the longest distance from the center point to each end face of the digital twin skeleton, and the center point as the center of the sphere; constructing a virtual camera per unit area of ​​the spherical surface of the sphere.

[0182] In one possible implementation, the processing unit 1502 is used to construct a virtual camera outside the digital twin skeleton, specifically including: constructing a sphere with a radius that is a multiple of the longest distance from the center point to each end face of the digital twin skeleton, and the center point as the center of the sphere; and constructing a virtual camera in the projection area of ​​each end face of the digital twin skeleton on the spherical surface of the sphere.

[0183] In one possible implementation, the annotation result includes at least one level at which each sub-skeleton is annotated; when the processing unit 1502 is used to normalize the annotation result, it specifically includes: when a sub-skeleton has two or more levels, the level of a sub-skeleton is determined as the outermost level.

[0184] In one possible implementation, the loading unit 1503 is used to load each sub-skeleton in sequence based on the configuration file, specifically including: loading from the outermost digital twin skeleton to the innermost digital twin skeleton in sequence; and re-determining the sub-skeletons marked as the same level as the sub-skeletons of the same level.

[0185] In one possible implementation, the digital twin model also includes texture map data; the processing unit 1502 is also used to: generate a texture map based on the texture map data, wherein the texture map has a corresponding relationship with the sub-skeleton corresponding to the smallest granularity of the digital twin; compress each texture map to generate an image placeholder corresponding to each texture map; generate a configuration file for the digital twin model, the configuration file includes a data file for each image placeholder, and the configuration file is also used to generate: an image of the surface of the sub-skeleton when loading each sub-skeleton.

[0186] In one possible implementation, the processing unit 1502 is also used to: generate a blurred animation based on each texture map and each image placeholder, and the blurred animation is used to display the animation effect loaded from each image placeholder to each texture map; the configuration file also includes a data file of the blurred animation; when the loading unit 1503 is used to load each sub-skeleton in turn, it is also used to: based on the configuration file, load and display the image placeholder corresponding to the sub-skeletons of the same level, load and display the blurred animation corresponding to the sub-skeletons of the same level, and replace the image placeholder corresponding to the sub-skeletons of the same level with the texture map.

[0187] In one possible implementation, the processing unit 1502 is used to compress each texture map and generate an image placeholder corresponding to each texture map, specifically including: performing lossless compression on each texture map to generate a compressed image; performing lossy compression on the compressed image to generate an image placeholder corresponding to each texture map.

[0188] In other embodiments:

[0189] The communication unit 1501 is used to receive an input digital twin model, where the digital twin model includes skeleton data and texture map data, and the digital twin model is composed of at least two digital twins of minimum granularity.

[0190] Processing unit 1502 is used to obtain a configuration file of the digital twin model, which includes information indicating the loading order of the digital twin skeleton of the digital twin model, as well as the image placeholder and blurred animation data file corresponding to the texture map of the digital twin model; wherein the digital twin skeleton includes a sub-skeleton corresponding to each digital twin of the smallest granularity, and the sub-skeleton has a corresponding relationship with the texture map. The loading order of the digital twin skeleton is from the outermost sub-skeleton to the innermost sub-skeleton in sequence.

[0191] Loading unit 1503 is used to perform the following operations in sequence according to the skeleton data, texture map data and configuration file in the loading order of the digital twin skeleton: load and display the sub-skeletons of the same level and the image placeholders corresponding to the sub-skeletons of the same level; load and display the blurred animation corresponding to the sub-skeletons of the same level, and replace the image placeholders corresponding to the sub-skeletons of the same level with texture maps.

[0192] For a more detailed description of the communication unit 1501, the processing unit 1502 and the loading unit 1503, please refer to Figure 4-Figure 14 The relevant descriptions in the method embodiment shown are not repeated here.

[0193] This application also provides a server for executing any of the above method embodiments. For the convenience and brevity of description, the specific working process of the server can refer to the corresponding process executed by the digital twin loading platform in the above method embodiments, and will not be repeated here.

[0194] The present application also provides a computer-readable medium having a computer program stored thereon, which implements the functions of any of the above method embodiments when executed by a computer.

[0195] The present application also provides a computer program product, which implements the functions of any of the above method embodiments when executed by a computer.

[0196] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments may be implemented in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated therein. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a high-density digital video disc (DVD)), or a semiconductor medium (eg, a solid state drive (SSD)).

[0197] It will be understood that the “embodiment” mentioned throughout the specification means that the specific features, structures or characteristics related to the embodiment are included in at least one embodiment of the present application. Therefore, the various embodiments in the entire specification do not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It will be understood that in the various embodiments of the present application, the size of the sequence numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0198] It can be understood that in this application, "when", "if" and "if" all mean that the device will perform corresponding processing under certain objective circumstances, and do not limit the time. It does not require that the device must perform a judgment action when it is implemented, nor does it mean that there are other limitations.

[0199] The same or similar parts between the various embodiments in this application can refer to each other. In the various embodiments in this application, and the various implementation methods / implementation methods / implementation methods in each embodiment, if there is no special explanation and logical conflict, the terms and / or descriptions between different embodiments and the various implementation methods / implementation methods / implementation methods in each embodiment are consistent and can be referenced to each other. The technical features in different embodiments and the various implementation methods / implementation methods / implementation methods in each embodiment can be combined to form new embodiments, implementation methods, implementation methods, or implementation methods according to their inherent logical relationships. The above-described implementation methods of this application do not constitute a limitation on the scope of protection of this application.

[0200] The above is only a specific implementation method of the present application, but the scope of protection of the present application is not limited thereto. Any technician familiar with this technical field can easily think of changes or replacements within the technical scope disclosed in this application, which should be covered by the scope of protection of the present application.

Claims

1. A method for loading a digital twin, characterized in that: The method comprises: Receive an input digital twin model, the digital twin model including skeleton data, the digital twin model consisting of at least two digital twins of minimum granularity; Constructing a digital twin skeleton based on the skeleton data; the digital twin skeleton includes a sub-skeleton corresponding to each digital twin of the smallest granularity; Determining the hierarchy of each sub-skeleton; comprising: constructing a virtual camera outside the digital twin skeleton, wherein the distance between the virtual camera and the center point of the digital twin skeleton is a multiple of the longest distance from the center point to each end face of the digital twin skeleton; emitting rays toward the center point based on the virtual camera, marking the hierarchy of the sub-skeleton to which each end face belongs according to the time sequence of different end faces of the digital twin skeleton contacted by the rays, and normalizing the marking results to thereby determine the hierarchy of each sub-skeleton; Generate a configuration file of the digital twin model, the configuration file including information indicating a loading order of the digital twin skeleton, where the loading order of the digital twin skeleton is determined by the level corresponding to each of the sub-skeletons; Based on the configuration file, each of the sub-skeletons is loaded in sequence; The digital twin model also includes texture map data; the method further includes: generating a texture map based on the texture map data, wherein the texture map corresponds to a sub-skeleton corresponding to the digital twin of the smallest granularity; Performing compression processing on each of the texture maps to generate an image placeholder corresponding to each of the texture maps; Generate a configuration file of the digital twin model, the configuration file including a data file of each of the image placeholders, the configuration file further configured to generate: an image of a surface of each of the sub-skeletons when loading the sub-skeletons; generating a blurred animation according to each of the texture maps and each of the image placeholders, wherein the blurred animation is used to display an animation effect loaded from each of the image placeholders to each of the texture maps; The configuration file also includes the data file of the blurred animation; When loading each of the sub-skeletons in sequence, the method also includes: based on the configuration file, loading and displaying the image placeholder corresponding to the sub-skeleton at the same level, loading and displaying the blurred animation corresponding to the sub-skeleton at the same level, and replacing the image placeholder corresponding to the sub-skeleton at the same level with the texture map.

2. The method according to claim 1, characterized in that The constructing of a virtual camera outside the digital twin skeleton includes: A sphere is constructed with the center point as the center, using the multiple of the longest distance from the center point to each end face of the digital twin skeleton as the radius; A virtual camera is constructed per unit area of ​​the spherical surface of the sphere.

3. The method according to claim 1, characterized in that The constructing of a virtual camera outside the digital twin skeleton includes: A sphere is constructed with the center point as the center, using the multiple of the longest distance from the center point to each end face of the digital twin skeleton as the radius; A virtual camera is constructed in the projection area of ​​each end face of the digital twin skeleton on the spherical surface of the sphere.

4. The method according to any one of claims 1 to 3, characterized in that The annotation result includes at least one level at which each sub-skeleton is annotated; The normalizing process of the annotation results includes: When a sub-skeleton has two or more levels, the level of the sub-skeleton is determined as the outermost level.

5. The method according to any one of claims 1 to 3, characterized in that Based on the configuration file, each of the sub-skeletons is loaded in sequence, including: loading from the outermost digital twin skeleton to the innermost digital twin skeleton in sequence; Sub-skeletons marked as being at the same level are redefined as sub-skeletons at the same level.

6. The method according to claim 1, characterized in that Compressing each of the texture maps to generate an image placeholder corresponding to each of the texture maps includes: Performing lossless compression on each of the texture maps to generate a compressed image; Lossy compression is performed on the compressed image to generate an image placeholder corresponding to each texture map.

7. A server, characterized in that: The server is used to execute the method according to any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions. When the program instructions are executed, the method according to any one of claims 1 to 6 is executed.

Images