Construction method, display method and training method of digital twinborn body and computing equipment

Abstract

The embodiment of the invention discloses a construction method, a display method and a training method of a digital twinborn body and computing equipment, relates to the technical field of image processing, and at least can improve the construction efficiency of the digital twinborn body. The method comprises the following steps: generating a 3D model and a first UV map of an object according to a multimedia file of the object; the UV chartlet is used for reflecting surface features of the object; the first UV map is configured onto the 3D model, generating a digital twin of the object.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of image processing technology, and in particular, to a method for constructing a digital twin, a display method, a training method, and a computing device. Background Art

[0002] With the continuous development of computer simulation technology, in order to visually display computing devices including servers, switches, and cabinets on a computer, a digital twin of a simulated physical device can be displayed on the computer by creating a three-dimensional (3D) model of the physical device.

[0003] Currently, it is often the case that new devices (such as servers, power supplies, switches, routers, etc.) need to be added to an existing digital twin scenario for display. Since new devices may not have corresponding 3D models (or the manufacturers of new devices do not disclose 3D models), modelers need to independently build models for new devices to obtain digital twins, which consumes human resources and time and has low modeling efficiency. Summary of the Invention

[0004] The embodiments of the present application provide a method for constructing a digital twin, a display method, a training method, and a computing device, which can at least improve the construction efficiency of the digital twin.

[0005] In a first aspect, the embodiments of the present application provide a method for constructing a digital twin, the method including: generating a 3D model of an object and a first UV map according to a multimedia file of the object; the UV map is used to display the surface features of the object; configuring the first UV map onto the 3D model to generate a digital twin of the object.

[0006] The method provided by the embodiments of the present application does not rely on the process of manual modeling. By obtaining the multimedia file of the object, the server can obtain information such as the skeleton and surface features required for constructing the digital twin from the multimedia file, thereby realizing the automated construction of the digital twin without excessive manual participation, effectively improving the modeling efficiency while reducing the waste of human and material resources. Among them, the 3D model is used to represent the skeleton of the object.

[0007] In a possible implementation manner, the method further includes: obtaining modification information; the modification information is used to indicate the change of the second UV map relative to the first UV map; adjusting the first UV map according to the modification information to obtain a second UV map; the surface features of the object displayed by the second UV map are different from those of the first UV map; configuring the second UV map onto the 3D model to generate an updated digital twin of the object.

[0008] In a possible implementation, the first UV map is adjusted according to the modification information to obtain a second UV map, including: inputting the first UV map and the modification information into a first model to obtain the second UV map output by the first model.

[0009] It can be understood that the embodiments of the present application adjust the UV map. The UV map is a two-dimensional image and is relatively simple to adjust. Moreover, the embodiments of the present application adopt a first model, which can realize obtaining the modified UV map according to the modification information, and then generate a digital twin that meets the user's needs, thereby effectively reducing the technical threshold requirements for users in the process of modifying the 3D model.

[0010] In a possible implementation, the 3D model is a 3D mesh model, including multiple mesh units in a three-dimensional space; the number of mesh units is used to characterize the display accuracy of the 3D mesh model. Generating a 3D model of an object according to the multimedia file of the object includes: generating a point cloud model of the object based on the multimedia file of the object; generating a 3D mesh model of the object based on the point cloud model; the display accuracy of the 3D mesh model is lower than that of the point cloud model. It should be understood that by providing the steps of obtaining the 3D mesh model, the feasibility of the present solution is improved.

[0011] In a possible implementation, generating a 3D model of an object based on the point cloud model includes: inputting the point cloud model into a second model to obtain the output 3D mesh model of the object. It should be understood that the conversion from the point cloud model to the 3D mesh model is realized through the inference of the second model. Compared with the traditional method of connecting lines to the point cloud to construct a 3D mesh model using an algorithm, the efficiency of obtaining the 3D mesh model is higher, thereby effectively improving the construction efficiency of the digital twin.

[0012] In a possible implementation, the number of mesh units on the surface of the 3D mesh model matches the target display accuracy of the digital twin; the target display accuracy is determined according to the application scenario of the object, and the display accuracies corresponding to the 3D mesh models of different objects in the same application scenario are the same.

[0013] In a possible implementation, generating a first UV map of an object according to the multimedia file of the object includes: obtaining the material information of the object; the material information is used to characterize the color information and / or light reflection intensity information of each pixel point on the surface of the object; rendering the surface of the 3D model based on the material information; obtaining the first UV map based on the two-dimensional unfolded view corresponding to the rendered 3D model. It should be understood that by providing the steps of obtaining the UV map, the feasibility of the present solution is improved.

[0014] In a possible implementation, the surface features include at least one of the following: the appearance parameters of the object; the components included in the object; the identification information of the components included in the object.

[0015] In a second aspect, an embodiment of the present application provides a display method, the method comprising: displaying a digital twin of the object; the digital twin includes a first UV map, and the first UV map is used to display the surface features of the object; in response to an adjustment operation on the first UV map, displaying the updated digital twin of the object, the updated digital twin of the object includes a second UV map; the second UV map is different from the surface features reflected by the first UV map; the second UV map is used to be configured on the 3D model of the object to obtain the updated digital twin of the object.

[0016] It should be understood that, in the embodiment of the present application, by displaying the first UV map and according to the adjustment of the first UV map by the user, the adjustment of the digital twin is realized. Compared with the way of directly adjusting the 3D model by the user, the way of adjusting the two-dimensional image is relatively simple, the technical threshold requirement for the user is relatively low, and the user can achieve the purpose of modifying the digital twin by performing simple operations (such as painting with a brush or inputting text description).

[0017] In a possible implementation manner, the adjustment operation includes at least one of the following: an input operation of text description information, and the text description information is used to describe the change of the second UV map relative to the first UV map; an editing operation on the first UV map, and the editing operation is used to edit the content included in the first UV map.

[0018] In a possible implementation manner, the text description information or the editing operation is used to adjust at least one of the following: the appearance parameters of the object; the components included in the object; the identification information of the components included in the object.

[0019] In a third aspect, an embodiment of the present application provides a training method, the method comprising: obtaining a first training data set; each pair of training data in the first training data set includes an initial UV map of the object, modification information, and a target UV map obtained by adjusting the initial UV map according to the modification information; wherein, the UV map is used to display the surface features of the object; the modification information is used to indicate the change of the target UV map relative to the initial UV map; using the initial UV map and the modification information as sample data, and using the target UV map as the label of the sample data to train a first initial model to obtain a first model.

[0020] Fourthly, an embodiment of the present application provides a training method, including: obtaining a second training dataset; each pair of training data in the second training dataset includes a point cloud model of an object and a 3D mesh model of the object; using the point cloud model as sample data and the 3D mesh model as the label of the sample data to train a second initial model to obtain a second model.

[0021] Fifthly, an embodiment of the present application provides a construction device for a digital twin, and the construction device for the digital twin is used to execute any one of the digital twin construction methods provided in the first aspect above.

[0022] Sixthly, an embodiment of the present application provides a display device, and the display device is used to execute any one of the display methods provided in the second aspect above.

[0023] Seventhly, an embodiment of the present application provides a training device, and the training device is used to execute any one of the training methods provided in the third aspect or the fourth aspect above.

[0024] Eighthly, an embodiment of the present application provides a computing device, which includes a processor and a memory, and the processor is coupled to the memory; the memory is used to store computer instructions, and the computer instructions are loaded and executed by the processor so that the computing device implements the methods described in the above aspects.

[0025] Ninthly, an embodiment of the present application provides a computer-readable storage medium, in which at least one computer program instruction is stored, and the computer program instruction is loaded and executed by a processor to implement the methods described in the above aspects.

[0026] Tenthly, an embodiment of the present application provides a computer program product, which includes computer instructions stored in a computer-readable storage medium. The processor of the computing device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions so that the computing device executes the methods provided in the various optional implementations of the above aspects.

[0027] For the beneficial effects of the fifth to tenth aspects and their various implementations in the embodiments of the present application, reference can be made to the analysis of the beneficial effects in the first to fourth aspects and their various implementations, which will not be elaborated here.

[0028] These aspects or other aspects of the embodiments of the present application will be more clearly understood in the following description. Description of the Drawings

[0029] Figure 1 It is a schematic diagram of an application scenario provided by an embodiment of the present application;

[0030] Figure 2 Schematic diagram of the hardware structure of a server provided by an embodiment of the present application;

[0031] Figure 3 Schematic diagram of the composition of a digital twin system provided by an embodiment of the present application;

[0032] Figure 4 Schematic flow diagram of a method for constructing a digital twin provided by an embodiment of the present application;

[0033] Figure 5 Schematic diagram of a 3D mesh model provided by an embodiment of the present application;

[0034] Figure 6 Schematic comparison diagram of 3D mesh models with different precisions provided by an embodiment of the present application;

[0035] Figure 7 Schematic flow diagram of another method for constructing a digital twin provided by an embodiment of the present application;

[0036] Figure 8 Schematic flow diagram of yet another method for constructing a digital twin provided by an embodiment of the present application;

[0037] Figure 9 Schematic 3D diagram of a server provided by an embodiment of the present application;

[0038] Figure 10 Schematic diagram of a UV map provided by an embodiment of the present application;

[0039] Figure 11 Schematic flow diagram of yet another method for constructing a digital twin provided by an embodiment of the present application;

[0040] Figure 12 Schematic flow diagram of a display method provided by an embodiment of the present application;

[0041] Figure 13 Schematic diagram of the modification of a UV map provided by an embodiment of the present application;

[0042] Figure 14 Schematic flow diagram of a training method provided by an embodiment of the present application;

[0043] Figure 15 Schematic diagram of the structure of a digital twin construction device provided by an embodiment of the present application;

[0044] Figure 16 Schematic diagram of the structure of a display device provided by an embodiment of the present application;

[0045] Figure 17Structural schematic diagram of a first training device provided by an embodiment of the present application;

[0046] Figure 18 Structural schematic diagram of a second training device provided by an embodiment of the present application. Detailed implementation manners

[0047] To make the objectives, technical solutions, and advantages of the present application clearer, the following will further describe the embodiments of the present application in detail with reference to the accompanying drawings.

[0048] As used herein, "a plurality of" means two or more. "And / or" describes the association relationship of associated objects and indicates that three relationships may exist. For example, A and / or B may represent: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally represents an "or" relationship between the associated objects before and after.

[0049] Moreover, in the description of the present application, unless otherwise specified, "a plurality of" means two or more than two. "At least one (item)" or similar expressions thereof refer to any combination of these items, including any combination of single item (s) or plural item (s). For example, at least one (item) of a, b, or c may represent: a, b, c, a - b, a - c, b - c, or a - b - c, where a, b, c may be single or multiple.

[0050] In addition, to facilitate a clear description of the technical solutions of the embodiments of the present application, in the embodiments of the present application, terms such as "first" and "second" are used to distinguish the same items or similar items with basically the same functions and roles. Those skilled in the art can understand that the terms "first", "second", etc. do not limit the quantity and execution order, and the terms "first", "second", etc. do not necessarily mean different. At the same time, in the embodiments of the present application, words such as "exemplary" or "for example" are used to represent examples, illustrations, or explanations. Any embodiment or design solution described as "exemplary" or "for example" in the embodiments of the present application should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Exactly speaking, using words such as "exemplary" or "for example" aims to present relevant concepts in a specific manner for easy understanding.

[0051] First, an exemplary introduction to the technical terms involved in the embodiments of the present application is provided.

[0052] 1. Digital Twin: It can be a simulation process that makes full use of data such as physical models, sensor updates, and operation history, integrates multiple disciplines, physical quantities, scales, and probabilities, and completes mapping in the virtual space to reflect the entire life cycle process of the corresponding physical equipment. Digital Twin is a concept beyond reality and can be regarded as a digital mapping system of one or more important and interdependent equipment systems.

[0053] 2. Digital Twin Body: It can be a digital "clone" created based on a device or system, which can be a virtual representation of the real world including physical objects, processes, relationships, and behaviors. The Digital Twin Body Skeleton can refer to the geometric bodies of each Digital Twin Body in the Digital Twin system. In geometry, a finite form enclosed by several geometric surfaces (plane or curved surfaces) can be called a geometric body. The surfaces enclosing the geometric body are called the interfaces or surfaces of the geometric body. The intersection lines of different interfaces are called the edge lines of the geometric body, and the intersection points of different edge lines are called the vertices of the geometric body. A geometric body can also be regarded as a finite space region segmented by several geometric surfaces in space.

[0054] 3. 3D Model: It is a polygonal representation of an object, usually displayed using a computer or other video device. The displayed object can be an entity in the real world or a fictional object. Anything that exists in the physical nature can be represented by a three-dimensional model.

[0055] 4. Digital Twin Scene: It is a virtual 3D scene composed of multiple Digital Twin Bodies and used to simulate the physical world. For example, the Digital Twin Scene can be a computer room, which includes the virtual representations of the styles and distribution positions of each cabinet. Another example is that the Digital Twin Scene can be a community, which includes the styles and distribution positions of each building.

[0056] 5. Diffusion Model: Among them, the diffusion model is a generative model based on probability theory. Its core idea is to simulate a gradual "diffusion" process from the data distribution to a simple noise distribution, and then learn the inverse process to reconstruct high-quality data samples from the noise. This process can be divided into two main stages: 1. Forward Process: Gradually add noise to the data to generate a series of data samples that gradually become noise. Each step in this process is conditionally independent and gradually approaches the Gaussian noise distribution. 2. Reverse Process: Start from pure noise and gradually "denoise" through a series of inverse steps to finally generate samples close to the original data distribution.

[0057] Among them, in the reverse process, the UNet network architecture can be adopted. The UNet network is an architecture specifically designed for image segmentation tasks. In image denoising tasks, the UNet network is used to learn to recover the original image from a noisy image. In addition, UNet is also a conditional denoising network, that is, it can not only remove the noise in the image, but also guide the denoising process according to the given label to generate an image that matches the label.

[0058] 6. UV mapping: It is a technology that maps a 2D texture image onto the surface of a 3D model in a 3D model. "UV" represents the texture mapping coordinates, similar to the X, Y, and Z axes of a spatial model. UV mapping makes the texture correctly map to each part of the model by precisely corresponding each point on the image to the surface of the model object, and ensures that the texture presents the correct shape and details during rendering.

[0059] 7. Web Graphics Library (WebGL) technology: WebGL is a technology for rendering interactive 2D (2 dimensions) and 3D graphics in any compatible web browser without using plugins. WebGL can be fully integrated into all web standards of the browser, and can use the GPU acceleration method of image processing and effects as part of the web canvas (canvas). WebGL elements can be added to other Hypertext Markup Language (HTML) elements and mixed with other parts of the web page or web page background. A WebGL program can consist of a handle written in JavaScript and shader code written in OpenGL Shading Language (GLSL) and execute on the GPU of a computing device.

[0060] 8. Web 3D technology: Web3D can refer to a method of displaying 3D graphics via a web browser.

[0061] The embodiment of this application provides a method for constructing a digital twin body, and a corresponding digital twin body is modeled according to the multimedia file of the object. It does not need to rely on the manual modeling process of a modeler, reduces the consumption of human and material resources, and effectively improves the modeling efficiency.

[0062] In one implementation, a 3D model of an object and a first UV map are generated based on a multimedia file. The UV map is a two-dimensional representation of the surface of the object and is used to reflect the surface characteristics of the object. Further, the first UV map is configured onto the 3D model to obtain a digital twin of the object, completing the modeling process. It can be seen that the method provided by the embodiments of the present application does not rely on the process of manual modeling. By obtaining the multimedia file of the object, the server can obtain information such as the skeleton and surface characteristics required for constructing the digital twin from the multimedia file, thereby realizing the automated construction of the digital twin without excessive manual participation, effectively improving the modeling efficiency while reducing the waste of human and material resources.

[0063] Figure 1 FIG. shows a schematic diagram of an application scenario provided by the embodiments of the present application. As Figure 1 shown, it includes a terminal device 100 and a server 110. The terminal device and the server can be collectively referred to as computing devices. The method for constructing a digital twin or the training method provided by the embodiments of the present application can be executed by the server 110. The display method provided by the embodiments of the present application can be executed by the terminal device 100.

[0064] Among them, the terminal device 100 and the server 110 can interact through a communication network. The communication network can be an Ethernet, a wireless access network, a wireless local area network (WLAN), etc.

[0065] Among them, the terminal device 100 is a device with an interface display function and is used to display the display interface of the digital twin.

[0066] Optionally, the terminal device 100 can be a personal computer, a smart phone, a tablet computer, an e-book reader, a portable computer, or other devices.

[0067] The server 110 deploys a digital twin system. The digital twin system is used to generate a digital twin of an object based on the multimedia file of the object (such as a video or an image), and send the digital twin to the terminal device 100 and display it to the user through the terminal device 100, so that the user can view and modify the digital twin.

[0068] Optionally, in terms of form, the server 110 mentioned here can be a blade server, a high-density server, a rack server, or an all-in-one cabinet server; in terms of function, the server 110 can be a general server, a graphics processing unit (GPU) server, an artificial intelligence (AI) server, etc.

[0069] Figure 2This is a schematic diagram of the hardware structure of a server provided by an embodiment of the present application. As Figure 2 shown, the server may include: a processor and a memory; the memory stores program instructions for the server to execute a certain step described in the following embodiments; when the processor is configured to run the program instructions, the server implements the method for constructing a digital twin in the embodiments of the present application.

[0070] It should be noted that the system architecture and application scenarios described in the embodiments of the present application are for more clearly explaining the technical solutions of the embodiments of the present application, and do not constitute a limitation to the technical solutions provided by the embodiments of the present application. Those skilled in the art know that with the evolution of the system architecture and the emergence of new business scenarios, the technical solutions provided by the embodiments of the present application are equally applicable to similar technical problems.

[0071] For the convenience of understanding, the following provides an exemplary introduction to the method for constructing a digital twin provided by the embodiments of the present application in conjunction with the accompanying drawings.

[0072] Figure 3 This is a schematic diagram of the composition of a digital twin system provided by an embodiment of the present application. As Figure 3 shown, it includes a sensing module, a generating module, and an enhancing module.

[0073] Among them, the sensing module is used to analyze the 3D model and material information of the object according to the multimedia file of the object (including video and / or image, etc.). Among them, the material information is used to characterize the color information and / or light reflection intensity information of each pixel point on the surface of the object.

[0074] The generating module is used to render the surface of the 3D model with the material information obtained by the sensing module to obtain the rendered 3D model, and then perform two-dimensional unfolding to obtain the first UV map. For the specific process, refer to the description of the corresponding Figure 4 embodiment below, and no detailed description will be given here.

[0075] The enhancing module is used to provide a display interface for the user, and modify the display style of the first UV map (such as the appearance parameters of the object, the components included, etc.) according to the modification information input by the user to obtain the second UV map. The second UV map is different from the surface characteristics of the object displayed by the first UV map. The second UV map obtained by the enhancing module can be used to update the digital twin to obtain a stylistically modified digital twin to meet the personalized needs of the user. For the specific modification process, refer to the description of the corresponding Figure 4 embodiment below, and no detailed description will be given here.

[0076] Figure 4The flowchart shows the construction method of the digital twin provided by an exemplary embodiment of the present application. The construction method of the digital twin is executed by a server and includes the following steps:

[0077] S401. Generate a 3D model and a first UV map of the object according to the multimedia file of the object.

[0078] Here, the object mentioned can be an entity device such as a server, a switch, or a router. In the embodiments of the present application, the server is taken as an example for illustration.

[0079] Among them, the multimedia file can be a video or an image including the object. Optionally, a multimedia file including multiple display angles of the object can be used to improve the accuracy of modeling.

[0080] Among them, the 3D model is a three-dimensional representation of the skeleton of the object. The UV map is a two-dimensional representation of the surface of the object, used to reflect the surface characteristics of the object, such as color, texture, pattern, light intensity, etc.

[0081] In a possible implementation manner, the server can obtain the multimedia file in the following two ways. 1. The server can download the display video or image of the object from the network or other places to obtain the multimedia file. 2. The user can take pictures of the object at one or more angles (such as a panoramic 360 degrees) with a camera to obtain a video or an image as the multimedia file and input it into the server.

[0082] It should be noted that the specific steps for generating the 3D model can be referred to the description of the corresponding embodiment below. Figure 5 The specific steps for generating the first UV map can be referred to the description of the corresponding embodiment below. Figure 6 The description of the corresponding embodiment.

[0083] S402. Configure the first UV map on the 3D model to generate the digital twin of the object.

[0084] In the embodiments of the present application, after the server obtains the 3D model and the first UV map respectively, it can align the coordinates of each point on the first UV map with the coordinates of the corresponding point on the surface of the 3D model to implement configuring the UV map on the 3D model, thereby obtaining a digital twin with rich surface details and a realistic appearance. The specific configuration process can be implemented based on 3D modeling tools, and the embodiments of the present application do not make specific limitations on this.

[0085] It should be noted that the 3D model and the UV map are interdependent. Without the UV map, the 3D model will lack texture and details and look dull. Without the 3D model, the UV map cannot play its role. In the embodiments of the present application, a digital twin is constructed by generating a 3D model and a UV map. On the one hand, the UV map can accurately reflect the details of the object surface, resulting in a more realistic digital twin. On the other hand, users can modify the digital twin through the UV map without directly modifying the digital twin itself, which can reduce the technical threshold requirements for users.

[0086] The following will describe the steps of S401 in combination with specific embodiments and the accompanying drawings of the specification.

[0087] In a possible implementation manner, the above 3D model is a 3D mesh model, including a plurality of mesh units in a three-dimensional space. The number of mesh units is used to characterize the display accuracy of the 3D mesh model. Specifically, the 3D mesh model is a three-dimensional geometric shape composed of a series of polygons (usually triangles). These polygons are connected to each other through vertices, edges, and faces, jointly constituting a 3D model for reflecting the external shape characteristics of an object. For example, Figure 5 is a schematic diagram of a 3D mesh model provided by an embodiment of the present application. For example, Figure 6 is a comparative schematic diagram of 3D mesh models with different precisions provided by an embodiment of the present application. As Figure 6 shown, the number of mesh units of the 3D mesh model on the left is more than that of the 3D mesh model on the right. Therefore, the display accuracy of the 3D mesh model on the left is higher than that of the 3D mesh model on the right.

[0088] Generating the 3D mesh model in the above S401, as Figure 7 shown, can be specifically implemented as S501 - S502:

[0089] S501. Generate a point cloud model of the object based on the multimedia file of the object.

[0090] Among them, the point cloud model refers to a large set of points expressing the distribution of the object in space and the characteristics of the target surface under the same spatial coordinate system. It can be understood as corresponding to the digital twin with the highest precision. Each point includes three-dimensional coordinate information (such as X, Y, Z), and each point can also include color information or light reflection intensity information.

[0091] It should be noted that in the related art, there is a way to obtain the production model of an object as the digital twin. Since the digital twin is generally used to display the appearance, but the production model generally also includes internal details, this will lead to an overly large model. Directly adding the production model to the digital twin scene will cause the scene to freeze and affect the user experience. In contrast, the embodiment of the present application models the surface of the object without caring about the internal structure of the object, and can achieve rapid modeling to generate a digital twin that meets the requirements of the digital twin scene.

[0092] In the embodiment of the present application, a neural radiance fields (NeRF) model can be used to perceive an object from a multimedia file to obtain a point cloud model. The NeRF model is a technology for view synthesis that can extract information such as the geometric shape, texture, and lighting of a target object from images taken from multiple perspectives, and uses a multilayer perceptron (MLP) to model the radiation field, thereby enabling the generation of realistic new perspective images. The NeRF model uses a multilayer perceptron (MLP) structure and enhances high-frequency information (the change in details between adjacent spatial positions, such as texture changes) through position encoding (the three-dimensional coordinates of a point in space) to improve the rendering details of the generated images.

[0093] The process of constructing a point cloud model based on the NeRF model is as follows:

[0094] 1. The multimedia file includes information about multiple angles of the object, as well as the parameter information of the corresponding camera when shooting each angle (including the pose during shooting, the set focal length, etc.). For each angle's image (when the multimedia file is a video, multiple images can be intercepted from the video), the server can construct a vector representation of the light based on the pose and focal length. Here, the light refers to the simulated light that starts from the camera and passes through each pixel point in the image. The vector representation of the light can be obtained by using the camera's projection matrix (used to convert the camera coordinate system to a three-dimensional coordinate system, which can be determined according to the camera's focal length and pose) to convert the two-dimensional coordinates of each pixel point on the image into three-dimensional coordinates.

[0095] 2. Further, after obtaining the light passing through each pixel point, for each pixel point, the server can uniformly sample multiple sampling points on the light corresponding to each pixel point. Each sampling point has three-dimensional coordinates and the direction attribute of the belonging light.

[0096] 3. The server inputs the three-dimensional coordinates of each sampling point and the direction attribute of the corresponding light into the NeRF model to obtain the color value and volume density of each output sampling point. Among them, the color value represents the numerical values of the red, green, and blue (RGB) three channels, and the volume density represents the material distribution or the degree of occupying space at this sampling point in the scene.

[0097] 4. Finally, according to the volume density of each sampling point, the points with a volume density greater than a certain threshold are selected (the greater the volume density, the more likely the sampling point is located on the surface of the object in the scene), and the point cloud data of the object is obtained. Based on this point cloud data (including the three-dimensional coordinates of each point in the 3D model corresponding to the object, as well as information such as the color and light intensity of each point), a point cloud model is constructed.

[0098] It should be noted that in order to enable the NeRF model to perceive the object more accurately, the NeRF model can be trained iteratively multiple times. The training process is as follows:

[0099] a. Obtain multiple training images and the parameter information of the corresponding cameras. The training images are obtained by taking pictures of the physical device at a certain angle.

[0100] b. For each training image, construct rays passing through each pixel point and sample multiple sampling points on the rays.

[0101] c. Input the three-dimensional coordinates of each sampling point and the corresponding ray direction into the NeRF model to obtain the color value and volume density of each sampling point.

[0102] d. For multiple sampling points on the same ray, use the volume rendering formula to integrate the color values and volume densities of the multiple sampling points to obtain the color of the pixel point passed through by the ray. Combining the distribution positions of each pixel point in the image and the colors of each pixel point, a two-dimensional image generated by the NeRF model is obtained.

[0103] e. Compare the two-dimensional image generated by the NeRF model with the input training image, and determine the loss value through the mean-square error (MSE). Then, based on the loss value, optimize the model parameters (or neural network weights) of the NeRF model. Repeat the above process of c - e until the difference between the generated two-dimensional image and the input training image is minimized, and the training of the NeRF model is completed. The trained NeRF model can accurately reconstruct the 3D scene.

[0104] S502. Based on the point cloud model, generate a 3D mesh model of the object.

[0105] Among them, the display accuracy of the 3D mesh model is lower than that of the point cloud model.

[0106] In the embodiments of the present application, the server can generate a 3D mesh model of the object based on the point cloud data of each point in the point cloud model. It should be understood that the point cloud model represents the surface of the object in the form of points, and the 3D mesh model represents the surface of the object with surfaces composed of points. Therefore, the display accuracy of the point cloud model is higher than that of the 3D mesh model.

[0107] In some embodiments, the server may convert the point cloud model into a 3D mesh model based on a grid generation algorithm (such as a ball rotation algorithm, a Poisson reconstruction algorithm, etc.).

[0108] In other embodiments, a second model is deployed on the server. The server may input the point cloud model into the second model to obtain a 3D mesh model of the output object.

[0109] It should be understood that the conversion of the point cloud model into a 3D mesh model through the inference of the second model has a higher efficiency of obtaining the 3D mesh model compared with the traditional method of connecting points of the point cloud to construct a 3D mesh model by an algorithm, thereby effectively improving the construction efficiency of the digital twin.

[0110] In some embodiments, the number of mesh units on the surface of the 3D mesh model matches the target display accuracy of the digital twin. Among them, the target display accuracy is determined according to the application scenario of the object, and the display accuracies corresponding to the 3D mesh models of different devices in the same application scenario are the same.

[0111] In some embodiments, the second model further includes a linear projection layer. Among them, in the field of machine learning, the linear projection layer can map data from one space to another through a linear transformation. In the embodiments of the present application, the role of the linear projection layer is to map a 3D mesh model of one accuracy to obtain a 3D mesh model of another accuracy. That is, the server may determine the target display accuracy according to the application scenario of the object, and then input the parameter representation corresponding to the target display accuracy and the point cloud model into the second model to obtain a 3D mesh model that matches the target display accuracy.

[0112] In a possible implementation manner, generating the first UV map in S401 above, as Figure 8 shown, may be specifically implemented as:[[]]

[0113] S601. Obtain the material information of the object.

[0114] Among them, the material information is used to characterize the color information and / or light reflection intensity information of each pixel point on the surface of the object.

[0115] Since each point in the point cloud model includes color information or light reflection intensity information, therefore, after the server obtains the point cloud model of the object, it may extract the material information from the point cloud model.

[0116] S602. Render the two-dimensional unfolded view of the 3D model based on the material information.

[0117] S603. Obtain the first UV map based on the two-dimensional unfolded view corresponding to the rendered 3D model.

[0118] Based on the material information, map the color information and / or light reflection intensity information of each pixel point on the surface of the object to the surface of the 3D model, and render (or bake and render) the 3D model, so as to obtain the rendered 3D model (including the surface features of the object). Further, perform a two-dimensional unfolding of the 3D model to obtain the first UV map. Among them, the two-dimensional unfolding of the 3D model is the process of mapping the 3D model from XYZ coordinates to the planar UV coordinates, so as to realize tiling the surface of the 3D mesh model onto a two-dimensional plane. The specific process of the two-dimensional unfolding in the embodiments of the present application is not limited. For example, it can be implemented through 3D modeling tools.

[0119] For example, taking the object as a server as an example, Figure 9 is a 3D schematic diagram of a server provided by an embodiment of the present application. Figure 10 is a VU map corresponding to the server provided by an embodiment of the present application. From Figure 10 it can be seen the details of the object surface, such as which components are included, the surface material and color, etc.

[0120] It should be understood that the above steps provide an implementable way to obtain the UV map. Through the obtained UV map, it is possible to endow the digital twin with realistic surface details during the construction process of the digital twin.

[0121] It should be noted that although the related art can also obtain the production model of the object (i.e., the 3D model provided by the equipment manufacturer) through some means, or scan the object through a scanning device on the market (such as a 3D profiler) to obtain the 3D model of the object. However, the 3D models obtained by the above methods are relatively large (including a lot of information that is useless for digital twin construction (such as information inside the object, surrounding environment information unrelated to the object itself, etc.)). If the models obtained by the above methods are directly added to the digital twin scene, it will cause the entire scene to freeze and affect the user experience.

[0122] For the problem of the relatively large 3D model mentioned above, the current solution is to further optimize the 3D model manually to obtain a 3D model suitable for use in the digital twin scene. However, this solution still consumes human resources and time and is difficult to achieve the rapid synchronization of the digital twin scene and the real scene. To solve the problem of manual optimization, the method for constructing a digital twin provided by the embodiments of the present application further performs the following:

[0123] In a possible implementation manner, as Figure 11 shown, the construction method provided by the embodiments of the present application further includes the following S403-S405:

[0124] S403. Obtain modification information.

[0125] Among them, the modification information is used to indicate the change of the second UV map relative to the first UV map.

[0126] The server can obtain the modification information from the terminal device on the user side. The modification information is used to represent the user's modification opinion on the first UV map. For the specific process of the user inputting the modification information to the terminal device, reference can be made to the description of the following Figure 12 corresponding embodiment.

[0127] Exemplarily, the modification information can be a text description or an instruction for modifying multiple coordinate points. The multiple coordinate points are the coordinate points marked on the first UV map.

[0128] S404. Adjust the first UV map according to the modification information to obtain the second UV map.

[0129] Among them, the surface feature details of the object displayed by the second UV map are different from those of the first UV map.

[0130] In some embodiments, the above S404 can be implemented as: inputting the first UV map and the modification information into the first model to obtain the second UV map output by the first model. Among them, the first model is used to adjust the surface features of the UV map according to the modification information.

[0131] For example, taking the modification information as a text description as an example, the server inputs the first UV map and the text description "Modify the color to blue" into the first model, and then a second UV map with an overall blue color can be generated.

[0132] In some embodiments, the surface details include at least one of the following:

[0133] Appearance parameters of the object (such as color, material, transparency, light intensity, etc.);

[0134] Components included in the object (for example, configuring the display or non-display of a certain component);

[0135] Identification information of the components included in the object (for example, configuring an identification (such as a name, number, etc.) for a certain component, or modifying the original identification).

[0136] S405. Configure the second UV map onto the 3D model to obtain the digital twin of the object.

[0137] Among them, for the description of S405, reference can be made to the description of S402 above, and no repeated elaboration will be made here.

[0138] It should be noted that the above S403 - S405 can be an execution solution to replace S402, or can be executed after S402. The embodiments of the present application do not make specific limitations on this.

[0139] It can be understood that, different from the related art, in the related art, the user adjusts the 3D model itself, and such an adjustment process is relatively cumbersome and requires the user to have certain modeling knowledge. In the embodiments of the present application, the UV map is adjusted. The UV map is a two-dimensional image and is relatively simple to adjust. Moreover, the embodiments of the present application adopt the first model, which can obtain the modified UV map according to the modification information, and then generate a digital twin that meets the user's needs, thereby effectively reducing the technical threshold requirements for the user in the process of modifying the 3D model.

[0140] Figure 12 The flowchart of the display method provided by an exemplary embodiment of the present application is shown. The display method can be executed by a terminal device and includes the following steps:

[0141] S1201. Display the digital twin of the object, and the digital twin includes a first UV map.

[0142] Wherein, the UV map is a two-dimensional representation of the surface of the object and is used to reflect the surface details of the object.

[0143] The terminal device can obtain the digital twin including the first UV map from the server and display it to the user on the display screen.

[0144] S1202. In response to the adjustment operation on the first UV map, display the digital twin of the updated object, and the digital twin of the updated object includes a second UV map.

[0145] Wherein, the 3D models of the above digital twin and the updated twin are the same, both are the 3D models of the same object; the second UV map is different from the surface features reflected by the first UV map. The second UV map is used to be configured on the 3D model of the object to obtain the digital twin of the updated object.

[0146] In the embodiments of the present application, the terminal device can obtain the adjustment operation of the user, convert it into modification information and send it to the server. The server can input the modification information and the first UV map into the first model to obtain the second UV map, configure the second UV map on the 3D model, and then return it to the terminal device for display.

[0147] In some possible implementation manners, the modification operation includes at least one of the following:

[0148] The input operation of text description information, and the text description information is used to describe the change of the second UV map relative to the first UV map;

[0149] The editing operation on the first UV map, and the editing operation is used to edit the content included in the first UV map.

[0150] In a possible implementation, the text description information or the editing operation is used to adjust at least one of the following:

[0151] Appearance parameters of the object (such as color, material, transparency, etc.);

[0152] Components included in the object (such as configuring the display or non-display of a certain component);

[0153] Identification information of the components included in the object (such as configuring an identifier (such as a name, number, etc.) for a certain component, or modifying the original identifier).

[0154] For example, if the user wants to modify the color of the digital twin, they can enter a text description on the terminal device, such as "Modify the color to blue". Correspondingly, the terminal device sends "Modify the color to blue" as the modification information to the server. The server inputs the first UV map and the text description into the first model to obtain the second UV map, configures the second UV map on the 3D model, and displays it to the user through the terminal device.

[0155] In addition, in addition to the text description, the user can directly perform an editing operation on the first UV map to modify the first UV map. For example, the user can use tools such as a brush to add an identifier to a certain component of the object in the first UV map. Another example is that the user can use tools such as a brush to smear on the first UV map to cover the components that the user does not want to display on the first UV map. As Figure 13 shown, the first UV map on the left includes a fan cooling port. If the user does not want to show this component in the digital twin, they can smear the part where the fan cooling port is located to obtain the second UV map on the right. Another example is that the first UV map includes a slider control for adjusting transparency. The user can move the slider control to adjust the transparency of the first UV map, etc., so as to increase or decrease the transparency of the digital twin.

[0156] It should be understood that in the embodiment of the present application, by displaying the first UV map and according to the adjustment of the first UV map by the user, the adjustment of the digital twin is realized. Compared with the way of directly adjusting the 3D model by the user, the way of adjusting the two-dimensional image is relatively simple, the technical threshold requirement for the user is relatively low, and the user can perform simple operations (such as painting or entering text descriptions) to achieve the purpose of modifying the digital twin.

[0157] The training process of the first model will be described below.

[0158] Figure 14 The flowchart of the training method provided by an exemplary embodiment of the present application is shown. This training method can be executed by the server and includes the following steps:

[0159] S1401. Obtain the first training dataset.

[0160] Among them, each pair of training data in the first training dataset includes the initial UV map of the device, the modification information, and the target UV map obtained by adjusting the initial UV map according to the modification information. Among them, the UV map is used to display the surface features of the object. The modification information is used to indicate the change of the target UV map relative to the initial UV map.

[0161] Users can collect the UV maps before and after modification according to the modification information in historical project experience and input them into the server so that the server can obtain the training dataset. Among them, the UV map before modification according to the modification information is the initial UV map, and the map after modification according to the modification information is the target UV map. Both the initial UV map and the target UV map are historical UV maps.

[0162] S1402. Use the initial UV map and the modification information as sample data, and use the target UV map as the label of the sample data to train the first initial model to obtain the first model.

[0163] Among them, the initial model refers to a basic model architecture selected before the start of training (which can be called the starting point of training). The parameters in this initial model are all preset initial values, and various parameters will be continuously adjusted during the training process until a model that can meet the user's needs is finally obtained. The first model is used to adjust the surface features of the UV map according to the modification information.

[0164] During the training process, the initial UV map and the modification information in the training dataset are used as sample data and input into the initial model to obtain the output result, which is compared with the target UV map used as the label data to determine the loss value. Then, the model parameters of the initial model are adjusted based on algorithms such as the gradient descent algorithm or the backpropagation algorithm. Repeat the above process until the obtained loss value is the smallest or the number of repetitions reaches a certain threshold, and the training is completed to obtain the first model.

[0165] It should be understood that for the convenience of the model to understand the feature information in the image or text, the server can convert the UV map and the modification information into vectors and then input them into the initial model. The specific method of converting into vectors in the embodiments of the present application is not limited. For example, the UV map is converted by using an autoencoder, and the modification information is converted by using a text encoder.

[0166] In one implementation, the first initial model used to train the first model is a diffusion model.

[0167] As described above, the principle of the diffusion model is to gradually add noise to the image in the forward process and then denoise and restore the clear image in the reverse process, so as to generate a new image closer to the original image. Since the diffusion model supports guiding the generation of images through labels, in the embodiments of the present application, by using the modification information as a label to train the first model, the first model is obtained with the function of generating a corresponding image according to the modification information, and the UV map can be adjusted according to the user's opinion, thereby realizing the stylized transformation of the digital twin.

[0168] The training process of the second model will be described below, including the following steps:

[0169] S1501. Obtain a second training dataset.

[0170] Among them, each pair of training data in the second training dataset includes a point cloud model of an object and a 3D mesh model of the object.

[0171] S1502. Use the point cloud model as sample data and the 3D mesh model as the label of the sample data to train the second initial model to obtain the second model.

[0172] The user can collect the point cloud models of multiple objects and their corresponding 3D mesh models and input them into the server, so that the server obtains the second training dataset. Further, the server inputs the point cloud model into the initial model, obtains the output result, compares it with the 3D mesh model as the label data to determine the loss value, and then adjusts the model parameters of the initial model based on the gradient descent algorithm or the backpropagation algorithm, etc. Repeat the above process until the obtained loss value is the smallest, or the number of repetitions reaches a certain threshold, and the training is completed to obtain the second model.

[0173] Among them, the point cloud model and the 3D mesh model used for training are both historical UV maps.

[0174] In one implementation, the second initial model used to train the second model can be a Transformer model. Among them, the Transformer model is a deep learning model that captures the dependency relationships between various elements in the input model (the feature representations of each pixel point in the embodiments of the present application) through the self-attention mechanism.

[0175] Optionally, during the process of training the second model, the 3D mesh model can be encoded into a feature vector and then noise (such as Gaussian noise) can be added. For example, a specific function in a matrix processing tool (such as numpy) can be used to implement adding noise to the data. It should be understood that adding noise to the data can increase the diversity and generalization ability of the data, and help improve the robustness of the model during the model training process. For example, when using a Transformer model for training, adding noise to the data can simulate the possible defects in the token sequence generated by the Transformer model to ensure that the model can generate a relatively accurate 3D mesh model even when the output token sequence is not perfect. Among them, a token is used to represent each element of the input or output of the Transformer model, and it is the basis for the Transformer model to understand and process the input data.

[0176] In one implementation, a linear projection layer can be added to the initial model. Each pair of training data in the second training dataset includes the point cloud model of the object and the 3D mesh model of the object corresponding to the display accuracy. Using the point cloud model and the display accuracy as sample data, and using the 3D mesh model corresponding to the display accuracy as the label of the sample data to train the second initial model, the second model is obtained. The second model trained in this way can obtain a 3D mesh model (i.e., the number of mesh cells on the model surface matches the target display accuracy) that matches the target display accuracy according to the input point cloud model and the target display accuracy.

[0177] The method for constructing a digital twin body provided by the embodiments of the present application generates a 3D model of an object and a first UV map according to a multimedia file. Among them, the UV map is used to reflect the surface details of the object. Further, the first UV map is configured onto the 3D model to obtain the digital twin body of the object, and the modeling process is completed. It can be seen that the method provided by the embodiments of the present application does not rely on the process of manual modeling. By obtaining the multimedia file of the object, information such as the skeleton and surface features required for constructing the digital twin body can be obtained from the multimedia file, so as to realize the automatic construction of the digital twin body without excessive manual participation, effectively improving the modeling efficiency while reducing the waste of human and material resources.

[0178] In addition, through the construction method of the embodiments of the present application, the diversity of the digital twin body can be greatly enhanced. Users do not need to spend a lot of time looking for or constructing a model, and only need to obtain the image or video of the device to generate the corresponding digital twin body. Moreover, through the modification of the UV map in the embodiments of the present application, the workload of directly adjusting the 3D model in the related technology is optimized, and the technical threshold of users is effectively reduced.

[0179] The method for constructing a digital twin provided by the embodiments of the present application can also be applied to scenarios such as the metaverse, industrial 3D display, and games.

[0180] The above mainly introduced the solution of the embodiments of the present application from the perspective of the method. It can be understood that in order for the page display device of the digital twin system to implement the above functions, it includes at least one of the corresponding hardware structures and software modules for executing each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the embodiments of the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the embodiments of the present application.

[0181] The embodiments of the present application can divide the functional units of the page display device of the digital twin system according to the above method examples. For example, each functional unit can be divided corresponding to each function, or two or more functions can be integrated into one processing unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit. It should be noted that the division of units in the embodiments of the present application is illustrative, only a logical function division, and there may be other division methods in actual implementation.

[0182] Exemplarily, Figure 15 The structural schematic diagram of a digital twin construction device provided by an exemplary embodiment of the embodiments of the present application is shown. The digital twin construction device is applied to a computing device, or the digital twin construction device can be a computing device. The digital twin construction device includes: a generation module 1501.

[0183] The generation module 1501 is used to generate a 3D model of the object and a first UV map according to the multimedia file of the object; the UV map is used to display the surface features of the object;

[0184] The generation module 1501 is further used to configure the first UV map onto the 3D model to generate a digital twin of the object.

[0185] For the specific description of the above optional methods, reference can be made to the foregoing method embodiments, which will not be elaborated herein. In addition, the explanations and descriptions of the beneficial effects of any of the above-provided digital twin construction devices can refer to the corresponding method embodiments above and will not be elaborated.

[0186] Exemplarily, Figure 16The figure shows a schematic structural diagram of a display device provided by an exemplary embodiment of the present application. The display device includes: a display module 1601.

[0187] The display module 1601 is configured to display the digital twin of an object; the digital twin includes a first UV map, and the first UV map is used to display the surface features of the object;

[0188] The display module 1601 is further configured to, in response to an adjustment operation on the first UV map, display the updated digital twin of the object, and the updated digital twin of the object includes a second UV map; the surface features reflected by the second UV map are different from those of the first UV map; the second UV map is used to be configured on the 3D model of the object to obtain the updated digital twin of the object.

[0189] For the specific description of the above optional manners, reference may be made to the foregoing method embodiments, which will not be elaborated herein. In addition, the explanations of any of the above-provided display devices and the descriptions of the beneficial effects may refer to the corresponding method embodiments above, and will not be elaborated.

[0190] Exemplarily, Figure 17 The figure shows a schematic structural diagram of a first training device provided by an exemplary embodiment of the present application. The display device includes: an acquisition module 1701 and a training module 1702.

[0191] The acquisition module 1701 is configured to acquire a first training dataset; each pair of training data in the first training dataset includes the initial UV map of the object, modification information, and the target UV map obtained by adjusting the initial UV map according to the modification information; wherein, the UV map is used to display the surface features of the object; the modification information is used to indicate the change of the target UV map relative to the initial UV map;

[0192] The training module 1702 is configured to use the initial UV map and the modification information as sample data, and use the target UV map as the label of the sample data to train a first initial model to obtain a first model.

[0193] Exemplarily, Figure 18 The figure shows a schematic structural diagram of a second training device provided by an exemplary embodiment of the present application. The display device includes: an acquisition module 1801 and a training module 1802.

[0194] The acquisition module 1801 is configured to acquire a second training dataset; each pair of training data in the second training dataset includes the point cloud model of the object and the 3D mesh model of the object;

[0195] The training module 1802 is configured to use the point cloud model as sample data, and use the 3D mesh model as the label of the sample data to train a second initial model to obtain a second model.

[0196] In an exemplary embodiment, a computer-readable storage medium is further provided, which is used to store at least one instruction, at least one program, a code set or an instruction set. The at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement all or part of the steps in the above-mentioned memory fault prediction method. For example, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, etc.

[0197] In an exemplary embodiment, a computer program product or a computer program is further provided. The computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computing device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computing device executes all or part of the steps of the method shown in any of the above Figure 4 embodiments.

[0198] In some embodiments, the method shown in the embodiments of the present application may be implemented as computer program instructions encoded in a machine-readable format on a computer-readable storage medium or encoded on other non-transitory media or articles.

[0199] Through the description of the above embodiments, those skilled in the art can clearly understand that for the convenience and conciseness of description, only the above division of each functional module is used as an example. In actual applications, the above functions may be allocated to different functional modules according to needs. For example, the internal structure of the device may be divided into different functional modules to complete all or part of the functions described above.

[0200] In several embodiments provided in the embodiments of the present application, it should be understood that the disclosed device and method may be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other may be through some interfaces. The indirect coupling or communication connection of the device or unit may be in an electrical, mechanical or other form.

[0201] The unit described as a separation component may or may not be physically separated. The component displayed as a unit may be a physical unit or multiple physical units, such as being located in one place or being distributed to multiple different places. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0202] In addition, in each embodiment of this application, each functional unit can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0203] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to enable a device (which can be a single-chip microcomputer, a chip, etc.) or a processor to execute all or part of the steps of the methods described in the embodiments of this application. The aforementioned storage medium includes: USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks, or optical discs and other various media that can store program codes.

[0204] The above are only optional embodiments of the embodiments of this application and are not intended to limit the embodiments of this application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the embodiments of this application shall be included in the protection scope of the embodiments of this application.

Claims

1. A method for constructing a digital twin, characterized in that, The method includes: Generating a 3D model and a first UV map of the object according to the multimedia file of the object; the UV map is used to display the surface features of the object; Configuring the first UV map onto the 3D model to generate a digital twin of the object.

2. The method according to claim 1, characterized in that, The method further includes: Obtaining modification information; the modification information is used to indicate the change of the second UV map relative to the first UV map; Adjusting the first UV map according to the modification information to obtain a second UV map; the surface features of the object displayed by the second UV map are different from those of the first UV map; Configuring the second UV map onto the 3D model to generate an updated digital twin of the object.

3. The method according to claim 2, characterized in that The adjusting the first UV map according to the modification information to obtain a second UV map includes: Inputting the first UV map and the modification information into a first model to obtain the second UV map output by the first model.

4. The method according to any one of claims 1-3, characterized in that The 3D model is a 3D mesh model, including a plurality of mesh units in a three-dimensional space; the number of the mesh units is used to characterize the display accuracy of the 3D mesh model. Generating the 3D model of the object according to the multimedia file of the object includes: Generating a point cloud model of the object based on the multimedia file of the object; Generating a 3D mesh model of the object based on the point cloud model; the display accuracy of the 3D mesh model is lower than that of the point cloud model.

5. The method according to claim 4, characterized in that The generating the 3D model of the object based on the point cloud model includes: Inputting the point cloud model into a second model to obtain the output 3D mesh model of the object.

6. The method according to claim 4 or 5, characterized in that The number of mesh units on the surface of the 3D mesh model matches the target display accuracy of the digital twin; the target display accuracy is determined according to the application scenario of the object, and the display accuracies corresponding to the 3D mesh models of different objects in the same application scenario are the same.

7. The method according to any one of claims 1-6, characterized in that, The generating the first UV map of the object according to the multimedia file of the object includes: Obtaining the material information of the object; the material information is used to characterize the color information and / or light reflection intensity information of each pixel point on the surface of the object; Rendering the surface of the 3D model based on the material information; Obtaining the first UV map based on the two-dimensional unfolded view corresponding to the rendered 3D model.

8. The method according to any one of claims 1 to 7, characterized in that, The surface features include at least one of the following: The appearance parameters of the object; The components included in the object; The identification information of the components included in the object.

9. A display method, characterized in that, Applied to a terminal device, the method includes: Displaying the digital twin of the object; the digital twin includes a first UV map, and the first UV map is used to display the surface features of the object; In response to an adjustment operation on the first UV map, displaying an updated digital twin of the object, and the updated digital twin of the object includes a second UV map; the surface features reflected by the second UV map are different from those of the first UV map; the second UV map is used to be configured onto the 3D model of the object to obtain the updated digital twin of the object.

10. The method according to claim 9, characterized in that The adjustment operation includes at least one of the following: An input operation of text description information for describing the change of the second UV map relative to the first UV map; An editing operation on the first UV map for editing the content included in the first UV map.

11. The method according to claim 10, wherein, The text description information or the editing operation is used to adjust at least one of the following: The appearance parameters of the object; The components included in the object; The identification information of the components included in the object.

12. A training method, characterized in that, The method further includes: Obtaining a first training data set; each pair of training data in the first training data set includes an initial UV map of an object, modification information, and a target UV map obtained by adjusting the initial UV map according to the modification information; wherein, the UV map is used to display the surface features of the object; the modification information is used to indicate the change of the target UV map relative to the initial UV map; Using the initial UV map and the modification information as sample data, and using the target UV map as the label of the sample data to train a first initial model to obtain a first model.

13. A training method, characterized in that, The method further includes: Obtaining a second training data set; each pair of training data in the second training data set includes a point cloud model of an object and a 3D mesh model of the object; Using the point cloud model as sample data, and using the 3D mesh model as the label of the sample data to train a second initial model to obtain a second model.

14. A computing device, characterized in that, The computing device includes a processor and a memory; the processor is coupled to the memory; The memory is used to store computer instructions; The computer instructions are loaded and executed by the processor to enable the computing device to implement the method according to any one of claims 1-13.

Images