Mesh model generation method and apparatus, and device

Abstract

The present application relates to the technical field of artificial intelligence. Provided are a mesh model generation method and apparatus, and a device. The method comprises: acquiring visual data of an object that is uploaded by a user, wherein the visual data comprises at least one of an image, a point cloud and a voxel model; performing geometric feature extraction on the visual data of the object, so as to obtain a first geometric feature of the object; acquiring from an a priori knowledge base a priori knowledge corresponding to an object category to which the object belongs, wherein the a priori knowledge is used for guiding the establishment of a geometric generative network; on the basis of the first geometric feature and the a priori knowledge corresponding to the object category to which the object belongs, establishing a first geometric generative network; and on the basis of the first geometric generative network, generating a first mesh model for the object. The mesh model can be automatically established without the need for manual participation.

Description

Method, device and apparatus for generating grid model

[0001] This application claims priority to Chinese patent application No. 202311034137.4 filed on August 16, 2023, entitled “Method, device, equipment and storage medium for converting three-dimensional digital assets” and Chinese patent application No. 202311654049.4 filed on November 30, 2023, entitled “Method, device and equipment for generating grid models”, the entire contents of which are incorporated herein by reference. Technical Field

[0002] The present application relates to the field of artificial intelligence technology, and in particular to a method, device and apparatus for generating a grid model. Background Art

[0003] Three-dimensional (3D) models are widely used in various fields, including industrial simulation and large-scale model training. 3D models generally include mesh models, point cloud models, and voxel models. Mesh models are the closest to actual objects in terms of both geometric accuracy and rendering quality.

[0004] Currently, modelers typically use computer software to construct mesh models of objects. However, manual modeling is inefficient and can take months to complete if the scene is complex and contains many objects.

[0005] Summary of the Invention

[0006] This application provides a method, apparatus, and device for generating a grid model, which can generate a grid model more efficiently. The corresponding technical solutions are as follows:

[0007] In a first aspect, a method for generating a grid model is provided, the method comprising:

[0008] Visual data of an object uploaded by a user is obtained, where the visual data includes at least one of an image, a point cloud, and a voxel model. Geometric features are then extracted from the visual data to obtain a first geometric feature of the object. Prior knowledge corresponding to the object category of the object is obtained from a priori knowledge base. This prior knowledge is used to guide the establishment of a geometry generative network. Furthermore, a first geometry generative network is established based on the first geometric feature and the prior knowledge corresponding to the object category. Finally, a first mesh model of the object is generated based on the first geometry generative network.

[0009] In the technical solution provided in this application, a geometric generation network is automatically constructed by combining prior knowledge and the geometric features of the object, so that the mesh model of the object can be automatically generated using the geometric generation network. The entire process does not require human participation, and the mesh model generation efficiency is higher.

[0010] In one possible implementation, the prior knowledge corresponding to the object category of the object includes a first geometric latent space network and hyperparameters of a first geometric generative network. Accordingly, based on the first geometric feature and the prior knowledge corresponding to the object category of the object, establishing the first geometric generative network includes:

[0011] The first geometric feature is input into a first geometric latent space network to obtain first geometric generative network parameters, and an initial geometric generative network is established based on the first geometric generative network parameters and the first geometric generative network hyperparameters. Then, supervised training is performed on the initial geometric generative network based on the first geometric feature to obtain a first geometric generative network.

[0012] In the technical solution provided in this application, the geometric features of the object are used to perform supervised training on the geometry generation network, so that the trained geometry generation network can deterministically generate a mesh model of the object, and the generated mesh model has higher accuracy.

[0013] In one possible implementation, the geometric latent space network in the prior knowledge base can also be optimized. Accordingly, the processing can be as follows:

[0014] The first geometric latent space network is trained based on the first geometric feature and the geometric generative network parameters of the first geometric generative network to obtain a trained geometric latent space network. Then, the first geometric latent space network stored in the prior knowledge base is replaced with the trained geometric latent space network.

[0015] In the technical solution provided in this application, by optimizing the geometric latent space network, the geometric generative network parameters output by the geometric latent space network can be made more usable, thereby reducing the number of training times of the geometric generative network and improving the generation efficiency of the mesh model.

[0016] In one possible implementation, a user can upload visual data of a scene. Accordingly, after obtaining the visual data of the first scene uploaded by the user, the object segmentation can be performed on the visual data of the first scene to obtain the visual data of each of the multiple objects. Thus, the technical solution provided in this application can be used to generate mesh models of all or part of the multiple objects.

[0017] In a possible implementation, when the user uploads a visual image of a scene, a mesh model of the scene can also be generated. Accordingly, the processing can be as follows:

[0018] Spatial features are extracted from multiple objects in the visual data of the first scene to obtain spatial features corresponding to the multiple objects, wherein the spatial features are used to represent the positions of the corresponding objects in the first scene. Then, mesh models corresponding to the multiple objects are placed based on the spatial features corresponding to the multiple objects to obtain a mesh model of the first scene.

[0019] In one possible implementation, considering that the positional relationship of objects calculated based on their spatial features may be inaccurate, the placement of the object's mesh model in the scene's mesh model can be adjusted based on prior knowledge. Accordingly, the processing can be as follows:

[0020] For each two objects among the plurality of objects, a confidence level is calculated for each of a plurality of preset positional relationships based on the spatial features corresponding to each of the two objects. A first object category and a second object category are determined to which each of the two objects belongs. From a priori knowledge base, a reference confidence level for each of the preset positional relationships between objects of the first object category and objects of the second object category in the first scene is obtained. Based on the confidence level and the reference confidence level, the positional relationship between each two objects is determined. Based on the positional relationship between the two objects, the positions of the grid models corresponding to the two objects in the grid model of the first scene are adjusted.

[0021] In the technical solution provided in this application, the prior knowledge base stores reference confidence levels for each preset positional relationship between objects of each object category in different scenarios. The reference confidence levels can be calculated based on the actual positional relationships between objects of each object category in different scenarios in reality, and can, to a certain extent, reflect the actual positional relationships between objects of each object category. On this basis, the mesh models of each object in the mesh model of the first scene are adjusted based on the reference confidence levels for each preset positional relationship between objects of the first object category and objects of the second object category in the first scene, so that the positional relationships between the mesh models of each object are more realistic.

[0022] In a possible implementation, a texture can be rendered on the mesh model, making the mesh model a textured mesh model that is closer to the real object. Accordingly, the processing can be as follows:

[0023] Obtain a first texture map for the object. Render the first texture map onto the first mesh model to obtain a first textured mesh model of the object. Obtaining the first texture map for the object can be performed before, after, or simultaneously with generating the first mesh model, and this application does not limit this.

[0024] In one possible implementation, when the visual data uploaded by the user is at least one of a non-color point cloud and a non-color voxel model, texture features cannot be directly obtained from the visual data. In this case, the texture can be specified by the user. Accordingly, the processing can be as follows:

[0025] A texture selection request is received from a user, wherein the texture selection request carries texture style indication information, color type indication information, and color brightness indication information. A first texture generation network corresponding to the texture type indication information, the color type indication information, and the color brightness indication information is obtained, and a first texture map of the object is generated based on the first texture generation network.

[0026] In one possible implementation, when the visual data uploaded by the user is at least one of a color point cloud, a color voxel model, and an image, texture features can be directly extracted from the visual data to generate a texture map. Accordingly, the processing can be as follows:

[0027] Texture features are extracted from the visual data of the object to obtain a first texture feature of the object. A second texture generation network is established based on the first texture feature and prior knowledge of the object. Based on the second texture generation network, a first texture map of the object is generated.

[0028] In a possible implementation, the prior knowledge further includes hyperparameters of the first texture latent space network and the first texture generation network. Accordingly, the process of establishing the texture generation network based on the prior knowledge may be as follows:

[0029] Input the first texture feature into the first texture latent space network to obtain first texture generation network parameters. Establish an initial texture generation network based on the first texture generation network parameters and the first texture generation network hyperparameters. Train the initial texture generation network based on the first texture feature to obtain a second texture generation network. Generate a first texture map of the object based on the second texture generation network.

[0030] In the technical solution provided in this application, the texture features of the object are used to supervise the training of the texture generation network, so that the trained texture generation network can deterministically generate the texture map of the object, and the generated texture map is closer to the texture of the actual object.

[0031] In a possible implementation, the texture latent space network in the prior knowledge base can also be optimized. Accordingly, the processing can be as follows:

[0032] The first texture latent space network is trained based on the first texture feature and the texture generation network parameters of the trained texture generation network to obtain a trained texture latent space network. The first texture latent space network stored in the prior knowledge base is replaced with the trained texture latent space network.

[0033] In the technical solution provided in this application, by optimizing the texture latent space network, the texture generation network parameters output by the texture latent space network can be made more usable, thereby reducing the number of training times of the texture generation network and improving the generation efficiency of texture maps.

[0034] In a possible implementation, the user may specify to generate a 3D model of a certain type and format. Accordingly, the processing may be as follows:

[0035] A 3D model conversion request is received, wherein the 3D model conversion request carries information indicating a target model type and a target model format, and a mesh model of the object is converted into a 3D model of the target model type and the target model format.

[0036] In a possible implementation, in order to facilitate archiving and management, objects can also be tagged. Accordingly, the processing can be as follows:

[0037] Based on the object category labels stored in the prior knowledge base, the object category label corresponding to the object is determined.

[0038] In a possible implementation, the technical solution provided by this application can also realize the self-growth of the grid model, and the processing can be as follows:

[0039] Receive a first 3D digital asset self-growth request sent by a user, wherein the first 3D digital asset self-growth request carries indication information of the first growth starting point object, indication information of the first growth end point object, and a target number, and the first growth starting point object and the first growth end point object are different objects under the third object category. In the prior knowledge base, obtain the geometric features of the first growth starting point object, the geometric features of the first growth end point object, and the prior knowledge corresponding to the third object type. With the geometric features of the first growth starting point object as the starting point and the geometric features of the first growth end point object as the end point, generate a target number of intermediate geometric features. For each of the target number of intermediate geometric features, generate a second geometric generation network based on the intermediate geometric features and the prior knowledge corresponding to the third object type, and generate a second mesh model based on the second geometric generation network.

[0040] In the technical solution provided by this application, users can specify the starting and ending objects for mesh model self-growth, as well as the target number of mesh models to be self-grown. This solution can then generate mesh models of objects with characteristics between the target number and the target number. Through mesh model self-growth, a large number of mesh models of a certain object category can be quickly generated, which can then be used for various applications such as large-scale model training and simulation.

[0041] In a possible implementation, the self-grown mesh model can be stored in a digital asset library. Accordingly, the processing can be as follows:

[0042] The target number of second mesh models and the target number of second geometry generation networks are stored in the digital asset library on the cloud.

[0043] In the technical solution provided in the present application, only a target number of second geometric generation networks can be stored without storing the second mesh model. When needed, the second mesh model can be temporarily generated through the second geometric generation network, thus saving storage space of the digital asset library.

[0044] In one possible implementation, when implementing the self-growth of a mesh model, the user may specify a target step size instead of a target number. Accordingly, the self-growth of the mesh model may be processed as follows:

[0045] A second 3D digital asset self-growth request sent by a user is received, wherein the second 3D digital asset self-growth request carries indication information of a second growth starting point object, indication information of a second growth end point object, and a target step length, and the second growth starting point object and the second growth end point object are different objects under a fourth object category. In a priori knowledge base, the geometric features of the second growth starting point object, the geometric features of the second growth end point object, and the prior knowledge corresponding to the fourth object type are obtained. At least one intermediate geometric feature is generated with the geometric features of the second growth starting point object as the starting point and the geometric features of the second growth end point object as the end point. For each of the at least one intermediate geometric feature, a third geometric generation network is generated based on the intermediate geometric feature and the prior knowledge corresponding to the fourth object type, and a third mesh model is generated based on the third geometric generation network.

[0046] In the technical solution provided by this application, users can specify the starting and ending objects for mesh model self-growth, as well as the target step size. This solution can then generate a mesh model with at least one object whose characteristics lie between those of the starting and ending objects. Through mesh model self-growth, mesh models of a specific object category can be rapidly generated, which can then be used for various applications such as large-scale model training and simulation.

[0047] In one possible implementation, the technical solution provided by this application can also realize the self-growth of the scene-level grid model, and the processing can be as follows:

[0048] Receive a third 3D digital asset self-growth request sent by a user, wherein the third 3D digital asset self-growth request carries indication information of the second scene and indication information of multiple objects in the second scene. Obtain mesh models of multiple objects in the second scene from the digital asset library. Obtain a reference confidence level for each of a plurality of preset positional relationships in the prior knowledge base. Determine the positional relationship between each two objects in the second scene based on the reference confidence level for each of a plurality of preset positional relationships in the positional relationship between each two objects in the second scene. Place mesh models of multiple objects in the second scene based on the positional relationship between each two objects in the second scene to obtain a mesh model of the second scene.

[0049] In the technical solution provided in this application, the user can specify the required scene and the objects included in the scene, and then generate a grid model of the scene containing these objects. The generation efficiency is high and the user does not need to provide visual data of the scene.

[0050] In a possible implementation, the self-grown mesh model can be stored in a digital asset library. Accordingly, the processing can be as follows:

[0051] The mesh model of the second scene is stored in the digital asset library.

[0052] In the technical solution provided in this application, self-growth of grid models can be achieved to obtain more grid models, thereby providing a large number of grid models required for various businesses such as large model training and simulation.

[0053] In a second aspect, a device for generating a grid model is provided, the device comprising:

[0054] A data preprocessing module, configured to obtain visual data of an object uploaded by a user, wherein the visual data includes at least one of an image, a point cloud, and a voxel model;

[0055] a feature extraction module, configured to extract geometric features from the visual data of the object to obtain a first geometric feature of the object;

[0056] An artificial intelligence-based digital asset generation AIGX module is used to obtain prior knowledge corresponding to the object category to which the object belongs from a prior knowledge base, wherein the prior knowledge is used to guide the establishment of a geometric generation network; based on the first geometric features and the prior knowledge corresponding to the object category to which the object belongs, a first geometric generation network is established; and based on the first geometric generation network, a first mesh model of the object is generated.

[0057] In one possible implementation, the prior knowledge corresponding to the object category to which the object belongs includes hyperparameters of a first geometric latent space network and a first geometric generative network, and the AIGX module is configured to:

[0058] Inputting the first geometric feature into the first geometric latent space network to obtain first geometric generative network parameters;

[0059] Establishing an initial geometry generative network based on the first geometry generative network parameters and the first geometry generative network hyperparameters;

[0060] The initial geometry generation network is trained based on the first geometric feature to obtain a first geometry generation network.

[0061] In one possible implementation, the AIGX module is configured to:

[0062] training the first geometric latent space network based on the first geometric features and the geometric generative network parameters of the first geometric generative network to obtain a trained geometric latent space network;

[0063] The first geometric latent space network stored in the prior knowledge base is replaced by the trained geometric latent space network.

[0064] In a possible implementation, the data preprocessing module is used to:

[0065] Obtaining visual data of a first scene uploaded by a user, wherein the first scene includes a plurality of objects;

[0066] Object segmentation is performed on the visual data of the first scene to obtain visual data of each of the multiple objects.

[0067] In a possible implementation, the apparatus further includes a positioning module, configured to:

[0068] Performing spatial feature extraction on the multiple objects in the visual data of the first scene to obtain spatial features corresponding to the multiple objects, wherein the spatial features are used to represent the positions of the corresponding objects in the first scene;

[0069] According to the spatial features corresponding to the multiple objects, the grid models corresponding to the multiple objects are placed to obtain the grid model of the first scene.

[0070] In a possible implementation, the apparatus further includes a scene optimization module, configured to:

[0071] For each two objects in the plurality of objects, calculating, based on the spatial features corresponding to the two objects, a confidence level that the positional relationship between the two objects is each of a plurality of preset positional relationships;

[0072] determining a first object category and a second object category to which the two objects respectively belong;

[0073] In the prior knowledge base, obtaining a positional relationship between an object of the first object category and an object of the second object category in the first scene as a reference confidence level of each preset positional relationship;

[0074] Determining a positional relationship between the two objects based on the confidence level and the reference confidence level;

[0075] According to the positional relationship between the two objects, positions of the grid models corresponding to the two objects in the grid model of the first scene are adjusted.

[0076] In a possible implementation, the AIGX module is further configured to:

[0077] Obtaining a first texture map of the object;

[0078] Rendering the first texture map onto the first mesh model to obtain a first textured mesh model of the object.

[0079] In one possible implementation, the visual data is at least one of a non-color point cloud and a non-color voxel model, and the AIGX module is configured to:

[0080] Receiving a texture selection request sent by a user, wherein the texture selection request carries texture style indication information, color type indication information, and color brightness indication information;

[0081] Acquire a first texture generation network corresponding to the texture type indication information, the color type indication information, and the color brightness indication information;

[0082] Based on the first texture generation network, a first texture map of the object is generated.

[0083] In a possible implementation, the prior knowledge is further used to guide the establishment of a texture generation network, and the feature extraction module is further used to:

[0084] Extracting texture features from the visual data of the object to obtain a first texture feature of the object;

[0085] Establishing a second texture generation network based on the first texture feature and prior knowledge of the object;

[0086] The AIGX module is used to:

[0087] Based on the second texture generation network, a first texture map of the object is generated.

[0088] In one possible implementation, the prior knowledge includes hyperparameters of a first texture latent space network and a first texture generation network, and the AIGX module is configured to:

[0089] Inputting the first texture feature into the first texture latent space network to obtain first texture generation network parameters;

[0090] Establishing an initial texture generation network based on the first texture generation network parameters and the first texture generation network hyperparameters;

[0091] Based on the first texture feature, the initial texture generation network is trained to obtain a second texture generation network;

[0092] Based on the second texture generation network, a first texture map of the object is generated.

[0093] In a possible implementation, the AIGX module is further configured to:

[0094] Training the first texture latent space network based on the first texture feature and the texture generation network parameters of the trained texture generation network to obtain a trained texture latent space network;

[0095] The first texture latent space network stored in the prior knowledge base is replaced by the trained texture latent space network.

[0096] In a possible implementation, the apparatus further includes a 3D digital asset conversion module, configured to:

[0097] A 3D model conversion request is received, wherein the 3D model conversion request carries information indicating a target model type and a target model format, and a mesh model of the object is converted into a 3D model of the target model type and the target model format.

[0098] In a possible implementation, the data preprocessing module is further configured to:

[0099] Based on the object category labels stored in the prior knowledge base, an object category label corresponding to the object is determined.

[0100] In a possible implementation, the AIGX module is further configured to:

[0101] Storing the first geometric feature of the object in the prior knowledge base;

[0102] receiving a first 3D digital asset self-growth request from a user, wherein the first 3D digital asset self-growth request carries information indicating a first growth starting object, information indicating a first growth ending object, and a target quantity, wherein the first growth starting object and the first growth ending object are different objects under a third object category;

[0103] Acquire, from the prior knowledge base, geometric features of the first growth starting point object, geometric features of the first growth end point object, and prior knowledge corresponding to the third object type;

[0104] Taking the geometric features of the first growth starting point object as a starting point and the geometric features of the first growth ending point object as an ending point, generating the target number of intermediate geometric features;

[0105] For each intermediate geometric feature of the target number of intermediate geometric features, a second geometry generation network is generated based on prior knowledge corresponding to the intermediate geometric feature and the third object type, and a second mesh model is generated based on the second geometry generation network.

[0106] In one possible implementation, the AIGX module is further used to:

[0107] The target number of second mesh models and the target number of second geometry generation networks are stored in the digital asset library on the cloud.

[0108] In a possible implementation, the AIGX module is further configured to:

[0109] Storing the first geometric feature of the object in the prior knowledge base;

[0110] receiving a second 3D digital asset self-growth request from a user, wherein the second 3D digital asset self-growth request carries information indicating a second growth starting object, information indicating a second growth ending object, and a target step size, and the second growth starting object and the second growth ending object are different objects under a fourth object category;

[0111] Acquiring, from the prior knowledge base, geometric features of the second growth starting point object, geometric features of the second growth end point object, and prior knowledge corresponding to the fourth object type;

[0112] generating at least one intermediate geometric feature by taking the geometric feature of the second growth starting point object as a starting point and the geometric feature of the second growth ending point object as an ending point;

[0113] For each intermediate geometric feature of the at least one intermediate geometric feature, a third geometry generation network is generated based on prior knowledge corresponding to the intermediate geometric feature and the fourth object type, and a third mesh model is generated based on the third geometry generation network.

[0114] In a possible implementation, the AIGX module is further configured to:

[0115] receiving a third 3D digital asset self-growth request sent by a user, wherein the third 3D digital asset self-growth request carries indication information of a second scene and indication information of a plurality of objects in the second scene;

[0116] Obtaining mesh models of multiple objects in the second scene from a digital asset library;

[0117] In the prior knowledge base, obtaining a reference confidence level of each of a plurality of preset positional relationships for a positional relationship between each two objects in the second scene;

[0118] determining a positional relationship between each two objects in the second scene based on a reference confidence level that the positional relationship between each two objects in the second scene is each of a plurality of preset positional relationships;

[0119] The grid models of the multiple objects in the second scene are placed according to the positional relationship between every two objects in the second scene to obtain the grid model of the second scene.

[0120] In one possible implementation, the AIGX module is further used to:

[0121] The mesh model of the second scene is stored in the digital asset library.

[0122] According to a third aspect, a computing device cluster is provided, wherein the computing device cluster includes at least one computing device, each computing device including a processor and a memory;

[0123] The processor of the at least one computing device is used to execute instructions stored in the memory of the device, so that the computing device cluster performs the method for generating a grid model as described in the first aspect above.

[0124] In a fourth aspect, a computer program product comprising instructions is provided. When the instructions are executed by a computing device cluster, the computing device cluster executes the method for generating a grid model as described in the first aspect.

[0125] In a fifth aspect, a computer-readable storage medium is provided, comprising computer program instructions. When the computer program instructions are executed by a computing device, the computing device executes the method for generating a grid model as described in the first aspect above. BRIEF DESCRIPTION OF THE DRAWINGS

[0126] FIG1 is a schematic diagram of an implementation scenario provided by an embodiment of the present application;

[0127] FIG2 is a flow chart of a method for generating a grid model provided in an embodiment of the present application;

[0128] FIG3 is a flow chart of a method for generating a grid model provided in an embodiment of the present application;

[0129] FIG4 is a schematic flow chart of a method for generating a grid model provided in an embodiment of the present application;

[0130] FIG5 is a flow chart of a method for generating a grid model provided in an embodiment of the present application;

[0131] FIG6 is a flow chart of a method for generating a grid model provided in an embodiment of the present application;

[0132] FIG7 is a schematic diagram of a structure of a device for generating a grid model provided in an embodiment of the present application;

[0133] FIG8 is a schematic diagram of the structure of a computing device provided in an embodiment of the present application;

[0134] FIG9 is a schematic diagram of a computing device cluster provided in an embodiment of the present application;

[0135] FIG10 is a schematic diagram of a computing device cluster provided in an embodiment of the present application. DETAILED DESCRIPTION

[0136] The following is an explanation of some of the terms involved in the embodiments of the present application.

[0137] 1. Mesh model

[0138] A type of 3D digital asset that uses a series of polygons (usually triangles) of similar size and shape to approximate the model of a three-dimensional object.

[0139] 2. Voxel model

[0140] A type of 3D digital asset that uses an ordered combination of a large number of regular voxels (such as cubes) to represent the model of a three-dimensional object.

[0141] 3. Point cloud

[0142] A type of 3D digital asset is a dataset of points in space that can represent a three-dimensional shape or object, typically acquired by a 3D scanner, including but not limited to a 3D lidar.

[0143] 4. 3D digital asset type conversion

[0144] Mesh model can be converted into voxel model: Mesh model can be converted into voxel model through voxelization method. Common voxelization methods include multi-view orthographic projection view rendering, object slicing, octree space partitioning, etc.

[0145] Mesh model conversion to point cloud: The mesh model can obtain the corresponding point cloud by sampling the vertices of the patch or taking more points on the patch.

[0146] Convert voxel models to mesh models: Voxel models can be converted into representations of vertices, edges, and faces through algorithms such as Marching Cube and Dual Contouring to obtain mesh models.

[0147] Convert voxel model to point cloud: By finding the centroid of each point in each voxel of the voxel model, or placing representative points on the voxel surface, a point cloud model can be sampled.

[0148] Convert point cloud to voxel model: Based on the given voxel size, the space is divided into regular cubes using the octree algorithm or other algorithms. For each cube, if the cube does not contain a point in the point cloud, it is discarded. If the cube contains a point in the point cloud, it is retained. In this way, the point cloud can be voxelized to obtain a voxel model.

[0149] Convert point cloud to mesh model: Use the rolling ball method, Poisson reconstruction method, and Delaunay triangulation projection method to fit the surface of the point cloud and form multiple triangular meshes to generate a mesh model.

[0150] 5. Artificial Intelligence Generated X (AIGX)

[0151] Generate text, images, audio, video, 3D models, etc. through AI technologies such as deep learning and reinforcement learning. In the disclosed embodiment, AIGX is used to generate 3D models.

[0152] 6. Geometry Generation Network

[0153] A type of neural network. In this application, a geometric generation network is used to generate a mesh model. The neural network can also be called a generative network, a 3D model generation network, a 3D geometry generation network, a mesh model generation network, etc. This application does not limit the name of the neural network.

[0154] 7. Texture Generation Network

[0155] A type of neural network. In this application, the texture generation network is used to generate texture maps. The neural network can also be called a generative network, a 3D model texture generation network, a mesh model texture generation network, etc. This application does not limit the name of the neural network.

[0156] The embodiment of the present application provides a method for generating a grid model, which can be implemented by a computing device, a computing device cluster, etc. The computing device and the computing device cluster can be a cloud platform or a user's local device. The method can be applied to various scenarios where there is a need to use 3D models. For example, in a simulation test in the manufacturing industry, it is necessary to establish a 3D model of the factory area. The user can collect visual data of the factory area, such as images, point clouds, etc., and the computing device uses the method provided by this application to generate a 3D model of the factory area.

[0157] Referring to FIG1 , a schematic diagram of an implementation scenario of an embodiment of the present application is shown. As shown in FIG1 , a user can upload visual data of an object for which a 3D model is to be created to a computing device based on their needs. The visual data can be one or more of an image, a point cloud, or a voxel model. The computing device then preprocesses the visual data to determine the object category to which the object belongs. Geometric features are then extracted from the visual data to obtain the object's geometric features. Furthermore, prior knowledge corresponding to the object category to which the object belongs is obtained from a priori knowledge base. This prior knowledge is used to guide the establishment of a geometry generative network. A geometry generative network is then established based on the object's geometric features and the prior knowledge corresponding to the object category to which the object belongs. This geometry generative network is then used to generate a mesh model of the object. Furthermore, to impart texture to the object's mesh model, in an embodiment of the present application, a texture map of the object can be obtained and rendered onto the object's mesh model to obtain a textured mesh model of the object. After generating the object's mesh model, the mesh model can be converted to a point cloud, voxel model, or other model based on the user's desired 3D model type. The method provided in the embodiment of the present application can combine prior knowledge and the geometric features of the object to automatically construct a geometric generation network, so that the geometric generation network can be used to automatically generate a mesh model of the object. The entire process does not require human participation, and the 3D model generation efficiency is higher.

[0158] 2 , which shows a flow chart of a method for generating a grid model according to an embodiment of the present application. The method may be implemented by a computing device. As shown in the figure, the flow of the method may include the following steps:

[0159] Step 201: Obtain visual data of an object uploaded by a user.

[0160] The visual data includes at least one of images, point clouds, and voxel models, and the images may include pictures and videos.

[0161] In implementation, users can collect visual data of objects based on actual needs and upload the collected visual data to a computing device. For example, users can use a mobile phone, tablet computer, camera, etc. to capture an image of the object as the object's visual data, or use a data acquisition device to collect a point cloud of the object as the object's visual data. Users can also create a voxel model of the object as the object's visual data, or obtain at least one of an image, point cloud, or voxel model of the object via the internet as the object's visual data.

[0162] The visual data of the objects mentioned here can be the visual data of a single object or the visual data of a scene including multiple objects. For example, if a user wants a 3D model of a car, he can obtain one or more of the pictures, videos, point clouds, and voxel models of the car and upload them to a computing device. For another example, if a user wants a 3D model of a car manufacturing plant, which includes multiple objects such as various machines and cars on a production line, in this case, the user can obtain one or more of the pictures, videos, point clouds, and voxel models of the plant and upload them to a computing device.

[0163] In a possible implementation, after receiving the visual data of the object uploaded by the user, the visual data may be first filtered to remove redundant data, sensitive data, abnormal data, etc.

[0164] Redundant data can be repeated data. For example, if the visual data uploaded by the user includes two identical images, it is redundant data. In this case, one of the images will be removed as redundant data through data filtering. Sensitive data can be data containing sensitive information. For example, if the visual data uploaded by the user includes images with clear faces, clear faces involve personal privacy and are sensitive data. In this case, images with clear faces will be removed through data filtering. Abnormal data can be data of poor quality. For example, if the visual data uploaded by the user includes overexposed or dark images, it is abnormal data. In this case, the overexposed or dark images will be removed through data filtering.

[0165] In one possible implementation, a computing device can obtain the object instance label and object category label for the visual data of an object uploaded by a user. For example, if the object is a car of brand A, the object instance label can be used to describe the object as a car or a car of brand A, while the object category label can be used to describe the object as a car.

[0166] If the visual data uploaded by the user is of a single object and the user also uploads an object instance label, the computing device does not need to label the visual data and can directly use the object instance label uploaded by the user. If the user does not upload the object instance label, the computing device can label the visual data to obtain the object instance label. After obtaining the object instance label, the object category label can also be obtained.

[0167] In the case where the visual data of the object uploaded by the user is the visual data of a scene including multiple objects, if the user also uploads the object instance labels of each object in the scene when uploading the visual data of the scene, the computing device does not need to segment and annotate the visual data of the scene, and can directly use the object instance labels uploaded by the user. If the user does not upload the object instance labels when uploading the visual data of the scene, the visual data of the scene can be segmented and annotated to obtain the object instance labels of each object in the scene. Among them, segmentation and annotation can be implemented in many ways. For example, the visual data of the scene is input into a segmentation and annotation model, and the segmentation and annotation model performs segmentation and annotation processing on the visual data and outputs the object instance labels of each object in the scene. The segmentation and annotation model mentioned here can be a neural network model. After obtaining the object instance labels of each object in the scene, the object category labels of each object can be further obtained.

[0168] There are many methods for obtaining object category labels, and several of them are listed below for illustration.

[0169] Method 1: A priori knowledge base containing multiple object category labels can be deployed on a computing device. Using natural language processing, the object instance labels are converted into numerical vectors. This numerical vector is then input into a classifier, which outputs the probability that the object belongs to the object category indicated by each object category label in the priori knowledge base. The object category label with the highest probability is determined as the object's corresponding label.

[0170] Method 2: A priori knowledge base can be deployed on the computing device, storing multiple object category labels. The object instance label and the object category labels stored in the priori knowledge base are input into the language model. The language model outputs the object category label corresponding to the object category. The object category label output by the language model is one of the object category labels stored in the priori knowledge base.

[0171] When the object category label of the object is obtained, the computing device may store the object category label and the visual data of the object in a digital asset library in correspondence with each other.

[0172] Step 202: Extract geometric features from the visual data of the object to obtain the first geometric features of the object.

[0173] Among them, geometric features can be expressed in vector form, and geometric features are used to characterize the contour features, area, perimeter, direction, etc. of an object.

[0174] In implementations, if the user uploads visual data of a single object, the computing device performs geometric feature extraction on the visual data of the object to obtain the first geometric feature of the object. If the user uploads visual data of a scene, the computing device may perform geometric feature extraction on the visual data of each object in the scene obtained by segmentation of the visual data of the scene to obtain the first geometric feature of each object.

[0175] There are many methods for extracting geometric features, and several of them are listed below for illustration.

[0176] The contour features of the object are extracted through the boundary feature method or the Fourier shape description method, and the area, perimeter, direction and other features of the object are extracted through the geometric parameter method or the shape invariant moment method.

[0177] Step 203: Acquire prior knowledge corresponding to the object category to which the object belongs from the prior knowledge base.

[0178] Among them, prior knowledge is used to guide the establishment of a geometric generative network, and the prior knowledge includes a first geometric latent space network corresponding to the object category to which the object belongs and a first geometric generative network hyperparameter.

[0179] During implementation, a priori knowledge base can be deployed on the computing device, and the prior knowledge base stores the geometric latent space network and geometric generative network hyperparameters corresponding to the object category. The geometric latent space network is used to generate the geometric generative network parameters. The geometric generative network parameters include weights, biases, etc. in the geometric generative network. The geometric generative network hyperparameters include the learning rate, number of iterations, optimization algorithm, number of perception layers, convolution kernel size, etc. of the geometric generative network. The geometric generative network hyperparameters will not change during the training process of the geometric generative network.

[0180] When the user uploads visual data of a single object, the computing device obtains the hyperparameters of the first geometric latent space network and the first geometric generative network corresponding to the object category to which the object belongs from the prior knowledge base.

[0181] When the user uploads visual data of a scene, for each object included in the scene, the computing device obtains the first geometric latent space network and the first geometric generative network hyperparameters corresponding to the object category to which the object belongs from the prior knowledge base.

[0182] Step 204: Establish a first geometric generative network based on the first geometric feature and the prior knowledge corresponding to the object category to which the object belongs.

[0183] In an implementation, when the user uploads visual data of a single object, the computing device establishes a first geometric generation network for generating a mesh model of the object.

[0184] When the user uploads visual data of a scene, for each object included in the scene, the computing device establishes a first geometry generation network for generating a mesh model of the object.

[0185] The establishment of the first geometric generation network is described below.

[0186] The user inputs the first geometric feature of an object into the first geometric latent space network, which then outputs the parameters of the first geometric generative network. An initial geometric generative network is established based on the first geometric generative network parameters and the first geometric generative network hyperparameters. The hyperparameters of the initial geometric generative network are the first geometric generative network hyperparameters, and the parameters of the initial geometric generative network are the first geometric generative network parameters. The initial geometric generative network is then trained based on the first geometric feature to obtain the first geometric generative network. During the training process, the hyperparameters of the initial geometric generative network remain unchanged, while the parameters of the initial geometric generative network are modified.

[0187] Step 205: Generate a first mesh model of the object based on the first geometry generation network.

[0188] In implementation, when the user uploads visual data of a single object, the computing device generates a first mesh model of the object based on a first geometric generation network corresponding to the object.

[0189] In the case where the user uploads visual data of a scene, for each object included in the scene, the computing device generates a first mesh model of the object based on a first geometric generation network corresponding to the object.

[0190] The following describes the process of training the initial geometry generation network and generating the first mesh model. Referring to FIG3 , the process may include the following steps.

[0191] Step 2051: Input the random signal into the initial geometry generation network, and the initial geometry generation network outputs a second mesh model.

[0192] Step 2052: extract the second geometric feature of the second mesh model, and calculate the first geometric loss of the first geometric feature and the second geometric feature.

[0193] In step 2053, if the first geometric loss is less than or equal to the first threshold, the second mesh model is used as the first mesh model of the object, and the initial geometric generation network is used as the first geometric generation network. Steps 2054 to 2059 are no longer executed. The first threshold can be configured based on the user's requirements for mesh model accuracy. For example, it can be set between 0 and 0.1, such as 0.01.

[0194] Step 2054: If the first geometric loss is greater than the first threshold, supervised training is performed on the initial geometric generative network based on the first geometric loss value to obtain a trained geometric generative network.

[0195] Step 2055: Input the random signal into the trained geometry generation network, and the trained geometry generation network outputs a third mesh model.

[0196] Step 2056: extract the third geometric feature of the third mesh model, and calculate the second geometric loss of the first geometric feature and the third geometric feature.

[0197] Step 2057: Determine whether the second geometric loss is greater than the first threshold.

[0198] Step 2058: If the second geometric loss is less than or equal to the first threshold, the third mesh model is used as the first mesh model of the object, the most recently trained geometry generation network is used as the first geometry generation network, and the loop is stopped.

[0199] Step 2059: If the second geometric loss is greater than the first threshold, the most recently trained geometric generative network is supervised again based on the second geometric loss value to obtain a trained geometric generative network, and the process then goes to step 2055.

[0200] The above steps can deterministically generate a mesh model of the object, and the generated mesh model is more accurate.

[0201] In one possible implementation, the geometric latent space network in the prior knowledge base can be optimized. The optimization process can be as follows:

[0202] When the trained first geometry generative network is used to generate a first mesh model of an object, the second geometry generative network parameters of the first geometry generative network can be obtained. The first geometric features are used as input samples, and the second geometry generative network parameters are used as output samples to perform supervised training on the first geometric latent space network to obtain a trained geometric latent space network. Furthermore, the first geometric latent space network stored in the prior knowledge base can be updated with the trained geometric latent space network.

[0203] In a possible implementation, the first geometric generative network of the object may be stored in a digital asset library. During storage, the object instance label, the object category label, and the first geometric generative network of the object may be stored correspondingly.

[0204] In a possible implementation, the first geometric feature of the object may be stored in a priori knowledge base. During storage, the first geometric feature of the object and the object category label and object instance label of the object may be stored in correspondence with each other.

[0205] In a possible implementation, the first mesh model of the object generated in step 205 is a geometric white model without texture. On this basis, referring to FIG. 4 , the computing device may further perform the following steps.

[0206] Step 206: Obtain a first texture map of the object.

[0207] In implementation, there may be two situations in which the computing device obtains the first texture map of the object, and these two situations are respectively described below.

[0208] Case 1: The visual data of the object uploaded by the user is one or more of a non-color point cloud and a non-color voxel model.

[0209] In this case, the computing device cannot directly extract the first texture map of the object from the visual data of the object. The user can specify the texture style, color type, color brightness, etc. of the texture to be used. The user can select the texture style, color type, color brightness, etc. of the texture to be used in the texture selection interface of the user device, and send a texture selection request to the computing device through the user device, wherein the texture selection request can carry texture style indication information, color type indication information, and color brightness indication information. Accordingly, the computing device can obtain a texture map corresponding to the texture type indication information, color type indication information, and color brightness indication information from the texture map library as the first texture map of the object. Alternatively, the computing device can also obtain a texture generation network corresponding to the texture type indication information, color type indication information, and color brightness indication information from the texture generation network library, input a random signal into the texture generation network, and the texture generation network outputs the first texture map of the object.

[0210] Case 2: The visual data of the object uploaded by the user is one or more of an image, a color point cloud, and a color voxel model.

[0211] In this case, when the computing device obtains the prior knowledge corresponding to the object category to which the object belongs, in addition to obtaining the first geometric latent space network and the first geometric generative network hyperparameters, it can also obtain the first texture latent space network and the first texture generative network hyperparameters, wherein the first texture latent space network and the first texture generative network hyperparameters also belong to the prior knowledge corresponding to the object category to which the object belongs. Then, the first texture map of the object can be generated by the following steps:

[0212] Step 2061: The computing device extracts texture features from the visual data of the object to obtain a first texture feature of the object.

[0213] Among them, the texture features of an object can be expressed in vector form. The texture features are used to characterize the RGB value of each pixel on the image of the object, the contrast of the image, the grayscale distribution of the image, the resolution of the image, the brightness of the pixel, etc.

[0214] In implementation, if the visual data of the object is an image, a color point cloud, or a color voxel model, the computing device can directly perform texture feature extraction on the visual data of the object to obtain the first texture feature of the object.

[0215] Step 2062: The computing device establishes a first texture generation network based on the first texture latent space network and the first texture generation network hyperparameters corresponding to the first texture feature and the object category to which the object belongs.

[0216] In implementation, a user inputs a first texture feature of an object into a first texture latent space network, which then outputs parameters of a first texture generation network. An initial texture generation network is established based on the first texture generation network parameters and the first texture generation network hyperparameters. The hyperparameters of the initial texture generation network are the first texture generation network hyperparameters, and the parameters of the initial texture generation network are the first texture generation network parameters. The initial texture generation network is then trained based on the first texture feature to obtain the first texture generation network. During the training process, the hyperparameters of the initial texture generation network remain unchanged, while the parameters of the initial texture generation network are modified.

[0217] Step 2063: Generate a first texture map of the object based on the first texture generation network.

[0218] The following describes the process of training the initial texture generation network and generating the first texture map, which may include the following steps:

[0219] S1. Input a random signal into an initial texture generation network, and the initial texture generation network outputs a second texture map.

[0220] S2. Extract a second texture feature from the second texture map, and calculate a first texture loss of the first texture feature and the second texture feature.

[0221] S3. If the first texture loss is less than or equal to a second threshold, the second texture map is used as the first texture map of the object, and the initial texture generation network is used as the first geometry generation network, and steps S4 to S8 are not performed. The second threshold can be configured based on the user's mesh model accuracy requirements, for example, it can be set between 0 and 0.1, such as 0.01.

[0222] S4. If the first texture loss is greater than the second threshold, supervised training is performed on the first texture generation network based on the first texture loss value to obtain a trained texture generation network.

[0223] S5. Input the random signal into the trained texture generation network, and the trained texture generation network outputs a third texture map.

[0224] S6. Extract a third texture feature of the third texture map, and calculate a second texture loss of the first texture feature and the third texture feature.

[0225] S7. If the second texture loss is less than or equal to the second threshold, the third texture map is used as the first texture map of the object, the texture generation network after the latest training is used as the above-mentioned first texture generation network, and the loop is stopped.

[0226] S8. If the second texture loss is greater than the second threshold, the texture generation network after the latest training is supervised and trained again based on the second texture loss value to obtain the trained texture generation network, and then go to step S5.

[0227] Step 207: Render the first texture map onto the first mesh model to obtain a first textured mesh model of the object.

[0228] During implementation, the computing device may render the first texture map onto the first mesh model through a renderer to obtain a first textured mesh model of the object.

[0229] In one possible implementation, the texture latent space network in the prior knowledge base can be optimized. The optimization process can be as follows:

[0230] When a trained texture generative network is used to generate a first texture model of an object, parameters of a second texture generative network of the trained texture generative network can be obtained. The first texture features are used as input samples, and the parameters of the second texture generative network are used as output samples to perform supervised training on the first texture latent space network to obtain a trained texture latent space network. Furthermore, the first texture latent space network stored in the prior knowledge base can be updated to the trained texture latent space network.

[0231] In a possible implementation, the first texture generation network of the object may be stored in a digital asset library. During storage, the object instance label, object category label, first geometry generation network, and first texture generation network of the object may be stored correspondingly.

[0232] In a possible implementation, the first texture feature of the object may be stored in a priori knowledge base. During storage, the first texture feature of the object and the object category label, object instance label, and first geometric feature of the object may be stored correspondingly.

[0233] In one possible implementation, when the user uploads visual data of a scene, a mesh model of the scene may be further generated based on the mesh models of the objects in the scene. Accordingly, the processing may be as follows:

[0234] The computing device extracts spatial features from each object in the visual data of the scene uploaded by the user to obtain spatial features corresponding to each object, wherein the spatial features are used to characterize the position and posture of the corresponding object in the scene.

[0235] For each two objects in the scene, the confidence level of each of multiple preset positional relationships is calculated based on the spatial features corresponding to the two objects. Preset positional relationships include intersecting, adhering, containing, being contained, and separating. Intersecting indicates that there is overlap between the two objects, adhering indicates that the two objects are in surface contact, containing indicates that the other object is within the object, being contained indicates that the object is within the other object, and separating indicates that the two objects are not in contact and are not in a containing or being contained relationship.

[0236] Then, a preset position relationship corresponding to the highest confidence level is determined as the first position relationship between the two objects.

[0237] Finally, a cubic space is constructed, and the mesh model of each object in the scene is placed in the cubic space according to the first position relationship between every two objects.

[0238] In addition, since the calculated positional relationship between objects may not be accurate, the position of the mesh model of each object can be adjusted based on prior knowledge. The processing can be as follows:

[0239] For every two objects in the scene, determine the first object category and the second object category to which the two objects belong respectively. In the prior knowledge base, obtain the positional relationship between the objects of the first object category and the objects of the second object category in the scene as the reference confidence of each preset positional relationship. For each preset positional relationship, multiply the confidence of the preset positional relationship by the reference confidence to obtain the comprehensive confidence of the preset positional relationship, and take the preset positional relationship corresponding to the highest comprehensive confidence as the second positional relationship between the two objects. According to the second positional relationship between every two objects in the scene, the positions of the grid models corresponding to the two objects in the grid model of the first scene are adjusted. For example, the scene includes object A and object B, and the first positional relationship between object A and object B is intersection, and the second positional relationship is contact, then the positions of object A and object B are adjusted from intersection to contact.

[0240] In addition, if a textured mesh model of an object is generated, a textured mesh model of a scene can be generated. The specific generation method is the same as the method for generating the mesh model of the scene, and will not be repeated here.

[0241] In one possible implementation, the user may specify the model type and model format of the generated 3D model, and accordingly, the computing device may convert the generated mesh model into a 3D model of the model type and model format specified by the user. Specifically, the processing may be as follows:

[0242] A user sends a 3D model conversion request to a computing device via a user device. The 3D model conversion request includes information indicating a target model type, and may also include information indicating a target model format and a target resolution. Upon receiving the 3D model conversion request, the computing device converts the mesh model of the object into a 3D model of the target model type, target model format, and target resolution.

[0243] There are several ways to convert 3D models:

[0244] 1. The user needs a mesh model in fbx format. The mesh model generated and stored by the computing device can be in obj format. Based on this, the computing device can read the obj format mesh model, store it in fbx format, and obtain the mesh model in fbx format.

[0245] Second, the user requires a point cloud model in PCD format at the target resolution. The computing device reads the mesh model in OBJ format, samples points at the target resolution, and stores them in PCD format. This yields a point cloud model in PCD format at the target resolution.

[0246] 3. The user requires a voxel model in binvox format at the target resolution. The computing device reads the mesh model in obj format, sets the number of voxel cells according to the target resolution, divides the space using an octree, and stores it in binvox format to obtain a voxel model in binvox format at the target resolution.

[0247] In one possible implementation, in order to obtain mesh models of more objects to meet the needs of large model training, the embodiment of the present application also provides a method for self-growth of 3D digital assets, through which a computing device can automatically generate a 3D model. Accordingly, as shown in FIG5 , the process may include the following steps:

[0248] Step 301: Receive a first 3D digital asset self-growth request sent by a user.

[0249] In implementations, if a user lacks visual data of an object, or needs to quickly generate textured mesh models for a large number of objects, they can specify on their device a first object in the third object category as the starting point for growth and a second object as the final point for growth. This allows the computing device to generate textured mesh models for objects with geometric and textural features between the first and second objects. For example, if the third object category is automobiles, the first object is a sports car, and the second object is a sport utility vehicle (SUV), the computing device can generate textured mesh models for automobiles with appearances between those of sports cars and SUVs.

[0250] In addition to specifying the third object category, the starting object for growth, and the target number of textured mesh models to be generated, the user on the user device can also specify. Accordingly, the user device can send a first 3D digital asset self-growth request to the computing device, where the first 3D digital asset self-growth request includes information indicating the first object, information indicating the second object, and the target number, where the first object and the second object are different objects under the third object category.

[0251] Step 302: From the prior knowledge base, obtain the geometric features of the first object, the texture features of the first object, the geometric features of the second object, the texture features of the second object, a second geometric latent space network corresponding to a third object type, hyperparameters of a second geometric generative network corresponding to the third object type, a second texture latent space network corresponding to the third object type, and parameters of a second texture generative network corresponding to the third object type.

[0252] Step 303: Taking the geometric features of the first object as the starting point and the geometric features of the second object as the end point, generate a target number of intermediate geometric features; taking the texture features of the first object as the starting point and the texture features of the second object as the end point, generate a target number of intermediate texture features, wherein the target number of intermediate texture features and the target number of intermediate geometric features correspond one to one.

[0253] The following describes how to generate a target number of intermediate geometric features, starting with the geometric features of the first object and ending with the geometric features of the second object:

[0254] The geometric features of the first object are vectors with n elements. The geometric features of the second object are also vectors with n elements. The first element of the geometric features of the first object is denoted as A1, and the first element of the geometric features of the second object is denoted as B1. The difference between B1 and A1 is calculated as C1. C1 is divided by (x+1) to obtain the target step length, denoted as D, where x is the target number. A1+D is used as the first element of the first intermediate geometric feature, A1+2D is used as the second element of the second intermediate geometric feature, and so on. The first element of each of the x intermediate geometric features can be obtained in the same way. The remaining elements of each of the x intermediate geometric features can be obtained. In this way, x intermediate geometric features can be obtained.

[0255] The method of generating the target number of intermediate texture features with the texture features of the first object as the starting point and the texture features of the second object as the end point is the same as the method of generating the intermediate geometric features described above, which will not be repeated here.

[0256] In one possible implementation, the user may also specify a target step size, and accordingly, the first 3D digital asset self-growth request may also carry the target step size. In this case, the computing device may use the target step size carried in the first 3D digital asset self-growth request to calculate intermediate geometric features and intermediate texture features.

[0257] Step 304: For the corresponding intermediate geometric features and intermediate texture features, the intermediate geometric features are input into a second geometric latent space network to obtain second geometric generative network parameters, a second geometric generative network is established based on the second geometric generative network parameters and the second geometric generative network hyperparameters, a second mesh model is generated based on the second geometric generative network, the second texture features are input into a second texture latent space network to obtain second texture generative network parameters, a third texture generative network is established based on the second texture generative network parameters and the second texture generative network hyperparameters, a second texture map is generated based on the third texture generative network, the second texture map is rendered onto the second mesh model to obtain a second textured mesh model.

[0258] Step 305: The generated target number of second textured mesh models may be stored in the digital asset library. When storing, they may be stored in correspondence with the third object category label.

[0259] In a possible implementation, when a 3D digital asset is self-grown, it is possible not to obtain texture features and texture maps, and ultimately only generate a mesh model without texture.

[0260] In a possible implementation, the embodiment of the present application further provides a method for automatically generating a grid model of a scene. Accordingly, as shown in FIG6 , the process may include the following steps:

[0261] Step 401: Receive a second 3D digital asset self-growth request sent by a user.

[0262] In implementation, if a user wishes to obtain a mesh model of a target scene, they can specify the target scene and the objects included in the target scene through their user device. In response, the user device can send a second 3D digital asset self-growth request to the computing device. The second 3D digital asset self-growth request can include information indicating the target scene, information indicating the third object, and information indicating the fourth object.

[0263] Step 402: Obtain a textured mesh model of a third object and a textured mesh model of a fourth object from the digital asset library.

[0264] Step 403: From the prior knowledge base, obtain the reference confidence level for each preset positional relationship between the third and fourth objects in the target scene. The preset positional relationship corresponding to the maximum parameter confidence level is used as the positional relationship between the third and fourth objects. A cubic space is constructed, and the textured mesh model of the third object and the textured mesh model of the fourth object are placed in the cubic space based on the positional relationship between the third and fourth objects to obtain a mesh model of the second scene.

[0265] Step 404: Store the generated mesh model of the target scene in the digital asset library.

[0266] During implementation, the computing device may store the generated mesh model of the target scene and the scene label of the target scene in a corresponding manner in the digital asset library.

[0267] In addition, here we use the example of using textured mesh models of two objects to generate a mesh model of a scene. During implementation, we can also use textured mesh models of more objects to generate a mesh model of a scene. When using textured mesh models of three or more objects to generate a mesh model of a scene, we need to determine the positional relationship between every two objects, and then place the textured mesh models of each object to generate a textured mesh model of the scene.

[0268] In a possible implementation, when the 3D digital asset is self-growing, a non-textured mesh model of the object may be obtained to generate a non-textured mesh model of the scene.

[0269] In one possible implementation, to save storage space, the digital asset library may not store the textured mesh model itself, but instead stores the geometry generation network used to generate the mesh model and the texture generation network used to generate the texture map. This way, when obtaining a textured mesh model for an object, one can first obtain the object's geometry generation network and texture generation network. The obtained geometry generation network is then used to generate the object's mesh model, and the texture generation network is used to generate the object's texture map. Finally, the object's texture map is rendered onto the object's mesh model to obtain the object's textured mesh model.

[0270] In a possible implementation, the embodiment of the present application further provides a method for establishing a priori knowledge base. Accordingly, the process may be as follows:

[0271] Initially, the prior knowledge base lacks any prior information. Public mesh models can be found in open source datasets or searched online. Currently, many manually constructed models are available online, covering most real-world object categories. Objects of different object categories correspond to different structures of the corresponding GNN, i.e., different GNN hyperparameters. For example, a car is a more complex object than an apple, and its corresponding GNN should have more layers. Therefore, for an unknown, never-before-seen object, multiple possible hyperparameter combinations need to be explored. A large model can be used to assess the object's complexity, providing several possible GNN hyperparameters. This generates neural networks with various structures, selecting the set of GNN hyperparameters that yields the highest generation efficiency and quality. The network structure defined by the optimal GNN hyperparameters is then trained to produce a mesh model that closely resembles the input object and a fine-tuned GNN. The extracted geometric features of the mesh model are used as input, and the fine-tuned GNN parameters are used as output to train the corresponding GNN for that object category. The establishment of prior knowledge related to textures is similar and will not be further elaborated here.

[0272] In one possible implementation, the steps shown in FIG. 2 may all be implemented by a cloud computing device, or may all be implemented by a user's local computing device.

[0273] In one possible implementation, in order to enhance user privacy protection, the visual data collected by the user does not need to be uploaded to the cloud. In this case, the steps shown in FIG2 can all be performed by the user's local computing device, and the prior knowledge base can be deployed in the cloud. When it comes to the processing of obtaining various types of prior knowledge in the prior library, the user's local computing device and the cloud can interact to obtain the corresponding prior knowledge.

[0274] The present application also provides a device for generating a grid model, which is applied to a computing device. Referring to FIG7 , the device includes: a data preprocessing module 410, a feature extraction module 420, and an AIGX module 430, wherein:

[0275] A data preprocessing module 410 is configured to obtain visual data of an object uploaded by a user, wherein the visual data includes at least one of an image, a point cloud, and a voxel model;

[0276] A feature extraction module 420 is configured to extract geometric features from the visual data of the object to obtain a first geometric feature of the object;

[0277] AIGX module 430 is used to obtain prior knowledge corresponding to the object category to which the object belongs from a prior knowledge base, wherein the prior knowledge is used to guide the establishment of a geometry generation network; establish a first geometry generation network based on the first geometric features and the prior knowledge corresponding to the object category to which the object belongs; and generate a first mesh model of the object based on the first geometry generation network.

[0278] In one possible implementation, the prior knowledge corresponding to the object category to which the object belongs includes hyperparameters of a first geometric latent space network and a first geometric generative network, and the AIGX module 430 is configured to:

[0279] Inputting the first geometric feature into the first geometric latent space network to obtain first geometric generative network parameters;

[0280] Establishing an initial geometry generative network based on the first geometry generative network parameters and the first geometry generative network hyperparameters;

[0281] The initial geometry generation network is trained based on the first geometric feature to obtain a first geometry generation network.

[0282] In one possible implementation, the AIGX module 430 is configured to:

[0283] training the first geometric latent space network based on the first geometric features and the geometric generative network parameters of the first geometric generative network to obtain a trained geometric latent space network;

[0284] The first geometric latent space network stored in the prior knowledge base is replaced by the trained geometric latent space network.

[0285] In a possible implementation, the data preprocessing module 410 is configured to:

[0286] Obtaining visual data of a first scene uploaded by a user, wherein the first scene includes a plurality of objects;

[0287] Object segmentation is performed on the visual data of the first scene to obtain visual data of each of the multiple objects.

[0288] In a possible implementation, the apparatus further includes a positioning module, configured to:

[0289] Performing spatial feature extraction on the multiple objects in the visual data of the first scene to obtain spatial features corresponding to the multiple objects, wherein the spatial features are used to represent the positions of the corresponding objects in the first scene;

[0290] According to the spatial features corresponding to the multiple objects, the grid models corresponding to the multiple objects are placed to obtain the grid model of the first scene.

[0291] In a possible implementation, the apparatus further includes a scene optimization module, configured to:

[0292] For each two objects in the plurality of objects, calculating, based on the spatial features corresponding to the two objects, a confidence level that the positional relationship between the two objects is each of a plurality of preset positional relationships;

[0293] determining a first object category and a second object category to which the two objects respectively belong;

[0294] In the prior knowledge base, obtaining a positional relationship between an object of the first object category and an object of the second object category in the first scene as a reference confidence level of each preset positional relationship;

[0295] Determining a positional relationship between the two objects based on the confidence level and the reference confidence level;

[0296] According to the positional relationship between the two objects, positions of the grid models corresponding to the two objects in the grid model of the first scene are adjusted.

[0297] In a possible implementation, the AIGX module 430 is further configured to:

[0298] Obtaining a first texture map of the object;

[0299] Rendering the first texture map onto the first mesh model to obtain a first textured mesh model of the object.

[0300] In one possible implementation, the visual data is at least one of a non-color point cloud and a non-color voxel model, and the AIGX module is configured to:

[0301] Receiving a texture selection request sent by a user, wherein the texture selection request carries texture style indication information, color type indication information, and color brightness indication information;

[0302] Acquire a first texture generation network corresponding to the texture type indication information, the color type indication information, and the color brightness indication information;

[0303] Based on the first texture generation network, a first texture map of the object is generated.

[0304] In a possible implementation, the prior knowledge is further used to guide the establishment of a texture generation network, and the feature extraction module 420 is further used to:

[0305] Extracting texture features from the visual data of the object to obtain a first texture feature of the object;

[0306] Establishing a second texture generation network based on the first texture feature and prior knowledge of the object;

[0307] The AIGX module 430 is configured to:

[0308] Based on the second texture generation network, a first texture map of the object is generated.

[0309] In one possible implementation, the prior knowledge includes hyperparameters of a first texture latent space network and a first texture generation network, and the AIGX module is configured to:

[0310] Inputting the first texture feature into the first texture latent space network to obtain first texture generation network parameters;

[0311] Establishing an initial texture generation network based on the first texture generation network parameters and the first texture generation network hyperparameters;

[0312] Based on the first texture feature, the initial texture generation network is trained to obtain a second texture generation network;

[0313] Based on the second texture generation network, a first texture map of the object is generated.

[0314] In a possible implementation, the AIGX module 430 is further configured to:

[0315] Training the first texture latent space network based on the first texture feature and the texture generation network parameters of the trained texture generation network to obtain a trained texture latent space network;

[0316] The first texture latent space network stored in the prior knowledge base is replaced by the trained texture latent space network.

[0317] In a possible implementation, the data preprocessing module is further configured to:

[0318] Based on the object category labels stored in the prior knowledge base, an object category label corresponding to the object is determined.

[0319] In a possible implementation, the AIGX module 430 is further configured to:

[0320] Storing the first geometric feature of the object in the prior knowledge base;

[0321] receiving a first 3D digital asset self-growth request from a user, wherein the first 3D digital asset self-growth request carries information indicating a first growth starting object, information indicating a first growth ending object, and a target quantity, wherein the first growth starting object and the first growth ending object are different objects under a third object category;

[0322] Acquire, from the prior knowledge base, geometric features of the first growth starting point object, geometric features of the first growth end point object, and prior knowledge corresponding to the third object type;

[0323] Taking the geometric features of the first growth starting point object as a starting point and the geometric features of the first growth ending point object as an ending point, generating the target number of intermediate geometric features;

[0324] For each intermediate geometric feature of the target number of intermediate geometric features, a second geometry generation network is generated based on prior knowledge corresponding to the intermediate geometric feature and the third object type, and a second mesh model is generated based on the second geometry generation network.

[0325] In a possible implementation, the AIGX module 430 is further configured to:

[0326] Storing the first geometric feature of the object in the prior knowledge base;

[0327] receiving a second 3D digital asset self-growth request from a user, wherein the second 3D digital asset self-growth request carries information indicating a second growth starting object, information indicating a second growth ending object, and a target step size, and the second growth starting object and the second growth ending object are different objects under a fourth object category;

[0328] Acquiring, from the prior knowledge base, geometric features of the second growth starting point object, geometric features of the second growth end point object, and prior knowledge corresponding to the fourth object type;

[0329] generating at least one intermediate geometric feature by taking the geometric feature of the second growth starting point object as a starting point and the geometric feature of the second growth ending point object as an ending point;

[0330] For each intermediate geometric feature of the at least one intermediate geometric feature, a third geometry generation network is generated based on prior knowledge corresponding to the intermediate geometric feature and the fourth object type, and a third mesh model is generated based on the third geometry generation network.

[0331] In a possible implementation, the AIGX module 430 is further configured to:

[0332] receiving a third 3D digital asset self-growth request sent by a user, wherein the third 3D digital asset self-growth request carries indication information of a second scene and indication information of a plurality of objects in the second scene;

[0333] Obtaining mesh models of multiple objects in the second scene from a digital asset library;

[0334] In the prior knowledge base, obtaining a reference confidence level of each of a plurality of preset positional relationships for a positional relationship between each two objects in the second scene;

[0335] determining a positional relationship between each two objects in the second scene based on a reference confidence level that the positional relationship between each two objects in the second scene is each of a plurality of preset positional relationships;

[0336] The grid models of the multiple objects in the second scene are placed according to the positional relationship between every two objects in the second scene to obtain the grid model of the second scene.

[0337] Data preprocessing module 410, feature extraction module 420, and AIGX module 430 can all be implemented via software or hardware. For example, the implementation of data preprocessing module 410 will be described below using data preprocessing module 410 as an example. Similarly, the implementation of feature extraction module 420 and AIGX module 430 can refer to the implementation of data preprocessing module 410.

[0338] As an example of a software functional unit, the data preprocessing module 410 may include code running on a computing instance. The computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Furthermore, the computing instance may be one or more. For example, the task mining training module 501 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed in the same region or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including one data center or multiple geographically close data centers. Typically, a region may include multiple AZs.

[0339] Similarly, multiple hosts / virtual machines / containers running the code can be distributed within the same virtual private cloud (VPC) or across multiple VPCs. Typically, a VPC is set up within a region. Cross-region communication between two VPCs within the same region, or between VPCs in different regions, requires a communication gateway within each VPC to interconnect the VPCs.

[0340] As an example of a hardware functional unit, the data pre-processing module 410 may include at least one computing device, such as a server. Alternatively, the data pre-processing module 410 may be implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0341] The multiple computing devices included in data preprocessing module 410 can be distributed in the same region or in different regions. The multiple computing devices included in module A can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in data preprocessing module 410 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of servers, ASICs, PLDs, CPLDs, FPGAs, GALs, and other computing devices.

[0342] It should be noted that in other embodiments, the data preprocessing module 410 can be used to execute any step performed by the computing device in the method for generating a mesh model, the feature extraction module 420 can be used to execute any step performed by the computing device in the method for generating a mesh model, and the AIGX module 430 can be used to execute any step performed by the computing device in the method for generating a mesh model. The steps that the data preprocessing module 410, the feature extraction module 420, and the AIGX module 430 are responsible for implementing can be specified as needed. The full functionality of the device for generating a mesh model is achieved by having the data preprocessing module 410, the feature extraction module 420, and the AIGX module 430 respectively implement different steps in the 3D model generation method.

[0343] The present application also provides a computing device 100, which can be the computing device in the above embodiment. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smartphone.

[0344] As shown in Figure 8, computing device 100 includes a bus 102, a processor 104, a memory 106, and a communication interface 108. Processor 104, memory 106, and communication interface 108 communicate with each other via bus 102. It should be understood that this application does not limit the number of processors and memories in computing device 100.

[0345] Bus 102 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, among others. Buses may be classified as address buses, data buses, control buses, and the like. For ease of illustration, FIG7 illustrates a single bus line, but this does not imply a single bus or type of bus. Bus 102 may include a path for transmitting information between various components of computing device 100 (e.g., memory 106, processor 104, and communication interface 108).

[0346] The processor 104 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0347] The memory 106 may include volatile memory, such as random access memory (RAM). The memory 106 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0348] Memory 106 stores executable code, which processor 104 executes to implement the functions of the aforementioned data preprocessing module 410, feature extraction module 420, and AIGX module 430, thereby implementing the processing performed by the computing device in the 3D model generation method. Specifically, memory 106 stores instructions for executing the mesh model generation method.

[0349] The communication interface 108 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 100 and other devices or a communication network.

[0350] Embodiments of the present application also provide a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smartphone.

[0351] As shown in Figure 9, the computing device cluster includes at least one computing device 100. The memory 106 of one or more computing devices 100 in the computing device cluster may store the same instructions for executing the method for generating a mesh model.

[0352] In some possible implementations, the memory 106 of one or more computing devices 100 in the computing device cluster may also store partial instructions for executing the method for generating a mesh model. In other words, the combination of one or more computing devices 100 can jointly execute the instructions for executing the method for generating a mesh model.

[0353] It should be noted that the memories 106 in different computing devices 100 in the computing device cluster may store different instructions, each for executing a portion of the functions of the method for generating a grid model.

[0354] In some possible implementations, the memory 106 of one or more computing devices 100 in the computing device cluster may also store partial instructions for executing the method for generating a mesh model. In other words, the combination of one or more computing devices 100 can jointly execute the instructions for executing the method for generating a 3D model.

[0355] It should be noted that the memory 106 in different computing devices 100 in the computing device cluster can store different instructions for executing portions of the 3D model generation function. In other words, the instructions stored in the memory 106 in different computing devices 100 can implement the functions of one or more nodes in the data preprocessing module 410, the feature extraction module 420, and the AIGX module 430.

[0356] In some possible implementations, one or more computing devices in a computing device cluster may be connected via a network. The network may be a wide area network (WAN) or a local area network (LAN), among others. FIG10 illustrates one possible implementation. As shown in FIG10 , two computing devices 100A and 100B are connected via a network. Specifically, the connection to the network is achieved via a communication interface within each computing device. In this type of possible implementation, the memory 106 within computing device 100A stores instructions for executing the AIGX module 430. Simultaneously, the memory 106 within computing device 100B stores instructions for executing the functions of the data preprocessing module 410 and the feature extraction module 420. It should be understood that the functions of computing device 100A shown in FIG10 may also be performed by multiple computing devices 100. Similarly, the functions of computing device 100B may also be performed by multiple computing devices 100.

[0357] The present application also provides another computing device cluster. The connection relationship between the computing devices in this computing device cluster can be similar to the connection relationship between the computing device clusters described in Figures 9 and 10. However, the memory 106 of one or more computing devices 100 in this computing device cluster can store the same instructions for executing the grid model generation method.

[0358] In some possible implementations, the memory 106 of one or more computing devices 100 in the computing device cluster may also store partial instructions for executing the method for generating a mesh model. In other words, the combination of one or more computing devices 100 can jointly execute the instructions for executing the method for generating a mesh model.

[0359] It should be noted that the memory 106 in different computing devices 100 in the computing device cluster may store different instructions for executing portions of the mesh model generation method. In other words, the instructions stored in the memory 106 in different computing devices 100 may implement the functions of one or more nodes of the data preprocessing module 410, the feature extraction module 420, and the AIGX module 430.

[0360] The present application also provides a computer program product containing instructions. The computer program product may be software or a program product containing instructions that can be run on a computing device or stored on any available medium. When the computer program product is run on a computing device, it causes at least one computing device to execute the method for generating a grid model provided in the present application.

[0361] The present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computing device, or a data storage device such as a data center that contains one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a magnetic tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive). The computer-readable storage medium includes instructions that instruct the computing device to execute the method for generating a grid model provided in the present application.

[0362] In this application, the terms "first", "second", etc. are used to distinguish between identical or similar items having substantially the same effects and functions. It should be understood that there is no logical or temporal dependency between "first" and "second", nor is there a limit on quantity and execution order. It should also be understood that although the following description uses the terms first, second, etc. to describe various elements, these elements should not be limited by the terms. These terms are simply used to distinguish one element from another. For example, without departing from the scope of the various examples, a first geometric feature may be referred to as a second geometric feature, and similarly, a second geometric feature may be referred to as a first geometric feature. The first geometric feature and the second geometric feature may both be collectively referred to as computational codes, and in some cases, may be separate and different geometric features.

[0363] The term "at least one" in this application means one or more, and the term "plurality" in this application means two or more.

[0364] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and such modifications or substitutions should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A method for generating a mesh model, characterized in that: The method comprises: Acquire visual data of an object uploaded by a user, wherein the visual data includes at least one of an image, a point cloud, and a voxel model; Extracting geometric features from the visual data of the object to obtain a first geometric feature of the object; Acquire prior knowledge corresponding to the object category to which the object belongs in a prior knowledge base, wherein the prior knowledge is used to guide the establishment of a geometric generation network; Establishing a first geometric generative network based on the first geometric feature and prior knowledge corresponding to the object category to which the object belongs; Based on the first geometry generation network, a first mesh model of the object is generated.

2. The method according to claim 1, characterized in that The prior knowledge corresponding to the object category to which the object belongs includes a first geometric latent space network and a first geometric generative network hyperparameter, and the first geometric generative network is established based on the first geometric feature and the prior knowledge corresponding to the object category to which the object belongs, including: Inputting the first geometric feature into the first geometric latent space network to obtain first geometric generative network parameters; Establishing an initial geometry generative network based on the first geometry generative network parameters and the first geometry generative network hyperparameters; Based on the first geometric feature, the initial geometry generation network is trained to obtain a first geometry generation network.

3. The method according to claim 2, characterized in that The method further comprises: Based on the first geometric features and the geometric generative network parameters of the first geometric generative network, the first geometric latent space network is trained to obtain a trained geometric latent space network; The first geometric latent space network stored in the prior knowledge base is replaced by the trained geometric latent space network.

4. The method according to any one of claims 1 to 3, characterized in that The obtaining of visual data of the object uploaded by the user includes: Acquire visual data of a first scene uploaded by a user, wherein the first scene includes a plurality of objects; Perform object segmentation on the visual data of the first scene to obtain visual data of each of the multiple objects.

5. The method according to claim 4, characterized in that The method further comprises: Extracting spatial features of the multiple objects in the visual data of the first scene to obtain spatial features corresponding to the multiple objects, wherein the spatial features are used to characterize the positions and postures of the corresponding objects in the first scene; According to the spatial features respectively corresponding to the multiple objects, the grid models respectively corresponding to the multiple objects are placed to obtain the grid model of the first scene.

6. The method according to claim 5, characterized in that The method further comprises: For each two objects among the plurality of objects, calculating, according to the spatial features respectively corresponding to the two objects, a confidence level that the positional relationship between the two objects is each of a plurality of preset positional relationships; Determine a first object category and a second object category to which the two objects respectively belong; In the prior knowledge base, obtaining the positional relationship between the objects of the first object category and the objects of the second object category in the first scene as a reference confidence of each preset positional relationship; Determining a positional relationship between the two objects based on the confidence level and the reference confidence level; According to the positional relationship between the two objects, positions of the grid models respectively corresponding to the two objects in the grid model of the first scene are adjusted.

7. The method according to any one of claims 1 to 6, characterized in that The method further comprises: Obtaining a first texture map of the object; The first texture map is rendered onto the first mesh model to obtain a first textured mesh model of the object.

8. The method according to claim 7, characterized in that The visual data is at least one of a non-color point cloud and a non-color voxel model, and the method further includes: Receiving a texture selection request sent by a user, wherein the texture selection request carries texture style indication information, color type indication information, and color brightness indication information; The obtaining of a first texture map of the object comprises: Acquire a first texture generation network corresponding to the texture type indication information, the color type indication information, and the color brightness indication information; Based on the first texture generation network, a first texture map of the object is generated.

9. The method according to claim 7, characterized in that: The prior knowledge is also used to guide the establishment of a texture generation network. The method further includes: Extracting texture features from visual data of the object to obtain first texture features of the object; Based on the first texture feature and the prior knowledge of the object, establishing a second texture generation network; The obtaining of a first texture map of the object comprises: Based on the second texture generation network, a first texture map of the object is generated.

10. The method according to claim 9, characterized in that The prior knowledge includes a first texture latent space network and a first texture generation network hyperparameter, and the second texture generation network is established based on the first texture feature and the prior knowledge of the object, including: Inputting the first texture feature into the first texture latent space network to obtain first texture generation network parameters; Establishing an initial texture generation network based on the first texture generation network parameters and the first texture generation network hyperparameters; Based on the first texture feature, training the initial texture generation network to obtain a second texture generation network; Based on the second texture generation network, a first texture map of the object is generated.

11. The method according to claim 10, characterized in that The method further comprises: Based on the first texture feature and the texture generation network parameters of the trained texture generation network, the first texture latent space network is trained to obtain a trained texture latent space network; The first texture latent space network stored in the prior knowledge base is replaced by the trained texture latent space network.

12. The method according to any one of claims 1 to 11, characterized in that The method further comprises: Based on the object category labels stored in the prior knowledge base, an object category label corresponding to the object is determined.

13. The method according to any one of claims 1 to 12, characterized in that The method further comprises: Storing the first geometric feature of the object in the prior knowledge base; receiving a first 3D digital asset self-growth request sent by a user, wherein the first 3D digital asset self-growth request carries indication information of a first growth starting point object, indication information of a first growth end point object, and a target quantity, and the first growth starting point object and the first growth end point object are different objects under a third object category; In the prior knowledge base, the geometric features of the first growth starting point object, the geometric features of the first growth end point object, and the prior knowledge corresponding to the third object type are obtained; Taking the geometric features of the first growth starting point object as a starting point and the geometric features of the first growth ending point object as an ending point, generating the target number of intermediate geometric features; For each intermediate geometric feature of the target number of intermediate geometric features, a second geometry generation network is generated based on prior knowledge corresponding to the intermediate geometric feature and the third object type, and a second mesh model is generated based on the second geometry generation network.

14. The method according to any one of claims 1 to 13, characterized in that The method further comprises: Storing the first geometric feature of the object in the prior knowledge base; receiving a second 3D digital asset self-growth request sent by a user, wherein the second 3D digital asset self-growth request carries indication information of a second growth starting point object, indication information of a second growth end point object, and a target step length, and the second growth starting point object and the second growth end point object are different objects under a fourth object category; In the prior knowledge base, the geometric features of the second growth starting point object, the geometric features of the second growth end point object, and the prior knowledge corresponding to the fourth object type are obtained; Taking the geometric feature of the second growth starting point object as a starting point and the geometric feature of the second growth ending point object as an ending point, generating at least one intermediate geometric feature according to the target step length; For each of the at least one intermediate geometric feature, a third geometry generation network is generated based on prior knowledge corresponding to the intermediate geometric feature and the fourth object type, and a third mesh model is generated based on the third geometry generation network.

15. The method according to any one of claims 1 to 14, characterized in that The method further comprises: receiving a third 3D digital asset self-growth request sent by a user, wherein the third 3D digital asset self-growth request carries indication information of a second scene and indication information of a plurality of objects in the second scene; In the digital asset library, obtain mesh models of multiple objects in the second scene; In the prior knowledge base, obtaining a reference confidence level of each preset position relationship among a plurality of preset position relationships, that is, a position relationship between each two objects in the second scene; determining a positional relationship between each two objects in the second scene based on a reference confidence level that the positional relationship between each two objects in the second scene is each of a plurality of preset positional relationships; According to the positional relationship between every two objects in the second scene, grid models of multiple objects in the second scene are placed to obtain a grid model of the second scene.

16. A device for generating a grid model, characterized in that: The device comprises: A data preprocessing module, used to obtain visual data of an object uploaded by a user, wherein the visual data includes at least one of an image, a point cloud, and a voxel model; A feature extraction module, used for extracting geometric features from the visual data of the object to obtain a first geometric feature of the object; An artificial intelligence-based digital asset generation AIGX module is used to obtain prior knowledge corresponding to the object category to which the object belongs in a prior knowledge base, wherein the prior knowledge is used to guide the establishment of a geometry generation network; based on the first geometric feature and the prior knowledge corresponding to the object category to which the object belongs, a first geometry generation network is established; based on the first geometry generation network, a first mesh model of the object is generated.

17. The device according to claim 16, characterized in that The prior knowledge corresponding to the object category to which the object belongs includes hyperparameters of a first geometric latent space network and a first geometric generative network, and the AIGX module is used to: Inputting the first geometric feature into the first geometric latent space network to obtain first geometric generative network parameters; Establishing an initial geometry generative network based on the first geometry generative network parameters and the first geometry generative network hyperparameters; Based on the first geometric feature, the initial geometry generation network is trained to obtain a first geometry generation network.

18. The device according to any one of claims 16-17, characterized in that The data preprocessing module is used to: Acquire visual data of a first scene uploaded by a user, wherein the first scene includes a plurality of objects; Perform object segmentation on the visual data of the first scene to obtain visual data of each of the multiple objects.

19. The device according to claim 18, characterized in that The device also includes a positioning module, which is used to: Extracting spatial features of the multiple objects in the visual data of the first scene to obtain spatial features corresponding to the multiple objects, wherein the spatial features are used to characterize the positions and postures of the corresponding objects in the first scene; According to the spatial features respectively corresponding to the multiple objects, the grid models respectively corresponding to the multiple objects are placed to obtain the grid model of the first scene.

20. The device according to claim 19, characterized in that The device also includes a scene optimization module, which is used to: For each two objects among the plurality of objects, calculating, according to the spatial features respectively corresponding to the two objects, a confidence level that the positional relationship between the two objects is each of a plurality of preset positional relationships; Determine a first object category and a second object category to which the two objects respectively belong; In the prior knowledge base, obtaining the positional relationship between the objects of the first object category and the objects of the second object category in the first scene as a reference confidence of each preset positional relationship; Determining a positional relationship between the two objects based on the confidence level and the reference confidence level; According to the positional relationship between the two objects, positions of the grid models respectively corresponding to the two objects in the grid model of the first scene are adjusted.

21. The device according to any one of claims 16 to 20, characterized in that The AIGX module is also used to: Storing the first geometric feature of the object in the prior knowledge base; receiving a first 3D digital asset self-growth request sent by a user, wherein the first 3D digital asset self-growth request carries indication information of a first growth starting point object, indication information of a first growth end point object, and a target quantity, and the first growth starting point object and the first growth end point object are different objects under a third object category; In the prior knowledge base, the geometric features of the first growth starting point object, the geometric features of the first growth end point object, and the prior knowledge corresponding to the third object type are obtained; Taking the geometric features of the first growth starting point object as a starting point and the geometric features of the first growth ending point object as an ending point, generating the target number of intermediate geometric features; For each intermediate geometric feature of the target number of intermediate geometric features, a second geometry generation network is generated based on prior knowledge corresponding to the intermediate geometric feature and the third object type, and a second mesh model is generated based on the second geometry generation network.

22. The device according to any one of claims 16 to 20, characterized in that The AIGX module is also used to: Storing the first geometric feature of the object in the prior knowledge base; receiving a second 3D digital asset self-growth request sent by a user, wherein the second 3D digital asset self-growth request carries indication information of a second growth starting point object, indication information of a second growth end point object, and a target step length, and the second growth starting point object and the second growth end point object are different objects under a fourth object category; In the prior knowledge base, the geometric features of the second growth starting point object, the geometric features of the second growth end point object, and the prior knowledge corresponding to the fourth object type are obtained; Taking the geometric feature of the second growth starting point object as a starting point and the geometric feature of the second growth ending point object as an ending point, generating at least one intermediate geometric feature according to the target step length; For each of the at least one intermediate geometric feature, a third geometry generation network is generated based on prior knowledge corresponding to the intermediate geometric feature and the fourth object type, and a third mesh model is generated based on the third geometry generation network.

23. The device according to any one of claims 16 to 22, characterized in that The AIGX module is also used to: receiving a third 3D digital asset self-growth request sent by a user, wherein the third 3D digital asset self-growth request carries indication information of a second scene and indication information of a plurality of objects in the second scene; In the digital asset library, obtain mesh models of multiple objects in the second scene; In the prior knowledge base, obtaining a reference confidence level of each preset position relationship among a plurality of preset position relationships, that is, a position relationship between each two objects in the second scene; determining a positional relationship between each two objects in the second scene based on a reference confidence level that the positional relationship between each two objects in the second scene is each of a plurality of preset positional relationships; According to the positional relationship between every two objects in the second scene, grid models of multiple objects in the second scene are placed to obtain a grid model of the second scene.

24. A computing device cluster, characterized in that: The computing device cluster includes at least one computing device, each computing device including a processor and a memory; The processor of the at least one computing device is used to execute instructions stored in the memory of the device, so that the computing device cluster executes the method for generating a grid model as described in any one of claims 1 to 15.

25. A computer program product comprising instructions, characterized in that When the instructions are executed by a computing device cluster, the computing device cluster executes the method for generating a grid model as described in any one of claims 1 to 15.

26. A computer-readable storage medium, characterized in that: The method comprises computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster executes the method for generating a grid model according to any one of claims 1 to 15.

Images