Method, device and equipment for generating grid model

Abstract

The invention provides a grid model generation method, device and equipment, and relates to the technical field of artificial intelligence, and the method comprises the steps: obtaining visual data of an object uploaded by a user, the visual data comprises at least one of an image, a point cloud and a voxel model, carrying out the geometric feature extraction of the visual data of the object, and carrying out the geometric feature extraction of the visual data of the object; the method comprises the following steps: obtaining a first geometric feature of an object, obtaining prior knowledge corresponding to an object category to which the object belongs from a prior knowledge base, establishing a first geometric generation network based on the first geometric feature and the prior knowledge corresponding to the object category to which the object belongs, and obtaining a second geometric generation network based on the first geometric feature and the prior knowledge corresponding to the object category to which the object belongs, and generating a first grid model of the object based on the first geometric generation network. A grid model can be automatically established, and manual participation is not needed.

Description

[0001] This application claims priority to Chinese patent application No. 202311034137.4 filed on August 16, 2023, with invention name “Method, device, equipment and storage medium for converting three-dimensional digital assets”, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The present application relates to the field of artificial intelligence technology, and in particular to a method, device and equipment for generating a grid model. Background Art

[0003] Three-dimensional (3D) models are widely used in many fields such as industrial simulation and large model training. 3D models generally include mesh models, point cloud models, voxel models, etc. Among them, mesh models are the closest to actual objects in terms of geometric accuracy and rendering effect.

[0004] Currently, modelers usually use computer software to build mesh models of objects. However, manual modeling is inefficient and may take months to complete if the scene is complex and has many objects. Summary of the invention

[0005] The present application provides a method, device and apparatus for generating a grid model, which can generate a grid model more efficiently. The corresponding technical solution is as follows:

[0006] In a first aspect, a method for generating a grid model is provided, the method comprising:

[0007] Obtain visual data of an object uploaded by a user, wherein the visual data includes at least one of an image, a point cloud, and a voxel model. Then, extract geometric features from the visual data of the object to obtain a first geometric feature of the object. Obtain prior knowledge corresponding to the object category to which the object belongs from a prior knowledge base, wherein the prior knowledge is used to guide the establishment of a geometry generation network. Furthermore, establish a first geometry generation network based on the first geometric feature and the prior knowledge corresponding to the object category to which the object belongs. Finally, generate a first mesh model of the object based on the first geometry generation network.

[0008] In the technical solution provided in the present application, a geometric generation network is automatically constructed by combining prior knowledge and the geometric features of the object, so that the mesh model of the object can be automatically generated using the geometric generation network. The entire process does not require human participation, and the mesh model generation efficiency is higher.

[0009] In a possible implementation, the prior knowledge corresponding to the object category to which the object belongs includes a first geometric latent space network and a first geometric generative network hyperparameter. Accordingly, based on the first geometric feature and the prior knowledge corresponding to the object category to which the object belongs, establishing the first geometric generative network includes:

[0010] The first geometric feature is input into the first geometric latent space network to obtain the first geometric generative network parameters, and an initial geometric generative network is established based on the first geometric generative network parameters and the first geometric generative network hyperparameters. Then, the initial geometric generative network is supervised trained based on the first geometric feature to obtain the first geometric generative network.

[0011] In the technical solution provided in the present application, the geometric features of the object are used to perform supervised training on the geometry generation network, so that the trained geometry generation network can deterministically generate a mesh model of the object, and the generated mesh model has higher accuracy.

[0012] In a possible implementation, the geometric latent space network in the prior knowledge base can also be optimized. Accordingly, the processing can be as follows:

[0013] Based on the first geometric feature and the geometric generative network parameters of the first geometric generative network, the first geometric latent space network is trained to obtain a trained geometric latent space network. Then, the first geometric latent space network stored in the prior knowledge base is replaced with the trained geometric latent space network.

[0014] In the technical solution provided in the present application, by optimizing the geometric latent space network, the geometric generative network parameters output by the geometric latent space network can be made more usable, thereby reducing the number of training times of the geometric generative network and improving the generation efficiency of the mesh model.

[0015] In a possible implementation, a user can upload visual data of a scene. Accordingly, after obtaining the visual data of a first scene uploaded by the user, the visual data of the first scene can be segmented to obtain visual data of each of the multiple objects. Thus, the technical solution provided by the present application can be used to generate a mesh model of all or part of the multiple objects.

[0016] In a possible implementation, when the user uploads a visual of a scene, a mesh model of the scene may also be generated. Accordingly, the processing may be as follows:

[0017] Extract spatial features of multiple objects in the visual data of the first scene to obtain spatial features corresponding to the multiple objects, wherein the spatial features are used to characterize the positions of the corresponding objects in the first scene. Then, according to the spatial features corresponding to the multiple objects, mesh models corresponding to the multiple objects are placed to obtain a mesh model of the first scene.

[0018] In a possible implementation, considering that the positional relationship of the objects calculated according to the spatial features of the objects may be inaccurate, the placement position of the mesh model of the objects in the mesh model of the scene can be adjusted in combination with prior knowledge. Accordingly, the processing can be as follows:

[0019] For each two objects among the multiple objects, the confidence that the positional relationship between each two objects is each preset positional relationship among the multiple preset positional relationships is calculated based on the spatial features corresponding to each two objects. Determine the first object category and the second object category to which each two objects belong. In the prior knowledge base, obtain the reference confidence of each preset positional relationship between the object of the first object category and the object of the second object category in the first scene. Based on the confidence and the reference confidence, determine the positional relationship between each two objects. According to the positional relationship between the two objects, adjust the positions of the grid models corresponding to the two objects in the grid model of the first scene.

[0020] In the technical solution provided in the present application, the prior knowledge base stores the reference confidence of each preset position relationship between objects of each object category in different scenes. The reference confidence can be calculated based on the actual position relationship between objects of each object category in different scenes in reality, and can reflect the actual position relationship between objects of each object category to a certain extent. On this basis, the position of the grid model of each object in the grid model of the first scene is adjusted according to the reference confidence of each preset position relationship between the object of the first object category and the object of the second object category in the first scene, so that the position relationship between the grid models of each object can be more in line with reality.

[0021] In a possible implementation, a texture may be rendered on the mesh model, so that the mesh model becomes a textured mesh model, which is closer to the real object. Accordingly, the processing may be as follows:

[0022] Obtain a first texture map of the object. Render the first texture map onto the first mesh model to obtain a first textured mesh model of the object. Here, obtaining the first texture map of the object can be performed at any time before, after, or simultaneously with generating the first mesh model, and this application does not limit this.

[0023] In a possible implementation, when the visual data uploaded by the user is at least one of a non-color point cloud and a non-color voxel model, it is impossible to directly obtain texture features from the visual data. In this case, the texture can be specified by the user. Accordingly, the processing can be as follows:

[0024] A texture selection request sent by a user is received, wherein the texture selection request carries texture style indication information, color type indication information, and color brightness indication information. A first texture generation network corresponding to the texture type indication information, the color type indication information, and the color brightness indication information is obtained, and a first texture map of the object is generated based on the first texture generation network.

[0025] In a possible implementation, when the visual data uploaded by the user is at least one of a color point cloud, a color voxel model, and an image, texture features can be directly extracted from the visual image to generate a texture map. Accordingly, the processing can be as follows:

[0026] The texture feature of the object is extracted from the visual data to obtain the first texture feature of the object. A second texture generation network is established based on the first texture feature and the prior knowledge of the object. A first texture map of the object is generated based on the second texture generation network.

[0027] In a possible implementation, the prior knowledge further includes hyperparameters of the first texture latent space network and the first texture generation network. Accordingly, the process of establishing the texture generation network based on the prior knowledge may be as follows:

[0028] Input the first texture feature into the first texture latent space network to obtain first texture generation network parameters. Establish an initial texture generation network based on the first texture generation network parameters and the first texture generation network hyperparameters. Train the initial texture generation network based on the first texture feature to obtain a second texture generation network. Generate a first texture map of the object based on the second texture generation network.

[0029] In the technical solution provided in the present application, the texture features of the object are used to perform supervised training on the texture generation network, so that the trained texture generation network can deterministically generate the texture map of the object, and the generated texture map is closer to the texture of the actual object.

[0030] In a possible implementation, the texture latent space network in the prior knowledge base can also be optimized. Accordingly, the processing can be as follows:

[0031] Based on the first texture feature and the texture generation network parameters of the trained texture generation network, the first texture latent space network is trained to obtain the trained texture latent space network. The first texture latent space network stored in the prior knowledge base is replaced with the trained texture latent space network.

[0032] In the technical solution provided in the present application, by optimizing the texture latent space network, the texture generation network parameters output by the texture latent space network can be made more usable, thereby reducing the number of training times of the texture generation network and improving the generation efficiency of texture maps.

[0033] In a possible implementation, the user may specify to generate a 3D model of a certain type and format. Accordingly, the processing may be as follows:

[0034] A 3D model conversion request is received, wherein the 3D model conversion request carries indication information of a target model type and indication information of a target model format. A mesh model of the object is converted into a 3D model of the target model type and the target model format.

[0035] In a possible implementation, in order to facilitate archiving management, objects can also be labeled. Accordingly, the processing can be as follows:

[0036] Based on the object category labels stored in the prior knowledge base, the object category label corresponding to the object is determined.

[0037] In a possible implementation, the technical solution provided by the present application can also realize the self-growth of the grid model, and the processing can be as follows:

[0038] Receive a first 3D digital asset self-growth request sent by a user, wherein the first 3D digital asset self-growth request carries indication information of a first growth starting point object, indication information of a first growth end point object, and a target number, and the first growth starting point object and the first growth end point object are different objects under a third object category. In a priori knowledge base, obtain the geometric features of the first growth starting point object, the geometric features of the first growth end point object, and the prior knowledge corresponding to the third object type. With the geometric features of the first growth starting point object as the starting point and the geometric features of the first growth end point object as the end point, generate a target number of intermediate geometric features. For each of the target number of intermediate geometric features, generate a second geometric generation network based on the intermediate geometric features and the prior knowledge corresponding to the third object type, and generate a second mesh model based on the second geometric generation network.

[0039] In the technical solution provided by the present application, the user can specify the starting point object, the end point object and the target number of self-growing mesh models for the self-growing mesh model, and then, the above solution can be used to generate mesh models of the target number of objects whose characteristics are between the starting point object and the end point object. Through the self-growth of the mesh model, a large number of mesh models under a certain object category can be quickly generated, and then, these mesh models can be provided for various business uses such as large model training and simulation.

[0040] In a possible implementation, the self-grown mesh model may be stored in a digital asset library. Accordingly, the processing may be as follows:

[0041] The obtained target number of second mesh models and target number of second geometry generation networks are stored in the digital asset library on the cloud.

[0042] In the technical solution provided in the present application, only a target number of second geometry generation networks may be stored without storing the second mesh model. When necessary, the second mesh model may be temporarily generated through the second geometry generation network, thus saving storage space of the digital asset library.

[0043] In a possible implementation, when implementing the self-growth of the mesh model, the user may specify the target step size instead of the target number. Accordingly, the process of the self-growth of the mesh model may be as follows:

[0044] Receive a second 3D digital asset self-growth request sent by a user, wherein the second 3D digital asset self-growth request carries indication information of a second growth starting point object, indication information of a second growth end point object, and a target step length, and the second growth starting point object and the second growth end point object are different objects under a fourth object category. In a priori knowledge base, obtain geometric features of the second growth starting point object, geometric features of the second growth end point object, and prior knowledge corresponding to the fourth object type. Generate at least one intermediate geometric feature with the geometric features of the second growth starting point object as the starting point and the geometric features of the second growth end point object as the end point. For each of the at least one intermediate geometric features, generate a third geometric generation network based on the intermediate geometric features and the prior knowledge corresponding to the fourth object type, and generate a third mesh model based on the third geometric generation network.

[0045] In the technical solution provided by the present application, the user can specify the starting point object, the end point object and the target step length of the self-growth of the mesh model, and then, the above solution can be used to generate a mesh model of an object with at least one feature between the starting point object and the end point object. Through the self-growth of the mesh model, the mesh model of a certain object category can be quickly generated, and then, these mesh models can be provided for various business uses such as large model training and simulation.

[0046] In a possible implementation, the technical solution provided by the present application can also realize the self-growth of the scene-level grid model, and the processing can be as follows:

[0047] Receive a third 3D digital asset self-growth request sent by a user, wherein the third 3D digital asset self-growth request carries indication information of the second scene and indication information of multiple objects in the second scene. In the digital asset library, obtain the grid models of the multiple objects in the second scene. In the prior knowledge base, obtain the reference confidence that the positional relationship between every two objects in the second scene is each of a plurality of preset positional relationships. Based on the reference confidence that the positional relationship between every two objects in the second scene is each of a plurality of preset positional relationships, determine the positional relationship between every two objects in the second scene. According to the positional relationship between every two objects in the second scene, place the grid models of the multiple objects in the second scene to obtain the grid model of the second scene.

[0048] In the technical solution provided in the present application, the user can specify the required scene and the objects included in the scene, and then a grid model of the scene containing these objects can be generated with high generation efficiency and without the user providing visual data of the scene.

[0049] In a possible implementation, the self-grown mesh model may be stored in a digital asset library. Accordingly, the processing may be as follows:

[0050] The mesh model of the second scene is stored in the digital asset library.

[0051] In the technical solution provided in the present application, self-growth of the grid model can be achieved to obtain more grid models, thereby providing a large number of grid models required for various businesses such as large model training and simulation.

[0052] In a second aspect, a device for generating a grid model is provided, the device comprising:

[0053] A data preprocessing module, used to obtain visual data of an object uploaded by a user, wherein the visual data includes at least one of an image, a point cloud, and a voxel model;

[0054] A feature extraction module, used for extracting geometric features from the visual data of the object to obtain a first geometric feature of the object;

[0055] An artificial intelligence-based digital asset generation AIGX module is used to obtain prior knowledge corresponding to the object category to which the object belongs in a prior knowledge base, wherein the prior knowledge is used to guide the establishment of a geometry generation network; based on the first geometric feature and the prior knowledge corresponding to the object category to which the object belongs, a first geometry generation network is established; based on the first geometry generation network, a first mesh model of the object is generated.

[0056] In a possible implementation, the prior knowledge corresponding to the object category to which the object belongs includes hyperparameters of a first geometric latent space network and a first geometric generative network, and the AIGX module is used to:

[0057] Inputting the first geometric feature into the first geometric latent space network to obtain first geometric generative network parameters;

[0058] Establishing an initial geometry generative network based on the first geometry generative network parameters and the first geometry generative network hyperparameters;

[0059] Based on the first geometric feature, the initial geometry generation network is trained to obtain a first geometry generation network.

[0060] In a possible implementation, the AIGX module is used to:

[0061] Based on the first geometric features and the geometric generative network parameters of the first geometric generative network, the first geometric latent space network is trained to obtain a trained geometric latent space network;

[0062] The first geometric latent space network stored in the prior knowledge base is replaced by the trained geometric latent space network.

[0063] In a possible implementation, the data preprocessing module is used to:

[0064] Acquire visual data of a first scene uploaded by a user, wherein the first scene includes a plurality of objects;

[0065] Perform object segmentation on the visual data of the first scene to obtain visual data of each of the multiple objects.

[0066] In a possible implementation, the device further includes a positioning module, configured to:

[0067] Extracting spatial features of the multiple objects in the visual data of the first scene to obtain spatial features corresponding to the multiple objects, wherein the spatial features are used to characterize the positions and postures of the corresponding objects in the first scene;

[0068] According to the spatial features respectively corresponding to the multiple objects, the grid models respectively corresponding to the multiple objects are placed to obtain the grid model of the first scene.

[0069] In a possible implementation, the device further includes a scene optimization module, which is used to:

[0070] For each two objects among the plurality of objects, calculating, according to the spatial features respectively corresponding to the two objects, a confidence level that the positional relationship between the two objects is each of a plurality of preset positional relationships;

[0071] Determine a first object category and a second object category to which the two objects respectively belong;

[0072] In the prior knowledge base, obtaining the positional relationship between the object of the first object category and the object of the second object category in the first scene as the reference confidence of each preset positional relationship;

[0073] Determining a positional relationship between the two objects based on the confidence level and the reference confidence level;

[0074] According to the positional relationship between the two objects, positions of the grid models respectively corresponding to the two objects in the grid model of the first scene are adjusted.

[0075] In a possible implementation, the AIGX module is further used for:

[0076] Obtaining a first texture map of the object;

[0077] The first texture map is rendered onto the first mesh model to obtain a first textured mesh model of the object.

[0078] In a possible implementation, the visual data is at least one of a non-color point cloud and a non-color voxel model, and the AIGX module is used to:

[0079] Receiving a texture selection request sent by a user, wherein the texture selection request carries texture style indication information, color type indication information, and color brightness indication information;

[0080] Acquire a first texture generation network corresponding to the texture type indication information, the color type indication information, and the color brightness indication information;

[0081] Based on the first texture generation network, a first texture map of the object is generated.

[0082] In a possible implementation, the prior knowledge is also used to guide the establishment of a texture generation network, and the feature extraction module is also used to:

[0083] Extracting texture features from visual data of the object to obtain a first texture feature of the object;

[0084] Based on the first texture feature and the prior knowledge of the object, establishing a second texture generation network;

[0085] The AIGX module is used to:

[0086] Based on the second texture generation network, a first texture map of the object is generated.

[0087] In a possible implementation, the prior knowledge includes hyperparameters of a first texture latent space network and a first texture generation network, and the AIGX module is used to:

[0088] Inputting the first texture feature into the first texture latent space network to obtain first texture generation network parameters;

[0089] Establishing an initial texture generation network based on the first texture generation network parameters and the first texture generation network hyperparameters;

[0090] Based on the first texture feature, training the initial texture generation network to obtain a second texture generation network;

[0091] Based on the second texture generation network, a first texture map of the object is generated.

[0092] In a possible implementation, the AIGX module is further used for:

[0093] Based on the first texture feature and the texture generation network parameters of the trained texture generation network, the first texture latent space network is trained to obtain a trained texture latent space network;

[0094] The first texture latent space network stored in the prior knowledge base is replaced by the trained texture latent space network.

[0095] In a possible implementation, the device further includes a 3D digital asset conversion module, which is used to:

[0096] A 3D model conversion request is received, wherein the 3D model conversion request carries indication information of a target model type and indication information of a target model format. A mesh model of the object is converted into a 3D model of the target model type and the target model format.

[0097] In a possible implementation, the data preprocessing module is further used to:

[0098] Based on the object category labels stored in the prior knowledge base, an object category label corresponding to the object is determined.

[0099] In a possible implementation, the AIGX module is further used for:

[0100] Storing the first geometric feature of the object in the prior knowledge base;

[0101] receiving a first 3D digital asset self-growth request sent by a user, wherein the first 3D digital asset self-growth request carries indication information of a first growth starting point object, indication information of a first growth end point object, and a target quantity, and the first growth starting point object and the first growth end point object are different objects under a third object category;

[0102] In the prior knowledge base, the geometric features of the first growth starting point object, the geometric features of the first growth end point object, and the prior knowledge corresponding to the third object type are obtained;

[0103] Taking the geometric features of the first growth starting point object as a starting point and the geometric features of the first growth ending point object as an ending point, generating the target number of intermediate geometric features;

[0104] For each intermediate geometric feature of the target number of intermediate geometric features, a second geometry generation network is generated based on prior knowledge corresponding to the intermediate geometric feature and the third object type, and a second mesh model is generated based on the second geometry generation network.

[0105] In one possible implementation, the AIGX module is also used to:

[0106] The obtained target number of second mesh models and target number of second geometry generation networks are stored in the digital asset library on the cloud.

[0107] In a possible implementation, the AIGX module is further used for:

[0108] Storing the first geometric feature of the object in the prior knowledge base;

[0109] receiving a second 3D digital asset self-growth request sent by a user, wherein the second 3D digital asset self-growth request carries indication information of a second growth starting point object, indication information of a second growth end point object, and a target step length, and the second growth starting point object and the second growth end point object are different objects under a fourth object category;

[0110] In the prior knowledge base, the geometric features of the second growth starting point object, the geometric features of the second growth end point object, and the prior knowledge corresponding to the fourth object type are obtained;

[0111] Taking the geometric feature of the second growth starting point object as a starting point and the geometric feature of the second growth ending point object as an ending point, generating at least one intermediate geometric feature;

[0112] For each of the at least one intermediate geometric feature, a third geometry generation network is generated based on prior knowledge corresponding to the intermediate geometric feature and the fourth object type, and a third mesh model is generated based on the third geometry generation network.

[0113] In a possible implementation, the AIGX module is further used for:

[0114] receiving a third 3D digital asset self-growth request sent by a user, wherein the third 3D digital asset self-growth request carries indication information of a second scene and indication information of a plurality of objects in the second scene;

[0115] In the digital asset library, obtain mesh models of multiple objects in the second scene;

[0116] In the prior knowledge base, obtaining a reference confidence level of each preset position relationship among a plurality of preset position relationships, that is, a position relationship between each two objects in the second scene;

[0117] determining a positional relationship between each two objects in the second scene based on a reference confidence level that the positional relationship between each two objects in the second scene is each of a plurality of preset positional relationships;

[0118] According to the positional relationship between every two objects in the second scene, grid models of multiple objects in the second scene are placed to obtain a grid model of the second scene.

[0119] In one possible implementation, the AIGX module is also used to:

[0120] The mesh model of the second scene is stored in the digital asset library.

[0121] In a third aspect, a computing device cluster is provided, the computing device cluster comprising at least one computing device, each computing device comprising a processor and a memory;

[0122] The processor of the at least one computing device is used to execute instructions stored in the memory of the device, so that the computing device cluster executes the method for generating a grid model as described in the first aspect above.

[0123] In a fourth aspect, a computer program product comprising instructions is provided. When the instructions are executed by a computing device cluster, the computing device cluster executes the method for generating a grid model as described in the first aspect.

[0124] In a fifth aspect, a computer-readable storage medium is provided, comprising computer program instructions. When the computer program instructions are executed by a computing device, the computing device executes the method for generating a grid model as described in the first aspect above. BRIEF DESCRIPTION OF THE DRAWINGS

[0125] Figure 1 This is a schematic diagram of an implementation scenario provided by an embodiment of the present application;

[0126] Figure 2 It is a schematic diagram of a method flow of generating a grid model provided in an embodiment of the present application;

[0127] Figure 3 It is a schematic diagram of a method flow of generating a grid model provided in an embodiment of the present application;

[0128] Figure 4 It is a schematic diagram of a method flow of generating a grid model provided in an embodiment of the present application;

[0129] Figure 5 It is a schematic diagram of a method flow of generating a grid model provided in an embodiment of the present application;

[0130] Figure 6 It is a schematic diagram of a method flow of generating a grid model provided in an embodiment of the present application;

[0131] Figure 7 It is a schematic diagram of the structure of a device for generating a grid model provided in an embodiment of the present application;

[0132] Figure 8 is a schematic diagram of the structure of a computing device provided in an embodiment of the present application;

[0133] Fig. 9 is a schematic diagram of a computing device cluster provided in an embodiment of the present application;

[0134] Fig.10 It is a schematic diagram of a computing device cluster provided in an embodiment of the present application. DETAILED DESCRIPTION

[0135] The following is an explanation of some of the terms involved in the embodiments of the present application.

[0136] 1. Mesh model

[0137] A type of 3D digital asset that uses a series of polygons (usually triangles) of similar size and shape to approximate the model of a three-dimensional object.

[0138] 2. Voxel model

[0139] A type of 3D digital asset that uses an ordered combination of a large number of regular voxels (such as cubes) to represent the model of a three-dimensional object.

[0140] 3. Point cloud

[0141] A type of 3D digital asset is a data set of points in space that can represent a three-dimensional shape or object, usually acquired by a three-dimensional scanner, where the three-dimensional scanner includes but is not limited to a three-dimensional lidar.

[0142] 4. 3D digital asset type conversion

[0143] Mesh model can be converted into voxel model: Mesh model can be converted into voxel model through voxelization method. Common voxelization methods include multi-view orthographic projection view rendering, object slicing, octree space division, etc.

[0144] Mesh model conversion to point cloud: The mesh model can obtain the corresponding point cloud by sampling the vertices of the patch or taking more points on the patch.

[0145] Convert voxel model to mesh model: The voxel model can be converted into representations of vertices, edges, and faces through algorithms such as Marching Cube and Dual Contouring to obtain a mesh model.

[0146] Convert voxel model to point cloud: By finding the centroid of each point in each voxel of the voxel model, or by placing representative points on the voxel surface, a point cloud model can be sampled.

[0147] Convert point cloud to voxel model: According to the given voxel size, divide the space into regular cubes through octree algorithm or other algorithms. For each cube, if the cube does not contain the points in the point cloud, it will be discarded. If the cube contains the points in the point cloud, it will be retained. In this way, the point cloud can be voxelized to obtain a voxel model.

[0148] Convert point cloud into mesh model: Use the rolling ball method, Poisson reconstruction method, and Delaunay triangulation projection method to fit the surface of the point cloud and form multiple triangular meshes to generate a mesh model.

[0149] 5. Artificial Intelligence Generated X (AIGX)

[0150] Through AI technologies such as deep learning and reinforcement learning, text, pictures, audio, video, 3D models, etc. are generated. In the disclosed embodiment, AIGX is used to generate 3D models.

[0151] 6. Geometry Generation Network

[0152] A type of neural network. In this application, the geometry generation network is used to generate a mesh model. The neural network can also be called a generative network, a mesh model generation network, a 3D geometry generation network, a mesh model generation network, etc. This application does not limit the name of the neural network.

[0153] 7. Texture Generation Network

[0154] A type of neural network. In this application, the texture generation network is used to generate texture maps. The neural network can also be called a generative network, a 3D model texture generation network, a mesh model texture generation network, etc. This application does not limit the name of the neural network.

[0155] The embodiment of the present application provides a method for generating a grid model, which can be implemented by a computing device, a computing device cluster, etc. The computing device and the computing device cluster can be a cloud platform or a user's local device. The method can be applied to various scenarios where there is a need to use 3D models. For example, in a simulation test in the manufacturing industry, it is necessary to establish a 3D model of the factory area. The user can collect visual data of the factory area, such as images, point clouds, etc., and the computing device uses the method provided by this application to generate a 3D model of the factory area.

[0156] See also Figure 1 , shows a schematic diagram of an implementation scenario of an embodiment of the present application, such as Figure 1 As shown, the user can upload the visual data of the object for which the 3D model needs to be established to the computing device according to his own needs. The visual data can be one or more of an image, a point cloud, and a voxel model. The computing device performs data preprocessing on the visual data to determine the object category to which the object belongs. Then. The geometric features of the visual data are extracted to obtain the geometric features of the object. Furthermore, the prior knowledge corresponding to the object category to which the object belongs is obtained in the prior knowledge base, and the prior knowledge is used to guide the establishment of a geometric generation network. Then, according to the geometric features of the object and the prior knowledge corresponding to the object category to which the object belongs, a geometric generation network is established, and the mesh model of the object is generated using the geometric generation network. In addition, in order to make the mesh model of the object have texture, in an embodiment of the present application, the texture map of the object can also be obtained, and the texture map of the object can be rendered onto the mesh model of the object to obtain a textured mesh model of the object. On the basis of generating the mesh model of the object, the mesh model of the object can also be converted into a point cloud, a voxel model, etc. according to the user's requirements for the type of 3D model. The method provided in the embodiment of the present application can combine prior knowledge and the geometric features of the object to automatically construct a geometric generation network, so that the geometric generation network can be used to automatically generate a mesh model of the object. The entire process does not require human participation, and the mesh model generation efficiency is higher.

[0157] See also Figure 2 , shows a flow chart of a method for generating a grid model provided in an embodiment of the present application. The method can be implemented by a computing device. As shown in the figure, the process of the method may include the following steps:

[0158] Step 201: Obtain visual data of an object uploaded by a user.

[0159] The visual data includes at least one of images, point clouds, and voxel models, and the images may include pictures and videos.

[0160] In implementation, users can collect visual data of objects according to actual needs and upload the collected visual data of objects to computing devices. For example, users can take images of objects through mobile phones, tablet computers, cameras, etc. as visual data of objects, or collect point clouds of objects through data acquisition equipment as visual data of objects, or create voxel models of objects themselves as visual data of objects, or obtain at least one of images, point clouds, and voxel models of objects through the Internet as visual data of objects.

[0161] The visual data of the object mentioned here can be the visual data of a single object or the visual data of a scene including multiple objects. For example, if a user wants a 3D model of a car, he can obtain one or more of the pictures, videos, point clouds, and voxel models of the car and upload them to a computing device. For another example, if a user wants a 3D model of a car manufacturing plant, which includes multiple objects, such as various machines and cars on a production line, in this case, the user can obtain one or more of the pictures, videos, point clouds, and voxel models of the plant and upload them to a computing device.

[0162] In a possible implementation, after receiving the visual data of the object uploaded by the user, the visual data may be first filtered to remove redundant data, sensitive data, abnormal data, etc.

[0163] Among them, redundant data can be repeated data. For example, if the visual data uploaded by the user includes two identical pictures, it is redundant data. Then, through data filtering, one of the pictures is removed as redundant data. Sensitive data can be data containing sensitive information. For example, if the visual data uploaded by the user includes pictures containing clear faces, clear faces involve personal privacy and are sensitive data. Then, through data filtering, pictures containing clear faces are removed. Abnormal data can be data of poor quality. For example, if the visual data uploaded by the user includes overexposed or dark pictures, it is abnormal data. Then, through data filtering, overexposed or dark pictures are removed.

[0164] In a possible implementation, for the visual data of the object uploaded by the user, the computing device may obtain the object instance label and the object category label of the object. For an object that is a car of brand A, the object instance label of the object may be used to describe that the object is a car, or that the object is a car of brand A, and the object category label of the object is used to describe that the object is a car.

[0165] In the case where the visual data of the object uploaded by the user is the visual data of a single object, if the user also uploads the object instance label when uploading the visual data of the object, the computing device does not need to label the visual data of the object, and can directly use the object instance label uploaded by the user. If the user does not upload the object instance label when uploading the visual data of the object, the computing device can label the visual data of the object to obtain the object instance label. After obtaining the object instance label, the object category label can be further obtained.

[0166] In the case where the visual data of the object uploaded by the user is the visual data of a scene including multiple objects, if the user also uploads the object instance labels of each object in the scene when uploading the visual data of the scene, the computing device does not need to segment and annotate the visual data of the scene, and can directly use the object instance labels uploaded by the user. If the user does not upload the object instance labels when uploading the visual data of the scene, the visual data of the scene can be segmented and annotated to obtain the object instance labels of each object in the scene. Among them, segmentation and annotation can be implemented in a variety of ways. For example, the visual data of the scene is input into a segmentation and annotation model, and the segmentation and annotation model performs segmentation and annotation processing on the visual data, and outputs the object instance labels of each object in the scene. The segmentation and annotation model mentioned here can be a neural network model. After obtaining the object instance labels of each object in the scene, the object category labels of each object can be further obtained.

[0167] There are many methods for obtaining object category labels of objects, and several of them are listed below for illustration.

[0168] Method 1: A priori knowledge base may be deployed on a computing device, and a plurality of object category labels are stored in the priori knowledge base. Through natural language processing, the object instance label of the object is converted into a numerical vector, and then the obtained numerical vector is input into a classifier, and the classifier outputs the probability that the object belongs to the object category indicated by each object category label in the priori knowledge base, and the object category label corresponding to the maximum probability is determined as the object category label corresponding to the object.

[0169] Method 2: A prior knowledge base may be deployed on a computing device, and a plurality of object category labels are stored in the prior knowledge base. The object instance label of the object and the object category labels stored in the prior knowledge base are input into the language model, and the language model outputs the object category label corresponding to the object category to which the object belongs, that is, the object category label corresponding to the object. The object category label output by the language model is one of the object category labels stored in the prior knowledge base.

[0170] When the object category label of the object is obtained, the computing device may store the object category label of the object and the visual data of the object in a digital asset library in correspondence.

[0171] Step 202: extract geometric features from the visual data of the object to obtain the first geometric features of the object.

[0172] Among them, geometric features can be expressed in the form of vectors, and geometric features are used to characterize the contour features, area, perimeter, direction, etc. of an object.

[0173] In implementation, when the user uploads visual data of a single object, the computing device extracts geometric features from the visual data of the object to obtain the first geometric features of the object. When the user uploads visual data of a scene, the computing device may extract geometric features from the visual data of each object in the scene obtained by segmenting the visual data of the scene to obtain the first geometric features of each object.

[0174] There are many methods for extracting geometric features, and several of them are listed below for illustration.

[0175] The contour features of the object are extracted through the boundary feature method or the Fourier shape description method, and the area, perimeter, direction and other features of the object are extracted through the geometric parameter method or the shape invariant moment method.

[0176] Step 203: Acquire prior knowledge corresponding to the object category to which the object belongs in the prior knowledge base.

[0177] The prior knowledge is used to guide the establishment of a geometric generative network, and the prior knowledge includes a first geometric latent space network and a first geometric generative network hyperparameter corresponding to the object category to which the object belongs.

[0178] In implementation, a priori knowledge base can be deployed on a computing device, in which a geometric latent space network and geometric generative network hyperparameters corresponding to object categories are stored. The geometric latent space network is used to generate geometric generative network parameters. The geometric generative network parameters include weights and biases in the geometric generative network. The geometric generative network hyperparameters include the learning rate, number of iterations, optimization algorithm, number of perception layers, convolution kernel size, etc. of the geometric generative network. The geometric generative network hyperparameters will not change during the training process of the geometric generative network.

[0179] When the user uploads visual data of a single object, the computing device obtains, from the prior knowledge base, first geometric latent space network and first geometric generative network hyperparameters corresponding to the object category to which the object belongs.

[0180] When the user uploads visual data of a scene, for each object included in the scene, the computing device obtains, from the prior knowledge base, first geometric latent space network and first geometric generative network hyperparameters corresponding to the object category to which the object belongs.

[0181] Step 204: Establish a first geometric generation network based on the first geometric feature and the prior knowledge corresponding to the object category to which the object belongs.

[0182] In an implementation, when the user uploads visual data of a single object, the computing device establishes a first geometric generation network for generating a mesh model of the object.

[0183] In the case that the user uploads visual data of a scene, for each object included in the scene, the computing device establishes a first geometric generation network for generating a mesh model of the object.

[0184] The establishment of the first geometric generation network is described below:

[0185] The user inputs the first geometric feature of the object into the first geometric latent space network, and the first geometric latent space network outputs the first geometric generative network parameters. Based on the first geometric generative network parameters and the first geometric generative network hyperparameters, an initial geometric generative network is established, and the hyperparameters of the initial geometric generative network are the first geometric generative network hyperparameters, and the parameters of the initial geometric generative network are the first geometric generative network parameters. Based on the first geometric feature, the initial geometric generative network is trained to obtain the first geometric generative network. During the training process, the hyperparameters of the initial geometric network remain unchanged, and the parameters of the initial geometric generative network change.

[0186] Step 205: Generate a first mesh model of the object based on the first geometry generation network.

[0187] In implementation, when the user uploads visual data of a single object, the computing device generates a first mesh model of the object based on a first geometric generation network corresponding to the object.

[0188] In the case where the user uploads visual data of a scene, for each object included in the scene, the computing device generates a first mesh model of the object based on a first geometric generation network corresponding to the object.

[0189] The following describes the process of training the initial geometry generation network to obtain the first geometry generation network and generating the first mesh model based on the first geometry generation network. Figure 3 , the process may include the following steps:

[0190] Step 2051: input the random signal into the initial geometry generation network, and the initial geometry generation network outputs a second mesh model.

[0191] Step 2052: extract the second geometric feature of the second mesh model, and calculate the first geometric loss of the first geometric feature and the second geometric feature.

[0192] Step 2053: If the first geometric loss is less than or equal to the first threshold, the second mesh model is used as the first mesh model of the object, and the initial geometric generation network is used as the first geometric generation network, and the following steps 2054 to 2059 are no longer performed. The first threshold can be configured according to the user's accuracy requirements for the mesh model, for example, it can be set between 0 and 0.1, such as 0.01.

[0193] Step 2054: If the first geometric loss is greater than the first threshold, supervised training is performed on the initial geometric generation network based on the first geometric loss value to obtain a trained geometric generation network.

[0194] Step 2055: input the random signal into the trained geometry generation network, and the trained geometry generation network outputs a third mesh model.

[0195] Step 2056: extract the third geometric feature of the third mesh model, and calculate the second geometric loss of the first geometric feature and the third geometric feature.

[0196] Step 2057: Determine whether the second geometric loss is greater than the first threshold.

[0197] Step 2058: If the second geometric loss is less than or equal to the first threshold, the third mesh model is used as the first mesh model of the object, the most recently trained geometry generation network is used as the above-mentioned first geometry generation network, and the loop is stopped.

[0198] Step 2059: If the second geometric loss is greater than the first threshold, the most recently trained geometric generative network is supervisedly trained again based on the second geometric loss value to obtain a trained geometric generative network, and then the process goes to step 2055.

[0199] In the present application, the first geometry generation network can deterministically generate a mesh model of an object, and the generated mesh model is more accurate.

[0200] In one possible implementation, the geometric latent space network in the prior knowledge base can be optimized. The optimization process can be as follows:

[0201] When the first mesh model of the object is generated using the trained first geometry generative network, the second geometry generative network parameters of the first geometry generative network can be obtained, the first geometric feature is used as an input sample, the second geometry generative network parameters are used as output samples, and the first geometry latent space network is supervised and trained to obtain the trained geometry latent space network. Furthermore, the first geometry latent space network stored in the prior knowledge base can be updated to the trained geometry latent space network.

[0202] In a possible implementation, the first geometric generative network of the object may be stored in the digital asset library. During storage, the object instance label, the object category label, and the first geometric generative network of the object may be stored correspondingly.

[0203] In a possible implementation, the first geometric feature of the object may be stored in a priori knowledge base. When storing, the first geometric feature of the object and the object category label and the object instance label of the object may be stored correspondingly.

[0204] In a possible implementation, the first mesh model of the object generated in step 205 is a geometric white model without texture. Figure 4 , the computing device may also perform the following steps.

[0205] Step 206: Obtain a first texture map of the object.

[0206] In implementation, there may be two situations in which the computing device obtains the first texture map of the object, and these two situations are respectively described below.

[0207] Case 1: The visual data of the object uploaded by the user is one or more of a non-color point cloud and a non-color voxel model.

[0208] In this case, the computing device cannot directly extract the first texture map of the object from the visual data of the object, and the user can specify the texture style, color type, color brightness, etc. of the texture to be used. The user can select the texture style, color type, color brightness, etc. of the texture to be used in the texture selection interface of the user device, and send a texture selection request to the computing device through the user device, wherein the texture selection request can carry texture style indication information, color type indication information, and color brightness indication information. Correspondingly, the computing device can obtain a texture map corresponding to the texture style indication information, color type indication information, and color brightness indication information in the texture map library as the first texture map of the object. Alternatively, the computing device can also obtain a texture generation network corresponding to the texture style indication information, color type indication information, and color brightness indication information in the texture generation network library, input a random signal into the texture generation network, and the texture generation network outputs the first texture map of the object.

[0209] Case 2: The visual data of the object uploaded by the user is one or more of an image, a color point cloud, and a color voxel model.

[0210] In this case, when the computing device acquires the prior knowledge corresponding to the object category to which the object belongs, in addition to acquiring the first geometric latent space network and the first geometric generative network hyperparameters, it can also acquire the first texture latent space network and the first texture generative network hyperparameters, wherein the first texture latent space network and the first texture generative network hyperparameters also belong to the prior knowledge corresponding to the object category to which the object belongs. Then, the first texture map of the object can be generated by the following steps:

[0211] Step 2061: The computing device extracts texture features from the visual data of the object to obtain a first texture feature of the object.

[0212] Among them, the texture features of the object can be expressed in vector form. The texture features are used to characterize the RGB value of each pixel on the image of the object, the contrast of the image, the grayscale distribution of the image, the resolution of the image, the brightness of the pixel, etc.

[0213] In implementation, if the visual data of the object is an image, a color point cloud, or a color voxel model, the computing device can directly perform texture feature extraction on the visual data of the object to obtain a first texture feature of the object.

[0214] Step 2062: The computing device establishes a first texture generation network based on the first texture latent space network and the first texture generation network hyperparameters corresponding to the first texture feature and the object category to which the object belongs.

[0215] In the implementation, the user inputs the first texture feature of the object into the first texture latent space network, and the first texture latent space network outputs the first texture generation network parameters. Based on the first texture generation network parameters and the first texture generation network hyperparameters, an initial texture generation network is established, and the hyperparameters of the initial texture generation network are the first texture generation network hyperparameters, and the parameters of the initial texture generation network are the first texture generation network parameters. Based on the first texture feature, the initial texture generation network is trained to obtain the first texture generation network. During the training process, the hyperparameters of the initial texture network remain unchanged, and the parameters of the initial texture generation network change.

[0216] Step 2063: Generate a first texture map of the object based on the first texture generation network.

[0217] The following describes a process of training an initial texture generation network to obtain a first texture generation network, and generating a first texture map based on the first texture generation network. The process may include the following steps:

[0218] S1. Input a random signal into an initial texture generation network, and the initial texture generation network outputs a second texture map.

[0219] S2. Extract a second texture feature of the second texture map, and calculate a first texture loss of the first texture feature and the second texture feature.

[0220] S3. If the first texture loss is less than or equal to the second threshold, the second texture map is used as the first texture map of the object, and the initial texture generation network is used as the first texture generation network, and the following steps S4 to S8 are no longer performed. The second threshold can be configured according to the user's accuracy requirements for the mesh model, for example, it can be set between 0 and 0.1, such as 0.01.

[0221] S4. If the first texture loss is greater than the second threshold, supervised training is performed on the first texture generation network based on the first texture loss value to obtain a trained texture generation network.

[0222] S5. Input the random signal into the trained texture generation network, and the trained texture generation network outputs a third texture map.

[0223] S6. Extract a third texture feature of the third texture map, and calculate a second texture loss of the first texture feature and the third texture feature.

[0224] S7. If the second texture loss is less than or equal to the second threshold, the third texture map is used as the first texture map of the object, the texture generation network after the latest training is used as the above-mentioned first texture generation network, and the loop is stopped.

[0225] S8. If the second texture loss is greater than the second threshold, the texture generation network after the latest training is supervised and trained again based on the second texture loss value to obtain the trained texture generation network, and then go to step S5.

[0226] Step 207: Render the first texture map onto the first mesh model to obtain a first textured mesh model of the object.

[0227] In implementation, the computing device may render the first texture map onto the first mesh model through a renderer to obtain a first textured mesh model of the object.

[0228] In one possible implementation, the texture latent space network in the prior knowledge base can be optimized, and the optimization process can be as follows:

[0229] When the trained texture generation network is used to generate the first texture model of the object, the second texture generation network parameters of the trained texture generation network can be obtained, the first texture feature is used as an input sample, the second texture generation network parameters are used as output samples, and the first texture latent space network is supervised and trained to obtain the trained texture latent space network. Furthermore, the first texture latent space network stored in the prior knowledge base can be updated to the trained texture latent space network.

[0230] In a possible implementation, the first texture generation network of the object may be stored in a digital asset library. During storage, the object instance label, the object category label, the first geometry generation network, and the first texture generation network of the object may be stored correspondingly.

[0231] In a possible implementation, the first texture feature of the object may be stored in a priori knowledge base. When storing, the first texture feature of the object and the object category label, object instance label, and first geometric feature of the object may be stored correspondingly.

[0232] In a possible implementation, when the user uploads visual data of a scene, a mesh model of the scene may be further generated based on the mesh models of the objects in the scene. Accordingly, the processing may be as follows:

[0233] The computing device extracts spatial features from each object in the visual data of the scene uploaded by the user to obtain spatial features corresponding to each object, wherein the spatial features are used to characterize the position and posture of the corresponding object in the scene.

[0234] For every two objects in the scene, the confidence level of each preset positional relationship among multiple preset positional relationships is calculated based on the spatial features corresponding to the two objects. The preset positional relationships include intersection, contact, inclusion, inclusion, and separation. Intersection means that there is an overlap between the two objects, contact means that the surfaces of the two objects are in contact, inclusion means that the other object is within the object, inclusion means that the object is within the other object, and separation means that the two objects are not in contact and are not in a containment or inclusion relationship.

[0235] Then, a preset position relationship corresponding to the highest confidence level is determined as the first position relationship between the two objects.

[0236] Finally, a cubic space is constructed, and the mesh model of each object in the scene is placed in the cubic space according to the first position relationship between every two objects.

[0237] In addition, since the calculated positional relationship between objects may not be accurate, the position of the grid model of each object can be adjusted based on prior knowledge. The processing can be as follows:

[0238] For every two objects in the scene, determine the first object category and the second object category to which the two objects belong respectively, and obtain the positional relationship between the objects of the first object category and the objects of the second object category in the scene as the reference confidence of each preset positional relationship in the prior knowledge base. For each preset positional relationship, multiply the confidence of the preset positional relationship by the reference confidence to obtain the comprehensive confidence of the preset positional relationship, and take the preset positional relationship corresponding to the highest comprehensive confidence as the second positional relationship between the two objects. According to the second positional relationship between every two objects in the scene, the positions of the grid models corresponding to the two objects in the grid model of the first scene are adjusted. For example, the scene includes object A and object B, and the first positional relationship between object A and object B is intersection, and the second positional relationship is contact, then the positions of object A and object B are adjusted from intersection to contact.

[0239] In addition, if a textured mesh model of an object is generated, a textured mesh model of a scene may be generated. The specific generation method is the same as the method for generating the mesh model of the scene, and will not be described in detail here.

[0240] In a possible implementation, the user may specify the model type and model format of the generated 3D model, and accordingly, the computing device may convert the generated mesh model into a 3D model of the model type and model format specified by the user. Specifically, the processing may be as follows:

[0241] The user sends a 3D model conversion request to the computing device through the user device, wherein the 3D model conversion request carries indication information of the target model type, and may also carry indication information of the target model format and / or indication information of the target resolution. After receiving the 3D model conversion request, the computing device converts the mesh model of the object into a 3D model of the target model type, target model format, and target resolution.

[0242] There are several ways to convert 3D models:

[0243] 1. The user needs a mesh model in fbx format. The mesh model generated and stored by the computing device can be in obj format. On this basis, the computing device can read the mesh model in obj format, store it in fbx format, and obtain the mesh model in fbx format.

[0244] 2. The user needs a point cloud model in pcd format with the target resolution. The computing device reads the mesh model in obj format, collects points according to the target resolution, and stores them in pcd format. The point cloud model in pcd format with the target resolution is obtained.

[0245] 3. The user needs a voxel model in binvox format with a target resolution. The computing device reads the grid model in obj format, sets the number of voxel cells according to the target resolution, divides the space through the octree, stores it in binvox format, and obtains a voxel model in binvox format with a target resolution.

[0246] In a possible implementation, in order to obtain mesh models of more objects to meet the needs of large model training, the embodiment of the present application also provides a method for self-growth of 3D digital assets, through which a computing device can automatically generate a 3D model. Figure 5 As shown, the process may include the following steps:

[0247] Step 301: Receive a first 3D digital asset self-growth request sent by a user.

[0248] In implementation, when there is no visual data of an object, or when it is necessary to quickly obtain textured mesh models of a large number of objects, the user can specify the first object under the third object category as the growth starting point object and the second object as the growth end point object on the user device, so that the computing device can generate textured mesh models of some objects whose geometric features and texture features are between the first object and the second object. For example, if the third object category is a car, the first object is a sports car, and the second object is a sport utility vehicle (SUV), the computing device can generate textured mesh models of some cars whose appearance is between a sports car and an SUV.

[0249] In addition to specifying the third object category, the growth starting point object, and the growth starting point object, the user can also specify the target number of the generated textured mesh model on the user device. Accordingly, the user device can send a first 3D digital asset self-growth request to the computing device, wherein the first 3D digital asset self-growth request carries the indication information of the first object, the indication information of the second object, and the target number, and the first object and the second object are different objects under the third object category.

[0250] Step 302: Obtain, from the prior knowledge base, the geometric features of the first object, the texture features of the first object, the geometric features of the second object, the texture features of the second object, the second geometric latent space network corresponding to the third object type, the second geometric generative network hyperparameters corresponding to the third object type, the second texture latent space network corresponding to the third object type, and the second texture generative network parameters corresponding to the third object type.

[0251] Step 303: Taking the geometric features of the first object as the starting point and the geometric features of the second object as the end point, a target number of intermediate geometric features are generated; taking the texture features of the first object as the starting point and the texture features of the second object as the end point, a target number of intermediate texture features are generated, wherein the target number of intermediate texture features and the target number of intermediate geometric features correspond one to one.

[0252] The following describes how to generate a target number of intermediate geometric features by taking the geometric features of the first object as the starting point and the geometric features of the second object as the end point:

[0253] The geometric features of the first object are vectors, including n elements. The geometric features of the second object are also vectors, also including n elements. The first element in the geometric features of the first object is recorded as A1, and the first element in the geometric features of the second object is recorded as B1. The difference between B1 and A1 is calculated, and the difference is recorded as C1. C1 is divided by (x+1) to obtain the target step length, recorded as D1, where x is the target number. A1+D1 is used as the first element of the first intermediate geometric feature, and A1+2*D1 is used as the first element of the second intermediate geometric feature. And so on, the first element of each of the x intermediate geometric features can be obtained. In the same way, the remaining elements of each of the x intermediate geometric features can be obtained. In this way, x intermediate geometric features can be obtained.

[0254] Taking the texture feature of the first object as the starting point and the texture feature of the second object as the end point, the method for generating the target number of intermediate texture features is the same as the method for generating the intermediate geometric features described above, and will not be repeated here. The first intermediate geometric feature generated corresponds to the first texture feature generated, the second intermediate geometric feature generated corresponds to the second texture feature generated, and so on, the target number of intermediate texture features corresponds to the target number of intermediate geometric features.

[0255] In one possible implementation, the user may also specify a target step size, and accordingly, the first 3D digital asset self-growth request may also carry the target step size. In this case, the computing device may use the target step size carried in the first 3D digital asset self-growth request to calculate intermediate geometric features and intermediate texture features.

[0256] Step 304: for the corresponding intermediate geometric features and intermediate texture features, the intermediate geometric features are input into the second geometric latent space network to obtain second geometric generative network parameters, a second geometric generative network is established based on the second geometric generative network parameters and the second geometric generative network hyperparameters, a second mesh model is generated based on the second geometric generative network, the second texture features are input into the second texture latent space network to obtain second texture generative network parameters, a third texture generative network is established based on the second texture generative network parameters and the second texture generative network hyperparameters, a second texture map is generated based on the third texture generative network, the second texture map is rendered onto the second mesh model to obtain a second textured mesh model.

[0257] Step 305: The generated target number of second textured mesh models may be stored in the digital asset library. When storing, they may be stored corresponding to the third object category label.

[0258] In a possible implementation, when the 3D digital asset is self-growing, it is also possible not to obtain texture features and texture maps, and ultimately only generate a mesh model of at least one object whose geometric features are between the first object and the second object.

[0259] In a possible implementation, when the 3D digital asset self-grows, the user may not specify the target number but only the target step size. Accordingly, the above step 303 may be replaced by: taking the geometric features of the first object as the starting point and the geometric features of the second object as the end point, generating at least one intermediate geometric feature according to the target step size; taking the texture features of the first object as the starting point and the texture features of the second object as the end point, generating at least one intermediate texture feature, wherein the number of the generated intermediate geometric features is the same as the number of the generated intermediate texture features.

[0260] The following describes how to generate at least one intermediate geometric feature based on the target step length, taking the geometric feature of the first object as the starting point and the geometric feature of the second object as the end point:

[0261] The geometric features of the first object are vectors, including n elements. The geometric features of the second object are also vectors, also including n elements. The first element of the geometric features of the first object is recorded as A1, the first element of the geometric features of the second object is recorded as B1, and the target step length specified by the user is recorded as D2. A1+D2 is used as the first element of the first intermediate geometric feature, A1+2*D2 is used as the first element of the second intermediate geometric feature, and so on, until B1 is between A1+y*D2 and A1+(y+1)*D2, the first element of each of the y intermediate geometric features can be obtained. According to the same method, the remaining elements of each of the y intermediate geometric features can be obtained, so that y intermediate geometric features can be obtained.

[0262] Taking the texture feature of the first object as the starting point and the texture feature of the second object as the end point, the method for generating at least one intermediate texture feature according to the target step length is the same as the method for generating the intermediate geometric feature described above, and will not be repeated here. Assuming that z intermediate texture features are generated, if y is greater than z, the first z intermediate geometric features generated are obtained, and these z intermediate geometric features correspond to the z intermediate texture features one by one. If y is less than z, the first y intermediate texture features generated are obtained, and these y intermediate texture features correspond to the y intermediate texture features one by one.

[0263] Accordingly, in step 304, n second textured mesh models may be generated, and when z and y are the same, the value of n is z or y, and when z and y are different, the value of n is the smaller value of y and z. In step 305, the generated n second textured mesh models may be stored in the digital asset library.

[0264] In a possible implementation, the embodiment of the present application also provides a method for automatically generating a grid model of a scene. Accordingly, Figure 6 As shown, the process may include the following steps:

[0265] Step 401: Receive a second 3D digital asset self-growth request sent by a user.

[0266] In implementation, if the user wants to obtain the mesh model of the target scene, the user can specify the target scene and the objects included in the target scene through the user device, and accordingly, the user device can send a second 3D digital asset self-growth request to the computing device. The second 3D digital asset self-growth request can carry the indication information of the target scene, the indication information of the third object, and the indication information of the fourth object.

[0267] Step 402: Obtain a textured mesh model of a third object and a textured mesh model of a fourth object in a digital asset library.

[0268] Step 403: In the prior knowledge base, the positional relationship between the third object and the fourth object in the target scene is obtained as a reference confidence of each preset positional relationship, and the preset positional relationship corresponding to the maximum parameter confidence is used as the positional relationship between the third object and the fourth object. A cubic space is constructed, and according to the positional relationship between the third object and the fourth object, the textured mesh model of the third object and the textured mesh model of the fourth object are placed in the cubic space to obtain a mesh model of the second scene.

[0269] Step 404: Store the generated mesh model of the target scene in the digital asset library.

[0270] In implementation, the computing device may store the generated mesh model of the target scene and the scene label of the target scene in correspondence in the digital asset library.

[0271] In addition, here the example of using textured mesh models of two objects to generate a mesh model of a scene is used for explanation. During implementation, more textured mesh models of objects may be used to generate a mesh model of the scene. When using textured mesh models of three or more objects to generate a mesh model of the scene, it is necessary to determine the positional relationship between every two objects, and then place the textured mesh models of each object to generate a textured mesh model of the scene.

[0272] In a possible implementation, when the 3D digital asset is self-growing, a non-textured mesh model of the object may also be obtained to generate a non-textured mesh model of the scene.

[0273] In a possible implementation, in order to save storage space, in the digital asset library, it is not necessary to store the textured mesh model itself, but rather to store the geometry generation network that generates the mesh model and the texture generation network that generates the texture map. In this way, when it is necessary to obtain a textured mesh model of an object, the geometry generation network and the texture generation network of the object can be obtained first, and the mesh model of the object can be generated using the obtained geometry generation network, and the texture map of the object can be generated using the texture generation network, and then the texture map of the object can be rendered onto the mesh model of the object to obtain the textured mesh model of the object.

[0274] In a possible implementation, the embodiment of the present application further provides a method for establishing a priori knowledge base. Accordingly, the processing may be as follows:

[0275] Initially, the prior knowledge base does not have any prior information. You can search for public mesh models from open source datasets or on the Internet. There are many models obtained by artificial modeling on the Internet, which can cover most of the object categories in reality. The structure of the geometric generative network corresponding to objects of different object categories is different, that is, the hyperparameters of the geometric generative network are different. For example, compared with an apple, a car is a more complex object, and the corresponding geometric generative network should have more network layers. So for an unknown object that has never been seen, it is necessary to try multiple possible hyperparameter combinations. The complexity of the object can be judged by a large model, several groups of possible geometric generative network hyperparameters are provided, and neural networks with multiple different structures are generated. A group of geometric generative network hyperparameters with the highest generation efficiency and the best generation result quality are selected. The network structure defined by the optimal geometric generative network hyperparameters is trained to obtain a mesh model similar to the input object and the final fine-tuned geometric generative network. The geometric features of the extracted mesh model are used as input, and the parameters of the fine-tuned geometric generative network are used as output to train the geometric latent space network corresponding to the object category. The establishment of relevant prior knowledge of texture is similar and will not be repeated here.

[0276] In one possible implementation, Figure 2 The steps shown can all be implemented by a cloud computing device, or can all be implemented by a user's local computing device.

[0277] In a possible implementation, in order to enhance user privacy protection, the visual data collected by the user does not need to be uploaded to the cloud. In this case, Figure 2 The steps shown can all be executed by the user's local computing device, and the prior knowledge base can be deployed in the cloud. When it comes to the processing of obtaining various types of prior knowledge in the prior library, the user's local computing device and the cloud can interact to obtain the corresponding prior knowledge.

[0278] The present application also provides a device for generating a grid model, which is applied to a computing device, see Figure 7 The device includes: a data preprocessing module 410, a feature extraction module 420 and an AIGX module 430, wherein:

[0279] A data preprocessing module 410 is used to obtain visual data of an object uploaded by a user, wherein the visual data includes at least one of an image, a point cloud, and a voxel model;

[0280] A feature extraction module 420 is used to extract geometric features from the visual data of the object to obtain a first geometric feature of the object;

[0281] AIGX module 430 is used to obtain prior knowledge corresponding to the object category to which the object belongs in a prior knowledge base, wherein the prior knowledge is used to guide the establishment of a geometry generation network; establish a first geometry generation network based on the first geometric feature and the prior knowledge corresponding to the object category to which the object belongs; and generate a first mesh model of the object based on the first geometry generation network.

[0282] In a possible implementation, the prior knowledge corresponding to the object category to which the object belongs includes a first geometric latent space network and a first geometric generative network hyperparameter, and the AIGX module 430 is used to:

[0283] Inputting the first geometric feature into the first geometric latent space network to obtain first geometric generative network parameters;

[0284] Establishing an initial geometry generative network based on the first geometry generative network parameters and the first geometry generative network hyperparameters;

[0285] Based on the first geometric feature, the initial geometry generation network is trained to obtain a first geometry generation network.

[0286] In a possible implementation, the AIGX module 430 is used to:

[0287] Based on the first geometric features and the geometric generative network parameters of the first geometric generative network, the first geometric latent space network is trained to obtain a trained geometric latent space network;

[0288] The first geometric latent space network stored in the prior knowledge base is replaced by the trained geometric latent space network.

[0289] In a possible implementation, the data preprocessing module 410 is used to:

[0290] Acquire visual data of a first scene uploaded by a user, wherein the first scene includes a plurality of objects;

[0291] Perform object segmentation on the visual data of the first scene to obtain visual data of each of the multiple objects.

[0292] In a possible implementation, the device further includes a positioning module, configured to:

[0293] Extracting spatial features of the multiple objects in the visual data of the first scene to obtain spatial features corresponding to the multiple objects, wherein the spatial features are used to characterize the positions and postures of the corresponding objects in the first scene;

[0294] According to the spatial features respectively corresponding to the multiple objects, the grid models respectively corresponding to the multiple objects are placed to obtain the grid model of the first scene.

[0295] In a possible implementation, the device further includes a scene optimization module, which is used to:

[0296] For each two objects among the plurality of objects, calculating, according to the spatial features respectively corresponding to the two objects, a confidence level that the positional relationship between the two objects is each of a plurality of preset positional relationships;

[0297] Determine a first object category and a second object category to which the two objects respectively belong;

[0298] In the prior knowledge base, obtaining the positional relationship between the object of the first object category and the object of the second object category in the first scene as the reference confidence of each preset positional relationship;

[0299] Determining a positional relationship between the two objects based on the confidence level and the reference confidence level;

[0300] According to the positional relationship between the two objects, positions of the grid models respectively corresponding to the two objects in the grid model of the first scene are adjusted.

[0301] In a possible implementation, the AIGX module 430 is further configured to:

[0302] Obtaining a first texture map of the object;

[0303] The first texture map is rendered onto the first mesh model to obtain a first textured mesh model of the object.

[0304] In a possible implementation, the visual data is at least one of a non-color point cloud and a non-color voxel model, and the AIGX module is used to:

[0305] Receiving a texture selection request sent by a user, wherein the texture selection request carries texture style indication information, color type indication information, and color brightness indication information;

[0306] Acquire a first texture generation network corresponding to the texture type indication information, the color type indication information, and the color brightness indication information;

[0307] Based on the first texture generation network, a first texture map of the object is generated.

[0308] In a possible implementation, the prior knowledge is also used to guide the establishment of a texture generation network, and the feature extraction module 420 is also used to:

[0309] Extracting texture features from visual data of the object to obtain a first texture feature of the object;

[0310] Based on the first texture feature and the prior knowledge of the object, establishing a second texture generation network;

[0311] The AIGX module 430 is used to:

[0312] Based on the second texture generation network, a first texture map of the object is generated.

[0313] In a possible implementation, the prior knowledge includes hyperparameters of a first texture latent space network and a first texture generation network, and the AIGX module is used to:

[0314] Inputting the first texture feature into the first texture latent space network to obtain first texture generation network parameters;

[0315] Establishing an initial texture generation network based on the first texture generation network parameters and the first texture generation network hyperparameters;

[0316] Based on the first texture feature, training the initial texture generation network to obtain a second texture generation network;

[0317] Based on the second texture generation network, a first texture map of the object is generated.

[0318] In a possible implementation, the AIGX module 430 is further configured to:

[0319] Based on the first texture feature and the texture generation network parameters of the trained texture generation network, the first texture latent space network is trained to obtain a trained texture latent space network;

[0320] The first texture latent space network stored in the prior knowledge base is replaced by the trained texture latent space network.

[0321] In a possible implementation, the data preprocessing module is further used to:

[0322] Based on the object category labels stored in the prior knowledge base, an object category label corresponding to the object is determined.

[0323] In a possible implementation, the AIGX module 430 is further configured to:

[0324] Storing the first geometric feature of the object in the prior knowledge base;

[0325] receiving a first 3D digital asset self-growth request sent by a user, wherein the first 3D digital asset self-growth request carries indication information of a first growth starting point object, indication information of a first growth end point object, and a target quantity, and the first growth starting point object and the first growth end point object are different objects under a third object category;

[0326] In the prior knowledge base, the geometric features of the first growth starting point object, the geometric features of the first growth end point object, and the prior knowledge corresponding to the third object type are obtained;

[0327] Taking the geometric features of the first growth starting point object as a starting point and the geometric features of the first growth ending point object as an ending point, generating the target number of intermediate geometric features;

[0328] For each intermediate geometric feature of the target number of intermediate geometric features, a second geometry generation network is generated based on prior knowledge corresponding to the intermediate geometric feature and the third object type, and a second mesh model is generated based on the second geometry generation network.

[0329] In a possible implementation, the AIGX module 430 is further configured to:

[0330] Storing the first geometric feature of the object in the prior knowledge base;

[0331] receiving a second 3D digital asset self-growth request sent by a user, wherein the second 3D digital asset self-growth request carries indication information of a second growth starting point object, indication information of a second growth end point object, and a target step length, and the second growth starting point object and the second growth end point object are different objects under a fourth object category;

[0332] In the prior knowledge base, the geometric features of the second growth starting point object, the geometric features of the second growth end point object, and the prior knowledge corresponding to the fourth object type are obtained;

[0333] Taking the geometric feature of the second growth starting point object as a starting point and the geometric feature of the second growth ending point object as an ending point, generating at least one intermediate geometric feature;

[0334] For each of the at least one intermediate geometric feature, a third geometry generation network is generated based on prior knowledge corresponding to the intermediate geometric feature and the fourth object type, and a third mesh model is generated based on the third geometry generation network.

[0335] In a possible implementation, the AIGX module 430 is further configured to:

[0336] receiving a third 3D digital asset self-growth request sent by a user, wherein the third 3D digital asset self-growth request carries indication information of a second scene and indication information of a plurality of objects in the second scene;

[0337] In the digital asset library, obtain mesh models of multiple objects in the second scene;

[0338] In the prior knowledge base, obtaining a reference confidence level of each preset position relationship among a plurality of preset position relationships, that is, a position relationship between each two objects in the second scene;

[0339] determining a positional relationship between each two objects in the second scene based on a reference confidence level that the positional relationship between each two objects in the second scene is each of a plurality of preset positional relationships;

[0340] According to the positional relationship between every two objects in the second scene, grid models of multiple objects in the second scene are placed to obtain a grid model of the second scene.

[0341] Among them, the data preprocessing module 410, the feature extraction module 420 and the AIGX module 430 can all be implemented by software, or can be implemented by hardware. Exemplarily, the implementation of the data preprocessing module 410 is introduced below by taking the data preprocessing module 410 as an example. Similarly, the implementation of the feature extraction module 420 and the AIGX module 430 can refer to the implementation of the data preprocessing module 410.

[0342] As an example of a software functional unit, the data preprocessing module 410 may include code running on a computing instance. Among them, the computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Further, the above-mentioned computing instance may be one or more. For example, the task mining training module 501 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code can be distributed in the same region (region) or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code can be distributed in the same availability zone (AZ) or in different AZs, each AZ including a data center or multiple data centers with close geographical locations. Among them, usually a region can include multiple AZs.

[0343] Similarly, multiple hosts / virtual machines / containers used to run the code can be distributed in the same virtual private cloud (VPC) or in multiple VPCs. Usually, a VPC is set up in a region. For cross-region communication between two VPCs in the same region and between VPCs in different regions, a communication gateway needs to be set up in each VPC to achieve interconnection between VPCs through the communication gateway.

[0344] As an example of a hardware functional unit, the data preprocessing module 410 may include at least one computing device, such as a server, etc. Alternatively, the data preprocessing module 410 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof.

[0345] The multiple computing devices included in the data preprocessing module 410 can be distributed in the same region or in different regions. The multiple computing devices included in the A module can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the data preprocessing module 410 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0346] It should be noted that, in other embodiments, the data preprocessing module 410 can be used to execute any step performed by the computing device in the method for generating a mesh model, the feature extraction module 420 can be used to execute any step performed by the computing device in the method for generating a mesh model, and the AIGX module 430 can be used to execute any step performed by the computing device in the method for generating a mesh model. The steps that the data preprocessing module 410, the feature extraction module 420, and the AIGX module 430 are responsible for implementing can be specified as needed, and the data preprocessing module 410, the feature extraction module 420, and the AIGX module 430 respectively implement different steps in the method for generating a mesh model to realize the full functions of the device for generating a mesh model.

[0347] The present application also provides a computing device 100, which may be the computing device in the above embodiment. The computing device may be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device may also be a terminal device such as a desktop computer, a laptop computer, or a smart phone.

[0348] like Figure 8 As shown, computing device 100 includes: bus 102, processor 104, memory 106 and communication interface 108. Processor 104, memory 106 and communication interface 108 communicate with each other via bus 102. It should be understood that the present application does not limit the number of processors and memories in computing device 100.

[0349] The bus 102 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 8 The bus 102 is represented by only one line, but it does not mean that there is only one bus or one type of bus. The bus 102 may include a path for transmitting information between various components of the computing device 100 (eg, the memory 106, the processor 104, and the communication interface 108).

[0350] The processor 104 may include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0351] The memory 106 may include a volatile memory, such as a random access memory (RAM). The memory 106 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0352] The memory 106 stores executable codes, and the processor 104 executes the executable codes to respectively implement the functions of the aforementioned data preprocessing module 410, feature extraction module 420, and AIGX module 430, thereby implementing the processing performed by the computing device in the method for generating a mesh model. That is, the memory 106 stores instructions for executing the method for generating a mesh model.

[0353] The communication interface 108 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 100 and other devices or a communication network.

[0354] The embodiment of the present application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smart phone.

[0355] like Fig. 9 As shown, the computing device cluster includes at least one computing device 100. The memory 106 in one or more computing devices 100 in the computing device cluster may store the same instructions for executing the method for generating a mesh model.

[0356] In some possible implementations, the memory 106 of one or more computing devices 100 in the computing device cluster may also store partial instructions for executing the method for generating a grid model. In other words, the combination of one or more computing devices 100 may jointly execute the instructions for executing the method for generating a grid model.

[0357] It should be noted that the memory 106 in different computing devices 100 in the computing device cluster may store different instructions, which are respectively used to execute part of the functions of the method for generating a grid model.

[0358] In some possible implementations, the memory 106 of one or more computing devices 100 in the computing device cluster may also store partial instructions for executing the method for generating a grid model. In other words, the combination of one or more computing devices 100 may jointly execute the instructions for executing the method for generating a grid model.

[0359] It should be noted that the memory 106 in different computing devices 100 in the computing device cluster may store different instructions for executing part of the functions of grid model generation. That is, the instructions stored in the memory 106 in different computing devices 100 may implement the functions of one or more nodes in the data preprocessing module 410, the feature extraction module 420 and the AIGX module 430.

[0360] In some possible implementations, one or more computing devices in the computing device cluster may be connected via a network, which may be a wide area network or a local area network. Fig.10 A possible implementation is shown. Fig.10 As shown, two computing devices 100A and 100B are connected via a network. Specifically, the network is connected via a communication interface in each computing device. In this possible implementation, the memory 106 in the computing device 100A stores instructions for executing the AIGX module 430. At the same time, the memory 106 in the computing device 100B stores instructions for executing the functions of the data preprocessing module 410 and the feature extraction module 420. It should be understood that Fig.10 The functions of the computing device 100A shown in FIG. 1 may also be completed by multiple computing devices 100. Similarly, the functions of the computing device 100B may also be completed by multiple computing devices 100.

[0361] The present application embodiment also provides another computing device cluster. The connection relationship between the computing devices in the computing device cluster can be similar to that of Fig. 9 and Fig.10 The connection mode of the computing device cluster is different in that the memory 106 in one or more computing devices 100 in the computing device cluster may store the same instructions for executing the grid model generation method.

[0362] In some possible implementations, the memory 106 of one or more computing devices 100 in the computing device cluster may also store partial instructions for executing the method for generating a grid model. In other words, the combination of one or more computing devices 100 may jointly execute the instructions for executing the method for generating a grid model.

[0363] It should be noted that the memory 106 in different computing devices 100 in the computing device cluster may store different instructions for executing part of the functions of the method for generating a grid model. That is, the instructions stored in the memory 106 in different computing devices 100 may implement the functions of one or more nodes of the data preprocessing module 410, the feature extraction module 420 and the AIGX module 430.

[0364] The present application also provides a computer program product including instructions. The computer program product may be a software or program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on a computing device, at least one computing device executes the method for generating a grid model provided in the present application.

[0365] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk). The computer-readable storage medium includes instructions that instruct the computing device to execute the method for generating a grid model provided in the embodiment of the present application.

[0366] In this application, the terms "first", "second", etc. are used to distinguish the same or similar items with basically the same effects and functions. It should be understood that there is no logical or temporal dependency between "first" and "second", nor is the quantity and execution order limited. It should also be understood that although the following description uses the terms first, second, etc. to describe various elements, these elements should not be limited by the terms. These terms are only used to distinguish one element from another. For example, without departing from the scope of various examples, the first geometric feature can be referred to as the second geometric feature, and similarly, the second geometric feature can be referred to as the first geometric feature. The first geometric feature and the second geometric feature can both be collectively referred to as calculation codes, and in some cases, can be separate and different geometric features.

[0367] The term "at least one" in this application means one or more, and the term "plurality" in this application means two or more.

[0368] The above description is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for generating a mesh model, characterized in that: The method comprises: Acquire visual data of an object uploaded by a user, wherein the visual data includes at least one of an image, a point cloud, and a voxel model; Extracting geometric features from the visual data of the object to obtain a first geometric feature of the object; Acquire prior knowledge corresponding to the object category to which the object belongs in a prior knowledge base, wherein the prior knowledge is used to guide the establishment of a geometric generation network; Establishing a first geometric generative network based on the first geometric feature and prior knowledge corresponding to the object category to which the object belongs; Based on the first geometry generation network, a first mesh model of the object is generated.

2. The method according to claim 1, characterized in that The prior knowledge corresponding to the object category to which the object belongs includes a first geometric latent space network and a first geometric generative network hyperparameter, and the first geometric generative network is established based on the first geometric feature and the prior knowledge corresponding to the object category to which the object belongs, including: Inputting the first geometric feature into the first geometric latent space network to obtain first geometric generative network parameters; Establishing an initial geometry generative network based on the first geometry generative network parameters and the first geometry generative network hyperparameters; Based on the first geometric feature, the initial geometry generation network is trained to obtain a first geometry generation network.

3. The method according to claim 2, characterized in that The method further comprises: Based on the first geometric features and the geometric generative network parameters of the first geometric generative network, the first geometric latent space network is trained to obtain a trained geometric latent space network; The first geometric latent space network stored in the prior knowledge base is replaced by the trained geometric latent space network.

4. The method according to any one of claims 1 to 3, characterized in that The obtaining of visual data of the object uploaded by the user includes: Acquire visual data of a first scene uploaded by a user, wherein the first scene includes a plurality of objects; Perform object segmentation on the visual data of the first scene to obtain visual data of each of the multiple objects.

5. The method according to claim 4, characterized in that The method further comprises: Extracting spatial features of the multiple objects in the visual data of the first scene to obtain spatial features corresponding to the multiple objects, wherein the spatial features are used to characterize the positions and postures of the corresponding objects in the first scene; According to the spatial features respectively corresponding to the multiple objects, the grid models respectively corresponding to the multiple objects are placed to obtain the grid model of the first scene.

6. The method according to claim 5, characterized in that The method further comprises: For each two objects among the plurality of objects, calculating, according to the spatial features respectively corresponding to the two objects, a confidence level that the positional relationship between the two objects is each of a plurality of preset positional relationships; Determine a first object category and a second object category to which the two objects respectively belong; In the prior knowledge base, obtaining the positional relationship between the objects of the first object category and the objects of the second object category in the first scene as a reference confidence of each preset positional relationship; Determining a positional relationship between the two objects based on the confidence level and the reference confidence level; According to the positional relationship between the two objects, positions of the grid models respectively corresponding to the two objects in the grid model of the first scene are adjusted.

7. The method according to any one of claims 1 to 6, characterized in that The method further comprises: Obtaining a first texture map of the object; The first texture map is rendered onto the first mesh model to obtain a first textured mesh model of the object.

8. The method according to claim 7, characterized in that The visual data is at least one of a non-color point cloud and a non-color voxel model, and the method further includes: Receiving a texture selection request sent by a user, wherein the texture selection request carries texture style indication information, color type indication information, and color brightness indication information; The obtaining of a first texture map of the object comprises: Acquire a first texture generation network corresponding to the texture type indication information, the color type indication information, and the color brightness indication information; Based on the first texture generation network, a first texture map of the object is generated.

9. The method according to claim 7, characterized in that: The prior knowledge is also used to guide the establishment of a texture generation network. The method further includes: Extracting texture features from visual data of the object to obtain first texture features of the object; Based on the first texture feature and the prior knowledge of the object, establishing a second texture generation network; The obtaining of a first texture map of the object comprises: Based on the second texture generation network, a first texture map of the object is generated.

10. The method according to claim 9, characterized in that The prior knowledge includes a first texture latent space network and a first texture generation network hyperparameter, and the second texture generation network is established based on the first texture feature and the prior knowledge of the object, including: Inputting the first texture feature into the first texture latent space network to obtain first texture generation network parameters; Establishing an initial texture generation network based on the first texture generation network parameters and the first texture generation network hyperparameters; Based on the first texture feature, training the initial texture generation network to obtain a second texture generation network; Based on the second texture generation network, a first texture map of the object is generated.

11. The method according to claim 10, characterized in that The method further comprises: Based on the first texture feature and the texture generation network parameters of the trained texture generation network, the first texture latent space network is trained to obtain a trained texture latent space network; The first texture latent space network stored in the prior knowledge base is replaced by the trained texture latent space network.

12. The method according to any one of claims 1 to 11, characterized in that The method further comprises: Based on the object category labels stored in the prior knowledge base, an object category label corresponding to the object is determined.

13. The method according to any one of claims 1 to 12, characterized in that The method further comprises: Storing the first geometric feature of the object in the prior knowledge base; receiving a first 3D digital asset self-growth request sent by a user, wherein the first 3D digital asset self-growth request carries indication information of a first growth starting point object, indication information of a first growth end point object, and a target quantity, and the first growth starting point object and the first growth end point object are different objects under a third object category; In the prior knowledge base, the geometric features of the first growth starting point object, the geometric features of the first growth end point object, and the prior knowledge corresponding to the third object type are obtained; Taking the geometric features of the first growth starting point object as a starting point and the geometric features of the first growth ending point object as an ending point, generating the target number of intermediate geometric features; For each intermediate geometric feature of the target number of intermediate geometric features, a second geometry generation network is generated based on prior knowledge corresponding to the intermediate geometric feature and the third object type, and a second mesh model is generated based on the second geometry generation network.

14. The method according to any one of claims 1 to 13, characterized in that The method further comprises: Storing the first geometric feature of the object in the prior knowledge base; receiving a second 3D digital asset self-growth request sent by a user, wherein the second 3D digital asset self-growth request carries indication information of a second growth starting point object, indication information of a second growth end point object, and a target step length, and the second growth starting point object and the second growth end point object are different objects under a fourth object category; In the prior knowledge base, the geometric features of the second growth starting point object, the geometric features of the second growth end point object, and the prior knowledge corresponding to the fourth object type are obtained; Taking the geometric feature of the second growth starting point object as a starting point and the geometric feature of the second growth ending point object as an ending point, generating at least one intermediate geometric feature according to the target step length; For each of the at least one intermediate geometric feature, a third geometry generation network is generated based on prior knowledge corresponding to the intermediate geometric feature and the fourth object type, and a third mesh model is generated based on the third geometry generation network.

15. The method according to any one of claims 1 to 14, characterized in that The method further comprises: receiving a third 3D digital asset self-growth request sent by a user, wherein the third 3D digital asset self-growth request carries indication information of a second scene and indication information of a plurality of objects in the second scene; In the digital asset library, obtain mesh models of multiple objects in the second scene; In the prior knowledge base, obtaining a reference confidence level of each preset position relationship among a plurality of preset position relationships, that is, a position relationship between each two objects in the second scene; determining a positional relationship between each two objects in the second scene based on a reference confidence level that the positional relationship between each two objects in the second scene is each of a plurality of preset positional relationships; According to the positional relationship between every two objects in the second scene, grid models of multiple objects in the second scene are placed to obtain a grid model of the second scene.

16. A device for generating a grid model, characterized in that: The device comprises: A data preprocessing module, used to obtain visual data of an object uploaded by a user, wherein the visual data includes at least one of an image, a point cloud, and a voxel model; A feature extraction module, used for extracting geometric features from the visual data of the object to obtain a first geometric feature of the object; An artificial intelligence-based digital asset generation AIGX module is used to obtain prior knowledge corresponding to the object category to which the object belongs in a prior knowledge base, wherein the prior knowledge is used to guide the establishment of a geometry generation network; based on the first geometric feature and the prior knowledge corresponding to the object category to which the object belongs, a first geometry generation network is established; based on the first geometry generation network, a first mesh model of the object is generated.

17. The device according to claim 16, characterized in that The prior knowledge corresponding to the object category to which the object belongs includes hyperparameters of a first geometric latent space network and a first geometric generative network, and the AIGX module is used to: Inputting the first geometric feature into the first geometric latent space network to obtain first geometric generative network parameters; Establishing an initial geometry generative network based on the first geometry generative network parameters and the first geometry generative network hyperparameters; Based on the first geometric feature, the initial geometry generation network is trained to obtain a first geometry generation network.

18. The device according to any one of claims 16-17, characterized in that The data preprocessing module is used to: Acquire visual data of a first scene uploaded by a user, wherein the first scene includes a plurality of objects; Perform object segmentation on the visual data of the first scene to obtain visual data of each of the multiple objects.

19. The device according to claim 18, characterized in that The device also includes a positioning module, which is used to: Extracting spatial features of the multiple objects in the visual data of the first scene to obtain spatial features corresponding to the multiple objects, wherein the spatial features are used to characterize the positions and postures of the corresponding objects in the first scene; According to the spatial features respectively corresponding to the multiple objects, the grid models respectively corresponding to the multiple objects are placed to obtain the grid model of the first scene.

20. The device according to claim 19, characterized in that The device also includes a scene optimization module, which is used to: For each two objects among the plurality of objects, calculating, according to the spatial features respectively corresponding to the two objects, a confidence level that the positional relationship between the two objects is each of a plurality of preset positional relationships; Determine a first object category and a second object category to which the two objects respectively belong; In the prior knowledge base, obtaining the positional relationship between the objects of the first object category and the objects of the second object category in the first scene as a reference confidence of each preset positional relationship; Determining a positional relationship between the two objects based on the confidence level and the reference confidence level; According to the positional relationship between the two objects, positions of the grid models respectively corresponding to the two objects in the grid model of the first scene are adjusted.

21. The device according to any one of claims 16 to 20, characterized in that The AIGX module is also used to: Storing the first geometric feature of the object in the prior knowledge base; receiving a first 3D digital asset self-growth request sent by a user, wherein the first 3D digital asset self-growth request carries indication information of a first growth starting point object, indication information of a first growth end point object, and a target quantity, and the first growth starting point object and the first growth end point object are different objects under a third object category; In the prior knowledge base, the geometric features of the first growth starting point object, the geometric features of the first growth end point object, and the prior knowledge corresponding to the third object type are obtained; Taking the geometric features of the first growth starting point object as a starting point and the geometric features of the first growth ending point object as an ending point, generating the target number of intermediate geometric features; For each intermediate geometric feature of the target number of intermediate geometric features, a second geometry generation network is generated based on prior knowledge corresponding to the intermediate geometric feature and the third object type, and a second mesh model is generated based on the second geometry generation network.

22. The device according to any one of claims 16 to 20, characterized in that The AIGX module is also used to: Storing the first geometric feature of the object in the prior knowledge base; receiving a second 3D digital asset self-growth request sent by a user, wherein the second 3D digital asset self-growth request carries indication information of a second growth starting point object, indication information of a second growth end point object, and a target step length, and the second growth starting point object and the second growth end point object are different objects under a fourth object category; In the prior knowledge base, the geometric features of the second growth starting point object, the geometric features of the second growth end point object, and the prior knowledge corresponding to the fourth object type are obtained; Taking the geometric feature of the second growth starting point object as a starting point and the geometric feature of the second growth ending point object as an ending point, generating at least one intermediate geometric feature according to the target step length; For each of the at least one intermediate geometric feature, a third geometry generation network is generated based on prior knowledge corresponding to the intermediate geometric feature and the fourth object type, and a third mesh model is generated based on the third geometry generation network.

23. The device according to any one of claims 16 to 22, characterized in that The AIGX module is also used to: receiving a third 3D digital asset self-growth request sent by a user, wherein the third 3D digital asset self-growth request carries indication information of a second scene and indication information of a plurality of objects in the second scene; In the digital asset library, obtain mesh models of multiple objects in the second scene; In the prior knowledge base, obtaining a reference confidence level of each preset position relationship among a plurality of preset position relationships, that is, a position relationship between each two objects in the second scene; determining a positional relationship between each two objects in the second scene based on a reference confidence level that the positional relationship between each two objects in the second scene is each of a plurality of preset positional relationships; According to the positional relationship between every two objects in the second scene, grid models of multiple objects in the second scene are placed to obtain a grid model of the second scene.

24. A computing device cluster, characterized in that: The computing device cluster includes at least one computing device, each computing device including a processor and a memory; The processor of the at least one computing device is used to execute instructions stored in the memory of the device, so that the computing device cluster executes the method for generating a grid model as described in any one of claims 1 to 15.

25. A computer program product comprising instructions, characterized in that When the instructions are executed by a computing device cluster, the computing device cluster executes the method for generating a grid model as described in any one of claims 1 to 15.

26. A computer-readable storage medium, characterized in that: The method comprises computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster executes the method for generating a grid model according to any one of claims 1 to 15.

Images