Three-dimensional model generation method, device, equipment and storage medium

Abstract

This application discloses a method, apparatus, device, and storage medium for generating a three-dimensional model. The method includes: acquiring a two-dimensional view of a three-dimensional object and extracting features from the two-dimensional view to obtain object feature vectors of the three-dimensional object; acquiring K feature vectors containing Gaussian features and, through an attention mechanism, performing feature enhancement processing on the object feature vectors of the three-dimensional object using each reference feature vector to obtain K enhanced feature vectors; constructing a Gaussian three-plane of the three-dimensional object using the Gaussian features of each enhanced feature vector, thereby performing three-dimensional reconstruction of the three-dimensional object based on the Gaussian three-plane to obtain a three-dimensional model of the three-dimensional object, thereby improving the object representation capability of the three-dimensional model and thus improving the model quality of the three-dimensional model.

Description

Technical Field

[0001] This application relates to the field of Internet technology, specifically to the field of image processing technology, and in particular to a method, apparatus, device, and storage medium for generating three-dimensional models. Background Technology

[0002] With the continuous development of large-scale modeling technology, large-scale 3D reconstruction models (LRM) can be used to reconstruct 3D objects, obtaining a 3D model of the object (i.e., a polygonal representation of the 3D object), thereby reducing the modeling cost for modelers. Currently, the workflow of existing large-scale 3D reconstruction models is roughly as follows: preprocessing of four vertical camera views to obtain view images of the 3D object; inputting these view images into the latent space to complete image feature learning; decomposing the learned image features into three image feature planes (referred to as the three planes); and then processing these three image feature planes through a multi-layer fully connected network to obtain the 3D model. It is evident that existing large-scale 3D reconstruction models mainly use the representation of three image feature planes as the basis for 3D reconstruction. This results in a 3D model that cannot accurately represent the 3D object, leading to poor model quality. Summary of the Invention

[0003] This application provides a method, apparatus, device, and storage medium for generating three-dimensional models, which can improve the object representation capability of three-dimensional models, thereby improving the model quality of three-dimensional models.

[0004] On one hand, embodiments of this application provide a method for generating a three-dimensional model, the method comprising:

[0005] Obtain a two-dimensional view of a three-dimensional object, and extract features from the two-dimensional view to obtain the object feature vector of the three-dimensional object;

[0006] Obtain K reference feature vectors, where each reference feature vector is a feature vector containing Gaussian features, where Gaussian features refer to features distributed in a Gaussian distribution, and K is a positive integer.

[0007] Through an attention mechanism, the object feature vector is enhanced by each reference feature vector to obtain K enhanced feature vectors, and each enhanced feature vector represents a Gaussian distribution.

[0008] Obtain the Gaussian features of each enhanced feature vector. The Gaussian features of any enhanced feature vector refer to the characteristics of the Gaussian distribution represented by the corresponding enhanced feature vector.

[0009] The obtained Gaussian features are used to construct the Gaussian three planes of the three-dimensional object, and the three-dimensional object is reconstructed according to the Gaussian three planes to obtain the three-dimensional model of the three-dimensional object.

[0010] On the other hand, embodiments of this application provide a three-dimensional model generation apparatus, the apparatus comprising:

[0011] The acquisition unit is used to acquire a two-dimensional view of a three-dimensional object and extract features from the two-dimensional view to obtain the object feature vector of the three-dimensional object.

[0012] The acquisition unit is further configured to acquire K reference feature vectors, wherein the reference feature vectors are feature vectors containing Gaussian features, and Gaussian features are features of a Gaussian distribution, and K is a positive integer;

[0013] The processing unit is used to perform feature enhancement processing on the object feature vector using each reference feature vector through an attention mechanism, to obtain K enhanced feature vectors, each enhanced feature vector representing a Gaussian distribution.

[0014] The processing unit is also used to obtain the Gaussian features of each enhanced feature vector, wherein the Gaussian features of any enhanced feature vector refer to the features of the Gaussian distribution represented by the corresponding enhanced feature vector.

[0015] The processing unit is further configured to construct a Gaussian three-plane of the three-dimensional object using the acquired Gaussian features, and to perform three-dimensional reconstruction of the three-dimensional object based on the Gaussian three-plane to obtain a three-dimensional model of the three-dimensional object.

[0016] In another aspect, embodiments of this application provide a computer device, the computer device including an input interface and an output interface, the computer device further including:

[0017] Processor and computer storage media;

[0018] The processor is adapted to implement one or more instructions, and the computer storage medium stores one or more instructions, which are adapted to be loaded and executed by the processor to perform the aforementioned three-dimensional model generation method.

[0019] In another aspect, embodiments of this application provide a computer storage medium storing one or more instructions, which are adapted to be loaded and executed by a processor using the aforementioned three-dimensional model generation method.

[0020] In another aspect, embodiments of this application provide a computer program product, which includes one or more instructions; when one or more instructions in the computer program product are executed by a processor, they implement the aforementioned three-dimensional model generation method.

[0021] This application embodiment can extract features from a two-dimensional view of a three-dimensional object to obtain the object feature vector of the three-dimensional object, and obtain K reference feature vectors containing Gaussian features. Then, through an attention mechanism, each reference feature vector is used to adaptively enhance the object feature vector of the three-dimensional object, resulting in K enhanced feature vectors. This enhances the feature vectors, making them more accurate in representing the three-dimensional object compared to the object feature vectors. Furthermore, a Gaussian three-plane can be constructed using the Gaussian features of the K enhanced feature vectors, and the three-dimensional object can be reconstructed based on the Gaussian three-plane to obtain a three-dimensional model of the three-dimensional object. This improves the object representation capability of the three-dimensional model, thereby enhancing its quality. It is understood that since the Gaussian three-plane is constructed using Gaussian features, it has fewer parameters. Therefore, using this Gaussian three-plane to reconstruct the three-dimensional model can reduce the parameters required for the three-dimensional model representation, thereby reducing the space occupied by the three-dimensional model and increasing its data size. Attached Figure Description

[0022] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1a This is a schematic diagram of a Gaussian three-plane provided in an embodiment of this application;

[0024] Figure 1b This is a schematic diagram illustrating the principle of a three-dimensional model generation method based on Gaussian three planes provided in an embodiment of this application;

[0025] Figure 2 This is a flowchart illustrating a three-dimensional model generation method provided in an embodiment of this application;

[0026] Figure 3a This is a schematic diagram illustrating an embodiment of the present application for generating object feature vectors of three-dimensional objects;

[0027] Figure 3b This is a schematic diagram illustrating the generation of a reference feature vector set provided in an embodiment of this application;

[0028] Figure 3cThis is a schematic diagram illustrating a method for obtaining Gaussian features of an enhanced feature vector according to an embodiment of this application;

[0029] Figure 4 This is a flowchart illustrating a three-dimensional model generation method provided in another embodiment of this application;

[0030] Figure 5a This is a schematic diagram of mapping the p-th three-dimensional spatial point to the feature planes in the Gaussian three-plane according to an embodiment of this application;

[0031] Figure 5b This is a schematic diagram illustrating the generation of a hybrid feature of the p-th three-dimensional spatial point according to an embodiment of this application;

[0032] Figure 5c This is a schematic diagram illustrating the generation of a three-dimensional model according to an embodiment of this application;

[0033] Figure 6a This is a schematic diagram of a three-dimensional reconstruction technique based on Gaussian three planes provided in an embodiment of this application;

[0034] Figure 6b This is a schematic diagram illustrating a process of preprocessing and extracting features from a single-view image, as provided in an embodiment of this application.

[0035] Figure 6c This is a schematic diagram illustrating the generation of enhanced feature vectors through a multi-layer attention mechanism, as provided in an embodiment of this application.

[0036] Figure 6d This is a schematic diagram illustrating the generation of a 3D image of a 3D object according to an embodiment of this application;

[0037] Figure 7 This is a schematic diagram of the structure of a three-dimensional model generation device provided in an embodiment of this application;

[0038] Figure 8 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0039] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0040] This application proposes a 3D model generation method based on Gaussian three planes to generate 3D models that can accurately represent 3D objects, thereby improving the object representation capability of 3D models and thus enhancing the model quality. The Gaussian three planes refer to three feature planes constructed based on Gaussian features, which can be denoted as the xy plane (first feature plane), the yz plane (second feature plane), and the xz plane (third feature plane). These three feature planes can be mutually orthogonal, such as... Figure 1a As shown; furthermore, the Gaussian features mentioned here refer to the characteristics of the Gaussian distribution, also known as the normal distribution or the normal distribution. It is a continuous probability distribution with a bell-shaped curve. For example, the features of a Gaussian distribution may include, but are not limited to: mean, variance, skewness, weights, etc. Among these, the mean of the Gaussian distribution indicates its central location, the variance indicates its dispersion, the skewness indicates the degree of skewness of the corresponding Gaussian distribution relative to a benchmark Gaussian distribution, and the weights indicate the importance of the Gaussian distribution.

[0041] See Figure 1b As shown in the embodiments of this application, the general principle of the 3D model generation method based on Gaussian three planes is as follows: A 2D view of the 3D object is obtained, and features are extracted from this 2D view to obtain the object feature vector of the 3D object. Additionally, a reference feature set can be obtained, which may include K reference feature vectors (K is a positive integer). The so-called reference feature vectors refer to feature vectors containing (implicit) Gaussian features. Furthermore, an attention mechanism can be used to perform feature enhancement processing on the object feature vector of the 3D object using each reference feature vector in the reference feature set, resulting in K enhanced feature vectors, each enhancing feature vector representing a Gaussian distribution. Since enhanced feature vectors are feature-enhancing processing of the object feature vectors of a 3D object, they can represent 3D objects more accurately than object feature vectors. Based on this, Gaussian features of each enhanced feature vector can be obtained, and Gaussian three planes of the 3D object can be constructed using the obtained Gaussian features. Then, the 3D object can be reconstructed based on the Gaussian three planes to obtain a 3D model of the 3D object. This improves the object representation capability of the 3D model, thereby improving the model quality of the 3D model and reducing the parameters required for 3D model representation, thus reducing the space occupied by the 3D model and increasing the data scale of the 3D model.

[0042] In practical implementation, the aforementioned 3D model generation method can be executed by a computer device, which can be a terminal or a server. Alternatively, the 3D model generation method can be executed jointly by a terminal and a server; for example, the terminal can be responsible for extracting the object feature vectors of the 3D object, performing feature enhancement processing on the object feature vectors, and sending K enhanced feature vectors to the server, which is responsible for obtaining the Gaussian features of each enhanced feature vector, constructing a Gaussian three-plane using the obtained Gaussian features, and reconstructing the 3D model of the 3D object based on the Gaussian three-plane; or, the server can be responsible for feature extraction, feature enhancement, and Gaussian three-plane construction, and then sending the constructed Gaussian three-plane to the terminal, which is responsible for reconstructing the 3D model of the 3D object based on the Gaussian three-plane, and so on.

[0043] The terminals mentioned above can be smartphones, computers (such as tablets, laptops, desktop computers, etc.), smart wearable devices (such as smartwatches, smart glasses), smart voice interaction devices, smart home appliances (such as smart TVs), vehicle terminals, or aircraft, etc. Furthermore, the servers mentioned above can be independent physical servers, server clusters or distributed systems composed of multiple physical servers, or cloud servers providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms, etc. Further, the terminals and servers can be located inside or outside the blockchain network, without limitation; even further, the terminals and servers can upload any data stored internally to the blockchain network for storage to prevent the internally stored data from being tampered with and to improve data security.

[0044] Based on the above description, the following section uses a computer device as the execution subject as an example, combined with... Figure 2 The flowchart shown illustrates the specific implementation process of the 3D model generation method proposed in this application embodiment. It is worth emphasizing that, in this application embodiment, if user information or other related data is involved, when any method embodiment proposed in this application is applied to a specific product or technology, this related data is collected with the user's permission or consent, and the collection, use, and processing of the related data comply with the relevant laws, regulations, and standards of the relevant region.

[0045] Please see Figure 2 As shown, the three-dimensional model generation method proposed in this application embodiment can generally include the following steps S201-S205:

[0046] S201: Obtain a two-dimensional view of the three-dimensional object, and extract features from the two-dimensional view to obtain the object feature vector of the three-dimensional object.

[0047] In this context, a three-dimensional object refers to an object defined in three-dimensional space. Specifically, the three-dimensional object can be an entity in the real world, such as a real person, object, or scene; or it can be an object in the virtual world (i.e., a fictional world), such as a virtual game character, a virtual game scene, a virtual game item, and so on.

[0048] A two-dimensional view of a three-dimensional object refers to a two-dimensional image drawn from the perspective of at least one camera. The camera perspective refers to the viewpoint from which the camera observes the three-dimensional object; a camera is a component used to present a three-dimensional object. Specifically, the two-dimensional view can be a single-view image, meaning it can include one viewpoint image; or it can be a multi-view image, meaning it can include N viewpoint images, where N is an integer greater than 1. Different viewpoint images are drawn from different camera perspectives, corresponding to different camera perspectives. This enriches the object information in the two-dimensional view, thereby improving the accuracy of the object feature vector obtained by subsequent feature extraction from the two-dimensional view.

[0049] In one specific implementation, if the two-dimensional view includes a single viewpoint image, when the computer device extracts features from the two-dimensional view to obtain the object feature vector of the three-dimensional object, it can directly input the two-dimensional view into the three-dimensional reconstruction model (i.e., the neural network model used to reconstruct and generate the three-dimensional model), and call the feature extraction network in the three-dimensional reconstruction model to extract features from the two-dimensional view, thereby using the features extracted by the feature extraction network as the object feature vector of the three-dimensional object. Alternatively, the two-dimensional view can be preprocessed to obtain a preprocessed two-dimensional view, which can then be input into the three-dimensional reconstruction model to call the feature extraction network in the three-dimensional reconstruction model to extract features from the preprocessed two-dimensional view and obtain the object feature vector of the three-dimensional object. The image preprocessing mentioned here may include at least one of the following: modifying the image size and modifying the image mean (i.e., the mean value between individual pixel values ​​in the image). It should be noted that when image preprocessing includes both modifying the image size and modifying the image mean, the computer device may first perform the operation of modifying the image size on the two-dimensional view and then perform the operation of modifying the image mean, or it may first perform the operation of modifying the image mean and then perform the operation of modifying the image size. This application embodiment does not limit this.

[0050] In another specific implementation, if the two-dimensional view includes N viewpoint images (N is an integer greater than 1), the computer device may include the following steps when extracting features from the two-dimensional view to obtain the object feature vector of the three-dimensional object: s11, extracting features from each viewpoint image in the two-dimensional view to obtain N image feature vectors, one image feature vector corresponding to one viewpoint image; s12, integrating (e.g., stitching) the N image feature vectors to obtain the object feature vector of the three-dimensional object. Specifically, for the nth viewpoint image (n∈[1,N]) in the two-dimensional view, the computer device may extract features from the nth viewpoint image to obtain the nth image feature vector (i.e., the image feature vector corresponding to the nth viewpoint image) by directly inputting the nth viewpoint image into the three-dimensional reconstruction model and calling the feature extraction network in the three-dimensional reconstruction model to extract features from the nth viewpoint image to obtain the nth image feature vector. Alternatively, the nth viewpoint image can be preprocessed to obtain a preprocessed nth viewpoint image. This preprocessed nth viewpoint image can then be input into a 3D reconstruction model. The feature extraction network within the 3D reconstruction model can then extract features from the preprocessed nth viewpoint image to obtain the nth image feature vector. Based on this, taking the image preprocessing process, which sequentially includes modifying the image size and the image mean, as an example, the specific process of generating the object feature vector of the 3D object in this case can be exemplarily described in [reference needed]. Figure 3a As shown.

[0051] Based on the above descriptions of the two specific implementations, the following points need to be explained:

[0052] ① The aforementioned modification of image size refers to scaling an image (such as a 2D view) according to a random image size, so that the scaled image has the same size as the random image size. By randomly scaling images, the sizes of images input to the 3D reconstruction model can be enriched, allowing the 3D reconstruction model to process images of various sizes, thereby enhancing the model's generalization ability.

[0053] ② The aforementioned modification of the image mean refers to: by modifying at least one pixel value in the image, the image mean is limited to a preset mean (e.g., a value of 0.5), and the image variance (i.e., the variance between individual pixel values ​​in the image) is also limited to a preset variance (e.g., a value of 1). By modifying the image mean, excessive fluctuations between pixel values ​​in the image input to the 3D reconstruction model can be avoided, thereby improving the information stability of the input image. This prevents overshooting by the various network layers in the 3D reconstruction model when processing the input image, thus improving the stability of the 3D reconstruction model.

[0054] ③ The feature extraction network mentioned above refers to a network with feature extraction capabilities. This application does not limit the specific structure of this feature extraction network. For example, the feature extraction network may include at least one convolutional layer and a feature extraction module. The feature extraction module may be a Feed Forward structure (containing multiple linear layers, interleaving activation functions (GeLU) between each layer, and dropout layers (network layers that improve the model's generalization ability by randomly dropping some neurons; the dropout rate can be a preset value (e.g., 0.05)). In this case, the way to call the feature extraction network to extract features from any image could be: calling at least one convolutional layer to perform convolution processing on any image, and calling the feature extraction module to extract features from the result of the convolution processing, thereby improving the accuracy of feature extraction. Alternatively, the feature extraction network may also include only at least one convolutional layer or only a feature extraction module, and so on.

[0055] S202, obtain K reference feature vectors, where K is a positive integer.

[0056] As mentioned above, a reference feature vector refers to a feature vector containing Gaussian features. That is, each of the K reference feature vectors obtained in step S202 can contain Gaussian features. It should be noted that the Gaussian features in each reference feature vector can be implicit features (i.e., features that are not directly visible). In other words, the Gaussian features in each reference feature vector are not directly visible in the corresponding reference feature vector, but can be extracted through learning or model inference. Optionally, in other embodiments, the Gaussian features in each reference feature vector can also be explicit features (i.e., directly visible features), and this is not limited.

[0057] In a practical implementation, the K reference feature vectors can be obtained from a pre-defined set of reference feature vectors, and this set of reference feature vectors can be generated by iteratively optimizing an initial feature vector set. For example, see [link to example]. Figure 3bAs shown, the specific process of iteratively optimizing the initial feature vector set to obtain the reference feature vector set can be roughly as follows: s21, obtain the sample object feature vectors of H sample objects and the 3D model label of each sample object, where H is a positive integer; the generation method of the sample object feature vector of any sample object is the same as the generation method of the object feature vector of the aforementioned 3D object, and will not be repeated here; the 3D model label of any sample object may include: display style parameters used to represent each 3D spatial point of the corresponding sample object. s22, obtain the initial feature vector set, which includes K initial feature vectors. Each initial feature vector can be zero or preset based on empirical values ​​or business requirements. s23, traverse each sample object and determine the h-th sample object currently traversed (h∈[1,H]). s24, through the attention mechanism, use each initial feature vector in the initial feature vector set to perform feature enhancement processing on the sample object feature vector of the h-th sample object, obtaining K feature enhancement results. Step s25: Construct a Gaussian triplane for the h-th sample object using the Gaussian features from the K feature enhancement results. Then, perform 3D reconstruction of the h-th sample object based on the constructed Gaussian triplane to obtain the sample 3D model of the h-th sample object. Step s26: Based on the difference between the sample 3D model and the 3D model label of the h-th sample object, optimize each initial feature vector in the initial feature vector set with the goal of reducing this difference. Step s27: Iteratively execute steps s23-s26 to iteratively optimize each initial feature vector in the initial feature vector set to obtain a reference feature vector set, ensuring that each reference feature vector in the reference feature vector set (i.e., the iteratively optimized initial feature vector) contains Gaussian features.

[0058] It is understood that the above is merely an exemplary example of how to obtain a reference feature vector set by iteratively optimizing an initial feature vector set, and is not an exhaustive list. In other embodiments, other implementation methods are also used to iteratively optimize the initial feature vector set to obtain a reference feature vector set. In addition, as an optional implementation method, the reference feature vector set can also be a feature vector set preset by an experienced person.

[0059] S203 uses an attention mechanism to perform feature enhancement processing on the object feature vector using each reference feature vector, resulting in K enhanced feature vectors.

[0060] In this context, an enhanced feature vector can be understood as the object's feature vector after feature enhancement processing. An enhanced feature vector is obtained by using an attention mechanism to enhance the object's feature vector with a reference feature vector, and an enhanced feature vector represents a Gaussian distribution.

[0061] Attention mechanisms are a data processing method within Machine Learning (ML). Machine learning is the core of Artificial Intelligence (AI), the fundamental way to endow computer devices with intelligence, and its applications span all areas of artificial intelligence. Specifically, machine learning is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory, and many other disciplines. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance.

[0062] In this embodiment, the operation rule of the attention mechanism can be cross attention, that is, taking the inner product of the reference feature vector and the object feature vector as the input pattern for feature enhancement. Specifically, the object feature vector of the 3D object may include multiple elements; the specific implementation of step S203 may include the following steps: calculating the similarity between each element in the object feature vector and the k-th reference feature vector, k∈[1,K]; specifically, the similarity between each element and the k-th reference feature vector can be obtained by taking the inner product of the object feature vector and the k-th reference feature vector. Further, according to the relationship that the similarity is proportional to the attention weight, an attention weight can be assigned to the corresponding element according to the similarity between each element and the k-th reference feature vector; and according to the attention weight of each element, the elements are weighted and summed to obtain the k-th enhanced feature vector.

[0063] It should be noted that computer devices can use an attention mechanism to perform feature enhancement processing on the object feature vector using the k-th reference feature vector to obtain the k-th enhanced feature vector. In this case, the specific method mentioned above for obtaining the k-th enhanced feature vector by weighted summation of each element based on its attention weight can be as follows: weighted summation of each element based on its attention weight, and the weighted summation result is directly used as the k-th enhanced feature vector; or, the weighted summation result is input into a fully connected network for processing to obtain the k-th enhanced feature vector.

[0064] Optionally, the computer device can also use an attention mechanism to iteratively perform feature enhancement processing on the object feature vector multiple times (i.e., at least twice) using the k-th reference feature vector to obtain the k-th enhanced feature vector, thereby improving the quality of the k-th enhanced feature vector. In this case, the specific method mentioned above for obtaining the k-th enhanced feature vector by weighted summation of each element based on the attention weight of each element can be as follows: weighted summation of each element based on the attention weight of each element to obtain an intermediate latent vector; and then using the attention mechanism to perform feature enhancement processing on the intermediate latent vector using the k-th enhanced feature vector to obtain a new intermediate latent vector, and so on, until multiple feature enhancement processing is performed iteratively. The intermediate latent vector obtained from the last feature enhancement processing is then used as the k-th enhanced feature vector, or the intermediate latent vector obtained from the last feature enhancement processing is input into a fully connected network for processing to obtain the k-th enhanced feature vector.

[0065] S204, obtain the Gaussian features of each enhanced feature vector.

[0066] In this context, the Gaussian feature of any enhanced feature vector refers to the characteristics of the Gaussian distribution represented by the corresponding enhanced feature vector. These characteristics may include, but are not limited to, the mean, variance, skewness, and weights.

[0067] See Figure 3c As shown: For the k-th enhanced feature vector, the computer device can obtain the mean, variance, skewness, and weights of the Gaussian distribution represented by the k-th enhanced feature vector, and then use the obtained mean, variance, skewness, and weights as the Gaussian features of the k-th enhanced feature vector. Therefore, the Gaussian features of the k-th enhanced feature vector can include: mean (μ), variance (σ), skewness (r), and weights (f).

[0068] Specifically, the 3D reconstruction model may include a Gaussian feature extraction module (or Gaussian feature separation module). Based on this, the computer device can obtain the mean, variance, deflection, and weight of the Gaussian distribution represented by the k-th enhanced feature vector by inputting the k-th enhanced feature vector into the Gaussian feature extraction module, thereby calling the Gaussian feature extraction module to extract features from the k-th enhanced feature vector, thus obtaining the mean, variance, deflection, and weight of the Gaussian distribution represented by the k-th enhanced feature vector.

[0069] S205: Construct a Gaussian three-plane model of the three-dimensional object using the obtained Gaussian features, and perform three-dimensional reconstruction of the three-dimensional object based on the Gaussian three-plane model to obtain the three-dimensional model of the three-dimensional object.

[0070] In a specific implementation, the method for constructing the Gaussian three planes of a 3D object using the acquired Gaussian features can be as follows: Three initial feature planes are acquired, each containing at least one mesh; the acquired Gaussian features are used to rasterize each mesh in each feature plane to obtain feature data for each mesh; the feature data of each mesh is then filled into the corresponding mesh to obtain the Gaussian three planes of the 3D object. Rasterization refers to the processing of feature data for constructing the mesh using Gaussian features. Specifically, this can be achieved using Gaussian features of mesh-related enhanced feature vectors to construct the feature data for the corresponding mesh. The mesh-related enhanced feature vector refers to the enhanced feature vector used to represent the Gaussian distribution intersecting with the mesh; that is, if the Gaussian distribution represented by an enhanced feature vector intersects with the mesh, then the enhanced feature vector can be considered to influence the mesh, and thus is considered to be related to the mesh.

[0071] Therefore, the embodiments of this application can construct a Gaussian three-plane structure by rasterizing each grid in the three feature planes using Gaussian features with K enhanced feature vectors. Compared with the existing technology of splitting image features to construct three planes, this construction method allows the resolution (size) of the Gaussian three-plane structure to be free from the constraint of the number of image features, thereby improving the resolution of the Gaussian three-plane structure and thus improving the resolution of the three-dimensional model subsequently constructed based on the Gaussian three-plane structure.

[0072] As described above, the Gaussian three-plane system of a 3D object comprises three feature planes, each containing at least one mesh filled with feature data. Therefore, a specific implementation for reconstructing a 3D model of a 3D object using the Gaussian three-plane system can be as follows: Each 3D spatial point representing the 3D object is mapped to a feature plane within the Gaussian three-plane system. For any given 3D spatial point, interpolation can be performed on the corresponding feature plane based on its mapping points, and the interpolation results can be blended in 3D space to obtain the blended features of that 3D spatial point. After processing each 3D spatial point and obtaining its blended features, the blended features can be used to construct the 3D model of the 3D object, such as by decoding the blended features of each 3D spatial point. It is understood that this is merely an illustrative example of one specific implementation for constructing a 3D model based on the Gaussian three-plane system, and not an exhaustive list.

[0073] This application embodiment can extract features from a two-dimensional view of a three-dimensional object to obtain the object feature vector of the three-dimensional object, and obtain K reference feature vectors containing Gaussian features. Then, through an attention mechanism, each reference feature vector is used to adaptively enhance the object feature vector of the three-dimensional object, resulting in K enhanced feature vectors. This enhances the feature vectors, making them more accurate in representing the three-dimensional object compared to the object feature vectors. Furthermore, a Gaussian three-plane can be constructed using the Gaussian features of the K enhanced feature vectors, and the three-dimensional object can be reconstructed based on the Gaussian three-plane to obtain a three-dimensional model of the three-dimensional object. This improves the object representation capability of the three-dimensional model, thereby enhancing its quality. It is understood that since the Gaussian three-plane is constructed using Gaussian features, it has fewer parameters. Therefore, using this Gaussian three-plane to reconstruct the three-dimensional model can reduce the parameters required for the three-dimensional model representation, thereby reducing the space occupied by the three-dimensional model and increasing its data size.

[0074] Based on the above Figure 2 The description of the method embodiments shown in this application also proposes a three-dimensional model generation method; in this application embodiment, the three-dimensional model generation method executed by a computer device is still used as an example for illustration. Please refer to... Figure 4 As shown, the method for generating this 3D model can be roughly divided into the following steps S401-S406:

[0075] S401: Obtain a two-dimensional view of the three-dimensional object, and extract features from the two-dimensional view to obtain the object feature vector of the three-dimensional object.

[0076] The two-dimensional view includes N viewpoint images, with different viewpoint images corresponding to different camera viewpoints. When extracting features from the two-dimensional view in step S401 to obtain the object feature vector of the three-dimensional object, the computer device can extract features from each viewpoint image in the two-dimensional view separately to obtain N image feature vectors, and then integrate these N image feature vectors to obtain the object feature vector of the three-dimensional object. The specific implementation process of extracting features from each viewpoint image in the two-dimensional view to obtain N image feature vectors can include at least the following steps 1-3:

[0077] Step 1: For the nth viewpoint image (n∈[1, N]) in the two-dimensional view, randomly generate an image size. Specifically, a preset image size range can be obtained, and an image size can be randomly selected from this range as the randomly generated image size. This image size range can be set based on actual needs or empirical values. For example, the image size range can be [108, 384], where 108 represents an image with both length and width of 108, and 384 represents an image with both length and width of 384. Optionally, the computer device can also use a random number generator or random number generation function to randomly generate an image size in real time; this embodiment does not limit this.

[0078] Step 2: Scale the nth viewpoint image according to the randomly generated image size to obtain an intermediate image. Specifically, the computer device can scale the nth viewpoint image according to the randomly generated image size, making the size of the scaled nth viewpoint image equal to the randomly generated image size. Furthermore, the computer device can directly use the scaled nth viewpoint image as the intermediate image, or it can crop a preset-size (e.g., 108) image region from the scaled nth viewpoint image as the intermediate image. This not only unifies the image size of the intermediate images corresponding to each viewpoint image when they have different image sizes, facilitating feature extraction by the subsequent feature extraction network and improving feature extraction efficiency, but also removes unimportant information from the viewpoint image to a certain extent through cropping, allowing for subsequent feature extraction of key information from the viewpoint image, thereby further improving feature extraction efficiency and accuracy.

[0079] Step 3: Extract features from the intermediate image to obtain the nth image feature vector. Specifically, the computer device can input the intermediate image into the feature extraction network of the 3D reconstruction model to extract features from the intermediate image and obtain the nth image feature vector. Alternatively, to improve the stability of the information in the image input to the feature extraction network and thus improve the accuracy of the feature extraction results, the computer device can also adjust at least one pixel value in the intermediate image to modify the image mean to a preset mean, obtaining a modified intermediate image. Feature extraction can then be performed on the modified intermediate image to obtain the nth image feature vector. In other words, the modified intermediate image is input into the feature extraction network of the 3D reconstruction model to extract features from the modified intermediate image and obtain the nth image feature vector.

[0080] The mean value of the intermediate image refers to the average value among all pixel values ​​in the intermediate image. The modified intermediate image is obtained by adjusting at least one pixel value in the intermediate image to change its mean value to a preset mean value. Specific implementation methods may include, but are not limited to: obtaining the actual pixel value of each pixel in the intermediate image (i.e., the pixel value of the pixel in the intermediate image), and obtaining a preset mean value (e.g., 0.5) and a preset variance (e.g., 1). Based on the actual pixel value, preset mean value, and preset variance of each pixel, the pixel value that each pixel should have is calculated in reverse. The calculated mean value among all pixel values ​​is equal to the preset mean value, and the calculated variance of each pixel value is equal to the preset variance. This application does not limit the specific calculation method; for example, mathematical formulas or neural network models related to mean and variance can be used to calculate the pixel value that each pixel should have in reverse. Furthermore, the actual pixel value of each pixel in the intermediate image can be modified to the pixel value that the corresponding pixel should have, thereby modifying the mean value of the intermediate image to the preset mean value.

[0081] S402, obtain K reference feature vectors, and use an attention mechanism to perform feature enhancement processing on the object feature vector using each reference feature vector to obtain K enhanced feature vectors.

[0082] S403, obtain the Gaussian features of each enhanced feature vector, and obtain the three initialized feature planes.

[0083] S404 uses the acquired Gaussian features to rasterize each grid in each feature plane to obtain the feature data of each grid.

[0084] The obtained Gaussian features include: Gaussian features of K enhanced feature vectors; the Gaussian features of any enhanced feature vector include, but are not limited to: mean, variance, bias, and weight. Based on this, the specific implementation process of step S404 may include the following steps (1)-(4):

[0085] (1) Traverse each grid in each feature plane and determine the currently traversed grid as the target grid.

[0086] (2) Obtain the positional relationship between the target grid and the Gaussian distribution represented by each enhanced feature vector. Any positional relationship is used to indicate whether the target grid and the corresponding Gaussian distribution intersect.

[0087] Specifically, the computer equipment can obtain the position coordinates of the target grid; in addition, it can obtain the mean of the Gaussian distribution represented by the k-th enhanced eigenvector as the k-th Gaussian mean, and the standard deviation of the Gaussian distribution represented by the k-th enhanced eigenvector as the k-th Gaussian standard deviation, where k∈[1,K]. Further, the difference between the position coordinates of the target grid and the k-th Gaussian mean can be calculated, and the k-th Gaussian standard deviation can be amplified using a preset coefficient to obtain the target Gaussian standard deviation. This preset coefficient can be set based on empirical values; for example, if the preset coefficient is 3, the target Gaussian standard deviation in this case is three times the k-th Gaussian standard deviation. Furthermore, the calculated difference can be compared to see if it is less than the target Gaussian standard deviation. If the calculated difference is less than the target Gaussian standard deviation, the intersection relationship is taken as the positional relationship between the target mesh and the Gaussian distribution represented by the k-th enhanced eigenvector. That is, this positional relationship is used to indicate that the target mesh and the Gaussian distribution represented by the k-th enhanced eigenvector intersect. If the calculated difference is greater than or equal to the target Gaussian standard deviation, the non-intersection relationship is taken as the positional relationship between the target mesh and the Gaussian distribution represented by the k-th enhanced eigenvector. That is, this positional relationship is used to indicate that the target mesh and the Gaussian distribution represented by the k-th enhanced eigenvector do not intersect.

[0088] It should be noted that the above is merely an exemplary illustration of one specific implementation method for obtaining the positional relationship between the target grid and the Gaussian distribution represented by the k-th enhanced feature vector, and is not exhaustive. For example, in other embodiments, a positional relationship prediction model can be pre-trained based on machine learning; in this case, the computer device can input the position coordinates of the target grid and the Gaussian distribution represented by the k-th enhanced feature vector into the pre-trained positional relationship prediction model, so as to call the positional relationship prediction model to predict and output the positional relationship between the target grid and the Gaussian distribution represented by the k-th enhanced feature vector based on the input data.

[0089] (3) Based on the obtained positional relationship, select the enhanced feature vector corresponding to the Gaussian distribution that intersects with the target mesh from the K enhanced feature vectors. Specifically, the K enhanced feature vectors can be traversed. If, based on the obtained positional relationship, it is determined that the target mesh intersects with the Gaussian distribution represented by the currently traversed enhanced feature vector, then the currently traversed enhanced feature vector can be selected from the K enhanced feature vectors, and the traversal can continue until all K enhanced feature vectors have been traversed.

[0090] (4) Construct the feature data of the target grid using the Gaussian features of each selected enhancement feature vector. Specifically, the following steps can be performed:

[0091] ① Iterate through each of the selected enhancement feature vectors, taking the currently iterated enhancement feature vector as the target enhancement feature vector. Using the variance and bias from the Gaussian features of this target enhancement feature vector, construct the covariance matrix corresponding to this target enhancement feature vector. This covariance matrix is ​​a 2×2 matrix, where the two elements on the diagonal are the variance, and the two elements at the bottom left and top right corners are the biases. For example, if the variance is 0.2 and the bias is 0.3, then the covariance matrix could be...

[0092] ② Determine the inverse matrix of the covariance matrix corresponding to the target enhancement feature vector, and calculate the difference between the position coordinates of the target grid and the mean of the Gaussian feature of the target enhancement feature vector as the target difference value; it can be understood that since the position coordinates are a vector and the mean of the Gaussian feature is also a vector, the calculated target difference value is also a vector.

[0093] ③ Call the exponential function (exp) to perform feature fitting based on the inverse matrix of the covariance matrix corresponding to the target enhanced feature vector and the target difference, and obtain the feature fitting result corresponding to the target enhanced feature vector; specifically, the transpose matrix of the target difference can be multiplied by the inverse matrix of the covariance matrix corresponding to the target enhanced feature vector to obtain the multiplication result, and the multiplication result can be multiplied by the target difference to obtain the exponent in the exponential function (exp). Call the exponential function to perform calculation based on the exponent to obtain the feature fitting result corresponding to the target enhanced feature vector.

[0094] ④ After all the selected enhancement feature vectors have been traversed, the feature fitting results corresponding to the respective enhancement feature vectors are integrated according to the weights in the Gaussian features of each selected enhancement feature vector to obtain the feature data of the target mesh. Practice has proven that using steps ①-④ to rasterize the target mesh can effectively reduce the structural problem of irregular Gaussian representation, thereby reducing the difficulty of inference and improving the quality of the 3D model.

[0095] Based on the description of steps (1)-(4) above, the specific method by which the obtained Gaussian features are used to rasterize any grid in any feature plane to obtain the feature data of the corresponding grid can be summarized as the following Gaussian rasterization mixing formulas 1.1-1.2:

[0096]

[0097] In formulas 1.1-1.2 above, F represents the feature data of the mesh, ∏ represents the product, and x represents the position coordinates of the mesh; μ i This represents the mean of the Gaussian features in the i-th enhanced feature vector. Let f represent the covariance matrix corresponding to the i-th enhanced eigenvector.i α represents the weight in the Gaussian feature of the i-th enhanced feature vector; i t represents the feature fitting result corresponding to the i-th enhanced feature vector; i It is a binary function, t, when the Gaussian distribution represented by the i-th enhancement eigenvector intersects with the target mesh. i The value of t is 1, when the Gaussian distribution represented by the i-th enhancement feature vector does not intersect with the target mesh. i The value of is 0.

[0098] S405, fill the feature data of each grid into the corresponding grid to obtain the Gaussian three planes of the three-dimensional object.

[0099] S406: Reconstruct the three-dimensional object using the Gaussian three-plane method to obtain the three-dimensional model of the object.

[0100] In practical implementation, the computer device can traverse all the three-dimensional spatial points used to represent the three-dimensional object and determine the p-th three-dimensional spatial point being traversed, where p is a positive integer less than or equal to the total number of three-dimensional spatial points. Based on the position coordinates of the p-th three-dimensional spatial point, the p-th three-dimensional spatial point is mapped to each feature plane in the Gaussian three-plane, obtaining the mapped points of the p-th three-dimensional spatial point on each feature plane, such as... Figure 5a As shown. Based on the mapping point on each feature plane, interpolation is performed on the feature data within at least one grid in the corresponding feature plane to obtain the feature of the p-th 3D spatial point in the corresponding feature plane; specifically, for any mapping point on a feature plane, Q grids (Q is a positive integer) adjacent to the mapping point can be selected from the feature plane based on a preset interpolation algorithm, and interpolation is performed on the feature data within the Q grids to obtain the feature of the p-th 3D spatial point in the feature plane.

[0101] Furthermore, the features of the p-th 3D spatial point in each feature plane can be blended to obtain the blended features of the p-th 3D spatial point. Specifically, the features of the p-th 3D spatial point in each feature plane can be blended directly in 3D space to obtain the blended features of the p-th 3D spatial point; or see [link to relevant documentation]. Figure 5b As shown, the encoder can be called to encode the position coordinates of the p-th 3D spatial point to obtain the position features of the p-th 3D spatial point. Then, the features and position features of the p-th 3D spatial point in each feature plane are mixed to obtain the mixed features of the p-th 3D spatial point. This mixed feature can contain the position information of the p-th 3D spatial point. This position information can improve the model quality of the subsequently constructed 3D model, avoid the problem of overly smooth display between adjacent 3D spatial points, and avoid the problem of loss of details near the 3D spatial point.

[0102] The process continues until the blending features of each 3D spatial point are obtained. These blending features are then used to construct a 3D model of the 3D object. Specifically, the computer device can decode the blending features of each 3D spatial point to obtain the display style parameters for each point (i.e., the decoding result of the blending features is used as the display style parameters). These display style parameters may include at least one of the following: color parameters and opacity parameters (or transparency parameters). It is evident that the display style parameters indicate at least one of the following: color and transparency. Furthermore, the display style parameters of each 3D spatial point can be summarized to obtain the 3D model of the 3D object. That is, the 3D model may include the display style parameters of each 3D spatial point. For example, taking M 3D spatial points as an example (M being a positive integer), a schematic diagram of summarizing the display style parameters of M 3D spatial points to obtain the 3D model can be found in [reference needed]. Figure 5c As shown.

[0103] This application embodiment can extract features from a two-dimensional view of a three-dimensional object to obtain the object feature vector of the three-dimensional object, and obtain K reference feature vectors containing Gaussian features. Then, through an attention mechanism, each reference feature vector is used to adaptively enhance the object feature vector of the three-dimensional object, resulting in K enhanced feature vectors. This enhances the feature vectors, making them more accurate in representing the three-dimensional object compared to the object feature vectors. Furthermore, a Gaussian three-plane can be constructed using the Gaussian features of the K enhanced feature vectors, and the three-dimensional object can be reconstructed based on the Gaussian three-plane to obtain a three-dimensional model of the three-dimensional object. This improves the object representation capability of the three-dimensional model, thereby enhancing its quality. It is understood that since the Gaussian three-plane is constructed using Gaussian features, it has fewer parameters. Therefore, using this Gaussian three-plane to reconstruct the three-dimensional model can reduce the parameters required for the three-dimensional model representation, thereby reducing the space occupied by the three-dimensional model and increasing its data size.

[0104] Based on the above Figure 2 and Figure 4 The description of the method embodiments shown in this application proposes a 3D reconstruction technique based on Gaussian three planes. This technique can be configured in a 3D reconstruction model to greatly enhance the generated quality of the 3D reconstruction model, improve the object representation capability and resolution of the 3D model generated through the 3D reconstruction model, thereby improving the model quality of the 3D model. See also... Figure 6a As shown, the three-dimensional reconstruction technology based on Gaussian three planes proposed in this application embodiment may include the following modules:

[0105] (1) Image preprocessing module: This module is mainly used to preprocess and extract features from images of a 3D object from various viewpoints, thereby integrating the image feature vectors corresponding to each viewpoint into a single object feature vector. The process of preprocessing and extracting features from a single viewpoint image can be found in [link to documentation]. Figure 6b As shown: Modify the image size and mean of the viewpoint image, perform multi-layer convolution processing on the modified viewpoint image, and use the feature extraction module to extract features from the result of the multi-layer convolution processing to obtain the image feature vector.

[0106] (2) Attention Feature Enhancement Module: This module primarily uses a multi-layer attention mechanism, employing K reference feature vectors from the reference feature vector set to enhance the object feature vectors output by the image preprocessing module, resulting in K enhanced feature vectors. See also... Figure 6c As shown, the process of using a multi-layer attention mechanism and a reference feature vector to enhance the object feature vector output by the image preprocessing module to obtain an enhanced feature vector is roughly as follows: The object feature vector is input into the first attention mechanism, which uses the reference feature vector to enhance the input vector, resulting in an intermediate latent vector. This intermediate latent vector is then input into the second attention mechanism, which uses the reference feature vector to enhance the input vector, resulting in a new intermediate latent vector. This new intermediate latent vector is then input into the third attention mechanism, which uses the reference feature vector to enhance the input vector, and so on. Finally, the intermediate latent vector obtained through the last attention mechanism is input into the fully connected network for processing, resulting in the enhanced feature vector. The specific process of using a reference feature vector to enhance the input vector through any attention mechanism can be found in the descriptions in the preceding embodiments, and will not be repeated here.

[0107] (3) Gaussian feature separation module. This module mainly performs Gaussian feature separation on each enhanced feature vector output by the attention feature enhancement module to extract the Gaussian features of each enhanced feature vector. The Gaussian features of any enhanced feature vector include: mean, variance, bias and weight.

[0108] (4) Gaussian feature rasterization module. This module is mainly used to rasterize each grid in the three initial feature planes according to the Gaussian features of the K enhanced feature vectors output by the Gaussian feature separation module, using the formulas 1.1-1.2 mentioned above, to obtain the feature data of each grid, and fill the feature data of each grid into the corresponding grid to obtain the Gaussian three planes of the three-dimensional object.

[0109] (5) Three-plane module. This module is mainly used to perform interpolation processing in each feature plane contained in the Gaussian three-plane according to the position coordinates of each three-dimensional space point of the three-dimensional object, so as to obtain the feature of each three-dimensional space point in the corresponding feature plane, and then perform feature mixing in three-dimensional space to obtain the mixed feature of each three-dimensional space point. The specific implementation process can be referred to the relevant description of the aforementioned method embodiment, and will not be repeated here.

[0110] (6) Gaussian feature decoding module. This module is mainly used to decode the mixed features of each three-dimensional space point output by the three-plane module to obtain the display style parameters (such as color parameters and opacity parameters) of each three-dimensional space point, and then use the display style parameters of each three-dimensional space point to construct a three-dimensional model.

[0111] (7) 3D Information Extraction Module: This module is mainly used to obtain rendering parameters (i.e., parameters required for image rendering, such as camera pose), and to perform image rendering based on the rendering parameters and the display style parameters of each 3D spatial point in the 3D module output by the Gaussian feature decoding module, in order to generate a 3D image of the 3D object, such as... Figure 6d As shown.

[0112] Based on the above description, the three-dimensional reconstruction technology based on Gaussian three planes proposed in this application embodiment can include at least the following beneficial effects:

[0113] (1) By proposing a novel Gaussian Triplane 3D Representation (GTR) technique, it is possible to represent the Gaussian Triplane using very few parameters (such as mean, variance, deflection and weight), thereby realizing the representation of a three-dimensional model using very few parameters.

[0114] (2) By proposing a novel Gaussian three-plane generation technique, it is possible not only to greatly improve the partial variation rate of the three planes under the condition of fixed parameter quantity, and generate ultra-high resolution Gaussian three planes, thereby generating ultra-high resolution three-dimensional models based on the Gaussian three planes, but also to reduce the structural problem of Gaussian representation irregularity by using Gaussian rasterization and feature mixing, thereby reducing the reasoning difficulty and improving the quality of the three-dimensional model.

[0115] (3) It can greatly reduce the space occupied by the three-dimensional model, increase the data scale of the three-dimensional model, and obtain a better three-dimensional model.

[0116] In summary, the 3D reconstruction technology based on Gaussian three planes proposed in this application can increase the resolution of the Gaussian three planes, reduce the irregularity of the Gaussian representation, and has good generalization ability for irregular 3D models.

[0117] Based on the descriptions of the above method embodiments, this application also discloses a three-dimensional model generation apparatus; the three-dimensional model generation apparatus may be a computer program (including one or more instructions) running on a computer device, and the three-dimensional model generation apparatus may execute each step in any of the above method flows. Please refer to... Figure 7 The three-dimensional model generation device can operate the following units:

[0118] The acquisition unit 701 is used to acquire a two-dimensional view of a three-dimensional object and extract features from the two-dimensional view to obtain the object feature vector of the three-dimensional object.

[0119] The acquisition unit 701 is further configured to acquire K reference feature vectors, wherein the reference feature vectors are feature vectors containing Gaussian features, and Gaussian features are features of a Gaussian distribution, and K is a positive integer;

[0120] Processing unit 702 is used to perform feature enhancement processing on the object feature vector by using each reference feature vector through an attention mechanism to obtain K enhanced feature vectors, where each enhanced feature vector represents a Gaussian distribution;

[0121] The processing unit 702 is further configured to obtain the Gaussian features of each enhanced feature vector, wherein the Gaussian features of any enhanced feature vector refer to the features of the Gaussian distribution represented by the corresponding enhanced feature vector.

[0122] The processing unit 702 is further configured to construct a Gaussian three-plane of the three-dimensional object using the acquired Gaussian features, and to perform three-dimensional reconstruction of the three-dimensional object based on the Gaussian three-plane to obtain a three-dimensional model of the three-dimensional object.

[0123] In one specific implementation, the object feature vector includes multiple elements; correspondingly, when the processing unit 702 performs feature enhancement processing on the object feature vector using each reference feature vector through an attention mechanism to obtain K enhanced feature vectors, it can specifically be used to:

[0124] Calculate the similarity between each element in the object's feature vector and the k-th reference feature vector, where k ∈ [1, K];

[0125] Based on the principle that similarity is proportional to attention weight, an attention weight is assigned to each element according to the similarity between each element and the k-th reference feature vector.

[0126] Based on the attention weights of each element, the elements are weighted and summed to obtain the k-th enhanced feature vector.

[0127] In another specific embodiment, when processing unit 702 is used to obtain the Gaussian features of each enhanced feature vector, it may specifically be used to:

[0128] For the k-th enhanced feature vector, obtain the mean, variance, skewness and weight of the Gaussian distribution represented by the k-th enhanced feature vector, k∈[1,K];

[0129] The obtained mean, variance, bias, and weight are all used as Gaussian features of the k-th enhanced feature vector.

[0130] In another specific embodiment, when the processing unit 702 is used to construct the Gaussian three planes of the three-dimensional object using the acquired Gaussian features, it may specifically be used to:

[0131] Obtain three initial feature planes, each of which includes at least one mesh;

[0132] The obtained Gaussian features are used to rasterize each grid in each feature plane to obtain the feature data of each grid.

[0133] The feature data of each grid is filled into the corresponding grid to obtain the Gaussian three planes of the three-dimensional object.

[0134] In another specific implementation, the obtained Gaussian features include: Gaussian features of the K enhanced feature vectors;

[0135] Accordingly, when processing unit 702 performs rasterization processing on each grid in each feature plane using the acquired Gaussian features to obtain feature data for each grid, it can specifically be used to:

[0136] Traverse each grid in each feature plane and determine the currently traversed grid as the target grid;

[0137] Obtain the positional relationship between the target grid and the Gaussian distribution represented by each enhanced feature vector, whereby any positional relationship is used to indicate whether the target grid intersects with the corresponding Gaussian distribution;

[0138] Based on the obtained positional relationship, the enhanced feature vector corresponding to the Gaussian distribution that intersects with the target grid is selected from the K enhanced feature vectors;

[0139] The feature data of the target grid is constructed using the Gaussian features of each selected enhanced feature vector.

[0140] In another specific embodiment, when processing unit 702 is used to obtain the positional relationship between the target mesh and the Gaussian distribution represented by each enhanced feature vector, it may specifically be used to:

[0141] Obtain the position coordinates of the target grid;

[0142] Obtain the mean of the Gaussian distribution represented by the k-th enhanced feature vector as the k-th Gaussian mean; and obtain the standard deviation of the Gaussian distribution represented by the k-th enhanced feature vector as the k-th Gaussian standard deviation, k∈[1,K];

[0143] Calculate the difference between the position coordinates of the target grid and the k-th Gaussian mean, and amplify the k-th Gaussian standard deviation using a preset coefficient to obtain the target Gaussian standard deviation;

[0144] If the calculated difference is less than the target Gaussian standard deviation, the intersection relationship is taken as the positional relationship between the target grid and the Gaussian distribution represented by the k-th enhanced feature vector;

[0145] If the calculated difference is greater than or equal to the target Gaussian standard deviation, then the disjoint relationship is taken as the positional relationship between the target grid and the Gaussian distribution represented by the k-th enhanced feature vector.

[0146] In another specific embodiment, the Gaussian triplane includes three feature planes, each feature plane including at least one grid and each grid being filled with feature data;

[0147] Accordingly, when processing unit 702 performs three-dimensional reconstruction of the three-dimensional object based on the Gaussian three planes to obtain a three-dimensional model of the three-dimensional object, it can specifically be used to:

[0148] Traverse each three-dimensional space point used to represent the three-dimensional object, and determine the p-th three-dimensional space point currently being traversed, where p is a positive integer less than or equal to the total number of three-dimensional space points;

[0149] Based on the position coordinates of the p-th three-dimensional spatial point, the p-th three-dimensional spatial point is mapped to each feature plane in the Gaussian three-plane, so as to obtain the mapping point of the p-th three-dimensional spatial point on each feature plane;

[0150] Based on the mapping points on each feature plane, interpolation is performed on the feature data in at least one grid in the corresponding feature plane to obtain the feature of the p-th three-dimensional space point in the corresponding feature plane.

[0151] The features of the p-th three-dimensional spatial point in each feature plane are blended to obtain the blended features of the p-th three-dimensional spatial point;

[0152] Continue traversing until the blended features of each three-dimensional spatial point are obtained, and use the blended features of each three-dimensional spatial point to construct a three-dimensional model of the three-dimensional object.

[0153] In another specific embodiment, when processing unit 702 mixes the features of the p-th three-dimensional spatial point in each feature plane to obtain the mixed features of the p-th three-dimensional spatial point, it can specifically be used to:

[0154] The encoder is invoked to encode the position coordinates of the p-th three-dimensional spatial point to obtain the position features of the p-th three-dimensional spatial point;

[0155] The features of the p-th three-dimensional spatial point in each feature plane and the positional features are mixed to obtain the mixed features of the p-th three-dimensional spatial point.

[0156] In another specific embodiment, when the processing unit 702 constructs a three-dimensional model of the three-dimensional object using the mixed features of the various three-dimensional spatial points, it may specifically be used to:

[0157] The blending features of each three-dimensional spatial point are decoded to obtain the display style parameters of each three-dimensional spatial point. The display style parameters are used to indicate at least one of the following: color and transparency.

[0158] The display style parameters of each three-dimensional spatial point are summarized to obtain the three-dimensional model of the three-dimensional object.

[0159] In another specific implementation, the two-dimensional view includes N viewpoint images, with different viewpoint images corresponding to different camera viewpoints, where N is an integer greater than 1;

[0160] Accordingly, when the acquisition unit 701 performs feature extraction on the two-dimensional view to obtain the object feature vector of the three-dimensional object, it can be specifically used for:

[0161] Feature extraction is performed on each viewpoint image in the two-dimensional view to obtain N image feature vectors, with one image feature vector corresponding to one viewpoint image;

[0162] The N image feature vectors are integrated to obtain the object feature vector of the three-dimensional object.

[0163] In another specific embodiment, when the acquisition unit 701 is used to extract features from each viewpoint image in the two-dimensional view to obtain N image feature vectors, it can be specifically used for:

[0164] For the nth viewpoint image in the two-dimensional view, a random image size is generated; where n∈[1,N];

[0165] The nth viewpoint image is scaled down according to the randomly generated image size to obtain the intermediate image;

[0166] Feature extraction is performed on the intermediate image to obtain the nth image feature vector.

[0167] In another specific embodiment, when the acquisition unit 701 is used to extract features from the intermediate image to obtain the nth image feature vector, it can be specifically used for:

[0168] By adjusting at least one pixel value in the intermediate image, the image mean of the intermediate image is modified to a preset mean, thus obtaining a modified intermediate image; wherein, the image mean of the intermediate image refers to the average value among all pixel values ​​in the intermediate image;

[0169] Feature extraction is performed on the modified intermediate image to obtain the nth image feature vector.

[0170] According to another embodiment of this application, Figure 7 The various units in the illustrated 3D model generation apparatus can be individually or entirely merged into one or more other units, or some of the units can be further divided into multiple functionally smaller units. This achieves the same operation without affecting the technical effects of the embodiments of this application. The above-mentioned units are based on logical function division. In practical applications, the function of one unit can also be implemented by multiple units, or the function of multiple units can be implemented by one unit. In other embodiments of this application, the 3D model generation apparatus may also include other units. In practical applications, these functions can also be implemented with the assistance of other units, and can be implemented collaboratively by multiple units.

[0171] According to another embodiment of this application, a computer program (including one or more instructions) capable of performing the steps involved in the various methods described above can be run on a general-purpose computing device, such as a computer, which includes processing elements and storage elements such as a central processing unit (CPU), random access memory (RAM), and read-only memory (ROM), to construct a system such as... Figure 7 The three-dimensional model generation apparatus shown herein, and the three-dimensional model generation method for implementing the embodiments of this application, are described. The computer program may be recorded on, for example, a computer-readable storage medium, loaded onto the aforementioned computing device via the computer-readable storage medium, and run therein.

[0172] It is worth noting that, in the embodiments of this application, the terms "module" or "unit" refer to a computer program or part of a computer program with a predetermined function, which works together with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can contain a portion of the overall module or unit's functionality.

[0173] This application embodiment can extract features from a two-dimensional view of a three-dimensional object to obtain the object feature vector of the three-dimensional object, and obtain K reference feature vectors containing Gaussian features. Then, through an attention mechanism, each reference feature vector is used to adaptively enhance the object feature vector of the three-dimensional object, resulting in K enhanced feature vectors. This enhances the feature vectors, making them more accurate in representing the three-dimensional object compared to the object feature vectors. Furthermore, a Gaussian three-plane can be constructed using the Gaussian features of the K enhanced feature vectors, and the three-dimensional object can be reconstructed based on the Gaussian three-plane to obtain a three-dimensional model of the three-dimensional object. This improves the object representation capability of the three-dimensional model, thereby enhancing its quality. It is understood that since the Gaussian three-plane is constructed using Gaussian features, it has fewer parameters. Therefore, using this Gaussian three-plane to reconstruct the three-dimensional model can reduce the parameters required for the three-dimensional model representation, thereby reducing the space occupied by the three-dimensional model and increasing its data size.

[0174] Based on the description of the above method and apparatus embodiments, this application also provides a computer device. Please refer to... Figure 8 The computer device includes at least a processor 801, an input interface 802, an output interface 803, and a computer storage medium 804. The processor 801, input interface 802, output interface 803, and computer storage medium 804 within the computer device can be connected via a bus or other means. The computer storage medium 804 can be stored in the computer device's memory. The computer storage medium 804 is used to store a computer program, which includes one or more instructions. The processor 801 is used to execute one or more instructions from the computer program stored in the computer storage medium 804. The processor 801 (or CPU (Central Processing Unit)) is the computing and control core of the computer device, adapted to implement one or more instructions, specifically adapted to load and execute one or more instructions to achieve a corresponding method flow or function.

[0175] In one embodiment, the processor 801 described in this application embodiment can be used to perform a series of three-dimensional reconstruction processes on a three-dimensional object, specifically including: acquiring a two-dimensional view of the three-dimensional object, and extracting features from the two-dimensional view to obtain an object feature vector of the three-dimensional object; acquiring K reference feature vectors, wherein the reference feature vectors are feature vectors containing Gaussian features, and Gaussian features refer to features of a Gaussian distribution, where K is a positive integer; using an attention mechanism, performing feature enhancement processing on the object feature vector using each reference feature vector to obtain K enhanced feature vectors, where each enhanced feature vector represents a Gaussian distribution; acquiring the Gaussian features of each enhanced feature vector, wherein the Gaussian features of any enhanced feature vector refer to the features of the Gaussian distribution represented by the corresponding enhanced feature vector; constructing a Gaussian three-plane of the three-dimensional object using the acquired Gaussian features, and performing three-dimensional reconstruction of the three-dimensional object based on the Gaussian three-plane to obtain a three-dimensional model of the three-dimensional object, etc.

[0176] This application embodiment also provides a computer storage medium (Memory), which is a memory device in a computer device used to store computer programs and data. It is understood that the computer storage medium here can include both the built-in storage medium in the computer device and extended storage media supported by the computer device. The computer storage medium provides storage space that stores the operating system of the computer device. Furthermore, the storage space also stores a computer program, which includes one or more instructions suitable for loading and execution by the processor 801. These instructions can be one or more program codes. It should be noted that the computer storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device; optionally, it can also be at least one computer storage medium located remotely from the aforementioned processor.

[0177] In one embodiment, a processor may load and execute one or more instructions stored in a computer storage medium to implement the corresponding steps in any of the above method embodiments; specifically, one or more instructions in the computer storage medium may be loaded and executed by the processor in the following steps:

[0178] Obtain a two-dimensional view of a three-dimensional object, and extract features from the two-dimensional view to obtain the object feature vector of the three-dimensional object;

[0179] Obtain K reference feature vectors, where each reference feature vector is a feature vector containing Gaussian features, where Gaussian features refer to features distributed in a Gaussian distribution, and K is a positive integer.

[0180] Through an attention mechanism, the object feature vector is enhanced by each reference feature vector to obtain K enhanced feature vectors, and each enhanced feature vector represents a Gaussian distribution.

[0181] Obtain the Gaussian features of each enhanced feature vector. The Gaussian features of any enhanced feature vector refer to the characteristics of the Gaussian distribution represented by the corresponding enhanced feature vector.

[0182] The obtained Gaussian features are used to construct the Gaussian three planes of the three-dimensional object, and the three-dimensional object is reconstructed according to the Gaussian three planes to obtain the three-dimensional model of the three-dimensional object.

[0183] In one specific implementation, the object feature vector includes multiple elements; correspondingly, when the object feature vector is enhanced using each reference feature vector through an attention mechanism to obtain K enhanced feature vectors, the one or more instructions can be loaded and executed by the processor:

[0184] Calculate the similarity between each element in the object's feature vector and the k-th reference feature vector, where k ∈ [1, K];

[0185] Based on the principle that similarity is proportional to attention weight, an attention weight is assigned to each element according to the similarity between each element and the k-th reference feature vector.

[0186] Based on the attention weights of each element, the elements are weighted and summed to obtain the k-th enhanced feature vector.

[0187] In another specific implementation, when acquiring the Gaussian features of each enhanced feature vector, the one or more instructions can be loaded and executed by the processor:

[0188] For the k-th enhanced feature vector, obtain the mean, variance, skewness and weight of the Gaussian distribution represented by the k-th enhanced feature vector, k∈[1,K];

[0189] The obtained mean, variance, bias, and weight are all used as Gaussian features of the k-th enhanced feature vector.

[0190] In another specific implementation, when constructing the Gaussian three planes of the 3D object using the acquired Gaussian features, the one or more instructions can be loaded and executed by the processor:

[0191] Obtain three initial feature planes, each of which includes at least one mesh;

[0192] The obtained Gaussian features are used to rasterize each grid in each feature plane to obtain the feature data of each grid.

[0193] The feature data of each grid is filled into the corresponding grid to obtain the Gaussian three planes of the three-dimensional object.

[0194] In another specific implementation, the obtained Gaussian features include: Gaussian features of the K enhanced feature vectors;

[0195] Accordingly, when using the acquired Gaussian features to rasterize each grid in each feature plane to obtain the feature data of each grid, the one or more instructions can be loaded and executed by the processor:

[0196] Traverse each grid in each feature plane and determine the currently traversed grid as the target grid;

[0197] Obtain the positional relationship between the target grid and the Gaussian distribution represented by each enhanced feature vector, whereby any positional relationship is used to indicate whether the target grid intersects with the corresponding Gaussian distribution;

[0198] Based on the obtained positional relationship, the enhanced feature vector corresponding to the Gaussian distribution that intersects with the target grid is selected from the K enhanced feature vectors;

[0199] The feature data of the target grid is constructed using the Gaussian features of each selected enhanced feature vector.

[0200] In another specific implementation, when obtaining the positional relationship between the target mesh and the Gaussian distribution represented by each enhanced feature vector, the one or more instructions can be loaded and executed by the processor:

[0201] Obtain the position coordinates of the target grid;

[0202] Obtain the mean of the Gaussian distribution represented by the k-th enhanced feature vector as the k-th Gaussian mean; and obtain the standard deviation of the Gaussian distribution represented by the k-th enhanced feature vector as the k-th Gaussian standard deviation, k∈[1,K];

[0203] Calculate the difference between the position coordinates of the target grid and the k-th Gaussian mean, and amplify the k-th Gaussian standard deviation using a preset coefficient to obtain the target Gaussian standard deviation;

[0204] If the calculated difference is less than the target Gaussian standard deviation, the intersection relationship is taken as the positional relationship between the target grid and the Gaussian distribution represented by the k-th enhanced feature vector;

[0205] If the calculated difference is greater than or equal to the target Gaussian standard deviation, then the disjoint relationship is taken as the positional relationship between the target grid and the Gaussian distribution represented by the k-th enhanced feature vector.

[0206] In another specific embodiment, the Gaussian triplane includes three feature planes, each feature plane including at least one grid and each grid being filled with feature data;

[0207] Accordingly, when performing 3D reconstruction of the 3D object based on the Gaussian three planes to obtain the 3D model of the 3D object, the one or more instructions can be loaded and executed by the processor:

[0208] Traverse each three-dimensional space point used to represent the three-dimensional object, and determine the p-th three-dimensional space point currently being traversed, where p is a positive integer less than or equal to the total number of three-dimensional space points;

[0209] Based on the position coordinates of the p-th three-dimensional spatial point, the p-th three-dimensional spatial point is mapped to each feature plane in the Gaussian three-plane, so as to obtain the mapping point of the p-th three-dimensional spatial point on each feature plane;

[0210] Based on the mapping points on each feature plane, interpolation is performed on the feature data in at least one grid in the corresponding feature plane to obtain the feature of the p-th three-dimensional space point in the corresponding feature plane.

[0211] The features of the p-th three-dimensional spatial point in each feature plane are blended to obtain the blended features of the p-th three-dimensional spatial point;

[0212] Continue traversing until the blended features of each three-dimensional spatial point are obtained, and use the blended features of each three-dimensional spatial point to construct a three-dimensional model of the three-dimensional object.

[0213] In another specific implementation, when the features of the p-th three-dimensional spatial point in each feature plane are blended to obtain the blended features of the p-th three-dimensional spatial point, the one or more instructions can be loaded and executed by the processor:

[0214] The encoder is invoked to encode the position coordinates of the p-th three-dimensional spatial point to obtain the position features of the p-th three-dimensional spatial point;

[0215] The features of the p-th three-dimensional spatial point in each feature plane and the positional features are mixed to obtain the mixed features of the p-th three-dimensional spatial point.

[0216] In another specific implementation, when constructing the three-dimensional model of the three-dimensional object using the mixed features of the various three-dimensional spatial points, the one or more instructions can be loaded and executed by the processor:

[0217] The blending features of each three-dimensional spatial point are decoded to obtain the display style parameters of each three-dimensional spatial point. The display style parameters are used to indicate at least one of the following: color and transparency.

[0218] The display style parameters of each three-dimensional spatial point are summarized to obtain the three-dimensional model of the three-dimensional object.

[0219] In another specific implementation, the two-dimensional view includes N viewpoint images, with different viewpoint images corresponding to different camera viewpoints, where N is an integer greater than 1;

[0220] Accordingly, when performing feature extraction on the two-dimensional view to obtain the object feature vector of the three-dimensional object, the one or more instructions can be loaded and executed by the processor:

[0221] Feature extraction is performed on each viewpoint image in the two-dimensional view to obtain N image feature vectors, with one image feature vector corresponding to one viewpoint image;

[0222] The N image feature vectors are integrated to obtain the object feature vector of the three-dimensional object.

[0223] In another specific embodiment, when the acquisition unit 701 is used to extract features from each viewpoint image in the two-dimensional view to obtain N image feature vectors, it can be specifically used for:

[0224] For the nth viewpoint image in the two-dimensional view, a random image size is generated; where n∈[1,N];

[0225] The nth viewpoint image is scaled down according to the randomly generated image size to obtain the intermediate image;

[0226] Feature extraction is performed on the intermediate image to obtain the nth image feature vector.

[0227] In another specific implementation, when performing feature extraction on the intermediate image to obtain the nth image feature vector, the one or more instructions can be loaded and executed by the processor:

[0228] By adjusting at least one pixel value in the intermediate image, the image mean of the intermediate image is modified to a preset mean, thus obtaining a modified intermediate image; wherein, the image mean of the intermediate image refers to the average value among all pixel values ​​in the intermediate image;

[0229] Feature extraction is performed on the modified intermediate image to obtain the nth image feature vector.

[0230] This application embodiment can extract features from a two-dimensional view of a three-dimensional object to obtain the object feature vector of the three-dimensional object, and obtain K reference feature vectors containing Gaussian features. Then, through an attention mechanism, each reference feature vector is used to adaptively enhance the object feature vector of the three-dimensional object, resulting in K enhanced feature vectors. This enhances the feature vectors, making them more accurate in representing the three-dimensional object compared to the object feature vectors. Furthermore, a Gaussian three-plane can be constructed using the Gaussian features of the K enhanced feature vectors, and the three-dimensional object can be reconstructed based on the Gaussian three-plane to obtain a three-dimensional model of the three-dimensional object. This improves the object representation capability of the three-dimensional model, thereby enhancing its quality. It is understood that since the Gaussian three-plane is constructed using Gaussian features, it has fewer parameters. Therefore, using this Gaussian three-plane to reconstruct the three-dimensional model can reduce the parameters required for the three-dimensional model representation, thereby reducing the space occupied by the three-dimensional model and increasing its data size.

[0231] It should be noted that, according to one aspect of this application, a computer program product or computer program is also provided, comprising one or more instructions stored in a computer storage medium. A processor of a computer device reads one or more instructions from the computer storage medium and executes the one or more instructions, causing the computer device to perform the methods provided in various optional embodiments of the above-described methods. It should be understood that the above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, equivalent variations made according to the claims of this application are still within the scope of this application.

Claims

1. A method for generating a three-dimensional model, characterized in that, include: Obtain a two-dimensional view of a three-dimensional object, and extract features from the two-dimensional view to obtain the object feature vector of the three-dimensional object; Obtain K reference feature vectors, where each reference feature vector is a feature vector containing Gaussian features, where Gaussian features refer to features distributed in a Gaussian distribution, and K is a positive integer. Through an attention mechanism, the object feature vector is enhanced by each reference feature vector to obtain K enhanced feature vectors, and each enhanced feature vector represents a Gaussian distribution. Obtain the Gaussian features of each enhanced feature vector. The Gaussian features of any enhanced feature vector refer to the characteristics of the Gaussian distribution represented by the corresponding enhanced feature vector. The obtained Gaussian features are used to construct the Gaussian three planes of the three-dimensional object, and the three-dimensional object is reconstructed according to the Gaussian three planes to obtain the three-dimensional model of the three-dimensional object.

2. The method as described in claim 1, characterized in that, The object feature vector includes multiple elements; the attention mechanism is used to perform feature enhancement processing on the object feature vector using each reference feature vector to obtain K enhanced feature vectors, including: Calculate the similarity between each element in the object's feature vector and the k-th reference feature vector, where k ∈ [1, K]; Based on the principle that similarity is proportional to attention weight, an attention weight is assigned to each element according to the similarity between each element and the k-th reference feature vector. Based on the attention weights of each element, the elements are weighted and summed to obtain the k-th enhanced feature vector.

3. The method as described in claim 1, characterized in that, The step of obtaining the Gaussian features of each enhanced feature vector includes: For the k-th enhanced feature vector, obtain the mean, variance, skewness and weight of the Gaussian distribution represented by the k-th enhanced feature vector, k∈[1,K]; The obtained mean, variance, bias, and weight are all used as Gaussian features of the k-th enhanced feature vector.

4. The method according to any one of claims 1-3, characterized in that, The step of constructing the Gaussian three planes of the 3D object using the acquired Gaussian features includes: Obtain three initial feature planes, each of which includes at least one mesh; The obtained Gaussian features are used to rasterize each grid in each feature plane to obtain the feature data of each grid. The feature data of each grid is filled into the corresponding grid to obtain the Gaussian three planes of the three-dimensional object.

5. The method as described in claim 4, characterized in that, The obtained Gaussian features include: Gaussian features of the K enhanced feature vectors; The method involves using the acquired Gaussian features to rasterize each grid in each feature plane, thereby obtaining feature data for each grid, including: Traverse each grid in each feature plane and determine the currently traversed grid as the target grid; Obtain the positional relationship between the target grid and the Gaussian distribution represented by each enhanced feature vector, whereby any positional relationship is used to indicate whether the target grid intersects with the corresponding Gaussian distribution; Based on the obtained positional relationship, the enhanced feature vector corresponding to the Gaussian distribution that intersects with the target grid is selected from the K enhanced feature vectors; The feature data of the target grid is constructed using the Gaussian features of each selected enhanced feature vector.

6. The method as described in claim 5, characterized in that, The step of obtaining the positional relationship between the target grid and the Gaussian distribution represented by each enhanced feature vector includes: Obtain the position coordinates of the target grid; Obtain the mean of the Gaussian distribution represented by the k-th enhanced feature vector as the k-th Gaussian mean; and obtain the standard deviation of the Gaussian distribution represented by the k-th enhanced feature vector as the k-th Gaussian standard deviation, k∈[1,K]; Calculate the difference between the position coordinates of the target grid and the k-th Gaussian mean, and amplify the k-th Gaussian standard deviation using a preset coefficient to obtain the target Gaussian standard deviation; If the calculated difference is less than the target Gaussian standard deviation, the intersection relationship is taken as the positional relationship between the target grid and the Gaussian distribution represented by the k-th enhanced feature vector; If the calculated difference is greater than or equal to the target Gaussian standard deviation, then the disjoint relationship is taken as the positional relationship between the target grid and the Gaussian distribution represented by the k-th enhanced feature vector.

7. The method as described in claim 1, characterized in that, The Gaussian triplane includes three feature planes, each feature plane includes at least one grid and each grid is filled with feature data; The step of reconstructing the three-dimensional object based on the Gaussian three planes to obtain a three-dimensional model of the three-dimensional object includes: Traverse each three-dimensional space point used to represent the three-dimensional object, and determine the p-th three-dimensional space point currently being traversed, where p is a positive integer less than or equal to the total number of three-dimensional space points; Based on the position coordinates of the p-th three-dimensional spatial point, the p-th three-dimensional spatial point is mapped to each feature plane in the Gaussian three-plane, so as to obtain the mapping point of the p-th three-dimensional spatial point on each feature plane; Based on the mapping points on each feature plane, interpolation is performed on the feature data in at least one grid in the corresponding feature plane to obtain the feature of the p-th three-dimensional space point in the corresponding feature plane. The features of the p-th three-dimensional spatial point in each feature plane are blended to obtain the blended features of the p-th three-dimensional spatial point; Continue traversing until the blended features of each three-dimensional spatial point are obtained, and use the blended features of each three-dimensional spatial point to construct a three-dimensional model of the three-dimensional object.

8. The method as described in claim 7, characterized in that, The process of blending the features of the p-th three-dimensional spatial point in each feature plane to obtain the blended features of the p-th three-dimensional spatial point includes: The encoder is invoked to encode the position coordinates of the p-th three-dimensional spatial point to obtain the position features of the p-th three-dimensional spatial point; The features of the p-th three-dimensional spatial point in each feature plane and the positional features are mixed to obtain the mixed features of the p-th three-dimensional spatial point.

9. The method as described in claim 7, characterized in that, The process of constructing a 3D model of the 3D object using the mixed features of the various 3D spatial points includes: The blending features of each three-dimensional spatial point are decoded to obtain the display style parameters of each three-dimensional spatial point. The display style parameters are used to indicate at least one of the following: color and transparency. The display style parameters of each three-dimensional spatial point are summarized to obtain the three-dimensional model of the three-dimensional object.

10. The method as described in claim 1, characterized in that, The two-dimensional view includes N viewpoint images, with different viewpoint images corresponding to different camera viewpoints, where N is an integer greater than 1; The step of extracting features from the two-dimensional view to obtain the object feature vector of the three-dimensional object includes: Feature extraction is performed on each viewpoint image in the two-dimensional view to obtain N image feature vectors, with one image feature vector corresponding to one viewpoint image; The N image feature vectors are integrated to obtain the object feature vector of the three-dimensional object.

11. The method as described in claim 10, characterized in that, The step involves extracting features from each viewpoint image in the two-dimensional view to obtain N image feature vectors, including: For the nth viewpoint image in the two-dimensional view, a random image size is generated; where n∈[1,N]; The nth viewpoint image is scaled down according to the randomly generated image size to obtain the intermediate image; Feature extraction is performed on the intermediate image to obtain the nth image feature vector.

12. The method as described in claim 11, characterized in that, The step of extracting features from the intermediate image to obtain the nth image feature vector includes: By adjusting at least one pixel value in the intermediate image, the image mean of the intermediate image is modified to a preset mean, thus obtaining a modified intermediate image; wherein, the image mean of the intermediate image refers to the average value among all pixel values ​​in the intermediate image; Feature extraction is performed on the modified intermediate image to obtain the nth image feature vector.

13. A three-dimensional model generation device, characterized in that, include: The acquisition unit is used to acquire a two-dimensional view of a three-dimensional object and extract features from the two-dimensional view to obtain the object feature vector of the three-dimensional object. The acquisition unit is further configured to acquire K reference feature vectors, wherein the reference feature vectors are feature vectors containing Gaussian features, and Gaussian features are features of a Gaussian distribution, and K is a positive integer; The processing unit is used to perform feature enhancement processing on the object feature vector using each reference feature vector through an attention mechanism, to obtain K enhanced feature vectors, each enhanced feature vector representing a Gaussian distribution. The processing unit is also used to obtain the Gaussian features of each enhanced feature vector, wherein the Gaussian features of any enhanced feature vector refer to the features of the Gaussian distribution represented by the corresponding enhanced feature vector. The processing unit is further configured to construct a Gaussian three-plane of the three-dimensional object using the acquired Gaussian features, and to perform three-dimensional reconstruction of the three-dimensional object based on the Gaussian three-plane to obtain a three-dimensional model of the three-dimensional object.

14. A computer device, comprising an input interface and an output interface, characterized in that, Also includes: Processor and computer storage media; The processor is adapted to implement one or more instructions, the computer storage medium stores one or more instructions, and the one or more instructions are adapted to be loaded by the processor and executed as described in any one of claims 1-12.

15. A computer storage medium, characterized in that, The computer storage medium stores one or more instructions, which are adapted to be loaded by a processor and executed as described in any one of claims 1-12.

16. A computer program product, characterized in that, The computer program product includes one or more instructions; when one or more instructions in the computer program are executed by the processor, they implement the three-dimensional model generation method as described in any one of claims 1-12.

Images