A model processing method, apparatus, device, storage medium, and program product

Abstract

Embodiments of the present application disclose a model processing method, device, equipment, storage medium and program product, which are used for efficiently and high-quality reconstructing a three-dimensional model and promoting the rendering effect of the three-dimensional model in a large-scale scene. The method comprises: performing spherical conversion processing on model data of an obtained initial three-dimensional model to obtain spherical mesh data of the initial three-dimensional model; generating a spherical feature map of the initial three-dimensional model based on the spherical mesh data of the initial three-dimensional model; updating model parameters of an initial spherical convolution model based on convolution parameters of a plurality of convolution layers and regularization term information of each convolution layer to obtain a target spherical convolution model, each convolution layer comprising a plurality of convolution kernels with different sizes, and the regularization term information of each convolution layer being calculated based on weight values of the plurality of convolution kernels in the corresponding convolution layer; and performing model reconstruction processing on the spherical feature map based on the target spherical convolution model to determine a target three-dimensional model, the target three-dimensional model being used to perform a rendering task.

Description

Technical Field

[0001] This application relates to the field of computer technology, specifically to a method, apparatus, device, storage medium, and program product for model processing. Background Technology

[0002] In the fields of three-dimensional (3D) computer vision, virtual reality (VR), augmented reality (AR), and game development, efficient and high-quality 3D model reconstruction is one of the key technologies. Reconstructed 3D models are typically optimized, removing redundant points and faces to make the model structure simpler and more efficient. Furthermore, the reconstruction process also involves refining the model, adding necessary texture details, etc., enabling the construction of rich and diverse rendering scenes to meet different creative needs and ensure the smoothness and realism of the product.

[0003] Traditional model reconstruction schemes typically utilize the 3D data of geometric samples directly for reconstruction, followed by manual processing. However, this traditional approach is susceptible to model complexity and relies heavily on hardware resources, resulting in poor reconstruction performance in large-scale scenes. Consequently, the rendering quality of the reconstructed 3D model deteriorates in rendering tasks. Summary of the Invention

[0004] This application provides a method, apparatus, device, storage medium, and program product for model processing, which can efficiently and with high quality reconstruct 3D models and improve the rendering effect of 3D models in large-scale scenes.

[0005] In a first aspect, embodiments of this application provide a model processing method. The method includes: performing spherical transformation processing on the model data of an acquired initial 3D model to obtain spherical mesh data of the initial 3D model; generating a spherical feature map of the initial 3D model based on the spherical mesh data, the spherical feature map being used to characterize the features of the initial 3D model; updating the model parameters of the initial spherical convolutional model based on the convolution parameters of multiple convolutional layers and the regularization term information of each convolutional layer to obtain a target spherical convolutional model, each convolutional layer including multiple convolutional kernels of different sizes, the regularization term information of each convolutional layer being calculated based on the weight values ​​of the multiple convolutional kernels in the corresponding convolutional layer; and performing model reconstruction processing on the spherical feature map based on the target spherical convolutional model to determine a target 3D model, the target 3D model being used to perform a rendering task.

[0006] Secondly, embodiments of this application provide a model processing apparatus. The model processing apparatus includes a conversion processing unit, a generation processing unit, an update processing unit, and a reconstruction processing unit.

[0007] The conversion processing unit is used to perform spherical conversion processing on the model data of the acquired initial three-dimensional model to obtain the spherical mesh data of the initial three-dimensional model.

[0008] A generation processing unit is used to generate a spherical feature map of the initial three-dimensional model based on the spherical mesh data of the initial three-dimensional model. The spherical feature map is used to characterize the features of the initial three-dimensional model.

[0009] An update processing unit is used to update the model parameters of the initial spherical convolution model based on the convolution parameters of multiple convolutional layers and the regularization term information of each convolutional layer, so as to obtain the target spherical convolution model. Each convolutional layer includes multiple convolutional kernels of different sizes, and the regularization term information of each convolutional layer is calculated based on the weight values ​​of the multiple convolutional kernels in the corresponding convolutional layer.

[0010] The reconstruction processing unit is used to perform model reconstruction processing on the spherical feature map based on the target spherical convolution model to determine the target three-dimensional model, which is used to perform rendering tasks.

[0011] In conjunction with one or more embodiments of the second aspect described above, in some other optional embodiments, the model processing apparatus provided in this application may further include an acquisition unit and a calculation processing unit.

[0012] The acquisition unit is used to update the model parameters of the initial spherical convolution model based on the convolution parameters of multiple convolutional layers and the regularization term information of each convolutional layer. Before obtaining the target spherical convolution model, the unit acquires the convolution parameters of multiple convolutional layers and the convolution information of the initial convolutional kernel of the initial spherical convolution model. The size of the multiple convolutional kernels in each convolutional layer is different from the size of the initial convolutional kernel.

[0013] The computational processing unit is used to calculate the regularization term information corresponding to the convolutional layer based on the weight values ​​of the multiple convolutional kernels in each convolutional layer;

[0014] The update processing unit is specifically used for:

[0015] Based on the convolution parameters of each convolutional layer and the regularization term information of the corresponding convolutional layer, the target convolution information of each convolutional layer is determined;

[0016] The target convolutional information of multiple convolutional layers is superimposed, and the convolutional information of the initial convolutional kernel in the initial spherical convolutional model is replaced by the superimposed target convolutional information to obtain the target spherical convolutional model. The input of the first convolutional layer of the target spherical convolutional model is the spherical feature map, and the input of the Lth convolutional layer of the target spherical convolutional model is the feature output by the (L-1)th convolutional kernel, the size of the Lth convolutional layer, and the regularization term information of the Lth convolutional layer, where L is an integer greater than or equal to 2.

[0017] In conjunction with one or more embodiments of the second aspect described above, in some alternative embodiments, the updating processing unit is specifically used for:

[0018] The spherical feature map is used as the input of the target sphere convolution model. The spherical feature map is convolved by multiple superimposed convolutional layers in the target sphere convolution model to obtain the deep features output by the Lth convolutional layer.

[0019] The deep features output by the Lth convolutional layer are deconvolutionally processed to obtain the model reconstruction data of the initial 3D model;

[0020] The target 3D model is generated from the model reconstruction data based on the initial 3D model.

[0021] In conjunction with one or more embodiments of the second aspect described above, in some alternative embodiments, the updating processing unit is specifically used for:

[0022] The spherical feature map is used as the input to the first convolutional layer in the target sphere convolutional model. The spherical feature map is convolved by multiple convolutional kernels of the specified size in the first convolutional layer to obtain the feature map output by multiple convolutional kernels in the Lth convolutional layer. The feature map output by multiple convolutional kernels and the regularization term information of the first convolutional layer are activated to obtain the output feature map of the first convolutional layer.

[0023] The output feature map of the first convolutional layer is used as the input of the Lth convolutional layer in the target sphere convolutional model. The output feature map of the first convolutional layer is convolved by multiple convolutional kernels of the specified size in the Lth convolutional layer to obtain the feature maps output by the multiple convolutional kernels in the Lth convolutional layer. The feature maps output by the multiple convolutional kernels are then activated with the regularization term information of the Lth convolutional layer to obtain the deep features output by the Lth convolutional layer.

[0024] In conjunction with one or more embodiments of the second aspect described above, in some alternative embodiments, the updating processing unit is specifically used for:

[0025] The spherical feature map is used as the input of the target spherical convolutional model. The spherical feature map is convolved by multiple superimposed convolutional layers in the target spherical convolutional model to obtain the deep features output by the Lth convolutional layer. Then, the deep features output by the Lth convolutional layer and the feature vector output by the Kth convolutional layer are fused based on a preset fusion coefficient to obtain the target fused features, where K is a positive integer not equal to L.

[0026] The target fusion features are deconvolutionally processed to obtain the model reconstruction data of the initial 3D model.

[0027] In conjunction with one or more embodiments of the second aspect described above, in some alternative embodiments, the updating processing unit is specifically used for:

[0028] Before performing deconvolution processing on the target fusion features to obtain the model reconstruction data of the initial 3D model, local convolution processing is performed on the target fusion features based on the local convolution kernel in the target sphere convolution model to obtain local convolution features;

[0029] The local convolutional features and the regularization term information of the local convolutional kernel are activated to obtain local enhanced features;

[0030] The target fusion features and the local convolution features are deconvolutionally processed to obtain the model reconstruction data of the initial 3D model.

[0031] In conjunction with one or more embodiments of the second aspect described above, in some alternative embodiments, the model processing apparatus provided in this application further includes a feature extraction unit, a sampling processing unit, and a texture mapping unit.

[0032] The feature extraction unit is specifically used to perform texture feature extraction processing on the model data of the initial three-dimensional model after the model reconstruction processing of the spherical feature map based on the target spherical convolution model is performed to determine the target three-dimensional model, thereby obtaining the texture feature information of the initial three-dimensional model.

[0033] The sampling processing unit is specifically used to resample the texture feature information of the initial three-dimensional model to obtain the target texture feature information;

[0034] The texture mapping unit is specifically used to perform texture mapping processing on the target 3D model based on the target texture feature information.

[0035] In conjunction with one or more embodiments of the second aspect described above, in some alternative embodiments, the target texture feature information includes target texture coordinates;

[0036] The computational processing unit is further configured to, after resampling the texture feature information of the initial three-dimensional model to obtain the target texture feature information, calculate the normal map feature based on the target texture coordinates, wherein the normal map feature is used to describe the normal situation of the target three-dimensional model;

[0037] The computational processing unit is specifically used to calculate ambient light occlusion features based on the vertex coordinates of multiple vertices in the target 3D model and preset ambient light occlusion parameters. The ambient light occlusion features are used to describe the degree to which the surface of the target 3D model is occluded by light.

[0038] The texture mapping unit is specifically used to perform texture mapping processing on the target 3D model based on the target texture feature information, the normal map features, and the ambient light occlusion features.

[0039] In conjunction with one or more embodiments of the second aspect above, in some alternative embodiments, the target texture feature information includes an initial texture resolution; the calculation processing unit is further configured to, after resampling the texture feature information of the initial three-dimensional model to obtain the target texture feature information, calculate the resolution texture feature of each of the multiple layers based on the initial texture resolution and the preset number of resolution layers.

[0040] The sampling processing unit is specifically used to perform upsampling processing based on the resolution texture features of each level, and to perform fusion processing on the resolution texture features obtained by the upsampling processing to obtain fused texture feature information;

[0041] The texture mapping unit is specifically used to perform texture mapping processing on the target 3D model based on the fused texture feature information.

[0042] In conjunction with one or more embodiments of the second aspect described above, in some alternative embodiments, the computational processing unit is further configured to:

[0043] After determining the target 3D model by reconstructing the spherical feature map based on the target spherical convolution model, the mean square error between the target 3D model and the initial 3D model is calculated.

[0044] The regularization information of multiple convolutional layers is summed to obtain the target regularization information;

[0045] The target loss value is determined based on the mean square error and the target regularization term information;

[0046] The update processing unit is specifically used to update the model parameters of the target sphere convolution model based on the target loss value.

[0047] In conjunction with one or more embodiments of the second aspect described above, in some alternative embodiments, the model data of the initial 3D model includes the vertex coordinates of the initial 3D model; the transformation processing unit is specifically used for:

[0048] The vertex coordinates are calculated based on the preset four-quadrant arctangent function to obtain the longitude coordinates of the spherical grid points in the spherical coordinate system corresponding to the target sphere.

[0049] The vertex coordinates are calculated and processed based on a preset inverse cosine function to obtain the latitude coordinates of the vertex coordinates as spherical grid points in the spherical coordinate system.

[0050] Based on the longitude and latitude coordinates, the Cartesian coordinates of the vertex coordinates in the spherical coordinate system are calculated to obtain the spherical mesh data of the initial three-dimensional model.

[0051] In conjunction with one or more embodiments of the second aspect described above, in some alternative embodiments, a processing unit is generated, specifically for:

[0052] The Cartesian coordinates are subjected to spherical position feature extraction processing to obtain the position feature vector of the vertex coordinates;

[0053] The texture map coordinates of the initial 3D model are determined based on the Cartesian coordinates, and the normal feature vector is determined based on the texture map coordinates.

[0054] Calculate the curvature feature vector of the target sphere based on the radius information of the target sphere;

[0055] Based on the position feature vector, the normal feature vector, and the curvature feature vector, a spherical feature map of the initial three-dimensional model is generated.

[0056] In conjunction with one or more embodiments of the second aspect described above, in some alternative embodiments, the computational processing unit is further configured to:

[0057] Before generating the spherical feature map of the initial three-dimensional model based on the position feature vector, the normal feature vector, and the curvature feature vector, the geometric feature vector of the first interpolation point is calculated based on a preset interpolation function to determine the interpolation value of the first interpolation point. The first interpolation point is a known vertex of the target sphere. The geometric feature vector includes one or more of the position feature vector, the normal feature vector, and the curvature feature vector.

[0058] Based on the interpolation value of the first interpolation point and the interpolation weight value between the first interpolation point and the second interpolation point, the interpolation value of the second interpolation point is calculated to obtain the geometric feature vector of the second interpolation point. The second interpolation point is an unknown vertex of the target sphere.

[0059] The generation processing unit is specifically used for:

[0060] Based on the position feature vector, the normal feature vector, the curvature feature vector, and the geometric feature vector of the second interpolation point, a spherical feature map of the initial three-dimensional model is generated.

[0061] A third aspect of this application provides a computer device, including a memory and a processor. The memory stores program instructions. The processor executes the program instructions in the memory to perform the model processing method corresponding to the embodiments of the first aspect described above.

[0062] A fourth aspect of this application provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the method corresponding to the embodiments of the first aspect described above.

[0063] The fifth aspect of this application provides a computer program product containing instructions that, when run on a computer or processor, causes the computer or processor to execute the method described above for performing the implementation method of the first aspect.

[0064] As can be seen from the above technical solutions, the embodiments of this application have the following advantages:

[0065] In this embodiment, after obtaining the model data of the initial 3D model, the obtained model data is first subjected to spherical transformation to obtain spherical mesh data of the initial 3D model. Subsequently, based on the spherical mesh data of the initial 3D model, a spherical feature map of the initial 3D model is generated. The spherical feature map is used to characterize the features of the initial 3D model. Furthermore, the model parameters of the initial spherical convolutional model are updated based on the convolution parameters of multiple convolutional layers and the regularization information of each convolutional layer to obtain the target spherical convolutional model. It should be noted that each convolutional layer includes multiple convolutional kernels of different sizes, and the regularization information of each convolutional layer is calculated based on the weight values ​​of the multiple convolutional kernels in the corresponding convolutional layer. Thus, by performing model reconstruction processing on the spherical feature map based on the target spherical convolutional model, a target 3D model suitable for performing rendering tasks can be determined. Through the above method, by converting the model data of the initial 3D model into spherical mesh data and extracting the corresponding spherical feature map, it can be ensured that the feature map input into the subsequent target spherical convolutional model is suitable for the processing of the spherical convolutional model, and the geometric details of the initial 3D model can be preserved to the greatest extent. Furthermore, this application updates the target sphere convolutional model based on multiple convolutional layers and the regularization terms of each layer. This allows the target sphere convolutional model to capture geometric features at different scales, better handle high-frequency details and complex topological structures, and reduce the use of hardware resources. Thus, using this target sphere convolutional model for efficient reconstruction of spherical feature maps ensures improved reconstruction speed and quality while retaining more model details, significantly enhancing the rendering effect of 3D models in large-scale scenes. Attached Figure Description

[0066] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0067] Figure 1 A schematic diagram of the system framework provided in an embodiment of this application is shown;

[0068] Figure 2 This illustration shows an implementation environment diagram of the model processing method provided in an embodiment of this application;

[0069] Figure 3 A flowchart illustrating a model processing method provided in an embodiment of this application is shown.

[0070] Figure 4 This illustration shows an optional framework diagram for generating spherical feature maps provided in an embodiment of this application;

[0071] Figure 5 This illustration shows an optional framework diagram for constructing a target sphere convolution model provided in an embodiment of this application;

[0072] Figure 6 Another schematic flowchart of the model processing method provided in this application embodiment is shown;

[0073] Figure 7 A schematic diagram of one embodiment of the model processing apparatus provided in this application is shown;

[0074] Figure 8 A schematic diagram of another embodiment of the computer device provided in this application is shown. Detailed Implementation

[0075] This application provides a method, apparatus, device, storage medium, and program product for model processing, which are used to efficiently and with high quality reconstruct 3D models and improve the rendering effect of 3D models in large-scale scenes.

[0076] It is understood that in the specific embodiments of this application, data such as user information are involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0077] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0078] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that implementations of the application described herein can be implemented, for example, in sequences other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0079] In fields such as computer graphics and 3D modeling, reconstructing original 3D models is an important task. This not only improves the quality of the model but also enables it to be used for rendering tasks, such as those in augmented reality, virtual reality, game development, and film production.

[0080] Traditional model reconstruction methods typically utilize the 3D data of geometric samples for reconstruction, which is then manually processed. However, this traditional approach relies heavily on hardware resources for processing 3D data and is susceptible to the complexity of the model. Consequently, using traditional methods for 3D model reconstruction can lead to reduced reconstruction efficiency and quality in large-scale scenes, resulting in suboptimal rendering performance in subsequent tasks.

[0081] To address the aforementioned technical problems, this application provides a model processing method. This method can be applied to various scenarios involving the reconstruction and rendering of 3D models. For example, scenarios involving the reconstruction or rendering of 3D models include, but are not limited to, virtual games, film and television, advertising, AR, and VR. In the model processing method provided in this application, the model data of the initial 3D model is converted into spherical mesh data, and the corresponding spherical feature maps are extracted. This ensures that the feature maps input into the subsequent target spherical convolutional model are suitable for the processing of the spherical convolutional model and can retain the geometric details of the initial 3D model to the greatest extent. Furthermore, this application updates the target spherical convolutional model based on multiple convolutional layers and the regularization terms of each layer, enabling the target spherical convolutional model to capture geometric features at different scales and better handle high-frequency details and complex topological structures. Thus, using the target spherical convolutional model for efficient reconstruction of spherical feature maps ensures improved reconstruction speed while preserving more model details, thereby improving rendering speed and model detail performance, and greatly enhancing the model rendering effect in large-scale scenes. For example, the model processing method provided in this application can be applied to one or more of the following application scenarios, as follows:

[0082] (I) Game Development Scenarios

[0083] For example, in game development scenarios, 3D models typically need to possess characteristics such as high precision, high quality, high efficiency, and reusability. They are often used to create game characters, game props, and virtual scenes in virtual games, providing players with an immersive gaming experience. Furthermore, 3D models can also be used to create animations and special effects in virtual games, such as character movements, combat effects, and environmental interactions. These animations and effects can enhance the visual impact of virtual games and improve their overall quality. Applying the model processing method of this application to game development scenarios considers the spherical features of the initial 3D model and fully leverages the ability of multiple convolutional layers and regularization terms in the spherical convolution model to capture high-frequency details and geometric features of different scales within the spherical features. Therefore, during the reconstruction of the 3D model, it ensures that reconstruction speed is improved while preserving more model details, thereby increasing rendering speed and model detail performance. This further enhances the rendering effect of game rendering tasks in game development scenarios, bringing players a more immersive gaming experience.

[0084] (II) AR / VR Scenarios

[0085] For example, in AR scenarios, 3D models can be used to combine virtual elements with the real world, allowing users to immerse themselves in a world where the virtual and real worlds intersect. For instance, applying the model processing method of this application to AR scenarios considers the spherical features of the initial 3D model and fully leverages the ability of multiple convolutional layers and regularization terms in the spherical convolution model to capture high-frequency details and geometric features of different scales within the spherical features. This ensures that while preserving more model details, the reconstruction speed is improved, thereby increasing rendering speed and model detail performance, ultimately enhancing the rendering effect and enabling users to fully understand product information after viewing or using the target 3D model. Similarly, in VR scenarios, 3D models can be used to create realistic virtual scenes. Applying the model processing method of this application to VR scenarios ensures that while preserving more model details, the reconstruction speed is improved, thereby increasing rendering speed and model detail performance, ultimately enhancing the rendering effect and providing users with an immersive experience.

[0086] (III) Digital Twins and Intelligent Manufacturing

[0087] For example, digital twins are a technology that combines the physical world with the virtual world. Applying the model processing method of this application to digital twin scenarios can improve reconstruction speed while preserving more model details, thereby improving rendering speed and model detail performance, further enhancing the rendering effect of rendering tasks, and using the target 3D model to create a high-quality 3D digital twin model of the physical object. This allows managers to monitor and analyze the status and performance of the physical object in real time to optimize production and management. Similarly, in smart manufacturing scenarios, applying the model processing method of this application can improve reconstruction speed, rendering speed, and model detail performance to enhance rendering effects, and using the target 3D model to generate physical products with high quality, thereby guiding the production process, optimizing production workflows, and improving production efficiency.

[0088] It should be noted that the model processing method of this application, in addition to the game development, AR / VR, digital twin, and intelligent manufacturing scenarios mentioned above, can also be applied to other scenarios in practical applications. For example, it can also be applied to: creating visual effects, film characters, and scenes using 3D models in film and television production; creating architectural models using 3D models in architectural design; creating product models and product animations using 3D models in industrial manufacturing; and creating virtual laboratories and simulated experimental scenarios, and training scenarios using 3D models in education and training scenarios, etc., without specific limitations in this application.

[0089] For example, the method provided in this application can be applied to Figure 1 The system framework is shown. For example... Figure 1 As shown, after obtaining the initial 3D model data, this application performs spherical transformation on it to obtain corresponding spherical mesh data. Further, multiple convolutional layers and regularization terms for each convolutional layer are introduced into the initial spherical convolutional model to adjust and obtain the target spherical convolutional model. Thus, the spherical feature map extracted from the spherical network data is used as input to the target spherical convolutional model, thereby reconstructing the corresponding target 3D model using this target spherical convolutional model. Optionally, the texture features of the initial 3D model can also be used to perform texture remapping and other processing on the target 3D model.

[0090] Combination Figure 1 , Figure 2 An implementation environment is shown where model processing device 100 interacts with user terminal 200 to perform operations. Figure 2The implementation environment shown includes a model processing device 100, and also includes, for example, a user terminal 200. The model processing device 100 and the user terminal 200 can be directly or indirectly connected via wired or wireless communication, etc., without specific limitations in this application. Optionally, the implementation environment may also include a database 300, etc. The model processing device 100 can also be connected to the database 300 via a network.

[0091] from Figure 2 As can be seen, in step S1, the user object first obtains the model data of the initial 3D model, such as vertex coordinates, through the user terminal 200. In step S2, after obtaining the model data of the initial 3D model, the user terminal 200 sends the model data of the initial 3D model to the model processing device 100. Alternatively, the model processing device 100 can directly obtain the model data of the initial 3D model stored in advance from a storage medium such as a database 300. This application does not limit the acquisition method.

[0092] Therefore, in step S3, the model processing device 100 performs spherical transformation processing on the acquired initial 3D model data to obtain spherical mesh data of the initial 3D model. In step S4, the model processing device 100 generates a spherical feature map of the initial 3D model based on the spherical mesh data of the initial 3D model. The described spherical feature map is used to characterize the features of the initial 3D model. In addition, in step S5, the model processing device 100 updates the model parameters of the initial spherical convolutional model based on the convolution parameters of multiple convolutional layers and the regularization information of each convolutional layer to obtain the target spherical convolutional model. Each convolutional layer includes multiple convolutional kernels of different sizes. The regularization information of each convolutional layer is calculated based on the weight values ​​of multiple convolutional kernels in the corresponding convolutional layer. Thus, in step S6, the model processing device 100 performs model reconstruction processing on the spherical feature map based on the target spherical convolutional model to determine the target 3D model. For example, the determined target 3D model can be used to perform rendering tasks in different scenes or different domains.

[0093] Optionally, in Figure 2 The implementation environment shown may also include step S7. That is, in step S7, after the model processing device 100 reconstructs the target 3D model, it can send the target 3D model to the user terminal 200. In this way, the user terminal 200 can use the target 3D model to complete operations such as creating and rendering related products.

[0094] It should be noted that this application Figure 2The initial spherical convolution model and the target spherical convolution model described herein are only illustrated using the example of storage in database 300. In practical applications, the initial spherical convolution model and the target spherical convolution model in this application can also be stored in other storage media, such as a distributed storage system; or they can be stored in a model processing device; or they can be stored in a blockchain. This application does not limit the specific storage media.

[0095] Optionally, the model processing device 100 and user terminal 200 mentioned above can be block nodes in a blockchain network. By applying the model processing method of this application to a blockchain network scenario, the logic of various operations can be run in the smart contract in the blockchain network, enabling both the user terminal 200 and the model processing device 100 to call the smart contract and participate in the business.

[0096] The user terminal 200 involved in this application includes, but is not limited to, mobile phones, tablets, laptops, desktop computers, intelligent voice interaction devices, smart home appliances, vehicle terminals, and aircraft. The client is deployed on the user terminal 200 and can run on the user terminal 200 via a browser or as a standalone application (APP).

[0097] The model processing device 100 involved in this application can be a processing device with data processing capabilities, such as, but not limited to, terminal devices, servers, etc. The server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence (AI) platforms.

[0098] The database 300 involved in this application is a collection of data stored together in a certain way, shared by multiple users, with minimal redundancy, and independent of application programs. A Database Management System (DBMS) is a computer software system designed to manage databases, generally possessing basic functions such as storage, retrieval, security, and backup. DBMSs can be classified according to the database model they support, such as relational or Extensible Markup Language (XML); or according to the type of computer they support, such as server clusters or mobile phones; or according to the query language used, such as Structured Query Language (SQL) or XQuery; or according to performance priorities, such as maximum scale or maximum operating speed; or other classification methods. Regardless of the classification method used, some DBMSs can cross categories, for example, simultaneously supporting multiple query languages. In this application, database 300 can be used to store initial spherical convolution models, target spherical convolution models, etc.

[0099] Based on the above introduction, the model processing method provided in this application will be described below with reference to the accompanying drawings. Figure 3 A flowchart illustrating the model processing method provided in this application is shown. Figure 3 As shown, the model processing method of this application includes at least the following steps:

[0100] 301. Perform spherical transformation on the model data of the acquired initial 3D model to obtain the spherical mesh data of the initial 3D model.

[0101] In one or more embodiments, the initial 3D model can be understood as the original 3D digital representation created, or it can be understood as the 3D model that needs to be reconstructed. When it is necessary to reconstruct certain 3D models, the model processing device can obtain the model data of the initial 3D model through different acquisition methods, which are not limited in this application. For example, a user can input the data related to the 3D model that needs to be reconstructed, i.e., the model data of the initial 3D model, into a user terminal using modeling software, etc. In this way, after the user terminal obtains the model data of the initial 3D model entered by the user, it sends the model data of the initial 3D model to the model processing device through a communication network, thereby allowing the model processing device to obtain the model data of the initial 3D model. Alternatively, since the model data of the initial 3D model is also stored in a storage medium such as a database in advance, the model processing device can directly obtain the relevant model data of the initial 3D model from the storage medium according to the reconstruction requirements.

[0102] The model data of the initial 3D model mentioned in this application includes information describing the shape, position, etc. of the initial 3D model. For example, the model data of the initial 3D model mentioned in this application includes, but is not limited to, the vertex coordinates, texture information, normals, texture coordinates, etc. of each vertex in the model, etc., which are not limited in this application. Alternatively, the model data of the described initial 3D model can be model data constructed from objects such as images, text, videos, products, virtual games, etc., which are not limited in this application.

[0103] In order for the subsequently updated target spherical convolutional model to effectively process the model data, the model processing device needs to preprocess the initial 3D model data after obtaining it. The preprocessing operations mentioned in this application include converting the initial 3D model data into a spherical representation and extracting relevant spherical features from the spherical representation.

[0104] For example, after obtaining the model data of an initial 3D model, the model processing device can perform spherical transformation processing on the initial 3D model data to obtain spherical mesh data of the initial 3D model. Through the spherical transformation processing method of this application, it can be ensured that the spherical features corresponding to the obtained spherical mesh data are suitable for convolution operations.

[0105] To perform spherical transformation, spherical projection techniques can be used. As an illustrative example, the vertex coordinates can be calculated based on a preset four-quadrant arctangent function to obtain the longitude coordinates of the spherical grid points in the spherical coordinate system corresponding to the target sphere. For instance, for a vertex with coordinates v = (x, y, z), the longitude coordinates of the corresponding spherical grid points satisfy the following expression: θ i =arctan2(y,x). Similarly, based on the preset inverse cosine function, the vertex coordinates are calculated to obtain the latitude coordinates of the spherical grid point in the spherical coordinate system. For example, for a vertex with coordinates v = (x,y,z), the latitude coordinates of its corresponding spherical grid point satisfy the following expression, such as: Thus, based on longitude and latitude coordinates, the Cartesian coordinates of the vertices in the spherical coordinate system can be calculated, thereby obtaining the spherical mesh data of the initial 3D model. For example, the Cartesian coordinates (x, y, y) of a spherical mesh point... i,j ,y i,j ,z i,j ) can be represented as: x i,j =sin(φ) j cos(θ) i ), y i,j =sin(φ) j sin(θ)i ), z i,j =cos(φ j ).

[0106] 302. Based on the spherical mesh data of the initial 3D model, generate the spherical feature map of the initial 3D model.

[0107] In one or more embodiments, after obtaining the spherical mesh data of an initial 3D model, the model processing device can generate a corresponding spherical feature map based on the spherical mesh data of the initial 3D model. For example, after obtaining the spherical mesh data, the model processing device can perform feature extraction processing on the spherical mesh data of the initial 3D model to generate a spherical feature map of the initial 3D model based on the extracted spherical features. By generating this spherical feature map, it is possible to better capture the geometric details in the initial 3D model and improve the representation of geometric details.

[0108] 303. Update the model parameters of the initial spherical convolution model based on the convolution parameters of multiple convolutional layers and the regularization term information of each convolutional layer to obtain the target spherical convolution model. Each convolutional layer includes multiple convolutional kernels of different sizes, and the regularization term information of each convolutional layer is calculated based on the weight values ​​of multiple convolutional kernels in the corresponding convolutional layer.

[0109] In one or more embodiments, spherical convolution is a technique for performing convolution operations on spherical data and is widely used in 3D model processing. However, traditional spherical convolution models often have limitations when handling complex 3D model details due to hardware resources and the complexity of the 3D model structure. Therefore, this application can, based on an initial spherical convolution model, update the model parameters of the initial spherical convolution model based on the convolution parameters of multiple convolutional layers and the regularization term information of each convolutional layer to obtain a target spherical convolution model. The target spherical convolution model constructed in this way can better handle high-frequency details and complex topological structures.

[0110] In other words, to capture geometric features at different scales, this application introduces multiple convolutional layers into the initial spherical convolutional model structure, and each convolutional layer includes multiple convolutional kernels of different sizes. These multiple convolutional kernels of different sizes can process geometric details of different frequencies. Furthermore, to enhance the processing capability for high-frequency details, this application adds regularization terms to each convolutional layer based on the initial spherical convolutional model structure. These regularization terms constrain the transformation range of different convolutional kernels in each convolutional layer, preventing the loss of high-frequency details.

[0111] 304. Based on the target spherical convolution model, the spherical feature map is reconstructed to determine the target 3D model, which is then used to perform the rendering task.

[0112] In one or more embodiments, after updating the model parameters of the initial spherical convolutional model using the convolution parameters of multiple convolutional layers and the regularization information of each convolutional layer to obtain the target spherical convolutional model, the aforementioned generated spherical feature map can be used as the input to the target spherical convolutional model. Thus, the spherical feature map is reconstructed using the target spherical convolutional model to determine the corresponding target 3D model. This reconstructed target 3D model can then be used to perform rendering tasks, etc.

[0113] In this embodiment, by converting the initial 3D model data into spherical mesh data and extracting the corresponding spherical feature maps, it is ensured that the feature maps input into the subsequent target spherical convolutional model are suitable for processing by the spherical convolutional model, and the geometric details of the initial 3D model are preserved to the greatest extent. Furthermore, this application updates the target spherical convolutional model based on multiple convolutional layers and the regularization terms of each layer, enabling the target spherical convolutional model to capture geometric features at different scales, better handle high-frequency details and complex topological structures, and reduce the use of hardware resources. Thus, using this target spherical convolutional model for efficient reconstruction of spherical feature maps ensures improved reconstruction speed and model reconstruction quality while retaining more model details, greatly enhancing the rendering effect of 3D models in large-scale scenes.

[0114] In some optional examples, in the aforementioned Figure 3 Based on the illustrated embodiment, the following describes how to generate the spherical feature map of the initial 3D model in step 302. Figure 4 Use the illustrated framework diagram to understand it. Figure 4 As shown, spherical features typically include, but are not limited to, vertex positions, normals, and spherical curvature.

[0115] like Figure 4 As shown, after obtaining the Cartesian coordinates, spherical position feature extraction can be performed on them to obtain the position feature vectors of the vertex coordinates. Similarly, the texture map coordinates of the initial 3D model also need to be determined based on the Cartesian coordinates. For example, the UV unwrapping tools (such as spherical mapping) provided by the modeling software can be used to automatically calculate the texture map coordinates of the initial 3D model based on the Cartesian coordinates. Then, the normal feature vector is determined based on the texture map coordinates. For example, the normal feature vector can satisfy the following formula: Where n represents the normal feature vector, u and v are texture map coordinates, and × represents the vector cross product.

[0116] In addition, based on the radius information of the target sphere, it is necessary to calculate the curvature feature vector of the target sphere. The described curvature feature vector includes, but is not limited to, principal curvature, Gaussian curvature, and mean curvature, etc., which are not limited in this application. For example, taking the principal curvatures including k1 and k2 as an example, the Gaussian curvature k' satisfies the following expression, such as k'=k1k2; the mean curvature k” satisfies the following expression, such as k”=(k1+k2) / 2.

[0117] Thus, after calculating the position eigenvector, normal eigenvector, and curvature eigenvector, a spherical feature map of the initial 3D model can be generated based on these vectors. For example, a position feature map can be generated from the position eigenvector, a normal map from the normal eigenvector, and a curvature map from the curvature eigenvector. The position feature map, normal map, and curvature map are then combined to form the spherical feature map.

[0118] The above method uses position feature vectors to describe the positions of each point on the surface of the initial 3D model in spherical space. Mapping these vectors onto a spherical feature map preserves the model's global positional information, facilitating the reconstruction of the model's shape and structure in spherical space. Furthermore, normal feature vectors describe the normal directions at each point on the model's surface, helping to simulate lighting effects and making the textures on the spherical feature map more realistic and three-dimensional. Additionally, curvature feature vectors describe the degree of curvature at each point on the model's surface. Mapping these vectors onto the spherical feature map highlights the model's details and contours, resulting in more delicate and richer textures. Thus, combining position, normal, and curvature feature vectors to generate a spherical feature map enhances model detail and improves subsequent rendering quality and efficiency, supporting complex scene rendering tasks.

[0119] In some alternative embodiments, if the vertex distribution in the initial 3D model is uneven, directly using spherical mesh data to generate a spherical feature map may result in the loss of model-related information.

[0120] Based on this, in the aforementioned Figure 4 Based on the described embodiments, this application can call a preset interpolation function before generating the spherical feature map of the initial 3D model based on the position feature vector, normal feature vector, and curvature feature vector. For example Where, p i f'() represents the interpolation point, and w represents the preset interpolation function. ij Indicates the interpolation point p i With interpolation point p j Interpolation weights between, f'(p) iThe expression represents the interpolated value at interpolation point pi. It should be noted that the interpolation weight w... ij The weights can be determined using either inverse distance weighting (IDW) or radial basis function (RBF), and this application does not specify a particular method. The described inverse distance weighting can be understood as a method for estimating the value of an unknown point in space. Specifically, it calculates the distance between the unknown point and known points and determines the weights between the known and unknown points based on the reciprocal of the distance. The described radial basis function method uses radial basis functions as interpolation basis functions and obtains the coefficients of the interpolation function by solving a system of linear equations to determine the weights between known and unknown points.

[0121] Thus, the geometric feature vector of the first interpolation point is calculated based on a preset interpolation function to determine the interpolation value of the first interpolation point. The first interpolation point is a known vertex of the target sphere. The geometric feature vector described here includes one or more of the aforementioned position feature vector, normal feature vector, and curvature feature vector. For example, the position feature vector of the first interpolation point is calculated based on the preset interpolation function to determine the position interpolation value of the first interpolation point. Similarly, the normal feature vector of the first interpolation point is calculated based on the preset interpolation function to determine the normal interpolation value of the first interpolation point. Additionally, the curvature feature vector of the first interpolation point also needs to be calculated based on the preset interpolation function to determine the curvature interpolation value of the first interpolation point.

[0122] Furthermore, based on the interpolated value of the first interpolation point and the interpolation weight value between the first and second interpolation points, the interpolated value of the second interpolation point is calculated, resulting in the geometric feature vector of the second interpolation point, which is an unknown vertex of the target sphere. For example, the position interpolated value of the second interpolation point is calculated based on the position interpolated value of the first interpolation point and the position interpolation weight value between the first and second interpolation points. Similarly, the normal interpolated value of the second interpolation point is calculated based on the normal interpolated value of the first interpolation point and the normal interpolation weight value between the first and second interpolation points. In addition, the curvature interpolated value of the second interpolation point also needs to be calculated based on the curvature interpolated value of the first interpolation point and the curvature interpolation weight value between the first and second interpolation points.

[0123] Thus, in the process of generating the spherical feature map of the initial 3D model based on the position feature vector, normal feature vector, and curvature feature vector, the spherical feature map of the initial 3D model can be specifically generated based on the position feature vector, normal feature vector, curvature feature vector, and the geometric feature vector of the second interpolation point. For example, the spherical feature map of the initial 3D model can be generated based on the position feature vector, normal feature vector, curvature feature vector, and the position interpolation value, normal interpolation value, and curvature interpolation value of the second interpolation point. For example, taking the spherical feature map as F, the components after interpolation can be represented as: F ps =(x i,j ,y i,j ,z i,j ), F nra =(n xi,j ,n yi,j ,n zi,j ), F craue =(k1) i,j k2 i,j ,k' i,j ,k” i,j ). Among them, F ps =(x i,j ,y i,j ,z i,j ) represents the location feature vector. F represents the eigenvector of the normal line. craue =(k1) i,j k2 i,j ,k' i,j ,k” i,j Then ) represents the curvature eigenvector.

[0124] By performing spherical interpolation on the extracted feature vectors, this application can ensure the continuity and smoothness of the feature vectors on the sphere, thereby mapping more geometric details in the initial 3D model onto the spherical mesh and solving the problem of information loss.

[0125] In some alternative embodiments, to adapt the feature data to the input range of the subsequent target sphere convolution model, this application may further normalize the aforementioned geometric feature vectors, such as position feature vectors, normal feature vectors, and curvature feature vectors, after extraction. For example, the normalized features can be represented as: Where f1 represents the geometric feature vector, such as the positional feature vector F. ps =(x i,j ,y i,j ,z i,j ), normal eigenvector and curvature eigenvector F craue=(k1) i,j k2 i,j ,k' i,j ,k” i,j f nr f represents the normalized feature. min f represents the minimum value in the eigenvector. max This represents the maximum value in the eigenvector.

[0126] For example, the maximum and minimum values ​​are found from the position eigenvectors, and these position eigenvectors, along with their maximum and minimum values, are used as inputs to the aforementioned normalization function to determine the normalized position eigenvector. Similarly, the maximum and minimum values ​​are found from the normal eigenvectors, and these normal eigenvectors, along with their maximum and minimum values, are used as inputs to the aforementioned normalization function to determine the normalized normal eigenvector. Likewise, the maximum and minimum values ​​are found from the curvature eigenvectors, and these curvature eigenvectors, along with their maximum and minimum values, are used as inputs to the aforementioned normalization function to determine the normalized curvature eigenvector. Thus, after calculating the normalized position eigenvector, normalized normal eigenvector, and normalized curvature eigenvector, the corresponding spherical feature map can be generated from these normalized position eigenvectors, normalized normal eigenvectors, and normalized curvature eigenvectors.

[0127] In some other alternative embodiments, in the foregoing Figure 3 Based on one or more embodiments shown, the following describes how to update the target sphere convolution model in step 303. Figure 5 Use the flowchart shown to understand the process.

[0128] like Figure 5 As shown, before executing step 303, the convolution parameters of multiple convolutional layers and the convolution information of the initial convolutional kernel of the initial spherical convolutional model can be obtained. It should be noted that the sizes of the multiple convolutional kernels in each convolutional layer described here are different from the size of the initial convolutional kernel. The described initial convolutional model can be represented by the following formula, such as... Where θ and φ represent longitude and latitude in the spherical coordinate system, respectively, Ω represents the spherical surface region, g represents the initial convolution kernel, f represents the spherical function, and (f*g)(θ,φ) represents the feature output by the initial convolution kernel g.

[0129] To capture geometric features at different scales, this application adds multiple convolutional layers, as mentioned above, to the initial spherical convolution model. Each convolutional layer includes multiple convolutional kernels, and each kernel has a different size to handle geometric details at different frequencies. Thus, after adjusting the initial spherical convolution model using the convolutional parameters of multiple convolutional layers, the multi-scale spherical convolution model of this application is obtained. For example, the multi-scale spherical convolution model satisfies the following expression: Among them, g s This represents a convolution kernel of size s. It should be noted that the size s mentioned in this application can take values ​​such as 1×1, 3×3, 5×5, etc., and is not limited in this application.

[0130] Furthermore, to enhance the processing capability for high-frequency details, this application can extract the weight values ​​of multiple convolutional kernels in each convolutional layer after obtaining the convolutional parameters of each convolutional layer. Thus, based on the weight values ​​of the multiple convolutional kernels in each convolutional layer, the regularization term information of the corresponding convolutional layer is calculated. For example, the regularization term information of each convolutional layer satisfies the following formula: in, r represents the regularization information of the l-th convolutional layer. i+1,j r i,j This represents the weight values ​​of the (i+1,j)th convolutional kernel and the (i,j)th convolutional kernel in the l-th convolutional layer, where λ represents the regularization coefficient.

[0131] In other words, during the process of updating the target spherical convolutional model in step 303, this application can first determine the target convolutional information of each convolutional layer based on the convolutional parameters and regularization information of each convolutional layer. Further, the target convolutional information of multiple convolutional layers is superimposed, and the convolutional information of the initial convolutional kernel in the initial spherical convolutional model is replaced by the superimposed target convolutional information to obtain the target spherical convolutional model. It should be noted that in the target spherical convolutional model of this application, the input of its first convolutional layer is a spherical feature map, and the input of the Lth convolutional layer of the target spherical convolutional model is the feature output of the (L-1)th convolutional layer, the size of the Lth convolutional layer, and the regularization information of the Lth convolutional layer, where L is an integer greater than or equal to 2.

[0132] As an exemplary description, the target sphere convolution model mentioned in this application can be represented by the following formula: in, h represents the regularization information of the l-th convolutional layer. (l) h represents the feature output by the l-th convolutional layer. (l-1) G represents the feature of the output of the (l-1)th layer. (l)Let represent the l-th convolutional layer, and σ represent the non-linear activation function. It should be noted that, since the target sphere convolutional model comprises multiple stacked convolutional layers, the described g... (l) This can be understood as the aforementioned combination of multiple convolutional kernels g of different sizes in the l-th layer. s It is obtained by stacking. h() can be regarded as the aforementioned f().

[0133] To capture geometric features at different scales, this application introduces multiple convolutional layers into the initial spherical convolutional model structure, with each layer containing multiple convolutional kernels of different sizes. These kernels can process geometric details of different frequencies. Furthermore, to enhance the processing capability for high-frequency details, this application adds regularization terms to each convolutional layer, constraining the transformation range of different convolutional kernels within each layer and preventing the loss of high-frequency details.

[0134] In some other alternative embodiments, in the foregoing Figure 5 Based on one or more embodiments shown, the determination of the target 3D model in step 304 can be implemented in the following manner:

[0135] First, the spherical feature map is used as input to the target spherical convolutional model. Multiple stacked convolutional layers within the target spherical convolutional model then convolve the spherical feature map to obtain the deep features output by the Lth convolutional layer. As an illustrative example, during the encoding stage, the spherical feature map is first used as input to the first convolutional layer of the target spherical convolutional model. Multiple convolutional kernels of different sizes within the first convolutional layer convolve the spherical feature map, resulting in feature maps output by these kernels. Subsequently, the feature maps output by these kernels and the regularization term of the first convolutional layer are activated to obtain the output feature map of the first convolutional layer. Further, the output feature map of the first convolutional layer is used as input to the second convolutional layer of the target spherical convolutional model. Multiple convolutional kernels of different sizes within the second convolutional layer convolve the spherical feature map, resulting in feature maps output by these kernels. Subsequently, the feature maps output by multiple convolutional kernels in the second convolutional layer and the regularization term information of the second convolutional layer are activated to obtain the output feature map of the second convolutional layer. Similarly, the output feature map of the (L-1)th convolutional layer is used as the input to the second convolutional layer in the target spherical convolutional model. The spherical feature map is then convolved with convolutional kernels of multiple sizes in the Lth convolutional layer to obtain the feature maps output by multiple convolutional kernels in the Lth convolutional layer. Then, the feature maps output by multiple convolutional kernels in the Lth convolutional layer and the regularization term information of the Lth convolutional layer are activated to obtain the deep features output by the Lth convolutional layer.

[0136] In other words, during the encoding stage, the spherical feature map of the initial 3D model is processed by multiple layers of spherical convolution in the target spherical convolution model to extract high-level geometric features. For example, if the spherical feature map is F, then the deep features output by the l-th convolutional layer can satisfy the following expression: Where σ is a nonlinear activation function, G (l) This can be understood as multiple convolutional kernels in the l-th layer, as mentioned above, g (l) ; This represents the regularization information of the l-th convolutional layer.

[0137] Thus, after determining the deep features output by the Lth convolutional layer, deconvolution processing can be performed on these deep features during the decoding stage to obtain the model reconstruction data of the initial 3D model. This deconvolution operation allows high-level geometric features (such as the deep features output by the Lth convolutional layer) to be restored to the original 3D model data. For example, if the feature output by the lth layer in the decoding stage is D... (l) Then the D (l) It can satisfy the following expression, that is: Where · represents the deconvolution operation, σ is the nonlinear activation function, and G (l) This can be understood as multiple convolutional kernels in the l-th layer, as mentioned above, g (l) ; This represents the regularization information of the l-th convolutional layer.

[0138] In this way, after determining the model reconstruction data of the initial 3D model through deconvolution operation, the target 3D model can be generated based on the model reconstruction data of the initial 3D model.

[0139] In some alternative embodiments, to improve the reconstruction effect, this application further fuses the deep features output by different convolutional layers to better capture the geometric details of the 3D model. For example, this application uses a spherical feature map as input to a target spherical convolutional model, performs convolution processing on the spherical feature map through multiple stacked convolutional layers in the target spherical convolutional model to obtain the deep features output by the Lth convolutional layer, and then obtains the feature vector output by the Kth convolutional layer. Here, K is a positive integer not equal to L. Then, based on a preset fusion coefficient, the deep features output by the Lth convolutional layer and the feature vector output by the Kth convolutional layer are fused to obtain the target fused feature. That is, through this target fused feature, the geometric details that multiple convolutional layers focus on can be understood. For example, the target fused feature satisfies the following formula: H fso =αH (L) +(1-α)H (K) , where H (L) H represents the deep features output by the Lth convolutional layer. (K) Let H be the feature vector output by the Kth convolutional layer, where α represents the fusion coefficient. fso This represents the target fusion feature. Thus, during the process of deconvolution processing the deep features output from the Lth convolutional layer to obtain the model reconstruction data of the initial 3D model, the target fusion feature can be deconvolutionally processed to output the model reconstruction data of the initial 3D model.

[0140] In the above manner, since the outputs of different convolutional layers reflect the feature representations of data at different levels, this application comprehensively considers the deep features output by different convolutional layers to determine the model reconstruction data. On the one hand, this enables the model to maintain high model performance when facing noise interference and improve model robustness; on the other hand, by comprehensively considering the outputs of different convolutional layers, the model can more accurately capture the key features in the data, thereby generating higher quality reconstruction data.

[0141] In some alternative embodiments, to further enhance the detail of the reconstructed model, this application can also utilize local convolution operations to enhance the local geometric details of the model. For example, before performing deconvolution processing on the target fusion features to obtain the model reconstruction data of the initial 3D model, this application can first perform local convolution processing on the target fusion features based on the local convolution kernels in the target spherical convolution model to obtain local convolution features. Subsequently, activation processing is performed on the regularization terms of the local convolution features and the local convolution kernels to obtain local enhanced features. For example, the local enhanced features can satisfy the following formula: Where * denotes the convolution operation, σ is the non-linear activation function, and G... (lcl) This can be understood as multiple local convolutional kernels in a local convolutional layer; This represents the regularization term information of the local convolutional layer. Thus, in the process of deconvolution processing the target fusion features to obtain the model reconstruction data of the initial 3D model, this application specifically performs deconvolution processing on the target fusion features and the local convolutional features to obtain the model reconstruction data of the initial 3D model.

[0142] In some alternative embodiments, to optimize the updated target spherical convolution model, this application can further optimize the model parameters of the target spherical convolution model by calculating the difference between the reconstructed target 3D model and the initial 3D model. For example, after determining the target 3D model by reconstructing the spherical feature map based on the target spherical convolution model, this application can also calculate the mean squared error between the target 3D model and the initial 3D model. For instance, the mean squared error can be expressed by the following formula: in, M represents the mean square error. q Represents the initial 3D model. Let represent the target 3D model, and q represent the q-th iteration of the model difference calculation N. In addition to the influence of differences between models, the regularization terms added in the convolutional layers can also affect the loss difference. Therefore, this application can also sum the regularization terms of multiple convolutional layers to obtain the target regularization term information. In other words, the target regularization term information satisfies the following formula: λ is a constant. Thus, after calculating the mean squared error and the target regularization term, the target loss value can be determined based on these information. As an illustrative example, the mean squared error and the target regularization term can be summed to obtain the target loss value. For instance, the target loss value might satisfy the following formula: in, This represents the target loss value. Then, the model parameters of the target sphere convolutional model are updated based on this target loss value.

[0143] In some alternative embodiments, this application may also employ a gradient descent algorithm to update the gradients in each convolutional layer during the determination of the target loss value, in order to further optimize the model parameters of the target spherical convolutional model. For example, the weights of the l-th convolutional layer can be used as variables to perform partial derivative processing on the aforementioned target loss function. This allows for the determination of the updated weights of the l-th convolutional layer by combining the difference between the weights and the partial derivative value. For instance, the updated weights of the l-th convolutional layer can satisfy the following formula: Where η represents the learning rate, w' (l) w represents the updated weight values ​​of the l-th convolutional layer. (l) This represents the weight values ​​of the l-th convolutional layer before the update. Represents the loss function The gradient of the parameters of the l-th convolutional layer.

[0144] In some alternative embodiments, after performing step 304 to determine the target 3D model, this application may also use the texture information of the original initial 3D model to achieve texture mapping for the target 3D model, so as to preserve and reproduce high-quality texture information in the target 3D model, thereby ensuring the realism of the visual effect and the preservation of details.

[0145] Based on this, in the aforementioned Figure 3 Based on one or more embodiments shown, Figure 6 Another flowchart illustrating the model processing method provided in this application is shown. Figure 6 As shown, the method provided in this application includes at least the following steps:

[0146] 601. Perform spherical transformation on the model data of the acquired initial 3D model to obtain the spherical mesh data of the initial 3D model.

[0147] 602. Based on the spherical mesh data of the initial 3D model, generate a spherical feature map of the initial 3D model. The spherical feature map is used to characterize the features of the initial 3D model.

[0148] 603. Update the model parameters of the initial spherical convolutional model based on the convolution parameters of multiple convolutional layers and the regularization term information of each convolutional layer to obtain the target spherical convolutional model. Each convolutional layer includes multiple convolutional kernels of different sizes, and the regularization term information of each convolutional layer is calculated based on the weight values ​​of multiple convolutional kernels in the corresponding convolutional layer.

[0149] 604. Based on the target spherical convolution model, the spherical feature map is reconstructed to determine the target 3D model, which is then used to perform the rendering task.

[0150] It should be noted that steps 601 to 604 described herein can be referred to the foregoing. Figure 3 The content described in steps 301 to 304 is for your understanding and will not be repeated here.

[0151] 605. Perform texture feature extraction processing on the model data of the initial 3D model to obtain the texture feature information of the initial 3D model.

[0152] In one or more embodiments, after obtaining the model data of the initial 3D model, texture feature extraction processing can be performed on the model data of the initial 3D model to obtain the texture feature information of the initial 3D model. For example, the texture feature information of the initial 3D model includes, but is not limited to, texture color, texture coordinates, texture resolution, etc., which are not limited in this application.

[0153] 606. Resample the texture feature information of the initial 3D model to obtain the target texture feature information.

[0154] In one or more embodiments, in order to ensure the authenticity and quality of subsequent texture mapping, after extracting the texture feature information of the initial three-dimensional model, this application also needs to resample the texture feature information of the initial three-dimensional model to obtain the target texture feature information.

[0155] As an illustrative example, resampling involves recalculating texture coordinates and calculating texture colors. For instance, in calculating texture colors, the texture colors of each vertex in the initial 3D model can be resampled to determine the corresponding target texture color, such as c′. i =T(u′) i ,v′ i ), where u′ i v′ i The texture coordinates t′ after resampling i =(u′ i ,v′ i The UV coordinates in ); T represents the texture image, c′ i This represents the texture color of the vertex. Similarly, in calculating texture coordinates, the target texture coordinates of the target vertex can be obtained by extracting the color values ​​of the four surrounding pixels of the target vertex and performing bilinear interpolation on these four pixel values ​​using an interpolation factor. For example, the target texture coordinates of the target vertex satisfy the following expression: c′ i =(1-α1)(1-β1)c 00 +α1(1-β1)c 10 +(1-α1)β1)c 01 +α1β1c 11Where α1 and β1 represent interpolation factors, respectively, and c 00 c 10 c 01 c 11 It is the color value of the four pixels surrounding the target vertex.

[0156] 607. Perform texture mapping processing on the target 3D model based on the target texture feature information.

[0157] In one or more embodiments, after resampling to obtain target texture feature information, the target texture feature information can be used for texture mapping processing of the target 3D model.

[0158] In the above manner, this application utilizes the texture information of the original initial three-dimensional model to achieve texture mapping of the target three-dimensional model, so as to preserve and reproduce high-quality texture information in the target three-dimensional model, thereby ensuring the realism of the visual effect and the preservation of details.

[0159] In some alternative embodiments, to enhance the texture realism of the target 3D model, this application may also combine ambient occlusion mapping and normal mapping to achieve texture mapping processing of the target 3D model. Exemplarily, the described target texture feature information includes target texture coordinates. In the foregoing Figure 6 Based on the illustrated embodiment, after performing step 606 to resample the texture feature information of the initial 3D model to obtain the target texture feature information, this embodiment can also calculate the normal map features based on the target texture coordinates. For example, the normal map features can be represented by the following formula: Among them, t x and t y Here, n represents the coordinate value in the target texture coordinate system, and n' represents the normal map feature. It should be noted that the normal map feature described in this application can be used to describe the normal characteristics of the target 3D model.

[0160] In addition to calculating the normal map features, this application also needs to calculate the ambient occlusion features based on the vertex coordinates of multiple vertices in the target 3D model and preset ambient occlusion parameters. For example, the ambient occlusion features satisfy the following expression: Where N represents the number of neighboring vertices around the target vertex, σ1 represents the parameter controlling the occlusion range (i.e., the aforementioned preset ambient light occlusion parameter), and A i Indicates the characteristics of ambient light shading, v i v j This represents the coordinates of a neighboring vertex. It should be noted that the ambient occlusion feature of this application can be used to describe the degree to which the surface of a target 3D model is occluded by light.

[0161] Thus, in step 607, during the process of texture mapping of the target 3D model based on the target texture feature information, the texture mapping of the target 3D model can be specifically performed based on the target texture feature information, normal map features, and ambient occlusion features. As an illustrative description, the normal map features and ambient occlusion features can be synthesized with the target texture feature information to obtain synthesized texture features, which are then used to perform texture mapping on the target 3D model. For example, the synthesized texture features described can satisfy the following expression: T'(u,v)=c(u,v)·A(u,v)+n(u,v)·L1. Where c(u,v) represents the target texture feature information, A(u,v) represents the ambient occlusion feature, n(u,v) represents the normal map feature, and L1 represents the lighting direction.

[0162] By taking into account both normal map features and ambient occlusion features in the texture mapping of the target 3D model based on the target texture feature information, the realism of the target 3D model can be enhanced on the one hand, and the subsequent model rendering efficiency can be improved on the other hand, resulting in richer and more realistic visual effects.

[0163] In some alternative embodiments, to enhance and improve the effect of texture mapping, this application can also perform texture mapping at different resolutions to better preserve details. Exemplarily, the described target texture feature information includes the initial texture resolution. In the foregoing Figure 6 Based on one or more embodiments shown, after performing step 606 to resample the texture feature information of the initial 3D model to obtain the target texture feature information, this application can further calculate the resolution texture features of each layer in multiple layers based on the initial texture resolution and the preset number of resolution layers. For example, the resolution texture features of each layer can be represented in the following way: p = 0, 1, ..., P-1. Where P is the resolution layer number, R... p Let R represent the resolution texture features of the p-th layer, and R be the initial texture resolution. Thus, after calculating the resolution texture features of each layer, during the texture mapping process of the target 3D model based on the target texture feature information, upsampling is first performed on the resolution texture features of each layer, and the upsampled resolution texture features are then fused to obtain fused texture feature information. Finally, texture mapping is performed on the target 3D model based on this fused texture feature information.

[0164] By employing the above method, the texture mapping process for the target 3D model fully considers the resolution texture features of different layers. On the one hand, by applying textures of different resolutions to different parts of the model, rendering speed can be improved while ensuring visual quality, resulting in a smoother rendering scene. On the other hand, for parts requiring high detail, such as the foreground or important features of the model, using high-resolution textures can present more details and textures, enhancing visual realism. Furthermore, this application achieves a smooth transition between textures of different resolutions, avoiding visual artifacts caused by abrupt changes in texture resolution, making the model look more natural and realistic.

[0165] The foregoing primarily describes the solutions provided by the embodiments of this application from a methodological perspective. It is understood that to achieve the above functions, corresponding hardware structures and / or software modules are included to execute each function. Those skilled in the art should readily recognize that, based on the modules and algorithm steps described in conjunction with the embodiments disclosed in this application, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0166] This application embodiment can divide the device into functional modules according to the above method example. For example, each function can be divided into its own functional module, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module. It should be noted that the module division in this application embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods.

[0167] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0168] The model processing apparatus in the embodiments of this application will now be described in detail. Figure 7 This is a schematic diagram of one embodiment of the model processing apparatus provided in this application. Figure 7As shown, the model processing device includes a conversion processing unit 701, a generation processing unit 702, an update processing unit 703, and a reconstruction processing unit 704.

[0169] The conversion processing unit 701 is used to perform spherical conversion processing on the model data of the acquired initial three-dimensional model to obtain the spherical mesh data of the initial three-dimensional model.

[0170] The generation processing unit 702 is used to generate a spherical feature map of the initial three-dimensional model based on the spherical mesh data of the initial three-dimensional model. The spherical feature map is used to characterize the features of the initial three-dimensional model.

[0171] The update processing unit 703 is used to update the model parameters of the initial spherical convolution model based on the convolution parameters of multiple convolutional layers and the regularization term information of each convolutional layer to obtain the target spherical convolution model. Each convolutional layer includes multiple convolutional kernels of different sizes, and the regularization term information of each convolutional layer is calculated based on the weight values ​​of the multiple convolutional kernels in the corresponding convolutional layer.

[0172] The reconstruction processing unit 704 is used to perform model reconstruction processing on the spherical feature map based on the target spherical convolution model to determine the target three-dimensional model, which is used to perform rendering tasks.

[0173] In combination with the above Figure 7 In one or more examples shown, and in some other optional embodiments, the model processing apparatus provided in this application may further include an acquisition unit 705 and a calculation processing unit 706.

[0174] The acquisition unit 705 is used to update the model parameters of the initial spherical convolution model based on the convolution parameters of multiple convolutional layers and the regularization term information of each convolutional layer. Before obtaining the target spherical convolution model, the convolution parameters of multiple convolutional layers and the convolution information of the initial convolution kernel of the initial spherical convolution model are acquired. The size of the multiple convolution kernels in each convolutional layer is different from the size of the initial convolution kernel.

[0175] The computational processing unit 706 is used to calculate the regularization term information corresponding to the convolutional layer based on the weight values ​​of the multiple convolutional kernels in each convolutional layer;

[0176] Update processing unit 703 is specifically used for:

[0177] Based on the convolution parameters of each convolutional layer and the regularization term information of the corresponding convolutional layer, the target convolution information of each convolutional layer is determined;

[0178] The target convolutional information of multiple convolutional layers is superimposed, and the convolutional information of the initial convolutional kernel in the initial spherical convolutional model is replaced by the superimposed target convolutional information to obtain the target spherical convolutional model. The input of the first convolutional layer of the target spherical convolutional model is the spherical feature map, and the input of the Lth convolutional layer of the target spherical convolutional model is the feature output by the (L-1)th convolutional kernel, the size of the Lth convolutional layer, and the regularization term information of the Lth convolutional layer, where L is an integer greater than or equal to 2.

[0179] In combination with the above Figure 7 In one or more examples shown, in some alternative embodiments, the update processing unit 703 is specifically used for:

[0180] The spherical feature map is used as the input of the target sphere convolution model. The spherical feature map is convolved by multiple superimposed convolutional layers in the target sphere convolution model to obtain the deep features output by the Lth convolutional layer.

[0181] The deep features output by the Lth convolutional layer are deconvolutionally processed to obtain the model reconstruction data of the initial 3D model;

[0182] The target 3D model is generated from the model reconstruction data based on the initial 3D model.

[0183] In combination with the above Figure 7 In one or more examples shown, in some alternative embodiments, the update processing unit 703 is specifically used for:

[0184] The spherical feature map is used as the input to the first convolutional layer in the target sphere convolutional model. The spherical feature map is convolved by multiple convolutional kernels of the specified size in the first convolutional layer to obtain the feature map output by multiple convolutional kernels in the Lth convolutional layer. The feature map output by multiple convolutional kernels and the regularization term information of the first convolutional layer are activated to obtain the output feature map of the first convolutional layer.

[0185] The output feature map of the first convolutional layer is used as the input of the Lth convolutional layer in the target sphere convolutional model. The output feature map of the first convolutional layer is convolved by multiple convolutional kernels of the specified size in the Lth convolutional layer to obtain the feature maps output by the multiple convolutional kernels in the Lth convolutional layer. The feature maps output by the multiple convolutional kernels are then activated with the regularization term information of the Lth convolutional layer to obtain the deep features output by the Lth convolutional layer.

[0186] In combination with the above Figure 7In one or more examples shown, in some alternative embodiments, the update processing unit 703 is specifically used for:

[0187] The spherical feature map is used as the input of the target spherical convolutional model. The spherical feature map is convolved by multiple superimposed convolutional layers in the target spherical convolutional model to obtain the deep features output by the Lth convolutional layer. Then, the deep features output by the Lth convolutional layer and the feature vector output by the Kth convolutional layer are fused based on a preset fusion coefficient to obtain the target fused features, where K is a positive integer not equal to L.

[0188] The target fusion features are deconvolutionally processed to obtain the model reconstruction data of the initial 3D model.

[0189] In combination with the above Figure 7 In one or more examples shown, in some alternative embodiments, the update processing unit 703 is specifically used for:

[0190] Before performing deconvolution processing on the target fusion features to obtain the model reconstruction data of the initial 3D model, local convolution processing is performed on the target fusion features based on the local convolution kernel in the target sphere convolution model to obtain local convolution features;

[0191] The local convolutional features and the regularization term information of the local convolutional kernel are activated to obtain local enhanced features;

[0192] The target fusion features and the local convolution features are deconvolutionally processed to obtain the model reconstruction data of the initial 3D model.

[0193] In combination with the above Figure 7 In one or more examples shown, and in some other optional embodiments, the model processing apparatus provided in this application further includes a feature extraction unit 707, a sampling processing unit 708, and a texture mapping unit 709.

[0194] The feature extraction unit 707 is specifically used to perform texture feature extraction processing on the model data of the initial three-dimensional model after the model reconstruction processing of the spherical feature map based on the target spherical convolution model is performed to determine the target three-dimensional model, thereby obtaining the texture feature information of the initial three-dimensional model.

[0195] The sampling processing unit 708 is specifically used to resample the texture feature information of the initial three-dimensional model to obtain the target texture feature information;

[0196] The texture mapping unit 709 is specifically used to perform texture mapping processing on the target 3D model based on the target texture feature information.

[0197] In combination with the above Figure 7 In one or more examples shown, in some alternative embodiments, the target texture feature information includes target texture coordinates;

[0198] The computational processing unit 706 is further configured to, after resampling the texture feature information of the initial three-dimensional model to obtain the target texture feature information, calculate the normal map feature based on the target texture coordinates, wherein the normal map feature is used to describe the normal situation of the target three-dimensional model;

[0199] The computational processing unit 706 is specifically used to calculate ambient light occlusion features based on the vertex coordinates of multiple vertices in the target 3D model and preset ambient light occlusion parameters. The ambient light occlusion features are used to describe the degree to which the surface of the target 3D model is occluded by light.

[0200] The texture mapping unit 709 is specifically used to perform texture mapping processing on the target 3D model based on the target texture feature information, the normal map features and the ambient light occlusion features.

[0201] In combination with the above Figure 7 In one or more examples shown, in some alternative embodiments, the target texture feature information includes an initial texture resolution; the calculation processing unit 706 is further configured to, after resampling the texture feature information of the initial three-dimensional model to obtain the target texture feature information, calculate the resolution texture features of each of the multiple layers based on the initial texture resolution and a preset number of resolution layers.

[0202] The sampling processing unit 708 is specifically used to perform upsampling processing based on the resolution texture features of each level, and to perform fusion processing on the resolution texture features obtained by the upsampling processing to obtain fused texture feature information;

[0203] The texture mapping unit 709 is specifically used to perform texture mapping processing on the target 3D model based on the fused texture feature information.

[0204] In combination with the above Figure 7 In one or more examples shown, in some alternative embodiments, the computation processing unit 706 is further configured to:

[0205] After determining the target 3D model by reconstructing the spherical feature map based on the target spherical convolution model, the mean square error between the target 3D model and the initial 3D model is calculated.

[0206] The regularization terms of multiple convolutional layers are summed to obtain the target regularization terms.

[0207] The target loss value is determined based on the mean square error and the target regularization term information;

[0208] The update processing unit 703 is specifically used to update the model parameters of the target sphere convolution model based on the target loss value.

[0209] In combination with the above Figure 7 In one or more examples shown, and in some alternative embodiments, the model data of the initial 3D model includes the vertex coordinates of the initial 3D model; the transformation processing unit 701 is specifically used for:

[0210] The vertex coordinates are calculated based on the preset four-quadrant arctangent function to obtain the longitude coordinates of the spherical grid points in the spherical coordinate system corresponding to the target sphere.

[0211] The vertex coordinates are calculated and processed based on a preset inverse cosine function to obtain the latitude coordinates of the vertex coordinates as spherical grid points in the spherical coordinate system.

[0212] Based on the longitude and latitude coordinates, the Cartesian coordinates of the vertex coordinates in the spherical coordinate system are calculated to obtain the spherical mesh data of the initial three-dimensional model.

[0213] In combination with the above Figure 7 In one or more examples shown, in some alternative embodiments, the generation processing unit 702 is specifically used for:

[0214] The Cartesian coordinates are subjected to spherical position feature extraction processing to obtain the position feature vector of the vertex coordinates;

[0215] The texture map coordinates of the initial 3D model are determined based on the Cartesian coordinates, and the normal feature vector is determined based on the texture map coordinates.

[0216] Calculate the curvature feature vector of the target sphere based on the radius information of the target sphere;

[0217] Based on the position feature vector, the normal feature vector, and the curvature feature vector, a spherical feature map of the initial three-dimensional model is generated.

[0218] In combination with the above Figure 7 In one or more examples shown, in some alternative embodiments, the computation processing unit 706 is further configured to:

[0219] Before generating the spherical feature map of the initial three-dimensional model based on the position feature vector, the normal feature vector, and the curvature feature vector, the geometric feature vector of the first interpolation point is calculated based on a preset interpolation function to determine the interpolation value of the first interpolation point. The first interpolation point is a known vertex of the target sphere. The geometric feature vector includes one or more of the position feature vector, the normal feature vector, and the curvature feature vector.

[0220] Based on the interpolation value of the first interpolation point and the interpolation weight value between the first interpolation point and the second interpolation point, the interpolation value of the second interpolation point is calculated to obtain the geometric feature vector of the second interpolation point. The second interpolation point is an unknown vertex of the target sphere.

[0221] The generation processing unit 702 is specifically used to generate a spherical feature map of the initial three-dimensional model based on the position feature vector, the normal feature vector, the curvature feature vector, and the geometric feature vector of the second interpolation point.

[0222] The model processing device in the embodiments of this application has been described above from the perspective of modular functional entities. The computer device in the embodiments of this application is described below from the perspective of hardware processing. Figure 8 This is a schematic diagram of a computer device provided in an embodiment of this application. The computer device can vary considerably due to differences in configuration or performance, for example, including but not limited to... Figure 7 The model processing device shown is an example.

[0223] like Figure 8 As shown, the computer device can vary significantly due to differences in configuration or performance, and may include one or more central processing units (CPUs) 322 (e.g., one or more processors) and memory 332, and one or more storage media 330 (e.g., one or more mass storage devices) for storing application programs 342 or data 344. The memory 332 and storage media 330 can be temporary or persistent storage. The program stored in the storage media 330 may include one or more modules (not shown in the figure), each module including a series of instruction operations on the processing device. Furthermore, the CPU 322 may be configured to communicate with the storage media 330 and execute a series of instruction operations stored in the storage media 330 on the computer device. Exemplarily, the CPU 322 is used to execute the application program 342 stored in the storage media 330, thereby implementing the model processing method provided in the above embodiments of this application.

[0224] The computer device may also include one or more power supplies 326, one or more wired or wireless network interfaces 350, one or more input / output interfaces 358, and / or one or more operating systems 341, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0225] For example, Figure 8 The central processing unit 322 can call computer execution instructions stored in memory 332 to cause the computer device to perform actions such as... Figures 3 to 6 The method in the corresponding method embodiment.

[0226] Specifically, Figure 7 The functions / implementation processes of the transformation processing unit 701, generation processing unit 702, update processing unit 703, reconstruction processing unit 704, feature extraction unit 707, calculation processing unit 706, sampling processing unit 708, and texture mapping unit 709 can be understood through... Figure 8 The central processing unit 322 in the memory calls computer execution instructions stored in the memory 332 to achieve this. Figure 7 The function / implementation process of the acquisition unit 705 can be achieved through... Figure 8 It is implemented using the input / output interface 358.

[0227] The steps performed by the computer device in the above embodiments can be based on this Figure 8 The computer device structure shown.

[0228] This application also provides a computer-readable storage medium storing a computer program or instructions thereon, which, when executed by a processor, implements the steps of the methods described in the foregoing embodiments.

[0229] This application also provides a computer program product, including a computer program or instructions, which, when executed by a processor, implement the steps of the methods described in the foregoing embodiments.

[0230] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0231] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0232] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between apparatuses or units, and may be electrical, mechanical, or other forms.

[0233] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0234] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0235] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0236] A computer program product includes one or more computer instructions. When these computer instructions are loaded and executed on a computer, they generate, in whole or in part, the processes or functions according to embodiments of this application. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions may be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., SSDs), etc.

[0237] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for model processing, characterized in that, include: The model data of the acquired initial three-dimensional model is subjected to spherical transformation processing to obtain the spherical mesh data of the initial three-dimensional model; Based on the spherical mesh data of the initial 3D model, a spherical feature map of the initial 3D model is generated, and the spherical feature map is used to characterize the features of the initial 3D model; The model parameters of the initial spherical convolution model are updated based on the convolution parameters of multiple convolutional layers and the regularization information of each convolutional layer to obtain the target spherical convolution model. Each convolutional layer includes multiple convolutional kernels of different sizes, and the regularization information of each convolutional layer is calculated based on the weight values ​​of the multiple convolutional kernels in the corresponding convolutional layer. Based on the target spherical convolution model, the spherical feature map is reconstructed to determine the target 3D model, which is then used to perform rendering tasks.

2. The method according to claim 1, characterized in that, Before updating the model parameters of the initial spherical convolution model based on the convolution parameters of multiple convolutional layers and the regularization term information of each convolutional layer to obtain the target spherical convolution model, the method further includes: Obtain the convolution parameters of multiple convolutional layers and the convolution information of the initial convolutional kernel of the initial spherical convolutional model, wherein the size of the multiple convolutional kernels in each convolutional layer is different from the size of the initial convolutional kernel; Based on the weight values ​​of the multiple convolutional kernels in each convolutional layer, calculate the regularization term information corresponding to the convolutional layer; The model parameters of the initial spherical convolution model are updated based on the convolution parameters of multiple convolutional layers and the regularization term information of each convolutional layer to obtain the target spherical convolution model, including: Based on the convolution parameters of each convolutional layer and the regularization term information of the corresponding convolutional layer, the target convolution information of each convolutional layer is determined; The target convolutional information of multiple convolutional layers is superimposed, and the convolutional information of the initial convolutional kernel in the initial spherical convolutional model is replaced by the superimposed target convolutional information to obtain the target spherical convolutional model. The input of the first convolutional layer of the target spherical convolutional model is the spherical feature map, and the input of the Lth convolutional layer of the target spherical convolutional model is the feature output by the (L-1)th convolutional kernel, the size of the Lth convolutional layer, and the regularization term information of the Lth convolutional layer, where L is an integer greater than or equal to 2.

3. The method according to claim 2, characterized in that, Based on the target spherical convolutional model, the spherical feature map is reconstructed to determine the target 3D model, including: The spherical feature map is used as the input of the target sphere convolution model. The spherical feature map is convolved by multiple superimposed convolutional layers in the target sphere convolution model to obtain the deep features output by the Lth convolutional layer. The deep features output by the Lth convolutional layer are deconvolutionally processed to obtain the model reconstruction data of the initial 3D model; The target 3D model is generated from the model reconstruction data based on the initial 3D model.

4. The method according to claim 3, characterized in that, The spherical feature map is used as input to the target sphere convolutional model. Multiple stacked convolutional layers in the target sphere convolutional model are used to convolve the spherical feature map to obtain the deep features output by the Lth convolutional layer, including: The spherical feature map is used as the input to the first convolutional layer in the target sphere convolutional model. The spherical feature map is convolved by multiple convolutional kernels of the specified size in the first convolutional layer to obtain the feature map output by multiple convolutional kernels in the Lth convolutional layer. The feature map output by multiple convolutional kernels and the regularization term information of the first convolutional layer are activated to obtain the output feature map of the first convolutional layer. The output feature map of the first convolutional layer is used as the input of the Lth convolutional layer in the target sphere convolutional model. The output feature map of the first convolutional layer is convolved by multiple convolutional kernels of the specified size in the Lth convolutional layer to obtain the feature maps output by the multiple convolutional kernels in the Lth convolutional layer. The feature maps output by the multiple convolutional kernels are then activated with the regularization term information of the Lth convolutional layer to obtain the deep features output by the Lth convolutional layer.

5. The method according to claim 4, characterized in that, The method further includes using the spherical feature map as input to the target sphere convolutional model, performing convolution processing on the spherical feature map through multiple stacked convolutional layers in the target sphere convolutional model to obtain the deep features output by the Lth convolutional layer: The deep features output by the Lth convolutional layer and the feature vector output by the Kth convolutional layer are fused based on a preset fusion coefficient to obtain the target fused features, where K is a positive integer not equal to L. The deep features output from the Lth convolutional layer are deconvolutionally processed to obtain the model reconstruction data of the initial 3D model, including: The target fusion features are deconvolutionally processed to obtain the model reconstruction data of the initial 3D model.

6. The method according to claim 5, characterized in that, Before performing deconvolution processing on the target fusion features to obtain the model reconstruction data of the initial 3D model, the method further includes: Based on the local convolution kernel in the target sphere convolution model, the target fusion features are subjected to local convolution processing to obtain local convolution features; The local convolutional features and the regularization term information of the local convolutional kernel are activated to obtain local enhanced features; The target fusion features are deconvolutionally processed to obtain the model reconstruction data of the initial 3D model, including: The target fusion features and the local convolution features are deconvolutionally processed to obtain the model reconstruction data of the initial 3D model.

7. The method according to any one of claims 1 to 6, characterized in that, After reconstructing the spherical feature map based on the target spherical convolution model to determine the target 3D model, the method further includes: The initial three-dimensional model data is subjected to texture feature extraction processing to obtain the texture feature information of the initial three-dimensional model; The texture feature information of the initial 3D model is resampled to obtain the target texture feature information; The target 3D model is subjected to texture mapping processing based on the target texture feature information.

8. The method according to claim 7, characterized in that, The target texture feature information includes target texture coordinates; after resampling the texture feature information of the initial 3D model to obtain the target texture feature information, the method further includes: The normal map features are calculated based on the target texture coordinates, and the normal map features are used to describe the normal characteristics of the target 3D model. Based on the vertex coordinates of multiple vertices in the target 3D model and preset ambient light occlusion parameters, ambient light occlusion features are calculated. These ambient light occlusion features are used to describe the degree to which the surface of the target 3D model is occluded by light. Based on the target texture feature information, texture mapping processing is performed on the target 3D model, including: Based on the target texture feature information, the normal map features, and the ambient occlusion features, texture mapping processing is performed on the target 3D model.

9. The method according to claim 7, characterized in that, The target texture feature information includes the initial texture resolution; after resampling the texture feature information of the initial 3D model to obtain the target texture feature information, the method further includes: Based on the initial texture resolution and the preset number of resolution levels, calculate the resolution texture features of each of the multiple levels; Based on the target texture feature information, texture mapping processing is performed on the target 3D model, including: Upsampling is performed on the resolution texture features of each level, and the resolution texture features obtained from the upsampling are then fused to obtain fused texture feature information. The target 3D model is subjected to texture mapping processing based on the fused texture feature information.

10. The method according to any one of claims 1 to 9, characterized in that, After determining the target 3D model by reconstructing the spherical feature map based on the target spherical convolution model, the method further includes: Calculate the mean square error between the target 3D model and the initial 3D model; The regularization terms of multiple convolutional layers are summed to obtain the target regularization terms. The target loss value is determined based on the mean square error and the target regularization term information; The model parameters of the target sphere convolution model are updated based on the target loss value.

11. The method according to claim 1, characterized in that, The model data of the initial 3D model includes the vertex coordinates of the initial 3D model; the model data of the acquired initial 3D model is subjected to spherical transformation processing to obtain the spherical mesh data of the initial 3D model, including: The vertex coordinates are calculated based on the preset four-quadrant arctangent function to obtain the longitude coordinates of the spherical grid points in the spherical coordinate system corresponding to the target sphere. The vertex coordinates are calculated and processed based on a preset inverse cosine function to obtain the latitude coordinates of the vertex coordinates as spherical grid points in the spherical coordinate system. Based on the longitude and latitude coordinates, the Cartesian coordinates of the vertex coordinates in the spherical coordinate system are calculated to obtain the spherical mesh data of the initial three-dimensional model.

12. The method according to claim 11, characterized in that, Based on the spherical mesh data of the initial 3D model, a spherical feature map of the initial 3D model is generated, including: The Cartesian coordinates are subjected to spherical position feature extraction processing to obtain the position feature vector of the vertex coordinates; The texture map coordinates of the initial 3D model are determined based on the Cartesian coordinates, and the normal feature vector is determined based on the texture map coordinates. Calculate the curvature feature vector of the target sphere based on the radius information of the target sphere; Based on the position feature vector, the normal feature vector, and the curvature feature vector, a spherical feature map of the initial three-dimensional model is generated.

13. The method according to claim 12, characterized in that, Before generating the spherical feature map of the initial 3D model based on the position feature vector, the normal feature vector, and the curvature feature vector, the method further includes: The geometric feature vector of the first interpolation point is calculated based on a preset interpolation function to determine the interpolation value of the first interpolation point. The first interpolation point is a known vertex of the target sphere. The geometric feature vector includes one or more of the position feature vector, the normal feature vector, and the curvature feature vector. Based on the interpolation value of the first interpolation point and the interpolation weight value between the first interpolation point and the second interpolation point, the interpolation value of the second interpolation point is calculated to obtain the geometric feature vector of the second interpolation point. The second interpolation point is an unknown vertex of the target sphere. Based on the position feature vector, the normal feature vector, and the curvature feature vector, a spherical feature map of the initial 3D model is generated, including: Based on the position feature vector, the normal feature vector, the curvature feature vector, and the geometric feature vector of the second interpolation point, a spherical feature map of the initial three-dimensional model is generated.

14. A model processing device, characterized in that, include: The conversion processing unit is used to perform spherical conversion processing on the model data of the acquired initial three-dimensional model to obtain the spherical mesh data of the initial three-dimensional model. A generation processing unit is used to generate a spherical feature map of the initial three-dimensional model based on the spherical mesh data of the initial three-dimensional model. The spherical feature map is used to characterize the features of the initial three-dimensional model. An update processing unit is used to update the model parameters of the initial spherical convolution model based on the convolution parameters of multiple convolutional layers and the regularization term information of each convolutional layer, so as to obtain the target spherical convolution model. Each convolutional layer includes multiple convolutional kernels of different sizes, and the regularization term information of each convolutional layer is calculated based on the weight values ​​of the multiple convolutional kernels in the corresponding convolutional layer. The reconstruction processing unit is used to perform model reconstruction processing on the spherical feature map based on the target spherical convolution model to determine the target three-dimensional model, which is used to perform rendering tasks.

15. A computer device, characterized in that, include: A processor and a memory, wherein the memory stores program instructions; The processor is configured to execute program instructions stored in the memory to perform the method as described in any one of claims 1 to 13.

16. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes instructions that, when executed on a computer device, cause the computer device to perform the method as described in any one of claims 1 to 13.

17. A computer program product, characterized in that, The computer program product includes instructions that, when executed on a computer device, cause the computer device to perform the method as described in any one of claims 1 to 13.

Images