Three-dimensional model processing method and device, equipment and storage medium

Abstract

The invention discloses a three-dimensional model processing method and device, equipment and a storage medium, and belongs to the technical field of model rendering. The method comprises the following steps: obtaining a model structure of a three-dimensional model, view angle images of an expected texture of the three-dimensional model under n view angles and confidence weights of a vertex in the three-dimensional model under the n view angles, wherein the confidence weight corresponding to a first view angle in the n view angles is in negative correlation with the offset degree of the vertex observed under the first view angle; taking the confidence weights under the n visual angles as color weights of visual angle images under the n visual angles, and performing texture baking on a model structure of the three-dimensional model based on the visual angle images under the n visual angles to obtain a three-dimensional color model; obtaining an observation image of the three-dimensional color model under n visual angles, and correcting the visual angle image according to the difference between the observation image and the visual angle image to obtain a corrected visual angle image; and performing texture baking on the model structure of the three-dimensional model based on the corrected visual angle image to obtain a corrected color model.

Description

Technical Field

[0001] The present application relates to the field of model rendering technology, and in particular to a three-dimensional model processing method, device, equipment and storage medium. Background Art

[0002] With the development of computer technology, the functions of application programs are becoming more and more diverse.

[0003] In the related art, texture baking is performed on a three-dimensional model based on a two-dimensional image, and the colors in the two-dimensional image are attached to the three-dimensional model to achieve texture filling of the three-dimensional model.

[0004] However, as the structural complexity of three-dimensional models increases, how to improve the accuracy of texture filling is an urgent problem to be solved. Summary of the invention

[0005] The present application provides a three-dimensional model processing method, device, equipment and storage medium, and the technical solution is as follows:

[0006] According to one aspect of the present application, a method for processing a three-dimensional model is provided, the method being executed by a computer device, the method comprising:

[0007] Acquire a model structure of the three-dimensional model, perspective images of the expected texture of the three-dimensional model at n perspectives, and confidence weights of vertices in the three-dimensional model at the n perspectives, where n is an integer greater than 1, and the confidence weight corresponding to a first perspective among the n perspectives is negatively correlated with a degree of deviation of the vertex observed at the first perspective;

[0008] The confidence weights at the n viewing angles are respectively used as color weights of the viewing images at the n viewing angles, and texture baking is performed on the model structure of the three-dimensional model based on the viewing images at the n viewing angles to obtain a three-dimensional color model;

[0009] Acquire observation images of the three-dimensional color model at the n viewing angles, and perform correction on the viewing angle images according to the difference between the observation images and the viewing angle images to obtain corrected viewing angle images;

[0010] Texture baking is performed on the model structure of the three-dimensional model based on the corrected viewing angle image to obtain a corrected color model.

[0011] According to another aspect of the present application, a three-dimensional model processing device is provided, the device comprising:

[0012] an acquisition module, used to acquire a model structure of the three-dimensional model, view images of the expected texture of the three-dimensional model at n view angles, and confidence weights of vertices in the three-dimensional model at the n view angles, where n is an integer greater than 1, and the confidence weight corresponding to a first view angle among the n view angles is negatively correlated with a degree of deviation of the vertex observed at the first view angle;

[0013] a processing module, configured to use the confidence weights at the n viewing angles as color weights of the viewing images at the n viewing angles, respectively, and perform texture baking on the model structure of the three-dimensional model based on the viewing images at the n viewing angles to obtain a three-dimensional color model;

[0014] a correction module, used for acquiring an observed image of the three-dimensional color model at the n viewing angles, and performing correction on the viewing angle image according to a difference between the observed image and the viewing angle image to obtain a corrected viewing angle image;

[0015] The processing module is further used to perform texture baking on the model structure of the three-dimensional model based on the corrected perspective image to obtain a corrected color model.

[0016] In an optional design of the present application, the processing module is further used for:

[0017] Based on the confidence weight at the i-th viewing angle, perform color weighting on the viewing image at the i-th viewing angle to obtain a weighted viewing image at the i-th viewing angle, where i is a positive integer not exceeding n;

[0018] Increasing the value of i, repeating the above steps, until the weighted viewing angle images under the n viewing angles are obtained;

[0019] According to the weighted viewing angle images under the n viewing angles, texture baking is performed on the model structure of the three-dimensional model to obtain the three-dimensional color model.

[0020] In an optional design of the present application, the processing module is further used for:

[0021] Constructing a weight matrix at the i-th viewing angle according to the confidence weights of the vertices in the three-dimensional model at the i-th viewing angle;

[0022] Based on the dot product between the weight matrix and the color matrix of the viewing image at the i-th viewing angle, a weighted viewing image at the i-th viewing angle is determined.

[0023] In an optional design of the present application, the processing module is further used for:

[0024] According to the mapping positions of a vertices in the three-dimensional model on the two-dimensional plane, the values ​​of a vertex matrix elements in the initial matrix are determined as the confidence weights of the a vertices at the i-th viewing angle, where a is an integer greater than 1;

[0025] According to the values ​​of the a vertex matrix elements and based on the positions of the a vertex matrix elements in the initial matrix, the values ​​of the model matrix elements whose matrix positions are between the a vertex matrix elements are obtained with numerical smoothness constraints, and the weight matrix at the i-th perspective is constructed based on the a vertex matrix elements and the model matrix elements.

[0026] In an optional design of the present application, the processing module is further used for:

[0027] Performing image segmentation on the viewing images at the n viewing angles respectively to obtain model image regions at the n viewing angles;

[0028] Performing edge erosion on the model image regions under the n viewing angles respectively to obtain the main image regions under the n viewing angles;

[0029] The confidence weights under the n viewing angles are respectively used as color weights of the main image area under the n viewing angles, and texture baking is performed on the model structure of the three-dimensional model based on the main image area under the n viewing angles to obtain the three-dimensional color model.

[0030] In an optional design of the present application, the acquisition module is further used for:

[0031] Obtaining the normal direction of each vertex in the three-dimensional model;

[0032] Obtaining an i-th observation position corresponding to an i-th viewing angle among the n viewing angles, where i is a positive integer not exceeding n;

[0033] Determining a confidence weight of the vertex according to a direction difference between an observation direction of a vertex in the three-dimensional model at the i-th observation position and the normal direction of the vertex;

[0034] Increase the value of i and repeat the above two steps until the confidence weights under the n perspectives are obtained.

[0035] In an optional design of the present application, the acquisition module is further used for:

[0036] Calculating the cosine value between the viewing direction and the normal direction;

[0037] The absolute value of the cosine value is determined as an independent variable of a normalization function, and the dependent variable of the normalization function is determined as the confidence weight of the vertex.

[0038] In an optional design of the present application, the acquisition module is further used for:

[0039] Obtaining a depth map of the three-dimensional model at the n viewing angles and a guide text of the desired texture;

[0040] An image generation network is called to perform prediction on the depth maps under the n viewing angles and the guide text to obtain viewing images under the n viewing angles.

[0041] In an optional design of the present application, the image generation network includes a text encoding subnetwork, an image encoding subnetwork and a prediction subnetwork;

[0042] The acquisition module is also used for:

[0043] Calling a text encoding sub-network to encode the guide text to obtain semantic features; and calling an image encoding sub-network to encode the depth images under the n viewing angles to obtain structural features;

[0044] The prediction subnetwork is called to perform prediction on the semantic features and the structural features to obtain the perspective images under the n perspectives.

[0045] In an optional design of the present application, the acquisition module is further used for:

[0046] Acquire a sample depth map of the sample model at the n viewing angles, a sample image of the sample texture of the sample model at the n viewing angles, and a sample guide text of the sample texture;

[0047] Calling the initial generation model to perform prediction on the sample depth maps under the n viewing angles and the sample guide text to obtain predicted viewing angle images under the n viewing angles;

[0048] Based on the difference between the predicted perspective images under the n perspectives and the sample images under the n perspectives, the initial generation model is trained to obtain the image generation network.

[0049] In an optional design of the present application, the acquisition module is further used to obtain three-dimensional position information of a color-vacant region in the corrected color model, wherein the color-vacant region is a region in the three-dimensional model where the texture is empty after texture baking is performed;

[0050] The processing module is further used to determine a spatially adjacent area in the modified color model based on the three-dimensional position information;

[0051] The processing module is further used to update the modified color model by filling the texture of the color vacant area on the modified color model based on the spatial adjacent area.

[0052] According to another aspect of the present application, a computer device is provided, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set are loaded and executed by the processor to implement the three-dimensional model processing method as described above.

[0053] According to another aspect of the present application, a computer-readable storage medium is provided, in which at least one instruction, at least one program, a code set or an instruction set is stored, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the three-dimensional model processing method as described above.

[0054] According to another aspect of the present application, a computer program product is provided, which includes computer instructions stored in a computer-readable storage medium, and a processor reads and executes the computer instructions from the computer-readable storage medium to implement the three-dimensional model processing method described above.

[0055] The beneficial effects of the technical solution provided by this application include at least:

[0056] By taking the confidence weight as the color weight of the perspective image, the degree of trust in the color information in the perspective image is quantitatively described according to the confidence weight used to indicate the degree of deviation of the observed vertex, thereby realizing the adjustment of the color information of the perspective image based on the confidence weight; reducing the impact of inaccurate texture baking caused by the large deviation of the observed vertex during the texture baking process; and adjusting the observed image through the difference between the observed image and the perspective image of the three-dimensional color model obtained through texture baking, thereby improving the accuracy of texture baking for the three-dimensional model. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0058] Figure 1 is a schematic diagram of a computer system provided by an exemplary embodiment of the present application;

[0059] Figure 2 is a schematic diagram of a method for processing a three-dimensional model provided by an exemplary embodiment of the present application;

[0060] Figure 3 is a flow chart of a method for processing a three-dimensional model provided by an exemplary embodiment of the present application;

[0061] Figure 4 is a flow chart of a method for processing a three-dimensional model provided by an exemplary embodiment of the present application;

[0062] Figure 5 is a schematic diagram of a weighted viewing angle image provided by an exemplary embodiment of the present application;

[0063] Figure 6 is a schematic diagram of confidence weights provided by an exemplary embodiment of the present application;

[0064] Figure 7 is a flow chart of a method for processing a three-dimensional model provided by an exemplary embodiment of the present application;

[0065] Figure 8 is a schematic diagram of a subject image area provided by an exemplary embodiment of the present application;

[0066] Fig. 9 is a flow chart of a method for processing a three-dimensional model provided by an exemplary embodiment of the present application;

[0067] Fig.10 is a schematic diagram of a depth map and a perspective image provided by an exemplary embodiment of the present application;

[0068] Fig.11 is a schematic diagram of a depth map provided by an exemplary embodiment of the present application;

[0069] Fig.12 is a flow chart of a method for processing a three-dimensional model provided by an exemplary embodiment of the present application;

[0070] Fig.13 is a schematic diagram of a modified color model provided by an exemplary embodiment of the present application;

[0071] Fig.14 is a schematic diagram of a modified color model provided by an exemplary embodiment of the present application;

[0072] Fig.15 is a structural block diagram of a three-dimensional model processing device provided by an exemplary embodiment of the present application;

[0073] Fig.16 It is a structural block diagram of a server provided by an exemplary embodiment of the present application.

[0074] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application. DETAILED DESCRIPTION

[0075] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0076] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0077] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0078] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions. For example, the model structure, view image and other information involved in this application are all obtained with full authorization.

[0079] It should be understood that, although the terms first, second, etc. may be used in the present application to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present application, the first parameter may also be referred to as the second parameter, and similarly, the second parameter may also be referred to as the first parameter. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0080] Figure 1 A schematic diagram of a computer system provided by an embodiment of the present application is shown. The computer system can be implemented as a system architecture of a three-dimensional model processing method. The computer system may include: a terminal 100 and a server 200.

[0081] The terminal 100 may be an electronic device such as a mobile phone, a tablet computer, a PC (Personal Computer), etc. A client running a target application may be installed in the terminal 100. The target application may be a 3D model processing application or other application that provides a 3D model processing function (such as a 3D model modeling application), which is not limited in this application. In addition, this application does not limit the form of the target application, including but not limited to App (Application) installed in the terminal 100, applets, etc., and may also be in the form of a web page.

[0082] The server 200 may be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud computing services. The server 200 may be a background server of the target application, used to provide background services for the client of the target application.

[0083] In the three-dimensional model processing method provided in the embodiment of the present application, the execution subject of each step may be a computer device, which refers to an electronic device with data calculation, processing and storage capabilities. Figure 1 Taking the implementation environment of the solution shown as an example, the three-dimensional model processing method can be executed by the terminal 100 (such as the client of the target application installed and running in the terminal 100 executes the three-dimensional model processing method), or the three-dimensional model processing method can be executed by the server 200, or the terminal 100 and the server 200 can interact and cooperate to execute it, and this application does not limit this.

[0084] In addition, the technical solution of the present application can be combined with blockchain technology. For example, in the method for processing a three-dimensional model disclosed in the present application, some data involved (such as structural information of the three-dimensional model, baking parameters of texture baking, etc.) can be stored on the blockchain. The terminal 100 and the server 200 can communicate through a network, such as a wired or wireless network.

[0085] Figure 2 A schematic diagram of a three-dimensional model processing method provided by an exemplary embodiment of the present application is shown.

[0086] The depth map 202 of the three-dimensional model under four viewing angles and the guide text 204 of the desired texture of the three-dimensional model are obtained. Taking the three-dimensional model as an example, the depth map 202 is used to indicate the distance from the virtual character to the imaging plane, and is used to indicate the spatial structure information of the virtual character. The guide text 204 is used to describe the appearance style of the desired texture in a natural language, such as describing at least one of the color, reflective material, pattern, and fill pattern of the desired texture. For example, the guide text 204 includes cartoon-style clothing, short-sleeved tops and wide leather shoes, the color of the hat is very different from the color of the clothes, and the clothing is made of cloth with only a few wrinkles.

[0087] The image generation network 310 is called to perform prediction on the depth map 202 and the guide text 204 to obtain the perspective images 212 under four perspectives; the image generation network 310 has the ability to generate image information based on the image and text, wherein the depth map 202 provides constraints for the shape of the generated perspective image 212, and the guide text 204 provides constraints for the appearance style of the desired texture in the perspective image.

[0088] The confidence weight 220 of the vertex in the three-dimensional model is obtained. The confidence weight 220 decreases as the directional difference between the observation direction and the normal direction of the vertex increases, showing a negative correlation. A weight matrix 222 is constructed based on the confidence weight 220; a matrix element in the weight matrix 222 is used to indicate the weight information for performing color weighting on a pixel point or a pixel block composed of multiple pixels in the perspective image at the i-th perspective. Based on the dot product between the weight matrix 222 and the color matrix of the perspective image 212, the product between the elements in the same matrix position in the weight matrix and the color matrix can be obtained, so as to realize weighting of the color information of the perspective image based on the confidence weight.

[0089] Based on the weight matrix 222, the color information of the color matrix of the perspective image 212 is weighted, and texture baking is performed on the model structure 230 of the three-dimensional model to obtain a three-dimensional color model 232; the three-dimensional color model 232 is obtained by filling colors on the triangular faces of the three-dimensional model based on the weighted color information, and is the result of texture baking on the model structure 230 of the three-dimensional model.

[0090] According to the difference between the observed image of the three-dimensional color model 232 and the view image 212, the quality of texture baking performed on the model structure 230 of the three-dimensional model is evaluated, and based on the difference between the observed image of the three-dimensional color model 232 and the view image 212, the view image 212 is corrected; and the texture baking is iterated to obtain the corrected color model 234. Compared with the three-dimensional color model 232, the corrected color model 234 has a higher model quality, and the observation results of the corrected color model 234 under four view angles are closer to the view image 212.

[0091] Next, the processing method of the three-dimensional model will be introduced through the following embodiments.

[0092] Figure 3 A flowchart of a method for processing a three-dimensional model provided by an exemplary embodiment of the present application is shown. The method can be executed by a computer device. The method includes:

[0093] Step 510: Obtain a model structure of the three-dimensional model, view images of the expected texture of the three-dimensional model at n view angles, and confidence weights of vertices in the three-dimensional model at n view angles.

[0094] The model structure of the three-dimensional model is used to indicate the stereoscopic space information of the three-dimensional model. The model structure is usually implemented as a model file of the three-dimensional model, indicating the model shape of the three-dimensional model.

[0095] The perspective images of the desired texture of the 3D model at n perspectives are n 2D images, which are used to indicate the observation images of the 3D model baked with texture information that are expected to be generated. Wherein, n is an integer greater than 1; in some examples, the value of n is 4, which is implemented as the observation images of the desired texture of the 3D model at the front view, rear view, left view, and right view; in some further examples, the value of n is 6, which also includes the observation images of the desired texture of the 3D model at the top view and the upward view.

[0096] The confidence weight of the vertex in the three-dimensional model under n viewing angles. The three-dimensional model is a space area surrounded by multiple triangular facets. The vertex is the vertex of the triangular facet. The three-dimensional model includes at least four vertices. The vertex has corresponding confidence weights under n viewing angles. The confidence weight corresponding to the first viewing angle among the n viewing angles is negatively correlated with the degree of deviation of the vertex observed under the first viewing angle. The degree of deviation is used to indicate the deviation angle between the viewing direction of the first viewing angle and the normal direction of the vertex.

[0097] Exemplarily, the model structure, viewpoint image, and confidence weight may be directly acquired by a computer device from stored information, or may be calculated or received by a computer device. This application does not limit the method for acquiring the above data.

[0098] Step 520: using the confidence weights at the n viewing angles as the color weights of the viewing images at the n viewing angles respectively, and performing texture baking on the model structure of the three-dimensional model based on the viewing images at the n viewing angles to obtain a three-dimensional color model.

[0099] The color weight is used to perform weighting on the color information of the perspective image; illustratively, the color information of the perspective image can be implemented in a format such as a hue-saturation-value color space (HSV), an RGB color space: a red-green-blue color space (RGB), and the weighting of the perspective image is achieved by calculating the product between at least one color channel in the hue-saturation-value color space or the red-green-blue color space and the color weight.

[0100] Exemplarily, the confidence weight is used to indicate the degree of deviation in the observation of vertices from a viewing angle. As the degree of deviation increases, the confidence weight decreases; that is, as the observation effect of the vertex from a viewing angle is poor, the color in the viewing angle image is faded in a weighted manner to reduce the impact of color on texture baking.

[0101] Exemplarily, by mapping the view images under n view angles onto the three-dimensional model, at least one of the properties such as color, material, and reflective property of each triangular face of the three-dimensional model is filled. Exemplarily, texture baking includes but is not limited to mapping the triangular face of the three-dimensional model onto at least one of a two-dimensional plane, a diffuse reflection map, and a normal map. The present application does not limit the specific method of model baking.

[0102] Step 530: Obtain observation images of the three-dimensional color model at n viewing angles, and perform correction on the viewing angle image according to the difference between the observation image and the viewing angle image to obtain a corrected viewing angle image.

[0103] Exemplarily, by calling a virtual camera, the three-dimensional color model obtained by texture baking is observed at n viewing angles to obtain observation images at n viewing angles.

[0104] The difference between the observed image and the view image is used to indicate the quality of texture baking performed based on the view image. As the difference between the observed image and the view image decreases, the accuracy of texture baking performed based on the view image increases, and the quality of the three-dimensional color model obtained by texture baking is higher.

[0105] In order to improve the quality of the three-dimensional color model, the perspective image is corrected according to the difference between the observed image and the perspective image to obtain a corrected perspective image; the perspective image is corrected by modifying the color information of the perspective image.

[0106] Step 540: Perform texture baking on the model structure of the three-dimensional model based on the corrected perspective image to obtain a corrected color model.

[0107] Exemplarily, the purpose of performing correction on the view image is to reduce the difference between the observed image and the view image. The corrected color model obtained by performing texture baking based on the corrected image has a higher model quality than the three-dimensional color model obtained by performing texture baking on the view image. Compared with the three-dimensional color model, the observation results of the corrected color model under n view angles are closer to the n view images.

[0108] In summary, the method provided in this embodiment, by using the confidence weight as the color weight of the perspective image, according to the confidence weight used to indicate the degree of deviation of the observed vertex, quantitatively describes the degree of trust in the color information in the perspective image, thereby realizing the adjustment of the color information of the perspective image based on the confidence weight; reduces the impact of inaccurate texture baking caused by the large degree of deviation of the observed vertex during the texture baking process; and then adjusts the observation image through the difference between the observation image and the perspective image of the three-dimensional color model obtained through texture baking, thereby improving the accuracy of texture baking for the three-dimensional model.

[0109] Figure 4 A flowchart of a method for processing a three-dimensional model provided by an exemplary embodiment of the present application is shown. The method can be executed by a computer device. Figure 3 In the illustrated embodiment, step 520 may be implemented as steps 522, 524, and 526:

[0110] Step 522: Based on the confidence weight at the i-th viewing angle, perform color weighting on the viewing image at the i-th viewing angle to obtain a weighted viewing image at the i-th viewing angle.

[0111] The color information of the view image is weighted based on the confidence weight; the color weighting of the view image is performed by calculating the product between at least one color channel in the hue-saturation-lightness color space or the red-green-blue color space and the confidence weight, so as to obtain a weighted view image corresponding to the view image at the i-th view, where i is a positive integer not exceeding n.

[0112] In an optional implementation, this step may be implemented as follows:

[0113] According to the confidence weights of the vertices in the 3D model at the i-th perspective, a weight matrix at the i-th perspective is constructed.

[0114] A matrix element in the weight matrix at the i-th viewing angle is used to indicate weight information for performing color weighting on a pixel point or a pixel block composed of multiple pixels in the viewing image at the i-th viewing angle. The matrix size of the weight matrix is ​​less than or equal to the image size of the observed image.

[0115] Further, according to the mapping positions of a vertices in the three-dimensional model on the two-dimensional plane, the values ​​of the a vertex matrix elements in the initial matrix are determined as the confidence weights of the a vertices at the i-th viewing angle;

[0116] In this embodiment, there are corresponding confidence weights for the vertices of the three-dimensional model, and the relative positions of a vertex matrix elements in the initial matrix are the same as the mapping positions of a vertices on the two-dimensional plane at the i-th viewing angle. Wherein, a is an integer greater than 1. The a vertex matrix elements indicate the positions of the vertices in the three-dimensional model through the positions of the matrix elements in the initial matrix; the confidence weights of the vertices are indicated by the values ​​of the matrix.

[0117] According to the values ​​of a vertex matrix elements and the positions of a vertex matrix elements in the initial matrix, the values ​​of the model matrix elements whose matrix positions are between the a vertex matrix elements are obtained with numerical smoothness constraints, and the weight matrix at the i-th perspective is constructed based on the a vertex matrix elements and the model matrix elements.

[0118] Since the area enclosed by the vertices of the triangular patch also includes one or more pixels, the vertex confidence weight cannot directly describe the weight information (i.e., color weight) of the color weighted pixel. According to the values ​​of a vertex matrix elements, the values ​​of the model matrix elements are obtained under the constraint of numerical smoothness, thus achieving the color weight filling in the area enclosed by the vertices of the triangular patch.

[0119] Based on the positions of a vertex matrix elements in the initial matrix, the values ​​of the obtained model matrix elements take into account the positions between the vertices; with the values ​​of the vertex matrix elements as a reference, the values ​​of the obtained model matrix elements meet the constraints of numerical smoothing, ensuring that the values ​​of the obtained model matrix elements change gradually between the values ​​of the vertex matrix elements.

[0120] Based on the dot product between the weight matrix and the color matrix of the perspective image at the i-th perspective, a weighted perspective image at the i-th perspective is determined.

[0121] Exemplarily, based on the dot product between the weight matrix and the color matrix, the product between the elements at the same matrix position in the weight matrix and the color matrix can be obtained, thereby implementing weighting of the color information of the perspective image based on the confidence weight.

[0122] Figure 5A schematic diagram of a weighted viewing angle image provided by an exemplary embodiment of the present application is shown. The weight matrix shown in grayscale in sub-image (a) shows that the confidence weight corresponding to a viewing angle is negatively correlated with the degree of deviation of the vertex observed at that viewing angle. The greater the deviation, the closer it is to black in sub-image (a); that is, the greater the deviation, the smaller the grayscale value. Sub-image (b) is a weighted viewing angle image determined based on the dot product between the weight matrix and the color matrix of the viewing angle image.

[0123] Figure 6 A schematic diagram of confidence weights provided by an exemplary embodiment of the present application is shown. Figure 6 The low confidence area 630 is an area where the execution weight is lower than the weight threshold. The confidence weights of the pixels in the low confidence weight area 630 are cleared to exclude the low confidence area 630 when performing texture baking, so as to avoid using the perspective image to perform texture baking when the displacement of the observed vertex is too large.

[0124] Step 524: Increase the value of i and repeat the above steps until the weighted viewing angle images under n viewing angles are obtained.

[0125] Exemplarily, the value of i is increased, and color weighting is performed for the viewing images at n viewing angles respectively; the above step 522 is repeated until weighted viewing images corresponding to the viewing images at n viewing angles are obtained.

[0126] Step 526: According to the weighted viewing angle images under n viewing angles, texture baking is performed on the model structure of the three-dimensional model to obtain a three-dimensional color model.

[0127] Exemplarily, by mapping the weighted view images under n view angles onto the three-dimensional model, at least one of the properties such as color, material, and reflective property of each triangular face of the three-dimensional model is filled. Exemplarily, texture baking is performed according to the weighted view images under n view angles to perform texture baking on the triangular face on the three-dimensional model under n view angles.

[0128] In one example, texture baking is performed on the model structure with reference to the weighted view image, and the color information in the weighted view image is mapped onto the three-dimensional model; the influence of vertex confidence weights on texture baking is considered. The loss function when performing correction on the view image is:

[0129]

[0130] Among them, W i is the weight matrix introduced above, is the i-th view image, It is based on the weighted view image X, at the i-th view (denoted as T i) is performed under the observation image of the three-dimensional color model obtained by texture baking.

[0131] In summary, the method provided in this embodiment quantitatively describes the degree of trust in the color information in the perspective image by using the confidence weight as the color weight of the perspective image and according to the confidence weight used to indicate the degree of deviation of the observed vertex; adjusts the color information of the perspective image based on the confidence weight by calculating the dot product; reduces the impact of inaccurate texture baking caused by the large deviation of the observed vertex during the texture baking process; and adjusts the observation image through the difference between the observation image and the perspective image of the three-dimensional color model obtained through texture baking, thereby improving the accuracy of texture baking for the three-dimensional model.

[0132] Figure 7 A flowchart of a method for processing a three-dimensional model provided by an exemplary embodiment of the present application is shown. The method can be executed by a computer device. Figure 3 Based on the embodiment shown, step 515 and step 516 are also included; step 520 can be implemented as step 520a:

[0133] Step 515: Perform image segmentation on the viewing images at n viewing angles respectively to obtain model image regions at n viewing angles.

[0134] By performing image segmentation on the perspective image, a model image region showing the desired texture of the three-dimensional model in the perspective image can be obtained. For example, due to the diversity of three-dimensional models, such as three-dimensional models implemented as virtual characters, virtual animals, virtual plants, virtual buildings, virtual clothing, and virtual objects, the model image region obtained by segmentation can be a regular region such as a rectangle or a circle, or an irregular region surrounded by multiple line segments.

[0135] By performing image segmentation, the observation area of ​​the three-dimensional model and the environment area around the three-dimensional model in the perspective image are separated from each other. Exemplarily, the image segmentation can be performed based on the annotation method, or it can be obtained by calling an artificial neural network (ANN) with image segmentation capability. This application does not limit the segmentation method of the image segmentation.

[0136] Step 516: performing edge erosion on the model image regions under n viewing angles respectively to obtain the main image regions under n viewing angles.

[0137] Edge erosion is performed on the model image regions under n viewing angles respectively, and on the basis of segmenting the environment region and the observation region of the three-dimensional model, the problem that the three-dimensional model cannot be completely separated from the environment region due to the complexity of the appearance is reduced, and the subject image region is obtained. In some examples, when edge erosion is performed on different positions of the model image region, the eroded pixel size is the same, reducing the impact of edge erosion on the shape of the model image region. Figure 8 A schematic diagram of a main image region provided by an exemplary embodiment of the present application is shown. Edge corrosion is performed on the model image region 610 at four viewing angles, and the corrosion region 602 is deleted to obtain the main image region 620 at four viewing angles. The model image region 610 is a model region of a virtual bird. Since the feather edge of the bird is irregular in shape, edge corrosion is performed to delete the area where the feather edge of the bird and the background environment intersect with each other, and the main image region 620 for performing texture baking is obtained.

[0138] Step 520a: Use the confidence weights under n viewing angles as the color weights of the main image area under n viewing angles respectively, perform texture baking on the model structure of the 3D model based on the main image area under n viewing angles, and obtain a 3D color model.

[0139] In this embodiment, texture baking is performed on the model structure of the three-dimensional model based on the main image area, and the edge part of the model image area obtained by image segmentation is avoided, and the influence of environmental information on model baking is avoided.

[0140] In summary, the method provided in this embodiment quantitatively describes the degree of trust in the color information in the perspective image based on the confidence weight indicating the degree of deviation of the observed vertex by taking the confidence weight as the color weight of the perspective image, and realizes the adjustment of the color information of the perspective image based on the confidence weight; by performing image segmentation, the observation area of ​​the three-dimensional model in the perspective image and the environmental area around the three-dimensional model are separated from each other. Edge erosion is performed on the model image area under n perspectives respectively to reduce the problem that the model cannot be completely separated from the environmental area due to the complexity of the appearance of the three-dimensional model. Texture baking is performed on the model structure of the three-dimensional model based on the main image area under n perspectives to obtain a three-dimensional color model; the influence of environmental information on model baking at the edge of the model image area obtained by image segmentation is avoided, and the accuracy of texture baking on the three-dimensional model is improved.

[0141] Next, Figure 3 The confidence weight and view image acquisition processes shown in are introduced respectively.

[0142] Fig. 9A flowchart of a method for processing a three-dimensional model provided by an exemplary embodiment of the present application is shown. The method can be executed by a computer device. Figure 3 On the basis of the illustrated embodiment, step 505 is also included, and step 510 can be implemented as step 511, step 512, step 513, step 515, and step 516.

[0143] Among them, step 505, step 511, step 512, and step 513 are introductions to the process of obtaining the confidence weight, and step 515 and step 516 are introductions to the process of obtaining the perspective image.

[0144] Step 505: Obtain the normal direction of each vertex in the three-dimensional model.

[0145] Exemplarily, in a three-dimensional model, a vertex has at least three connected triangular facets, and the average value of the normal directions of the triangular facets connected to the vertex is determined as the normal direction of the vertex. Exemplarily, the normal direction of each vertex is determined based on the normal direction of each triangular facet of the three-dimensional model.

[0146] Step 511: Obtain the i-th observation position corresponding to the i-th viewing angle among the n viewing angles.

[0147] Exemplarily, the n viewing angles correspond one-to-one to n observation positions, where the observation positions are positions where virtual cameras are deployed to observe the three-dimensional model.

[0148] Exemplarily, there is an association relationship between the i-th viewing angle image and the i-th observation position among the n viewing angle images, and the i-th viewing angle image is used to indicate an image obtained by observing the desired texture of the three-dimensional model when the virtual camera is deployed at the i-th observation position, and the observation position is also called the optical center position of the virtual camera. Wherein, i is a positive integer not exceeding n.

[0149] Step 512: Determine the confidence weight of the vertex according to the direction difference between the observation direction of the vertex in the three-dimensional model at the i-th observation position and the normal direction of the vertex.

[0150] Exemplarily, the viewing direction of a vertex is the direction in which the viewing position points to the vertex; as described above, the normal direction of a vertex is the average of the normal directions of the triangles connected to the vertex. The confidence weight of the vertex is determined based on the direction difference between the viewing direction and the normal direction of the vertex. The confidence weight decreases as the direction difference between the viewing direction and the normal direction of the vertex increases, showing a negative correlation.

[0151] In an optional implementation, this step may be implemented as follows:

[0152] Calculate the cosine between the viewing direction and the normal direction.

[0153] The absolute value of the cosine value is determined as the independent variable of the normalization function, and the dependent variable of the normalization function is determined as the confidence weight of the vertex.

[0154] Exemplarily, when the observation direction and the normal direction are perpendicular to each other, the cosine value between the observation direction and the normal direction is 0; as the observation direction and the normal direction become closer to parallel, the absolute value of the cosine value between the observation direction and the normal direction increases and approaches 1.

[0155] As described above, as the degree of deviation between the viewing direction and the normal direction increases, the absolute value of the calculated cosine value decreases; until the viewing direction and the normal direction are perpendicular to each other, the absolute value of the calculated cosine value is 0.

[0156] The absolute value of the cosine value is determined as the independent variable of the normalization function, the calculated absolute value of the cosine value is normalized, and the calculated absolute value of the cosine value is mapped to a fixed interval to obtain the confidence weight, so as to achieve a stable description of the deviation degree of the vertex observed under a viewing angle based on the confidence weight.

[0157] In some examples, the normalization function is a natural exponential function, that is, an exponential function whose base is a natural constant (Natural Constant), which is also called the Euler number (Euler Number). The absolute value of the cosine value is determined as the independent variable of the natural exponential function, and the dependent variable of the natural exponential function is determined as the confidence weight of the vertex. By determining the absolute value of the cosine value as the independent variable of the natural exponential function, the discrimination of the obtained confidence weight can be increased in the form of an exponential function.

[0158] In one example, for the j-th vertex on the three-dimensional model, the confidence weight of the j-th vertex is implemented as:

[0159]

[0160] Among them, w j is the confidence weight of the jth vertex, abs is the absolute value calculation, p j is the position coordinate of the jth vertex in three-dimensional space, c is the observation position of the virtual camera deployed at a certain viewing angle, also known as the optical center position of the virtual camera; n j is the normal direction of the j-th vertex, and exp is the natural exponential function.

[0161] Step 513: Increase the value of i and repeat the above two steps until the confidence weights under n perspectives are obtained.

[0162] Exemplarily, the value of i is increased, and the confidence weight of each perspective under n perspectives is calculated respectively; by repeating the above steps 511 and 512, the confidence weights under n perspectives are obtained.

[0163] As described above, steps 505, 511, 512, and 513 in this embodiment are introductions to obtaining confidence weights; in an optional implementation, reference may also be made to the introduction in step 510, where the method for obtaining the perspective view is not limited, and steps 505 to 513 are similar to the steps in step 510. Figure 3 Steps 520 to 540 in the embodiment are combined into a new embodiment and implemented separately, and this application does not limit this.

[0164] Step 515: Obtain depth maps of the 3D model at n viewing angles and guide text of the desired texture.

[0165] Exemplarily, the depth map is used to indicate the distance from the three-dimensional model to the imaging plane, and is used to indicate the spatial structure information of the three-dimensional model. The guide text of the expected texture is used to describe the appearance style of the expected texture in a natural language, such as describing at least one of the color, reflective material, pattern, and fill pattern of the expected texture.

[0166] Step 516: Call the image generation network to perform prediction on the depth map and the guide text under n viewing angles to obtain viewing images under n viewing angles.

[0167] Exemplarily, the image generation network has the ability to generate image information based on images and text, wherein the depth map provides constraints for the shape of the generated perspective image and the guidance text provides constraints for the appearance style of the desired texture in the perspective image.

[0168] The exemplary image generation network is an artificial neural network with image generation capability, and the present application does not impose any restrictions on the network structure of the artificial neural network.

[0169] Fig.10 A schematic diagram of a depth map and a perspective image provided by an exemplary embodiment of the present application is shown.

[0170] Sub-image (a) is a depth map under four viewing angles. The depth map has an image style in which there is a corresponding relationship between depth information and image grayscale. Fig.11 ; Fig.11 A schematic diagram of a depth map provided by an exemplary embodiment of the present application is shown.

[0171] In the three-dimensional space, the three-dimensional object is photographed by the camera 650 to obtain an observation picture of the three-dimensional space, that is, an image including the three-dimensional object. Fig.11In the figure, the three-dimensional objects include a sphere 662, a pyramid 664 and a cylinder 666; however, it is not excluded that in other examples, the three-dimensional space includes more or fewer three-dimensional objects, and / or the three-dimensional space includes other three-dimensional objects with delayed appearance.

[0172] Exemplarily, the imaging plane 670 is perpendicular to the shooting direction 650a of the camera 650; Fig.11 The imaging plane 670 in the figure is an exemplary description; further, in some examples, the imaging plane is the plane where the lens of the camera 650 is located, or the plane where the light passes through the focal position of the lens of the camera 650. Exemplarily, the distance between the sphere 662 and the imaging plane 670 is a first distance 672, the distance between the quadrangular pyramid 664 and the imaging plane 670 is a second distance 674, and the distance between the cylinder 666 and the imaging plane 670 is a third distance 676. The first distance 672 is smaller than the second distance 674, and the second distance 674 is smaller than the third distance 676.

[0173] Exemplarily, the depth map 680 includes projection images of a sphere 662 , a pyramid 664 , and a cylinder 666 on a plane;

[0174] The fill color of the projection area of ​​the sphere 662 in the depth map 680 is the first gray value 682; the fill color of the projection area of ​​the pyramid 664 in the depth map 680 is the second gray value 684; the fill color of the projection area of ​​the cylinder 666 in the depth map 680 is the third gray value 686.

[0175] Exemplarily, the grayscale value filled in the depth map 680 increases as the distance between the object and the imaging plane 670 increases; therefore, the first grayscale value 682 is smaller than the second grayscale value 684 , and the second grayscale value 684 is smaller than the third grayscale value 686 .

[0176] exist Fig.10 In the figure, sub-image (b) calls the image generation network to predict the depth map and guide text under four viewing angles, and obtains the viewing images under four viewing angles. It can be seen that the depth map shown in sub-image (a) provides constraints for the shape of the generated viewing image; the predicted viewing image is an observed image of the expected texture of the virtual character of the spatial structure such as sub-image (a).

[0177] In an optional implementation, the image generation network includes a text encoding subnetwork, an image encoding subnetwork and a prediction subnetwork; this step is implemented as follows:

[0178] Call the text encoding subnetwork to encode the guide text to obtain semantic features; and call the image encoding subnetwork to encode the depth map under n perspectives to obtain structural features.

[0179] Exemplarily, the semantic feature is the feature representation of the guide text in the latent space, or the semantic feature is or the vector information composed of real numbers obtained by mapping the guide text from high-dimensional data in natural language format to a low-dimensional continuous vector space, where the text encoding subnetwork is also called a text embedding subnetwork.

[0180] Exemplarily, the structural feature is a characteristic representation of the depth map in the latent space, and the structural feature can be implemented as at least one of an eigenvalue, an eigenvector, and a feature matrix to quantitatively describe the spatial structure of the three-dimensional model carried in the perspective image in the three-dimensional space.

[0181] Call the prediction subnetwork to perform predictions on the semantic features and structural features to obtain the perspective images under n perspectives.

[0182] Exemplarily, the prediction subnetwork takes semantic features and structural features as input parameters to predict the perspective images under n perspectives. Exemplarily, the prediction subnetwork predicts the structural features encoded under n perspectives as a whole to obtain the perspective images under n perspectives; the structural features under n perspectives constrain each other to generate the perspective images of n perspectives at one time. In this process, the structural features under n perspectives are spliced ​​during the image prediction process, and the calculation is shared in the prediction subnetwork based on the self-attention mechanism to ensure the consistency of the generated perspective images under n perspectives, avoid conflicts in the information displayed by images of different perspectives, and avoid conflicts in the generated observation images under different perspectives, such as avoiding the problem of multiple faces (Janus Face) for the three-dimensional model of the character image.

[0183] Optionally, the training process of the image generation network is described as follows:

[0184] Obtain sample depth maps of the sample model at n viewing angles, sample images of the sample texture of the sample model at n viewing angles, and sample guide text of the sample texture.

[0185] The sample image is an observed image of the sample model, the sample guide text is descriptive information of the texture on the sample model, and the sample depth map is used to describe the depth information of the sample model.

[0186] Call the initial generation model to perform prediction on the sample depth map and sample guide text under n perspectives to obtain the predicted perspective images under n perspectives.

[0187] Exemplarily, the model structure of the initial generation model is the same as that of the image generation model mentioned above. In the training phase of the initial generation model, the initial generation model is called to perform prediction on the sample depth maps and sample guide texts under n viewing angles.

[0188] In some examples, the initial generation model includes an initial text encoding subnetwork, an initial image encoding subnetwork, and an initial prediction subnetwork; for the introduction of each subnetwork, please refer to the text encoding subnetwork, image encoding subnetwork, and prediction subnetwork mentioned above, and no repetition will be given.

[0189] Based on the difference between the predicted view images under n view angles and the sample images under n view angles, the initial generation model is trained to obtain an image generation network.

[0190] Exemplarily, the generation capability of the initial generation model for the perspective image is described according to the difference between the predicted perspective image under n perspectives and the sample image under n perspectives. According to the difference between the predicted perspective image and the sample image, backward error propagation training is performed on the initial generation model, and the generation capability for the perspective image is improved by adjusting the model parameters of the initial generation model, thereby obtaining an image generation network.

[0191] In one example, the difference between the predicted view image under n views and the sample image under n views is implemented as:

[0192]

[0193] Among them, c t is a control signal obtained based on the sample guide text, such as the semantic features of the sample guide text, c f is the control signal obtained based on the sample depth map, such as the structural features of the sample depth map, z0,t is the model parameter of the initial generation model, ee q is the difference between the sampling information and the sampling noise information of the initial generation model. For example, the perspective image is predicted based on the attention mechanism in the initial generation model:

[0194]

[0195] Among them, Q is the prompt word (Query), K, V are the control signals obtained based on the sample guidance text and the control signals obtained from the sample depth map, and d is the dimension size of K.

[0196] Fig.12 A flowchart of a method for processing a three-dimensional model provided by an exemplary embodiment of the present application is shown. The method can be executed by a computer device. Figure 3 Based on the embodiment shown, step 552, step 554, and step 556 are also included:

[0197] Step 552: Obtain the three-dimensional position information of the color-deficient region in the modified color model.

[0198] The color-vacant area is an area in the 3D model where the texture is empty after texture baking is performed; in one example, due to the complex spatial structure of the 3D model, different sub-parts of the 3D model will occlude each other, resulting in an area where the texture is empty after texture baking is performed. In other examples, due to the small number of perspective images, such as a small value of n, the perspective images cannot fully describe the expected texture of the 3D model at all angles, resulting in an area where the texture is empty after texture baking is performed.

[0199] Exemplarily, the three-dimensional position information is the position of the color-vacancy region described in the three-dimensional space coordinate system. The number of color-vacancy regions may be one or more, and each color-vacancy region may be a region of one or more triangular facets; it is understood that the present application does not exclude the situation where a part of a triangular facet belongs to the color-vacancy region.

[0200] Step 554: Based on the three-dimensional position information, determine the spatially adjacent region in the modified color model.

[0201] Exemplarily, the spatially adjacent region is an area of ​​the modified color model in the three-dimensional space coordinate system that is adjacent to the color-deficient region and has texture information. Exemplarily, the spatially adjacent region can be obtained by marking on the modified color model, or by searching on the modified color model, such as performing a nearest neighbor search (K-Nearest Neighbors search, KNN) on the modified color model in the three-dimensional space coordinate system based on the three-dimensional position information, so as to match the color-deficient region to the position of the nearest neighbor with texture information in the modified color model.

[0202] Step 556: Based on the spatial neighboring regions, update the modified color model by filling the texture of the color vacant region on the modified color model.

[0203] The color information of the spatially adjacent areas is used as a reference to fill in the texture of the color-missing areas to update and correct the color model.

[0204] Exemplarily, the color information of the spatially adjacent areas can be directly used as the color information of the color-deficient areas to fill the texture of the color-deficient areas; or a color gradient can be performed on the color information of the spatially adjacent areas to fill the color-deficient areas with gradient colors to achieve a smooth transition of colors from the color-deficient areas to the surrounding spatially adjacent areas.

[0205] Fig.13A schematic diagram of a corrected color model provided by an exemplary embodiment of the present application is shown. The texture information baked on the corrected color model can be stored in the form of a two-dimensional image as shown in sub-image (b), and each pixel position in the two-dimensional image has a corresponding spatial position on the three-dimensional model after the texture is baked. The texture information of the spatially adjacent area can be searched based on the spatially adjacent area to obtain the corresponding area position on the two-dimensional image. Based on the texture information of the spatially adjacent area, the texture of the color vacant area is filled to achieve the update of the corrected color model. Sub-image (a) is the corresponding position of the color vacant area on the two-dimensional image.

[0206] Fig.14 A schematic diagram of a modified color model provided by an exemplary embodiment of the present application is shown. Sub-figure (a) is a schematic diagram of the modified color model, and the black area is a color vacancy area 690 where the texture is empty after texture baking is performed in the three-dimensional model; based on the spatial neighboring area, the texture of the color vacancy area is filled on the modified color model to obtain the updated modified color model shown in sub-figure (b).

[0207] In summary, the method provided in this embodiment, by using the confidence weight as the color weight of the perspective image, according to the confidence weight used to indicate the degree of deviation of the observed vertex, quantitatively describes the degree of trust in the color information in the perspective image, and realizes the adjustment of the color information of the perspective image based on the confidence weight; reduces the impact of inaccurate texture baking caused by the large degree of deviation of the observed vertex during the texture baking process; fills the texture of the color vacant area with reference to the color information of the spatially adjacent area to achieve the update and correction of the color model, thereby ensuring the completeness of the texture baking performed on the three-dimensional model and avoiding the problem of some areas not being filled with texture information due to the occlusion of the model itself.

[0208] In an application scenario of the present application, a computer device is installed with a virtual environment development program, and the three-dimensional model is a virtual item model in a virtual environment constructed based on the virtual environment development program. The virtual environment development program is used to construct a virtual environment in a three-dimensional space. Accordingly,

[0209] Obtaining a model structure of a virtual item model, perspective images of an expected texture of the virtual item model at n perspectives, and confidence weights of vertices in the virtual item model at n perspectives;

[0210] n is an integer greater than 1, and the confidence weight corresponding to the first perspective among the n perspectives is negatively correlated with the degree of deviation of the vertex observed at the first perspective;

[0211] Optional, for execution weight:

[0212] Get the normal direction of each vertex in the virtual item model;

[0213] Get the confidence weights of the vertices in the virtual item model under n perspectives, including:

[0214] Get the i-th observation position corresponding to the i-th perspective among n perspectives, where i is a positive integer not exceeding n;

[0215] Determine the confidence weight of the vertex according to the direction difference between the observation direction of the vertex in the virtual object model at the i-th observation position and the normal direction of the vertex;

[0216] Increase the value of i and repeat the above two steps until the confidence weights under n perspectives are obtained.

[0217] Further, the cosine value between the viewing direction and the normal direction is calculated;

[0218] The absolute value of the cosine value is determined as the independent variable of the normalization function, and the dependent variable of the normalization function is determined as the confidence weight of the vertex.

[0219] Optional, for perspective images:

[0220] Obtaining the depth map of the virtual object model at n viewing angles and the guiding text of the expected texture;

[0221] The image generation network is called to perform prediction on the depth map and guide text under n perspectives to obtain perspective images under n perspectives.

[0222] Furthermore, the image generation network includes a text encoding subnetwork, an image encoding subnetwork, and a prediction subnetwork;

[0223] Calling the text encoding sub-network to encode the guide text to obtain semantic features; and calling the image encoding sub-network to encode the depth map under n perspectives to obtain structural features;

[0224] The prediction subnetwork is called to perform prediction on the semantic features and structural features to obtain the perspective images under n perspectives.

[0225] For the training of image generation network, it also includes:

[0226] Obtaining sample depth maps of the sample model at n viewing angles, sample images of the sample texture of the sample model at n viewing angles, and sample guide text of the sample texture;

[0227] Call the initial generation model to perform prediction on the sample depth map and sample guide text under n perspectives to obtain the predicted perspective images under n perspectives;

[0228] Based on the difference between the predicted view images under n view angles and the sample images under n view angles, the initial generation model is trained to obtain an image generation network.

[0229] The confidence weights under n viewing angles are used as the color weights of the viewing images under n viewing angles respectively, and texture baking is performed on the model structure of the virtual object model based on the viewing images under n viewing angles to obtain a three-dimensional color model;

[0230] The color weight is used to perform weighting on the color information of the perspective image; illustratively, the confidence weight is used to indicate the degree of deviation in the observation of the vertex at a certain perspective, and as the deviation increases, the confidence weight decreases; that is, as the observation effect of the vertex at a certain perspective is not good, the color in the perspective image is faded in a weighted manner to reduce the influence of the color on texture baking.

[0231] Optionally, based on the confidence weight at the i-th viewing angle, color weighting is performed on the viewing image at the i-th viewing angle to obtain a weighted viewing image at the i-th viewing angle, where i is a positive integer not exceeding n;

[0232] Increase the value of i and repeat the above steps until the weighted perspective images under n perspectives are obtained;

[0233] According to the weighted perspective images under n perspectives, texture baking is performed on the model structure of the virtual object model to obtain a three-dimensional color model.

[0234] Further optionally, a weight matrix at the i-th perspective is constructed according to the confidence weights of the vertices in the virtual item model at the i-th perspective;

[0235] Based on the dot product between the weight matrix and the color matrix of the viewing image at the i-th viewing angle, a weighted viewing image at the i-th viewing angle is determined.

[0236] Furthermore, according to the mapping positions of a vertices in the virtual item model on the two-dimensional plane, the values ​​of a vertex matrix elements in the initial matrix are determined as the confidence weights of the a vertices at the i-th viewing angle, where a is an integer greater than 1;

[0237] According to the values ​​of a vertex matrix elements and the positions of a vertex matrix elements in the initial matrix, the values ​​of the model matrix elements whose matrix positions are between the a vertex matrix elements are obtained with numerical smoothness constraints, and the weight matrix at the i-th perspective is constructed based on the a vertex matrix elements and the model matrix elements.

[0238] Obtaining observation images of the three-dimensional color model at n viewing angles, and performing correction on the viewing angle image according to the difference between the observation image and the viewing angle image to obtain a corrected viewing angle image;

[0239] Exemplarily, by calling a virtual camera, the three-dimensional color model obtained by texture baking is observed at n viewing angles to obtain observation images at n viewing angles.

[0240] The difference between the observed image and the view image is used to indicate the quality of texture baking performed based on the view image. As the difference between the observed image and the view image decreases, the accuracy of texture baking performed based on the view image increases, and the quality of the three-dimensional color model obtained by texture baking is higher.

[0241] Perform texture baking on the model structure of the virtual object model based on the corrected perspective image to obtain a corrected color model.

[0242] Exemplarily, the purpose of performing correction on the view image is to reduce the difference between the observed image and the view image. The corrected color model obtained by performing texture baking based on the corrected image has a higher model quality than the three-dimensional color model obtained by performing texture baking on the view image. Compared with the three-dimensional color model, the observation results of the corrected color model under n view angles are closer to the n view images.

[0243] Optionally, the method further includes: obtaining three-dimensional position information of a color-deficient region in the corrected color model, wherein the color-deficient region is a region in the virtual object model where the texture is empty after texture baking is performed;

[0244] Based on the three-dimensional position information, determining the spatially adjacent regions in the modified color model;

[0245] Based on the spatial neighboring regions, the modified color model is updated by filling the texture of the color vacant region on the modified color model.

[0246] In another application scenario of the present application, a computer device is installed with an image processing program, such as a photo editing software, and the three-dimensional model is a three-dimensional model of an object in the image information (i.e., the perspective image) obtained by the photo editing software, also known as an image model. Exemplarily, after obtaining the corrected color model, it also includes calling a virtual camera model to obtain the expected observation image of the corrected color model under the expected perspective; the expected perspective is the perspective selected through human-computer interaction, which provides the photo editing software with the function of changing the angle of the object in the image to be processed. Accordingly,

[0247] Obtain the model structure of the image model, the view images of the expected texture of the image model under n view angles, and the confidence weights of the vertices in the image model under n view angles;

[0248] n is an integer greater than 1, and the confidence weight corresponding to the first perspective among the n perspectives is negatively correlated with the degree of deviation of the vertex observed at the first perspective;

[0249] Optional, for execution weight:

[0250] Get the normal direction of each vertex in the image model;

[0251] Get the confidence weights of the vertices in the image model under n viewing angles, including:

[0252] Get the i-th observation position corresponding to the i-th perspective among n perspectives, where i is a positive integer not exceeding n;

[0253] Determine the confidence weight of the vertex according to the direction difference between the observation direction of the vertex in the image model at the i-th observation position and the normal direction of the vertex;

[0254] Increase the value of i and repeat the above two steps until the confidence weights under n perspectives are obtained.

[0255] Further, the cosine value between the viewing direction and the normal direction is calculated;

[0256] The absolute value of the cosine value is determined as the independent variable of the normalization function, and the dependent variable of the normalization function is determined as the confidence weight of the vertex.

[0257] Optional, for perspective images:

[0258] Obtain the depth map of the image model at n viewing angles and the guiding text of the expected texture;

[0259] The image generation network is called to perform prediction on the depth map and guide text under n perspectives to obtain perspective images under n perspectives.

[0260] Furthermore, the image generation network includes a text encoding subnetwork, an image encoding subnetwork, and a prediction subnetwork;

[0261] Calling the text encoding sub-network to encode the guide text to obtain semantic features; and calling the image encoding sub-network to encode the depth map under n perspectives to obtain structural features;

[0262] The prediction subnetwork is called to perform prediction on the semantic features and structural features to obtain the perspective images under n perspectives.

[0263] For the training of image generation network, it also includes:

[0264] Obtaining sample depth maps of the sample model at n viewing angles, sample images of the sample texture of the sample model at n viewing angles, and sample guide text of the sample texture;

[0265] Call the initial generation model to perform prediction on the sample depth map and sample guide text under n perspectives to obtain the predicted perspective images under n perspectives;

[0266] Based on the difference between the predicted view images under n view angles and the sample images under n view angles, the initial generation model is trained to obtain an image generation network.

[0267] The confidence weights under n viewing angles are used as the color weights of the viewing images under n viewing angles respectively, and texture baking is performed on the model structure of the image model based on the viewing images under n viewing angles to obtain a three-dimensional color model;

[0268] The color weight is used to perform weighting on the color information of the perspective image; illustratively, the confidence weight is used to indicate the degree of deviation in the observation of the vertex at a certain perspective, and as the deviation increases, the confidence weight decreases; that is, as the observation effect of the vertex at a certain perspective is not good, the color in the perspective image is faded in a weighted manner to reduce the influence of the color on texture baking.

[0269] Optionally, based on the confidence weight at the i-th viewing angle, color weighting is performed on the viewing image at the i-th viewing angle to obtain a weighted viewing image at the i-th viewing angle, where i is a positive integer not exceeding n;

[0270] Increase the value of i and repeat the above steps until the weighted perspective images under n perspectives are obtained;

[0271] According to the weighted view images under n view angles, texture baking is performed on the model structure of the image model to obtain a three-dimensional color model.

[0272] Further optionally, a weight matrix at the i-th perspective is constructed according to the confidence weights of the vertices in the image model at the i-th perspective;

[0273] Based on the dot product between the weight matrix and the color matrix of the viewing image at the i-th viewing angle, a weighted viewing image at the i-th viewing angle is determined.

[0274] Furthermore, according to the mapping positions of a vertices in the image model on the two-dimensional plane, the values ​​of a vertex matrix elements in the initial matrix are determined as the confidence weights of the a vertices at the i-th viewing angle, where a is an integer greater than 1;

[0275] According to the values ​​of a vertex matrix elements and the positions of a vertex matrix elements in the initial matrix, the values ​​of the model matrix elements whose matrix positions are between the a vertex matrix elements are obtained with numerical smoothness constraints, and the weight matrix at the i-th perspective is constructed based on the a vertex matrix elements and the model matrix elements.

[0276] Obtaining observation images of the three-dimensional color model at n viewing angles, and performing correction on the viewing angle image according to the difference between the observation image and the viewing angle image to obtain a corrected viewing angle image;

[0277] Exemplarily, by calling a virtual camera, the three-dimensional color model obtained by texture baking is observed at n viewing angles to obtain observation images at n viewing angles.

[0278] The difference between the observed image and the view image is used to indicate the quality of texture baking performed based on the view image. As the difference between the observed image and the view image decreases, the accuracy of texture baking performed based on the view image increases, and the quality of the three-dimensional color model obtained by texture baking is higher.

[0279] Perform texture baking on the model structure of the image model based on the corrected view angle image to obtain a corrected color model.

[0280] Exemplarily, the purpose of performing correction on the view image is to reduce the difference between the observed image and the view image. The corrected color model obtained by performing texture baking based on the corrected image has a higher model quality than the three-dimensional color model obtained by performing texture baking on the view image. Compared with the three-dimensional color model, the observation results of the corrected color model under n view angles are closer to the n view images.

[0281] Optionally, the method further includes: obtaining three-dimensional position information of a color-vacancy region in the corrected color model, where the color-vacancy region is a region in the image model where the texture is empty after texture baking is performed;

[0282] Based on the three-dimensional position information, determining the spatially adjacent regions in the modified color model;

[0283] Based on the spatial neighboring regions, the modified color model is updated by filling the texture of the color vacant region on the modified color model.

[0284] Those skilled in the art can understand that the above embodiments can be implemented independently, or the above embodiments can be freely combined to form new embodiments to implement the three-dimensional model processing method of the present application.

[0285] Fig.15 The following is a block diagram of a three-dimensional model processing device provided by an exemplary embodiment of the present application. The device includes:

[0286] An acquisition module 810 is used to acquire a model structure of the three-dimensional model, view images of the expected texture of the three-dimensional model at n view angles, and confidence weights of vertices in the three-dimensional model at the n view angles, where n is an integer greater than 1, and the confidence weight corresponding to a first view angle among the n view angles is negatively correlated with a degree of deviation of the vertex observed at the first view angle;

[0287] A processing module 820 is configured to use the confidence weights at the n viewing angles as color weights of the viewing images at the n viewing angles, respectively, and perform texture baking on the model structure of the three-dimensional model based on the viewing images at the n viewing angles to obtain a three-dimensional color model;

[0288] A correction module 830, configured to obtain an observed image of the three-dimensional color model at the n viewing angles, and to correct the viewing angle image according to a difference between the observed image and the viewing angle image to obtain a corrected viewing angle image;

[0289] The processing module 820 is further configured to perform texture baking on the model structure of the three-dimensional model based on the corrected viewing angle image to obtain a corrected color model.

[0290] In an optional implementation of this embodiment, the processing module 820 is further configured to:

[0291] Based on the confidence weight at the i-th viewing angle, perform color weighting on the viewing image at the i-th viewing angle to obtain a weighted viewing image at the i-th viewing angle, where i is a positive integer not exceeding n;

[0292] Increasing the value of i, repeating the above steps, until the weighted viewing angle images under the n viewing angles are obtained;

[0293] According to the weighted viewing angle images under the n viewing angles, texture baking is performed on the model structure of the three-dimensional model to obtain the three-dimensional color model.

[0294] In an optional implementation of this embodiment, the processing module 820 is further configured to:

[0295] Constructing a weight matrix at the i-th viewing angle according to the confidence weights of the vertices in the three-dimensional model at the i-th viewing angle;

[0296] Based on the dot product between the weight matrix and the color matrix of the viewing image at the i-th viewing angle, a weighted viewing image at the i-th viewing angle is determined.

[0297] In an optional implementation of this embodiment, the processing module 820 is further configured to:

[0298] According to the mapping positions of a vertices in the three-dimensional model on the two-dimensional plane, the values ​​of a vertex matrix elements in the initial matrix are determined as the confidence weights of the a vertices at the i-th viewing angle, where a is an integer greater than 1;

[0299] According to the values ​​of the a vertex matrix elements and based on the positions of the a vertex matrix elements in the initial matrix, the values ​​of the model matrix elements whose matrix positions are between the a vertex matrix elements are obtained with numerical smoothness constraints, and the weight matrix at the i-th perspective is constructed based on the a vertex matrix elements and the model matrix elements.

[0300] In an optional implementation of this embodiment, the processing module 820 is further configured to:

[0301] Performing image segmentation on the viewing images at the n viewing angles respectively to obtain model image regions at the n viewing angles;

[0302] Performing edge erosion on the model image regions under the n viewing angles respectively to obtain the main image regions under the n viewing angles;

[0303] The confidence weights under the n viewing angles are respectively used as color weights of the main image area under the n viewing angles, and texture baking is performed on the model structure of the three-dimensional model based on the main image area under the n viewing angles to obtain the three-dimensional color model.

[0304] In an optional implementation of this embodiment, the acquisition module 810 is further configured to:

[0305] Obtaining the normal direction of each vertex in the three-dimensional model;

[0306] Obtaining an i-th observation position corresponding to an i-th viewing angle among the n viewing angles, where i is a positive integer not exceeding n;

[0307] Determining a confidence weight of the vertex according to a direction difference between an observation direction of a vertex in the three-dimensional model at the i-th observation position and the normal direction of the vertex;

[0308] Increase the value of i and repeat the above two steps until the confidence weights under the n perspectives are obtained.

[0309] In an optional implementation of this embodiment, the acquisition module 810 is further configured to:

[0310] Calculating the cosine value between the viewing direction and the normal direction;

[0311] The absolute value of the cosine value is determined as an independent variable of a normalization function, and the dependent variable of the normalization function is determined as the confidence weight of the vertex.

[0312] In an optional implementation of this embodiment, the acquisition module 810 is further configured to:

[0313] Obtaining a depth map of the three-dimensional model at the n viewing angles and a guide text of the desired texture;

[0314] An image generation network is called to perform prediction on the depth maps under the n viewing angles and the guide text to obtain viewing images under the n viewing angles.

[0315] In an optional implementation of this embodiment, the image generation network includes a text encoding subnetwork, an image encoding subnetwork and a prediction subnetwork;

[0316] The acquisition module 810 is also used for:

[0317] Calling a text encoding sub-network to encode the guide text to obtain semantic features; and calling an image encoding sub-network to encode the depth images under the n viewing angles to obtain structural features;

[0318] The prediction subnetwork is called to perform prediction on the semantic features and the structural features to obtain the perspective images under the n perspectives.

[0319] In an optional implementation of this embodiment, the acquisition module 810 is further configured to:

[0320] Acquire a sample depth map of the sample model at the n viewing angles, a sample image of the sample texture of the sample model at the n viewing angles, and a sample guide text of the sample texture;

[0321] Calling the initial generation model to perform prediction on the sample depth maps under the n viewing angles and the sample guide text to obtain predicted viewing angle images under the n viewing angles;

[0322] Based on the difference between the predicted perspective images under the n perspectives and the sample images under the n perspectives, the initial generation model is trained to obtain the image generation network.

[0323] In an optional implementation of this embodiment, the acquisition module 810 is further used to acquire three-dimensional position information of a color-vacancy region in the modified color model, wherein the color-vacancy region is a region in the three-dimensional model where the texture is empty after texture baking is performed;

[0324] The processing module 820 is further configured to determine a spatially adjacent region in the modified color model based on the three-dimensional position information;

[0325] The processing module 820 is further configured to update the modified color model by filling the texture of the color-deficient area on the modified color model based on the spatially adjacent area.

[0326] One point that needs to be explained is that the device provided in the above embodiment only uses the division of the above-mentioned functional modules as an example to implement its functions. In actual applications, the above-mentioned functions can be assigned to different functional modules according to actual needs, that is, the content structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0327] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method; the technical effects achieved by each module performing operations are the same as those in the embodiment of the method, and will not be elaborated here.

[0328] An embodiment of the present application also provides a computer device, which includes: a processor and a memory, wherein a computer program is stored in the memory; the processor is used to execute the computer program in the memory to implement the three-dimensional model processing method provided in the above-mentioned method embodiments.

[0329] Optionally, the computer device is a server. Fig.16 It is a structural block diagram of a server provided by an exemplary embodiment of the present application.

[0330] Typically, the server 2300 includes: a processor 2301 and a memory 2302 .

[0331] The processor 2301 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 2301 may be implemented in at least one hardware form of digital signal processing (DSP), field programmable gate array (FPGA), and programmable logic array (PLA). The processor 2301 may also include a main processor and a coprocessor. The main processor is a processor for processing data in an awake state, also known as a central processing unit (CPU); the coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 2301 may be integrated with a graphics processing unit (GPU), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 2301 may also include an artificial intelligence (AI) processor, which is used to process computing operations related to machine learning.

[0332] The memory 2302 may include one or more computer-readable storage media, which may be non-transitory. The memory 2302 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 2302 is used to store at least one instruction, which is used to be executed by the processor 2301 to implement the three-dimensional model processing method provided in the method embodiment of the present application.

[0333] In some embodiments, the server 2300 may also optionally include: an input interface 2303 and an output interface 2304. The processor 2301, the memory 2302, the input interface 2303, and the output interface 2304 may be connected via a bus or a signal line. Each peripheral device may be connected to the input interface 2303 and the output interface 2304 via a bus, a signal line, or a circuit board. The input interface 2303 and the output interface 2304 may be used to connect at least one peripheral device related to input / output (I / O) to the processor 2301 and the memory 2302. In some embodiments, the processor 2301, the memory 2302, the input interface 2303, and the output interface 2304 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 2301, the memory 2302, the input interface 2303, and the output interface 2304 may be implemented on a separate chip or circuit board, which is not limited in the embodiments of the present application.

[0334] Those skilled in the art will appreciate that the structure shown above does not constitute a limitation on the server 2300 , and may include more or fewer components than shown in the figure, or combine certain components, or adopt a different component arrangement.

[0335] In an exemplary embodiment, a chip is also provided, which includes a programmable logic circuit and / or program instructions. When the chip is run on a computer device, it is used to implement the three-dimensional model processing method described in the above aspects.

[0336] In an exemplary embodiment, a computer program product is also provided, the computer program product includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor reads and executes the computer instructions from the computer-readable storage medium to implement the three-dimensional model processing method provided by each of the above method embodiments.

[0337] In an exemplary embodiment, a computer-readable storage medium is also provided, in which a computer program is stored. The computer program is loaded and executed by a processor to implement the three-dimensional model processing method provided by the above-mentioned method embodiments.

[0338] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0339] Those skilled in the art should be aware that in one or more of the above examples, the functions described in the embodiments of the present application can be implemented with hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein the communication media include any media that facilitates the transmission of a computer program from one place to another. The storage medium can be any available medium that a general or special-purpose computer can access.

[0340] The above description is only an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for processing a three-dimensional model, characterized in that: The method is performed by a computer device, and the method comprises: Acquire a model structure of the three-dimensional model, perspective images of the expected texture of the three-dimensional model at n perspectives, and confidence weights of vertices in the three-dimensional model at the n perspectives, where n is an integer greater than 1, and the confidence weight corresponding to a first perspective among the n perspectives is negatively correlated with a degree of deviation of the vertex observed at the first perspective; The confidence weights at the n viewing angles are respectively used as color weights of the viewing images at the n viewing angles, and texture baking is performed on the model structure of the three-dimensional model based on the viewing images at the n viewing angles to obtain a three-dimensional color model; Acquire observation images of the three-dimensional color model at the n viewing angles, and perform correction on the viewing angle images according to the difference between the observation images and the viewing angle images to obtain corrected viewing angle images; Texture baking is performed on the model structure of the three-dimensional model based on the corrected viewing angle image to obtain a corrected color model.

2. The method according to claim 1, characterized in that The method of using the confidence weights under the n viewing angles as color weights of the viewing images under the n viewing angles, and performing texture baking on the model structure of the three-dimensional model based on the viewing images under the n viewing angles to obtain a three-dimensional color model includes: Based on the confidence weight at the i-th viewing angle, perform color weighting on the viewing image at the i-th viewing angle to obtain a weighted viewing image at the i-th viewing angle, where i is a positive integer not exceeding n; Increasing the value of i, repeating the above steps, until the weighted viewing angle images under the n viewing angles are obtained; According to the weighted viewing angle images under the n viewing angles, texture baking is performed on the model structure of the three-dimensional model to obtain the three-dimensional color model.

3. The method according to claim 2, characterized in that The step of performing color weighting on the viewing image at the i-th viewing angle based on the confidence weight at the i-th viewing angle to obtain the weighted viewing image at the i-th viewing angle includes: Constructing a weight matrix at the i-th viewing angle according to the confidence weights of the vertices in the three-dimensional model at the i-th viewing angle; Based on the dot product between the weight matrix and the color matrix of the viewing image at the i-th viewing angle, a weighted viewing image at the i-th viewing angle is determined.

4. The method according to claim 3, characterized in that The step of constructing a weight matrix at the i-th viewing angle according to the confidence weights of the vertices in the three-dimensional model at the i-th viewing angle comprises: According to the mapping positions of a vertices in the three-dimensional model on the two-dimensional plane, the values ​​of a vertex matrix elements in the initial matrix are determined as the confidence weights of the a vertices at the i-th viewing angle, where a is an integer greater than 1; According to the values ​​of the a vertex matrix elements and based on the positions of the a vertex matrix elements in the initial matrix, the values ​​of the model matrix elements whose matrix positions are between the a vertex matrix elements are obtained with numerical smoothness constraints, and the weight matrix at the i-th perspective is constructed based on the a vertex matrix elements and the model matrix elements.

5. The method according to claim 1, characterized in that The method further comprises: Performing image segmentation on the viewing images at the n viewing angles respectively to obtain model image regions at the n viewing angles; Performing edge erosion on the model image regions under the n viewing angles respectively to obtain the main image regions under the n viewing angles; The method of using the confidence weights under the n viewing angles as color weights of the viewing images under the n viewing angles, and performing texture baking on the model structure of the three-dimensional model based on the viewing images under the n viewing angles to obtain a three-dimensional color model includes: The confidence weights under the n viewing angles are respectively used as color weights of the main image area under the n viewing angles, and texture baking is performed on the model structure of the three-dimensional model based on the main image area under the n viewing angles to obtain the three-dimensional color model.

6. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: Obtaining the normal direction of each vertex in the three-dimensional model; The obtaining of confidence weights of vertices in the three-dimensional model at the n viewing angles includes: Obtaining an i-th observation position corresponding to an i-th viewing angle among the n viewing angles, where i is a positive integer not exceeding n; Determining a confidence weight of the vertex according to a direction difference between an observation direction of a vertex in the three-dimensional model at the i-th observation position and the normal direction of the vertex; Increase the value of i and repeat the above two steps until the confidence weights under the n perspectives are obtained.

7. The method according to claim 6, characterized in that The determining the confidence weight of the vertex according to the direction difference between the observation direction of the vertex in the three-dimensional model at the i-th observation position and the normal direction of the vertex includes: Calculating the cosine value between the viewing direction and the normal direction; The absolute value of the cosine value is determined as an independent variable of a normalization function, and the dependent variable of the normalization function is determined as the confidence weight of the vertex.

8. The method according to any one of claims 1 to 5, characterized in that: The obtaining of the perspective images of the desired texture of the three-dimensional model at n perspectives includes: Obtaining a depth map of the three-dimensional model at the n viewing angles and a guide text of the desired texture; An image generation network is called to perform prediction on the depth maps under the n viewing angles and the guide text to obtain viewing images under the n viewing angles.

9. The method according to claim 8, characterized in that The image generation network includes a text encoding subnetwork, an image encoding subnetwork and a prediction subnetwork; The calling image generation network performs prediction on the depth map under the n viewing angles and the guide text to obtain the viewing angle images under the n viewing angles, including: Calling a text encoding sub-network to encode the guide text to obtain semantic features; and calling an image encoding sub-network to encode the depth images under the n viewing angles to obtain structural features; The prediction subnetwork is called to perform prediction on the semantic features and the structural features to obtain the perspective images under the n perspectives.

10. The method according to claim 8, characterized in that The method further comprises: Acquire a sample depth map of the sample model at the n viewing angles, a sample image of the sample texture of the sample model at the n viewing angles, and a sample guide text of the sample texture; Calling the initial generation model to perform prediction on the sample depth maps under the n viewing angles and the sample guide text to obtain predicted viewing angle images under the n viewing angles; Based on the difference between the predicted perspective images under the n perspectives and the sample images under the n perspectives, the initial generation model is trained to obtain the image generation network.

11. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: Acquire three-dimensional position information of a color-deficient region in the modified color model, wherein the color-deficient region is a region in the three-dimensional model where the texture is empty after texture baking is performed; Based on the three-dimensional position information, determining a spatially adjacent region in the modified color model; Based on the spatially adjacent region, the modified color model is updated by filling the texture of the color-deficient region on the modified color model.

12. A three-dimensional model processing device, characterized in that: The device comprises: an acquisition module, used to acquire a model structure of the three-dimensional model, view images of the expected texture of the three-dimensional model at n view angles, and confidence weights of vertices in the three-dimensional model at the n view angles, where n is an integer greater than 1, and the confidence weight corresponding to a first view angle among the n view angles is negatively correlated with a degree of deviation of the vertex observed at the first view angle; a processing module, configured to use the confidence weights at the n viewing angles as color weights of the viewing images at the n viewing angles, respectively, and perform texture baking on the model structure of the three-dimensional model based on the viewing images at the n viewing angles to obtain a three-dimensional color model; a correction module, used for acquiring an observed image of the three-dimensional color model at the n viewing angles, and performing correction on the viewing angle image according to a difference between the observed image and the viewing angle image to obtain a corrected viewing angle image; The processing module is further used to perform texture baking on the model structure of the three-dimensional model based on the corrected perspective image to obtain a corrected color model.

13. A computer device, characterized in that: The computer device includes: a processor and a memory, wherein the memory stores at least one program; the processor is used to execute the at least one program in the memory to implement the three-dimensional model processing method as described in any one of claims 1 to 11.

14. A computer-readable storage medium, characterized in that: The readable storage medium stores executable instructions, and the executable instructions are loaded and executed by the processor to implement the three-dimensional model processing method as described in any one of claims 1 to 11.

15. A computer program product, characterized in that The computer program product includes computer instructions, which are stored in a computer-readable storage medium. The processor reads and executes the computer instructions from the computer-readable storage medium to implement the three-dimensional model processing method as described in any one of claims 1 to 11.

Images