Three-dimensional model generation method and device, medium and product

Abstract

The invention discloses a three-dimensional model generation method and device, a medium and a product, and relates to the field of artificial intelligence. The method comprises the following steps: acquiring an object image of a first object; extracting texture feature representation of the first object from the object image; obtaining a first object model; adjusting a model structure of the first object model based on the texture feature representation, so that the first object model is matched with the texture feature representation to obtain a second object model; and converting the texture feature representation into a texture image, and fitting the texture image to the second object model to obtain a three-dimensional model corresponding to the first object. The contour of the first object model is adjusted under the constraint of the appearance performance, so that the texture is matched with the adjusted first object model, and the three-dimensional model corresponding to the first object is obtained after the texture image is mapped and covered to the adjusted first object model, so that the generation efficiency of the three-dimensional model and the man-machine interaction efficiency are improved, and the user experience is improved. And time resources and human resources are saved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and in particular to a method, apparatus, medium, and product for generating three-dimensional models. Background Technology

[0002] In virtual scene-based applications, there is often a need for 3D modeling and rendering. For example, virtual characters in a virtual scene need to be generated through 3D modeling so that they can move around in the virtual scene.

[0003] In related technologies, artists and programmers work together to create 3D models in virtual scenes. Artists create 3D models through design and construction, while programmers configure the 3D models to enable them to be rendered and displayed in the virtual scene.

[0004] However, the above-mentioned method of generating 3D models requires a lot of time resources, resulting in low efficiency in generating 3D models. Summary of the Invention

[0005] This application provides a method, apparatus, medium, and product for generating three-dimensional models, which can improve the efficiency of generating three-dimensional models. The technical solution is as follows:

[0006] On the one hand, a method for generating a three-dimensional model is provided, the method comprising:

[0007] Obtain the image of the first object, where the first object is the object to be generated into a 3D model;

[0008] Extract the texture feature representation of the first object from the object image;

[0009] Obtain the first object model, which is a pre-provided basic 3D model;

[0010] The model structure of the first object model is adjusted based on the texture feature representation so that the model structure of the first object model matches the texture structure of the texture feature representation, thereby obtaining the second object model;

[0011] The texture feature representation is converted into a texture image, and the texture image is attached to the second object model to obtain the three-dimensional model corresponding to the first object.

[0012] On the other hand, a three-dimensional model generation apparatus is provided, the apparatus comprising:

[0013] The acquisition module is used to acquire the object image of the first object, which is the object to be generated into a 3D model;

[0014] The extraction module is used to extract the texture feature representation of the first object from the object image;

[0015] The acquisition module is further configured to acquire a first object model, wherein the first object model is a pre-provided basic three-dimensional model;

[0016] An adjustment module is used to adjust the model structure of the first object model based on the texture feature representation, so that the model structure of the first object model matches the texture structure of the texture feature representation, thereby obtaining a second object model;

[0017] The generation module is used to convert the texture feature representation into a texture image and attach the texture image to the second object model to obtain the three-dimensional model corresponding to the first object.

[0018] On the other hand, a computer device is provided, the computer device including a processor and a memory, the memory storing at least one instruction, at least one program, code set or instruction set, the at least one instruction, the at least one program, the code set or instruction set being loaded and executed by the processor to implement the method for generating a three-dimensional model as provided in the embodiments of this application above.

[0019] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction, at least one program, code set, or instruction set is stored in the storage medium, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the method for generating a three-dimensional model as provided in the embodiments of this application above.

[0020] On the other hand, a computer program product is provided that, when the computer program product is run on a computer, causes the computer to execute the method for generating a three-dimensional model as provided in the embodiments of this application above.

[0021] The beneficial effects of the technical solutions provided in this application include at least the following:

[0022] By acquiring the object image of the first object and extracting its texture feature representation, the appearance of the first object is obtained. After acquiring the first object model as the base model, the outline of the first object model is adjusted under the constraints of the appearance representation to match the texture with the adjusted first object model. The texture image is then mapped and overlaid onto the adjusted first object model to obtain the 3D model corresponding to the first object. This eliminates the need for artists to design and program the 3D model corresponding to the first object separately, improving the efficiency of 3D model generation and human-computer interaction, and saving the time and human resources consumed in 3D model generation. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a schematic diagram of a three-dimensional model generation framework provided in an exemplary embodiment of this application;

[0025] Figure 2 This is a flowchart of a method for generating a three-dimensional model provided in an exemplary embodiment of this application;

[0026] Figure 3 This is a schematic diagram illustrating the control point adjustment effect provided in an exemplary embodiment of this application;

[0027] Figure 4 This is a flowchart of a method for generating a three-dimensional model provided in another exemplary embodiment of this application;

[0028] Figure 5 This is a flowchart of a method for generating a three-dimensional model provided in yet another exemplary embodiment of this application;

[0029] Figure 6 This is a flowchart of a method for generating a three-dimensional model provided in another exemplary embodiment of this application;

[0030] Figure 7 This is a flowchart of a method for generating three-dimensional virtual characters in a game, provided in an exemplary embodiment of this application;

[0031] Figure 8 This is an overall flowchart of a three-dimensional virtual character generation method provided in an exemplary embodiment of this application;

[0032] Figure 9 This is a schematic diagram illustrating the effect of generating a three-dimensional virtual character according to an exemplary embodiment of this application;

[0033] Figure 10 This is a flowchart of a method for generating a three-dimensional vehicle model provided in an exemplary embodiment of this application;

[0034] Figure 11 This is a structural block diagram of a three-dimensional model generation apparatus provided in an exemplary embodiment of this application;

[0035] Figure 12 This is a structural block diagram of a three-dimensional model generation apparatus provided in another exemplary embodiment of this application;

[0036] Figure 13This is a structural block diagram of a computer device provided in an exemplary embodiment of this application. Detailed Implementation

[0037] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0038] To improve the efficiency of generating 3D models, this application provides a method for generating 3D models. This method involves providing a 2D modeling image, extracting texture features from the 2D modeling image, and obtaining a pre-designed basic 3D model. The method then combines the texture features of the 2D image and the basic 3D model to generate a 3D model corresponding to the 2D modeling image. The method provided in this application eliminates the need for artists to design and manually model the 3D model based on drawings, thus improving the efficiency of 3D modeling.

[0039] Indicative Figure 1 This is a schematic diagram of a three-dimensional model generation framework provided in an exemplary embodiment of this application, such as... Figure 1 As shown, the implementation of a 3D model into a face model is illustrated as an example. That is, after obtaining a 2D image of a face, a corresponding 3D face model is constructed based on the 2D image.

[0040] First, a face image 110 is acquired, and a texture feature representation 120 is extracted from the face image 110 using a pre-trained feature extraction network. This texture feature representation 120 is used to characterize the appearance features of the face image 110, such as color, lighting, and facial wrinkles. Next, a pre-set 3D reference model 130 is acquired. This 3D reference model 130 is a pre-provided head 3D model, and it includes multiple control points. These control points are used to control the contour structure of the 3D reference model 130. By adjusting the coordinate positions of the control points, the positions of various parts on the 3D reference model 130 are adjusted. For example, by adjusting the coordinate positions of the control points, the positions of the facial features on the 3D reference model 130 are adjusted. For instance, by moving the control point corresponding to the nose closer to the eyes, the nose on the 3D reference model 130 is brought closer to the eyes, and the distance between the nose and eyes is shortened. This can also be understood as lengthening the philtrum on the 3D reference model 130.

[0041] In this embodiment, the contour of the three-dimensional reference model 130 is adjusted according to the texture structure of the texture feature representation 120, so that the contour of the three-dimensional reference model 130 matches the texture feature representation 120. After adjusting the three-dimensional reference model 130, the texture feature representation 120 is restored to a texture image, and the texture image is attached to the adjusted three-dimensional reference model 130 to obtain a three-dimensional model 140 corresponding to the face image 110.

[0042] In some embodiments, such as Figure 1 As shown, after the terminal 100 acquires the face image 110, it sends the face image 110 to the server 101. The server 101 extracts the texture feature representation 120 of the face image 110, obtains the three-dimensional reference model 130 from the model library, and generates the three-dimensional model 140 corresponding to the face image 110 based on the three-dimensional reference model 130 and the face image 110. The data of the three-dimensional model 140 is then fed back to the terminal 100.

[0043] It is worth noting that the aforementioned servers can be independent physical servers, server clusters or distributed systems composed of multiple physical servers, or cloud servers that provide basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0044] Cloud technology refers to a hosting technology that unifies hardware, software, and network resources within a wide area network (WAN) or local area network (LAN) to achieve data computation, storage, processing, and sharing. Based on the cloud computing business model, cloud technology encompasses network technology, information technology, integration technology, management platform technology, and application technology. It can form resource pools, providing flexible and convenient on-demand access. Cloud computing technology will become a crucial support. Backend services of technical network systems require substantial computing and storage resources, such as video websites, image websites, and many portal websites. With the rapid development and application of the internet industry, every item may have its own identification mark in the future, requiring transmission to backend systems for logical processing. Data at different levels will be processed separately, and various industry data will require robust system support, which can only be achieved through cloud computing.

[0045] In some embodiments, the server described above can also be implemented as a node in a blockchain system.

[0046] It should be noted that all information (including but not limited to image information, user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals involved in this application have been authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the image data, 3D models, and other content involved in this application were obtained with full authorization.

[0047] It should be understood that although the terms first, second, etc., may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, a first parameter may also be referred to as a second parameter, and similarly, a second parameter may also be referred to as a first parameter. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0048] Figure 2 This is a flowchart of a method for generating a 3D model provided in an exemplary embodiment of this application. This method can be executed by a terminal, by a server, or by a combination of both. In this embodiment, the method is illustrated using an application on a server as an example. Figure 2 As shown, the method includes the following steps.

[0049] Step 201: Obtain the object image of the first object.

[0050] The first object is the object to be generated as a 3D model. A 3D model refers to a model structure constructed on a computer device that is presented in a three-dimensional form. A 3D model can also be implemented as a 3D virtual model, which includes models that act as virtual entities in a 3D virtual scene. For example, a 3D virtual model can be implemented as a virtual character model, virtual animal model, virtual building model, etc., within a virtual scene.

[0051] The first object can be implemented as at least one of the following types: character object, building object, animal object, decoration object, prop object, container object, etc. Taking a character object as an example, a 3D model of the first character is generated based on the character image. Specifically, a 3D head model of the first character is generated based on the facial image.

[0052] In some embodiments, the facial image of the first person can be a frontal facial image or a side facial image. If it is a frontal facial image, a model contour of the frontal facial region is generated based on the frontal facial image, and the back of the head contour is supplemented based on a pre-provided base model. If it is a side facial image, a model contour of a first side profile is generated based on the side facial image, and a model contour of a second side profile is generated based on symmetry and a pre-provided base model. That is, a partial facial model contour is generated based on an existing facial image, and the facial model contour of the missing image portion is supplemented based on the base model.

[0053] In some embodiments, the object image of the first object can be an image acquired from a single viewpoint or multiple images acquired from multiple viewpoints.

[0054] In some embodiments, obtaining the object image of the first object includes at least one of the following methods:

[0055] First, receive the object image uploaded by the account. The object image includes the image content corresponding to the first object. For example, if the first object is implemented as a person object, then receive the object image uploaded by the account. The object image includes the person's head.

[0056] Among them, obtaining object images by receiving object images uploaded by the account is simple and quick. Users can select existing images from their albums as object images for uploading without having to collect them again.

[0057] Second, an object image of the first object is captured using a camera. Specifically, if this embodiment is executed by a terminal, image capture is performed through a camera module connected to the terminal; when the first object is within the camera module's capture range, an object image of the first object is captured. If this embodiment is executed by a server, the terminal performs image capture through a camera module; when the first object is within the camera module's capture range, an object image of the first object is captured, and the object image of the first object is uploaded to the server.

[0058] In this embodiment, when the first object is within the capture range of the camera module, the terminal receives a shooting operation and captures an image of the first object; or, the terminal performs real-time image content recognition on the image captured by the camera module. When it recognizes that the image captured by the camera module includes image content of a first type, it automatically triggers a shooting operation to capture an image of the first object. The first object corresponds to the first type. For example, if the first object is implemented as a person, then when the terminal recognizes that the image captured by the camera module includes person content, it automatically triggers a shooting operation to obtain an image of the person. In some embodiments, when the terminal recognizes that the image captured by the camera module includes image content of the first type, it selects and highlights the image content of the first type, such as highlighting the area of ​​the person in the image and displaying a countdown. The shooting operation is automatically triggered when the countdown ends. The first type of image content is identified by a pre-trained machine learning model.

[0059] By acquiring the image of the first object through a capture method, the user avoids the need to take photos with a third-party camera when the image of the object is not in the user's album. The method directly calls the camera to capture the image of the object, which improves the efficiency and real-time performance of the object image acquisition.

[0060] Third, video data is captured by a camera, and video frames are obtained from the video data as the object image of the first object. Specifically, if this embodiment is executed by a terminal, video data is captured through a camera module connected to the terminal, and when the first object is within the camera module's capture range, video frames of the first object are obtained as the object image. If this embodiment is executed by a server, the terminal captures video data through the camera module and uploads the video data to the server.

[0061] In some embodiments, when video data is collected through the camera, if the first type of image content is detected within the collection range of the camera module, the collection and storage of video data is triggered; if the first type of image content is not detected within the collection range of the camera module, the collection and storage of video data is paused. Alternatively, video data is continuously collected and stored through the camera module, and the first type of image content is identified in the collected and stored video data. The identified video segments are then automatically edited and stored separately.

[0062] In some embodiments, when obtaining video frames of a first object from video data, a selection operation on video frames in the video data is received, and the selected video frames are used as object images of the first object; or, if the video data includes video segments of the first object and other video segments, a video segment containing image content of the first type is first identified from the video data using a pre-trained machine learning model, and used as a video segment of the first object, and then video frames are obtained from the video segments of the first object as object images of the first object; if the video data only includes video segments of the first object and does not include other video segments, then video frames are directly obtained from the video segments of the first object as object images of the first object. Wherein, when obtaining video frames from video segments of the first object, parameters such as sharpness, contrast, and object display area of ​​each video frame or keyframe in the video segment are identified using a pre-trained machine learning model, and the video frame with the largest object display area, the highest sharpness, or the highest contrast is selected; or, the sharpness, contrast, and object display area are comprehensively calculated to obtain a comprehensive score for each video frame, and the video frame with the highest comprehensive score is selected as the object image of the first object. To illustrate, a weighted fusion calculation is performed on sharpness, contrast, and the area of ​​the object display area to obtain a comprehensive score. The weights for sharpness, contrast, and the area of ​​the object display area are pre-designed.

[0063] By acquiring the object image of the first object through video capture, and using a machine learning model to obtain image frames with better clarity, contrast, and display area as the object image while continuously capturing multiple frames, the quality of the object image acquisition of the first object is improved, thereby improving the quality of the generated 3D model of the first object.

[0064] It is worth noting that the above-described method for obtaining the object image of the first object is merely an illustrative example, and the specific method for obtaining the object image of the first object is not limited in the embodiments of this application.

[0065] Step 202: Extract the texture feature representation of the first object from the object image.

[0066] Texture features represent the appearance characteristics of a first object in an image, specifically texture features.

[0067] In some embodiments, a texture feature representation of a first object is extracted from an object image using a pre-trained feature extraction model. Here, texture feature representation refers to the feature vector obtained by extracting features such as color, illumination, and texture of the first object from the object image. The texture feature representation of the first object is extracted from the object image using a convolutional neural network or other image processing techniques. Taking a person as an example, the feature extraction model extracts features such as color, illumination, and wrinkles of the first object to obtain the texture feature representation.

[0068] In this embodiment, the extraction of texture feature representation of the first object in an image using Convolutional Neural Networks (CNNs) is used as an example. A Convolutional Neural Network is a deep learning model composed of multiple convolutional layers, pooling layers, fully connected layers, and other network layers, capable of automatically learning high-level semantic features of an image.

[0069] The following section introduces the various layers in a convolutional neural network: 1. Convolutional Layers: Convolutional layers extract primary features of an image through convolution operations, such as edges, color, and texture. Each neuron in a convolutional layer is connected to an image region of the object image and performs a dot product operation with the pixel values ​​of pixels in the image region through a convolution kernel (or filter). By moving the filter kernel, the convolution kernel covers the entire image, capturing local features. 2. Activation Functions: The convolutional feature map is usually processed by a non-linear activation function, such as the ReLU (Rectified Linear Unit) function, to introduce non-linear characteristics into the image. 3. Pooling Layers: Pooling layers are used to reduce the dimensionality and number of parameters of the convolutional feature map, reducing computational cost. Pooling operations help extract rotation- and translation-invariant features by preserving the main features in the feature map (such as maximum or average values), while preventing overfitting. 4. Repeated Convolution and Pooling Operations: By repeating convolutional and pooling layers, convolutional neural networks (CNNs) can gradually transition from extracting simple features to extracting complex features. In different layers of the network, lower layers focus on general features (such as edges and textures), while higher layers can recognize more complex and abstract concepts (such as parts and the whole of an object). 5. Fully Connected Layers: After multiple convolutional and pooling layers, the final layer of a CNN typically contains one or more fully connected layers. Fully connected layers combine the features learned from previous layers to perform classification or other tasks. Complex relationships between features are learned in fully connected layers, and the final prediction is output.

[0070] Based on the above description of each network layer in the convolutional neural network, the extraction process of texture feature representation of the convolutional neural network in the embodiments of this application will be introduced.

[0071] First, the convolutional layer is the core part of the convolutional neural network. It uses a set of convolutional kernels to perform convolution operations on the input object image to obtain a set of feature maps (or activation maps). The convolution operation can be expressed as Equation 1 below.

[0072] Formula 1:

[0073] Where k represents the convolution operation performed through the k-th convolution kernel, F k,i,j W represents the value in the i-th row and j-th column of the feature map extracted by the k-th convolutional kernel. k,m,n b represents the weight used in the m-th row and n-th column of the k-th convolutional kernel during the convolution operation. k This represents the bias term of the k-th convolutional kernel, where M and N are the height and width of the kernel, respectively. The function of a convolutional kernel is to extract local features from an image, such as edges, corners, and textures. Different convolutional kernels can extract different features; therefore, multiple convolutional kernels are used to increase the diversity and richness of features.

[0074] After performing a convolution operation on the object image through a convolutional layer, nonlinear activation is applied to the feature image output by the convolutional layer to increase the model's expressive power and nonlinear fitting ability. Commonly used activation functions include ReLU, Sigmoid, and Tanh. In this embodiment, the Sigmoid function is used as an example for illustration. The definition of the Sigmoid function is as follows: Formula 2.

[0075] Formula 2: A(x) = max(0, x)

[0076] Here, x is the input, and A(x) is the output, which is the maximum value of x among 0 and x in the input feature image. The advantages of the Sigmoid function are its simplicity, speed, and sparsity, which can effectively solve the gradient vanishing problem and improve the training efficiency and performance of the model.

[0077] Then, the feature image output by the activation layer is pooled (or downsampled) to reduce the dimensionality and computational cost of the features, while preserving the main information of the features and enhancing their robustness and translation invariance. Commonly used pooling methods include max pooling, average pooling, and random pooling. Max pooling will be used as an example for explanation. Max pooling is defined as shown in Formula 3 below.

[0078] Formula 3: P k,i,j =max m,n∈R i,j A k,m,n

[0079] Among them, P k,i,j A represents the value in the i-th row and j-th column of the k-th pooling layer. k,m,n R represents the value in the m-th row and n-th column of the k-th activation layer. i,j This represents the pooling region corresponding to the i-th row and j-th column of the k-th pooling layer. Max pooling aims to retain the maximum response in the feature map, i.e., the most salient feature, while suppressing other interfering information.

[0080] The output features of the pooling layer are flattened into a one-dimensional vector, which serves as the texture feature representation of the image for subsequent classification or matching tasks. The flattening operation can be expressed as shown in Formula 4 below.

[0081] Formula 4: V = flatten(P)

[0082] Where V is a vector of length K×M′×N′, P is a tensor of length K×M′×N′, K is the number of feature maps, and M′ and N′ are the height and width of the pooling layer, respectively. In some embodiments, the flattening operation can be implemented by expanding all elements of the tensor into a continuous vector in a specific order (usually row-major or column-major). In the embodiments of this application, the flatten algorithm is implemented by expanding the K×M′×N′ tensor into a K×M′×N′ vector in row-major order.

[0083] In some embodiments, the object image is preprocessed before extracting texture feature representation. The preprocessing methods include at least one of grayscale conversion, histogram equalization, and normalization to reduce the influence of factors such as noise and illumination in the object image and enhance its contrast and clarity.

[0084] Grayscale conversion refers to the process of transforming the pixel values ​​of an object image into grayscale, thereby converting a color object image into a grayscale image. Grayscale conversion includes at least one of the following methods: 1、 Average Method: The average of the red (R), green (G), and blue (B) color channel values ​​of each pixel in a color image is used to replace the original RGB values, resulting in a grayscale image. 2、 Weighted Average Method: The human eye has varying sensitivities to different colors. To make the grayscale image more visually appealing, the values ​​of the RGB channels are typically weighted and averaged. For example: Gray = 0.299R + 0.587G + 0.114B, meaning the red channel has a weight of 0.299, the green channel has a weight of 0.587, and the blue channel has a weight of 0.114. 3、Maximum Method: This method selects the maximum value among the R, G, and B channels as the grayscale value of a pixel. This method is simple and fast, but may result in a loss of image detail. 4、 Component Method: This method uses only one component of the RGB color spectrum as the grayscale value. It is simple, but the grayscale value can vary greatly depending on the selected color component. 5、 Custom conversion method: The conversion formula is customized according to specific application requirements. For example, in certain situations, it may be necessary to adjust the weights of different color channels in grayscale processing based on ambient light, background color, or other factors. It is worth noting that the above grayscale processing methods are merely illustrative examples, and this embodiment does not limit them.

[0085] Histogram equalization is used to improve the contrast of an image. By adjusting the histogram distribution, it broadens the overall histogram range, thus enhancing the overall contrast. Histogram equalization can be applied not only to grayscale images but also to individual channels of color images. The basic steps of histogram equalization are as follows: 1. Calculate the histogram: First, calculate the histogram of the image, i.e., count the frequency of each gray level. 2. Calculate the Cumulative Distribution Function (CDF): Calculate the CDF based on the histogram. The CDF is the cumulative number of pixels in the image that are less than or equal to a certain gray value. 3. Histogram equalization: Using the CDF, map each pixel value in the original image to a new gray level, thereby increasing the image contrast. 4. Create a new image: Using the mapping relationship from the previous step, assign a new gray value to each pixel in the original image, thus creating the equalized image.

[0086] Normalization is used to adjust image data to bring pixel values ​​within a specific range, typically between 0 and 1 or -1 and 1. Normalization helps improve the efficiency and accuracy of subsequent image processing or machine learning models when processing image data. Normalization methods include at least one of the following: 1、 Simple scaling normalization: This is achieved by dividing each pixel value by the maximum possible pixel value (255 for an 8-bit image). This ensures that all pixel values ​​are between 0 and 1. 2、Min-Max Normalization: Also known as feature scaling, its purpose is to scale image pixel values ​​to a range of 0 to 1. This is achieved by subtracting the minimum pixel value and then dividing by the range of pixel values. That is, the minimum and maximum pixel values ​​of each pixel in the object image are determined, and the pixel difference between the maximum and minimum pixel values ​​is determined. During normalization, the minimum pixel value is subtracted from the pixel value, and then the difference is calculated to obtain the normalized value. It is worth noting that the above normalization method is only an illustrative example, and this embodiment does not limit it.

[0087] Step 203: Obtain the first object model.

[0088] The first object model is a pre-provided basic 3D model. Specifically, the first object model serves as the foundational template structure for generating the 3D representation of the first object. That is, based on the first object model, after adjusting the first object model by incorporating the texture features of the first object, the 3D model of the first object is obtained.

[0089] When obtaining the first object model, at least one of the following methods is included:

[0090] The first method involves obtaining a predefined template model, which is a predefined basic 3D model used as the first object model. For example, if the first object is implemented as a human figure, then the first object model is implemented as a predefined basic human face model. This first object model is applicable to the 3D model generation process of any human figure, such as children, boys, girls, etc. All human figures are generated using this first object model.

[0091] By using a predefined template model as the first object model, the same template model can be applied to different types of objects, avoiding the design of multiple template models and reducing the amount of data stored in the computer.

[0092] The second method is to obtain the first object type of the first object, and then obtain the first object model that matches the first object type from multiple candidate object models based on the first object type.

[0093] In this model, multiple candidate object models correspond to different object types. In some embodiments, the multiple candidate object models are multiple candidate parameterized models; that is, since the multiple candidate object models correspond to different object types, their model parameters are also different. For example, children correspond to candidate object model 1, girls to candidate object model 2, and boys to candidate object model 3. When the first object is a child, candidate object model 1 is selected as the first object model. In some embodiments, after recognizing the above texture feature representation using a pre-trained image recognition model, the first object type corresponding to the first object is obtained. The image recognition model is used to classify the first object among different object types to obtain the first object type.

[0094] Different parametric models are provided for different types of objects, so that different types of objects only need to be fine-tuned based on the parametric model. When the model adjustment range is small, it is not easy to produce distortion problems caused by large adjustments, and the generation accuracy of 3D models is high.

[0095] The third method involves using a pre-trained model to generate a network that recognizes object images or the aforementioned texture feature representations to generate a basic first object model.

[0096] Optionally, the first object model includes a set of control points and a set of basis functions.

[0097] Control points are used to control the shape of the mesh (or fragment) in the first object model, as illustrated below. Figure 3 As shown, meshes 310 and 320 of the 3D model are connected as part of the outline of the 3D model. By adjusting the position of control point 311, meshes 310 and 320 of the 3D model are deformed, thereby causing the outline of the 3D model to change.

[0098] Basis functions for 3D models are mathematical functions used to define shapes, surfaces, or volumes when constructing 3D models. Basis functions are a core concept in 3D modeling and computer graphics, used to describe the geometry and properties of objects. The selection and application of basis functions vary across different modeling methods. Basis functions for 3D models include at least one of the following forms: 1. Polynomial basis functions: In many 3D modeling techniques, polynomial basis functions are used to represent curves, surfaces, or volumes. For example, Bézier curves and surfaces use specific polynomial basis functions to define the shape of the curve or surface. Combinations of these basis functions can construct complex geometric shapes. 2. Spline functions: A special type of piecewise polynomial function widely used in computer graphics and 3D modeling to create smooth curves and surfaces. B-spline (B-spline) basis functions are mathematical tools for constructing these smooth curves and surfaces, allowing precise control of the shape by adjusting control points. 3. Subdivision surfaces: A technique for generating smooth surfaces by recursively subdividing the control mesh. This method relies on specific subdivision rules (i.e., basis functions), which define how to modify the vertices (i.e., the control points mentioned above) and edges of the mesh in each subdivision step. It is worth noting that the types of basis functions described above are merely illustrative examples, and the embodiments of this application do not limit the types of basis functions.

[0099] After obtaining the first object model, the first object model is represented as shown in Formula 5 below.

[0100] Formula 5:

[0101] Where M is the basic 3D model, c i B is the coordinate of the i-th control point. i Here, is the basis function corresponding to the i-th control point, and n is the number of control points. The advantage of parametric models is that the shape of the model can be changed by adjusting the positions of the control points to adapt to different object model characteristics.

[0102] Step 204: Adjust the model structure of the first object model based on the texture feature representation so that the model structure of the first object model matches the texture structure of the texture feature representation, thereby obtaining the second object model.

[0103] The texture feature representation includes feature points, and the first object model includes control points. The position of the control points is adjusted based on the feature points so that the model structure of the first object model matches the texture structure of the texture feature representation.

[0104] In this context, the feature points in the texture feature representation are the feature representations of the image regions corresponding to specified parts in multiple image regions when performing texture analysis on the object image of the first object. The control points in the first object model are pre-set mesh vertices used to control the structure of the first object model.

[0105] The model structure refers to the outline structure exhibited by the outer contour of the first object model, that is, the relative positional relationship between different object parts of the first object as exhibited by the outer contour of the first object model. In some embodiments, the model structure is determined based on the structural units that construct the first object model. For example, the structural units that construct the first object model are implemented as mesh units that construct the first object model. Taking a triangular mesh unit as an example, multiple triangular meshes are spliced ​​together to obtain the outer contour of the first object model, that is, the model structure. The vertices of the triangular mesh are implemented as control points that control the shape of the triangular mesh. Alternatively, the structural units that construct the first object model are implemented as voxel blocks that construct the first object model. Multiple voxel blocks are used to build the outer contour of the first object model, that is, the model structure. The outer contour of the first object model is adjusted by controlling the position of each voxel block.

[0106] Texture structure refers to the relative positional relationship between different parts of a first object as represented by texture features. In some embodiments, since different parts of the object exhibit different texture features, the texture feature representation can represent each part of the first object, thereby showing the distribution and relative positional relationship of each part of the first object, which is the texture structure.

[0107] In some embodiments, feature points are obtained in the texture feature representation. Feature points are points used to characterize the features of image regions when extracting the texture feature representation of an object image. For example, image feature representation is extracted from a 500×500 image, where feature points are extracted from the 20×20 image region in the upper left corner and used as an element in the vector of the texture feature representation. In some embodiments, a feature point refers to a point used to characterize the features of a specified image region when extracting the texture feature representation of an object image. For example, if the image region corresponding to the facial features and contours of a person's face in the object image is used as a feature point, and when extracting the texture feature representation of image region A in the object image, the nose region of the face corresponding to image region A is identified, then the texture feature representation extracted from image region A is used as a feature point.

[0108] Optionally, the texture feature representation is identified to obtain feature points corresponding to specified key points in the texture feature representation. For example, facial key points mark important features and parts of the face, such as the eyes, nose, mouth, chin, and facial contours. For illustration, 68 key points are used as an example; the eyes, eyebrows, nose, mouth, chin, and facial contours are labeled using 68 facial key points to obtain feature points corresponding to the facial key points in the texture feature representation.

[0109] The control points used to control the model structure in the first object model are obtained. Based on the feature points, the positions of the control points in the first object model are adjusted so that the positions of the feature points and the control points are relatively consistent. This means that the positions of the feature points in the texture feature representation are relatively consistent with the positions of the control points in the first object model, thus matching the model structure of the first object model with the texture structure of the texture feature representation, resulting in the second object model. Here, "the positions of the feature points in the texture feature representation are relatively consistent with the positions of the control points in the first object model" means that after mapping the first object model to a two-dimensional plane, the positions of the control points on the mapped two-dimensional plane are consistent with the positions of the feature points in the texture feature representation.

[0110] In some embodiments, control points corresponding to specified key point positions in the first object model are obtained, and the control points corresponding to specified key point positions are adjusted based on feature points so that the positions of the control points corresponding to specified key point positions are relatively consistent with those of the feature points, thereby making the model structure of the first object model match the texture structure represented by the texture features.

[0111] In some embodiments, when the object image of the first object is acquired from the side of the first object, the texture feature representation also corresponds to the representational features of the side of the first object. Therefore, when adjusting the control points of the side region of the first object model according to the texture feature representation, the control points of the other side of the first object model are simultaneously adjusted according to the symmetry of the first object model. When the object image of the first object is acquired from the front of the first object, the texture feature representation also corresponds to the representational features of the front of the first object. Therefore, the control point coordinates of the front region of the first object model are adjusted according to the texture feature representation, and the back region of the first object model is represented according to the structure of the first object model itself, without the need for additional adjustments based on the texture feature representation. Alternatively, there may be a relationship between the control points of the first object model, such as: adjusting the position of the first control point will lead to a synchronous adjustment of the second control point. Therefore, the control points of the back region are adjusted synchronously according to the adjustment of the front control points. The relationship between different control points is pre-defined, that is, an association is configured between multiple control points, and this association corresponds to a relationship where one control point is adjusted when a pair of control points is affected. This application does not limit the representation of the first object model outside the image acquisition range of the first object.

[0112] The method provided in this embodiment adjusts the control points within the first object image acquisition range accordingly, while the control points outside the first object image acquisition range are positioned based on their location on the first object model, thus improving the efficiency of model adjustment.

[0113] In some embodiments, the first object image comprises multiple images, and the multiple first object images correspond to different angles of the first object. The coordinates of each control point on the first object model are then adjusted according to the angles corresponding to the multiple first object images.

[0114] Step 205: Convert the texture feature representation into a texture image, and attach the texture image to the second object model to obtain the three-dimensional model corresponding to the first object.

[0115] In some embodiments, the texture feature representation is decoded to obtain a texture image, which is then mapped to a second object model to generate a mapping relationship between the texture image and the second object model. This mapping relationship includes the alignment between the k-th pixel in the texture image and the p-th 3D vertex in the object model, where k and p are positive integers. Based on this mapping relationship, the texture image is fitted to the second object model to obtain the 3D model corresponding to the first object.

[0116] In some embodiments, since the texture image is a two-dimensional planar image and the second object model is a three-dimensional model, after determining the mapping relationship between the texture image and the second object model, the target pixels in the texture image that exist in the mapping relationship are attached to the three-dimensional vertex surface of the object model based on the mapping relationship, and the texture image is extended between multiple target pixels so that the texture image covers the surface of the object model and completely envelops the surface of the object model.

[0117] In summary, the 3D model generation method provided in this embodiment obtains the object image of a first object and extracts the texture feature representation of the object image to obtain the appearance of the first object. After obtaining the first object model as the base model, the outline of the first object model is adjusted under the constraint of the appearance representation so that the texture matches the adjusted first object model. The texture image is then mapped and overlaid onto the adjusted first object model to obtain the 3D model corresponding to the first object. This eliminates the need for artists to design and program the 3D model corresponding to the first object separately, improving the generation efficiency of 3D models and the efficiency of human-computer interaction, and saving the time and human resources consumed in the generation of 3D models.

[0118] The method provided in this embodiment obtains a first object model corresponding to the object type from an object model library based on the object type of the first object, and then generates a three-dimensional model of the first object after adjusting the first object model, thereby improving the generation efficiency and accuracy of the object model.

[0119] In some embodiments, when adjusting the model structure of the first object model based on the texture feature representation, the control points of the first object model are positioned according to the feature points in the texture feature representation. Figure 4This is a flowchart of a method for generating a 3D model provided in another exemplary embodiment of this application. This method can be executed by a terminal, by a server, or by a combination of both. In this embodiment, the method is illustrated using an application to a server as an example. Figure 4 As shown, step 204 above can be implemented as steps 2041 to 2044 below.

[0120] Step 2041: Obtain feature points in the texture feature representation.

[0121] In this context, feature points are used to characterize image region features. Feature points are points used to characterize image region features when extracting texture feature representations of an object image. In some embodiments, feature points are feature vectors extracted from the image region corresponding to a specified part of the first object in the texture feature representation. The feature vectors of these feature points are used to characterize the regional pixel features of the image region corresponding to the specified part.

[0122] In some embodiments, feature points are pre-specified points in the texture feature representation used to characterize specified parts of the first object. Illustratively, taking a face object as an example, feature points refer to pre-specified facial key points in a face image. In some embodiments, facial key point recognition is performed on the texture feature representation to obtain feature points corresponding to the facial key points in the texture feature representation. In some embodiments, the object image is an image obtained by capturing the frontal view of the first object, or an image obtained by capturing the side view of the first object. Facial key points are points corresponding to key facial parts obtained after key point recognition in the texture feature representation, such as: eyes, nose, eyebrows, contours, etc. Feature points are feature representations in the texture feature representation corresponding to the facial key points.

[0123] In some embodiments, facial key points are used as an example for illustration. Facial key points are associated with key point identifiers. For example, the key point at the left corner of the mouth corresponds to key point identifier 10. In the texture feature representation, the feature representation extracted from the left corner of the mouth region corresponds to key point identifier 10.

[0124] Step 2042: Obtain the control points in the first object model used to control the model structure.

[0125] In some embodiments, control points corresponding to specified parts of the first object are pre-annotated in the first object model. For example, taking a person object as an illustration, control points corresponding to facial features and contours are pre-annotated in the first object model. There is a correspondence between the control points and the aforementioned feature points. For example, taking a face as an illustration, the control points are the mesh vertices in the first object model corresponding to facial keypoints. Optionally, control points have corresponding control point identifiers. Taking facial keypoints as an example, facial keypoints have corresponding keypoint identifiers, such as keypoint identifier 10 for the left corner of the mouth. Therefore, in the first object model, the control point at the left corner of the mouth corresponds to control point identifier 10. It is worth noting that control point identifiers and feature point identifiers can be different; the above example uses the same identifier.

[0126] In some embodiments, the control points in the first object model are preset. When the first object model is obtained as the base 3D model, the positions of the control points in the first object model are default and preset. The control points themselves are implemented as dynamically changeable points, meaning that the positions of the control points can be dynamically changed by adjustment. In this embodiment, after obtaining the first object model as the base 3D model, the contour structure, i.e., the model structure, of the first object model is changed by adjusting the control points in the first object model, thereby obtaining the second object model.

[0127] Step 2043: Determine the loss data between the first coordinates of the feature point and the second coordinates of the control point using a preset loss function.

[0128] Optionally, the first reference point in the texture feature representation is used as the origin to determine the first coordinates corresponding to the feature points in the texture feature representation; the first object model is mapped from three dimensions to a two-dimensional plane, and the second reference point on the two-dimensional plane is used as the origin to determine the second coordinates corresponding to the control points on the two-dimensional plane, wherein the first reference point and the second reference point are points corresponding to preset positions; for the i-th feature point, the comprehensive loss data between the first coordinates of the i-th feature point and the second coordinates of multiple control points is determined by a preset loss function to obtain the loss data between multiple feature points and multiple control points, where i is a positive integer.

[0129] In some embodiments, a texture coordinate system is constructed using a first reference point pre-specified in the texture feature representation as the origin, such as the top-left corner of the texture feature representation. The first coordinates of the feature points are determined based on the texture coordinate system.

[0130] In some embodiments, when mapping the first object model from three dimensions to a two-dimensional plane, the acquisition angle of the first object in the object image is identified based on texture feature representation. The first object model is then mapped to the two-dimensional plane using this acquisition angle. For example, by identifying the texture feature representation, it is determined that the first object in the object image is an image acquired from a side profile view from the left side. Therefore, the first object model is mapped to the two-dimensional plane using the acquisition angle from the left side, and the control points of the left side profile of the first object model are mapped to the two-dimensional plane along the acquisition angle. On the mapped two-dimensional plane, a mapping coordinate system is constructed using a pre-specified second reference point as the origin. For example, the second reference point is the upper left corner of the mapped two-dimensional plane. The second coordinates of the control points are determined based on the mapping coordinate system.

[0131] The loss data is used to characterize the difference between the first coordinate and the second coordinate. The first coordinate refers to the position coordinate of the feature point in the texture feature representation, and the second coordinate refers to the position coordinate of the control point on the first object model. The loss data represents the difference between the relative positions of the feature points in the texture feature representation and the relative positions of the control points on the first object model. Since the first object model is a 3D model, it is first mapped to a 2D plane. The second coordinate refers to the position coordinate of the control point on the mapped 2D plane. In other words, the loss data represents the difference between the relative positions of the feature points in the texture feature representation and the relative positions of the control points on the mapped 2D plane.

[0132] In some embodiments, determining loss data includes at least one of the following methods:

[0133] 1. There is a correspondence between feature points and control points. For the i-th feature point, the control point corresponding to the i-th feature point is obtained according to the correspondence between feature points and control points. Based on the first coordinate of the i-th feature point and the second coordinate of the corresponding control point, the loss data corresponding to the i-th feature point is determined. Thus, the loss data between multiple feature points and multiple control points is obtained based on the loss data corresponding to each feature point.

[0134] As an illustration, the calculation method for this loss data is shown in Formula Six below:

[0135] Formula Six:

[0136] Where L is the loss function, Fi is the coordinates of the i-th texture feature point, Mi is the two-dimensional coordinates of the control point corresponding to the i-th texture feature point on the first object model, N is the number of texture feature points, λ is a pre-set regularization coefficient, and R(M) is the regularization term for the first object model M. There are many methods to optimize the loss value of the loss function, such as gradient descent, Newton's method, and genetic algorithms. By adjusting the coordinate positions of the control points on the first object model, the loss value calculated by the loss function can be reduced.

[0137] 2. The correspondence between irrelevant feature points and control points: For the i-th feature point, determine the comprehensive loss data between the i-th feature point and each control point. For example, sum the loss data between the i-th feature point and each control point to obtain the comprehensive loss data between the i-th feature point and each control point. Based on the comprehensive loss data corresponding to each feature point, determine the loss data between multiple feature points and multiple control points.

[0138] Step 2044: With the goal of reducing data loss, adjust the position of the control points of the first object model so that the model structure of the first object model matches the texture structure of the texture feature representation, and obtain the second object model.

[0139] In some embodiments, adjusting the position of the control points of the first object model includes at least one of the following methods:

[0140] 1. Adjust the positions of the control points on the three-dimensional structure of the first object model, and continue to perform two-dimensional mapping based on the adjusted control points. Calculate the loss data between the adjusted control points and feature points until the loss data meets the requirements.

[0141] When adjusting the position of control points in a three-dimensional structure, the position of the control points can be adjusted on the plane perpendicular to the mapping direction; or, the position of the control points can be adjusted on the preset adjustment curve corresponding to the control points. The preset adjustment curve is designed independently for each control point and is used to adjust the position of the control points in three-dimensional space.

[0142] 2. Adjust the positions of the control points on the 2D plane after 2D mapping, and recalculate the loss data between the feature points and control points based on the adjusted control points until the loss data meets the requirements. After the loss data meets the requirements, remap the adjusted control points on the 2D plane to the 3D structure.

[0143] In some embodiments, the mapping depth of the control points in the mapping direction is kept constant, and the coordinates of the adjusted control point positions on the two-dimensional plane are mapped in three dimensions in the mapping direction to obtain the second object model.

[0144] In some embodiments, each control point corresponds to a preset adjustment curve. After adjusting the position of the control point on the two-dimensional plane, the control point is mapped along the direction perpendicular to the two-dimensional plane. The intersection of the mapping line and the preset adjustment curve of the control point is taken as the position point of the control point after adjustment on the three-dimensional structure.

[0145] In some embodiments, the adjusted first object model, i.e., the second object model, is converted into a triangular mesh structure to facilitate subsequent texture mapping and rendering. Optionally, triangulation or other mesh generation algorithms are used to generate the triangular mesh structure. The advantage of triangular meshes is that they can conveniently represent the topological structure of the model and facilitate various geometric transformations and lighting calculations.

[0146] In summary, the 3D model generation method provided in this embodiment obtains the object image of a first object and extracts the texture feature representation of the object image to obtain the appearance of the first object. After obtaining the first object model as the base model, the outline of the first object model is adjusted under the constraint of the appearance representation so that the texture matches the adjusted first object model. The texture image is then mapped and overlaid onto the adjusted first object model to obtain the 3D model corresponding to the first object. This eliminates the need for artists to design and program the 3D model corresponding to the first object separately, improving the generation efficiency of 3D models and the efficiency of human-computer interaction, and saving the time and human resources consumed in the generation of 3D models.

[0147] The method provided in this embodiment obtains feature points in the texture feature representation and control points in the first object model, and adjusts the position of the control points based on the feature points, thereby making the model structure of the first object model match the texture feature representation and improving the generation efficiency of the three-dimensional model.

[0148] The method provided in this embodiment calculates the loss value between feature points and control points using a loss function, thereby adjusting the position of control points with the goal of reducing the loss value, which improves the efficiency and accuracy of adjusting the model structure of the first object model.

[0149] The method provided in this embodiment performs two-dimensional mapping on the first object model, so that the feature points in the texture feature representation are compared with the control points of the first object model in a two-dimensional angle, thereby calculating the loss value, which improves the accuracy and efficiency of loss value calculation.

[0150] In an optional embodiment, after adjusting the control points of the first object model, it is also necessary to attach the texture image corresponding to the texture feature representation to the adjusted first object model. Figure 5This is a flowchart of a method for generating a 3D model provided in another exemplary embodiment of this application. This method can be executed by a terminal, by a server, or by a combination of both. In this embodiment, the method is illustrated using an application to a server as an example. Figure 5 As shown, step 205 above can be implemented as steps 2051 to 2053.

[0151] Step 2051: Decode the texture feature representation to obtain the texture image.

[0152] Optionally, the texture feature representation V is reconstructed into a texture image T to facilitate the fitting of the texture image onto the second object model. This involves using a decoder network to decode the texture feature representation into a texture image. This decoder network has the opposite structure to a convolutional neural network, consisting of multiple deconvolutional layers, unpooling layers, and deactivation layers, and is capable of automatically learning to reconstruct the image from the feature vector. The deconvolution operation can be expressed as shown in Equation 7 below.

[0153] Formula 7:

[0154] Among them, T i,j F represents the pixel value in the i-th row and j-th column of the texture image. k,i,j W represents the value in the i-th row and j-th column of the k-th deconvolutional layer. k,i,j b represents the weight of the k-th deconvolution kernel in the i-th row and j-th column. i,j Let K represent the bias term in the i-th row and j-th column, and K be the number of deconvolution layers. The purpose of the deconvolution kernel is to expand each value in the feature map into a small region, thereby increasing the resolution and detail of the image.

[0155] In some embodiments, the decoder network is a pre-trained neural network. Optionally, during the training phase, the decoder network and the convolutional neural network first acquire sample images, extract image feature representations of the sample images through the convolutional neural network, and then decode the image feature representations through the decoder network to obtain decoded images. A loss value is then obtained based on the difference between the decoded images and the sample images. The convolutional neural network and the decoder network are trained based on this loss value until the features extracted by the convolutional neural network and the loss value corresponding to the images decoded by the decoder network converge.

[0156] Here, a texture image is an image represented by pixels obtained by decoding a texture feature representation. The texture feature representation is feature data expressed in a vector format, while the texture image is data expressed in pixel format by decoding the texture feature representation. Specifically, the texture image is the image obtained by converting the texture feature representation of the first object. Optionally, the texture image does not include the background portion of the object image; the texture image only contains the image content corresponding to the first object.

[0157] Step 2052: Map the texture image to the second object model to generate the mapping relationship between the texture image and the second object model.

[0158] The mapping relationship includes the alignment between the k-th pixel in the texture image and the p-th 3D vertex in the second object model, where k and p are positive integers. The 3D vertex is the vertex of the mesh in the second object model.

[0159] Optionally, when there is a correspondence between feature points and control points, the texture image is mapped to the second object model according to the correspondence, generating a mapping relationship between the texture image and the second object model. That is, the texture image is mapped to the second object model according to the correspondence between feature points and control points, obtaining the correspondence between the pixels in the texture image and the three-dimensional vertices on the second object model.

[0160] Alternatively, when there is no pre-defined correspondence between feature points and control points, a pre-trained matching model can be used to map and match the texture image and the second object model to obtain the mapping relationship between the texture image and the second object model.

[0161] Mapping the texture image T onto the second object model M forms a texture map U, which is a bidirectional mapping from the surface of the second object model to the texture image, and can be expressed as Equation 8 below.

[0162] Formula 8: U = {u i} i＝1,...,N

[0163] Among them, u i =(x i ,y i ,z i ,s i ,t i ) is a quintuple representing the i-th 3D vertex on the second object model and the (s-th)-th vertex on the texture image. i ,t i The correspondence of ) pixels, x i ,y i ,z iThe coordinates of the i-th 3D vertex on the second object model are s. i ,t i is the two-dimensional coordinate of a pixel on the texture image, and N is the number of three-dimensional vertices on the second object model. The purpose of texture mapping is to transfer the color and texture information of the texture image to the surface of the model to enhance the realism and visual effect of the model.

[0164] Step 2053: Based on the mapping relationship, the texture image is attached to the second object model to obtain the three-dimensional model corresponding to the first object.

[0165] In some embodiments, when applying the texture image to the second object model based on the mapping relationship, the texture mapping relationship also needs to be optimized. That is, energy loss data corresponding to the mapping relationship is obtained, wherein the energy loss data includes pixel difference data between adjacent vertices. The mapping relationship is adjusted with the goal of reducing the energy loss data to obtain the mapping adjustment result. Based on the mapping adjustment result, the texture image is applied to the second object model to obtain the 3D model corresponding to the first object.

[0166] The pixel difference data includes at least one of pixel value difference data and pixel distance difference data. In this embodiment, pixel difference data including pixel value difference data will be used as an example for explanation.

[0167] Optionally, the point identifiers corresponding to each three-dimensional vertex in the second object model are obtained, wherein the three-dimensional vertices adjacent to the point identifiers are adjacent in the second object model, that is, there are connecting lines between the three-dimensional vertices corresponding to adjacent point identifiers as edges of the mesh.

[0168] For the first and second vertices adjacent to each other, the first pixel value of the first pixel corresponding to the first vertex is obtained based on the mapping relationship, and the second pixel value of the second pixel corresponding to the second vertex is obtained based on the mapping relationship. Based on the difference between the first pixel value and the second pixel value, pixel difference sub-data is determined. Based on the pixel difference sub-data corresponding to multiple adjacent vertices, pixel difference data is determined, and energy loss data is obtained.

[0169] By calculating the differences between adjacent 3D vertices as pixel difference data, the problem of large differences in pixel values ​​between adjacent 3D vertices due to inaccurate mapping in the mapping relationship is avoided, which would lead to problems such as deformation and distortion of pixel values ​​of 3D vertices after mapping.

[0170] Optionally, the energy loss data also includes distortion data of the 3D vertices themselves. Vertex distortion data refers to the degree of change in a 3D vertex as it deforms from its original position to a new position during 3D modeling, graphic deformation, or animation. In model deformation, animation, and simulation of physical phenomena (such as elastic body dynamics), distortion data means that parts of the model have undergone extreme stretching, compression, or twisting, which can affect the model's visual quality and physical accuracy. Controlling vertex distortion data is one of the key factors in ensuring a natural and realistic model in animation and model creation.

[0171] Obtain the first vertex position of the j-th 3D vertex in the first object model and the second vertex position of the j-th 3D vertex in the second object model, where j is a positive integer. Based on the difference between the first and second vertex positions, determine the distortion data corresponding to the j-th 3D vertex, and determine the energy loss data based on the distortion data corresponding to multiple 3D vertices respectively.

[0172] By calculating the distortion data of 3D vertices, the problem of excessive positional changes of 3D vertices before and after deformation, which leads to large stretching or compression of the mesh corresponding to the 3D vertices and poor realism of the 3D model, is avoided, thus improving the realism and accuracy of the 3D model.

[0173] The texture mapping U is optimized to reduce distortions and artifacts that may occur during the mapping process, such as stretching, compression, overlap, and cracks. In this embodiment, this is achieved by optimizing an energy function, which measures the quality of the mapping. This energy function is shown in Formula 9 below.

[0174] Formula Nine:

[0175] Where E is the energy function, D(u i S(u) is the distorted data of the i-th 3D vertex. i ,u j ) represents the similarity between the i-th and j-th 3D vertices, u i It is the pixel value corresponding to the i-th 3D vertex, u j It is the pixel value corresponding to the j-th 3D vertex, N is the number of 3D vertices, and w i and w ij These are weighting coefficients. Optionally, energy loss data can be optimized using methods such as Laplace smoothing, least squares, or optimal transport.

[0176] The method provided in this application determines the energy loss of the mapping relationship by designing a calculation method for energy loss data, thereby optimizing the mapping relationship based on the energy loss and reducing distortions and artifacts that may occur during the mapping process, such as stretching, compression, overlap, and cracks.

[0177] In summary, the 3D model generation method provided in this embodiment obtains the object image of a first object and extracts the texture feature representation of the object image to obtain the appearance of the first object. After obtaining the first object model as the base model, the outline of the first object model is adjusted under the constraint of the appearance representation so that the texture matches the adjusted first object model. The texture image is then mapped and overlaid onto the adjusted first object model to obtain the 3D model corresponding to the first object. This eliminates the need for artists to design and program the 3D model corresponding to the first object separately, improving the generation efficiency of 3D models and the efficiency of human-computer interaction, and saving the time and human resources consumed in the generation of 3D models.

[0178] In some embodiments, when the texture image is attached to the second object model to obtain a three-dimensional model, optimization is also required using a diffusion model. Figure 6 This is a flowchart of a method for generating a 3D model provided in another exemplary embodiment of this application. This method can be executed by a terminal, by a server, or by a combination of both. In this embodiment, the method is illustrated using an application to a server as an example. Figure 6 As shown, step 205 above can be implemented as steps 2054 to 2057.

[0179] Step 2054: Convert the texture feature representation into a texture image and attach the texture image to the second object model to obtain the candidate 3D model.

[0180] Optionally, after converting the texture feature representation into a texture image, and determining the mapping relationship through steps 2051 to 2053 above, the texture image is then fitted to the second object model according to the mapping relationship to obtain a candidate 3D model. The process of fitting the texture image according to the mapping relationship has been described in detail and will not be repeated here.

[0181] Step 2055: Collect the planar feature map corresponding to the candidate 3D model.

[0182] Optionally, planar feature maps are acquired from the candidate 3D model, wherein the planar feature maps are feature maps acquired from the candidate 3D model from a specified viewpoint. In some embodiments, the planar feature maps are feature maps acquired from a single viewpoint of the candidate 3D model; or, the planar feature maps are multiple feature maps acquired from multiple viewpoints of the candidate 3D model.

[0183] In some embodiments, a tri-planar feature map of the candidate 3D model is acquired. The tri-planar feature map is used to map textures and details onto the model's surface to improve visual quality without requiring complex UV unwrapping. Every corner and surface of the model is covered by mapping textures from three orthogonal planes (typically the X, Y, and Z planes in space) onto the model. In this embodiment, acquiring the tri-planar feature map involves acquiring texture images of the candidate 3D model in the X, Y, and Z planes.

[0184] Step 2056: Diffusion feature map is obtained by diffusing the planar feature map through the pre-trained diffusion model.

[0185] Diffusion feature maps offer higher clarity than planar feature maps. The Diffusion Model Neural Network (DMNN) is a generative model based on a diffusion process, capable of progressively reconstructing a clear image from noise. The core idea of ​​the diffusion model is to transform the image generation problem into an image denoising problem; that is, starting with a randomly initialized noisy image, it gradually approaches the target image through a series of diffusion steps. The advantages of the diffusion model are its ability to effectively utilize the prior distribution of image data and the use of conditional probabilities in each diffusion step to guide image generation.

[0186] In this embodiment of the application, a diffusion process is defined, namely a random transformation from the target image to the noisy image and an inverse transformation from the noisy image to the target image. The diffusion process can be expressed as Equations 10 and 11 below.

[0187] Formula 10: x t =sqrt(1-β) t )·x0+sqrt(β t )·∈ t t = 1, ..., T

[0188] Formula 11: x0 = sqrt(1 - β) t )·x t +sqrt(β t )·∈0

[0189] Where, x t Let x be the image at the t-th diffusion step, x0 be the target image, and ∈ t And ∈0 are independent and identically distributed Gaussian noise, β t Let be the noise coefficient at the t-th diffusion step, and T be the total number of diffusion steps. The purpose of the diffusion process is to gradually increase the proportion of noise while maintaining the statistical properties of the image, making the image blurry and random, thus forming a distribution that is easy to model.

[0190] In this embodiment, a reverse diffusion process also needs to be defined for the corresponding diffusion process, which is a random transformation from the noisy image to the target image. The reverse diffusion function is the inverse of the diffusion process and can be expressed as the following formula 12.

[0191] Formula 12: x t-1 =(x t -sqrt(β t )·∈ t ) / sqrt(1-β t )

[0192] Where, x t-1 This is the image from the (t-1)th diffusion step. The purpose of the reverse diffusion process is to gradually reduce the proportion of noise while maintaining the statistical properties of the image, making the image clearer and more defined, thereby reconstructing the target image.

[0193] Combining the above diffusion and anti-diffusion processes, a diffusion model is defined, which is a conditional probability distribution from a noisy image to a target image. This diffusion model can be expressed as Equation Thirteen below.

[0194] Formula Thirteen:

[0195] Where p(x) t-1 |x t Let be the probability distribution of the image at the (t-1)th diffusion step given the image at the tth diffusion step, where N is a Gaussian distribution and μ is a constant. t (x t )and It is composed of a neural network f θ The parameterized mean and variance functions, θ, are the parameters of the neural network. The role of the diffusion model is to predict the probability distribution of the next image based on the current image at each diffusion step, thereby generating a clearer and closer image to the target.

[0196] Step 2057: Adjust the candidate 3D model based on the diffusion feature map to obtain the 3D model corresponding to the first object.

[0197] In some embodiments, the three-plane feature map is a texture image of the candidate 3D model in the X, Y, and Z planes. After obtaining the diffusion feature map, based on the acquisition process of the three-plane feature map, the diffusion feature map is reversed and mapped onto the candidate 3D model in the X, Y, and Z planes to obtain the 3D model corresponding to the first object.

[0198] In summary, the 3D model generation method provided in this embodiment obtains the object image of a first object and extracts the texture feature representation of the object image to obtain the appearance of the first object. After obtaining the first object model as the base model, the outline of the first object model is adjusted under the constraint of the appearance representation so that the texture matches the adjusted first object model. The texture image is then mapped and overlaid onto the adjusted first object model to obtain the 3D model corresponding to the first object. This eliminates the need for artists to design and program the 3D model corresponding to the first object separately, improving the generation efficiency of 3D models and the efficiency of human-computer interaction, and saving the time and human resources consumed in the generation of 3D models.

[0199] The method provided in this embodiment optimizes the three-plane feature map of the candidate 3D model by defining a diffusion model, thereby improving the clarity and accuracy of the texture on the candidate 3D model and thus improving the texture clarity and accuracy of the generated 3D model.

[0200] The method for generating 3D models provided in this application can be applied to a variety of different scenarios. Illustratively, this method for generating 3D models includes at least the following application scenarios, as illustrated in the following embodiments.

[0201] First, let's take the character generation scene in a game as an example for explanation.

[0202] In the game, players can upload or capture facial images to create a 3D model corresponding to the uploaded facial image in the game's virtual environment.

[0203] Figure 7 This is a flowchart illustrating a method for generating a 3D virtual character in a game, provided in an exemplary embodiment of this application. This method can be executed by a terminal, a server, or a combination of both. In this embodiment, the method is described using a server as an example. Figure 7 As shown, the method includes the following steps.

[0204] Step 701: Collect the player's facial image.

[0205] When facial images are used to generate 3D virtual characters, the facial features of the 3D virtual characters are generated based on the facial images.

[0206] In some embodiments, the facial image is collected after obtaining user authorization. That is, before collecting the facial image, the user is first notified of the information collection in the form of a pop-up window or other means, informing the user that the facial image is about to be collected and specifying the specific purpose of collecting the facial image. After obtaining the user's authorization, the facial image is collected.

[0207] In some embodiments, a facial image uploaded by a user is received; or, a facial image of a player is captured by a camera module; or, a facial video of a player is captured by a camera module, and a facial image is obtained from the facial video.

[0208] Step 702: Extract the texture feature representation of the player's face from the face image.

[0209] Texture feature representation is used to characterize the facial features of a player in a face image, such as feature vectors extracted from features like color, lighting, and facial wrinkles. In some embodiments, a texture feature representation of the player's face in a face image is extracted using a pre-trained feature extraction model.

[0210] Texture feature representations of human faces can be extracted from object images using convolutional neural networks or other image processing techniques.

[0211] Step 703: Obtain the first face model.

[0212] The first face model is a pre-provided basic 3D model.

[0213] The first face model is used to provide the basic template structure for the three-dimensional generation of face images. That is, based on the first face model, the first face model is adjusted by combining the texture feature representation of the face image to obtain the model of the three-dimensional virtual character corresponding to the face image.

[0214] In some embodiments, after acquiring a face image, the face type corresponding to the face image is first identified, such as male, female, child, etc. After identification, a basic 3D model corresponding to the face type is obtained as the first face model. That is, the model library provides basic 3D models corresponding to different face types, and the basic 3D model is selected as the first face model for adjustment based on the face type adaptability of the face image.

[0215] Step 704: Adjust the model structure of the first face model based on the texture feature representation so that the model structure of the first face model matches the texture structure of the texture feature representation to obtain the second face model.

[0216] The texture feature representation includes feature points, and the first face model includes control points. The position of the control points is adjusted based on the feature points so that the model structure of the first face model matches the texture structure of the texture feature representation.

[0217] Model structure refers to the outline structure represented by the outer contour of the first face model, that is, the relative positional relationship between different facial features of the player represented by the outer contour of the first face model. Texture structure refers to the relative positional relationship between different facial features of the player represented by texture features.

[0218] Optionally, the texture feature representation is identified to obtain the feature points corresponding to the specified key points in the texture feature representation. For example, facial key points mark important features and parts of the face, such as the eyes, nose, mouth, chin, and facial contours.

[0219] The control points used to control the model structure in the first face model are obtained. The positions of the control points in the first face model are adjusted based on the feature points so that the model structure of the first face model matches the texture structure represented by the texture features, thus obtaining the second face model.

[0220] In some embodiments, control points corresponding to specified key point positions in the first face model are obtained, and the control points corresponding to specified key point positions are adjusted based on feature points so that the model structure of the first face model matches the texture structure represented by texture features.

[0221] Step 705: Convert the texture feature representation into a texture image and attach the texture image to the second face model to obtain the player's corresponding 3D character model.

[0222] In some embodiments, the texture feature representation is decoded to obtain a texture image, which is then mapped onto a second face model to generate a mapping relationship between the texture image and the second face model. Based on this mapping relationship, the texture image is fitted onto the second face model to obtain the 3D character model corresponding to the player. In some embodiments, the face of the 3D character model is generated based on the face image, while the body parts of the 3D character model are pre-designed.

[0223] In some embodiments, since the texture image is a two-dimensional planar image and the second face model is a three-dimensional model, after determining the mapping relationship between the texture image and the second face model, the target pixels in the texture image that exist in the mapping relationship are attached to the three-dimensional vertex surface of the face model based on the mapping relationship, and the texture image is extended between multiple target pixels so that the texture image covers the surface of the face model and completely covers the surface of the face model.

[0224] In some embodiments, when converting texture feature representations into texture images and attaching them to a second face model, sharpness optimization is performed using a pre-trained diffusion model.

[0225] Indicative, such as Figure 8As shown, it illustrates the overall flowchart of the three-dimensional character model generation method provided in this embodiment. First, a face image 810 is acquired, and texture feature representation 820 is extracted; a parameterized face model 830 is acquired, and a three-dimensional avatar 840 is generated based on the texture feature representation 820 and the parameterized face model 830. The three-dimensional avatar 840 is then optimized to obtain a three-dimensional character model 850.

[0226] like Figure 9 As shown, it illustrates the effect of generating a three-dimensional character model provided in this embodiment. The three-dimensional mesh model 910 is generated using the modeling technology provided in this embodiment based on the image 900 of the avatar.

[0227] In some embodiments, control points are designed for the first face model. These control points can be used to control the dynamic performance of the first face model, such as expressions and movements. When converted into a second face model and applied to a 3D character model, these control points can be used to control the dynamic performance of the 3D character model, such as expressions and movements. The control points of the 3D character model are consistent with the control points on the first face model. That is, the control points of the 3D character model are obtained by adjusting the positions of the control points of the first face model as described above.

[0228] In summary, the 3D model generation method provided in this embodiment obtains the player's facial image and extracts its texture feature representation to obtain the appearance of the player's face. After obtaining a first facial model as the base model, the outline of the first facial model is adjusted under the constraint of the appearance representation to match the texture with the adjusted first facial model. The texture image is then mapped and overlaid onto the adjusted first facial model to obtain the 3D character model corresponding to the player. This eliminates the need for artists to design and program the 3D model corresponding to the player separately, improving the generation efficiency of 3D models and human-computer interaction efficiency, and saving the time and human resources consumed in generating 3D character models.

[0229] Secondly, let's take the scenario of generating vehicle models in navigation software as an example for explanation.

[0230] In navigation or driving racing games, players can upload or capture vehicle images to create a 3D model in a virtual environment that corresponds to the uploaded vehicle image.

[0231] Figure 10 This is a flowchart illustrating a method for generating a 3D vehicle model according to an exemplary embodiment of this application. This method can be executed by a terminal, a server, or both. In this embodiment, the method is described using a server as an example. Figure 10 As shown, the method includes the following steps.

[0232] Step 1001: Acquire vehicle images.

[0233] When vehicle images are used to generate 3D vehicle models, the shape of the 3D vehicle model is generated based on the vehicle images.

[0234] In some embodiments, the vehicle image is captured after obtaining user authorization. That is, before capturing the vehicle image, the user is first notified of the information collection in the form of a pop-up window or other means, informing the user that the vehicle image is about to be captured and specifying the specific purpose of capturing the vehicle image. After obtaining the user's authorization, the vehicle image is captured.

[0235] In some embodiments, vehicle images uploaded by users are received; or vehicle images are captured by a camera module; or road videos are captured by a camera module and vehicle images are obtained from the road videos.

[0236] Step 1002: Extract the texture feature representation of the vehicle from the vehicle image.

[0237] Texture feature representation is used to characterize the features of a vehicle in a vehicle image, such as feature vectors extracted from features like color, lighting, and shape. In some embodiments, a pre-trained feature extraction model is used to extract the texture feature representation of a vehicle in a vehicle image.

[0238] Texture feature representations of vehicle images can be extracted from road images using convolutional neural networks or other image processing techniques.

[0239] Step 1003: Obtain the first vehicle model.

[0240] The first vehicle model is a pre-provided basic 3D model.

[0241] The first vehicle model is used to provide the basic template structure for the three-dimensional generation of vehicle images. That is, based on the first vehicle model, the first vehicle model is adjusted by combining the texture feature representation of the vehicle image to obtain the three-dimensional vehicle model corresponding to the vehicle image.

[0242] In some embodiments, after acquiring a vehicle image, the vehicle type corresponding to the image is first identified, such as sports car, bus, SUV, etc. After identification, a basic 3D model corresponding to the vehicle type is obtained as the first vehicle model. That is, the model library provides basic 3D models corresponding to different vehicle types, and the basic 3D model is selected as the first vehicle model for adjustment based on the vehicle type adaptability of the vehicle image.

[0243] Step 1004: Adjust the model structure of the first vehicle model based on the texture feature representation so that the model structure of the first vehicle model matches the texture structure of the texture feature representation, thereby obtaining the second vehicle model.

[0244] The texture feature representation includes feature points, and the first vehicle model includes control points. The position of the control points is adjusted based on the feature points so that the model structure of the first vehicle model matches the texture structure of the texture feature representation.

[0245] Model structure refers to the outline structure exhibited by the outer contour of the first vehicle model, that is, the relative positional relationship between different vehicle parts exhibited by the outer contour of the first vehicle model. Texture structure refers to the relative positional relationship between different vehicle parts exhibited by texture features.

[0246] The control points used to control the model structure in the first vehicle model are obtained. The positions of the control points in the first vehicle model are adjusted based on the feature points so that the model structure of the first vehicle model matches the texture structure represented by the texture features, thus obtaining the second vehicle model.

[0247] In some embodiments, control points corresponding to specified key point positions in the first vehicle model are obtained, and the control points corresponding to specified key point positions are adjusted based on feature points so that the model structure of the first vehicle model matches the texture structure represented by texture features.

[0248] Step 1005: Convert the texture feature representation into a texture image and attach the texture image to the second vehicle model to obtain the player's corresponding 3D vehicle model.

[0249] In some embodiments, the texture feature representation is decoded to obtain a texture image, which is then mapped to a second vehicle model to generate a mapping relationship between the texture image and the second vehicle model. Based on the mapping relationship, the texture image is fitted onto the second vehicle model to obtain the player's corresponding 3D vehicle model.

[0250] In some embodiments, since the texture image is a two-dimensional planar image and the second vehicle model is a three-dimensional model, after determining the mapping relationship between the texture image and the second vehicle model, the target pixels in the texture image that exist in the mapping relationship are attached to the three-dimensional vertex surface of the vehicle model based on the mapping relationship, and the texture image is extended between multiple target pixels so that the texture image covers the surface of the vehicle model and completely covers the surface of the vehicle model.

[0251] It is worth noting that the above application scenarios are merely illustrative examples. The three-dimensional model generation method provided in this application embodiment can also be applied to various scenarios such as road model generation and prop model generation. This application embodiment does not limit this application.

[0252] Figure 11 This is a structural block diagram of a three-dimensional model generation apparatus provided in an exemplary embodiment of this application, such as... Figure 11 As shown, the device includes:

[0253] The acquisition module 1110 is used to acquire the object image of the first object, which is the object to be generated as a 3D model;

[0254] Extraction module 1120 is used to extract the texture feature representation of the first object from the object image;

[0255] The acquisition module 1110 is further configured to acquire a first object model, wherein the first object model is a pre-provided basic three-dimensional model;

[0256] The adjustment module 1130 is used to adjust the model structure of the first object model based on the texture feature representation, so that the model structure of the first object model matches the texture structure of the texture feature representation, thereby obtaining a second object model;

[0257] The generation module 1140 is used to convert the texture feature representation into a texture image and attach the texture image to the second object model to obtain a three-dimensional model corresponding to the first object.

[0258] In an optional embodiment, the acquisition module 1110 is further configured to acquire feature points in the texture feature representation, the feature points being used to characterize image region features; and acquire control points in the first object model used to control the model structure;

[0259] The adjustment module 1130 is further configured to adjust the position of the control points in the first object model based on the feature points, so that the model structure of the first object model matches the texture structure represented by the texture features, thereby obtaining the second object model.

[0260] In an optional embodiment, such as Figure 12 As shown, the adjustment module 1130 includes:

[0261] The determining unit 1131 is used to determine the loss data between the first coordinates of the feature point and the second coordinates of the control point through a preset loss function;

[0262] The adjustment unit 1132 is used to adjust the position of the control points of the first object model with the goal of reducing the loss data, so that the model structure of the first object model matches the texture structure represented by the texture features, thereby obtaining the second object model.

[0263] In an optional embodiment, the determining unit 1131 is further configured to determine the first coordinates corresponding to the feature points in the texture feature representation, with the first reference point represented by the texture feature as the origin;

[0264] The determining unit 1131 is further configured to map the first object model from three dimensions to a two-dimensional plane, and determine the second coordinates corresponding to the control points on the two-dimensional plane with the second reference point on the two-dimensional plane as the origin, wherein the first reference point and the second reference point are points corresponding to preset positions;

[0265] The determining unit 1131 is further configured to, for the i-th feature point, determine the comprehensive loss data between the first coordinate of the i-th feature point and the second coordinate of the multiple control points through the preset loss function, thereby obtaining the loss data between the multiple feature points and the multiple control points, where i is a positive integer.

[0266] In an optional embodiment, the adjustment unit 1132 is further configured to adjust the coordinates of the control points of the first object model on the two-dimensional plane with the goal of reducing the loss data, to obtain the adjusted coordinates corresponding to the control points; keeping the mapping depth of the control points in the mapping direction unchanged, and mapping the adjusted coordinates on the two-dimensional plane in the mapping direction in three dimensions to obtain the second object model.

[0267] In an optional embodiment, the control point is configured with a preset adjustment curve;

[0268] The adjustment unit 1132 is further configured to adjust the position of the control points of the first object model on the preset adjustment curve, and recalculate the loss data based on the adjusted control points until the loss data converges, so that the model structure of the first object model matches the texture structure represented by the texture features.

[0269] In an optional embodiment, the generation module 1140 is further configured to decode the texture feature representation to obtain the texture image; map the texture image to the second object model to generate a mapping relationship between the texture image and the second object model, wherein the mapping relationship includes the alignment relationship between the k-th pixel in the texture image and the p-th three-dimensional vertex on the second object model, where k and p are positive integers; and attach the texture image to the second object model based on the mapping relationship to obtain the three-dimensional model corresponding to the first object.

[0270] In an optional embodiment, the acquisition module 1110 is further configured to acquire energy loss data corresponding to the mapping relationship, the energy loss data including pixel difference data between adjacent vertices; adjust the mapping relationship with the goal of reducing the energy loss data, and obtain a mapping adjustment result;

[0271] The generation module 1140 is further configured to attach the texture image to the second object model based on the mapping adjustment result, so as to obtain the three-dimensional model corresponding to the first object.

[0272] In an optional embodiment, the acquisition module 1110 is further configured to acquire the point identifiers corresponding to each three-dimensional vertex in the second object model;

[0273] The acquisition module 1110 is further configured to: for a first vertex and a second vertex adjacent to a point identifier, acquire a first pixel value corresponding to a first pixel point of the first vertex based on the mapping relationship; acquire a second pixel value corresponding to a second pixel point of the second vertex based on the mapping relationship; determine pixel difference sub-data based on the difference between the first pixel value and the second pixel value; determine the pixel difference data based on the pixel difference sub-data corresponding to multiple adjacent vertices respectively, and obtain the energy loss data.

[0274] In an optional embodiment, the energy loss data further includes distortion data;

[0275] The acquisition module 1110 is further configured to acquire the first vertex position of the j-th three-dimensional vertex in the first object model, and the second vertex position of the j-th three-dimensional vertex in the second object model, where j is a positive integer;

[0276] The generation module 1140 is further configured to determine the distortion data corresponding to the j-th three-dimensional vertex based on the difference between the first vertex position and the second vertex position; and to determine the energy loss data based on the distortion data corresponding to multiple three-dimensional vertices respectively.

[0277] In an optional embodiment, the generation module 1140 is further configured to convert the texture feature representation into a texture image, and attach the texture image to the second object model to obtain a candidate 3D model; and to acquire a planar feature map corresponding to the candidate 3D model, wherein the planar feature map is a feature map obtained by acquiring the candidate 3D model from a specified perspective;

[0278] The generation module 1140 is further configured to process the planar feature map through a pre-trained diffusion model to obtain a diffusion feature map, wherein the clarity of the diffusion feature map is higher than that of the planar feature map; and adjust the candidate 3D model based on the diffusion feature map to obtain the 3D model corresponding to the first object.

[0279] In an optional embodiment, the acquisition module 1110 is further configured to acquire the first object type of the first object; and acquire the first object model that matches the first object type from a plurality of candidate object models based on the first object type, wherein the plurality of candidate object models correspond to different object types respectively.

[0280] In summary, the 3D model generation device provided in this embodiment obtains the object image of a first object and extracts the texture feature representation of the object image to obtain the appearance of the first object. After obtaining the first object model as the base model, the outline of the first object model is adjusted under the constraint of the appearance representation so that the texture matches the adjusted first object model. The texture image is then mapped and overlaid onto the adjusted first object model to obtain the 3D model corresponding to the first object. This eliminates the need for artists to design and program the 3D model corresponding to the first object separately, improving the generation efficiency of 3D models and the efficiency of human-computer interaction, and saving the time and human resources consumed in the generation of 3D models.

[0281] It should be noted that the 3D model generation apparatus provided in the above embodiments is only an example of the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the 3D model generation apparatus provided in the above embodiments belongs to the same concept as the 3D model generation method embodiments, and its specific implementation process is detailed in the method embodiments, which will not be repeated here.

[0282] This application also provides a computer device including a processor and a memory, wherein the memory stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the three-dimensional model generation method provided in the above-described method embodiments. It should be noted that the computer device may be as follows: Figure 13 The computer equipment provided.

[0283] Please refer to Figure 13This illustration shows a schematic diagram of a computer device provided in an exemplary embodiment of this application. Specifically, the computer device 1300 includes a central processing unit (CPU) 1301, a system memory 1304 including random access memory (RAM) 1302 and read-only memory (ROM) 1303, and a system bus 1305 connecting the system memory 1304 and the CPU 1301. The computer device 1300 also includes a basic input / output system (I / O system) 1306 that facilitates the transfer of information between various devices within the computer, and a mass storage device 1307 for storing the operating system 1313, application programs 1314, and other program modules 1315.

[0284] The basic input / output system 1306 includes a display 1308 for displaying information and an input device 1309 for user input, such as a mouse or keyboard. Both the display 1308 and the input device 1309 are connected to the central processing unit 1301 via an input / output controller 1310 connected to the system bus 1305. The basic input / output system 1306 may also include the input / output controller 1310 for receiving and processing input from multiple other devices such as a keyboard, mouse, or electronic stylus. Similarly, the input / output controller 1310 also provides output to a display screen, printer, or other types of output devices.

[0285] The mass storage device 1307 is connected to the central processing unit 1301 via a mass storage controller (not shown) connected to the system bus 1305. The mass storage device 1307 and its associated computer-readable media provide non-volatile storage for the computer device 1300. That is, the mass storage device 1307 may include computer-readable media (not shown) such as a hard disk or a CD-ROM drive.

[0286] Without loss of generality, the computer-readable medium may include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media include RAM, ROM, EPROM, EEPROM, flash memory or other solid-state storage technologies, CD-ROM, DVD or other optical storage, magnetic tape cassettes, magnetic tape, disk storage, or other magnetic storage devices. Of course, those skilled in the art will recognize that the computer storage media are not limited to the above-mentioned types. The system memory 1304 and mass storage device 1307 described above can be collectively referred to as memory.

[0287] The memory stores one or more programs, which are configured to be executed by one or more central processing units 1301. The one or more programs contain instructions for implementing the above-described method for generating a three-dimensional model or a translation method based on a machine translation model. The central processing unit 1301 executes the one or more programs to implement the method for generating a three-dimensional model or a translation method based on a machine translation model provided in the above-described method embodiments.

[0288] According to various embodiments of the present invention, the computer device 1300 can also be connected to a remote computer on a network, such as the Internet. That is, the computer device 1300 can be connected to a network 1312 via a network interface unit 1311 connected to the system bus 1305, or the network interface unit 1311 can be used to connect to other types of networks or remote computer systems (not shown).

[0289] The memory further includes one or more programs stored in the memory, and the one or more programs include steps executed by a computer device in the method for generating a three-dimensional model provided in the embodiments of the present invention.

[0290] This application also provides a computer device, which includes a memory and a processor. The memory stores at least one instruction, at least one program, code set, or instruction set. The processor loads the at least one instruction, at least one program, code set, or instruction set to implement the above-described method for generating the three-dimensional model.

[0291] This application also provides a computer-readable storage medium storing at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the above-described method for generating a three-dimensional model.

[0292] This application also provides a computer program product that, when run on a computer, causes the computer to execute the three-dimensional model generation method provided in the above-described method embodiments.

[0293] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. This program can be stored in a computer-readable storage medium, which may be a computer-readable storage medium included in the memory described in the above embodiments; or it may be a standalone computer-readable storage medium not assembled into the terminal. The computer-readable storage medium stores at least one instruction, at least one program segment, a code set, or an instruction set. The at least one instruction, the at least one program segment, the code set, or the instruction set is loaded and executed by the processor to implement the above-described method for generating a three-dimensional model or a translation method based on a machine translation model. Optionally, the computer-readable storage medium may include: read-only memory (ROM), random access memory (RAM), solid-state drives (SSDs), or optical discs, etc. The random access memory may include resistive random access memory (ReRAM) and dynamic random access memory (DRAM). The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0294] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware, or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk. The above descriptions are merely preferred embodiments of this application and are not intended to limit the application. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for generating a three-dimensional model, characterized in that, The method includes: Obtain the image of the first object, where the first object is the object to be generated into a 3D model; Extract the texture feature representation of the first object from the object image; Obtain the first object model, which is a pre-provided basic 3D model; The model structure of the first object model is adjusted based on the texture feature representation so that the model structure of the first object model matches the texture structure of the texture feature representation, thereby obtaining the second object model; The texture feature representation is converted into a texture image, and the texture image is attached to the second object model to obtain the three-dimensional model corresponding to the first object.

2. The method according to claim 1, characterized in that, The step of adjusting the model structure of the first object model based on the texture feature representation, so that the model structure of the first object model matches the texture structure of the texture feature representation, to obtain the second object model, includes: Obtain feature points from the texture feature representation, whereby the feature points are used to characterize image region features; Obtain the control points in the first object model used to control the model structure; Based on the feature points, the positions of the control points in the first object model are adjusted so that the model structure of the first object model matches the texture structure represented by the texture features, thereby obtaining the second object model.

3. The method according to claim 2, characterized in that, The step of adjusting the position of control points in the first object model based on the feature points, so that the model structure of the first object model matches the texture structure represented by the texture features, to obtain the second object model, includes: The loss data between the first coordinates of the feature point and the second coordinates of the control point is determined by a preset loss function; With the goal of reducing the lost data, the positions of the control points of the first object model are adjusted so that the model structure of the first object model matches the texture structure represented by the texture features, thereby obtaining the second object model.

4. The method according to claim 3, characterized in that, The loss data determined by using a preset loss function to compare the first coordinates of the feature point with the second coordinates of the control point includes: Using the first reference point represented by the texture feature as the origin, determine the first coordinates corresponding to the feature points in the texture feature representation; The first object model is mapped from three dimensions to a two-dimensional plane, and the second reference point on the two-dimensional plane is used as the origin to determine the second coordinates corresponding to the control points on the two-dimensional plane. The first reference point and the second reference point are points corresponding to preset positions. For the i-th feature point, the comprehensive loss data between the first coordinate of the i-th feature point and the second coordinates of multiple control points is determined by the preset loss function, so as to obtain the loss data between multiple feature points and multiple control points, where i is a positive integer.

5. The method according to claim 4, characterized in that, The step of adjusting the position of the control points of the first object model to reduce the loss data, so that the model structure of the first object model matches the texture structure represented by the texture features, to obtain the second object model, includes: With the goal of reducing the lost data, the coordinates of the control points of the first object model on the two-dimensional plane are adjusted to obtain the adjusted coordinates of the control points; Keeping the mapping depth of the control point in the mapping direction unchanged, the adjusted coordinates on the two-dimensional plane are mapped in three dimensions in the mapping direction to obtain the second object model.

6. The method according to claim 3, characterized in that, Each control point is configured with a preset adjustment curve; The step of adjusting the position of the control points of the first object model to reduce the lost data, so that the model structure of the first object model matches the texture structure represented by the texture features, includes: The positions of the control points of the first object model are adjusted on the preset adjustment curve, and the loss data is recalculated based on the adjusted control points until the loss data converges, so that the model structure of the first object model matches the texture structure represented by the texture features.

7. The method according to any one of claims 1 to 6, characterized in that, The step of converting the texture feature representation into a texture image and then attaching the texture image to the second object model to obtain the 3D model corresponding to the first object includes: Decode the texture feature representation to obtain the texture image; The texture image is mapped to the second object model to generate a mapping relationship between the texture image and the second object model. The mapping relationship includes the alignment relationship between the k-th pixel in the texture image and the p-th three-dimensional vertex on the second object model, where k and p are positive integers. Based on the mapping relationship, the texture image is attached to the second object model to obtain the three-dimensional model corresponding to the first object.

8. The method according to claim 7, characterized in that, The step of applying the texture image to the second object model based on the mapping relationship to obtain the 3D model corresponding to the first object includes: Obtain the energy loss data corresponding to the mapping relationship, wherein the energy loss data includes pixel difference data between adjacent vertices; The mapping relationship is adjusted with the goal of reducing the energy loss data, and the mapping adjustment result is obtained; Based on the mapping adjustment result, the texture image is applied to the second object model to obtain the three-dimensional model corresponding to the first object.

9. The method according to claim 8, characterized in that, The step of obtaining the energy loss data corresponding to the mapping relationship includes: Obtain the point identifiers corresponding to each 3D vertex in the second object model; For a first vertex and a second vertex that are adjacent to each other, the first pixel value of the first pixel corresponding to the first vertex is obtained based on the mapping relationship; and the second pixel value of the second pixel corresponding to the second vertex is obtained based on the mapping relationship. Pixel difference sub-data is determined based on the difference between the first pixel value and the second pixel value; Based on the pixel difference sub-data corresponding to multiple adjacent vertices, the pixel difference data is determined, and the energy loss data is obtained.

10. The method according to claim 7, characterized in that, The energy loss data also includes distorted data; The method further includes: Obtain the position of the first vertex of the j-th 3D vertex in the first object model, and the position of the second vertex of the j-th 3D vertex in the second object model, where j is a positive integer; Based on the difference between the first vertex position and the second vertex position, the distortion data corresponding to the j-th 3D vertex is determined; The energy loss data is determined based on the distortion data corresponding to multiple three-dimensional vertices.

11. The method according to any one of claims 1 to 6, characterized in that, The step of converting the texture feature representation into a texture image and then attaching the texture image to the second object model to obtain the 3D model corresponding to the first object includes: The texture feature representation is converted into a texture image, and the texture image is then attached to the second object model to obtain a candidate 3D model; Collect planar feature maps corresponding to the candidate 3D model. The planar feature maps are feature maps obtained by collecting the candidate 3D model from a specified viewpoint. The planar feature map is processed by a pre-trained diffusion model to obtain a diffused feature map, which has a higher clarity than the planar feature map. The candidate 3D model is adjusted based on the diffusion feature map to obtain the 3D model corresponding to the first object.

12. The method according to any one of claims 1 to 6, characterized in that, The process of obtaining the first object model includes: Obtain the first object type of the first object; Based on the first object type, the first object model matching the first object type is obtained from multiple candidate object models, wherein the multiple candidate object models correspond to different object types.

13. A device for generating a three-dimensional model, characterized in that, The device includes: The acquisition module is used to acquire the object image of the first object, which is the object to be generated into a 3D model; The extraction module is used to extract the texture feature representation of the first object from the object image; The acquisition module is further configured to acquire a first object model, wherein the first object model is a pre-provided basic three-dimensional model; An adjustment module is used to adjust the model structure of the first object model based on the texture feature representation, so that the model structure of the first object model matches the texture structure of the texture feature representation, thereby obtaining a second object model; The generation module is used to convert the texture feature representation into a texture image and attach the texture image to the second object model to obtain the three-dimensional model corresponding to the first object.

14. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing at least one instruction, which is loaded and executed by the processor to implement the method for generating a three-dimensional model as described in any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that, The readable storage medium stores at least one instruction, which is loaded and executed by a processor to implement the method for generating a three-dimensional model as described in any one of claims 1 to 12.

16. A computer program product, characterized in that, It includes a computer program or instructions that, when executed by a processor, implement the method for generating a three-dimensional model as described in any one of claims 1 to 12.

Images