Data processing method and related device

Abstract

The embodiment of the invention discloses a data processing method and a related device, and provides a position distribution image corresponding to a three-dimensional model for an image generation model in the process of generating a texture image. The distribution mode of each model vertex in the texture image can be represented through the position of the image point corresponding to each model vertex on the three-dimensional model in the position distribution image, and the position distribution of each model vertex in the three-dimensional space can be identified through the position information corresponding to each image point. Through the position distribution image, in the image generation process, on one hand, the comprehensiveness of texture effect analysis is improved, the probability that abnormal texture effects occur to all model parts is reduced, and on the other hand, the texture image can be directly processed based on the position distribution image, so that the accuracy of texture effect analysis is improved. The processed texture image does not need to be obtained through image synthesis, and the image generation process is more efficient.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a data processing method and related devices. Background Art

[0002] With the continuous development of computer technology, 3D rendering has become one of the commonly used technologies in many fields. For example, in the field of games, 3D rendering is needed to display various 3D models in the game interface. Among them, the texture effect added to the 3D model is an important factor affecting the 3D rendering effect. The texture effect is usually added based on the texture image corresponding to the 3D model. Therefore, how to accurately and efficiently generate texture images has become one of the key research directions in the field of 3D rendering.

[0003] In the related art, a texture image can generate a desired problem image by providing an initial three-dimensional model without texture effects and requirement information for expressing the need for texture effects to an image generation model. The texture image can be used to add the desired texture effects to the initial three-dimensional model.

[0004] However, the texture image generation method in the related art needs to analyze the texture effect brought by the texture image in combination with the preview image, and adjust the texture image by adjusting the preview image, and the image generation efficiency is low; at the same time, the texture generation effect of the model part not displayed in the preview image is poor, and it is difficult to guarantee the texture image generation effect. Summary of the invention

[0005] In order to solve the above technical problems, the present application provides a data processing method, which provides position distribution information for the image generation model, so that the generated texture image can add more accurate and reasonable texture to the entire model to be processed. At the same time, there is no need to perform the texture image synthesis step, thereby improving the texture image generation efficiency.

[0006] The embodiments of the present application disclose the following technical solutions:

[0007] In a first aspect, an embodiment of the present application discloses a data processing method, the method comprising:

[0008] Acquire sample requirement information and a sample texture image, wherein the sample requirement information is used to characterize the sample texture effect required by the sample model, and the sample texture image is used to add a texture satisfying the sample texture effect to the sample model;

[0009] Acquire a sample position distribution image corresponding to the sample model, wherein the model vertices on the sample model have corresponding image points on the sample position distribution image, the image points corresponding to the model vertices of the sample model in the sample texture image have the same positions as the corresponding image points in the sample position distribution image, and the image points in the sample position distribution image have corresponding position information, and the position information is used to identify the positions of the model vertices corresponding to the image points in the sample position distribution image in three-dimensional space;

[0010] Generate a pending texture image according to the sample requirement information and the sample position distribution image through an initial image generation model;

[0011] According to the difference between the sample texture image and the texture image to be determined, the model parameters corresponding to the initial image generation model are adjusted to obtain an image generation model, wherein the image generation model is used to generate a texture image according to the requirement information and the position distribution image, wherein the requirement information is used to characterize the texture effect required by the model to be processed, and the position distribution image is used to identify the positions of the model vertices corresponding to the image points in the texture image on the model to be processed in three-dimensional space, and the texture image is used to add a texture that satisfies the texture effect to the model to be processed.

[0012] In a second aspect, an embodiment of the present application discloses a data processing method, the method comprising:

[0013] Obtaining demand information corresponding to the model to be processed, wherein the demand information is used to characterize the texture effect required by the model to be processed;

[0014] Acquire a position distribution image corresponding to the model to be processed, wherein the model vertices on the model to be processed have corresponding image points in the position distribution image, and the image points in the position distribution image have corresponding position information, and the position information is used to identify the position of the model vertex corresponding to the image point in the position distribution image in the three-dimensional space;

[0015] A texture image is generated according to the demand information and the position distribution image through an image generation model, the image points corresponding to the model vertices on the model to be processed in the position distribution image are at the same position as the corresponding image points in the texture image, and the texture image is used to add a texture that satisfies the texture effect to the model to be processed.

[0016] In a third aspect, an embodiment of the present application discloses a data processing device, the device comprising a first acquisition unit, a second acquisition unit, a first generation unit and an adjustment unit:

[0017] The first acquisition unit is used to acquire sample requirement information and a sample texture image, wherein the sample requirement information is used to characterize the sample texture effect required by the sample model, and the sample texture image is used to add a texture satisfying the sample texture effect to the sample model;

[0018] The second acquisition unit is used to acquire a sample position distribution image corresponding to the sample model, the model vertices on the sample model have corresponding image points on the sample position distribution image, the image points corresponding to the model vertices of the sample model in the sample texture image have the same positions as the corresponding image points in the sample position distribution image, and the image points in the sample position distribution image have corresponding position information, and the position information is used to identify the positions of the model vertices corresponding to the image points in the sample position distribution image in three-dimensional space;

[0019] The first generating unit is used to generate a to-be-determined texture image according to the sample requirement information and the sample position distribution image by using an initial image generating model;

[0020] The adjustment unit is used to adjust the model parameters corresponding to the initial image generation model according to the difference between the sample texture image and the to-be-determined texture image to obtain the image generation model, the image generation model is used to generate the texture image according to the requirement information and the position distribution image, the requirement information is used to characterize the texture effect required by the to-be-processed model, the position distribution image is used to identify the positions of the model vertices corresponding to the image points in the texture image on the to-be-processed model in three-dimensional space, and the texture image is used to add a texture that satisfies the texture effect to the to-be-processed model.

[0021] In a possible implementation, the color corresponding to the image point in the sample position distribution image is determined by the position information corresponding to the image point, the initial image generation model includes an initial first module and a second module, and the first generation unit is specifically used to:

[0022] Extracting a first undetermined image feature corresponding to the sample position distribution image through the initial first module;

[0023] The second module generates the pending texture image according to the first pending image features and the sample requirement information.

[0024] In a possible implementation manner, the first generating unit is specifically configured to:

[0025] Randomly generate a first initial texture image feature;

[0026] Performing denoising processing on the first initial texture image feature according to the first undetermined image feature and the sample requirement information to generate a first undetermined texture image feature;

[0027] The first undetermined texture image feature is decoded to generate the undetermined texture image.

[0028] In a possible implementation, the second module is a pre-trained model, and the pre-trained model is used to generate an image that meets the image requirement according to information representing the image requirement, and the adjustment unit is specifically used to:

[0029] According to the difference between the sample texture image and the pending texture image, the model parameters corresponding to the initial first module are adjusted to obtain the first module. The first module and the second module are used to constitute the image generation model. The first module is used to extract image features corresponding to the position distribution image, and the second module is used to generate the texture image according to the image features and the required information.

[0030] In a possible implementation manner, the apparatus further includes a third acquiring unit:

[0031] The third acquisition unit is used to acquire a sample depth image corresponding to the sample model, the sample depth image is used to display the sample model, the image points in the sample depth image have corresponding depth information, and the depth information is used to identify the distance between the model vertex displayed by the image point and the camera plane corresponding to the sample depth image;

[0032] The first generating unit is specifically used for:

[0033] The undetermined texture image is generated through an initial image generation model according to the sample requirement information, the sample position distribution image and the sample depth image.

[0034] In a possible implementation, the color corresponding to the image point in the sample depth image is determined by the depth information corresponding to the image point, the initial image generation model includes an initial first module, a third module and a fourth module, and the first generation unit is specifically used to:

[0035] Extracting a first undetermined image feature corresponding to the sample position distribution image through the initial first module;

[0036] Extracting the second undetermined image feature corresponding to the sample depth image through the third module;

[0037] The fourth module generates the undetermined texture image according to the first undetermined image feature, the second undetermined image feature and the sample requirement information.

[0038] In a possible implementation manner, the first generating unit is specifically configured to:

[0039] Randomly generating a second initial texture image feature, where the second initial texture image feature is used to characterize the initial texture image;

[0040] generating an initial preview image feature according to the second undetermined image feature and the second initial texture image feature, wherein the initial preview image feature is used to characterize an initial preview image, wherein the initial preview image is used to display the sample model at a first angle, wherein the first angle is an angle at which the sample model is displayed through the sample depth image, wherein the sample model corresponds to a first texture effect in the initial preview image, and wherein the first texture effect is a texture effect added through the initial texture image;

[0041] According to the initial preview image feature, the first undetermined image feature, the second undetermined image feature and the sample requirement information, denoising the second initial texture image feature to generate a second undetermined texture image feature;

[0042] The second undetermined texture image feature is decoded to generate the undetermined texture image.

[0043] In a possible implementation manner, the first generating unit is specifically configured to:

[0044] Using the second initial texture image feature and the initial preview image feature as input features corresponding to the first round of denoising processing, performing N rounds of denoising processing on the second initial texture image feature to generate the second undetermined texture image feature;

[0045] In the i-th round of denoising among the N rounds of denoising, the fourth module is used to perform the following steps:

[0046] According to the preview image features in the input features corresponding to the i-th round of denoising, the sample requirement information, the first to-be-determined image features and the second to-be-determined image features, denoising the texture image features in the input features corresponding to the i-th round of denoising to generate the first texture image features output by the i-th round of denoising, where N is a positive integer and i is a positive integer not greater than N;

[0047] Based on i being equal to N, determining the first texture image feature as the second undetermined texture image feature;

[0048] Based on i being less than N, generating a first preview image feature output by the i-th round of denoising processing according to the second undetermined image feature and the first texture image feature, wherein the first preview image feature is used to characterize a first preview image, the first preview image is used to display the sample model from the first angle, the sample model corresponds to a second texture effect in the first preview image, and the second texture effect is a texture effect added to the texture image characterized by the first texture image feature;

[0049] The first texture image feature and the first preview image feature are determined as input features corresponding to the (i+1)th round of denoising processing.

[0050] In a possible implementation, the denoising process is performed on the texture image features in the input features corresponding to the i-th round of denoising process according to the preview image features in the input features corresponding to the i-th round of denoising process, the sample requirement information, the first to-be-determined image features, and the second to-be-determined image features to generate the first texture image features output by the i-th round of denoising process, including:

[0051] The input features corresponding to the i-th round of denoising are concatenated to generate the image features to be processed corresponding to the i-th round of denoising;

[0052] Determine a query matrix, a key matrix, and a value matrix in an attention mechanism according to the first undetermined image feature, the second undetermined image feature, the sample requirement information, and a model parameter corresponding to the fourth module;

[0053] The image features to be processed corresponding to the i-th round of denoising are used as input features of the attention mechanism, and the image features to be processed are denoised by the attention mechanism to generate the first texture image features output by the i-th round of denoising.

[0054] In a possible implementation manner, the sample depth image is any one of a plurality of sample depth images, and the plurality of sample depth images are used to display the sample model from a plurality of angles, and the first generating unit is specifically used to:

[0055] The to-be-determined texture image is generated through an initial image generation model according to the sample requirement information, the sample position distribution image and the multiple sample depth images.

[0056] In a fourth aspect, an embodiment of the present application discloses a data processing device, the device comprising a fourth acquisition unit, a fifth acquisition unit and a second generation unit:

[0057] The fourth acquisition unit is used to acquire requirement information corresponding to the model to be processed, where the requirement information is used to characterize the texture effect required by the model to be processed;

[0058] The fifth acquisition unit is used to acquire a position distribution image corresponding to the model to be processed, wherein the model vertices on the model to be processed have corresponding image points in the position distribution image, and the image points in the position distribution image have corresponding position information, and the position information is used to identify the position of the model vertex corresponding to the image point in the position distribution image in the three-dimensional space;

[0059] The second generating unit is used to generate a texture image through an image generating model according to the demand information and the position distribution image, the image points corresponding to the model vertices on the model to be processed in the position distribution image are at the same position as the corresponding image points in the texture image, and the texture image is used to add a texture that satisfies the texture effect to the model to be processed.

[0060] In a possible implementation manner, the apparatus further includes a sixth acquiring unit:

[0061] The sixth acquisition unit is used to acquire a depth image corresponding to the model to be processed, the depth image is used to display the model to be processed, the image points in the depth image have corresponding depth information, and the depth information is used to identify the distance between the model vertex displayed by the image point and the camera plane corresponding to the depth image;

[0062] The second generating unit is specifically used for:

[0063] A texture image is generated according to the demand information, the position distribution image and the depth image through an image generation model.

[0064] In a possible implementation manner, the demand information is provided by a provider, and the device further includes an adding unit:

[0065] The adding unit is used to add texture to the model to be processed through the texture image to obtain a processed model, and the processed model is used to display it to the provider.

[0066] In a fifth aspect, an embodiment of the present application discloses a computer device, wherein the computer device includes a processor and a memory:

[0067] The memory is used to store a computer program and transmit the computer program to the processor;

[0068] The processor is used to execute the data processing method described in any one of the first aspects, or execute the data processing method described in any one of the second aspects according to the instructions in the computer program;

[0069] In a sixth aspect, an embodiment of the present application discloses a computer-readable storage medium, wherein the computer-readable storage medium is used to store a computer program, wherein the computer program is used to execute the data processing method described in any one of the first aspects, or execute the data processing method described in any one of the second aspects;

[0070] In the seventh aspect, an embodiment of the present application discloses a computer program product including a computer program, which, when running on a computer device, enables the computer device to execute the data processing method described in any one of the first aspect, or execute the data processing method described in any one of the second aspect.

[0071] It can be seen from the above technical solution that when training an image generation model for generating a texture image, the present application provides a sample position distribution image corresponding to the sample model on the basis of providing sample requirement information for characterizing texture requirements. The image points corresponding to the model vertices on the sample model in the sample position distribution image are the same as the positions of the corresponding image points in the texture image corresponding to the sample model. Therefore, the sample position distribution image can characterize the position distribution of the model vertices on the sample model in the texture image. By adding position information to each image point in the sample position distribution image to identify the position of the model vertex corresponding to the image point in three-dimensional space, the sample position distribution image can characterize the overall structure of the three-dimensional model composed of the model vertices. Therefore, when the texture image is generated based on the sample position distribution image and the requirement information through the initial image generation model, on the one hand, the sample requirement information can be used to analyze how to generate a texture image that meets the required texture effect. On the other hand, the sample position distribution image can be used to accurately analyze the texture effect brought by the generated texture image on the entire sample model to generate a pending texture image. Furthermore, the initial image generation model is adjusted based on the difference between the pending texture image and the sample texture image. On the one hand, the model can learn how to accurately generate a texture image that meets the texture requirements based on the demand information. On the other hand, the model can learn how to make the texture image have high-quality texture effects as a whole after being added to the model, which effectively solves the problem of abnormal texture effects caused by insufficient analysis of the model's texture effects. At the same time, since the demand information represents the texture effect requirements and the position distribution image represents the position distribution of the model's vertices on the texture image, the model can directly generate the texture image based on the demand information and the position distribution image, and directly adjust the texture image during the generation process. The entire image generation process does not involve the synthesis process of the texture image based on the image of the display model. Therefore, the image generation process involves fewer links and has a higher generation efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0073] Figure 1 A schematic diagram of a data processing method in a related technology provided in an embodiment of the present application;

[0074] Figure 2 A schematic diagram of a data processing method in an actual application scenario provided by an embodiment of the present application;

[0075] Figure 3 A flowchart of a data processing method provided in an embodiment of the present application;

[0076] Figure 4 A flowchart of a data processing method provided in an embodiment of the present application;

[0077] Figure 5 A schematic diagram of a data processing method provided in an embodiment of the present application;

[0078] Figure 6 A schematic diagram of a data processing method provided in an embodiment of the present application;

[0079] Figure 7 A schematic diagram of a data processing method provided in an embodiment of the present application;

[0080] Figure 8 A schematic diagram of a data processing method provided in an embodiment of the present application;

[0081] Fig. 9 A schematic diagram of a data processing method provided in an embodiment of the present application;

[0082] Fig.10 A flowchart of a data processing method in an actual application scenario provided by an embodiment of the present application;

[0083] Fig.11 A schematic diagram of a data processing method in an actual application scenario provided by an embodiment of the present application;

[0084] Fig.12 A schematic diagram of a data processing method in an actual application scenario provided by an embodiment of the present application;

[0085] Fig.13 A structural block diagram of a data processing device provided in an embodiment of the present application;

[0086] Fig.14 A structural block diagram of a data processing device provided in an embodiment of the present application;

[0087] Fig.15 A structural diagram of a terminal provided in an embodiment of the present application;

[0088] Fig.16 A structural diagram of a server provided in an embodiment of the present application. DETAILED DESCRIPTION

[0089] The embodiments of the present application are described below in conjunction with the accompanying drawings.

[0090] Adding textures to 3D models is a common operation in the field of 3D modeling technology. By adding a variety of textures, the 3D model can have different material effects. Among them, the addition of textures usually requires the generation of a texture image corresponding to the 3D model. The texture image can be approximately regarded as an image obtained by unfolding the surface of the 3D model with the desired texture effect in two dimensions, such as a UV texture image. Each model vertex on the 3D model has a corresponding image point in the texture image. The texture on the image point is the texture used to add to the corresponding model vertex, such as Figure 1 As shown. For example, each model vertex has a uv parameter value, which is used to identify the image point corresponding to the model vertex in the uv texture image, u represents the distribution of the image point in the horizontal coordinate, and v represents the distribution of the image point in the vertical coordinate. The model vertex is the vertex of the model mesh that constitutes the model. When adding texture, the texture corresponding to the model vertex is usually rendered first, and then the texture of the unrendered model part is rendered through various methods such as interpolation.

[0091] In the related art, in order to efficiently generate texture images, an image generation model is trained. During the image generation process, the image generation model generates preview images corresponding to the three-dimensional model at multiple angles based on the initial texture image in the generation process. The texture effect of the three-dimensional model in the preview image is the texture effect brought by adding texture to the initial texture image. The image generation model processes these preview images to make the texture effect in the preview image more in line with the texture effect requirements represented by the demand information, and then synthesizes the processed texture image based on the processed multi-angle preview image, and re-executes the above operation until the number of iterations is reached, and the final texture image is output.

[0092] There are mainly two problems with the texture image generation method in the related art. On the one hand, the image generation model in the related art cannot analyze the texture effect brought by the texture image on the entire three-dimensional model. Even if the texture effect can be analyzed through preview images at multiple angles, the texture effect added to the model part that cannot be displayed at multiple angles through the generated texture image is difficult to guarantee; on the other hand, since the model cannot directly process the texture image, it can only synthesize the processed texture image based on the processed preview image. Therefore, the entire image generation process requires multiple texture image synthesis processes, which is time-consuming and difficult to provide efficient texture generation capabilities.

[0093] In order to solve the above technical problems, the present application provides a data processing method, in the process of generating a texture image, providing a position distribution image corresponding to a three-dimensional model to an image generation model. The distribution mode of each model vertex in the texture image is characterized by the position of the image points corresponding to each model vertex on the three-dimensional model in the position distribution image, and at the same time, the position distribution of each model vertex in the three-dimensional space can be identified by the position information corresponding to each image point, and then the overall model structure of the three-dimensional model can be identified. Therefore, through the position distribution image, the texture effect that the texture image can add to the overall three-dimensional model can be accurately analyzed during the image generation process, and then the texture image can be directly processed to obtain a texture image that better meets the texture effect requirements. On the one hand, the comprehensiveness of the texture effect analysis is improved, and the probability of abnormal texture effects in each model part is reduced. On the other hand, the texture image can be directly processed without the need to obtain the processed texture image through image synthesis, and the image generation process is more efficient.

[0094] It is understandable that the method can be applied to a computer device, which is a computer device capable of data processing, such as a terminal device or a server. The method can be executed independently by a terminal device or a server, and can also be applied to a network scenario in which a terminal device and a server communicate, and is executed by the cooperation of a terminal device and a server. Among them, the terminal device can be a mobile phone, a tablet computer, a laptop computer, a desktop computer and other devices. The terminal device can also include a variety of virtual reality devices, for example, it can include augmented reality (AR) devices, such as AR glasses, AR screens and other devices, and can include virtual reality technology (VR) devices, such as head-mounted VR glasses and other devices. The server can be understood as an application server, or a Web server. In actual deployment, the server can be an independent server, a cluster server, or a cloud server.

[0095] In order to facilitate understanding of the technical solution provided by this application, the data processing method provided by this application will be introduced below in combination with an actual application scenario.

[0096] See also Figure 2 , Figure 2 A schematic diagram of a data processing method in an actual application scenario provided in an embodiment of the present application. In this actual application scenario, the computer device may be a server 101 with an image processing function, and the server 101 is used to train the initial image generation model to obtain an image generation model that can be used to generate a texture image.

[0097] During the training process, the server 101 may first obtain sample requirement information, sample texture images, and sample position distribution images. The sample requirement information is used to characterize the sample texture effect required by the sample model. The sample model may be Figure 1 The sample texture image is used to add a texture that satisfies the sample texture effect to the sample model. For example, the sample requirement information may be "add a black and white texture to one side of the model". Through the sample texture image, a texture composed of a black and white grid can be added to the front of the sample model to satisfy the sample texture effect.

[0098] The sample position distribution image includes image points corresponding to each model vertex on the sample model, wherein the image points corresponding to the same model vertex in the sample position distribution image and the sample texture image have the same position, so that the position distribution of each model vertex of the sample model in the texture image can be characterized by the position distribution of the image points in the sample position distribution image. At the same time, each image point in the sample position distribution image has corresponding position information, and the position information is used to identify the position of the corresponding image point in three-dimensional space. For example, the position information can be presented in the form of color, so that on the one hand, the corresponding relationship between the image point and the model vertex can be identified, so that the sample image generation model can accurately generate a texture image that can be used to add texture to the sample model; on the other hand, it can enable the initial image generation model to understand the position distribution of each model vertex constituting the sample model in three-dimensional space based on the sample position distribution image, and then understand the model structure of the sample model, so that in the image generation process, the texture image and the sample position distribution image can be combined to analyze the texture effect brought by the texture image on the sample model.

[0099] Based on this, the server 101 can input the sample position distribution image and sample requirement information into the initial image generation model, so that the initial image generation model can generate a texture image that can be used to add texture to the sample model during the image generation process, without synthesizing the texture image, and without generating model images at multiple angles, thereby simplifying the image generation process; and can accurately analyze the texture effect brought by the texture image to the sample model as a whole based on the sample position distribution image and the texture image, without being restricted by the angle, making the texture effect analysis more comprehensive, so that the texture image can be accurately adjusted based on the sample requirement information to generate a texture image that satisfies the sample texture effect. The server 101 can adjust the initial image generation model according to the difference between the pending texture image output by the initial image generation model and the sample texture image, so as to obtain an image generation model that can accurately generate a texture image based on the requirement information and the position distribution image. It can be seen from the above that the image generation model can generate texture images more efficiently and accurately.

[0100] Next, the technical solution provided by this application will be introduced in detail in conjunction with the accompanying drawings.

[0101] See also Figure 3 , Figure 3 A flowchart of a data processing method provided in an embodiment of the present application. In this embodiment, the computer device may be any of the above-mentioned computer devices with data processing functions. The method includes:

[0102] S301: Obtain sample requirement information and sample texture image.

[0103] Among them, the sample requirement information is used to characterize the sample texture effect required by the sample model. For example, it can be text information describing the sample texture effect, such as "make the sample model have a medieval aristocratic style", "make the sample model have a wooden effect", etc. The sample model can be any three-dimensional model that can add texture. The sample texture image is used to add a texture that satisfies the sample texture effect to the sample model. That is, if the model can accurately generate a texture image based on the sample requirement information, the generated texture image should be close to the sample texture image. The higher the degree of closeness, the better the generation effect.

[0104] S302: Obtain a sample position distribution image corresponding to the sample model.

[0105] In the related art, the reason why the preview image needs to be processed first and then the processed texture image needs to be synthesized is that the model in the related art cannot directly analyze the distribution of all model vertices on the three-dimensional model in the texture image, and therefore cannot analyze the texture effect brought by the texture image on each model part of the three-dimensional model. It is necessary to generate a preview image to analyze the texture effect brought by the texture image. As a result, due to the limitations of the display angle of the preview image in displaying the three-dimensional model, when adding texture to the three-dimensional model through the finally generated texture image, it is easy to have abnormal texture effects on some models, and in the image generation process, it is necessary to synthesize the texture image based on the preview image, which leads to a long image generation time and difficulty in bringing efficient texture image generation capabilities.

[0106] In order to solve the above technical problems, the present application innovatively proposes a model input, namely, a position distribution image. Taking the sample position distribution image as an example, the model vertices on the sample model have corresponding image points on the sample position distribution image, and the image points corresponding to the model vertices of the sample model in the sample texture image are in the same position as the corresponding image points in the sample position distribution image, so that the position distribution of the image points corresponding to the model vertices in the sample texture image can characterize the position distribution of the image points corresponding to the model vertices in the texture image. The image points in the sample position distribution image have corresponding position information, and the position information is used to identify the positions of the model vertices corresponding to the image points in the sample position distribution image in three-dimensional space, so that the mapping relationship between the image points in the sample position distribution image and the model vertices can be established through the position information. The position information may include a variety of information, such as the coordinate information of the model vertices in the three-dimensional coordinate system, etc., which is not limited here.

[0107] On the one hand, this mapping relationship, combined with the distribution pattern of image points in the sample position distribution image, can characterize the distribution pattern of image points corresponding to model vertices in the texture image; on the other hand, by identifying the distribution pattern of model vertices of the sample model in three-dimensional space, the model structure of the sample model in three-dimensional space can be characterized.

[0108] S303: Generate a pending texture image through an initial image generation model according to sample requirement information and sample position distribution image.

[0109] The initial image generation model can be any model that can generate an image based on input information. By using the sample requirement information and the sample position distribution image as the intermediate model input, on the one hand, the initial image generation model can understand the distribution of the image points corresponding to the model vertices on the sample model in the texture image based on the sample position distribution image, and then accurately generate a texture image that can be used to add texture to the sample model. At the same time, since the texture effect added by the texture image on the sample model can be accurately analyzed through the sample position distribution image, the initial image generation model can directly analyze the texture effect based on the texture image, and then directly process the texture image during the image generation process to generate a texture image that is more in line with the sample texture effect represented by the sample requirement information, and finally obtain the processed pending texture image. Among them, there are many ways to generate a pending texture image based on the above information, which will be introduced in detail below and will not be repeated here.

[0110] S304: According to the difference between the sample texture image and the pending texture image, the model parameters corresponding to the initial image generation model are adjusted to obtain the image generation model.

[0111] The difference between the sample texture image and the pending texture image can characterize the accuracy of the texture image generation by the initial image generation model, and the difference is inversely correlated with the accuracy. Based on this, the computer device can adjust the model parameters corresponding to the initial image generation model according to the difference, which can enable the model to learn how to accurately generate a texture image that can be used to add texture to the sample model based on the sample position distribution image, and on the other hand, enable the model to learn how to accurately analyze the texture effect brought by the texture image based on the sample position distribution image, and how to accurately analyze the required sample texture effect based on the sample requirement information, so as to learn how to effectively process the texture image based on the sample requirement information and the sample position distribution image, and finally generate a texture image that meets the sample texture effect.

[0112] Based on this, the image generation model trained in this way can be used to accurately generate texture images according to the demand information and position distribution images. The high demand information is used to characterize the texture effect required by the model to be processed, and the position distribution image is used to identify the positions of the model vertices corresponding to the image points in the texture image on the model to be processed in three-dimensional space. The texture image is used to add a texture that meets the texture effect to the model to be processed.

[0113] It can be seen from the above content that the data processing method of the present application has the following technical effects compared with the related art:

[0114] 1. During the image generation process, the present application does not need to generate preview images for previewing the texture effects of the sample model from multiple angles, nor does it need to synthesize texture images based on preview images from multiple angles. The texture image can be directly generated and processed, making the image generation process more efficient, while ensuring the accuracy of image generation and improving the efficiency of image generation.

[0115] 2. The present application can accurately analyze the overall texture effect of the sample model based on the sample position distribution image, which can effectively reduce the probability of abnormal texture effects appearing in various parts of the model, and further improve the generation effect of the texture image.

[0116] See also Figure 4 , Figure 4 A flowchart of a data processing method provided in an embodiment of the present application, when using an image generation model to generate a texture image, a computer device can execute Figure 4 The steps shown in the figure include:

[0117] S401: Obtain demand information corresponding to the model to be processed.

[0118] The model to be processed may be any three-dimensional model to which texture can be added, and the requirement information is used to characterize the texture effect required by the model to be processed, and the texture effect may be any texture effect.

[0119] S402: Obtain a position distribution image corresponding to the model to be processed.

[0120] The position distribution image is used to characterize the distribution of image points corresponding to model vertices on the model to be processed in a texture image that can add texture to the model to be processed. The model vertices on the model to be processed have corresponding image points in the position distribution image, and the image points in the position distribution image have corresponding position information. The position information is used to identify the position of the model vertex corresponding to the image point in the position distribution image in three-dimensional space. Through the mapping relationship between the position information and the image point, a mapping relationship between the image point and the model vertex can be established.

[0121] S403: Generate a texture image according to the required information and the position distribution image through an image generation model.

[0122] like Figure 5As shown, in the image generation process, the demand information and the position distribution image are the input of the image generation model, and the output is the texture image. Through the position distribution image, the image generation model can analyze how to generate a texture image that can add texture to the model to be processed, that is, the image points corresponding to the model vertices on the model to be processed in the position distribution image are in the same position as the corresponding image points in the texture image. At the same time, in the image processing process, the texture effect brought by the generated texture image on the entire model to be processed can be analyzed based on the position distribution image, and the required texture effect can be analyzed based on the demand information, so that the texture image in the generation process can be continuously processed in combination with these two parts of information, and finally the output texture image is obtained, which is used to add a texture that satisfies the texture effect on the model to be processed.

[0123] Next, the technical details involved in this application will be introduced in detail.

[0124] First, in a possible implementation, in order to enable the model to fully understand the information contained in the position distribution image, the computer device can convert the position information into the color corresponding to the image point in the position distribution information. Taking the sample position distribution image as an example, the color corresponding to the image point in the sample position distribution image is determined by the position information corresponding to the image point. For example, the position information can be three-dimensional coordinate information. The computer device can determine the pixel value corresponding to the image point, the color value of the three color channels of red, green and blue, and other information that can affect the color of the image point based on the three-dimensional coordinate information, so that the image points corresponding to different position information can have different colors, and then the image generation model can perceive the position information contained in the sample position distribution image.

[0125] like Figure 6 As shown, the initial image generation model may include an initial first module and a second module. When executing step S303, the computer device may execute steps S3031-S3032 (not shown in the figure). Steps S3031-S3032 are a possible implementation of step S303, including:

[0126] S3031: extracting the first undetermined image features corresponding to the sample position distribution image through the initial first module.

[0127] The first undetermined image feature is used to characterize the image content corresponding to the sample position distribution image. Since the image content of an image is determined based on the position distribution of image points and the colors corresponding to the image points, the first undetermined image feature can characterize the positional characteristics of the model vertices represented by the sample position distribution image in the texture image, as well as characterize the model structure corresponding to the sample model, so that the information contained in the sample position distribution image can be fully utilized by the initial image generation model.

[0128] S3032: Generate a pending texture image according to the first pending image features and sample requirement information through the second module.

[0129] The second module can use the information represented by the features of the first pending image to generate a texture image that can add texture to the sample model. At the same time, it can analyze the overall texture effect of the texture image added to the sample model, and directly adjust the texture image based on the texture effect represented by the sample demand information to generate a pending texture image.

[0130] During the application process, the image generation model can also generate texture images through the above process, which will not be elaborated here.

[0131] In the image generation process of the present application, the image processing methods adopted may include multiple methods. For example, in a possible implementation method, the computer device may process the texture image by a denoising method.

[0132] In a possible implementation, when executing step S3032, the computer device may execute steps S30321-S30323 (not shown in the figure), where steps S30321-S30323 are a possible implementation of step S3032, including:

[0133] S30321: Randomly generate a first initial texture image feature.

[0134] In the denoising process, the initial image generation model will first randomly generate a first initial texture image feature, which can characterize a random image. The denoising process is to remove the noise information in this random image that is irrelevant to the final generated texture image to obtain the final required texture image.

[0135] S30322: Perform denoising processing on the first initial texture image feature according to the first undetermined image feature and the sample requirement information to generate a first undetermined texture image feature.

[0136] The initial image generation model can perform denoising on the first initial texture image features based on the distribution of image points corresponding to the model structure and model vertices in the texture image represented by the first pending image features, and the sample texture effect represented by the sample requirement information, so as to remove image features in the first initial texture image features that are irrelevant to the required sample texture effect, and generate the first pending texture image features. The denoising process can make the texture image represented by the first pending texture image features more consistent with the sample texture effect, and can be effectively used to add texture to the sample model.

[0137] S30323: Decode the first undetermined texture image feature to generate an undetermined texture image.

[0138] By decoding the first undetermined texture image feature, an image represented by the first undetermined texture image feature can be restored, and the image is the undetermined texture image.

[0139] Through the above method, the information involved in the entire texture image generation process can be converted into feature information that can be understood by the model, while retaining the information content contained in the original information, so that the initial image generation model can perform effective image processing and generate the required texture image through a more reasonable image processing process.

[0140] The above denoising process may include one or more rounds of denoising process, and the input of each round of denoising process is the image features generated by the previous round of denoising process (the input of the first round of denoising process is the first initial texture image features), and the information on which each round of denoising process is based is the first undetermined image features and sample requirement information, as shown in the following formula:

[0141] x t-1 =x t -∈ θ (x t ,t,c t ,c uv )

[0142] Among them, x t-1 is the input of the penultimate t-1 round of denoising, x t is the input of the t-th round of denoising, t is the number of rounds of denoising, c t is the control signal generated based on the sample demand information, that is, the feature information extracted from the sample demand information, c uv is a control signal generated based on the sample position distribution image, that is, the first undetermined image feature. These two control signals are used to indicate the t The processing method for denoising, ∈ θ is the model parameter, x t -∈ θ (x t ,t,c t ,c uv ) is the penultimate t-th round of denoising process.

[0143] When adjusting the parameters of the above-mentioned initial image generation model, in a possible implementation method, the computer device can adjust all model parameters involved in the initial image generation model, for example, the parameters of both the initial first module and the second module can be adjusted. By adjusting the parameters of the initial first module, the initial first module can learn how to accurately extract features of the position distribution image to effectively retain the relevant information contained in the position distribution image that is helpful for texture image generation; by adjusting the parameters of the second module, the second module can learn how to combine the image features and required information of the position distribution image to accurately generate the required texture image.

[0144] In another possible implementation, in order to further improve the generalization of the image generation model so that the image generation model can accurately generate texture images that have not appeared during the training process, the computer device can also construct the image generation model in combination with a pre-trained model.

[0145] In one possible implementation, the second module may be a pre-trained model, which refers to a model that is pre-trained with a large number of training samples and has a relatively stable model function. In the present application, the pre-trained model is used to generate an image that meets the image requirements based on information that characterizes the image requirements. The pre-trained model may be any model with the above functions, for example, it may be part or all of the modules in any image generation model.

[0146] When executing step S304, the computer device may execute step S3041 (not shown in the figure), which is a possible implementation of step S304, including:

[0147] S3041: According to the difference between the sample texture image and the to-be-determined texture image, the model parameters corresponding to the initial first module are adjusted to obtain the first module.

[0148] Since the second module already has a relatively mature image generation capability, when adjusting parameters, the computer device can only adjust the parameters of the initial first module, so that the initial first module can learn how to extract the first undetermined image feature from the sample position distribution image, so that the first undetermined image feature can retain the information contained in the sample position distribution image on the one hand, and on the other hand, the second module can effectively use the feature to generate a texture image. Finally, the first module and the second module can be used to form an image generation model. In the application process, the first module can be used to extract the image features corresponding to the position distribution image, and the second module can be used to generate a texture image according to the image features and the required information.

[0149] Since there is no need to adjust the parameters of the second module, on the one hand, the model knowledge for image generation learned through the pre-training process in the second module can be effectively retained, so that the image generation model can also have a better image generation effect for texture images that did not appear in this training process, thereby improving the generalization of the image generation model; on the other hand, it can reduce the amount of model parameters that need to be adjusted during the model training process, so that the model can converge more quickly, while ensuring the model training effect, further improving the model training efficiency.

[0150] In addition, in the present application, in order to further improve the generation effect of the texture image, the computer device can also provide a depth image corresponding to the three-dimensional model, so that the image generation model can more accurately analyze the texture effect brought by the texture image on the three-dimensional model.

[0151] In one possible implementation, see Figure 7 The computer device can also obtain a sample depth image corresponding to the sample model, which is used to display the sample model. The image points in the sample depth image have corresponding depth information. The depth information is used to identify the distance between the model vertex displayed by the image point and the camera plane corresponding to the sample depth image. The camera plane is a virtual plane used to generate the sample depth image. Therefore, through the depth information and the distribution of the image points in the sample depth image, the position distribution of the model vertices of the sample model displayed by the sample depth image in the three-dimensional space can be accurately known, and then the model structure of the sample model displayed by the sample depth image can be accurately analyzed.

[0152] Based on this, when executing step S303, the computer device may execute step S3033 (not shown in the figure), which is a possible implementation of step S303, including:

[0153] S3033: Generate a pending texture image through an initial image generation model according to sample requirement information, sample position distribution image and sample depth image.

[0154] Through the sample depth image, the initial image generation model can more accurately analyze the model structure of the sample model at a specific display angle, and thus can more accurately analyze the texture effect brought by the texture image on the sample model at a specific display angle. Furthermore, based on the analysis of the texture effect, the initial image generation model can more accurately determine whether the currently generated texture image can meet the required sample texture effect, so as to achieve more accurate processing of the texture image and generate a texture image that better meets the texture effect requirements. For example, in Figure 7In the present invention, through the sample depth image, the sample image generation model can more accurately analyze the model structure of the cube sample model in the three-dimensional space, so as to more accurately analyze the texture effect brought by the texture image when rendering the three model surfaces displayed by the sample depth image.

[0155] From the above content, it can be seen that, on the one hand, the initial image generation model can analyze the overall model structure of the sample model based on the position information carried by the sample position distribution image, so as to analyze the texture effect brought by the texture image to the sample model as a whole; on the other hand, it can make a more detailed analysis of the sample model structure at a specific angle based on the sample depth image, so as to make a more detailed analysis of the texture effect at this angle, so as to enhance the analysis accuracy of the texture effect. In the above way, the overall and local texture effect analysis can be combined, so that the final generated texture image can better meet the texture effect requirements and further improve the image generation quality.

[0156] Similarly, during the model application process, the computer device can also combine the depth image to generate a more accurate texture image. In one possible implementation, when generating a texture image for the model to be processed, the computer device can also obtain a depth image corresponding to the model to be processed, and the depth image is used to display the model to be processed. The image points in the depth image have corresponding depth information, and the depth information is used to identify the distance between the model vertex displayed by the image point and the camera plane corresponding to the depth image, and the camera plane is a virtual plane used to generate the depth image.

[0157] When executing step S403, the computer device may execute step S4031 (not shown in the figure), which is a possible implementation of step S403, including:

[0158] S4031: Generate a texture image according to the required information, the position distribution image and the depth image through an image generation model.

[0159] See also Figure 8 In this implementation, the input of the image generation model may include demand information, position distribution image and depth image. Through the depth image, the image generation model can perform a more accurate analysis of the structure of the model to be processed at the display angle corresponding to the depth image during the image generation process, thereby being able to perform a more detailed analysis of the texture effect brought by the texture image at the display angle. Combined with the analysis of the texture effect brought by the texture image to the entire model to be processed through the position distribution image, the image generation model can process the texture image more accurately to generate a texture image that better meets the texture effect represented by the demand information.

[0160] Next, we will introduce in detail how to combine depth images to generate texture images.

[0161] In a possible implementation, similar to the position distribution image, in order to enable the initial image generation model to understand the information contained in the depth image, the computer device can reflect this information through the image color corresponding to the depth image. Taking the sample depth image as an example, the color corresponding to the image point in the sample depth image is determined by the depth information corresponding to the image point. For example, Figure 7 In the image, the closer the image point is to the camera plane, the lighter the color corresponding to the image point is.

[0162] In this implementation, if Fig. 9 As shown, the initial image generation model may include an initial first module, a third module and a fourth module. When executing step S3033, the computer device may execute steps S30331-S30333 (not shown in the figure). Steps S30331-S30333 are a possible implementation of step S3033, including:

[0163] S30331: Extracting the first undetermined image features corresponding to the sample position distribution image through the initial first module.

[0164] The function of the initial first module is similar to that of the initial first module in the above embodiment, and will not be described in detail here.

[0165] S30332: Extract the second undetermined image features corresponding to the sample depth image through the third module.

[0166] The third module is used to extract features from the depth image of the input model. Taking the sample depth image as an example, the second image feature to be determined is used to characterize the image content of the sample depth image. Since the image content is determined based on the position distribution and color of the image points in the image, and the position distribution combined with the depth information represented by the color can characterize the position distribution of the model vertices displayed by the sample depth image in three-dimensional space, the second image feature to be determined can characterize the model structure displayed by the sample depth image.

[0167] S30333: Generate a pending texture image according to the first pending image feature, the second pending image feature and the sample requirement information through the fourth module.

[0168] Through the second pending image feature, the fourth module can accurately analyze the model structure displayed by the sample model in the sample depth image, and then can combine the second pending image feature to accurately analyze the texture effect brought about by this part of the model structure of the texture image, so as to realize the above-mentioned image generation process that combines the overall texture effect and the local texture effect to more accurately process the texture image, and finally output a more accurate pending texture image.

[0169] Among them, similar to the above-mentioned model architecture, in the process of parameter adjustment, the computer device can simultaneously adjust the initial first module, the third module and the fourth module to obtain the first module, the third module and the fourth module for constituting the image generation model. Among them, by adjusting the parameters of the initial first module, the initial first module can learn how to accurately extract features from the position distribution image to effectively retain the relevant information contained in the position distribution image that is helpful for texture image generation; by adjusting the parameters of the third module, the third module can learn how to accurately extract features from the depth image to effectively retain the relevant information contained in the depth image that is helpful for texture image generation. By adjusting the parameters of the fourth module, the fourth module can learn how to combine the image features extracted from the depth image, the image features extracted from the position distribution image and the required information to accurately generate the required texture image.

[0170] Of course, in this implementation, the computer device can also use the above-mentioned pre-trained model as the fourth module to further improve the generalization of the image generation model, which will not be described here. During the model application process, the first module, the third module, and the fourth module can perform the image processing steps performed by the above-mentioned initial first module, the third module, and the fourth module to generate the required texture image according to the input demand information, the position distribution image, and the depth image.

[0171] Similarly, in the fourth module, the image processing methods for generating the texture image in combination with the above-mentioned feature information may also include multiple methods, and a specific image processing method will be introduced below.

[0172] In a possible implementation, when executing step S30333, the computer device may execute steps S303331-S303334, where steps S303331-S303334 are a possible implementation of step S30333, including:

[0173] S303331: Randomly generate a second initial texture image feature.

[0174] The second initial texture image feature is used to characterize the initial texture image, which is a randomly generated image. Subsequent denoising processing can remove image content in the initial texture image that is irrelevant to the desired texture image to obtain the desired texture image.

[0175] S303332: Generate an initial preview image feature according to the second pending image feature and the second initial texture image feature.

[0176] Since the second undetermined image feature can characterize the model structure of the sample model at the corresponding display angle in the sample depth image, therefore, in combination with the second undetermined image feature and the second initial texture image feature, the texture effect produced when the initial texture image is added to the sample model displayed by the sample depth image can be analyzed, and then the initial preview image feature can be generated. The initial preview image feature is used to characterize the initial preview image, and the initial preview image is used to display the sample model at a first angle, and the first angle is the angle at which the sample model is displayed through the sample depth image. The sample model corresponds to the first texture effect in the initial preview image, and the first texture effect is the texture effect added by the initial texture image. Therefore, through the initial preview image feature, the fourth module can analyze the texture effect that the currently generated texture image can bring to the sample model at the first angle.

[0177] Among them, in order to have more accurate initial preview image features, the fourth module can also generate the initial preview image features in combination with the first pending image features that can identify the mapping relationship between image points in the texture image and model vertices on the sample model, which is not limited here.

[0178] S303333: According to the initial preview image feature, the first to-be-determined image feature, the second to-be-determined image feature and the sample requirement information, perform denoising on the second initial texture image feature to generate a second to-be-determined texture image feature.

[0179] On the first aspect, in combination with the first pending image feature and the second initial texture image feature, the fourth module can analyze the overall texture effect obtained by adding the initial texture image to the sample model; on the second aspect, in combination with the initial preview image feature and the second pending image feature, the fourth module can analyze the texture effect brought by the initial texture image on the sample model portion displayed by the sample depth image; on the third aspect, based on the sample requirement information, the fourth module can obtain the texture effect required for this image generation process. In combination with the above three aspects, on the one hand, the fourth module can obtain the processing method for denoising the overall image of the initial texture image represented by the initial texture image feature, and on the other hand, it can obtain the processing method for denoising the partial texture image displayed by the initial preview image in the initial texture image, so as to determine the processing method for denoising the second initial texture image feature, and finally obtain the second pending texture image feature that can meet the sample texture effect represented by the sample requirement information both overall and locally, and the second pending texture image feature is used to characterize the above-mentioned pending texture image.

[0180] S303334: Decode the second undetermined texture image features to generate an undetermined texture image.

[0181] Through the above method, the fourth module can make full use of the input information to analyze the texture effects brought by the texture image on the sample model as a whole and locally in the process of generating the pending texture image, so that the texture image features can be accurately denoised from both the overall and local dimensions, and finally a texture image that meets the texture effect requirements represented by the sample demand information is obtained, further improving the rationality and accuracy of the texture image generation process.

[0182] In order to further improve the accuracy of texture image generation, the computer device can also set multiple rounds of denoising processing in the image generation model to improve the accuracy of denoising processing on the texture image.

[0183] In a possible implementation, when executing step S303333, the computer device may execute step S3033331 (not shown in the figure), where step S3033331 is a possible implementation of step S303333, including:

[0184] S3033331: Using the second initial texture image features and the initial preview image features as input features corresponding to the first round of denoising processing, performing N rounds of denoising processing on the second initial texture image features to generate second to-be-determined texture image features.

[0185] Wherein, N is a positive integer, that is, the computer device can perform one or more rounds of denoising on the second initial texture image feature. Generally, the more rounds of denoising processing, the higher the processing accuracy. The value of N can be set based on actual image generation requirements and is not limited here.

[0186] Taking the i-th round of denoising as an example, in the i-th round of denoising among N rounds of denoising, the fourth module may be used to perform the following steps:

[0187] First, the fourth module can perform denoising on the texture image features in the input features corresponding to the i-th round of denoising according to the preview image features, sample requirement information, first to-be-determined image features and second to-be-determined image features in the input features corresponding to the i-th round of denoising, and generate the first texture image features output by the i-th round of denoising, where i is a positive integer not greater than N. The denoising process is similar to the denoising process described in step S303333 and will not be described in detail here. The texture image represented by the first texture image feature generated in this way can better fit the sample texture effect represented by the sample requirement information than the texture image represented by the texture image feature in the input features corresponding to the i-th round of denoising.

[0188] Then, the fourth module can determine whether the N rounds of denoising are completed, that is, whether i is equal to N. Since i is equal to N, it means that the N rounds of denoising have been completed, and the fourth module can determine the first texture image feature as the second undetermined texture image feature, that is, directly output the first texture image feature;

[0189] Since i is less than N, it means that the Nth round of denoising has not yet ended. At this time, the fourth module can generate the first preview image feature output by the i-th round of denoising based on the second pending image feature and the first texture image feature. The first preview image feature is used to characterize the first preview image. The first preview image is used to display the sample model from a first angle. The sample model corresponds to the second texture effect in the first preview image. The second texture effect is the texture effect added by the texture image represented by the first texture image feature. The generation method of the first preview image feature is similar to the generation method of the initial preview image feature generated by step S303332, and will not be repeated here. Through the first preview image feature, the model can accurately analyze the texture effect obtained by adding texture to the sample model displayed by the sample depth image based on the texture image represented by the first texture image feature, and can then be used to locally analyze the texture effect that the texture image feature obtained after the i-th round of denoising can bring.

[0190] Then, the fourth module can determine the first texture image feature and the first preview image feature as the input features corresponding to the i+1th round of denoising, and perform the i+1th round of denoising so that the texture image represented by the texture image feature can be more consistent with the sample texture effect represented by the sample demand information. It can be seen that through the above-mentioned N rounds of denoising, the texture image represented by the texture image feature generated by the initial image generation model can be continuously more consistent with the sample texture effect, whether for the sample model as a whole or for the texture effect brought about by the local sample model, and then the texture image generation process can be refined to improve the texture image generation accuracy.

[0191] Among them, the present application may also include multiple processing methods for denoising. Next, a specific denoising processing method will be introduced.

[0192] In one possible implementation, when denoising the texture image features in the input features corresponding to the i-th round of denoising based on the preview image features, sample requirement information, first to-be-determined image features, and second to-be-determined image features in the input features corresponding to the i-th round of denoising, and generating the first texture image features output by the i-th round of denoising, the computer device may set an attention mechanism in the fourth module to denoise the texture image features through the attention mechanism.

[0193] Attention Mechanism is a mechanism that allows the model to learn to perceive the important and unimportant parts of feature data, and to process specific stimuli by selectively focusing on and concentrating attention, thereby filtering out irrelevant or useless information and focusing on important stimuli, tasks or goals. In this application, the important part is the image features that match the desired texture effect, and the unimportant part is the graphic features that differ greatly from the desired texture effect. The goal is to remove unimportant image features and retain those image features that match the desired texture effect.

[0194] First, the fourth module can splice the input features corresponding to the i-th round of denoising to generate the image features to be processed corresponding to the i-th round of denoising. The purpose of this step is to splice multiple image features into the same input feature. Then, the fourth module can determine the query matrix, key matrix and value matrix in the attention mechanism based on the first pending image features, the second pending image features, the sample requirement information and the model parameters corresponding to the fourth module. These three matrices determine the processing method for the input features.

[0195] The fourth module can use the image features to be processed corresponding to the i-th round of denoising as the input features of the attention mechanism, perform denoising on the image features to be processed through the attention mechanism, and generate the first texture image features output by the i-th round of denoising. During the denoising process, the fourth module can process the image features to be processed through the query matrix, the key matrix and the value matrix respectively to obtain the query vector, the key vector and the value vector. The query vector is used to perform similarity calculation with the key vector to determine which image features in the image features to be processed need to be paid attention to. The image content represented by this part of the image features is the image content that is more in line with the sample texture effect. The weight can be generated by the similarity, and the weight is used to perform weighted processing on the value vector. The result after this part of the weighted processing is the image feature obtained by denoising the image features to be processed. The feature part that is more in line with the sample texture effect in the image features to be processed can be retained through weighted processing, and the part that is more different from the sample texture effect can be removed. The texture image feature contained in the image feature is the above-mentioned first texture image feature.

[0196] Since the fourth module merges the two parts of the image features into the same image feature for input during feature input, the two parts of the image features are visible to each other during denoising through the attention mechanism, so that the texture effect on the local model represented by the preview image features can be effectively combined, and the image feature part with a strong correlation with the sample texture effect in the texture image features can be analyzed, and then the texture image feature part can be denoised directly based on the preview image feature part in the image features to be processed, without the need to synthesize the texture image features based on the preview image features, thereby simplifying the texture image generation process to improve the image generation efficiency while ensuring the texture image denoising effect.

[0197] In addition, in order to further improve the denoising accuracy, in one possible implementation, the computer device can provide the initial image generation model with depth images showing the sample model from multiple angles, so that the model can combine the texture effects of multiple model parts on the sample model to perform denoising during the denoising process.

[0198] In this implementation, the sample depth image may be any one of a plurality of sample depth images, and the plurality of sample depth images are used to display the sample model from a plurality of angles. When executing step S3033, the computer device may execute step S30334 (not shown in the figure), which is a possible implementation of step S3033, including:

[0199] S30334: Generate a pending texture image through an initial image generation model according to sample requirement information, a sample position distribution image and a plurality of sample depth images.

[0200] By inputting multiple sample depth images, the initial image generation model can analyze the texture effects added by texture images on sample models at multiple angles during the denoising process, thereby providing more references for texture effects at more angles for the model's denoising process. This enables the model to combine the texture effects at multiple angles to perform more accurate and comprehensive denoising on the texture image, further improving the accuracy of the denoising process.

[0201] The image processing process involved in the above training process can not only be applied to the training process, but also to the model application process, with the same effect. The model application process will be introduced in detail below.

[0202] As mentioned above, texture images are used to add textures to models to achieve various texture effects. Based on this, in one possible implementation, a computer device can automatically add corresponding model textures to the model to be processed based on the generated texture images during the model application process.

[0203] In this implementation, the demand information may be information provided by the provider. After the computer device generates the texture image through the image generation model, it can add texture to the model to be processed by itself through the texture image to obtain a processed model, and the processed model is used to display to the provider. Thus, the provider can intuitively know the texture effect brought by the generated texture image through the processed model, and then analyze whether the texture image can meet the texture effect requirements, so as to determine whether to regenerate the texture image or directly apply the texture image, etc., which provides effective effect analysis support for subsequent operations based on the texture image.

[0204] In order to facilitate understanding of the technical solution provided by this application, the data processing method provided by this application will be introduced below in combination with an actual application scenario.

[0205] See also Fig.10 , Fig.10 A flowchart of a data processing method in an actual application scenario provided in an embodiment of the present application. In the actual application scenario, the computer device can be any computer device with a data processing function. The computer device includes an image generation model trained in the above manner. The model architecture of the image generation model can be as follows: Fig.11 As shown, the method includes:

[0206] S1001: Obtain the model to be processed and requirement information provided by the provider.

[0207] The model to be processed can be any three-dimensional model to which texture can be added, and the requirement information is used to characterize the texture effect that needs to be added to the three-dimensional model. Fig.12As shown, the computer device can first display a model selection interface to the provider, through which the three-dimensional model for which a texture image needs to be generated can be selected, including multiple candidate models, such as a cube model, a sphere model, a cylinder model, etc. Fig.12 In the example, the provider can select a cube model as the model to be processed to generate a texture image.

[0208] Then, the computer device may display an information input interface to the provider, and the information input interface is used to input the demand information corresponding to the texture image generation this time, and the demand information may include multiple types, such as text information and image information, etc. The text information may directly describe the texture effect brought by the texture image to be generated, and the image information may be an image referenced when generating the texture image, etc.

[0209] S1002: Generate a position distribution image corresponding to the model to be processed and corresponding depth images at multiple angles.

[0210] The computer device can generate a position distribution image corresponding to the model to be processed and corresponding depth images at multiple angles by analyzing the position distribution of the model vertices of the model to be processed in three-dimensional space. The position distribution image is used to characterize the position distribution of image points corresponding to the model vertices on the texture image, and can characterize the mapping relationship between the image points on the texture image and the model vertices. The depth image can display the model to be processed from various angles, and combined with the position distribution of the image points on the depth image and the depth information corresponding to the image points, the three-dimensional structure of the model to be processed can be displayed from various angles.

[0211] The color corresponding to the image point in the position distribution image is determined by the position information corresponding to the image point, and the color corresponding to the image point in the depth image is determined by the depth information corresponding to the image point, so that the position information and depth information corresponding to the image point can be represented by the image content.

[0212] S1003: Input the position distribution image, demand information, and corresponding depth images at multiple angles into an image generation model, and extract first image features corresponding to the position distribution image through a first module in the image generation model.

[0213] The first image feature is used to characterize the image content corresponding to the position distribution image, so as to characterize the position distribution corresponding to the image point in the position distribution image on the one hand, and the color corresponding to each image point on the other hand, and further characterize the information contained in the position distribution image. The first module is a module obtained by adjusting the model parameters corresponding to the initial first module based on the above training method.

[0214] S1004: Extract second image features corresponding to the multiple depth images respectively through the third module in the image generation model.

[0215] The second image feature is used to characterize the image content corresponding to the depth image, so that on the one hand it can characterize the corresponding position distribution of image points in the depth image, and on the other hand it can characterize the color corresponding to each image point, and thus can characterize the information contained in the depth image. The third module can be a module obtained by adjusting the model parameters corresponding to the third module based on the above-mentioned training method.

[0216] S1005: Input the first image feature, the second image features corresponding to the multiple angles, and the required information into the fourth module, and randomly generate an initial texture image feature through the fourth module.

[0217] The initial texture image feature is used to characterize the randomly generated initial texture image.

[0218] S1006: Generate preview image features corresponding to multiple angles according to the initial texture image features and the second image features corresponding to multiple angles.

[0219] The preview image feature is used to characterize the image content corresponding to the preview image. The preview image is used to show the texture effect at a specific angle after the texture image is added to the model to be processed.

[0220] S1007: The initial texture image features and the preview image features corresponding to the multiple angles are concatenated to obtain the image features input for the first round of denoising processing, and N rounds of denoising processing are performed.

[0221] like Fig.11 As shown, in the i+1th round of denoising in N rounds of denoising, the input image features are the image features output by the i-th round of denoising, and the image features output by the i-th round of denoising include texture image features, preview image features corresponding to angle 1, and preview image features corresponding to angle 2. The feature sizes of these three image features can be, for example, (4, h / 8, w / 8), and image features of size (3*4, h / 8, w / 8) are obtained by splicing.

[0222] Each round of denoising needs to be processed by the encoder for feature extraction and the decoder for decoding. The process between the encoder and the decoder is the denoising process. This application can use the attention mechanism for denoising. The denoising process has been introduced in the above content and will not be repeated here. Through the i+1th round of denoising, the image features of the i+1th round of denoising output can be obtained. The process can be shown in the following formula:

[0223]

[0224] in is the preview image feature corresponding to angle 1 output by the i-th round of denoising, is the preview image feature corresponding to angle 2 output by the i-th round of denoising, x i is the texture image feature output by the i-th round of denoising, i is the number of rounds of denoising, c t is the control signal generated based on the demand information, that is, the feature information extracted from the demand information, c uv is a control signal generated based on the first image feature, c d1 is the control signal generated based on the second image feature corresponding to angle 1, c d2 are control signals generated based on the second image features corresponding to angle 2, and these control signals are used to control the processing method of noise reduction processing. This is the i+1th round of noise reduction process, where is the preview image feature corresponding to angle 1 output by the i+1th round of denoising, is the preview image feature corresponding to angle 2 output by the i+1th round of denoising, x i+1 It is the texture image feature output by the i+1th round of denoising processing.

[0225] In addition, in order to improve the generalization of the image generation model, the fourth module can use a pre-trained image generation model that generates images based on demand information.

[0226] S1008: Decoding the texture image features output by the Nth round of denoising processing through an image decoder to generate a texture image.

[0227] When decoding the image features output by the denoising process, the image generation model may only decode the texture image features to obtain the generated texture image, without decoding the preview image features.

[0228] S1009: Add texture to the model to be processed based on the texture image.

[0229] The computer device can directly add the generated texture image to the model to be processed to further meet the texture image generation needs of the provider.

[0230] S1010: Display the model to be processed and the texture image after adding texture.

[0231] like Fig.12 As shown, in the result display interface, the computer device can simultaneously display the model to be processed with the texture added and the generated texture image to meet the different texture requirements of the provider.

[0232] It can be seen from the above content that compared with the related art, this application has the following technical effects:

[0233] 1. During the image generation process, the present application does not need to generate preview images for previewing the texture effects of the model to be processed from multiple angles, nor does it need to synthesize texture images based on preview images from multiple angles. The texture image can be directly generated and processed, and the image generation process is more efficient, which improves the image generation efficiency while ensuring the accuracy of image generation.

[0234] 2. The present application can accurately analyze the texture effect of the entire model to be processed based on the position distribution image, which can effectively reduce the probability of abnormal texture effects appearing in various parts of the model, and further improve the generation effect of the texture image for the entire model to be processed.

[0235] 3. The present application can utilize a pre-trained model to construct a module for denoising in an image generation model, so as to improve the generalization of the image generation capability of the image generation model without being limited by training samples. At the same time, it reduces the model parameter adjustments required in the model training process and improves the parameter adjustment efficiency.

[0236] 4. The present application can combine depth images at one or more angles to enable the image generation model to more accurately analyze the texture effects added to the local part of the model by the generated texture effects, thereby combining the texture effects of the overall model and the texture effects of the local model to perform more accurate denoising on the texture image, and ultimately generate a texture image that better meets the texture effect requirements.

[0237] Based on the data processing method on the training side provided in the above embodiment, the present application also provides a data processing device, see Fig.13 , Fig.13 This is a structural block diagram of a data processing device provided in an embodiment of the present application. The device 1300 includes a first acquisition unit 1301, a second acquisition unit 1302, a first generation unit 1303 and an adjustment unit 1304:

[0238] The first acquisition unit 1301 is used to acquire sample requirement information and a sample texture image, wherein the sample requirement information is used to characterize the sample texture effect required by the sample model, and the sample texture image is used to add a texture satisfying the sample texture effect to the sample model;

[0239] The second acquisition unit 1302 is used to acquire a sample position distribution image corresponding to the sample model, the model vertices on the sample model have corresponding image points on the sample position distribution image, the image points corresponding to the model vertices of the sample model in the sample texture image have the same positions as the corresponding image points in the sample position distribution image, and the image points in the sample position distribution image have corresponding position information, and the position information is used to identify the positions of the model vertices corresponding to the image points in the sample position distribution image in three-dimensional space;

[0240] The first generating unit 1303 is used to generate a to-be-determined texture image according to the sample requirement information and the sample position distribution image by using an initial image generating model;

[0241] The adjustment unit 1304 is used to adjust the model parameters corresponding to the initial image generation model according to the difference between the sample texture image and the texture image to be determined, so as to obtain an image generation model. The image generation model is used to generate a texture image according to the requirement information and the position distribution image. The requirement information is used to characterize the texture effect required by the model to be processed. The position distribution image is used to identify the positions of the model vertices corresponding to the image points in the texture image on the model to be processed in three-dimensional space. The texture image is used to add a texture that satisfies the texture effect to the model to be processed.

[0242] In a possible implementation, the color corresponding to the image point in the sample position distribution image is determined by the position information corresponding to the image point, the initial image generation model includes an initial first module and a second module, and the first generation unit 1303 is specifically used to:

[0243] Extracting a first undetermined image feature corresponding to the sample position distribution image through the initial first module;

[0244] The second module generates the pending texture image according to the first pending image features and the sample requirement information.

[0245] In a possible implementation, the first generating unit 1303 is specifically configured to:

[0246] Randomly generate a first initial texture image feature;

[0247] Performing denoising processing on the first initial texture image feature according to the first undetermined image feature and the sample requirement information to generate a first undetermined texture image feature;

[0248] The first undetermined texture image feature is decoded to generate the undetermined texture image.

[0249] In a possible implementation, the second module is a pre-trained model, and the pre-trained model is used to generate an image that meets the image requirement according to information representing the image requirement, and the adjustment unit 1304 is specifically used to:

[0250] According to the difference between the sample texture image and the pending texture image, the model parameters corresponding to the initial first module are adjusted to obtain the first module. The first module and the second module are used to constitute the image generation model. The first module is used to extract image features corresponding to the position distribution image, and the second module is used to generate the texture image according to the image features and the required information.

[0251] In a possible implementation manner, the apparatus further includes a third acquiring unit:

[0252] The third acquisition unit is used to acquire a sample depth image corresponding to the sample model, the sample depth image is used to display the sample model, the image points in the sample depth image have corresponding depth information, and the depth information is used to identify the distance between the model vertex displayed by the image point and the camera plane corresponding to the sample depth image;

[0253] The first generating unit 1303 is specifically used for:

[0254] The undetermined texture image is generated through an initial image generation model according to the sample requirement information, the sample position distribution image and the sample depth image.

[0255] In a possible implementation, the color corresponding to the image point in the sample depth image is determined by the depth information corresponding to the image point, the initial image generation model includes an initial first module, a third module and a fourth module, and the first generation unit 1303 is specifically used to:

[0256] Extracting a first undetermined image feature corresponding to the sample position distribution image through the initial first module;

[0257] Extracting the second undetermined image feature corresponding to the sample depth image through the third module;

[0258] The fourth module generates the undetermined texture image according to the first undetermined image feature, the second undetermined image feature and the sample requirement information.

[0259] In a possible implementation, the first generating unit 1303 is specifically configured to:

[0260] Randomly generating a second initial texture image feature, where the second initial texture image feature is used to characterize the initial texture image;

[0261] generating an initial preview image feature according to the second undetermined image feature and the second initial texture image feature, wherein the initial preview image feature is used to characterize an initial preview image, wherein the initial preview image is used to display the sample model at a first angle, wherein the first angle is an angle at which the sample model is displayed through the sample depth image, wherein the sample model corresponds to a first texture effect in the initial preview image, and wherein the first texture effect is a texture effect added through the initial texture image;

[0262] According to the initial preview image feature, the first undetermined image feature, the second undetermined image feature and the sample requirement information, denoising the second initial texture image feature to generate a second undetermined texture image feature;

[0263] The second undetermined texture image feature is decoded to generate the undetermined texture image.

[0264] In a possible implementation, the first generating unit 1303 is specifically configured to:

[0265] Using the second initial texture image feature and the initial preview image feature as input features corresponding to the first round of denoising processing, performing N rounds of denoising processing on the second initial texture image feature to generate the second undetermined texture image feature;

[0266] In the i-th round of denoising among the N rounds of denoising, the fourth module is used to perform the following steps:

[0267] According to the preview image features in the input features corresponding to the i-th round of denoising, the sample requirement information, the first to-be-determined image features and the second to-be-determined image features, denoising the texture image features in the input features corresponding to the i-th round of denoising to generate the first texture image features output by the i-th round of denoising, where N is a positive integer and i is a positive integer not greater than N;

[0268] Based on i being equal to N, determining the first texture image feature as the second undetermined texture image feature;

[0269] Based on i being less than N, generating a first preview image feature output by the i-th round of denoising processing according to the second undetermined image feature and the first texture image feature, wherein the first preview image feature is used to characterize a first preview image, the first preview image is used to display the sample model from the first angle, the sample model corresponds to a second texture effect in the first preview image, and the second texture effect is a texture effect added to the texture image characterized by the first texture image feature;

[0270] The first texture image feature and the first preview image feature are determined as input features corresponding to the (i+1)th round of denoising processing.

[0271] In a possible implementation, the denoising process is performed on the texture image features in the input features corresponding to the i-th round of denoising process according to the preview image features in the input features corresponding to the i-th round of denoising process, the sample requirement information, the first to-be-determined image features, and the second to-be-determined image features to generate the first texture image features output by the i-th round of denoising process, including:

[0272] The input features corresponding to the i-th round of denoising are concatenated to generate the image features to be processed corresponding to the i-th round of denoising;

[0273] Determine a query matrix, a key matrix, and a value matrix in an attention mechanism according to the first undetermined image feature, the second undetermined image feature, the sample requirement information, and a model parameter corresponding to the fourth module;

[0274] The image features to be processed corresponding to the i-th round of denoising are used as input features of the attention mechanism, and the image features to be processed are denoised by the attention mechanism to generate the first texture image features output by the i-th round of denoising.

[0275] In a possible implementation, the sample depth image is any one of a plurality of sample depth images, and the plurality of sample depth images are used to display the sample model from a plurality of angles, and the first generating unit 1303 is specifically used to:

[0276] The to-be-determined texture image is generated through an initial image generation model according to the sample requirement information, the sample position distribution image and the multiple sample depth images.

[0277] Based on the data processing method on the application side provided in the above embodiment, the present application also provides a data processing device, see Fig.14 , Fig.14 This is a structural block diagram of a data processing device provided in an embodiment of the present application. The device 1400 includes a fourth acquisition unit 1401, a fifth acquisition unit 1402, and a second generation unit 1403:

[0278] The fourth acquisition unit 1401 is used to acquire requirement information corresponding to the model to be processed, where the requirement information is used to characterize the texture effect required by the model to be processed;

[0279] The fifth acquisition unit 1402 is used to acquire a position distribution image corresponding to the model to be processed, wherein the model vertices on the model to be processed have corresponding image points in the position distribution image, and the image points in the position distribution image have corresponding position information, and the position information is used to identify the position of the model vertex corresponding to the image point in the position distribution image in the three-dimensional space;

[0280] The second generation unit 1403 is used to generate a texture image through an image generation model according to the demand information and the position distribution image, the image points corresponding to the model vertices on the model to be processed in the position distribution image are at the same position as the corresponding image points in the texture image, and the texture image is used to add a texture that satisfies the texture effect to the model to be processed.

[0281] In a possible implementation manner, the apparatus further includes a sixth acquiring unit:

[0282] The sixth acquisition unit is used to acquire a depth image corresponding to the model to be processed, the depth image is used to display the model to be processed, the image points in the depth image have corresponding depth information, and the depth information is used to identify the distance between the model vertex displayed by the image point and the camera plane corresponding to the depth image;

[0283] The second generating unit 1403 is specifically used for:

[0284] A texture image is generated according to the demand information, the position distribution image and the depth image through an image generation model.

[0285] In a possible implementation manner, the demand information is provided by a provider, and the device further includes an adding unit:

[0286] The adding unit is used to add texture to the model to be processed through the texture image to obtain a processed model, and the processed model is used to display it to the provider.

[0287] The present application also provides a computer device, see Fig.15 As shown, the computer device may be a terminal device, and a mobile phone is taken as an example:

[0288] Fig.15 FIG. 1 is a block diagram showing a partial structure of a mobile phone related to a terminal device provided in an embodiment of the present application. Fig.15The mobile phone includes: a radio frequency (RF) circuit 710, a memory 720, an input unit 730, a display unit 740, a sensor 750, an audio circuit 760, a wireless fidelity (WiFi) module 770, a processor 780, and a power supply 790. Those skilled in the art will understand that Fig.15 The mobile phone structure shown in the figure does not constitute a limitation on the mobile phone, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0289] Combine the following Fig.15 A detailed introduction to the various components of the mobile phone:

[0290] The RF circuit 710 can be used for receiving and sending signals during the process of sending and receiving information or making calls. In particular, after receiving the downlink information of the base station, it is sent to the processor 780 for processing; in addition, the designed uplink data is sent to the base station. Usually, the RF circuit 710 includes but is not limited to an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier (Low Noise Amplifier, referred to as LNA), a duplexer, etc. In addition, the RF circuit 710 can also communicate with the network and other devices through wireless communication. The above-mentioned wireless communication can use any communication standard or protocol, including but not limited to the Global System of Mobile communication (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, Short Messaging Service (SMS), etc.

[0291] The memory 720 can be used to store software programs and modules. The processor 780 executes various functional applications and data processing of the mobile phone by running the software programs and modules stored in the memory 720. The memory 720 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory 720 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0292] The input unit 730 can be used to receive input digital or character information, and to generate key signal input related to the user settings and function control of the mobile phone. Specifically, the input unit 730 may include a touch panel 731 and other input devices 732. The touch panel 731, also known as a touch screen, can collect the user's touch operation on or near it (such as the user's operation on the touch panel 731 or near the touch panel 731 using any suitable object or accessory such as a finger, stylus, etc.), and drive the corresponding connection device according to a pre-set program. Optionally, the touch panel 731 may include two parts: a touch detection device and a touch controller. Among them, the touch detection device detects the user's touch orientation, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into contact coordinates, and then sends it to the processor 780, and can receive and execute commands sent by the processor 780. In addition, the touch panel 731 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic waves. In addition to the touch panel 731, the input unit 730 may further include other input devices 732. Specifically, the other input devices 732 may include but are not limited to one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, and the like.

[0293] The display unit 740 can be used to display information input by the user or information provided to the user and various menus of the mobile phone. The display unit 740 may include a display panel 741. Optionally, the display panel 741 may be configured in the form of a liquid crystal display (Liquid Crystal Display, LCD), an organic light-emitting diode (Organic Light-Emitting Diode, OLED), etc. Further, the touch panel 731 may cover the display panel 741. When the touch panel 731 detects a touch operation on or near it, it is transmitted to the processor 780 to determine the type of touch event. Subsequently, the processor 780 provides a corresponding visual output on the display panel 741 according to the type of touch event. Although in Fig.15 In the embodiment, the touch panel 731 and the display panel 741 are used as two independent components to realize the input and output functions of the mobile phone, but in some embodiments, the touch panel 731 and the display panel 741 can be integrated to realize the input and output functions of the mobile phone.

[0294] The mobile phone may also include at least one sensor 750, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display panel 741 according to the brightness of the ambient light, and the proximity sensor may turn off the display panel 741 and / or the backlight when the mobile phone is moved to the ear. As a type of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in all directions (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that identify the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as for other sensors that can be configured in the mobile phone, such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., they will not be repeated here.

[0295] The audio circuit 760, the speaker 761, and the microphone 762 can provide an audio interface between the user and the mobile phone. The audio circuit 760 can transmit the received audio data to the speaker 761 after converting the received audio data into an electrical signal, which is converted into a sound signal for output; on the other hand, the microphone 762 converts the collected sound signal into an electrical signal, which is received by the audio circuit 760 and converted into audio data, and then the audio data is output to the processor 780 for processing, and then sent to another mobile phone through the RF circuit 710, or the audio data is output to the memory 720 for further processing.

[0296] WiFi is a short-range wireless transmission technology. The mobile phone can help users send and receive emails, browse web pages and access streaming media through the WiFi module 770. It provides users with wireless broadband Internet access. Fig.15A WiFi module 770 is shown, but it is understandable that it is not an essential component of the mobile phone and can be omitted as needed without changing the essence of the invention.

[0297] The processor 780 is the control center of the mobile phone. It uses various interfaces and lines to connect various parts of the entire mobile phone. By running or executing software programs and / or modules stored in the memory 720, and calling data stored in the memory 720, it executes various functions of the mobile phone and processes data, thereby performing overall detection of the mobile phone. Optionally, the processor 780 may include one or more processing units; preferably, the processor 780 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 780.

[0298] The mobile phone also includes a power supply 790 (such as a battery) for supplying power to various components. Preferably, the power supply can be logically connected to the processor 780 through a power management system, so that the power management system can manage charging, discharging, power consumption and other functions.

[0299] Although not shown, the mobile phone may also include a camera, a Bluetooth module, etc., which will not be described in detail here.

[0300] In this embodiment, the processor 780 included in the terminal device also has the following functions:

[0301] Acquire sample requirement information and a sample texture image, wherein the sample requirement information is used to characterize the sample texture effect required by the sample model, and the sample texture image is used to add a texture satisfying the sample texture effect to the sample model;

[0302] Acquire a sample position distribution image corresponding to the sample model, wherein the model vertices on the sample model have corresponding image points on the sample position distribution image, the image points corresponding to the model vertices of the sample model in the sample texture image have the same positions as the corresponding image points in the sample position distribution image, and the image points in the sample position distribution image have corresponding position information, and the position information is used to identify the positions of the model vertices corresponding to the image points in the sample position distribution image in three-dimensional space;

[0303] Generate a pending texture image according to the sample requirement information and the sample position distribution image through an initial image generation model;

[0304] According to the difference between the sample texture image and the texture image to be determined, the model parameters corresponding to the initial image generation model are adjusted to obtain an image generation model, wherein the image generation model is used to generate a texture image according to the requirement information and the position distribution image, wherein the requirement information is used to characterize the texture effect required by the model to be processed, and the position distribution image is used to identify the positions of the model vertices corresponding to the image points in the texture image on the model to be processed in three-dimensional space, and the texture image is used to add a texture that satisfies the texture effect to the model to be processed.

[0305] In this embodiment, the processor 780 included in the terminal device also has the following functions:

[0306] Obtaining demand information corresponding to the model to be processed, wherein the demand information is used to characterize the texture effect required by the model to be processed;

[0307] Acquire a position distribution image corresponding to the model to be processed, wherein the model vertices on the model to be processed have corresponding image points in the position distribution image, and the image points in the position distribution image have corresponding position information, and the position information is used to identify the position of the model vertex corresponding to the image point in the position distribution image in the three-dimensional space;

[0308] A texture image is generated according to the demand information and the position distribution image through an image generation model, the image points corresponding to the model vertices on the model to be processed in the position distribution image are at the same position as the corresponding image points in the texture image, and the texture image is used to add a texture that satisfies the texture effect to the model to be processed.

[0309] The present application also provides a server. Fig.16 As shown, Fig.16 The structural diagram of the server 800 provided in the embodiment of the present application, the server 800 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPU) 822 (for example, one or more processors) and a memory 832, and one or more storage media 830 (for example, one or more mass storage devices) storing application programs 842 or data 844. Among them, the memory 832 and the storage medium 830 can be temporary storage or permanent storage. The program stored in the storage medium 830 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the server. Furthermore, the central processing unit 822 can be configured to communicate with the storage medium 830 and execute a series of instruction operations in the storage medium 830 on the server 800.

[0310] The server 800 may also include one or more power supplies 826, one or more wired or wireless network interfaces 850, one or more input and output interfaces 858, and / or one or more operating systems 841, such as Windows Server 2000. TM , Mac OS X TM , Unix TM ,Linux TM , FreeBSD TM etc.

[0311] The steps performed by the server in the above embodiment can be based on Fig.16 The server structure shown.

[0312] An embodiment of the present application also provides a computer-readable storage medium for storing a computer program, wherein the computer program is used to execute any one of the data processing methods described in the aforementioned embodiments.

[0313] An embodiment of the present application further provides a computer program product including a computer program, which, when executed on a computer device, enables the computer device to execute the data processing method described in any one of the above embodiments.

[0314] It can be understood that in the specific implementation of this application, related data such as user information (such as demand information, texture images, etc.) are involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0315] A person of ordinary skill in the art can understand that all or part of the steps of implementing the above-mentioned method embodiment can be completed by hardware related to program instructions, and the above-mentioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above-mentioned method embodiment; and the above-mentioned storage medium can be at least one of the following media: read-only memory (English: read-only memory, abbreviated: ROM), RAM, magnetic disk or optical disk, etc. Various media that can store program codes.

[0316] It should be noted that each embodiment in this specification is described in a progressive manner, and the same and similar parts between the embodiments can refer to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can refer to the partial description of the method embodiments. The device and system embodiments described above are merely schematic, in which the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative work.

[0317] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A data processing method, characterized in that: The method comprises: Acquire sample requirement information and a sample texture image, wherein the sample requirement information is used to characterize the sample texture effect required by the sample model, and the sample texture image is used to add a texture satisfying the sample texture effect to the sample model; Acquire a sample position distribution image corresponding to the sample model, wherein the model vertices on the sample model have corresponding image points on the sample position distribution image, the image points corresponding to the model vertices of the sample model in the sample texture image have the same positions as the corresponding image points in the sample position distribution image, and the image points in the sample position distribution image have corresponding position information, and the position information is used to identify the positions of the model vertices corresponding to the image points in the sample position distribution image in three-dimensional space; Generate a pending texture image according to the sample requirement information and the sample position distribution image through an initial image generation model; According to the difference between the sample texture image and the texture image to be determined, the model parameters corresponding to the initial image generation model are adjusted to obtain an image generation model, wherein the image generation model is used to generate a texture image according to the requirement information and the position distribution image, wherein the requirement information is used to characterize the texture effect required by the model to be processed, and the position distribution image is used to identify the positions of the model vertices corresponding to the image points in the texture image on the model to be processed in three-dimensional space, and the texture image is used to add a texture that satisfies the texture effect to the model to be processed.

2. The method according to claim 1, characterized in that The color corresponding to the image point in the sample position distribution image is determined by the position information corresponding to the image point. The initial image generation model includes an initial first module and a second module. The initial image generation model generates a pending texture image according to the sample requirement information and the sample position distribution image, including: Extracting a first undetermined image feature corresponding to the sample position distribution image through the initial first module; The second module generates the pending texture image according to the first pending image features and the sample requirement information.

3. The method according to claim 2, characterized in that The step of generating the undetermined texture image according to the first undetermined image feature and the sample requirement information includes: Randomly generate a first initial texture image feature; Performing denoising processing on the first initial texture image feature according to the first undetermined image feature and the sample requirement information to generate a first undetermined texture image feature; The first undetermined texture image feature is decoded to generate the undetermined texture image.

4. The method according to claim 2, characterized in that: The second module is a pre-trained model, and the pre-trained model is used to generate an image that meets the image requirement according to the information representing the image requirement, and the model parameters corresponding to the initial image generation model are adjusted according to the difference between the sample texture image and the undetermined texture image to obtain the image generation model, including: According to the difference between the sample texture image and the pending texture image, the model parameters corresponding to the initial first module are adjusted to obtain the first module. The first module and the second module are used to constitute the image generation model. The first module is used to extract image features corresponding to the position distribution image, and the second module is used to generate the texture image according to the image features and the required information.

5. The method according to claim 1, characterized in that The method further comprises: Acquire a sample depth image corresponding to the sample model, the sample depth image is used to display the sample model, image points in the sample depth image have corresponding depth information, and the depth information is used to identify the distance between the model vertex displayed by the image point and the camera plane corresponding to the sample depth image; The generating model through the initial image, generating a pending texture image according to the sample requirement information and the sample position distribution image, comprises: The undetermined texture image is generated through an initial image generation model according to the sample requirement information, the sample position distribution image and the sample depth image.

6. The method according to claim 5, characterized in that The color corresponding to the image point in the sample depth image is determined by the depth information corresponding to the image point. The initial image generation model includes an initial first module, a third module and a fourth module. The initial image generation model generates a texture image to be determined according to the sample requirement information, the sample position distribution image and the sample depth image, including: Extracting a first undetermined image feature corresponding to the sample position distribution image through the initial first module; Extracting the second undetermined image feature corresponding to the sample depth image through the third module; The fourth module generates the undetermined texture image according to the first undetermined image feature, the second undetermined image feature and the sample requirement information.

7. The method according to claim 6, characterized in that The generating the undetermined texture image according to the first undetermined image feature, the second undetermined image feature and the sample requirement information comprises: Randomly generating a second initial texture image feature, where the second initial texture image feature is used to characterize the initial texture image; generating an initial preview image feature according to the second undetermined image feature and the second initial texture image feature, wherein the initial preview image feature is used to characterize an initial preview image, wherein the initial preview image is used to display the sample model at a first angle, wherein the first angle is an angle at which the sample model is displayed through the sample depth image, wherein the sample model corresponds to a first texture effect in the initial preview image, and wherein the first texture effect is a texture effect added through the initial texture image; According to the initial preview image feature, the first undetermined image feature, the second undetermined image feature and the sample requirement information, denoising the second initial texture image feature to generate a second undetermined texture image feature; The second undetermined texture image feature is decoded to generate the undetermined texture image.

8. The method according to claim 7, characterized in that The step of performing denoising on the second initial texture image feature to generate a second to-be-determined texture image feature according to the initial preview image feature, the first to-be-determined image feature, the second to-be-determined image feature and the sample requirement information comprises: Using the second initial texture image feature and the initial preview image feature as input features corresponding to the first round of denoising processing, performing N rounds of denoising processing on the second initial texture image feature to generate the second undetermined texture image feature; In the i-th round of denoising among the N rounds of denoising, the fourth module is used to perform the following steps: According to the preview image features in the input features corresponding to the i-th round of denoising, the sample requirement information, the first to-be-determined image features and the second to-be-determined image features, denoising the texture image features in the input features corresponding to the i-th round of denoising to generate the first texture image features output by the i-th round of denoising, where N is a positive integer and i is a positive integer not greater than N; Based on i being equal to N, determining the first texture image feature as the second undetermined texture image feature; Based on i being less than N, generating a first preview image feature output by the i-th round of denoising processing according to the second undetermined image feature and the first texture image feature, wherein the first preview image feature is used to characterize a first preview image, the first preview image is used to display the sample model from the first angle, the sample model corresponds to a second texture effect in the first preview image, and the second texture effect is a texture effect added to the texture image characterized by the first texture image feature; The first texture image feature and the first preview image feature are determined as input features corresponding to the (i+1)th round of denoising processing.

9. The method according to claim 8, characterized in that The method of performing denoising on the texture image features in the input features corresponding to the i-th round of denoising according to the preview image features in the input features corresponding to the i-th round of denoising, the sample requirement information, the first to-be-determined image features, and the second to-be-determined image features, to generate the first texture image features output by the i-th round of denoising, comprises: The input features corresponding to the i-th round of denoising are concatenated to generate the image features to be processed corresponding to the i-th round of denoising; Determine a query matrix, a key matrix, and a value matrix in an attention mechanism according to the first undetermined image feature, the second undetermined image feature, the sample requirement information, and a model parameter corresponding to the fourth module; The image features to be processed corresponding to the i-th round of denoising are used as input features of the attention mechanism, and the image features to be processed are denoised by the attention mechanism to generate the first texture image features output by the i-th round of denoising.

10. The method according to claim 5, characterized in that The sample depth image is any one of a plurality of sample depth images, and the plurality of sample depth images are used to display the sample model from a plurality of angles. The generating model through the initial image generates a pending texture image according to the sample requirement information, the sample position distribution image and the sample depth image, including: The to-be-determined texture image is generated through an initial image generation model according to the sample requirement information, the sample position distribution image and the multiple sample depth images.

11. A data processing method, characterized in that: The method comprises: Obtaining demand information corresponding to the model to be processed, wherein the demand information is used to characterize the texture effect required by the model to be processed; Acquire a position distribution image corresponding to the model to be processed, wherein the model vertices on the model to be processed have corresponding image points in the position distribution image, and the image points in the position distribution image have corresponding position information, and the position information is used to identify the position of the model vertex corresponding to the image point in the position distribution image in the three-dimensional space; A texture image is generated according to the demand information and the position distribution image through an image generation model, the image points corresponding to the model vertices on the model to be processed in the position distribution image are at the same position as the corresponding image points in the texture image, and the texture image is used to add a texture that satisfies the texture effect to the model to be processed.

12. The method according to claim 11, characterized in that The method further comprises: Acquire a depth image corresponding to the model to be processed, the depth image being used to display the model to be processed, the image points in the depth image having corresponding depth information, the depth information being used to identify the distance between the model vertex displayed by the image point and the camera plane corresponding to the depth image; The step of generating a texture image according to the demand information and the position distribution image by using an image generation model includes: A texture image is generated according to the demand information, the position distribution image and the depth image through an image generation model.

13. The method according to claim 11, characterized in that The demand information is provided by a provider, and the method further includes: The texture is added to the model to be processed by using the texture image to obtain a processed model, and the processed model is used to display to the provider.

14. A data processing device, characterized in that: The device comprises a first acquisition unit, a second acquisition unit, a first generation unit and an adjustment unit: The first acquisition unit is used to acquire sample requirement information and a sample texture image, wherein the sample requirement information is used to characterize the sample texture effect required by the sample model, and the sample texture image is used to add a texture satisfying the sample texture effect to the sample model; The second acquisition unit is used to acquire a sample position distribution image corresponding to the sample model, the model vertices on the sample model have corresponding image points on the sample position distribution image, the image points corresponding to the model vertices of the sample model in the sample texture image have the same positions as the corresponding image points in the sample position distribution image, and the image points in the sample position distribution image have corresponding position information, and the position information is used to identify the positions of the model vertices corresponding to the image points in the sample position distribution image in three-dimensional space; The first generating unit is used to generate a to-be-determined texture image according to the sample requirement information and the sample position distribution image by using an initial image generating model; The adjustment unit is used to adjust the model parameters corresponding to the initial image generation model according to the difference between the sample texture image and the to-be-determined texture image to obtain the image generation model, the image generation model is used to generate the texture image according to the requirement information and the position distribution image, the requirement information is used to characterize the texture effect required by the to-be-processed model, the position distribution image is used to identify the positions of the model vertices corresponding to the image points in the texture image on the to-be-processed model in three-dimensional space, and the texture image is used to add a texture that satisfies the texture effect to the to-be-processed model.

15. A data processing device, characterized in that: The device comprises a fourth acquisition unit, a fifth acquisition unit and a second generation unit: The fourth acquisition unit is used to acquire requirement information corresponding to the model to be processed, where the requirement information is used to characterize the texture effect required by the model to be processed; The fifth acquisition unit is used to acquire a position distribution image corresponding to the model to be processed, wherein the model vertices on the model to be processed have corresponding image points in the position distribution image, and the image points in the position distribution image have corresponding position information, and the position information is used to identify the position of the model vertex corresponding to the image point in the position distribution image in the three-dimensional space; The second generating unit is used to generate a texture image through an image generating model according to the demand information and the position distribution image, the image points corresponding to the model vertices on the model to be processed in the position distribution image are at the same position as the corresponding image points in the texture image, and the texture image is used to add a texture that satisfies the texture effect to the model to be processed.

16. A computer device, characterized in that: The computer device comprises a processor and a memory: The memory is used to store a computer program and transmit the computer program to the processor; The processor is used to execute the data processing method described in any one of claims 1 to 10 according to the instructions in the computer program, or to execute the data processing method described in any one of claims 11 to 13.

17. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store a computer program, and the computer program is used to execute the data processing method described in any one of claims 1 to 10, or execute the data processing method described in any one of claims 11 to 13.

18. A computer program product comprising a computer program, which, when executed on a computer device, enables the computer device to execute the data processing method according to any one of claims 1 to 10, or the data processing method according to any one of claims 11 to 13.

Images