Image-based texture filling method, device, equipment and storage medium

Abstract

The present disclosure provides an image-based texture filling method, device, equipment and storage medium, which relates to the field of image processing technology. The method includes: performing image segmentation processing on a filling object in an image to obtain a first image of the filling object and an area of ​​the filling object on the image; determining a three-dimensional grid model of the filling object based on the first image, wherein the grid points on the three-dimensional grid model have normal information and depth information; filling texture on the area of ​​the image based on the three-dimensional grid model to obtain a filling image, so that the normal and depth of the texture at the corresponding position on the filling image are consistent with the normal and depth of the grid points at the corresponding position on the three-dimensional grid model of the filling object. The present disclosure can effectively improve the naturalness and realism of the texture effect.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, and in particular to an image-based texture filling method, device, equipment and storage medium. Background Art

[0002] The application provided by the related technology can provide users with an image editing function, through which users can add interesting texture effects to images. However, the combination of the texture added based on the related technology and the original image is relatively stiff, unnatural, and has a poor sense of reality. Summary of the invention

[0003] In order to solve the above technical problem or at least partially solve the above technical problem, the present disclosure provides an image-based texture filling method, apparatus and terminal device.

[0004] A first aspect of an embodiment of the present disclosure provides an image-based texture filling method, comprising: performing image segmentation processing on a filling object in an image to obtain a first image of the filling object and a region of the filling object in the image; determining a three-dimensional mesh model of the filling object based on the first image, wherein mesh points in the three-dimensional mesh model have normal information and depth information; and filling texture on the region of the image based on the three-dimensional mesh model to obtain a filling image, such that the normal and depth of the texture at a corresponding position on the filling image are consistent with the normal and depth of the mesh points at a corresponding position on the three-dimensional mesh model of the filling object.

[0005] A second aspect of an embodiment of the present disclosure provides a texture filling device, including:

[0006] A segmentation processing module, used for performing image segmentation processing on the filling object in the image to obtain a first image of the filling object and a region of the filling object in the image;

[0007] A determination module, configured to determine, based on the first image, a three-dimensional mesh model of the filling object, wherein mesh points in the three-dimensional mesh model have normal information and depth information;

[0008] A texture filling module is used to fill texture on the area of ​​the image according to the three-dimensional mesh model to obtain a filled image, so that the normal and depth of the texture at the corresponding position on the filled image are consistent with the normal and depth of the grid point at the corresponding position on the three-dimensional mesh model of the filled object.

[0009] A third aspect of an embodiment of the present disclosure provides a terminal device, comprising: a memory and a processor; wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the above-mentioned texture filling method.

[0010] A fourth aspect of an embodiment of the present disclosure provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the processor executes the above-mentioned texture filling method.

[0011] Compared with the prior art, the technical solution provided by the embodiments of the present disclosure has the following advantages:

[0012] In the disclosed embodiment, a first image of the filled object and the area of ​​the filled object in the image are obtained by performing image segmentation processing on the filled object in the image; then, a three-dimensional mesh model of the filled object is determined based on the first image, wherein the mesh points in the three-dimensional mesh model of the filled object have normal information and depth information; and then, based on the three-dimensional mesh model of the filled object, a texture is filled in the area where the filled object is located to obtain a filled image, so that the normal and depth of the texture at the corresponding position on the filled image are consistent with the normal and depth of the mesh points at the corresponding position on the three-dimensional mesh model of the filled object. In the texture filling method provided in the disclosed embodiment, since the normal and depth of the texture at the corresponding position on the filled image are consistent with the normal and depth of the mesh points at the corresponding position on the three-dimensional mesh model of the filled object, the texture effect can reflect the real three-dimensional effect of the texture applied on the surface of the object, effectively improving the naturalness and realism of the texture effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0014] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0015] Figure 1 A flowchart of the image-based texture filling method according to an embodiment of the present disclosure;

[0016] Figure 2A is a schematic diagram of the first image according to an embodiment of the present disclosure;

[0017] Figure 2B is a schematic diagram of the second image according to an embodiment of the present disclosure;

[0018] Figure 3 A schematic diagram of a first image taking a person as an example according to an embodiment of the present disclosure;

[0019] Figure 4 A schematic diagram of a texture map according to an embodiment of the present disclosure;

[0020] Figure 5 This is a schematic diagram of an image after adding texture according to an embodiment of the present disclosure;

[0021] Figure 6 This is a schematic diagram of the UV distortion method described in the embodiment of the present disclosure;

[0022] Figure 7 It is a structural block diagram of the texture filling device according to an embodiment of the present disclosure;

[0023] Figure 8 It is a schematic diagram of the structure of the terminal device described in the embodiment of the present disclosure. DETAILED DESCRIPTION

[0024] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0025] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.

[0026] At present, the texture added based on the image editing function is relatively stiff and unnatural in combination with the original image, and has poor realism. In order to improve the naturalness and realism of the texture effect, the embodiment of the present disclosure provides an image-based texture filling method, device and terminal device, which are described in detail below.

[0027] Embodiment 1:

[0028] Reference Figure 1 The flowchart of the image-based texture filling method shown in FIG. 1 is applicable to devices with image editing functions, such as digital cameras, mobile phones, tablet computers, etc. The image-based texture filling method includes the following steps:

[0029] Step 110 , performing image segmentation processing on the filling object in the image to obtain a first image of the filling object and a region of the filling object in the image.

[0030] In this embodiment, the obtained image can be an image captured by an image acquisition device, or an image downloaded from the network, stored locally, or manually uploaded. The image may include at least one filling object, wherein the filling object may be a person, an animal, a machine, or any other object to be filled with texture.

[0031] This embodiment may adopt a preset segmentation method, such as a region-based segmentation method or an edge-based segmentation method, to segment and obtain a first image of the filling object and a region of the filling object in the image. Figure 2A The filling object shown in the figure is a person's clothes. The first image a is an image of the person's clothes. Figure 2B The area b where the character's clothes are located is the area consistent with the outline of the character's clothes. Of course, this is only an example and not a sole limitation.

[0032] Step 120: Determine and obtain a three-dimensional mesh model of the filling object according to the first image, wherein mesh points in the three-dimensional mesh model have normal information and depth information.

[0033] In some possible implementations, the first image can be input into a 3D reconstruction model, which is pre-trained. A 3D mesh model of the filling object is reconstructed based on the 3D reconstruction model. Generally, the grid points on the 3D mesh model of the filling object have normal information and depth information. In another implementation, the image of each position on the filling object can be obtained by image segmentation technology. That is, in this method, the first image includes multiple sub-images, which are images of each position on the filling object. Then, based on the preset 3D reconstruction model, the 3D mesh model of each sub-image is constructed respectively, and the normal and depth of the grid points on the 3D mesh model of each sub-image are determined according to the 3D mesh model of each sub-image. At this point, it is known that each sub-image obtained by image segmentation is an image of each position on the filling object, but it is not known which position on the filling object these sub-images specifically correspond to. In order to obtain the 3D mesh model of the filling object or the normal and depth of each grid point on the 3D mesh model of the filling object, it is necessary to determine the position on the filling object corresponding to each sub-image. In the actual scene, the normal and depth of the points on different parts of the object have certain regularities, and the position of the sub-image can be judged according to these regularities. Therefore, in an exemplary implementation of the present embodiment, a prediction model can be pre-trained, and the input of the training model can be the normal and depth of the grid points on the three-dimensional grid model, and the output can be the position of the corresponding image on the object image. Therefore, in the present embodiment, by inputting the normal and depth of the grid points on the three-dimensional grid model of each sub-image into the prediction model, the position of each sub-image on the filling object can be obtained, and then according to the position of each sub-image on the filling object and the three-dimensional grid model of each sub-image, the three-dimensional grid model of the filling object can be reconstructed, and the normal and depth of the grid points on the three-dimensional grid model of the filling object can be obtained.

[0034] Step S130, filling texture on the area of ​​the image according to the three-dimensional mesh model to obtain a filled image, so that the normal and depth of the texture at the corresponding position on the filled image are consistent with the normal and depth of the grid point at the corresponding position on the three-dimensional mesh model of the filled object.

[0035] In this embodiment, the texture filled in the area where the filling object is located can be at least one of a texture map and a program texture. Among them, a texture map refers to a two-dimensional image with a specific texture pattern. A program texture refers to a texture generated by a mathematical description method such as a noise algorithm. Due to the texture filled in the way of texture maps and / or program textures, the normal direction of the texture of each point on the image is perpendicular to the image surface and upward, and the depth is the depth of the image surface. Therefore, in order to make the filled texture reflect the three-dimensional effect in the actual three-dimensional scene, it is necessary to adjust the coordinates of the texture after the texture is filled to present a real three-dimensional effect. In an exemplary implementation of this embodiment, the coordinates of the filled texture can be distorted by an image distortion method to obtain a filled object, so that the normal and depth of the texture at the corresponding position on the filled image are consistent with the normal and depth of the grid point at the corresponding position on the three-dimensional grid model of the filled object, so that the final texture filling effect can present a real three-dimensional stereoscopic effect, and the naturalness and realism of the texture effect are improved.

[0036] Specifically, according to the three-dimensional mesh model of the filling object, there are multiple methods for filling the texture on the area where the filling object is located to obtain the filling image. In an exemplary method, based on the image segmentation technology, the filling object can be cut out from the area of ​​the filling object in the image to obtain the mask of the area, and then the texture rendering is performed on the mask of the area according to the depth and normal of the grid points on the three-dimensional mesh model of the filling object, so that the normal and depth of the corresponding position on the rendered filling image are consistent with the normal and depth of the grid points at the corresponding position on the three-dimensional mesh model of the filling object. In another exemplary implementation of the present embodiment, after obtaining the area of ​​the filling object on the image and the three-dimensional mesh model of the filling object, the texture can be directly added to the area of ​​the image, and the added texture can be distorted so that the normal and depth of the corresponding position on the filling image are consistent with the normal and depth of the grid points at the corresponding position on the three-dimensional mesh model of the filling object. Of course, the above two filling methods are only examples and are not all implementation methods of the present disclosure.

[0037] The disclosed embodiment performs image segmentation processing on the filling object in the image to obtain a first image of the filling object and the area of ​​the filling object in the image; then, based on the first image, a three-dimensional mesh model of the filling object is determined, wherein the mesh points in the three-dimensional mesh model of the filling object have normal information and depth information; and then, based on the three-dimensional mesh model of the filling object, a texture is filled in the area where the filling object is located to obtain a filling image, so that the normal and depth of the texture at the corresponding position on the filling image are consistent with the normal and depth of the mesh points at the corresponding position on the three-dimensional mesh model of the filling object. In the texture filling method provided by the disclosed embodiment, since the normal and depth of the texture at the corresponding position on the filling image are consistent with the normal and depth of the mesh points at the corresponding position on the three-dimensional mesh model of the filling object, the texture effect can reflect the real three-dimensional effect of the texture applied on the surface of the object, effectively improving the naturalness and realism of the texture effect.

[0038] This embodiment also provides a method for constructing a three-dimensional grid model of a filling object. Considering that in practical applications, a filling object generally has multiple regions of interest. Figure 3 Taking the person shown as an example, the person may have multiple regions of interest such as head, coat, chest and abdomen, and legs. Accordingly, the first image may include multiple sub-images, which are images filling different positions on the object, such as: head sub-image, sleeve image, chest and abdomen sub-image, and leg sub-image.

[0039] In the case where the first image includes a plurality of sub-images, the specific implementation method of constructing the three-dimensional mesh model of the filling object according to the first image may refer to the following steps 1 to 3:

[0040] Step 1: construct a three-dimensional grid model of each sub-image based on a preset three-dimensional reconstruction model, wherein the grid points of the three-dimensional grid model of each sub-image have normal information and depth information.

[0041] The method of constructing a 3D mesh model of a sub-image based on a 3D reconstruction model is only an exemplary method of this embodiment and is not the only method. For example, in other embodiments, a 3D mesh model of an object can be obtained by matching from a sub-image based on a preset object matching algorithm. For example, for a face, a face matching algorithm can be used to match a 3D mesh model of the face based on a face image. Of course, this is only an example and is not the only limitation on the object matching algorithm referred to in this embodiment.

[0042] In this embodiment, there are multiple methods for obtaining the normal and depth of the grid points on the three-dimensional grid model of the filling object. For example, in an exemplary method, a sub-image can be input into a preset data analysis model, and the normal prediction map and depth prediction map of the grid points on the three-dimensional grid model of the sub-image are output through the data analysis model. The above-mentioned normal prediction map is used to represent the direction of the normal vector of the grid point on the three-dimensional grid model of the sub-image perpendicular to the section between the point and the three-dimensional grid model. The depth prediction map is used to represent the Z-axis coordinate value of the grid point on the three-dimensional grid model of the sub-image.

[0043] In addition, in addition to using the 3D reconstruction model to determine the normal and depth of the grid points on the 3D grid model of the sub-image, other methods can also be used in practical applications to determine the normal and depth of the points on the sub-image. For example, the normal of the points on the sub-image can be determined based on the photometric stereo method, and the depth of the points can be determined based on the grayscale value of the points on the sub-image.

[0044] Step 2: According to the normal and depth of the grid points on the three-dimensional grid model of each sub-image, a preset prediction model is used to determine the position of each sub-image on the filling object.

[0045] The prediction model is a pre-trained model, such as a PRN (Pose Residual Network), which is used to calculate the position of a point on an image based on the normal and depth of a grid point on a three-dimensional grid model of the image.

[0046] In this embodiment, the normal and depth of the grid points on the three-dimensional grid model of the sub-image can be input into the prediction model, and the prediction model calculates the position coordinates of the grid points on the three-dimensional grid model of the sub-image on the three-dimensional grid model of the filling object based on the normal and depth of the grid points. According to the position coordinates of the grid points on the three-dimensional grid model of the sub-image on the three-dimensional grid model of the filling object, the coordinates of the projection points of the grid points on the three-dimensional grid model of the sub-image on the filling object are determined, and the parameters of the bounding box of the sub-image on the filling object are determined based on the coordinates of the projection points of the grid points on the three-dimensional grid model of the sub-image on the filling object. The parameters of the bounding box of the sub-image are compared with the preset position parameters of each sub-region such as the sleeves, chest and abdomen, and legs of the filling object, and the position of the sub-image on the filling object is determined based on the comparison result.

[0047] Step 3: Determine and obtain the three-dimensional mesh model of the filling object according to the position of each sub-image on the filling object and the three-dimensional mesh model of each sub-image.

[0048] In addition, the embodiments of the present disclosure may have multiple ways of filling the area where the filling object is located with a texture, such as at least one of the following ways.

[0049] Method 1: Add a preset texture map to the area formed after the fill object is segmented from the image. Texture maps are pre-stored texture data, such as Figure 4 Examples of texture maps for the three different shapes shown.

[0050] Method 2: Generate a procedural texture on an area formed after the fill object is segmented from the image according to preset texture parameters; and insert a background material of the procedural texture on the area.

[0051] In the specific implementation, a set of preset texture parameters is used by the computer to generate a program texture consistent with the area outline, and the generated program texture is added to the area. In practical applications, the surfaces of many filled objects have personalized materials, such as different paint materials of vehicles (such as high gloss paint, matte paint, pearl paint) present different visual effects, and different clothing materials of characters (such as linen, silk, leather) present different visual effects. Therefore, in order to further improve the realism of the texture, this embodiment can also continue to insert background materials of program textures in the area, and the background materials are such as wood grain, metal, stone and other materials.

[0052] When combining the above two methods, you can refer to Figure 5 The schematic diagram of the image after adding texture shown in the figure, first add the preset texture map to the area, for example, add the texture map in the shape of horizontal stripes to the area, and obtain the first texture image. Then, based on the first texture image, according to the preset texture parameters, generate a program texture on the area, and the program texture takes a circle set as an example to obtain the second texture image. Finally, insert the background material of the program texture on the area and perform UV distortion processing to obtain the third texture image.

[0053] However, for the above texture map, it cannot truly reflect the dynamic changes of the surface of the filled object, such as the changes in wrinkles on the surface of clothes caused by the action of the character. For the procedural texture, although it is created based on the texture parameters, the texture meets certain requirements of the user in terms of scale and direction, and there is no concept of distortion, but it still cannot completely match the real surface of the filled object. At the same time, the position of each sub-image determined in the above step 2 on the filled object does not require high accuracy. A relatively blurred position will not reflect some details. Therefore, when the above texture is added to the above area, whether it is a texture map or a procedural texture, the richness of details cannot be completely consistent with the real dynamic changes on the surface of the filled object.

[0054] In order to improve the above-mentioned problem of the difference between the texture and the filled object, this embodiment can distort the texture on the filled object so that the normal and depth of the texture of the points on the filled object on the three-dimensional mesh model of the filled object are consistent with the normal and depth of the points on the filled object on the three-dimensional mesh model of the filled object.

[0055] In specific implementation, the texture can be distorted by image distortion algorithms such as UV distortion, so that the texture produces directional changes that conform to real logic, so as to simulate the real three-dimensional effect of corresponding points on the filled object. Figure 6 The UV distortion algorithm shown distorts a regular-shaped texture into an irregular-shaped texture to make the texture effect closer to the real effect.

[0056] In summary, the texture filling method provided in the above embodiment fills the texture on the area where the filling object is located according to the normal and depth of the points on the three-dimensional mesh of the filling object, and distorts the texture so that the normal and depth of the texture of the points on the area on the three-dimensional mesh model of the filling object are consistent with the normal and depth of the points on the area on the three-dimensional mesh model of the filling object; therefore, the texture effect can be closer to the real surface of the filling object, effectively improving the naturalness and realism of the texture effect.

[0057] Embodiment 2:

[0058] For the texture filling method provided in the first embodiment, the present disclosure provides a texture filling device, see Figure 7 A structural block diagram of a texture filling device is shown, the device comprising:

[0059] A segmentation processing module 702 is used to perform image segmentation processing on the filling object in the image to obtain a first image of the filling object and a region of the filling object in the image;

[0060] A determination module 704 is used to determine a three-dimensional mesh model of the filling object according to the first image, wherein mesh points in the three-dimensional mesh model have normal information and depth information;

[0061] The texture filling module 706 is used to fill the texture on the area of ​​the image according to the three-dimensional mesh model to obtain a filled image, so that the normal and depth of the texture at the corresponding position on the filled image are consistent with the normal and depth of the grid point at the corresponding position on the three-dimensional mesh model of the filled object.

[0062] In one implementation, the determination module 704 is configured to:

[0063] According to the first image, a preset three-dimensional reconstruction model is used to construct a three-dimensional mesh model of the filling object.

[0064] In one embodiment, the first image includes a plurality of sub-images, and the plurality of sub-images are images filling different positions on the object respectively; the determination module 704 includes:

[0065] A model building submodule is used to build a three-dimensional grid model of each sub-image based on a preset three-dimensional reconstruction model, wherein the grid points of the three-dimensional grid model of each sub-image have normal information and depth information;

[0066] A first determination submodule is used to determine the position of each sub-image on the filling object using a preset prediction model according to the normal and depth of the grid points on the three-dimensional grid model of each sub-image;

[0067] The second determination submodule is used to determine the three-dimensional mesh model of the filling object according to the position of each sub-image on the filling object and the three-dimensional mesh model of each sub-image.

[0068] In one embodiment, the texture filling module 706 includes:

[0069] An image cutting submodule, used for cutting the filling object out of the area of ​​the image;

[0070] The filling submodule is used to fill the texture on the area of ​​the image according to the three-dimensional grid model of the filling object to obtain a filled image.

[0071] In one embodiment, the filling submodule comprises:

[0072] A filling subunit, used for filling the area with a texture;

[0073] The texture distortion subunit is used to distort the texture on the area according to the normal and depth of the grid points on the three-dimensional grid model of the filling object to obtain a filling image.

[0074] In one embodiment, the filling subunit is used to add a preset texture map to the area.

[0075] In one embodiment, the filling subunit is used to: generate a program texture on the area according to preset texture parameters; and insert a background material of the program texture on the area.

[0076] Based on the foregoing embodiments, this embodiment provides a terminal device, including: a memory and a processor; wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the above method.

[0077] See also Figure 8 As shown, the terminal device may include:

[0078] Processor 801, memory 802, input device 803 and output device 804. The number of processors 801 in the atlas packaging device can be one or more. Figure 8 In some embodiments of the present invention, the processor 801, the memory 802, the input device 803 and the output device 804 may be connected via a bus or other means, wherein: Figure 8 The example of connecting through bus is taken in the following.

[0079] The memory 802 can be used to store software programs and modules. The processor 801 executes various functional applications and data processing of the atlas packaging device by running the software programs and modules stored in the memory 802. The memory 802 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function, etc. In addition, the memory 802 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. The input device 803 may be used to receive input digital or character information, and to generate signal input related to user settings and function control of the atlas packaging device.

[0080] Specifically in this embodiment, the processor 801 will load the executable files corresponding to the processes of one or more applications into the memory 802 according to the following instructions, and the processor 801 will run the applications stored in the memory 802, thereby realizing the various functions of the above-mentioned atlas packaging device.

[0081] This embodiment also provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the processor executes the above method.

[0082] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0083] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A texture filling method based on an image, characterized in that: include: Performing image segmentation processing on the filling object in the image to obtain a first image of the filling object and a region of the filling object in the image; Determine, according to the first image, a three-dimensional mesh model of the filling object, wherein mesh points in the three-dimensional mesh model have normal information and depth information; Filling the area of ​​the image with a texture according to the three-dimensional mesh model to obtain a filled image, so that the normal and depth of the texture at a corresponding position on the filled image are consistent with the normal and depth of the mesh point at a corresponding position on the three-dimensional mesh model of the filled object; Determining, according to the first image, to obtain a three-dimensional mesh model of the filling object includes: According to the first image, a preset three-dimensional reconstruction model is used to construct a three-dimensional mesh model of the filling object; The first image includes a plurality of sub-images, and the plurality of sub-images are images at different positions on the filling object respectively; The step of constructing a three-dimensional mesh model of the filling object using a preset three-dimensional reconstruction model according to the first image includes: Based on the preset three-dimensional reconstruction model, a three-dimensional grid model of each sub-image is constructed, and the grid points of the three-dimensional grid model of each sub-image have normal information and depth information; According to the normal and depth of the grid points on the three-dimensional grid model of each sub-image, a preset prediction model is used to determine the position of each sub-image on the filling object; The three-dimensional mesh model of the filling object is determined based on the position of each sub-image on the filling object and the three-dimensional mesh model of each sub-image.

2. The method according to claim 1, characterized in that Filling the texture on the region of the image according to the three-dimensional grid model to obtain a filled image comprises: cutting the filled object out of the region of the image; According to the three-dimensional mesh model of the filling object, a texture is filled on the area of ​​the image to obtain a filled image.

3. The method according to claim 2, characterized in that Filling the texture on the area of ​​the image according to the three-dimensional grid model of the filling object to obtain a filled image includes: filling the area with a texture; The texture on the region is distorted according to the normal and depth of the grid points on the three-dimensional grid model of the filling object to obtain a filling image.

4. The method according to claim 3, characterized in that The filling the area with a texture comprises: Add a preset texture map to the area.

5. The method according to claim 3, characterized in that: The filling the area with a texture comprises: Generating a procedural texture on the area according to preset texture parameters; Inserts the background material of the procedural texture over the area.

6. A texture filling device, characterized in that: include: A segmentation processing module, used for performing image segmentation processing on the filling object in the image to obtain a first image of the filling object and a region of the filling object in the image; A determination module, configured to determine, based on the first image, a three-dimensional mesh model of the filling object, wherein mesh points in the three-dimensional mesh model have normal information and depth information; A texture filling module, used to fill the texture on the area of ​​the image according to the three-dimensional mesh model to obtain a filled image, so that the normal and depth of the texture at the corresponding position on the filled image are consistent with the normal and depth of the grid point at the corresponding position on the three-dimensional mesh model of the filled object; The determining module is used to: According to the first image, a preset three-dimensional reconstruction model is used to construct a three-dimensional mesh model of the filling object; The first image includes a plurality of sub-images, and the plurality of sub-images are images at different positions on the filling object respectively; The determination module comprises: A model building submodule is used to build a three-dimensional grid model of each sub-image based on a preset three-dimensional reconstruction model, wherein the grid points of the three-dimensional grid model of each sub-image have normal information and depth information; A first determination submodule is used to determine the position of each sub-image on the filling object using a preset prediction model according to the normal and depth of the grid points on the three-dimensional grid model of each sub-image; The second determination submodule is used to determine the three-dimensional mesh model of the filling object according to the position of each sub-image on the filling object and the three-dimensional mesh model of each sub-image.

7. The device according to claim 6, characterized in that The texture filling module comprises: An image cutting submodule, used for cutting the filling object out of the area of ​​the image; The filling submodule is used to fill the texture on the area of ​​the image according to the three-dimensional grid model of the filling object to obtain a filled image.

8. The device according to claim 7, characterized in that The filling submodule comprises: A filling subunit, used for filling the area with a texture; The texture distortion subunit is used to distort the texture on the area according to the normal and depth of the grid points on the three-dimensional grid model of the filling object to obtain a filling image.

9. The device according to claim 8, characterized in that The filling subunit is used for: Add a preset texture map to the area.

10. The device according to claim 8, characterized in that The filling subunit is used for: Generating a procedural texture on the area according to preset texture parameters; and Inserts the background material of the procedural texture over the area.

11. A terminal device, characterized in that: include: Memory and processor; The memory stores a computer program, and when the computer program is executed by the processor, the processor executes the method as claimed in any one of claims 1 to 5.

12. A computer-readable storage medium, characterized in that: The storage medium stores a computer program. When the computer program is executed by a processor, the processor executes the method according to any one of claims 1 to 5.

Images