Texture mapping processing method, device and equipment and storage medium

Abstract

Embodiments of the present disclosure provide a texture mapping processing method and device, equipment and storage medium, by determining the visibility list of the patches in the three-dimensional mesh model, the visibility list of the patches records the images visible to the image perspective of the patches, and based on the visibility list of the patches, the interface patch corresponding to any image is determined, the interface patch corresponding to any image is the patch of the intersection position of the foreground region and the background region in the three-dimensional mesh model under the image perspective of any image, then based on the interface patch corresponding to any image and the modeling error, the mapping participation weight of the patch participating in the mapping with any image is set, and then based on the visibility list of the patches and the mapping participation weight corresponding to the images in the visibility list, the patches in the three-dimensional mesh model are mapped, which helps to avoid the foreground image being mapped to the background geometric position of the model, and can enhance the visual reality.

Description

Technical Field

[0001] This disclosure relates to three-dimensional reconstruction technology and data processing technology, and in particular to a method, apparatus, device and storage medium for processing texture maps. Background Technology

[0002] During 3D reconstruction, a series of errors in the reconstruction process, such as camera calibration error, pose estimation error, and geometric error, inevitably lead to problems in the precise alignment of color images and geometric positions during texture mapping. For example, a color image in the foreground position may be applied to the background geometric position, resulting in poor texture mapping effect.

[0003] When problems arise where color images and geometric positions are not aligned, related technologies typically employ manual model repair methods, such as manually adjusting texture coordinates and modifying texture maps to optimize the texture map. This process is time-consuming and costly. Summary of the Invention

[0004] To address the aforementioned technical problems, embodiments of this disclosure provide a method, apparatus, device, and storage medium for processing texture mapping.

[0005] According to one aspect of the present disclosure, a method for processing texture maps is provided, comprising: A visibility list of patches in a 3D mesh model is determined, wherein the visibility list of patches records the images that the patches are visible from the image viewpoint; Based on the visibility list of the patches, the interface patch corresponding to any image is determined. The interface patch corresponding to any image is the patch at the boundary between the foreground region and the background region in the three-dimensional mesh model under the image view of any image. Based on the interface patch corresponding to any one of the images and the modeling error, the patch is set with the texture participation weight of any one of the images participating in the texture mapping. Based on the visibility list of the facets and the texture participation weights corresponding to the images in the visibility list, texture processing is performed on the facets in the 3D mesh model.

[0006] According to another aspect of the present disclosure, a texture mapping processing apparatus is provided, comprising: The list determination module is used to determine the visibility list of patches in the 3D mesh model. The visibility list of patches records the images of the patches that are visible from the image viewpoint. The interface patch determination module is used to determine the interface patch corresponding to any image based on the visibility list of the patches. The interface patch corresponding to any image is the patch at the boundary between the foreground region and the background region in the three-dimensional mesh model under the image view of the image of the image. The weight setting module is used to set the texture participation weight of the texture with the image as the texture based on the interface patch corresponding to any image and the modeling error. The texture mapping module is used to perform texture mapping on the faces in the 3D mesh model based on the visibility list of the faces and the texture participation weights corresponding to the images in the visibility list.

[0007] According to another aspect of the present disclosure, a computer-readable storage medium is provided, the storage medium storing computer program instructions, which, when executed, implement the above-described texture mapping processing method.

[0008] According to another aspect of the present disclosure, an electronic device is provided, the electronic device comprising: Memory, used to store computer program products; A processor is used to execute a computer program product stored in memory, and when the computer program product is executed, it implements the above-described texture mapping processing method.

[0009] According to another aspect of the present disclosure, a computer program product is provided, including computer program instructions, characterized in that, when the computer program instructions are executed by a processor, they implement the above-described texture mapping processing method.

[0010] Based on the embodiments of this disclosure, when performing texture mapping, a visibility list of patches in the 3D mesh model is first determined. The visibility list of each patch records the image visible to the patch from the image viewpoint. Then, based on the visibility list of the patches, the interface patch corresponding to any image is determined. The interface patch corresponding to any image is the patch at the boundary between the foreground region and the background region in the 3D mesh model from the image viewpoint of any image. Then, based on the interface patch corresponding to any image and the modeling error, a mapping participation weight is set for the patch to participate in the mapping of any image. Finally, based on the visibility list of the patches and the mapping participation weight corresponding to the images in the visibility list, the patches in the 3D mesh model are mapped. By setting the mapping participation weight for the patches in the region where the interface patch corresponding to any image is located based on the modeling error, the participation of the image in mapping the interface patch can be reduced in the mapping, which helps to avoid the foreground image being mapped to the background geometry of the model, thereby enhancing visual realism and improving the mapping effect.

[0011] The technical solutions of this disclosure will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0012] The above and other objects, features, and advantages of this disclosure will become more apparent from the more detailed description of the embodiments thereof in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this disclosure and form part of the specification. They are used together with the embodiments of this disclosure to explain the disclosure and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0013] This disclosure will become clearer with reference to the accompanying drawings and the following detailed description, wherein: Figure 1 A flowchart illustrating one embodiment of the texture mapping method disclosed herein; Figure 2 This is a schematic diagram illustrating that the facets are not visible to the image in the texture mapping method disclosed herein; Figure 3 This is a schematic diagram illustrating the visibility of a patch in the image during the texture mapping process of this disclosure. Figure 4 This is a schematic diagram of the interface piece in the texture mapping method disclosed herein; Figure 5 This is a flowchart of step 101 in the texture mapping processing method of this disclosure; Figure 6 This is a flowchart of step 102 in the texture mapping processing method of this disclosure; Figure 7 This is a flowchart of step 103 in the texture mapping processing method of this disclosure; Figure 8 The image shows the effect of applying the texture mapping method disclosed herein; Figure 9 This is a schematic diagram of the structure of one embodiment of the texture mapping processing apparatus of this disclosure; Figure 10 This is a schematic diagram of the structure of yet another embodiment of the texture mapping processing apparatus of this disclosure; Figure 11 This is a structural diagram of an electronic device provided as an illustrative embodiment of the present disclosure. Detailed Implementation

[0014] Hereinafter, exemplary embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present disclosure, and not all embodiments of the present disclosure, and it should be understood that the present disclosure is not limited to the exemplary embodiments described herein.

[0015] It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of this disclosure.

[0016] Those skilled in the art will understand that the terms "first," "second," etc., in the embodiments of this disclosure are only used to distinguish different steps, devices, or modules, and do not represent any specific technical meaning, nor do they indicate a necessary logical order between them.

[0017] It should also be understood that in the embodiments disclosed herein, "a plurality of" may refer to two or more, and "at least one" may refer to one, two or more.

[0018] It should also be understood that any component, data or structure mentioned in the embodiments of this disclosure can generally be understood as one or more unless expressly defined or given to the contrary in the context.

[0019] Furthermore, the term "and / or" in this disclosure is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this disclosure generally indicates that the preceding and following related objects have an "or" relationship.

[0020] It should also be understood that the description of the various embodiments in this disclosure emphasizes the differences between the various embodiments, and the similarities or similarities can be referred to each other. For the sake of brevity, they will not be described in detail.

[0021] At the same time, it should be understood that, for ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn according to actual scale.

[0022] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit this disclosure or its application or use.

[0023] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0024] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0025] The embodiments disclosed herein can be applied to electronic devices such as terminal devices, computer systems, and servers, and can operate together with a wide range of other general-purpose or special-purpose computing system environments or configurations. Examples of well-known terminal devices, computing systems, environments, and / or configurations suitable for use with electronic devices such as terminal devices, computer systems, and servers include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments including any of the above systems, etc.

[0026] Electronic devices such as terminal devices, computer systems, and servers can be described in the general context of computer system executable instructions (such as program modules) executed by a computer system. Typically, program modules can include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems / servers can be implemented in distributed cloud computing environments, where tasks are executed by remote processing devices linked through communication networks. In distributed cloud computing environments, program modules can reside on local or remote computing system storage media, including storage devices.

[0027] This disclosure outlines In developing this disclosure, the inventors discovered that during 3D reconstruction, errors in various stages, such as camera calibration, pose estimation, and geometric errors, inevitably lead to misalignment between color images and geometric positions during texture mapping. For example, a color image from the foreground might be applied to a background geometric position, resulting in poor texture mapping. In related technologies, to improve texture mapping quality, manual model retouching is typically used after texture mapping is completed, which is time-consuming and labor-intensive.

[0028] In order to improve the texture mapping effect and reduce labor costs, the inventors proposed the technical solution disclosed herein.

[0029] Exemplary methods Figure 1 This is a flowchart illustrating one embodiment of the texture mapping processing method disclosed herein; this texture mapping processing method can be applied to electronic devices (such as computer systems, servers, etc.). Figure 1 As shown, the processing method for this texture map includes the following steps 101 to 104. Each step is explained below.

[0030] In step 101, a list of the visibility of patches in the 3D mesh model is determined. The list of the visibility of patches records the images in which the patches are visible from the image viewpoint.

[0031] In a 3D mesh model, a patch is the basic geometric unit used to represent the surface of the model. It is usually formed by connecting three or more vertices, and common patch shapes include triangular patches or quadrilateral patches.

[0032] The images can be used to generate or optimize texture maps, and can include images from multiple perspectives and positions in a 3D scene captured by an image acquisition device, such as a camera. A 3D mesh model can be obtained through 3D reconstruction based on the images from multiple perspectives.

[0033] In this embodiment of the disclosure, for an image viewpoint, the patches in the 3D mesh model include patches in the foreground region, patches in the background region, and patches at the boundary positions. Patches in the foreground region are visible from this image viewpoint, patches in the background region are not visible from this image viewpoint, and patches at the boundary positions are partially visible and partially invisible from this image viewpoint. It is understood that the patches in the foreground region, the patches in the background region, and the patches at the boundary positions are different for different image viewpoints. Specifically, "partially visible and partially invisible" means that only a portion of the patch at the boundary position is visible from this image viewpoint.

[0034] In this embodiment of the disclosure, the visibility list of patches records an image in which each patch is fully visible from the image viewpoint.

[0035] See Figure 2 The image illustrates three faces from a 3D scene model: face a, face b, and face c. It also shows image 1 captured from an image viewpoint. As can be seen, the ray pointing from the image viewpoint to face a (indicated by label 21) is occluded by the foreground region, so face a is not visible from the image viewpoint of image 1. On the other hand, the ray pointing from the image viewpoint to face c (indicated by label 22) is not occluded, so face c is visible from the image viewpoint of image 1.

[0036] See Figure 3 The diagram illustrates three faces from multiple faces in a 3D scene model, along with three images from three different viewpoints: face a, face b, face c, image 1, image 2, and image 3. It can be seen that rays from the viewpoints of images 1, 2, and 3 pointing to face c are not occluded. Therefore, face c is visible from the viewpoints of images 1, 2, and 3. The visibility list of face c includes images 1, 2, and 3.

[0037] In step 102, the interface patch corresponding to any image is determined based on the patch visibility list.

[0038] Here, the interface patch corresponding to any image is the patch at the boundary between the foreground and background regions in the 3D mesh model from the image viewpoint of that image. The boundary position is used to indicate the location where the visibility of the patch in the 3D scene model changes from the corresponding image viewpoint.

[0039] See Figure 4 The diagram illustrates six faces from a 3D scene model: face a, face b, face c, face d, face e, and face f. It also shows image 1 captured from a single viewpoint. It can be seen that rays pointing from the viewpoint to faces a, e, and f are occluded by the foreground region, making faces a, e, and f invisible from the viewpoint of image 1. Rays pointing from the viewpoint to faces b, c, and d are not occluded, making faces b, c, and d visible from the viewpoint of image 1. Furthermore, faces b, c, and d are adjacent to faces a, e, and f, respectively. Therefore, it can be determined that the positions of faces b, c, and d with faces a, e, and f from the viewpoint of image 1 represent the boundary points. Faces a, b, c, d, e, and f are considered boundary faces.

[0040] In this embodiment of the disclosure, the visibility list of each patch and the visibility list of each patch's neighboring patches can be traversed one by one. If an image exists in the visibility list of a patch, but the image is not included in the visibility list of any of the patch's neighboring patches, it can be determined that the patch is located at the boundary between the foreground region and the background region in the image's view.

[0041] In step 103, based on the interface patch corresponding to any image and the modeling error, the texture participation weight of any image is set for the interface patch.

[0042] The modeling error is used to indicate the cumulative error of each stage in the 3D reconstruction process. The error of each stage can be output by the respective stage, such as the errors generated by image acquisition, feature extraction (detection and matching key points), sparse reconstruction, dense reconstruction, and meshing. The modeling error can be an object space error, such as a modeling error of 20cm in the object space, or an image space error, such as a modeling error of 5 pixels.

[0043] In some implementations, the modeling error can be obtained by summing the errors output at each stage of the 3D reconstruction process; in other implementations, the error output at the last stage of the 3D reconstruction process can be determined as the modeling error.

[0044] In this embodiment of the disclosure, after determining the interface patch and the modeling error, the patches within the range indicated by the modeling error near the interface patch can be determined, and the patches visible to the image viewpoint can be selected from the patches within the range indicated by the modeling error near the interface patch. The patches are then assigned a texture participation weight for any image participating in the texture mapping.

[0045] The texture participation weight of any image used in the texture mapping is used to indicate the degree of participation of that image in the texture mapping of the face. The smaller the texture participation weight, the lower the degree of participation of that image in the texture mapping of the face.

[0046] For example, if the visibility list of patch a includes image 1, image 2, and image 3, and after judgment, it is determined that patch a is the interface patch of the image view corresponding to image 3, and the modeling error is 20cm in the object space, then a region growing operation can be performed from the position of patch a towards the foreground region until a range matching the modeling error is obtained. The patches within this range are then set with the texture participation weight of image 3, reducing the participation of image 3 in texture mapping of patch a.

[0047] In step 104, texture processing is performed on the patches in the 3D mesh model based on the visibility list of the patches and the textures corresponding to the images in the visibility list as weights.

[0048] In this embodiment of the disclosure, a pre-configured texture mapping algorithm can be used to perform texture mapping on the patches in the three-dimensional mesh model.

[0049] In some alternative implementations, a selection-based texture mapping algorithm can be employed. This algorithm selects the optimal image from multiple candidate texture sources (multiple images in a visibility list) of a patch to generate a high-quality, consistent texture map. In this embodiment, by setting texture participation weights for images, it helps to reduce the probability of images with low texture participation weights being selected when choosing the optimal image, and may even remove such images from the patch's visibility list.

[0050] In other alternative implementations, a blending-based texture mapping algorithm can also be employed. This algorithm selects at least one (e.g., m) optimal image from multiple (e.g., n) candidate texture sources (multiple images in the visibility list) of a patch to generate a high-quality, consistent texture map. In this embodiment, by setting texture participation weights for images, it helps to reduce the probability of images with low texture participation weights being selected, or even to remove such images from the patch's visibility list, when selecting the optimal image.

[0051] In this embodiment of the disclosure, other mapping algorithms for texture mapping can also be used to process the patches in the 3D mesh model. Combining the image mapping participation weight can ensure that erroneous images will not participate in the mapping generation process.

[0052] Through steps 101 to 104 above, when performing texture mapping, the visibility list of patches in the 3D mesh model is first determined. The visibility list of each patch records the image that the patch is visible to from the image viewpoint. Then, based on the visibility list of the patches, the interface patch corresponding to any image is determined. The interface patch corresponding to any image is the patch at the boundary between the foreground and background regions in the 3D mesh model from the image viewpoint of any image. Then, based on the interface patch corresponding to any image and the modeling error, the texture participation weight of any image is set on the patch. Finally, based on the visibility list of the patches and the texture participation weight of the images in the visibility list, the patches in the 3D mesh model are textured. By setting the texture participation weight of the patches in the region where the interface patch corresponding to any image is located based on the modeling error, the participation of the image in the texture mapping of the interface patch can be reduced, which helps to avoid the foreground image being mapped to the background geometry of the model, which can enhance the visual realism and improve the texture mapping effect.

[0053] Figure 5 This is a flowchart of step 101 in the texture mapping processing method of this disclosure. For example... Figure 5 As shown, in Figure 1 Based on the illustrated embodiment, step 101 may include the following steps 111 to 112. Each step is described below.

[0054] In step 111, it is determined whether the visibility condition is satisfied between the patch and any image.

[0055] The visibility condition is used to indicate the criteria for determining whether a patch is fully visible from any image viewpoint.

[0056] In this embodiment of the disclosure, the vertices of the face can be projected based on the camera extrinsic and intrinsic parameters corresponding to any image to obtain the two-dimensional point coordinates corresponding to the vertices; in response to the two-dimensional point coordinates corresponding to the vertices indicating that the projection point of the vertices is located on any image, it is determined whether there are other faces between any image and the face; in response to the absence of other faces between any image and the face, it is determined that the face and any image satisfy the visibility condition.

[0057] In this embodiment of the disclosure, if the two-dimensional point coordinates of the vertex indicate that the projection points of the vertex are not all located on any image, it can be determined that the visibility condition is not met; or if the two-dimensional point coordinates of the vertex indicate that the projection points of the vertex are all located on any image, but there are other patches between any image and the patch, it can also be determined that the visibility condition is not met.

[0058] In this embodiment of the disclosure, the specific process of determining whether there are other patches between any image and a patch may include: determining the ray formed by the camera position corresponding to any image and the vertex of the patch; and determining that there are no other patches between any image and a patch in response to the fact that there is no intersection between the ray and other patches in the three-dimensional mesh model.

[0059] For example, if the patch is a triangular patch, the three-dimensional coordinates of the three vertices of the patch can be projected onto two-dimensional point coordinates based on the camera's extrinsic and intrinsic parameters. It can then be determined whether these two-dimensional point coordinates fall within the image. If all three vertices fall within the image, it can be further determined that the ray from the vertex to the image's optical center (camera position) does not intersect with other patches of the 3D mesh model. Specifically, it can be calculated whether each patch intersects with a ray, or whether the intersection point with the patch falls within the triangular patch. If it does not intersect with other patches of the 3D mesh model, it means the patch is fully visible from the image's perspective. After determining the visibility list for each patch, the visibility information of the entire 3D mesh model (the visibility list of each patch) is obtained.

[0060] In step 112, in response to the visibility condition being met between the patch and any image, any image is added to the visibility list of the patch to obtain the visibility list of the patch.

[0061] Based on the embodiments of this disclosure, by determining the visibility of each facet and each image in the three-dimensional mesh model one by one, the visibility information of the three-dimensional mesh model can be obtained. Subsequently, the interface facets under different image perspectives can be determined by the visibility list of each facet and the visibility list of adjacent facets.

[0062] Figure 6 This is a flowchart of step 102 in the texture mapping method of this disclosure. For example... Figure 6 As shown, in Figure 1Based on the illustrated embodiment, step 102 may include the following steps 121 to 122. Each step is described below.

[0063] In step 121, for any patch, the image in the visibility list of the patch is determined to be different from the image in the visibility list of the patch's neighboring patches.

[0064] The image in the visibility list of a patch and the image in the visibility list of its neighboring patches are used to indicate an image that exists in the visibility list of a patch but does not exist in the visibility list of its neighboring patches; or an image that exists in the visibility list of its neighboring patches but does not exist in the visibility list of the patch itself.

[0065] In step 122, adjacent patches of a patch are determined to be interface patches corresponding to different images, thus obtaining the interface patch corresponding to any image.

[0066] For example, see Figure 4 The diagram illustrates that from the perspective of image 1, patches a, e, and f are invisible, while patches b, c, and d are visible. Therefore, the visibility lists of patches b, c, and d contain image 1, but the visibility lists of patches a, e, and f do not contain image 1. Furthermore, patches b, c, and d are adjacent to patches a, e, and f, respectively. Thus, it can be determined that patches b, c, d, a, e, and f are the interfaces of image 1.

[0067] Based on the embodiments of this disclosure, by traversing the visibility list of the facet, the interface facet at the boundary position of the foreground region and the background region under different image viewpoints can be determined. In other words, the interface facet at the boundary position where the visibility deteriorates under different image viewpoints can be determined. This helps to reduce the participation of the image with the facet as the interface facet when performing texture mapping processing on the facet, thereby improving the texture mapping effect.

[0068] Figure 7 This is a flowchart of step 103 in the texture mapping processing method of this disclosure. For example... Figure 7 As shown, in Figure 1 Based on the illustrated embodiment, step 103 may include the following steps 131 to 132. Each step is described below.

[0069] In step 131, the facets whose distance from the interface facets is within the modeling error range are identified as the target facets whose texture weights need to be adjusted.

[0070] The target facets whose texture weights need to be adjusted are used to indicate faces within a certain error range (modeling error range) at the boundary. When determining the distance to the boundary facets, the distance between facets can be determined by calculating the distance between their center points.

[0071] For example, if the modeling error is 20cm in the object space, the patches within a 20cm radius of the boundary can be taken as target patches. First, the center point coordinates of the boundary patch can be determined, and then the center point coordinates of the adjacent patches can be determined. The distance between the patches can be determined based on the center point coordinates of the boundary patch and the center point coordinates of the adjacent patches. If the distance is less than 20cm, the distance between the boundary patch and the adjacent patches can be calculated again by region growing, until the location of the patch with a distance greater than the modeling error is determined.

[0072] Understandably, since the image's facets are not included in the visibility list, the image itself is not involved in the texture mapping process for those facets. Therefore, when determining the target facets based on the region growing method, growth can be directed towards the foreground region rather than the background region. For example, Figure 4 It can grow in the direction of facets b, c, and d, but not in the direction of facets a, e, and f, which can reduce the amount of computation required to determine the target facet.

[0073] The modeling error is used to indicate the cumulative error of each stage in the 3D reconstruction process. The error of each stage can be output by that stage, such as the errors generated by image acquisition, feature extraction (detection and matching key points), sparse reconstruction, dense reconstruction, and meshing. The modeling error can be object space error or image space error, such as a modeling error of 5 pixels.

[0074] In step 132, based on the distance between the target patch and the interface patch, the texture participation weight for any image participating in the texture mapping is set.

[0075] The texture participation weight of any image used in the texture mapping is used to indicate the degree of participation of that image in the texture mapping of the face. The smaller the texture participation weight, the lower the degree of participation of that image in the texture mapping of the face.

[0076] Based on the embodiments of this disclosure, the target patch is determined by region growing based on the boundary position, which helps to determine the real object boundary of the 3D reconstruction model that can be better distinguished and reduce boundary blur; moreover, the implementation process is relatively simple and highly flexible.

[0077] Among some optional implementation methods, refer to Figure 8The left image shows the texture effect before adopting the technical solution of this disclosure. The positions indicated by labels 81, 82 and 83 show the texture effect caused by the misalignment between the texture and the geometric position. The right image shows the texture effect after adopting the technical solution of this disclosure. It can be seen that after adopting the texture processing method of this disclosure, the texture is completely aligned with the geometric position after texturing the 3D mesh model in the 3D scene, which improves the visual consistency.

[0078] Corresponding to the embodiments of the aforementioned texture mapping processing method, this disclosure also provides embodiments of the texture mapping processing apparatus.

[0079] Exemplary device Figure 9 This is a schematic diagram illustrating one embodiment of the texture mapping processing apparatus of this disclosure. The apparatus is used in electronic devices (such as computer systems or servers). Figure 9 As shown, the texture mapping processing apparatus may include: The list determination module 91 is used to determine the visibility list of patches in the 3D mesh model. The visibility list of patches records the images that the patches are visible from the image viewpoint. Interface Patch Determination Module 92 is used to determine the interface patch corresponding to any image based on the patch visibility list. The interface patch corresponding to any image is the patch at the boundary between the foreground region and the background region in the 3D mesh model under the image view of any image. The weight setting module 93 is used to set the texture participation weight of any image on the interface patch based on the interface patch corresponding to any image and the modeling error. The texture module 94 is used to perform texture processing on the faces in the 3D mesh model based on the visibility list of the face and the texture corresponding to the image in the visibility list.

[0080] Figure 10 This is a schematic diagram of another embodiment of the texture mapping processing apparatus of this disclosure, as shown below. Figure 10 As shown, in Figure 9 Based on the illustrated embodiment, in some optional implementations, the list determination module 91 may include: The first determining submodule 911 is used to determine whether the visibility condition is satisfied between the patch and any image. The visibility condition is used to indicate the judgment condition that the patch is completely visible to the image view of any image. The processing submodule 912 is used to add any image to the visibility list of the face in response to the face and any image satisfying the visibility condition, thereby obtaining the visibility list of the face.

[0081] In some optional implementations, the first determining submodule 911 can be used to project the vertices of the face based on the camera extrinsic and intrinsic parameters corresponding to any image to obtain the two-dimensional point coordinates corresponding to the vertices; in response to the two-dimensional point coordinates corresponding to the vertices indicating that the projected point of the vertex is located on any image, determine whether there are other faces between any image and the face; in response to the absence of other faces between any image and the face, determine that the face and any image satisfy the visibility condition.

[0082] In some alternative implementations, the first determining submodule 911 may be used to determine the ray formed by the camera position corresponding to any image and the vertex of the patch; in response to the fact that there is no intersection between the ray and other patches in the three-dimensional mesh model, it is determined that there are no other patches between any image and the patch.

[0083] In some alternative implementations, the interface patch determination module 92 may include: The second determining submodule 921 is used to determine, for any patch, different images in the visibility list of the patch from those in the visibility list of the adjacent patches. The third determining submodule 922 is used to determine that adjacent patches of a patch are interface patches corresponding to different images, thereby obtaining the interface patch corresponding to any image.

[0084] In some alternative implementations, the weight setting module 93 may include: The fourth determination submodule 931 is used to determine the facets whose distance from the interface facet is within the modeling error range as the target facets whose texture weights need to be adjusted. Setting submodule 932 is used to set the texture participation weights for any image participating in the texture mapping based on the distance between the target patch and the interface patch.

[0085] The modules and units in this disclosed device can be further divided into finer-grained units according to actual needs, and the specific configuration can be set according to actual needs.

[0086] The apparatus of this disclosure embodiment can be used to implement the methods of the above embodiments of this disclosure. The two correspond to each other in specific implementation, and the specific implementation of related parts can be referred to each other, which will not be repeated here.

[0087] Exemplary systems, electronic devices, computer program products, and computer-readable storage media This disclosure also provides an electronic device, including: a memory for storing a computer program; and a processor for executing the computer program stored in the memory, wherein when the computer program is executed, it implements the texture mapping processing method of any of the above embodiments of this disclosure.

[0088] Below, for reference Figure 11This describes an electronic device according to embodiments of the present disclosure, wherein apparatus for implementing methods according to embodiments of the present disclosure may be integrated. Figure 11 This is a structural diagram of an electronic device provided in an illustrative embodiment of the present disclosure, such as... Figure 11 As shown, the electronic device includes one or more processors 11, one or more memory 12 of computer-readable storage media, and a computer program stored in the memory and executable on the processor. When the program in the memory 12 is executed, the texture mapping processing method described above can be implemented.

[0089] Specifically, in practical applications, the electronic device may also include components such as an input device 13 and an output device 14, which are interconnected via a bus system and / or other forms of connection mechanisms (not shown). Those skilled in the art will understand that... Figure 11 The structure of the electronic device shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or certain components, or different component arrangements. Wherein: The processor 11 may be a central processing unit (CPU) or other form of processing unit with texture mapping processing capabilities and / or instruction execution capabilities. It performs various functions and processes data by running or executing software programs and / or modules stored in the memory 12 and calling data stored in the memory 12, thereby providing overall monitoring of the electronic device.

[0090] The memory 12 can store one or more computer program products. The memory can include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program products can be stored on the computer-readable storage medium, and the processor 11 can run the computer program products to implement the texture mapping processing methods of the various embodiments of this disclosure described above and / or other desired functions.

[0091] The input device 13 can be used to receive input numerical or character information. The input device 13 may include a keyboard, mouse, joystick, etc., related to user settings and function control.

[0092] The output device 14 can output various information to the outside, including determined distance information, direction information, etc. The output device 14 may include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0093] The electronic device may also include a power supply for powering various components, which can be logically connected to the processor 11 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and any other components.

[0094] Of course, for the sake of simplicity, Figure 11 Only some of the components of the electronic device relevant to this disclosure are shown, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device may include any other suitable components depending on the specific application.

[0095] In addition to the methods and apparatus described above, embodiments of this disclosure may also be computer program products, including computer program instructions that, when executed by a processor, cause the processor to perform the steps in the texture mapping processing methods according to various embodiments of this disclosure as described in the "Exemplary Methods" section of this specification.

[0096] Computer program products can be written in any combination of one or more programming languages ​​to perform the operations of embodiments of this disclosure. The programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on a user's computing device, partially on a user's computing device, as a standalone software package, partially on a user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0097] Furthermore, embodiments of this disclosure may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps in the texture mapping processing methods according to various embodiments of this disclosure as described in the "Exemplary Methods" section above.

[0098] Computer-readable storage media may take the form of any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0099] The basic principles of this disclosure have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, effects, etc. mentioned in this disclosure are merely examples and not limitations. These advantages, benefits, effects, etc. should not be considered as essential features of each embodiment of this disclosure.

[0100] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For system embodiments, since they largely correspond to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0101] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as ROM, RAM, magnetic disk, or optical disk.

[0102] The methods and apparatus of this disclosure may be implemented in many ways. For example, they may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above-described order of steps for the method is for illustrative purposes only, and the steps of the method of this disclosure are not limited to the order specifically described above, unless otherwise specifically stated. Furthermore, in some embodiments, this disclosure may also be implemented as a program recorded on a recording medium, the program including machine-readable instructions for implementing the method according to this disclosure. Thus, this disclosure also covers recording media storing programs for performing the method according to this disclosure.

[0103] The description in this disclosure is provided for illustrative and descriptive purposes only and is not intended to be exhaustive or to limit the disclosure to its forms. Many modifications and variations will be apparent to those skilled in the art. The embodiments were chosen and described in order to better illustrate the principles and practical application of this disclosure and to enable those skilled in the art to understand this disclosure and to design various embodiments with various modifications suitable for a particular purpose.

Claims

1. A method for processing texture mapping, characterized in that, include: A visibility list of patches in a 3D mesh model is determined, wherein the visibility list of patches records the images that the patches are visible from the image viewpoint; Based on the visibility list of the patches, the interface patch corresponding to any image is determined. The interface patch corresponding to any image is the patch at the boundary between the foreground region and the background region in the three-dimensional mesh model under the image view of any image. Based on the interface patch corresponding to any one of the images and the modeling error, the patch is set with the texture participation weight of any one of the images participating in the texture mapping. Based on the visibility list of the facets and the texture participation weights corresponding to the images in the visibility list, texture processing is performed on the facets in the 3D mesh model.

2. The method according to claim 1, characterized in that, The list of visible patches in the 3D mesh model includes: Determine whether the visibility condition is satisfied between the patch and any of the images, wherein the visibility condition is used to indicate the judgment condition that the patch is fully visible to the image view of any of the images; In response to the fact that the visibility condition between the patch and any of the images is met, the any of the images is added to the visibility list of the patch, thereby obtaining the visibility list of the patch.

3. The method according to claim 2, characterized in that, Determining whether the visibility condition is satisfied between the patch and any of the images includes: Based on the camera extrinsic and intrinsic parameters corresponding to any of the images, the vertices of the patch are projected to obtain the two-dimensional point coordinates corresponding to the vertices; In response to the two-dimensional point coordinates corresponding to the vertex indicating that the projection point of the vertex is located on any image, it is determined whether there are other patches between any image and the patch; In response to the absence of other patches between any image and the patch, it is determined that the patch and any image satisfy the visibility condition.

4. The method according to claim 3, characterized in that, Determining whether there are other patches between any image and the patch includes: Determine the ray formed by the camera position corresponding to any image and the vertex of the patch; In response to the absence of intersections between the ray and other patches in the 3D mesh model, it is determined that there are no other patches between any image and the patch.

5. The method according to any one of claims 1-4, characterized in that, The determination of the interface patch corresponding to any image based on the visibility list of the patches includes: For any of the aforementioned facets, determine the different images in the visibility list of the facet compared to the visibility lists of the adjacent facets of the facet; The interface patches corresponding to different images are determined by identifying the adjacent patches of the patch. Thus, the interface patches corresponding to any image are obtained.

6. The method according to any one of claims 1-5, characterized in that, The step of setting texture participation weights for the interface patches corresponding to any one of the images and the modeling error, with the textures participating in the texture mapping based on any one of the images, includes: The facets whose distance from the interface facets is within the modeling error range are identified as the target facets whose texture weights need to be adjusted. Based on the distance between the target patch and the interface patch, a texture participation weight is set for each of the images participating in the texture mapping.

7. A texture mapping processing apparatus, characterized in that, include: The list determination module is used to determine the visibility list of patches in the 3D mesh model. The visibility list of patches records the images of the patches that are visible from the image viewpoint. The interface patch determination module is used to determine the interface patch corresponding to any image based on the visibility list of the patches. The interface patch corresponding to any image is the patch at the boundary between the foreground region and the background region in the three-dimensional mesh model under the image view of the image of the image. The weight setting module is used to set the texture participation weight of the texture with the image as the texture based on the interface patch corresponding to any image and the modeling error. The texture mapping module is used to perform texture mapping on the faces in the 3D mesh model based on the visibility list of the faces and the texture participation weights corresponding to the images in the visibility list.

8. The apparatus according to claim 7, characterized in that, The list determination module includes: The first determining submodule is used to determine whether the visibility condition is satisfied between the patch and any image, wherein the visibility condition is used to indicate the judgment condition that the patch is fully visible to the image view of any image; A processing submodule is configured to add any image to the visibility list of the patch in response to the visibility condition being met between the patch and any image, thereby obtaining the visibility list of the patch.

9. A computer-readable storage medium, characterized in that, The storage medium stores computer program instructions, which, when executed, implement the method described in any one of claims 1-6.

10. An electronic device, characterized in that, The electronic device includes: Memory, used to store computer program products; A processor is configured to execute a computer program product stored in the memory, wherein, when the computer program product is executed, it implements the method described in any one of claims 1-6.

Images