A multi-view texture synthesis method, device, electronic equipment and storage medium

Abstract

The present disclosure provides a multi-view texture synthesis method and device, electronic equipment and storage medium. The vertex view weight is constructed by comprehensively constructing the Euclidean distance and the view angle at the vertex level, and the vertex weight is interpolated into the view mixing weight at the sampling point level by using the barycentric coordinates of the texture sampling point relative to the target triangular patch in the texture parameter domain. Then, the weighted fusion is realized by combining the visibility suppression, weight normalization and multi-view projection interpolation sampling, so that the fusion weight continuously changes in the patch, reduces the view jump and seam artifacts of adjacent sampling points, improves the continuity and consistency of texture synthesis, and suppresses the invalid view interference under the conditions of occlusion or unfavorable view angle. Thus, a more stable and natural final texture map is obtained, and the texture block is packaged and exported.

Description

Technical Field

[0001] This disclosure relates to the field of three-dimensional reconstruction technology, and more specifically, to a multi-view texture synthesis method, apparatus, electronic device, and storage medium. Background Technology

[0002] In applications such as 3D reconstruction, digital twins, digitization of cultural heritage, and virtual reality / augmented reality, it is often necessary to generate high-quality textures for 3D mesh models to give the model surface a realistic, continuous, and richly detailed appearance. Existing multi-view texture synthesis typically uses multiple images with known camera poses as texture sources, projects points on the model surface onto several views for sampling, and selects or blends colors among the multiple views using certain strategies to form the final texture map.

[0003] However, in practical engineering, the view selection or weight allocation in multi-view texture synthesis is unstable. Some solutions select views for patches or points based on only a single factor (such as distance, viewing angle, or sharpness), which easily leads to frequent switching of the source view for adjacent pixels, resulting in obvious seams and color jumps within or between texture blocks, affecting the overall continuity. If the weight is calculated only for the entire patch or the center point of the patch, it is difficult to provide continuously changing fusion weights for each texture sampling point within the texture parameter domain, especially at complex curved surfaces, slender triangles, or view coverage boundaries, which can easily produce artifacts such as local blurring, jagged edges, or local streaking. Summary of the Invention

[0004] This disclosure provides at least one multi-view texture synthesis method, apparatus, electronic device, and storage medium. It constructs vertex view weights by integrating Euclidean distance and the viewing angle at the vertex level, and interpolates the vertex weights into sample point-level view blending weights in the texture parameter domain using the centroid coordinates of the texture sampling points relative to the target triangle facet. Then, it combines visibility suppression, weight normalization, and multi-view projection interpolation sampling to achieve weighted fusion. This allows the fusion weights to continuously change within the facet, reducing view jumps and stitching artifacts between adjacent sampling points, improving the continuity and consistency of texture synthesis. Simultaneously, it suppresses invalid view interference under occlusion or unfavorable viewing conditions, thereby obtaining a more stable and natural final texture map and facilitating the packaging and export of texture blocks.

[0005] This disclosure provides a multi-view texture synthesis method, including:

[0006] Obtain the 3D mesh model to be textured, the multi-view image set, and the camera parameters corresponding to the multi-view image set. Based on the 3D mesh model and the camera parameters, determine the vertex view weights of the mesh vertices of the 3D mesh model in each view.

[0007] Construct the texture parameter domain of the three-dimensional mesh model, and determine the set of texture sampling points to be synthesized in the texture parameter domain;

[0008] For any texture sampling point, determine the target triangle facet corresponding to the texture sampling point, and calculate the centroid coordinates of the texture sampling point within the target triangle facet. Based on the centroid coordinates, interpolate the vertex view weights of the three vertices of the target triangle facet to obtain the view blending weight of the texture sampling point.

[0009] Based on the view blending weights, the texture sampling points are mapped to at least one view using the camera parameters and color sampling is performed. The sampled colors are then weighted and fused to obtain the output color of the texture sampling points.

[0010] A texture map is generated based on the output color of each texture sampling point in the texture parameter domain, and the texture map is exported.

[0011] In one optional implementation, based on the 3D mesh model and the camera parameters, the vertex view weights of the mesh vertices of the 3D mesh model in each view are determined, specifically including:

[0012] For any grid vertex and any viewpoint, calculate the Euclidean distance between the grid vertex and the viewpoint, as well as the angle between their lines of sight.

[0013] The unnormalized vertex view weights of the mesh vertices in the view are constructed based on the Euclidean distance and the angle between the lines of sight.

[0014] The unnormalized vertex view weights are determined by the distance term and the included angle term, and adjustable parameters are introduced to adjust the contribution ratio of the distance term and the included angle term to the unnormalized vertex view weights respectively.

[0015] In one optional implementation, based on the 3D mesh model and the camera parameters, the vertex view weights of the mesh vertices of the 3D mesh model in each view are determined, specifically including:

[0016] When the mesh vertex is not visible in the target view or is outside the preset viewing range, the unnormalized vertex view weight corresponding to the target view is set to 0.

[0017] The unnormalized vertex view weights of the same mesh vertex in multiple views are normalized to obtain the vertex view weights of the mesh vertex in each view.

[0018] In one optional implementation, a texture parameter domain of the three-dimensional mesh model is constructed, and a set of texture sampling points to be synthesized is determined in the texture parameter domain, specifically including:

[0019] The three-dimensional mesh model is textured to obtain a UV atlas, and the UV atlas is divided into multiple texture blocks, each texture block consisting of a set of adjacent triangular faces;

[0020] The bounding box size of each texture block determines the texture map range corresponding to that texture block, and the texture resolution of that texture block is also determined.

[0021] For each triangular facet within each texture block, a correspondence is established between the UV vertex coordinates of the triangular facet in the UV atlas and the geometric vertex coordinates of the triangular facet to ensure texture mapping consistency.

[0022] In one optional implementation, for any texture sampling point, a target triangular facet corresponding to the texture sampling point is determined, and the centroid coordinates of the texture sampling point within the target triangular facet are calculated, specifically including:

[0023] For each texture block, perform pixel traversal on the texture map to obtain the current texture sampling point;

[0024] For the current texture sampling point, calculate the centroid coordinates of the current texture sampling point based on the UV vertex coordinates of the target triangle facet;

[0025] When the centroid coordinates meet the preset internal determination conditions, the current texture sampling point is determined to belong to the target triangle facet.

[0026] In one optional implementation, the vertex view weights of the three vertices of the target triangular facet are interpolated based on the centroid coordinates to obtain the view blending weights of the texture sampling points, specifically including:

[0027] Using the centroid coordinates as interpolation coefficients, the vertex view weights of the three vertices of the target triangle facet in each view are linearly combined to obtain the view blending weights of the texture sampling points for each view, and the view blending weights corresponding to the invisible views are set to 0.

[0028] In one optional implementation, based on the view blending weights, the texture sampling points are mapped to at least one view using the camera parameters and color sampling is performed. The sampled colors are then weighted and fused to obtain the output color of the texture sampling points. Specifically, this includes:

[0029] Normalize the view blending weights;

[0030] The texture sampling points are projected onto the image plane positions of each visible view using the camera parameters;

[0031] Interpolation sampling is used to obtain the sampled color for the image plane position of each visible view;

[0032] The sampled colors are weighted and fused according to the normalized view mixing weights to obtain the output color.

[0033] This disclosure also provides a multi-view texture synthesis apparatus, including:

[0034] The data acquisition module is used to acquire the 3D mesh model to be textured, the multi-view image set, and the camera parameters corresponding to the multi-view image set, and to determine the vertex view weights of the mesh vertices of the 3D mesh model in each view based on the 3D mesh model and the camera parameters.

[0035] The sampling point determination module is used to construct the texture parameter domain of the three-dimensional mesh model and determine the set of texture sampling points to be synthesized in the texture parameter domain;

[0036] The hybrid weight determination module is used to determine the target triangle facet corresponding to any texture sampling point, calculate the centroid coordinates of the texture sampling point in the target triangle facet, and interpolate the vertex view weights of the three vertices of the target triangle facet based on the centroid coordinates to obtain the view hybrid weight of the texture sampling point.

[0037] The output color determination module is used to map the texture sampling points to at least one view based on the view blending weight and the camera parameters, and perform color sampling, and perform weighted fusion of the sampled colors to obtain the output color of the texture sampling points.

[0038] The texture map output module is used to generate a texture map based on the output color of each texture sampling point in the texture parameter domain, and to export the texture map.

[0039] This disclosure also provides an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, the steps of the above-described multi-view texture synthesis method, or any possible implementation of the above-described multi-view texture synthesis method, are performed.

[0040] This disclosure also provides a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of the above-described multi-view texture synthesis method or any possible implementation thereof.

[0041] This disclosure also provides a computer program product, including a computer program / instructions, which, when executed by a processor, implement the steps of the above-described multi-view texture synthesis method, or any possible implementation of the above-described multi-view texture synthesis method.

[0042] This disclosure provides a multi-view texture synthesis method, apparatus, electronic device, and storage medium. By constructing vertex view weights at the vertex level by integrating Euclidean distance and the viewing angle, and interpolating the vertex weights into sample point-level view mixing weights in the texture parameter domain using the centroid coordinates of the texture sampling points relative to the target triangle facet, and then combining visibility suppression, weight normalization, and multi-view projection interpolation sampling to achieve weighted fusion, the fusion weights change continuously within the facet, reducing view jumps and stitching artifacts between adjacent sampling points, improving the continuity and consistency of texture synthesis, and suppressing invalid view interference under occlusion or unfavorable viewing conditions, thereby obtaining a more stable and natural final texture map and facilitating the packaging and export of texture blocks.

[0043] To make the above-mentioned objects, features and advantages of this disclosure more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0044] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings used in the embodiments will be briefly described below. These drawings are incorporated in and constitute a part of this specification. They illustrate embodiments conforming to this disclosure and, together with the specification, serve to explain the technical solutions of this disclosure. It should be understood that the following drawings only show some embodiments of this disclosure and should not be considered as limiting the scope. Those skilled in the art can obtain other related drawings based on these drawings without creative effort.

[0045] Figure 1 A flowchart of a multi-view texture synthesis method provided by an embodiment of this disclosure is shown;

[0046] Figure 2 A flowchart of another multi-view texture synthesis method provided by an embodiment of this disclosure is shown;

[0047] Figure 3 A schematic diagram of a multi-view texture synthesis apparatus provided in an embodiment of this disclosure is shown;

[0048] Figure 4 A schematic diagram of an electronic device provided in an embodiment of this disclosure is shown. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. The components of the embodiments of this disclosure described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this disclosure provided in the accompanying drawings is not intended to limit the scope of the claimed disclosure, but merely represents selected embodiments of this disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without inventive effort are within the scope of protection of this disclosure.

[0050] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0051] In this document, the term "and / or" merely describes a relationship, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Furthermore, the term "at least one" in this document means any combination of at least two of any one or more elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0052] Research has revealed that in practical engineering, the view selection or weight allocation in multi-view texture synthesis is unstable. Some solutions select views for patches or points based on only a single factor (such as distance, viewing angle, or sharpness), which easily leads to frequent switching of source views for adjacent pixels, resulting in obvious seams and color jumps within or between texture blocks, affecting overall continuity. If weights are calculated only for the entire patch or its center point, it is difficult to provide continuously changing fusion weights for each texture sampling point within the texture parameter domain, especially at complex curved surfaces, slender triangles, or view coverage boundaries, easily producing artifacts such as local blurring, jagged edges, or local streaking.

[0053] Based on the above research, this disclosure provides a multi-view texture synthesis method, apparatus, electronic device, and storage medium. By constructing vertex view weights by integrating Euclidean distance and viewing angle at the vertex level, and interpolating vertex weights into view mixing weights at the sampling point level in the texture parameter domain using the centroid coordinates of texture sampling points relative to the target triangle facet, and then combining visibility suppression, weight normalization, and multi-view projection interpolation sampling to achieve weighted fusion, thereby enabling the fusion weights to change continuously within the facet, reducing view jumps and stitching artifacts between adjacent sampling points, improving the continuity and consistency of texture synthesis, and suppressing invalid view interference under occlusion or unfavorable viewing conditions, thus obtaining a more stable and natural final texture map and facilitating the packaging and export of texture blocks.

[0054] To facilitate understanding of this embodiment, a multi-view texture synthesis method disclosed in this disclosure will first be described in detail. The execution entity of the multi-view texture synthesis method provided in this disclosure is generally a computer device with certain computing capabilities. This computer device may include, for example, a terminal device, a server, or other processing devices. The terminal device may be a user equipment (UE), mobile device, user terminal, terminal, cellular phone, cordless phone, personal digital assistant (PDA), handheld device, computing device, in-vehicle device, wearable device, etc. In some possible implementations, this multi-view texture synthesis method can be implemented by a processor calling computer-readable instructions stored in memory.

[0055] See Figure 1 The diagram shows a flowchart of a multi-view texture synthesis method provided in an embodiment of this disclosure. The method includes steps S101 to S105, wherein:

[0056] S101. Obtain the 3D mesh model to be textured, the multi-view image set, and the camera parameters corresponding to the multi-view image set. Based on the 3D mesh model and the camera parameters, determine the vertex view weights of the mesh vertices of the 3D mesh model in each view.

[0057] In its specific implementation, the multi-view texture synthesis method of the present invention first acquires a three-dimensional mesh model to be textured, a multi-view image set, and camera parameters corresponding to the multi-view image set. The three-dimensional mesh model is used to characterize the geometry of the object to be textured; it can be in the form of a triangular mesh, including a vertex set and a triangular facet set. Each vertex in the vertex set contains three-dimensional coordinate information, and each triangular facet is determined by three vertex indices. The multi-view image set includes multiple images acquired or rendered from different viewpoints, each image corresponding to a view. The camera parameters are used to describe the imaging geometry of each view, including camera intrinsic and camera extrinsic parameters. The camera intrinsic parameters describe imaging parameters such as focal length, principal point position, and pixel scale, while the camera extrinsic parameters describe the pose transformation relationship of the camera coordinate system relative to the world coordinate system / model coordinate system.

[0058] In a specific implementation, the three-dimensional mesh model can be obtained by the output of a three-dimensional reconstruction system, such as a mesh model generated by structured light, binocular / multi-view vision reconstruction or SLAM reconstruction; the multi-view image set can be real images captured or synthetic images rendered; the camera parameters can be obtained by calibration or output simultaneously by the reconstruction process, and correspond one-to-one with each image in the multi-view image set.

[0059] It should be noted that, in order to ensure the geometric consistency of subsequent projection and sampling, the same coordinate reference is used to describe the three-dimensional mesh model and camera extrinsic parameters in this embodiment. For example, the mesh vertex coordinates are represented in the world coordinate system, and the camera extrinsic parameters of each view are represented as rigid body transformations from the world coordinate system to the camera coordinate system.

[0060] Here, after completing the above data acquisition, this embodiment further determines the vertex view weights of the mesh vertices of the 3D mesh model in each view based on the 3D mesh model and the camera parameters. The vertex view weights are used to characterize the availability or reliability of a certain mesh vertex in a certain view. They can reflect the degree to which the view adapts to the texture information provided by the surface area where the vertex is located, so that the weights at the vertex level can be continuously propagated to the texture sampling point level through centroid interpolation, thereby stably completing multi-view color fusion.

[0061] In one embodiment, the determination of vertex view weights may at least consider the spatial distance between the grid vertex and the viewpoint, as well as the angle between the vertex normal and the line of sight.

[0062] Specifically, for any mesh vertex, the camera center position corresponding to that view is first determined based on the camera extrinsic parameters, and the Euclidean distance between the mesh vertex and the camera center is calculated. Simultaneously, the viewing direction vector from the mesh vertex to the camera center is determined, and the viewing angle is calculated by combining this vector with the surface normal vector of the mesh vertex. The surface normal vector can be obtained by a weighted average of the normal vectors of adjacent faces, or it can be pre-defined in the mesh model. By comprehensively evaluating distance and angle, views that are closer and have a more direct viewpoint (i.e., a smaller angle and better observability) can receive higher weights, thereby improving the clarity and consistency of subsequent texture fusion.

[0063] In one embodiment, unnormalized vertex view weights of mesh vertices in the view can be constructed based on the Euclidean distance and the angle between the lines of sight. The unnormalized vertex view weights are determined by the distance term and the angle term, and adjustable parameters are introduced to adjust the contribution ratio of the distance term and the angle term to the unnormalized vertex view weights, respectively.

[0064] For example, the distance term can decrease monotonically as the distance increases, and the included angle term can decrease monotonically as the included angle increases, so that the view at close range and from the front angle contributes more to the vertex; the adjustable parameters can be used to adapt to factors such as image resolution, viewing distance range or shooting baseline differences under different acquisition scenarios, so as to make a trade-off between a strategy that is more biased towards distance priority or a strategy that is more biased towards viewpoint priority.

[0065] In a further implementation, to avoid invalid views interfering with weight allocation, this embodiment can also combine visibility constraints to suppress the weights of unnormalized vertex views.

[0066] Specifically, when a mesh vertex is not visible in the target view, the unnormalized vertex view weight corresponding to the target view is set to 0; where invisible may include, but is not limited to, situations where the mesh vertex projection falls outside the image range, the vertex is behind the camera, or the vertex is occluded by the model itself. As one possible approach, the occlusion relationship can be determined by using a depth map obtained through depth buffering / rasterization or by using ray detection to determine whether a vertex is occluded.

[0067] As another possible approach, the back view point can be determined based on the viewing angle threshold. That is, when the angle is greater than the preset upper limit of the viewing angle, the view weight is reset to zero to reduce the risk of stretching and blurring caused by a large oblique viewing angle.

[0068] Furthermore, after calculating and suppressing the unnormalized vertex view weights, this embodiment normalizes the unnormalized vertex view weights of the same mesh vertex in multiple views to obtain the vertex view weights of the mesh vertex in each view. The normalization process is used to map the weights of the same vertex in different views to comparable relative proportions, and to ensure that subsequent fusion satisfies stable weighted summation constraints.

[0069] For example, the unnormalized weights of the vertex in each view can be summed as a normalization factor, and the unnormalized weights of each view can be divided by the normalization factor. When the normalization factor is 0 (e.g., the vertex is not visible or is suppressed in all views), the vertex view weights of the vertex can be set to 0, or a preset fallback strategy can be adopted (e.g., selecting the nearest view, or assigning a preset color to the corresponding texture sampling point in a later stage) to ensure the robustness of the process.

[0070] Thus, through the above processing, this embodiment obtains a vertex-view weight matrix / weight set for each mesh vertex and each view at the vertex level. This vertex-view weight serves as the basic input for subsequent centroid interpolation and texture sampling fusion, enabling view contributions to naturally transition from discrete vertices to the interior of the facet, reducing seams and color jumps caused by frequent local view switching, and providing support for generating continuous and stable fusion weights per sampling point within the texture parameter domain.

[0071] S102. Construct the texture parameter domain of the three-dimensional mesh model, and determine the set of texture sampling points to be synthesized in the texture parameter domain.

[0072] In practical implementation, after obtaining the 3D mesh model and the vertex view weights of its vertices in each view, this invention further constructs a texture parameter domain for the 3D mesh model and determines the set of texture sampling points to be synthesized within this domain. The texture parameter domain is used to map the surface of the 3D mesh model to a 2D texture space, so as to generate a texture map in the 2D space and ultimately paste it back onto the surface of the 3D mesh model. The texture parameter domain can be represented using UV parameterization, which is used to establish a one-to-one correspondence between mesh patches in the 3D geometric space and the 2D texture space.

[0073] In a specific implementation, constructing the texture parameter domain may include performing texture unwrapping processing on the 3D mesh model to obtain a UV atlas. The texture unwrapping can be implemented using any implementable mesh parameterization algorithm, such as kerf-based parameterization, angle-preserving or area-preserving parameterization, etc.

[0074] It should be noted that this embodiment does not limit the specific algorithm type, as long as it can generate corresponding UV vertex coordinates for each triangular facet in the triangular mesh, so that the triangular facet can be represented as a UV triangle on the two-dimensional UV plane and a mapping relationship can be established with its triangular facet in the three-dimensional geometric space.

[0075] In one embodiment, to facilitate texture generation and subsequent packaging, the present invention divides the UV atlas into multiple texture blocks, each texture block consisting of a set of adjacent or connected triangular faces in UV space. A texture block can be understood as an island or a further segmentation unit within the UV atlas. By processing the UV atlas in blocks, the computational and storage pressure of single texture generation can be reduced, and parallel texture generation and subsequent independent packaging and stitching of texture blocks can be facilitated, thereby improving overall processing efficiency and the controllability of the output texture.

[0076] In one embodiment, for each texture block, its bounding box in UV space can be calculated based on the coordinates of all UV vertices contained in the texture block, and the bounding box is used to define the range of the texture map corresponding to the texture block.

[0077] Furthermore, the texture resolution of the texture block can be determined based on the bounding box size. The texture resolution can be determined by a preset texture pixel density and bounding box size, or by preset constraints such as maximum texture size and minimum texture size, so that the texture block has a reasonable sampling density at different scales while taking into account both sharpness and resource consumption.

[0078] In a further implementation, to ensure the consistency of the mapping between the texture parameter domain and the geometric domain, this embodiment establishes a correspondence between the UV vertex coordinates of the triangular facet in the UV atlas and the geometric vertex coordinates of the triangular facet for each triangular facet within each texture block.

[0079] Specifically, for any triangular facet, it is determined by three mesh vertices in geometric space and by three corresponding UV vertices in UV space. By recording the correspondence between geometric vertex indices and UV vertex coordinates, it is possible to ensure that the selected geometric facets and UV facets are strictly consistent when performing facet assignment determination, centroid coordinate calculation, and color projection sampling on texture sampling points in the future, thus avoiding problems such as texture misalignment, mapping flipping, or boundary cracks.

[0080] After constructing the texture parameter domain, this invention determines the set of texture sampling points to be synthesized within the texture parameter domain. The set of texture sampling points can be understood as the set of texture space sampling positions for generating texture colors, which can correspond one-to-one with the pixels of the texture map.

[0081] As one implementation method, this embodiment uses the texture map corresponding to each texture block as the sampling carrier, defines sampling points on the pixel grid of the texture map, that is, the center point of each pixel in the texture map corresponds to a texture sampling point, and converts the position of the texture sampling point in the texture map coordinate system into UV space coordinates to form the texture sampling point set.

[0082] In another alternative implementation, to improve sampling stability and reduce jagged edges, a multi-sampling strategy can be introduced based on pixel center sampling. For example, multiple sub-sampling points can be selected within each pixel and their output colors aggregated, or a denser distribution of sampling points can be used in the texture block boundary region to enhance the smoothness of edge transitions. This invention does not limit this approach; the definition of the sampling point set can be flexibly set according to actual engineering accuracy and performance requirements.

[0083] Thus, through the above steps, this embodiment completes the construction of the texture parameter domain and the determination of the set of texture sampling points to be synthesized, enabling subsequent steps to determine the corresponding geometric triangle facet in UV space for each texture sampling point and calculate the centroid coordinates, thereby continuously propagating the vertex view weights to the texture sampling point level, and performing projection sampling and fusion in multiple views to generate the final texture map.

[0084] S103. For any texture sampling point, determine the target triangle facet corresponding to the texture sampling point, and calculate the centroid coordinates of the texture sampling point within the target triangle facet. Based on the centroid coordinates, interpolate the vertex view weights of the three vertices of the target triangle facet to obtain the view blending weight of the texture sampling point.

[0085] In specific implementation, after constructing the texture parameter domain and determining the set of texture sampling points to be synthesized, the present invention determines the target triangular facet corresponding to any texture sampling point and calculates the centroid coordinates of the texture sampling point within the target triangular facet.

[0086] Here, the target triangular facet is the triangular facet in the 3D mesh model that corresponds to the texture sampling point in the texture parameter domain (UV space). By establishing an accurate correspondence between the texture sampling point and the target triangular facet in the UV space, a geometric basis can be provided for the subsequent continuous propagation of the vertex-level weights to the texture sampling point level, thereby ensuring the stability and consistency of texture synthesis.

[0087] In a practical implementation, the above-mentioned determination can be performed on a per-texture-block basis. For the texture map of each texture block, the present invention can perform pixel traversal of the texture map to obtain the current texture sampling point, and convert the position of the current texture sampling point into UV space coordinates. Subsequently, in the set of triangle faces contained in the texture block, the UV vertex coordinates corresponding to each candidate triangle face are obtained one by one. The UV vertex coordinates of the candidate triangle faces are used to perform face assignment determination on the current texture sampling point to determine the target triangle face containing the current texture sampling point. Since the same texture block contains only a set of faces that are adjacent in UV space, the above-mentioned method of limiting the search range can reduce the computational amount of traversal determination and improve the overall efficiency.

[0088] As one possible implementation method, see Figure 2 The diagram shows a flowchart of another multi-view texture synthesis method provided in this embodiment of the present disclosure. The method includes steps S1031 to S1033, wherein:

[0089] S1031. Perform pixel traversal on the texture map of each texture block to obtain the current texture sampling point.

[0090] S1032. For the current texture sampling point, calculate the centroid coordinates of the current texture sampling point based on the UV vertex coordinates of the target triangle facet.

[0091] S1033. When the centroid coordinates meet the preset internal judgment conditions, determine that the current texture sampling point belongs to the target triangle facet.

[0092] In one embodiment, the determination of a facet's affiliation can be based on internal criteria using centroid coordinates. Specifically, for any candidate triangular facet, it corresponds to three vertices in UV space, denoted as UV1, UV2, and UV3; the current texture sampling point is denoted as UVp. By calculating the centroid coordinates (α, β, γ) of UVp relative to the triangles (UV1, UV2, UV3), and determining whether α, β, and γ satisfy preset internal criteria (e.g., α, β, and γ are all not less than 0 and α+β+γ is 1 or within the tolerance range), if satisfied, the current texture sampling point is determined to be located inside the candidate triangular facet, and thus the candidate triangular facet is identified as the target triangular facet. For texture sampling points located at the triangle boundary or vertex neighborhood, a tolerance threshold can be introduced to improve numerical stability and avoid misjudgments or holes caused by floating-point errors.

[0093] Here, after determining the target triangular facet, the present invention calculates the centroid coordinates of the texture sampling points within the target triangular facet. The centroid coordinates characterize the area ratio or linear combination coefficient of the texture sampling point relative to the three vertices of the target triangular facet, satisfying its affine consistency within the target triangular facet. Using these centroid coordinates, vertex-level attributes can be continuously interpolated to any position within the facet, thereby ensuring that the weights and color blending of subsequent texture sampling points change continuously in space, reducing texture seams and abrupt changes.

[0094] In this embodiment, based on the centroid coordinates, the vertex view weights of the three vertices of the target triangular facet are interpolated to obtain the view blending weights of the texture sampling points.

[0095] Specifically, the target triangular facet corresponds to three mesh vertices in the geometric domain, denoted as V1, V2, and V3. Based on the preceding steps, the vertex view weights for each mesh vertex in each view have been obtained, denoted as W(V1), W(V2), and W(V3), respectively. When the centroid coordinates of the texture sampling point UVp relative to the facet are (α, β, γ), these centroid coordinates are used as interpolation coefficients to linearly combine W(V1), W(V2), and W(V3) to obtain the view blending weight W(UVp) for the texture sampling point in each view, expressed as: W(UVp) = α·W(V1) + β·W(V2) + γ·W(V3). This linear combination is performed in each view dimension to obtain the blending weight vector of the texture sampling point for each view.

[0096] In one embodiment, to further suppress the influence of invalid views, the blending weights corresponding to invisible views can be set to 0 during view blending weight interpolation. Specifically, the visibility determination results at the vertex level can be inherited; that is, when the target view is not visible to the face region or has no effect on the projection of the sampling points, the blending weights corresponding to that view are reset to zero; or visibility masking processing can be performed on the blending weights after interpolation to ensure that subsequent color sampling and fusion only use valid views. Through the above processing, noise or incorrect colors introduced into the fusion result by occluded views or views with large oblique angles can be avoided, improving the robustness of texture synthesis.

[0097] Thus, through the above steps, this embodiment realizes the continuous propagation from the view weight of the mesh vertex to the view blending weight of the texture sampling point: First, the target triangle facet of the texture sampling point is accurately determined in the UV space and the centroid coordinates are calculated. Then, the vertex view weights of the three vertices of the facet are interpolated using the centroid coordinates to obtain the view blending weights at the pixel / sampling point level, providing a stable and continuous weight foundation for subsequent multi-view projection color sampling and weighted fusion based on camera parameters.

[0098] S104. Based on the view blending weight, the texture sampling points are mapped to at least one view using the camera parameters and color sampling is performed. The sampled colors are then weighted and fused to obtain the output color of the texture sampling points.

[0099] In practical implementation, after obtaining the view blending weights of the texture sampling points, this invention maps the texture sampling points to at least one view using the camera parameters based on the view blending weights and performs color sampling. The sampled colors are then weighted and fused to obtain the output color of the texture sampling point. The output color is the color value of the final texture map at that texture sampling point, obtained by fusing observations of the same surface location from multiple views. This balances sharpness and continuity, and reduces the effects of occlusion, noise, or local distortion caused by single-view sampling.

[0100] In this implementation, to map the texture sampling point to each view, this embodiment first determines the corresponding three-dimensional position of the texture sampling point in the geometric domain. Since the previous steps have determined the target triangle facet and its centroid coordinates corresponding to the texture sampling point, this embodiment can interpolate the three grid vertex coordinates based on the coordinates of the three grid vertices of the target triangle facet in the geometric domain, and use the centroid coordinates as interpolation coefficients, thereby obtaining the three-dimensional coordinates of the texture sampling point on the model surface.

[0101] In one embodiment, the present invention normalizes the view blending weights of the texture sampling points. The purpose of normalization is to ensure that the weights of different views have uniform scale constraints, thereby guaranteeing that subsequent color blending satisfies a stable weighted summation relationship.

[0102] Specifically, the blending weights of the texture sampling points in each view can be summed, and the blending weight of each view can be divided by the summed value. When the summed value is 0, it means that the texture sampling point is unavailable or suppressed in all views. In this embodiment, a fallback strategy can be triggered, such as setting the output color of the texture sampling point to a preset color, or performing interpolation filling based on the colors of adjacent available sampling points, in order to avoid holes in the final texture map.

[0103] Furthermore, after weight normalization, this embodiment uses camera parameters to project the geometric space points onto the image plane positions of each visible view. Specifically, for any view, the geometric space points are transformed from the world coordinate system / model coordinate system to the camera coordinate system using the camera extrinsic parameters corresponding to that view, and then the points in the camera coordinate system are projected onto the image plane using the camera intrinsic parameters to obtain the corresponding two-dimensional pixel coordinates.

[0104] Here, if the projection falls outside the image range, is behind the camera, or has an abnormal projection depth, the view can be determined to be invalid for the texture sampling point, and the view can be removed from the subsequent sampling set or its weight can be reset to zero to ensure the validity of the sampling results.

[0105] In one embodiment, the present invention performs color sampling on the image at the two-dimensional pixel coordinates to obtain a sampled color. Since the projected pixel coordinates are usually non-integer coordinates, this embodiment can use interpolation sampling to obtain sub-pixel precision color values, for example, using bilinear interpolation: reading the colors of the four pixels adjacent to the two-dimensional pixel coordinates and interpolating them according to distance weights, thereby obtaining the sampled color of the texture sampling point in this view. Interpolation sampling can reduce jagged edges and jitter caused by discrete pixels and improve the smoothness and detail fidelity of the blended texture.

[0106] After obtaining the sampled colors under each visible view, this embodiment performs weighted fusion of the sampled colors according to the normalized view blending weights to obtain the output color of the texture sampling point. Specifically, for the texture sampling point, the sampled color of each view is multiplied by its corresponding normalized blending weight and summed to obtain the fused color value.

[0107] It should be noted that the fusion can be performed in the RGB color space or in other color spaces, and the present invention does not limit it; when using the RGB color space, the weighted summation of the R, G, and B channels can be performed separately to obtain the output color of the texture sampling point.

[0108] In a further implementation, to improve the robustness of the fusion result, this embodiment can also perform outlier suppression processing on the sampled colors before fusion. For example, when the sampled color of a certain view differs from the sampled color of other views by more than a preset threshold, the weight contribution of that view can be reduced or the view can be removed to reduce local color casts caused by reflections, highlights, shadows, or inconsistent exposure; or, after fusion, boundary smoothing or light filtering processing can be performed on the output colors to reduce fusion discontinuities at texture block boundaries. This invention allows for the selection of whether to enable the above-mentioned enhancement processing according to the actual application scenario.

[0109] Thus, through the above steps, this embodiment can utilize the view blending weights of texture sampling points to perform projection sampling and weighted fusion on the same surface position in multiple effective views, thereby generating a stable output color for each texture sampling point, providing a point-by-point color basis for the subsequent generation of a complete texture map, and effectively reducing the impact of stitching, jump and occlusion errors in multi-view texture synthesis.

[0110] S105. Generate a texture map based on the output color of each texture sampling point in the texture parameter domain, and export the texture map.

[0111] In practical implementation, after determining the output color of each texture sampling point within the texture parameter domain, this invention generates a texture map based on the output color of each texture sampling point in the texture parameter domain and exports the texture map. The texture map is used in conjunction with the UV parameterization relationship of the 3D mesh model to achieve texture mapping on the surface of the 3D mesh model, thereby giving the 3D mesh model a realistic appearance. The texture map can be one or more 2D image files, depending on the UV atlas partitioning method, texture packing strategy, and target output format.

[0112] In a specific implementation, this invention can generate texture maps separately for each texture block. For each texture block, this embodiment determines the resolution of the texture map corresponding to the texture block as the target size, and writes the output color of the texture sampling point pixel by pixel on the pixel grid of the texture map, thereby generating a local texture map of the texture block.

[0113] Specifically, when the set of texture sampling points corresponds one-to-one with the pixels of the texture map, the output color of each texture sampling point can be directly written to the corresponding pixel position; when using a multi-sampling or sub-sampling point strategy, the output colors of multiple sub-sampling points within the same pixel can be aggregated (e.g., by taking the average or weighted average) and then written to the pixel position to obtain a smoother texture result.

[0114] In one embodiment, to avoid jagged edges or gaps at the edges of texture blocks, the present invention can perform boundary processing on the boundary areas of texture blocks. The boundary processing may include, but is not limited to: appropriately expanding the boundary pixels, that is, expanding the color inside the patch outwards by a certain pixel width into the UV blank area to reduce edge leakage during texture sampling; or performing smooth transition processing on pixels near the boundary to reduce the impact of minor color differences between adjacent texture blocks or adjacent patches on visual continuity.

[0115] It should be noted that this invention does not limit the boundary processing method. Appropriate strategies and edge widths can be selected according to the specific rendering pipeline and texture filtering method (such as bilinear / trilinear filtering, anisotropic filtering).

[0116] In a further implementation, when the texture sampling point is not visible in all views or the weight normalization factor is 0 and a fallback strategy is triggered, this embodiment can write the corresponding pixel with a preset color or fill the pixel with a neighborhood-based completion method.

[0117] For example, interpolation filling can be performed using the colors of adjacent valid pixels, or nearest neighbor propagation can be performed within the same texture block to reduce hole areas and improve the integrity of the texture map. This invention does not limit the type of completion algorithm, as long as it ensures that the output texture map can be used for subsequent texture rendering.

[0118] In one embodiment, the present invention may further perform pre-output formatting processing on the generated texture map. The formatting processing may include: color space conversion (e.g., conversion between linear space and sRGB space), channel range clipping (limiting pixel values ​​to a preset range), and optional compression encoding processing. Through the above formatting processing, the exported texture map can be adapted to different rendering engines or 3D asset format requirements, improving the compatibility of the texture map in practical applications.

[0119] Furthermore, after generating local texture maps for each texture block, this embodiment can perform texture packing to export the final texture map. Specifically, multiple texture blocks can be arranged into one or more texture maps according to preset packing rules. These packing rules may include inter-block spacing, rotation restrictions, filling strategies, and maximum texture size restrictions, etc., to improve the space utilization of the texture map and reduce the number of textures. After packing is completed, the corresponding UV atlas information is updated or output so that the position and scale of each texture block in the final texture map can match the UV coordinates of the 3D mesh model, thereby ensuring the correctness of the final texture map.

[0120] Finally, the present invention exports the generated texture map and its corresponding UV parameterization information. The export may include saving the texture map as a preset image file format and associating it with a 3D mesh model file for output; the preset image file format may be PNG, JPEG, or other common texture formats, and the present invention does not limit this. Through the above export steps, a textured 3D model asset that can be directly used for 3D display, rendering, or subsequent processing can be obtained, thereby completing the entire output process of multi-view texture synthesis.

[0121] This disclosure provides a multi-view texture synthesis method that constructs vertex view weights by integrating Euclidean distance and viewing angle at the vertex level. In the texture parameter domain, the vertex weights are interpolated into view mixing weights at the sampling point level using the centroid coordinates of the texture sampling points relative to the target triangle facet. Then, by combining visibility suppression, weight normalization, and multi-view projection interpolation sampling, weighted fusion is achieved. This allows the fusion weights to change continuously within the facet, reducing view jumps and stitching artifacts between adjacent sampling points, improving the continuity and consistency of texture synthesis. At the same time, it suppresses invalid view interference under occlusion or unfavorable viewing conditions, thereby obtaining a more stable and natural final texture map and facilitating the packaging and export of texture blocks.

[0122] The above scheme will now be described in conjunction with specific implementation methods.

[0123] For each mesh vertex in the 3D reconstruction model, the weight of each view for that vertex is first determined by calculating the distance and angle between that vertex and the relevant viewpoints in multiple views. Assuming viewpoints... and vertex The positions are respectively and Calculate the Euclidean distance between the vertex and the viewpoint. The specific calculation method is shown in formula (1). Furthermore, the angle between the viewpoint and the vertex is calculated. Weights between vertices and viewpoints It can be calculated using formula (2):

[0124]

[0125]

[0126] The 3D reconstructed model is textured to obtain a corresponding texture map (UV map). Based on this, the entire texture map is divided into multiple texture blocks, each composed of a set of adjacent triangular faces. Each texture block has an independent texture map, facilitating subsequent multi-view texture compositing.

[0127] For each triangle in a patch, a texture map (UV map) is defined. The UV coordinates of each triangle represent its position in the texture map. The calculation of the texture map coordinates corresponds to the geometric position of the triangle to ensure consistency in the final texture composition.

[0128] Iterate through the UV coordinates in each texture map and find the triangle facet corresponding to that coordinate. For each triangle, calculate the centroid coordinates of the current texture coordinates within the triangle, i.e., the relative position of the texture coordinates within the triangle. Assume the three vertices of the triangle are... , , Then the point corresponding to the current texture coordinates is Its centroid coordinates , , Calculate according to formula (3):

[0129]

[0130] For each triangle, the barycenter coordinates are interpolated with the relevant view weights of the three vertices of the triangle to obtain the blended weights for each texture coordinate. Specifically, assume that the weight sets for each view corresponding to the three vertices are as follows: , , Then, the set of blending weights for each texture coordinate across all views. This can be represented by formula (4). Subsequently, these texture coordinates are projected onto each view to obtain texture images on multiple views, color sampling is completed, and the sampled colors are weighted and mixed to obtain the final color. In this way, the continuity of texture sampling is effectively improved, thereby smoothing the texture transition and eliminating color inconsistency and distortion caused by factors such as viewing angle differences and lighting changes.

[0131]

[0132] The above steps are applied to the texture coordinates of all texture blocks, ultimately synthesizing the texture maps of each block into a complete texture map. Finally, all synthesized texture maps are packaged to generate the final multi-view texture map. This method enables seamless fusion of textures from multiple perspectives, achieving high-quality 3D texture restoration.

[0133] Those skilled in the art will understand that, in the above-described method of the specific implementation, the order in which each step is written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0134] Based on the same inventive concept, this disclosure also provides a multi-view texture synthesis apparatus corresponding to the multi-view texture synthesis method. Since the principle of the apparatus in this disclosure for solving the problem is similar to the multi-view texture synthesis method described above in this disclosure, the implementation of the apparatus can refer to the implementation of the method, and the repeated parts will not be described again.

[0135] Please see Figure 3 , Figure 3 This is a schematic diagram of a multi-view texture synthesis apparatus provided in an embodiment of this disclosure. Figure 3 As shown in the embodiments of this disclosure, the multi-view texture synthesis apparatus 300 includes:

[0136] The data acquisition module 310 is used to acquire the three-dimensional mesh model to be textured, the multi-view image set, and the camera parameters corresponding to the multi-view image set, and to determine the vertex view weights of the mesh vertices of the three-dimensional mesh model in each view based on the three-dimensional mesh model and the camera parameters.

[0137] The sampling point determination module 320 is used to construct the texture parameter domain of the three-dimensional mesh model and determine the set of texture sampling points to be synthesized in the texture parameter domain.

[0138] The blending weight determination module 330 is used to determine the target triangle facet corresponding to any texture sampling point, calculate the centroid coordinates of the texture sampling point in the target triangle facet, and interpolate the vertex view weights of the three vertices of the target triangle facet based on the centroid coordinates to obtain the view blending weight of the texture sampling point.

[0139] The output color determination module 340 is used to map the texture sampling points to at least one view and perform color sampling based on the view blending weight and the camera parameters, and to perform weighted fusion of the sampled colors to obtain the output color of the texture sampling points.

[0140] The texture map output module 350 is used to generate a texture map based on the output color of each texture sampling point in the texture parameter domain, and to export the texture map.

[0141] The processing flow of each module in the device and the interaction flow between each module can be referred to the relevant descriptions in the above method embodiments, and will not be detailed here.

[0142] This disclosure provides a multi-view texture synthesis apparatus that constructs vertex view weights by integrating Euclidean distance and viewing angle at the vertex level. In the texture parameter domain, the vertex weights are interpolated into view mixing weights at the sampling point level using the centroid coordinates of the texture sampling points relative to the target triangle facet. Then, by combining visibility suppression, weight normalization, and multi-view projection interpolation sampling, weighted fusion is achieved. This allows the fusion weights to change continuously within the facet, reducing view jumps and stitching artifacts between adjacent sampling points, improving the continuity and consistency of texture synthesis. At the same time, it suppresses invalid view interference under occlusion or unfavorable viewing conditions, thereby obtaining a more stable and natural final texture map and facilitating the packaging and export of texture blocks.

[0143] Corresponding to Figure 1 and Figure 2 In addition to the multi-view texture synthesis method, this disclosure also provides an electronic device 400, such as... Figure 4 The diagram shown is a structural schematic of an electronic device 400 provided in an embodiment of this disclosure, including:

[0144] Processor 41, memory 42, and bus 43; memory 42 is used to store execution instructions, including main memory 421 and external memory 422; the main memory 421, also called internal memory, is used to temporarily store the computational data in processor 41, as well as the data exchanged with external memory 422 such as hard disk. Processor 41 exchanges data with external memory 422 through main memory 421. When the electronic device 400 is running, processor 41 and memory 42 communicate through bus 43, enabling processor 41 to execute... Figure 1 and Figure 2 The steps of the multi-view texture synthesis method.

[0145] This disclosure also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the multi-view texture synthesis method described in the above-described method embodiments. The storage medium can be a volatile or non-volatile computer-readable storage medium.

[0146] This disclosure also provides a computer program product, which includes computer instructions. When the computer instructions are executed by a processor, they can perform the steps of the multi-view texture synthesis method described in the above method embodiments. For details, please refer to the above method embodiments, which will not be repeated here.

[0147] The aforementioned computer program product can be implemented through hardware, software, or a combination thereof. In one optional embodiment, the computer program product is specifically embodied in a computer storage medium; in another optional embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0148] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the device described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here. In the several embodiments provided in this disclosure, it should be understood that the disclosed device and method can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection may be through some communication interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0149] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0150] In addition, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0151] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this disclosure, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this disclosure. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0152] Finally, it should be noted that the above-described embodiments are merely specific implementations of this disclosure, used to illustrate the technical solutions of this disclosure, and not to limit it. The protection scope of this disclosure is not limited thereto. Although this disclosure has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this disclosure. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this disclosure, and should all be covered within the protection scope of this disclosure. Therefore, the protection scope of this disclosure should be determined by the protection scope of the claims.

Claims

1. A method for synthesizing multi-view textures, characterized in that, include: Obtain the 3D mesh model to be textured, the multi-view image set, and the camera parameters corresponding to the multi-view image set. Based on the 3D mesh model and the camera parameters, determine the vertex view weights of the mesh vertices of the 3D mesh model in each view. Specifically, for any grid vertex and any viewpoint, the Euclidean distance and the angle of view between the grid vertex and the viewpoint are calculated; based on the Euclidean distance and the angle of view, the unnormalized vertex view weight of the grid vertex in the view is constructed; wherein, the unnormalized vertex view weight is determined by the distance term and the angle term, and adjustable parameters are introduced to adjust the contribution ratio of the distance term and the angle term to the unnormalized vertex view weight respectively; Construct the texture parameter domain of the three-dimensional mesh model, and determine the set of texture sampling points to be synthesized in the texture parameter domain; Specifically, the 3D mesh model is texture-unfolded to obtain a UV atlas, and the UV atlas is divided into multiple texture blocks, each texture block consisting of a set of adjacent triangular faces; the bounding box size of each texture block determines the texture map range corresponding to that texture block, and the texture resolution of that texture block is determined; for each triangular face within each texture block, a correspondence is established between the UV vertex coordinates of the triangular face in the UV atlas and the geometric vertex coordinates of the triangular face to ensure texture mapping consistency; For any texture sampling point, determine the target triangle facet corresponding to the texture sampling point, and calculate the centroid coordinates of the texture sampling point within the target triangle facet. Based on the centroid coordinates, interpolate the vertex view weights of the three vertices of the target triangle facet to obtain the view blending weight of the texture sampling point. Based on the view blending weights, the texture sampling points are mapped to at least one view using the camera parameters and color sampling is performed. The sampled colors are then weighted and fused to obtain the output color of the texture sampling points. A texture map is generated based on the output color of each texture sampling point in the texture parameter domain, and the texture map is exported.

2. The method according to claim 1, characterized in that, Based on the 3D mesh model and the camera parameters, the vertex view weights of the mesh vertices in each view are determined, specifically including: When the mesh vertex is not visible in the target view or is outside the preset viewing range, the unnormalized vertex view weight corresponding to the target view is set to 0. The unnormalized vertex view weights of the same mesh vertex in multiple views are normalized to obtain the vertex view weights of the mesh vertex in each view.

3. The method according to claim 1, characterized in that, For any texture sampling point, determine the target triangular facet corresponding to the texture sampling point, and calculate the centroid coordinates of the texture sampling point within the target triangular facet, specifically including: For each texture block, perform pixel traversal on the texture map to obtain the current texture sampling point; For the current texture sampling point, calculate the centroid coordinates of the current texture sampling point based on the UV vertex coordinates of the target triangle facet; When the centroid coordinates meet the preset internal determination conditions, the current texture sampling point is determined to belong to the target triangle facet.

4. The method according to claim 1, characterized in that, Based on the centroid coordinates, the vertex view weights of the three vertices of the target triangular facet are interpolated to obtain the view blending weights of the texture sampling points, specifically including: Using the centroid coordinates as interpolation coefficients, the vertex view weights of the three vertices of the target triangle facet in each view are linearly combined to obtain the view blending weights of the texture sampling points for each view, and the view blending weights corresponding to the invisible views are set to 0.

5. The method according to claim 1, characterized in that, Based on the view blending weights, the texture sampling points are mapped to at least one view using the camera parameters, and color sampling is performed. The sampled colors are then weighted and fused to obtain the output color of the texture sampling points. Specifically, this includes: Normalize the view blending weights; The texture sampling points are projected onto the image plane positions of each visible view using the camera parameters; Interpolation sampling is used to obtain the sampled color for the image plane position of each visible view; The sampled colors are weighted and fused according to the normalized view mixing weights to obtain the output color.

6. A multi-view texture synthesis device, characterized in that, include: The data acquisition module is used to acquire the 3D mesh model to be textured, the multi-view image set, and the camera parameters corresponding to the multi-view image set, and to determine the vertex view weights of the mesh vertices of the 3D mesh model in each view based on the 3D mesh model and the camera parameters. Specifically, for any grid vertex and any viewpoint, the Euclidean distance and the angle of view between the grid vertex and the viewpoint are calculated; based on the Euclidean distance and the angle of view, the unnormalized vertex view weight of the grid vertex in the view is constructed; wherein, the unnormalized vertex view weight is determined by the distance term and the angle term, and adjustable parameters are introduced to adjust the contribution ratio of the distance term and the angle term to the unnormalized vertex view weight respectively; The sampling point determination module is used to construct the texture parameter domain of the three-dimensional mesh model and determine the set of texture sampling points to be synthesized in the texture parameter domain; Specifically, the 3D mesh model is texture-unfolded to obtain a UV atlas, and the UV atlas is divided into multiple texture blocks, each texture block consisting of a set of adjacent triangular faces; the bounding box size of each texture block determines the texture map range corresponding to that texture block, and the texture resolution of that texture block is determined; for each triangular face within each texture block, a correspondence is established between the UV vertex coordinates of the triangular face in the UV atlas and the geometric vertex coordinates of the triangular face to ensure texture mapping consistency; The hybrid weight determination module is used to determine the target triangle facet corresponding to any texture sampling point, calculate the centroid coordinates of the texture sampling point in the target triangle facet, and interpolate the vertex view weights of the three vertices of the target triangle facet based on the centroid coordinates to obtain the view hybrid weight of the texture sampling point. The output color determination module is used to map the texture sampling points to at least one view based on the view blending weight and the camera parameters, and perform color sampling, and perform weighted fusion of the sampled colors to obtain the output color of the texture sampling points. The texture map output module is used to generate a texture map based on the output color of each texture sampling point in the texture parameter domain, and to export the texture map.

7. An electronic device, characterized in that, include: The device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, the steps of the multi-view texture synthesis method as described in any one of claims 1 to 5 are performed.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the multi-view texture synthesis method as described in any one of claims 1 to 5.

Images