Texture mapping method and device, electronic equipment and storage medium

Abstract

The texture mapping method, device, electronic equipment and storage medium provided by the application relate to the field of scanning data processing. The method is point cloud conversion processing on a grid model of a measured object to obtain point cloud data; according to the pose of each collection image of the measured object, the mapping relationship between each collection image and the point cloud data and a global texture atlas are obtained; based on the global texture atlas, the mapping relationship between each collection image and the point cloud data is corrected to obtain the corrected mapping relationship between each collection image and the point cloud data; the global texture atlas is updated according to the corrected mapping relationship between each collection image and the point cloud data to obtain an updated global texture atlas; and based on the updated global texture atlas, the grid model is converted into a texture map, so that the color misplacement, ghosting and joint fracture problems caused by the pose error are eliminated, and the realism, continuity and engineering robustness of the texture map in the high-precision three-dimensional reconstruction scene such as intraoral scanning are improved.

Description

Technical Field

[0001] This invention relates to the field of scan data processing, and more specifically, to a texture mapping method, apparatus, electronic device, and storage medium. Background Technology

[0002] In 3D reconstruction technology based on multi-view images, texture mapping is a key step in accurately assigning color information from the acquired images to the surface of the reconstructed mesh model, directly affecting the realism and usability of the final rendering result.

[0003] Currently, texture mapping in 3D reconstruction mainly employs two methods: weighted fusion and optimal image selection. Weighted fusion improves overall consistency by averaging colors from multiple perspectives, but the projection position of the same spatial point in different images may shift, leading to blurring, ghosting, and color bleeding in high-frequency detail areas. Optimal image selection, while preserving the sharpness of individual images, can easily cause seam misalignment and texture breakage in texture-rich areas because adjacent patches may select images from different perspectives. Summary of the Invention

[0004] In view of this, the object of the present invention is to provide a texture mapping method, apparatus, electronic device and computer-readable storage medium.

[0005] To achieve the above objectives, the technical solutions adopted in the embodiments of the present invention are as follows: In a first aspect, the present invention provides a texture mapping method, the method comprising: Acquire the mesh model of the object under test and multiple acquired images; The mesh model is subjected to point cloud transformation processing to obtain point cloud data; Based on the pose of each acquired image, obtain the mapping relationship between each acquired image and the point cloud data, and the global texture map; Based on the global texture map, the mapping relationship between each acquired image and the point cloud data is corrected to obtain the corrected mapping relationship between each acquired image and the point cloud data; The global texture map is updated based on the corrected mapping relationship between each acquired image and the point cloud data to obtain the updated global texture map. Based on the updated global texture map, the mesh model is converted into a texture map.

[0006] Optionally, the step of performing point cloud transformation processing on the mesh model to obtain point cloud data includes: Based on the area of ​​each facet in the mesh model and the preset sampling interval, multiple spatial points are generated within each facet to obtain the point cloud data; the point cloud data includes all spatial points within the facets.

[0007] Optionally, the step of obtaining the mapping relationship between each acquired image and the point cloud data and the global texture map based on the pose of each acquired image includes: For each of the acquired images, the pixel coordinates of each spatial point in the point cloud data mapped to the acquired image are determined based on the pose of the acquired image. Based on the pixel coordinates of each spatial point mapped to the acquired image and the range of pixel coordinates in the acquired image, visible spatial points are determined, and the mapping relationship between the acquired image and the point cloud data is obtained; the mapping relationship between the acquired image and the point cloud data includes the pixel coordinates of each visible spatial point mapped to the acquired image, and the pixel coordinates of the visible spatial points mapped to the acquired image are located within the range of pixel coordinates in the acquired image; The global texture map is obtained based on the mapping relationship between each acquired image and the point cloud data.

[0008] Optionally, the step of obtaining the global texture map based on the mapping relationship between each acquired image and the point cloud data includes: For each spatial point, a first target image is determined from all the acquired images based on the mapping relationship between each acquired image and the point cloud data; the pixel coordinates of the spatial point mapped to the first target image are located within the pixel range of the first target image. Obtain the color value at the pixel coordinates of each of the spatial points mapped to the first target image; The color values ​​at the pixel coordinates of each first target image mapped to the spatial point are weighted and summed, and the sum is used as the target color value and associated with the spatial point. Traverse all the spatial points to obtain the global texture map; the global texture map includes the target color value associated with each spatial point.

[0009] Optionally, the step of correcting the mapping relationship between each acquired image and the point cloud data based on the global texture map to obtain the corrected mapping relationship between each acquired image and the point cloud data includes: For each of the acquired images, deformation control parameters for the acquired image are constructed based on the mapping relationship between the acquired image and the point cloud data and the global texture map; The acquired image is resampled using the deformation control parameters to obtain a resampled acquired image; Based on the resampled acquired image, the mapping relationship between the acquired image and the point cloud data is corrected to obtain the corrected mapping relationship between the acquired image and the point cloud data.

[0010] Optionally, the step of updating the global texture map based on the corrected mapping relationship between each acquired image and the point cloud data to obtain the updated global texture map includes: For each spatial point, based on the corrected mapping relationship between each acquired image and the point cloud data, each second target image is determined from all corrected acquired images, wherein the pixel coordinates of the spatial point mapped to the second target image are within the pixel range of the second target image; Obtain the color value at the pixel coordinates of each second target image mapped from the spatial point; The color values ​​at the pixel coordinates of each second target image mapped from the spatial point are weighted and summed, and the sum is used as the new target color value and associated with the spatial point. Traverse all the said spatial points to obtain an updated global texture map, which includes the new target color value associated with each of the said spatial points.

[0011] Optionally, the step of converting the mesh model into a texture map based on the updated global texture map includes: Based on the normal vector and adjacency relationship of each facet in the mesh model, the mesh model is unfolded into a planar image; For each spatial point, based on the position of the spatial point in the mesh model, a target region on the planar image corresponding to the spatial point is determined, and the color value of the target region is set as a new target color value associated with the spatial point; By traversing all the aforementioned spatial points, the texture map is obtained.

[0012] In a second aspect, the present invention provides a texture mapping apparatus, the apparatus comprising: The acquisition module is used to acquire the mesh model of the object under test and multiple acquired images; The processing module is used to perform point cloud conversion processing on the mesh model to obtain point cloud data; based on the pose of each acquired image, obtain the mapping relationship between each acquired image and the point cloud data and the global texture map of the point cloud data; based on the global texture map, correct the mapping relationship between each acquired image and the point cloud data to obtain the corrected mapping relationship between each acquired image and the point cloud data; update the global texture map based on the corrected mapping relationship between each acquired image and the point cloud data to obtain the updated global texture map; and based on the updated global texture map, convert the mesh model into a texture map.

[0013] Thirdly, the present invention provides an electronic device including a processor and a memory, the memory storing machine-executable instructions executable by the processor, the processor executing the machine-executable instructions to implement the texture mapping method described in the first aspect above.

[0014] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the texture mapping method as described in the first aspect above.

[0015] The texture mapping method, apparatus, electronic device, and storage medium provided in this invention acquire a mesh model of the object under test and multiple acquired images; perform point cloud conversion processing on the mesh model to obtain point cloud data; obtain the mapping relationship between each acquired image and the point cloud data and a global texture map based on the pose of each acquired image; correct the mapping relationship between each acquired image and the point cloud data based on the global texture map to obtain the corrected mapping relationship; update the global texture map based on the corrected mapping relationship between each acquired image and the point cloud data to obtain the updated global texture map; and convert the mesh model into a texture map based on the updated global texture map. Because this invention uses the global texture map to correct the spatial mapping deviation between each frame of acquired images and the point cloud in real time, it eliminates color misalignment, ghosting, and seam breaks caused by pose errors while preserving the robustness of weighted fusion, thus improving the realism, continuity, and engineering robustness of texture maps in high-precision 3D reconstruction scenarios such as intraoral scanning.

[0016] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This figure shows a schematic block diagram of an electronic device provided by an embodiment of the present invention; Figure 2 A flowchart illustrating a texture mapping method provided by an embodiment of the present invention is shown; Figure 3 A schematic diagram illustrating the implementation process of step S103 provided in an embodiment of the present invention is shown; Figure 4 A schematic diagram illustrating the implementation process of step S104 provided in an embodiment of the present invention is shown; Figure 5 A schematic diagram illustrating the implementation process of step S105 provided in an embodiment of the present invention is shown; Figure 6 A schematic diagram illustrating the implementation process of step S106 provided in an embodiment of the present invention is shown; Figure 7 A functional block diagram of a texture mapping device provided in an embodiment of the present invention is shown.

[0019] Icons: 100 - Electronic device; 110 - Memory; 120 - Processor; 130 - Communication module; 200 - Texture mapping device; 201 - Acquisition module; 202 - Processing module. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0021] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0022] It should be noted that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0023] Traditional texture mapping generation methods generally employ multi-view projection, which directly utilizes multiple color images taken from different perspectives during 3D reconstruction. These images are projected frame by frame onto the surface of a 3D mesh model with known geometry, establishing a correspondence between pixels and model vertices or faces. Finally, the entire 3D model is unfolded into a single 2D image, completing texture mapping generation. Based on this, mainstream methods fall into two main categories: the first is a fusion scheme, which uses a weighted average of the pixel colors corresponding to each mesh face across all visible images to obtain the final texture for that face; the second is an optimization scheme, which selects a single image with the best imaging quality and the most correct viewing angle for each face as its sole texture source. While both approaches have their own applicable scenarios, they both have inherent flaws in practical applications: the fusion approach relies on precise spatial alignment between the image and the model. If there is a slight error in the pose of the acquired image or the camera intrinsic calibration is not accurate enough, the same spatial location will be projected onto different pixel coordinates in different images, resulting in color blurring and ghosting after weighted averaging. The preferred approach, while avoiding aliasing caused by cross-image fusion, is prone to seam misalignment due to differences in perspective and projection mismatch in areas with rich texture (such as text markings, scratches, and enamel boundaries on the tooth surface) when different images are selected for adjacent patches, causing visual breaks.

[0024] In this regard, embodiments of the present invention provide a texture mapping method, apparatus, electronic device, and storage medium, which will be described in detail below.

[0025] Please refer to Figure 1 This is a block diagram of an electronic device 100. The electronic device 100 includes a memory 110, a processor 120, and a communication module 130. The memory 110, processor 120, and communication module 130 are electrically connected to each other directly or indirectly to realize data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines.

[0026] The memory 110 is used to store programs or data. The memory 110 may be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.

[0027] The processor 120 is used to read / write data or programs stored in the memory 110 and to perform corresponding functions.

[0028] The communication module 130 is used to establish a communication connection between the electronic device 100 and other communication terminals through the network, and to send and receive data through the network.

[0029] It should be understood that, Figure 1 The structure shown is only a schematic diagram of the electronic device 100. The electronic device 100 may also include components that are larger than... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown. Figure 1 The components shown can be implemented using hardware, software, or a combination thereof.

[0030] Please refer to Figure 2 The texture mapping method provided in this embodiment of the invention includes steps S101 to S106.

[0031] S101, acquire the mesh model of the object under test and multiple acquired images.

[0032] S102, perform point cloud transformation processing on the mesh model to obtain point cloud data.

[0033] In a possible implementation, step S102 can be implemented by generating multiple spatial points within each facet based on the area of ​​each facet in the mesh model and a preset sampling interval, thereby obtaining point cloud data; the point cloud data includes spatial points within all facets.

[0034] 1. In other words, iterate through each facet of the mesh model, with each triangle containing three vertices. , , This means that the area of ​​each triangular facet is calculated using the following formula. :

[0035]

[0036]

[0037]

[0038] Based on the area of ​​each triangular facet, a sampling interval is determined for each facet. The preset sampling interval is directly proportional to the facet area; that is, the larger the facet area, the larger the preset sampling interval. Within each triangular facet, a corresponding number of points are randomly generated according to the preset sampling interval. These generated points are used as spatial points to form point cloud data. .

[0039] Understandably, point cloud data is not obtained by uniform sampling. Instead, it is composed of multiple spatial points randomly generated within each facet based on the area of ​​each facet in the mesh model and the preset sampling interval. This allows the distribution of texture details to be dynamically adjusted according to geometric complexity. For example, the sampling density is increased in areas with rich textures or dramatic curvature changes, such as tooth edges and occlusal grooves, while the sampling points are reduced in large areas of smooth enamel, thereby achieving optimal perceptual quality with limited computing resources.

[0040] S103: Based on the pose of each acquired image, obtain the mapping relationship between each acquired image and point cloud data and the global texture map.

[0041] Among them, the global texture map is an intermediate data structure that is built up frame by frame with the image. It is used to record the color observation value of each spatial point under all effective viewpoints and its confidence weight.

[0042] In possible implementations, please refer to Figure 3 Step S103 may include sub-steps S103-1 to S103-3.

[0043] S103-1, For each acquired image, determine the pixel coordinates of each spatial point in the point cloud data mapped to the acquired image based on the pose of the acquired image.

[0044] In this embodiment of the invention, each spatial point The image coordinates are transformed to the image coordinate system of the currently acquired image using the pose matrix RT, and then projected to pixel coordinates using the following transformation relationship. :

[0045]

[0046] In the formula, , The focal length is in the camera intrinsic parameter matrix. and These are the coordinates of the principal point in the camera intrinsic parameter matrix.

[0047] S103-2, Based on the pixel coordinates of each spatial point mapped to the acquired image and the range of pixel coordinates of the acquired image, the visible spatial points are determined, and the mapping relationship between the acquired image and the point cloud data is obtained; the mapping relationship between the acquired image and the point cloud data includes the pixel coordinates of each visible spatial point mapped to the acquired image, and the pixel coordinates of the visible spatial points mapped to the acquired image are within the range of pixel coordinates of the acquired image.

[0048] Understandably, when the calculation results If a point falls within the effective pixel range of an image and is not occluded by other surfaces from the current viewpoint, it is identified as a "visible spatial point." The "mapping relationship between the acquired image and the point cloud data" constructed in this way is a sparse index table that clearly records which spatial points can be observed in which images and which pixel positions they correspond to.

[0049] S103-3, based on the mapping relationship between each acquired image and point cloud data, a global texture map is obtained.

[0050] Further, the implementation process of step S103-3 can be as follows: for each spatial point, based on the mapping relationship between each acquired image and point cloud data, determine each first target image from all acquired images; the pixel coordinates of the spatial point mapped to the first target image are located within the pixel range of the first target image; obtain the color value at the pixel coordinates of the spatial point mapped to each first target image; perform a weighted summation on the color values ​​at the pixel coordinates of the spatial point mapped to each first target image, and use the resulting sum as the target color value and associate it with the spatial point; traverse all spatial points to obtain a global texture map; the global texture map includes the target color value associated with each spatial point.

[0051] Understandably, for a given spatial point, all first target images satisfying the condition that "the projected coordinates of the spatial point are within its pixel range" are identified from all acquired images. The RGB color values ​​at the corresponding pixels of each first target image are extracted, and different weights are assigned based on the view angle (i.e., the angle between the spatial point's normal and the camera's optical axis) and the confidence level of the depth measurement. The color values ​​are then weighted and summed, and the resulting sum is the "target color value" of that spatial point. The target color values ​​of all spatial points together constitute a global texture map. This global texture map is not a static snapshot, but an intermediate representation with temporal accumulation and statistical robustness. It is not bound to any single frame image, but carries the most reliable color consensus among all valid observations.

[0052] S104, based on the global texture map, corrects the mapping relationship between each acquired image and point cloud data to obtain the corrected mapping relationship between each acquired image and point cloud data.

[0053] Traditional texture mapping methods directly proceed to color fusion or texture generation after obtaining the mapping relationship between the acquired image and point cloud data. This implicitly assumes that the pose (RT) and camera intrinsic parameters are completely accurate, the mesh model strictly represents the surface of the real object, and the image is free from motion blur or glare interference. However, in practical applications such as intraoral scanning, these assumptions are often difficult to meet: slight handheld device jitter can cause sub-pixel level deviations in pose estimation; gingival occlusion or saliva reflection can cause some surface depth measurements to fail, thus affecting projection reliability; and high-contrast textures on the tooth surface (such as fissures, filling edges, and plaque deposition zones) are particularly sensitive to registration errors. If the image is directly projected onto the point cloud according to the initial mapping relationship, the same spatial point will be mapped to different pixel positions in different frames, inevitably resulting in color diffusion, boundary blurring, and even double image overlap after weighted averaging.

[0054] In this regard, the embodiments of the present invention do not modify the pose parameters themselves, but use the global texture map as the color truth reference to construct a two-dimensional non-rigid deformation field in the image pixel plane. The projection misalignment caused by pose error, calibration deviation and model geometric approximation is dynamically compensated by spatial deformation resampling. Thus, without changing the semantic content of the original image, the local texture structure of the currently acquired image is aligned with the historical fusion colors that have been deposited in the point cloud space at the pixel level.

[0055] In possible implementations, please refer to Figure 4 Step S104 may include sub-steps S104-1 to S104-3.

[0056] S104-1, For each acquired image, based on the mapping relationship between the acquired image and the point cloud data and the global texture map, construct the deformation control parameters of the acquired image.

[0057] S104-2, using deformation control parameters, the acquired image is resampled to obtain a resampled acquired image.

[0058] S104-3, Based on the resampled acquired image, the mapping relationship between the acquired image and the point cloud data is corrected to obtain the corrected mapping relationship between the acquired image and the point cloud data.

[0059] This embodiment of the invention first constructs deformation control parameters for the acquired image based on the "mapping relationship between the acquired image and point cloud data" obtained in step S103 and the already constructed global texture map A; these deformation control parameters are embodied in a parameter defined at the image pixel coordinates. Two-dimensional vector field Its physical meaning is that the image is located in The pixel at that location should be resampled from the original image at position [location missing]. The color value at that location; this vector field is not a manually defined regular function (such as an affine transformation), but is automatically obtained by solving an energy minimization problem, the objective function of which is:

[0060]

[0061]

[0062] In the formula, For data items; This represents the set of all valid pixels in the current image; Indicates the acquired image After resampling, at pixel coordinates The color obtained from it; This represents the pixel coordinates of the current spatial point retrieved from the global texture map. The reference color corresponding to the projection position (this reference color is a historical consensus obtained from previous multi-frame fusion, representing a more robust and stable texture ground truth). The confidence weight is determined by the view angle (the angle between the spatial point normal and the camera optical axis) and the depth uncertainty. The smaller the angle (frontal view) and the more certain the depth, the greater the weight, ensuring that deformation is prioritized for high-quality observation areas. As a smoothing term, this term constrains the difference in deformation offset vectors between adjacent pixels to be not too large. Essentially, it's a Laplacian regularization term that prevents sharp jumps or local tearing in the deformation field, ensuring a visually natural and continuous image after deformation. (Hyperparameter) Used to balance the relationship between data fidelity and deformation smoothing, its value is experimentally calibrated to ensure that moderate deformation is allowed in textured areas (such as crown text and caries edges) to eliminate misalignment, while excessive distortion is suppressed in large smooth areas. for The neighborhood (such as the top, bottom, left, and right four neighbors).

[0063] The optimal solution obtained after solving for this energy function These parameters constitute the deformation control parameters; then, the original acquired image is resampled bilinearly or bicubicly using these parameters to generate a geometrically corrected resampled image; finally, based on the new sampling source relationship of each pixel in the resampled image, the mapping between the original point cloud and the image is updated, so that the original mapping to... The spatial point now corresponds to the new pixel position in the resampled image that still falls within the effective range after deformation, thus forming a "corrected mapping relationship between the acquired image and the point cloud data".

[0064] Understandably, the embodiments of the present invention do not change the three-dimensional geometric structure or pose parameters, but achieve equivalent compensation for three-dimensional spatial errors in the two-dimensional image domain. The originally uncontrollable "projection mismatch" problem is transformed into an image registration optimization problem guided by the global texture map and constrained by smooth priors. This avoids the computational overhead and convergence risk caused by iteratively optimizing the pose, and ensures that the subsequent global texture map update (step S105) and texture map generation (step S106) based on the corrected mapping relationship are always based on data with consistent colors and aligned structures. This fundamentally alleviates the blurring and ghosting phenomena in traditional fusion schemes, while retaining the inherent robustness of the seams in weighted fusion.

[0065] S105, the global texture map is updated according to the corrected mapping relationship between each acquired image and point cloud data, resulting in the updated global texture map.

[0066] Traditional fusion-based texture mapping methods typically perform only one static aggregation: given the initial pose and original image, they determine which faces or vertices are visible, which images to sample colors from, and then perform a weighted average to obtain the final texture. This process is irreversible once completed. If there are systematic biases in the initial projection (such as camera calibration errors, model scaling mismatch, or slight motion blur), these biases will be directly embedded in the fusion result, leading to blurred textures, blurred boundaries, or local color shifts. Especially in intraoral scanning scenarios, high-contrast structures such as tiny enamel cracks, patch edges, or plaque deposits on the tooth surface are easily smoothed out by subpixel-level misalignment, resulting in the loss of crucial details needed for clinical diagnosis.

[0067] In response, this embodiment of the invention, based on the image domain deformation correction, uses the corrected mapping relationship as a new benchmark to perform a multi-view color aggregation again, and fuses the spatially aligned and optimized image observations of each frame into an updated global texture map according to a unified weight strategy, so that the global texture map converges to a more accurate, more stable and more consistent color ground truth value.

[0068] In possible implementations, please refer to Figure 5 Step S105 may include sub-steps S105-1 to S105-3.

[0069] S105-1, For each spatial point in the point cloud data, based on the corrected mapping relationship between each acquired image and the point cloud data, determine each second target image from all corrected acquired images, wherein the pixel coordinates of the spatial point mapped to the second target image are within the pixel range of the second target image.

[0070] S105-2, obtain the color value of the spatial point mapped to the pixel coordinates in each second target image.

[0071] S105-3, perform a weighted summation of the color values ​​at the pixel coordinates of the spatial point mapped to each second target image, and use the sum as the new target color value and associate it with the spatial point.

[0072] Traverse all spatial points to obtain the updated global texture map, which includes the new target color value associated with each spatial point.

[0073] In this embodiment of the invention, for each spatial point, firstly, based on the corrected mapping relationship between each acquired image and point cloud data, a second target image satisfying the condition that "the spatial point is mapped to a pixel coordinate within the effective pixel range of the image" is identified from all corrected acquired images. This filtering process is consistent with the visibility judgment logic in S103-2, but the judgment criterion has been upgraded from the original mapping to the corrected mapping. This means that spatial points that were previously incorrectly judged as "invisible" or "projected outside the image" due to pose deviation may fall back into the effective pixel area after deformation correction, and thus be included in this fusion. Next, the color value of the spatial point at the corresponding pixel coordinate in each second target image is obtained. The color value here comes from the corrected image obtained by resampling in S104, and its pixel content has been processed by a two-dimensional deformation field. Geometric alignment was completed, so the pixel positions of the same spatial point in different second target images carry colors that are closer to the true physical reflection, rather than misaligned sampling under the original projection mismatch. Finally, these corrected color values ​​are weighted and summed, where the weights are still calculated based on the view angle and depth confidence. Among them, the smaller the included angle (the more upright the viewpoint) and the more stable the depth measurement, the higher the weight, ensuring that high-quality observation dominates the fusion result; the sum obtained is the "new target color value" of the spatial point, and the new target color values ​​of all spatial points together constitute the updated global texture map.

[0074] Understandably, in this embodiment of the invention, the global texture map is no longer regarded as a static snapshot built at one time, but is positioned as a "consensus database" that is continuously optimized as the correction accuracy improves. By re-aggregating colors under the corrected mapping relationship, the contamination of the fusion result by the initial error is eliminated, while the inherent noise suppression and seam smoothing capabilities of weighted fusion are retained.

[0075] Furthermore, when extremely high texture quality is required (such as for training data generation for AI-assisted tooth decay recognition), multiple rounds of "S104 correction → S105 update" loops can be triggered (i.e., each S105 update provides a more reliable reference color for the next S104; each S104 correction provides a more accurate mapping relationship for the next S105) until the change in the global texture map is lower than a preset threshold, thereby achieving the best balance between perceptual quality and engineering efficiency with limited computing resources.

[0076] S106 converts the mesh model into a texture map based on the updated global texture map.

[0077] Traditional texture mapping methods typically employ a fixed strategy to unfold the 3D mesh into 2D UV coordinates after color fusion (e.g., generating globally continuous UVs based on angle collapse or Laplacian parameterization), and then directly write the facet colors into the corresponding UV regions. While this approach ensures topological continuity, it struggles to meet the actual visual needs of different regions. Key diagnostic areas such as the fine pits and fissures on the occlusal surfaces of teeth, the tiny edges of the proximal contact areas, and the color transition at the junction of restorations and natural teeth, if over-compressed during UV unwrapping, will lose details during rendering even if the original colors are accurate due to insufficient pixel density. Conversely, allocating too much UV space to large areas of smooth enamel will result in wasted video memory and sampling redundancy.

[0078] In response, this embodiment of the invention maps the updated global texture map to a two-dimensional planar image space through an adaptive UV unwrapping strategy that optimizes perceptual quality and resource constraints. This allows the high-fidelity color information carried by each spatial point to obtain a spatial resolution allocation in the texture map that matches its geometric importance in the three-dimensional model.

[0079] In possible implementations, please refer to Figure 6 Step S106 may include sub-steps S106-1 to S106-2.

[0080] S106-1, based on the normal vector and adjacency relationship of each facet in the mesh model, unfold the mesh model into a planar image.

[0081] S106-2, For each spatial point, determine the target region on the corresponding planar image based on the position of the spatial point in the mesh model, and set the color value of the target region to the new target color value associated with the spatial point.

[0082] Traverse all spatial points to obtain the texture map.

[0083] In this embodiment of the invention, the mesh model is first unfolded into a planar image based on the normal vector and adjacency relationship of each facet in the mesh model. The "unfolding" here is not a one-time global parameterization, but a strategy selection based on preset quality parameters: when the quality parameter is set to high precision mode, local optimal unfolding can be performed on each facet individually, that is, each triangular facet is independently mapped to a dedicated rectangular area in the UV plane, and the size of the area is adaptively scaled according to its area and curvature to ensure that textured areas (such as boundary bands with drastic changes in normal vectors and sharp angles between adjacent faces) obtain higher pixel density; when the quality parameter is set to lightweight mode, multiple coplanar or nearly coplanar faces are jointly unfolded into the same UV block, and their gaps and white space are compressed, thereby accommodating a larger-scale mesh model within a limited texture size (such as 1024×1024).

[0084] The "unfolding" process records the precise correspondence between the position of each spatial point in the original grid and its target region on the planar image. For each spatial point, based on this correspondence, its target region on the planar image is located (this target region is not a fixed pixel coordinate, but a continuous coordinate range determined by the unfolding parameters of the patch where the spatial point is located, the interpolation weights within the patch, and the preset sampling density). The new target color value associated with this spatial point is written into the pixel position within its target region using bilinear interpolation or nearest neighbor method.

[0085] Since the new target color value is derived from the global texture map updated by S105, which is itself a corrected and consistent consensus, no matter how the target area is divided or whether the patches are spliced ​​across UV blocks, there will be no misalignment of seams caused by differences in the source of single images in the traditional preferred scheme, nor will there be local blurring caused by initial projection mismatch in the traditional fusion scheme.

[0086] Understandably, the embodiments of the present invention achieve dual decoupling in the texture mapping generation process: firstly, decoupling "color accuracy" from "unfolding method," so that high-quality color information is no longer limited by a fixed UV layout; secondly, decoupling "geometric details" from "resource overhead," allowing users to flexibly adjust quality parameters according to clinical application scenarios (such as initial screening requiring only an overall look, while precise diagnosis requires sub-millimeter textures), achieving a configurable balance between memory usage, loading speed, and rendering realism; the final output texture map not only fully retains the high-fidelity color structure obtained through closed-loop optimization, but also possesses the size controllability and memory friendliness required for engineering deployment, truly meeting the differentiated needs of intraoral scanning devices in real-time rendering on embedded platforms and high-precision reconstruction in the cloud.

[0087] To perform the corresponding steps in the above embodiments and various possible methods, an implementation of a texture mapping device 200 is given below. Further, please refer to... Figure 7 , Figure 7 This is a functional block diagram of a texture mapping device 200 provided in an embodiment of the present invention. It should be noted that the texture mapping device 200 provided in this embodiment has the same basic principle and technical effects as those in the above embodiments. For the sake of brevity, any parts not mentioned in this embodiment can be referred to the corresponding content in the above embodiments. The texture mapping device 200 includes: The acquisition module 201 is used to acquire the mesh model of the object under test and multiple acquired images.

[0088] The processing module 202 is used to perform point cloud conversion processing on the mesh model to obtain point cloud data; based on the pose of each acquired image, it obtains the mapping relationship between each acquired image and the point cloud data and the global texture map of the point cloud data; based on the global texture map, it corrects the mapping relationship between each acquired image and the point cloud data to obtain the corrected mapping relationship between each acquired image and the point cloud data; it updates the global texture map based on the corrected mapping relationship between each acquired image and the point cloud data to obtain the updated global texture map; and based on the updated global texture map, it converts the mesh model into a texture map.

[0089] Optionally, the above modules can be stored in the form of software or firmware. Figure 1 The memory 110 shown is either stored in or embedded in the operating system (OS) of the electronic device 100, and can be used by... Figure 1 The processor 120 executes the program. Meanwhile, the data and program code required to execute the above modules can be stored in the memory 110.

[0090] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative; for example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0091] In addition, the functional modules in the various embodiments of the present invention can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0092] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0093] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A texture mapping method, characterized in that, The method includes: Acquire the mesh model of the object under test and multiple acquired images; The mesh model is subjected to point cloud transformation processing to obtain point cloud data; Based on the pose of each acquired image, obtain the mapping relationship between each acquired image and the point cloud data, and the global texture map; Based on the global texture map, the mapping relationship between each acquired image and the point cloud data is corrected to obtain the corrected mapping relationship between each acquired image and the point cloud data; The global texture map is updated based on the corrected mapping relationship between each acquired image and the point cloud data to obtain the updated global texture map. Based on the updated global texture map, the mesh model is converted into a texture map.

2. The texture mapping method as described in claim 1, characterized in that, The step of performing point cloud transformation processing on the mesh model to obtain point cloud data includes: Based on the area of ​​each facet in the mesh model and the preset sampling interval, multiple spatial points are generated within each facet to obtain the point cloud data; the point cloud data includes all spatial points within the facets.

3. The texture mapping method as described in claim 1, characterized in that, The step of obtaining the mapping relationship between each acquired image and the point cloud data and the global texture map based on the pose of each acquired image includes: For each of the acquired images, the pixel coordinates of each spatial point in the point cloud data mapped to the acquired image are determined based on the pose of the acquired image. Based on the pixel coordinates of each spatial point mapped to the acquired image and the range of pixel coordinates in the acquired image, visible spatial points are determined, and the mapping relationship between the acquired image and the point cloud data is obtained; the mapping relationship between the acquired image and the point cloud data includes the pixel coordinates of each visible spatial point mapped to the acquired image, and the pixel coordinates of the visible spatial points mapped to the acquired image are located within the range of pixel coordinates in the acquired image; The global texture map is obtained based on the mapping relationship between each acquired image and the point cloud data.

4. The texture mapping method as described in claim 3, characterized in that, The step of obtaining the global texture map based on the mapping relationship between each acquired image and the point cloud data includes: For each spatial point, a first target image is determined from all the acquired images based on the mapping relationship between each acquired image and the point cloud data; the pixel coordinates of the spatial point mapped to the first target image are located within the pixel range of the first target image. Obtain the color value at the pixel coordinates of each of the spatial points mapped to the first target image; The color values ​​at the pixel coordinates of each first target image mapped to the spatial point are weighted and summed, and the sum is used as the target color value and associated with the spatial point. Traverse all the spatial points to obtain the global texture map; the global texture map includes the target color value associated with each spatial point.

5. The texture mapping method as described in claim 1, characterized in that, The step of correcting the mapping relationship between each acquired image and the point cloud data based on the global texture map to obtain the corrected mapping relationship between each acquired image and the point cloud data includes: For each of the acquired images, deformation control parameters for the acquired image are constructed based on the mapping relationship between the acquired image and the point cloud data and the global texture map; The acquired image is resampled using the deformation control parameters to obtain a resampled acquired image; Based on the resampled acquired image, the mapping relationship between the acquired image and the point cloud data is corrected to obtain the corrected mapping relationship between the acquired image and the point cloud data.

6. The texture mapping method as described in claim 5, characterized in that, The step of updating the global texture map based on the corrected mapping relationship between each acquired image and the point cloud data to obtain the updated global texture map includes: For each spatial point in the point cloud data, based on the corrected mapping relationship between each acquired image and the point cloud data, each second target image is determined from all corrected acquired images, wherein the pixel coordinates of the spatial point mapped to the second target image are within the pixel range of the second target image; Obtain the color value at the pixel coordinates of each second target image mapped from the spatial point; The color values ​​at the pixel coordinates of each second target image mapped from the spatial point are weighted and summed, and the sum is used as the new target color value and associated with the spatial point. Traverse all the said spatial points to obtain an updated global texture map, which includes the new target color value associated with each of the said spatial points.

7. The texture mapping method as described in claim 6, characterized in that, The step of converting the mesh model into a texture map based on the updated global texture map includes: Based on the normal vector and adjacency relationship of each facet in the mesh model, the mesh model is unfolded into a planar image; For each spatial point, based on the position of the spatial point in the mesh model, a target region on the planar image corresponding to the spatial point is determined, and the color value of the target region is set as a new target color value associated with the spatial point; By traversing all the aforementioned spatial points, the texture map is obtained.

8. A texture mapping device, characterized in that, The device includes: The acquisition module is used to acquire the mesh model of the object under test and multiple acquired images; The processing module is used to perform point cloud conversion processing on the mesh model to obtain point cloud data; based on the pose of each acquired image, obtain the mapping relationship between each acquired image and the point cloud data and the global texture map of the point cloud data; based on the global texture map, correct the mapping relationship between each acquired image and the point cloud data to obtain the corrected mapping relationship between each acquired image and the point cloud data; update the global texture map based on the corrected mapping relationship between each acquired image and the point cloud data to obtain the updated global texture map; and based on the updated global texture map, convert the mesh model into a texture map.

9. An electronic device, characterized in that, It includes a processor and a memory, the memory storing machine-executable instructions that can be executed by the processor to implement the texture mapping method according to any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the texture mapping method as described in any one of claims 1-7.

Images