Intraoral texture mapping method, device and equipment

Abstract

The application is suitable for the field of data processing, and provides an intraoral texture mapping method, device and equipment, wherein the method comprises the following steps: acquiring a multi-view image sequence collected by an intraoral scanner and camera parameters corresponding to each view image; for each triangular facet in a three-dimensional mesh model, screening qualified views capable of observing the current facet from the multi-view image sequence; based on the screened qualified views, constructing a candidate texture color set for each triangular facet; decomposing the candidate texture color set through a low-rank sparse decomposition model, eliminating abnormal color values in the candidate texture color set, and obtaining a reference color of each triangular facet; determining a target source view for each triangular facet from the qualified views according to the reference color and a view quality parameter; and performing texture mapping on the three-dimensional mesh model according to the target source views of all the triangular facets. The scheme effectively ensures the real-time effectiveness of texture mapping processing and the model reconstruction effect.

Description

Technical Field

[0001] This application belongs to the field of data processing, and in particular relates to a method, apparatus and device for intraoral texture mapping. Background Technology

[0002] In modern dental treatment, oral scanning technology, as an important digital aid, is gradually changing the clinical diagnosis and treatment process. Through 3D modeling and texture acquisition technology, scanning systems can acquire high-precision geometric and realistic texture information of the internal structures of the oral cavity, providing doctors with visualized and quantitative diagnostic information. This texture information is particularly crucial for identifying details such as dental caries, enamel defects, and early mucosal lesions, greatly improving the efficiency and accuracy of diagnosis and treatment.

[0003] Thanks to advancements in sensor hardware, mainstream dental scanning devices can now operate stably at frame rates of 20Hz or even higher, achieving near real-time data acquisition. However, building a truly high-precision real-time dental scanning system requires not only focusing on data acquisition speed but also addressing the significant computational and graphics optimization challenges in the post-processing stage. Particularly in the texture mapping phase, current research primarily concentrates on offline rendering schemes after mesh completion, lacking real-time and high-quality texture mapping processing solutions. Summary of the Invention

[0004] This application provides an intraoral texture mapping method, apparatus, and device to address the problem of the lack of real-time and effective texture mapping processing solutions in the prior art.

[0005] A first aspect of this application provides an intraoral texture mapping method, including:

[0006] Acquire multi-view image sequences and camera parameters corresponding to each view image from an intraoral scanner;

[0007] Based on the camera parameters, for each triangular facet in the 3D mesh model, qualified views that can observe the current facet are selected from the multi-view image sequence;

[0008] Based on the selected qualified views, a candidate texture color set is constructed for each of the triangular facets;

[0009] The candidate texture color set is decomposed using a low-rank sparse decomposition model, and abnormal color values ​​in the candidate texture color set are removed to obtain the base color of each triangular facet.

[0010] Based on the reference color and viewing angle quality parameters, a target source view is determined from the qualified view for each of the triangular facets;

[0011] Texture mapping is performed on the 3D mesh model based on the target source view of all the said triangular facets.

[0012] A second aspect of this application provides an intraoral texture mapping apparatus, comprising:

[0013] The acquisition module is used to acquire multi-view image sequences and camera parameters corresponding to each view image acquired by the intraoral scanner.

[0014] A filtering module is used to filter out qualified views from the multi-view image sequence that can observe the current facet for each triangular facet in the 3D mesh model based on the camera parameters.

[0015] The construction module is used to construct a candidate texture color set for each of the three triangular faces based on the selected qualified views;

[0016] The elimination module is used to decompose the candidate texture color set through a low-rank sparse decomposition model, eliminate abnormal color values ​​in the candidate texture color set, and obtain the base color of each triangular facet.

[0017] The determination module is used to determine a target source view from the qualified views for each of the triangular facets based on the reference color and view quality parameters;

[0018] An execution module is used to perform texture mapping on the 3D mesh model based on the target source view of all the said triangular facets.

[0019] A third aspect of this application provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method described in the first aspect.

[0020] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method described in the first aspect.

[0021] The fifth aspect of this application provides a computer program product including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in an electronic device, a processor in the electronic device performs the steps of the method described in the first aspect above.

[0022] In this embodiment of the application, the above-described scheme acquires a multi-view image sequence and camera parameters corresponding to each view image from an intraoral scanner. Based on these camera parameters, for each triangular facet in the 3D mesh model, qualified views that can observe the current facet are selected from the multi-view image sequence. Based on the selected qualified views, a candidate texture color set is constructed for each triangular facet. The candidate texture color set is decomposed using a low-rank sparse decomposition model to remove abnormal color values ​​from the candidate texture color set, thereby obtaining the base color of each triangular facet. According to the base color and view quality parameters, a target source view is determined for each triangular facet from the qualified views. Based on the target source views of all triangular facets, texture mapping is performed on the 3D mesh model, effectively ensuring the real-time effectiveness of texture mapping processing and the model reconstruction effect. Attached Figure Description

[0023] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiments below. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0024] Figure 1 This is a flowchart of an intraoral texture mapping method according to some embodiments of this application. Figure 1 ;

[0025] Figure 2 This is a flowchart of an intraoral texture mapping method according to some embodiments of this application. Figure 2 ;

[0026] Figure 3 These are structural diagrams of an intraoral texture mapping apparatus according to some embodiments of this application;

[0027] Figure 4 This is a structural diagram of the computer device provided in the embodiments of this application. Detailed Implementation

[0028] The embodiments of the technical solution of this application will now be described in detail with reference to the accompanying drawings. These embodiments are only used to more clearly illustrate the technical solution of this application and are therefore merely examples, and should not be used to limit the scope of protection of this application.

[0029] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the application; the terms “comprising” and “having”, and any variations thereof, in the specification, claims, and foregoing description of the drawings are intended to cover non-exclusive inclusion.

[0030] In the description of the embodiments of this application, technical terms such as "first" and "second" are used only to distinguish different objects and should not be construed as indicating or implying relative importance or implicitly specifying the number, specific order, or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly defined.

[0031] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0032] In the description of the embodiments in this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0033] Currently, there are two common texture rendering algorithms in computer graphics: unfolded texture mapping and non-unwrap texture fusion. The core idea of ​​unfolded texture mapping is to unfold the 3D mesh structure into a 2D space, establishing a UV coordinate mapping relationship for each vertex, thereby achieving an accurate correspondence between the image texture and the mesh. Common unfolding algorithms include Least Squares Conformal Mapping (LSCM), Tutte Mapping, and angle-based enhanced unfolding algorithms (ABF++), which can preserve the angular characteristics of the mesh and control surface deformation. This method is widely used in offline rendering, 3D printing, and virtual reality scenes, with advantages such as strong compatibility, support for general rendering pipelines and hardware acceleration, and ease of post-editing and compression. However, in medical scenes or intraoral scanning systems, this method has high requirements for surface complexity and time efficiency. The unfolding process is not only computationally expensive, but also prone to stretching or overlapping under high curvature or non-manifold structures, affecting texture accuracy and visual consistency. Therefore, it is usually used in situations where the model structure is stable and texture requirements are highly controllable.

[0034] Texture fusion without mesh unwrapping bypasses the traditional UV unwrapping process, directly projecting texture information onto the surface of a 3D model using multi-view images, and constructing the final texture map through fusion or selection mechanisms. In this method, each triangular facet is evaluated for reliability across multiple views based on metrics such as geometric visibility, image projection area, and color consistency. A fusion strategy (e.g., multi-band pyramid fusion, Poisson image blending) or a selection strategy (e.g., a scoring mechanism to select the best image) is then used to generate the texture map. This approach is particularly suitable for dental scanning because tooth surfaces are complex and highly reflective; direct mapping avoids the facet stretching problem during unwrapping. However, challenges exist: fusion methods may produce ghosting, especially at view edges; while selection methods are prone to creating seams at image transitions, affecting continuity and visual uniformity. For applications involving high-frequency data streams and dynamic modeling, this type of method is becoming increasingly popular.

[0035] While "unfolded texture mapping" is widely used in current dental scanning techniques, it still faces several technical bottlenecks in high-precision tooth structure modeling. This method is prone to local stretching and patch overlap during UV unwrapping, especially for dental meshes with complex curvatures and non-manifold structures. The unwrapping process is not only computationally complex but also relies on stable topological relationships, severely limiting its efficiency in real-time scanning scenarios. Furthermore, the atlas packaging stage typically employs heuristic rectangular arrangement algorithms, which have limited space utilization and gap control capabilities, easily leading to atlas size redundancy or edge occlusion.

[0036] While non-expansion-based texture fusion strategies effectively reduce UV processing overhead and can quickly project image textures onto the model surface, they face several technical obstacles in image selection and fusion. On the one hand, multi-view fusion methods are prone to introducing ghosting, especially in areas with inconsistent exposure or large angle deviations, and the fusion algorithm lacks robustness to changes in illumination. On the other hand, although image optimization strategies can avoid ghosting problems, they can produce significant seams at image transitions, affecting texture continuity and visual consistency.

[0037] This application proposes several intraoral texture mapping processing schemes that can accurately remove and seamlessly integrate color anomalies in multiple views while ensuring real-time performance, thereby improving the stability of texture mapping and the quality of 3D model reconstruction.

[0038] To illustrate the technical solution described in this application, specific embodiments are provided below.

[0039] Combination Figure 1 As shown, in some embodiments, an intraoral texture mapping method is proposed, including:

[0040] Step 101: Obtain the multi-view image sequence acquired by the intraoral scanner and the camera parameters corresponding to each view image.

[0041] The camera parameters may include, for example, camera intrinsic parameters, camera extrinsic parameters, camera line of sight, etc., and may also include the camera projection matrix obtained by multiplying the camera extrinsic parameters and the pre-calibrated camera intrinsic parameters to obtain images at each viewpoint.

[0042] A multi-view image sequence includes intraoral images acquired from various perspectives. When acquiring multi-angle oral cavity images, in addition to obtaining the corresponding camera pose, image parameters such as illumination parameters and depth data can also be obtained.

[0043] An intraoral scanner can be equipped with sensor or camera modules capable of acquiring images from multiple perspectives. For example, multiple sensors or cameras can be used, positioned at different locations, or the sensor heads and camera lenses can be adjusted to face different directions for image acquisition. This ensures the acquisition of high-resolution RGB images of the oral cavity from various angles, and depth data can also be acquired simultaneously.

[0044] Optionally, each frame in the multi-view image sequence is calibrated to obtain the camera intrinsic parameter matrix K, and the extrinsic parameter matrix is ​​obtained by combining the rotation matrix R and the translation vector T. The product of intrinsic and extrinsic parameters yields the camera projection matrix. The camera projection matrix is ​​used to subsequently project 3D points in the 3D mesh model onto pixel space to achieve mapping.

[0045] Alternatively, noise and specular highlights in individual images can be suppressed through distortion correction and pre-filtering.

[0046] Optionally, the resulting multi-view image sequence is represented as I represents image brightness. These are the image coordinates.

[0047] Step 102: Based on the camera parameters, for each triangular facet in the 3D mesh model, select qualified views from the multi-view image sequence that can observe the current facet.

[0048] The three-dimensional mesh model is a mesh model corresponding to the internal structure of the oral cavity. This model includes multiple triangular facets, which are the basic units of the mesh in the three-dimensional mesh model.

[0049] Alternatively, a complex 3D mesh model (such as a tooth) is composed of a large number of tiny triangles "stitched together". Each triangle is defined by three vertices. The entire 3D mesh model consists of these vertices and the topological relationships that define how they connect to form triangles.

[0050] Optionally, based on the camera's line-of-sight direction, camera pose, and camera projection matrix when the camera acquires images, it can be determined which triangular facets in the multi-view image sequence are observable and which are occluded and unobservable. Observable triangular facets have a projection area in the corresponding viewpoint image and a corresponding texture color; otherwise, they do not.

[0051] In an optional implementation, step 102, based on the camera parameters, for each triangular facet in the 3D mesh model, filters qualified views from the multi-view image sequence that allow observation of the current facet, including:

[0052] For each of the triangular facets, calculate the angle between the normal vector of the triangular facet and the camera's line of sight corresponding to each view image, and retain the first view from the multi-view image sequence whose angle is less than a threshold.

[0053] Using image depth data, a depth test is performed on each of the first views to remove views in which the projection area of ​​the triangular facet is obscured, thus obtaining the second view.

[0054] Calculate the contrast of the projected area in the second view, and discard views whose contrast is not within the set range to obtain the qualified view.

[0055] In the optional implementation process, images that are severely mislit or almost invisible need to be removed from all captured views.

[0056] When retaining the first view from the multi-view image sequence based on the aforementioned included angle, in an optional implementation, for any triangular facet, the cosine of the angle between the normal vector of the current triangular facet and the camera's line-of-sight direction of the intraoral texture image acquired from each angle is calculated. Images with a cosine angle less than a set value are discarded, and images with a cosine angle greater than the set value (the first view) are retained. The larger the cosine angle, the smaller the included angle.

[0057] In the optional implementation, the normal vector of each triangular facet... relative to the camera's line of sight Calculate the cosine of the included angle:

[0058] ,reserve Views that are too far back or sideways are excluded. This is a threshold value, which can be, for example, 0.3 or other values.

[0059] When performing depth tests on each first view using image depth data, in an optional implementation process, triangular patches can be projected onto each first view to determine the projection area of ​​the triangular patches. Based on the depth map of each first view, views whose pixels are occluded in the projection area are removed from each first view, and the remaining views (second views) are retained.

[0060] During implementation, based on the depth maps of each first view, views where pixels in the projection area are occluded are removed from the first views, and the remaining views are retained. This can be done by:

[0061] For each first view, for each pixel in the projection area of ​​the first view, obtain the theoretical depth value of the current triangular facet in the current pixel (which can be calculated by interpolation). From the depth map of the first view (i.e. the aforementioned depth information), determine the actual depth value of the current pixel (this value represents the depth of the object closest to the camera when this view is taken). If the theoretical depth value of the current pixel is greater than the actual depth value (meaning that the object is farther away from the camera than the object closest to the camera), then the current pixel is determined to be an occluded pixel. If the theoretical depth value of the current pixel is equal to or deviates from the actual depth value within a set range, then the current pixel is determined to be a visible pixel. Remove the view whose projection area contains the occluded pixel from the first view and retain the remaining view (second view).

[0062] This process effectively determines whether a triangular facet is obscured by other structures in each view, effectively eliminates views obscured by other structures, and ensures that the current facet is visible.

[0063] When calculating the contrast of the projected area in the second view and discarding views whose contrast is not within the set range to obtain a qualified view, the optional implementation process may be as follows:

[0064] Based on the Laplacian coefficient of variation of each second view, the contrast of the projected area in each second view is determined. From multiple second views, the view with a contrast less than the upper limit and greater than the lower limit is selected as the qualified view of the current triangular facet, and the blurred or overexposed views are discarded.

[0065] The Laplacian coefficient of variation of an image can be , I is the Laplacian operator operated on the image I. It is a standard deviation calculation; this measures the contrast, discarding views with too low or too high contrast to ensure that the next step is a high-quality view.

[0066] The contrast of the projection area is used to measure the sharpness of local details and edges in an image. In this embodiment, a range of contrast values ​​is set. When the contrast is too low, the image is blurry, out of focus, and lacks effective texture details. When the contrast is too high, the image is overexposed, has specular highlights, or is subject to noise interference, resulting in drastic changes in local pixel values. The true texture information of these areas may be lost.

[0067] Through the above processing, a set of high-quality, visible, and qualified views are determined for each patch.

[0068] Step 103: Based on the selected qualified views, construct a candidate texture color set for each of the triangular facets.

[0069] Each triangular facet has a candidate texture color set. The candidate texture color set includes candidate colors for the projected regions of the current triangular facet in the corresponding qualified views. Each qualified view corresponds to one candidate color. This candidate color can be a color determined based on artificial intelligence processing and adapted to the current intraorific structure; it can be the average pixel value of the projected regions of the triangular facet in the qualified views; or it can be a color value selected through other methods.

[0070] In an optional implementation, step 103, based on the selected qualified views, constructs a candidate texture color set for each of the triangular facets, including:

[0071] Based on each of the triangular facets, the average pixel value of the projection area of ​​the triangular facet in each of the corresponding qualified views is calculated to obtain the candidate color of the projection area in each of the qualified views;

[0072] For each of the triangular facets, the candidate colors of the corresponding qualified views are divided into the same set to obtain the candidate texture color set for each of the triangular facets.

[0073] During implementation, the pixel positions of the vertices of the triangular facets mapped to different qualified views can be determined based on the camera projection matrix, and the region defined by the pixel positions is defined as the projection region. For each triangular facet, the average pixel value of the projection region on each corresponding qualified view is calculated as a candidate color, forming a candidate texture color set for each triangular facet.

[0074] Optionally, for each vertex of a triangle Application of camera projection matrix Get pixel position Then, the projection area of ​​the triangular facet onto the image plane of the qualified view. Internal sampling of all pixels to calculate the average color:

[0075]

[0076] This allows the generation of an RGB value for the triangular facet in the i-th qualified view. It enables the acquisition of the average color of all pixels in the projection area of ​​the triangular facet in each qualified view as a candidate color. The average colors of all qualified views are used to form a candidate color list, thereby forming a candidate texture color set for the candidate colors of the projection areas of each triangular facet in its corresponding multiple qualified views, which will be used for subsequent color anomaly removal.

[0077] Step 104: Decompose the candidate texture color set using a low-rank sparse decomposition model, remove abnormal color values ​​from the candidate texture color set, and obtain the base color of each triangular facet.

[0078] This base color serves as a reference factor for subsequently assigning target source views to each triangular facet.

[0079] By decomposing the candidate texture color set using a low-rank sparse decomposition model, the low-rank part and the sparse outlier part can be obtained. By removing the outlier color values, the base color of each triangular facet can be obtained.

[0080] In an optional implementation, step 104 decomposes the candidate texture color set using a low-rank sparse decomposition model, removes outlier color values ​​from the candidate texture color set, and obtains the base color of each triangular facet, including:

[0081] For each of the triangular facets, the candidate texture color sets are numerically stacked to form a candidate color matrix;

[0082] The candidate color matrix is ​​decomposed into a low-rank part and a sparse outlier part.

[0083] The objective function is to minimize the low-rank portion and the sparse outlier portion, and the mean value of all rows in the low-rank portion is used as the base color of the triangular facet.

[0084] In an optional implementation, the n candidate colors from the aforementioned candidate texture color set can be stacked into a matrix. Robust Principal Component Analysis (Robust PCA) is applied to decompose it into a low-rank component and a sparse outlier component. The objective function is designed as follows:

[0085]

[0086] in, For the low-rank part constrained by the nuclear norm , Constraints on sparse anomalies .in, For the candidate color matrix, The lower-rank part, This is the sparse anomaly. These are the weight parameters.

[0087] The above objective function is iterated using the Inexact ALM algorithm:

[0088]

[0089] Where Y is the Lagrange multiplier, and μ is the penalty parameter (i.e., the step size factor of the augmented Lagrange multiplier method). SVDshrink (singular value soft thresholding) performs singular value decomposition (SVD) on the matrix, then soft-thresholds the singular values ​​to suppress smaller singular values, thus approximating a low-rank matrix. Shrink (element-by-element soft thresholding) soft-thresholds each element of the matrix individually, suppressing small-amplitude noise while retaining large-amplitude outliers (i.e., sparse components). The superscripts of the parameters represent different iteration numbers.

[0090] Final decision Mean of all rows: As the net color (i.e., the reference color) of the current triangular facet, it can strictly identify and remove outliers in the candidate texture color set, automatically remove shadows and highlights, remove abnormal light spots, and obtain a reference color that is closer to the real reflectivity, ensuring high image processing accuracy and fewer artifacts.

[0091] Step 105: Based on the reference color and viewing angle quality parameters, determine a target source view from the qualified views for each triangular facet.

[0092] The criteria for assigning a target source view to each triangular facet may include: camera viewing angle, the difference between the base color and candidate colors, and the projected area of ​​the triangular facet in the qualified views. Based on these criteria, each qualified view corresponding to each triangular facet is scored, and then one view is selected as the target source view.

[0093] Optionally, the degree of similarity between the candidate color and the reference color of the triangular facet is positively correlated with the score. The size of the camera viewing angle is negatively correlated with the score. The size of the projected area of ​​the triangular facet in the qualified view is positively correlated with the score.

[0094] In an optional implementation, determining a target source view from the qualified views for each of the triangular facets based on the reference color and viewing angle quality parameters includes:

[0095] Obtain the view quality parameters of each qualified view corresponding to each of the triangular facets. The view quality parameters include: the angle between the normal vector of the triangular facet and the camera line of sight corresponding to the qualified view, and the area of ​​the projection region of each triangular facet in the corresponding qualified view.

[0096] Obtain the deviation color value between the reference color of each of the triangular facets and the candidate color of the corresponding qualified view;

[0097] Based on the view quality parameters and the deviation color values, a quality score is determined for each qualified view corresponding to each triangular facet.

[0098] Based on the quality score, the target source view is selected from the corresponding qualified views for each of the triangular facets.

[0099] The angle between the normal vector of the triangular facet and the camera's line of sight corresponding to the qualified view can measure whether the shooting angle of the qualified view is a direct frontal shot, avoiding views with excessive back or side views, ensuring the visibility of the facet, and at the same time, the more frontal the viewing angle, the smaller the perspective distortion, ensuring the view quality.

[0100] The deviation color value can specifically be the difference between the RGB value of the reference color and the RGB value of the candidate color. The color quality of a single qualified view is measured by the deviation color value between the color of the single qualified view and the reference color determined by the candidate texture color set. This effectively eliminates highlights and shadows, retains views with moderate exposure and consistent colors, and effectively solves the problem of color jumps caused by differences in white balance and exposure between multiple views, ensuring the color consistency (exposure consistency) of the selected views.

[0101] The larger the area of ​​the triangular facet projected onto each of the corresponding qualified views, the higher the texture resolution, the richer the details, and the better the clarity in the view, ensuring the texture resolution and clarity of the selected view.

[0102] In the optional implementation process, for each qualified view of a patch, the view quality score is calculated using the following formula to obtain the overall score:

[0103]

[0104] in, Let cosine be the angle between the viewpoint (i.e., the angle between the normal vector of the triangular facet and the camera's line of sight corresponding to the qualified view). As the base color, For the i-th qualified view, The area of ​​the projected region. , , Set coefficients (which can be set and adjusted according to needs).

[0105] Based on this comprehensive score, the highest-scoring qualified view can be selected as the target source view. This allows for the further selection of views with better image quality for each triangular facet from the qualified views, ensuring that the target source view selected for each triangular facet has good facet visibility, sharpness, and exposure consistency.

[0106] In an optional implementation, it may further include:

[0107] Construct an adjacency graph to reflect the adjacency relationships between the triangular facets;

[0108] Based on the adjacency graph, when the target source view of the triangular facet is different from that of the adjacent facet, the number of adjacent facets with different target source views is obtained;

[0109] If the number exceeds a set ratio, the target source view of the triangular facet is replaced with the target source view of the adjacent facets, which also exceed the set ratio.

[0110] Optionally, if the number does not exceed the set ratio, the target source view of the triangular facet is retained.

[0111] The above process involves selecting the target source view for the triangular facets, constructing a facet adjacency graph, and replacing small patches using different views if a triangular facet uses the same view as most of its neighbors. This achieves neighborhood and global color smoothing, eliminates small-scale abrupt changes, and ensures neighborhood consistency in the view effect.

[0112] In an optional implementation, it may further include:

[0113] For each of the triangular facets, separate the illumination component and the reflection component in the target source view;

[0114] The illumination component is removed, and the reflection component is enhanced to obtain the updated target source view.

[0115] In the optional implementation process, all target source views are considered. The reflectance component of each image is calculated using the single-scale Retinex algorithm:

[0116]

[0117] in, For Gaussian kernel; This is a convolution operation; j={1,2,3} represents the red, green, and blue channels respectively. This is a reflectance estimate obtained based on the Retinex visual theory algorithm; This is the enhanced output of the j-th channel.

[0118] The above operation separates the illumination component and the reflection component. Based on this, the illumination component can be eliminated and the reflection component can be enhanced to restore a color image with consistent brightness. The color tone and brightness of multiple views can be corrected to solve the problem of inconsistent exposure in multiple views and achieve global color correction.

[0119] Step 106: Apply texture mapping to the 3D mesh model based on the target source view of all the triangular facets.

[0120] The above-described embodiments of this application acquire multi-view image sequences and camera parameters corresponding to each view image from an intraoral scanner. Based on these camera parameters, for each triangular facet in the 3D mesh model, qualified views that can observe the current facet are selected from the multi-view image sequence. Based on the selected qualified views, a candidate texture color set is constructed for each triangular facet. The candidate texture color set is decomposed using a low-rank sparse decomposition model to remove abnormal color values ​​from the candidate texture color set, thereby obtaining the base color of each triangular facet. According to the base color and view quality parameters, a target source view is determined for each triangular facet from the qualified views. Based on the target source views of all triangular facets, texture mapping is performed on the 3D mesh model, effectively ensuring the real-time effectiveness of texture mapping processing and the model reconstruction effect.

[0121] In one optional implementation, combined with Figure 2 As shown, step 106 involves texture mapping of the 3D mesh model based on the target source view of all the triangular facets, including:

[0122] Step 201: For each of the triangular facets, extract the corresponding texture region from the target source view;

[0123] Step 202: Pack all texture regions into a single texture atlas;

[0124] Step 203: On the texture atlas, feathering and blending processing is performed on the seams between the texture regions from different views;

[0125] Step 204: Control the shader to perform texture mapping on the 3D mesh model based on the fused texture atlas.

[0126] All texture regions are packaged into a single texture atlas, using a rectangular arrangement algorithm. Feathering and blending can be performed in real-time in the GPU shader based on the calculated distance field.

[0127] Extracting the corresponding texture region from the target source view can be achieved by determining the projection area of ​​the triangular facet in the target source view, where the projection area has a texture color, and the area where the texture color is located is the texture region.

[0128] In an optional implementation, the size of the texture block corresponding to each triangle can be determined based on the texture region of each triangle in the target source view. A two-dimensional binning algorithm is used to calculate the placement coordinates of each texture block in the texture atlas according to the size. Based on the placement coordinates, the texture color corresponding to each texture region is written into the texture atlas. On the texture atlas, the seams between texture regions from different views are feathered and blended.

[0129] The above placement coordinates are associated with the corresponding triangular facets. The shader reads the texture color corresponding to each triangular facet from the texture atlas based on the association relationship, and performs texture rendering on the corresponding triangular facets of the 3D spatial model based on the texture color.

[0130] In an optional implementation, the textured area of ​​each triangular facet is... Sort the textures in descending order of height, and then use the MaxRects algorithm for 2D binning. Specifically, this involves continuously selecting the best-fitting rectangular gaps in the remaining space of the texture atlas and placing them into the current texture region, dynamically updating the list of free rectangles, until all regions are filled with the smallest square texture atlas, maximizing utilization and minimizing blank space.

[0131] In an exemplary implementation of a texture atlas packaging algorithm, the region list formed by the texture regions of all triangular facets and the side length of the texture atlas can be used as input to the algorithm model. These texture regions are sorted in descending order of region height, and a list of free rectangles and a list of initial placement results are initialized. The region list is then traversed to find the optimal free rectangle.

[0132] When the optimal free rectangle is found, the placement position is recorded in the placement result list, the free rectangle list is updated, and rectangles with an inclusion relationship in the free rectangle list are merged and removed. When the optimal free rectangle is not found, the side length of the atlas is multiplied by 2, the algorithm model is recursively called, and the step of finding the optimal free rectangle is returned.

[0133] In the above process, after merging and removing rectangles with an inclusive relationship in the list of free rectangles, it can also be determined whether all texture regions have been processed. If they have been processed, the placement result list and the side length of the texture atlas are output. If they have not been processed, the region list is traversed until all regions are processed, thus achieving efficient packaging of texture atlases and saving memory.

[0134] In some alternative implementations, the shader can perform feathering and blending of the seam based on the distance field.

[0135] Mark the seam edges from different view sources on the texture atlas to generate a binary mask M(x,y). Approximate the range field using the jump flooding algorithm.

[0136]

[0137] And in the fragment shader Apply linear blending to adjacent color regions:

[0138]

[0139] in, For distance field, The mixing coefficient; It is the color of the view from one side of the gap. It is the color of the view from the other side of the gap. The final color after feathering and blending treatment of the seam.

[0140] Through the above processing, the seam edges are feathered smoothly using a distance field in the shader, which eliminates the seams and suppresses ghosting. At the same time, only a few texture lookups and interpolations are required to meet the needs of real-time rendering.

[0141] Compared to traditional texture mapping methods that require mesh unpacking and offline stitching, the above-described implementation of this application integrates the multi-view texture mapping process into a real-time algorithm that is unpacked and parallelizable end-to-end. Through rapid candidate view filtering and patch projection sampling, low-rank plus sparse decomposition is used to automatically remove shadows and specular artifacts. Combined with neighborhood consistency filtering and Retinex global color correction, it ensures that each patch obtains a clean and uniform reflectivity color. At the same time, a compact packing texture atlas algorithm is used to save memory, and distance field feathering is used in the fragment shader to replace the time-consuming Poisson fusion, eliminating stitching and ghosting in one go. The overall process balances computational efficiency and rendering quality, achieving high frame rate, high precision, and seamless texture mapping processing effects in oral scanning.

[0142] The intraoral texture mapping algorithm provided in this application can complete high-precision texture mapping and compact packaging in real time without mesh unpacking, providing intuitive color information for subsequent medical diagnosis. The algorithm process covers steps such as candidate view acquisition, low-rank decomposition and elimination, neighborhood optimization, atlas generation and edge feathering, taking into account both computational efficiency and texture quality, effectively eliminating ghosting and gaps, and improving visual consistency.

[0143] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0144] Based on the same inventive concept, this application also provides an intraoral texture mapping device. The intraoral texture mapping device provided in this application can implement the various processes of the above-described intraoral texture mapping method embodiments and achieve the same technical effect. Therefore, the specific limitations of one or more intraoral texture mapping device embodiments provided below can be found in the limitations of the intraoral texture mapping method above. To avoid repetition, they will not be repeated here.

[0145] This embodiment can divide the computing side into functional modules according to the above method. For example, it can be divided into functional modules corresponding to each function, or two or more functions can be integrated into one processing module.

[0146] See Figure 3 , Figure 3 This is a structural diagram of an intraoral texture mapping device provided in an embodiment of this application. For ease of explanation, only the parts related to the embodiment of this application are shown.

[0147] The intraoral texture mapping device 300 includes:

[0148] The acquisition module 301 is used to acquire the multi-view image sequence and the camera parameters corresponding to each view image acquired by the intraoral scanner.

[0149] The filtering module 302 is used to filter out qualified views from the multi-view image sequence that can observe the current facet for each triangular facet in the three-dimensional mesh model based on the camera parameters.

[0150] Module 303 is configured to construct a candidate texture color set for each of the three triangular faces based on the selected qualified views;

[0151] The elimination module 304 is used to decompose the candidate texture color set through a low-rank sparse decomposition model, eliminate abnormal color values ​​in the candidate texture color set, and obtain the base color of each triangular facet.

[0152] The determination module 305 is used to determine a target source view from the qualified views for each of the triangular facets based on the reference color and view quality parameters.

[0153] The execution module 306 is used to perform texture mapping on the three-dimensional mesh model based on the target source view of all the triangular facets.

[0154] Optionally, the filtering module 302 is specifically used for:

[0155] For each of the triangular facets, calculate the angle between the normal vector of the triangular facet and the camera's line of sight corresponding to each view image, and retain the first view from the multi-view image sequence whose angle is less than a threshold.

[0156] Using image depth data, a depth test is performed on each of the first views to remove views in which the projection area of ​​the triangular facet is obscured, thus obtaining the second view.

[0157] Calculate the contrast of the projected area in the second view, and discard views whose contrast is not within the set range to obtain the qualified view.

[0158] Optionally, module 303 is constructed specifically for:

[0159] Based on each of the triangular facets, the average pixel value of the projection area of ​​the triangular facet in each of the corresponding qualified views is calculated to obtain the candidate color of the projection area in each of the qualified views;

[0160] For each of the triangular facets, the candidate colors of the corresponding qualified views are divided into the same set to obtain the candidate texture color set for each of the triangular facets.

[0161] Optionally, module 304 is excluded, specifically for:

[0162] For each of the triangular facets, the candidate texture color sets are numerically stacked to form a candidate color matrix;

[0163] The candidate color matrix is ​​decomposed into a low-rank part and a sparse outlier part.

[0164] The objective function is to minimize the low-rank portion and the sparse outlier portion, and the mean value of all rows in the low-rank portion is used as the base color of the triangular facet.

[0165] Optionally, module 305 is specifically used for:

[0166] Obtain the view quality parameters of each qualified view corresponding to each of the triangular facets. The view quality parameters include: the angle between the normal vector of the triangular facet and the camera line of sight corresponding to the qualified view, and the area of ​​the projection region of each triangular facet in the corresponding qualified view.

[0167] Obtain the deviation color value between the reference color of each of the triangular facets and the candidate color of the corresponding qualified view;

[0168] Based on the view quality parameters and the deviation color values, a quality score is determined for each qualified view corresponding to each triangular facet.

[0169] Based on the quality score, the target source view is selected from the corresponding qualified views for each of the triangular facets.

[0170] Optionally, it also includes:

[0171] Replacement module, used for:

[0172] Construct an adjacency graph to reflect the adjacency relationships between the triangular facets;

[0173] Based on the adjacency graph, when the target source view of the triangular facet is different from that of the adjacent facet, the number of adjacent facets with different target source views is obtained;

[0174] If the number exceeds a set ratio, the target source view of the triangular facet is replaced with the target source view of the adjacent facets, which also exceed the set ratio.

[0175] Optionally, execution module 306 is specifically used for:

[0176] For each of the triangular facets, extract the corresponding texture region from the target source view;

[0177] Pack all texture regions into a single texture atlas;

[0178] On the texture atlas, the seams between the texture regions originating from different views are feathered and blended.

[0179] The control shader performs texture mapping on the 3D mesh model based on the blended texture atlas.

[0180] Optionally, it also includes:

[0181] The update module is used for:

[0182] For each of the triangular facets, separate the illumination component and the reflection component in the target source view;

[0183] Based on the target source view, the illumination component is removed and the reflection component is enhanced to obtain the updated target source view.

[0184] The integrated modules described above can be implemented in hardware. It should be noted that the module division in this embodiment is illustrative and represents only one logical functional division; in actual implementation, other division methods may be used.

[0185] It should be noted that the relevant content of each step involved in the above method embodiments can be referenced from the functional description of the corresponding functional module, and will not be repeated here.

[0186] In one embodiment, such as Figure 4 As shown, a computer device is provided. The computer device 4 of this embodiment includes: at least one processor 400 (… Figure 4 (Only one is shown in the diagram), memory 401, and computer program 402 stored in said memory 401 and executable on said at least one processor 400, which, when executed by said processor 400, implements the steps in any of the above method embodiments.

[0187] The computer device 4 can be a desktop computer, laptop, handheld computer, or other computing device. The computer device 4 may include, but is not limited to, a processor 400 and a memory 401. Those skilled in the art will understand that... Figure 4 This is merely an example of computer device 4 and does not constitute a limitation on computer device 4. It may include more or fewer components than shown, or combine certain components, or different components. For example, the computer device may also include input / output devices, network access devices, buses, etc.

[0188] The processor 400 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0189] The memory 401 can be an internal storage unit of the computer device 4, such as a hard disk or memory of the computer device 4. The memory 401 can also be an external storage device of the computer device 4, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the computer device 4. Furthermore, the memory 401 can include both internal and external storage units of the computer device 4. The memory 401 is used to store the computer program and other programs and data required by the computer device. The memory 401 can also be used to temporarily store data that has been output or will be output.

[0190] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0191] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0192] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0193] In the embodiments provided in this application, it should be understood that the disclosed apparatus / computer devices and methods can be implemented in other ways. For example, the apparatus / computer device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings or direct couplings or communication connections may be through some interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0194] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0195] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0196] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0197] The methods described in this application can be implemented in whole or in part by a computer program product. When the computer program product is run on a computer device, the computer device executes the steps in the various method embodiments described above.

[0198] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. An intraoral texture mapping method, characterized by, The method comprises the following steps: obtaining a multi-view image sequence collected by an intraoral scanner and camera parameters corresponding to each view image; based on the camera parameters, for each triangular facet in the three-dimensional mesh model, screening out qualified views that can observe the current facet from the multi-view image sequence; based on the screened qualified views, constructing a candidate texture color set for each triangular facet; by a low-rank sparse decomposition model, decomposing the candidate texture color set, eliminating abnormal color values in the candidate texture color set, and obtaining a reference color of each triangular facet; obtaining a view quality parameter of each qualified view corresponding to each triangular facet, the view quality parameter including: an angle between a normal vector of the triangular facet and a camera line-of-sight direction corresponding to the qualified view, and a projection area size of each triangular facet in the corresponding qualified view; obtaining a deviation color value of the reference color of each triangular facet and the candidate color of the corresponding qualified view; based on the view quality parameter and the deviation color value, determining a quality score for each qualified view corresponding to each triangular facet; based on the quality score, selecting a target source view for each triangular facet from the corresponding qualified views; according to the target source view of all the triangular facets, performing texture mapping on the three-dimensional mesh model.

2. The method of claim 1, wherein, The method comprises the following steps: for each triangular facet, calculating an angle between a normal vector of the triangular facet and a camera line-of-sight direction corresponding to each view image, and retaining a first view whose angle is less than a threshold value from the multi-view image sequence; using image depth data, performing a depth test on each first view to eliminate a view in which a projection area of the triangular facet is blocked in the first view, and obtaining a second view; calculating a contrast of the projection area in the second view, and eliminating a view whose contrast is not within a set range, and obtaining the qualified view.

3. The method of claim 1, wherein, The method comprises the following steps: based on each triangular facet, calculating a pixel average value of a projection area of the triangular facet in the corresponding qualified view, and obtaining a candidate color of the projection area in each qualified view; for each triangular facet, dividing the candidate colors of the corresponding qualified views into the same set, and obtaining a candidate texture color set of each triangular facet.

4. The method of claim 1, wherein, The method comprises the following steps: for each triangular facet, stacking the candidate texture color set to form a candidate color matrix; decomposing the candidate color matrix into a low-rank part and a sparse abnormal part; Solving a function with a target of minimizing the low-rank part and the sparse abnormal part, and obtaining a mean value of all rows of the low-rank part as the reference color of the triangular patch.

5. The method of claim 1, wherein, Before the three-dimensional mesh model is texture mapped according to the target source view of all the triangular patches, the method further comprises: An adjacency graph is constructed to reflect the adjacency relationship between the triangular patches; Based on the adjacency graph, the number of adjacent patches with different target source views is obtained in the case that the target source view of the triangular patch is different from the target source view of the adjacent patch; If the number exceeds a set proportion, the target source view of the triangular patch is replaced by the target source view of the adjacent patch whose number exceeds the set proportion.

6. The method of claim 1, wherein, The texture mapping of the three-dimensional mesh model according to the target source view of all the triangular patches comprises: For each triangular patch, a corresponding texture region is extracted from the target source view; All the texture regions are packed into a texture atlas; The seams between the texture regions from different views are feathered and fused on the texture atlas; A shader is controlled to perform texture mapping on the three-dimensional mesh model based on the fused texture atlas.

7. The method of claim 1, wherein, Before the three-dimensional mesh model is texture mapped according to the target source view of all the triangular patches, the method further comprises: For each triangular patch, the illumination component and the reflection component in the target source view are separated; Based on the target source view, the illumination component is removed, and the reflection component is enhanced to obtain an updated target source view.

8. An intraoral texture mapping device, comprising: The method comprises: An acquisition module is configured to acquire a multi-view image sequence collected by an intraoral scanner and camera parameters corresponding to each view image; A screening module is configured to screen, based on the camera parameters, qualified views capable of observing a current triangular patch from the multi-view image sequence for each triangular patch in a three-dimensional mesh model; A construction module is configured to construct a candidate texture color set for each triangular patch based on the screened qualified views; A removal module is configured to remove abnormal color values in the candidate texture color set by using a low-rank sparse decomposition model to decompose the candidate texture color set, and obtain a reference color of each triangular patch; A determination module is configured to determine a target source view for each triangular patch from the qualified views according to the reference color and a view quality parameter; An execution module is configured to texture map the three-dimensional mesh model according to the target source view of all the triangular patches. The determining module is specifically configured to: acquire a view angle quality parameter of each of the qualified views corresponding to each of the triangular patches, the view angle quality parameter comprising: an included angle between a normal vector of the triangular patch and a camera line-of-sight direction corresponding to the qualified view and an area size of a projection region of each of the triangular patches in the corresponding qualified view; acquire a deviation color value of the reference color of each of the triangular patches and a candidate color of the corresponding qualified view; and determine a quality score of each of the qualified views corresponding to each of the triangular patches based on the view angle quality parameter and the deviation color value. Based on the quality score, the target source view is selected from the corresponding qualified views for each of the triangular patches.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 7.

Images