Dental arch map defect repairing method and system

Abstract

The invention discloses a dental arch map defect repair method and system. The method comprises the following steps: S1, extracting an edge girdle Band region; s2, performing multi-scale decomposition on the dental arch chartlet to obtain a low-frequency sub-graph Ilow and a high-frequency sub-graph Ihigh; s3, filling a defect area omega of the low-frequency sub-graph Ilow by adopting a gradient driving filling algorithm; s4, repairing the defect area omega of the high-frequency sub-graph Ihigh by adopting a structure constraint matching algorithm; s5, carrying out inverse reconstruction to obtain a complete map Iout; and S6, mapping the complete map Iout back to the 3D dental arch model through UV. According to the method, the repairing color of the dental arch map is smooth, texture details are considered, the texture is natural, and false textures are avoided; natural transition between the defect area and the peripheral area is realized; according to the embodiment, training data and neural network reasoning are not needed, and the method has real-time performance and cross-device adaptability and is suitable for clinical rapid repair.

Description

[0001] The present application relates to the technical field of image inpainting, and in particular to a tooth arch map defect repairing method and system. BACKGROUND

[0002] In the field of digital dental restoration (CAD / CAM) and oral three-dimensional visualization, the tooth arch map is used to restore the color and texture of the tooth arch surface in reality. However, due to the reflection, scanning dead angle, saliva interference and other problems in the oral cavity, the generated tooth arch map often has local defects or artifacts, such as color loss, texture interruption or map holes. In addition, when editing a three-dimensional model using a map to restore color and texture, the edited area often cannot be restored due to the change of the relationship between the map and the mesh.

[0003] The current common mainstream repair method for tooth arch map is as follows:

[0004] 1. Global color interpolation: as shown in the following formula, a method of interpolating and filling based on the color of the pixels on the edge of the defect, although it can quickly eliminate the holes, it is difficult to restore the fine lines or wear features, and it cannot reconstruct the real enamel texture, the surface of the repaired result is too smooth, lacks naturalness, and the repaired result is not real enough. Figure 2 2. Deep learning repair: using neural network to automatically generate texture in the defect area, but this method needs a large amount of high-quality tooth arch map data for training, and the model parameters are large and the computing resources are consumed, and the calculation time is long, which cannot meet the demand of clinical interactive design scene; in addition, the color and luster of different patients' teeth are obviously different, the network generated result is easy to appear color difference and false texture, it cannot adapt to the color difference of in-mouth light gradient and different tooth regions, it is easy to produce obvious color blocks or joints, and it is difficult to be quickly applied in clinic.

[0005] SUMMARY

[0006] The present application provides a tooth arch map defect repairing method and system to solve the technical problems of data dependence, loss of detail texture, poor color consistency and insufficient real-time during repairing of the tooth arch map.

[0007] To solve the above technical problems, in a first aspect, the present application provides a tooth arch map defect repairing method, comprising the following steps:

[0008] S1: extracting the edge band Band region of the tooth arch map defect region Omega, obtaining the color information and texture direction information of the Band region;

[0009] S2: performing multi-scale decomposition on the tooth arch map to obtain a low-frequency sub-map I_low and a high-frequency sub-map I_high;

[0010] ​S3: based on the color information of the Band region, using a gradient-driven filling algorithm to fill the missing region Ω of the low-frequency subgraph I_low, to obtain the filled low-frequency subgraph I_low_filled;

[0011] S4: based on the texture direction information of the Band region, using a structure-constrained matching algorithm to repair the missing region Ω of the high-frequency subgraph I_high, to obtain the repaired high-frequency subgraph I_high_filled;

[0012] S5: performing color statistical matching on the filled low-frequency subgraph I_low_filled, to obtain the corrected low-frequency subgraph I', and inversely reconstructing the corrected low-frequency subgraph I' and the repaired high-frequency subgraph I_high_filled to obtain the complete map I_out;

[0013] S6: mapping the complete map I_out back to the 3D dental arch model through UV.

[0014] In some embodiments, the step S1 of extracting the edge ring Band region of the dental arch map missing region Ω specifically comprises the following steps:

[0015] S11: dynamically adjusting the ring width based on the edge curvature, to obtain the Band region;

[0016] S12: performing a closing operation on the extracted Band region to fill the edge breakage and smooth the profile.

[0017] In some embodiments, the step S1 of obtaining the color information of the Band region specifically comprises:

[0018] Converting the Band region to Lab or HSV space, calculating the average value μ of the color channel through formula (1), and calculating the variance σ of the color channel through formula (2),

[0019] Formula (1):

[0020] Formula (2):

[0021] wherein c represents the L / a / b or H / S / V channel, N represents the total number of pixels in the Band region, represents the value of the i-th pixel in the Band region in the c channel.

[0022] In some embodiments, the step S1 of obtaining the texture direction information of the Band region specifically comprises:

[0023] Applying Sobel gradient operator to the Band region, a local main texture direction field G is obtained, and the gradient direction θ is calculated by formula (3),

[0024] Formula (3):

[0025] Wherein, the direction field G(x, y) represents the local texture direction of each pixel.

[0026] In some embodiments, the step S2 specifically comprises:

[0027] Wavelet decomposition is performed on the dental arch map to obtain low frequency LL and high frequency LH / HL / HH, or Gaussian pyramid decomposition is performed on the dental arch map to obtain low frequency subgraph I_low and high frequency subgraph I_high.

[0028] In some embodiments, the gradient-driven filling algorithm is a Poisson blending algorithm or a multi-point interpolation algorithm, and the step S3 specifically comprises:

[0029] Based on the color information of the Band region, Poisson blending is performed on the missing area Ω of the low frequency subgraph I_low, and the Poisson blending formula is as follows:

[0030]

[0031] Wherein, u represents the generated repair color, represents the low frequency value of the Band region, represents the color change gradient, I represents the original dental arch map, represents the contour line of the missing area Ω,

[0032] Or, a multi-point interpolation algorithm is used, a plurality of sampling points are selected in the Band region, and the color is smoothly expanded to the missing area Ω according to the distance weight or the triangular interpolation.

[0033] In some embodiments, the step S4 specifically comprises the following steps:

[0034] S41: In the area outside the missing area Ω of the high frequency subgraph I_high, a small block Patch with the most similar boundary texture of the missing area Ω is searched;

[0035] S42: The small block Patch is spread into the missing area Ω according to the main direction of the direction field G;

[0036] S43: Through multi-round minimum error block selection and weight fusion iteration optimization, a preliminary high frequency filling graph is generated;

[0037] S44: The repair boundary is Gaussian feathered to obtain the repaired high frequency subgraph I_high_filled.

[0038] In some embodiments, the step S5 specifically comprises the following steps:

[0039] S51: Align the filled low-frequency sub-map I_low_filled with the mean value μ and variance σ of the Band region by formula (4) to obtain a corrected low-frequency sub-map I' after correction,

[0040] Formula (4):

[0041] Wherein, μ Band is the mean value of the color channel of the Band region, σ Band is the variance of the color channel of the Band region, σ filled is the variance of the filled low-frequency sub-map I_low_filled in the missing area Ω, I low_filled is the patching background color of the filled low-frequency sub-map I_low_filled, μ filled is the mean value of the filled low-frequency sub-map I_low_filled in the missing area Ω;

[0042] S52: Input the corrected low-frequency sub-map I' and the patched high-frequency sub-map I_high_filled into an inverse transform by inverse wavelet / pyramid reconstruction, and output a complete map I_out.

[0043] In some embodiments, the step S6 specifically comprises the following steps:

[0044] S61: Map the complete map I_out back to the 3D dental arch surface according to the original three-dimensional model UV layout;

[0045] S62: Apply small-range Poisson fusion or feathering smoothing at the splicing gap of the complete map I_out and the 3D dental arch surface to eliminate color and texture joints;

[0046] S63: Generate a map file and an updated UV coordinate file that can be directly used for CAD / CAM rendering.

[0047] In a second aspect, the present application also provides a computer system, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method of the first aspect when executing the computer program.

[0048] The technical scheme provided by the embodiments of the present disclosure can have the following beneficial effects:

[0049] 1. By decomposing the original dental arch map into a low-frequency sub-map containing low-frequency color information and a high-frequency sub-map containing high-frequency texture information, and then fusing them after repairing by different algorithms respectively, color smoothing and texture details are realized.

[0050] 2. The missing area of the high-frequency subgraph is repaired by combining the Band region texture direction information and the structure constraint matching algorithm, so that the texture is natural and false texture is avoided;

[0051] 3. The missing area of the low-frequency subgraph is filled by the gradient-driven filling algorithm, and the filled low-frequency subgraph is color statistical matched based on the color information of the Band region, so that the natural transition of the missing area and the surrounding area is realized;

[0052] 4. The tooth arch map missing repair method of the present application does not need training data and neural network inference, has real-time and cross-device adaptability, and is suitable for clinical rapid repair.

[0053] In addition to the purposes, features and advantages described above, the present application has other purposes, features and advantages. The present application will be further described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0054] The accompanying drawings, which form a part of this application, are intended to provide further understanding of the present application, and the illustrative embodiments of the present application and their description serve the purpose of explaining the present application. The present application is not limited by the accompanying drawings. In the drawings:

[0055] Figure 1 is a flowchart of the tooth arch map missing repair method of the preferred embodiment of the present application;

[0056] Figure 2 is a tooth arch map repaired by the existing global color interpolation method;

[0057] Figure 3 is a tooth arch map repaired by the tooth arch map missing repair method of the preferred embodiment of the present application. DETAILED DESCRIPTION

[0058] The embodiments of the present application will be described in detail below with reference to the accompanying drawings, but the present application can be implemented in various different ways limited and covered by the claims.

[0059] In addition, unless otherwise defined, technical terms or scientific terms used in the description of the present application shall be understood as having the common meaning to those of ordinary skill in the art to which the present application belongs. The words "upper", "lower", "left", "right", "center", "vertical", "horizontal", "inner", "outer" and the like used in the description of the present application only indicate relative directions or positional relationships, and do not imply that the device or element must have a particular orientation, be constructed and operated in a particular orientation, and the relative positional relationship may also change accordingly when the absolute position of the described object changes, therefore it cannot be understood as a limitation to the present application. The "first", "second", "third" and the like used in the description of the present application are only for the purpose of description, to distinguish different components, and cannot be understood as indicating or implying relative importance. The "one", "an" or "the" and the like used in the description of the present application should not be understood as an absolute limitation on the quantity, but should be understood as the presence of at least one. The "includes" or "contains" and the like used in the description of the present application means that the elements or objects before the word are encompassed by the elements or objects listed after the word and their equivalents, without excluding other elements or objects.

[0060] It should also be noted that, unless otherwise specified and limited, the "installation", "connection", "connection" and the like used in the description of the present application should be understood broadly, for example, the connection can be fixed connection, or detachable connection, or integral connection; can be mechanical connection, or electrical connection; can be directly connected, or indirectly connected through intermediate medium, or the internal communication of two elements, and those skilled in the art can understand its specific meaning in the present application according to the specific circumstances.

[0061] As shown in Figure 1 The dental arch map defect repairing method of the embodiment comprises the following steps:

[0062] S1: extracting the edge ring band Band region of the dental arch map defect region Omega, obtaining the color information and texture direction information of the Band region;

[0063] S2: performing multi-scale decomposition on the dental arch map to obtain a low-frequency subgraph I_low and a high-frequency subgraph I_high;

[0064] S3: based on the color information of the Band region, using gradient driven filling algorithm to fill the defect region Omega of the low-frequency subgraph I_low, to obtain the filled low-frequency subgraph I_low_filled;

[0065] S4: based on the texture direction information of the Band region, a structure constraint matching algorithm is used to repair the missing area Ω of the high-frequency subgraph I_high, to obtain a repaired high-frequency subgraph I_high_filled;

[0066] S5: the filled low-frequency subgraph I_low_filled is subjected to color statistical matching to obtain a corrected low-frequency subgraph I', and the corrected low-frequency subgraph I' and the repaired high-frequency subgraph I_high_filled are inversely reconstructed to obtain a complete map I_out;

[0067] S6: the complete map I_out is mapped back to the 3D dental arch model through UV.

[0068] The dental arch map missing repair method of the embodiment can be run on a standard dental workstation, meets the real-time operation needs of the clinic, and realizes the balance between color smoothing and texture details by decomposing the original dental arch map into a low-frequency subgraph containing low-frequency color information and a high-frequency subgraph containing high-frequency texture information, respectively repairing them by using different algorithms, and then fusing them. The missing area of the high-frequency subgraph is repaired by combining the Band region texture direction information and the structure constraint matching algorithm, so that the texture is natural and false texture is avoided. The missing area of the low-frequency subgraph is filled by the gradient-driven filling algorithm, and the filled low-frequency subgraph is subjected to color statistical matching based on the color information of the Band region, to realize the natural transition between the missing area and the surrounding area. The embodiment does not need training data and neural network inference, has real-time performance and cross-device adaptability, and is suitable for clinical rapid repair.

[0069] In some possible embodiments, the edge ring Band region of the dental arch map missing area Ω extracted in step S1 specifically includes the following steps:

[0070] S11: based on the edge curvature, the ring width is dynamically adjusted to obtain the Band region;

[0071] S12: the extracted Band region is subjected to a closing operation to fill the edge fracture and smooth the profile.

[0072] The width of the edge ring band is dynamically adjusted based on the edge curvature, specifically, a smaller width is used for regions with large curvature and a larger width is used for flat regions; in places where the tooth edge is significantly curved, a narrow ring band can accurately capture the subtle features of the high curvature area while avoiding the inclusion of too many irrelevant textures; in flat areas, a wide ring band can obtain more rich color features and cover the color gradient in the flat area. This adaptive mechanism is more scientific and reasonable than a fixed width. In some embodiments, a curvature threshold value can be set according to experience, when the curvature is greater than the threshold value, a smaller width edge ring band is used, for example, 3-5 pixels, when the curvature is less than the threshold value, a larger width edge ring band is used, for example, 8-12 pixels; in other embodiments, the width of the edge ring band can also be inversely proportional to the curvature through a mathematical formula.

[0073] After determining the edge ring band Band area according to the edge curvature, a closing operation of inflation and then corrosion is performed on the Band area to fill the edge fracture of the Band area and smooth the profile, ensuring that the edge ring band Band area is a continuous ring. The steps of the present embodiment can ensure the width of the edge ring band Band and ensure that it is a continuous ring.

[0074] In some possible embodiments, the color information of the Band area obtained in step S1 specifically includes:

[0075] The Band area is converted to Lab or HSV space, the average value μ of the color channel is calculated by formula (1), and the variance σ of the color channel is calculated by formula (2),

[0076] Formula (1):

[0077] Formula (2):

[0078] Wherein, c represents L / a / b or H / S / V channel, N represents the total number of pixels in the Band area, represents the value of the i-th pixel in the Band area in the c channel.

[0079] Wherein, when the Band area is converted to Lab space, Lab space is a color model, and the Lab color model takes coordinates Lab, wherein L represents luminance; positive a represents red, and negative a represents green; positive b represents yellow, and negative b represents blue; when the Band area is converted to HSV space, the parameters in the HSV color model are hue (H), saturation (S), and brightness (V). The average value μ of the three channels of the Band area can be obtained by formula (1), and the variance σ of the three channels of the Band area can be obtained by formula (2).

[0080] In some possible embodiments, the texture direction information of the Band area obtained in step S1 specifically includes:

[0081] Applying Sobel gradient operator to the Band region, the texture direction field G(x, y) of each pixel is obtained, and the gradient direction θ is calculated by formula (3),

[0082] Formula (3):

[0083] Wherein, Gx represents the color change intensity in horizontal direction, and Gy represents the color change intensity in vertical direction.

[0084] The gradient direction of each pixel in the Band region is calculated, and then the texture direction field of the Band region is obtained through the direction distribution histogram.

[0085] The gradient components (Gx, Gy) of each pixel in the Band region are detected through Sobel gradient operator, the gradient direction θ of each pixel is calculated through formula (3), and thus the direction field G(x, y) of each pixel is obtained. The direction field G is a two-dimensional vector, which includes direction and gradient intensity information, and θ is the scalar representation of the direction field G(x, y).

[0086] The gradient direction θ of each pixel in the Band region is calculated, and then the main direction field of the Band region can be obtained through the direction distribution histogram. The direction distribution histogram is specifically that the continuous gradient direction is discretized into several intervals, for example, 0°-180° (because the texture direction has periodicity, 180° and 0° are the same direction) is divided into K intervals, and the width of each interval is 180° / K. All pixels in the Band region are counted, and each pixel is classified into the corresponding interval according to the gradient direction θ. After obtaining the histogram, the peak value of the histogram is found, that is, the angle range corresponding to the interval containing the most pixels, and the center angle of the interval is usually taken as the main direction θ°. Thus, the main direction field of the entire Band region can be obtained.

[0087] In some possible embodiments, step S2 specifically comprises:

[0088] The wavelet decomposition is performed on the dental arch map to obtain low frequency LL and high frequency LH / HL / HH, or the Gaussian pyramid decomposition is performed on the dental arch map to obtain low frequency subgraph I_low and high frequency subgraph I_high.

[0089] In the embodiment, the image is separated into four independent subbands by a high-pass filter and a low-pass filter through wavelet decomposition, wherein the low-frequency approximation subband LL saves the overall light / dark and base tone information of the dental arch map, the horizontal high-frequency detail subband LH saves the vertical enamel vertical lines in the dental arch map, the vertical high-frequency detail subband HL saves the horizontal gingival lines in the dental arch map, and the diagonal high-frequency detail subband HH saves the diagonal line texture information of the dental arch map. In other possible embodiments, the image can also be decomposed by a Gaussian pyramid. Specifically, the original dental arch map is taken as the bottom layer, and a hierarchical structure with decreasing resolution is generated through progressive down-sampling and Gaussian blur operation to obtain a high-level blurred low-frequency subgraph I_low and a bottom clear high-frequency subgraph I_high. The original dental arch map is decomposed into a blurred low-frequency subgraph I_low that retains color information and a clear high-frequency subgraph I_high that retains texture and edges. After being repaired by different algorithms respectively, the two subgraphs are fused to realize the balance between color smoothing and texture details.

[0090] In some possible embodiments, the gradient-driven filling algorithm is a Poisson fusion algorithm or a multi-point interpolation algorithm, and step S3 specifically includes:

[0091] Based on the color information of the Band region, Poisson fusion is performed on the missing area Ω of the low-frequency subgraph I_low, and the Poisson fusion formula is as follows:

[0092]

[0093] wherein u represents the repaired color to be generated, I ref represents the low-frequency value of the Band region, represents the color change gradient, I represents the original dental arch map, represents the boundary of the missing area Ω,

[0094] Or a multi-point interpolation algorithm is used to select multiple sampling points in the Band region, and the color is smoothly extended to the missing area Ω according to the distance weight or the triangular interpolation.

[0095] When the gradient-driven filling algorithm is a Poisson fusion algorithm, the low-frequency value of the Band region is specifically a set of color values of all pixels of the low-frequency subgraph I_low in the Band region, and the pixel value includes Lab or HSV color information, is the gradient field of the Band region, that is, the reference gradient field, which is used to provide an ideal color gradient mode of the Band region for the repaired area, and the boundary of the missing area Ω represents the contour line around the missing area Ω, and the boundary in the embodiment is a single-pixel contour line, represents a set of pixel values of the original dental arch image on the boundary of the missing area, The value of the image forced to be patched as a constraint condition on the boundary is equal to the pixel value of the boundary The missing area Ω is filled by performing Poisson fusion to obtain the filled low-frequency subgraph I_low_filled. In this embodiment, the gradient consistency constraint ensures the natural color transition of the patched area (missing area Ω) and the Band area, which can overcome the color block mutation defect of the traditional interpolation method,

[0096] It should be noted that in other embodiments, the missing area Ω of the low-frequency subgraph I_low can also be filled by a multi-point interpolation algorithm. Specifically, a plurality of sampling points are first selected in the Band area, and weights are assigned according to the distance between the missing point and each sampling point, or the Band area is triangulated and the barycentric coordinate interpolation is used to extend the color to the missing area Ω to obtain the filled low-frequency subgraph I_low_filled.

[0097] In some possible embodiments, step S4 specifically comprises the following steps:

[0098] S41: In the area outside the missing area Ω of the high-frequency subgraph I_high, search for a small block Patch most similar to the boundary texture of the missing area Ω;

[0099] S42: Spread the small block Patch into the missing area Ω according to the main direction of the texture direction field G;

[0100] S43: Perform iterative optimization by multiple rounds of minimum error block selection and weight fusion to generate a preliminary high-frequency filled graph;

[0101] S44: Perform Gaussian feathering on the patched boundary to obtain the patched high-frequency subgraph I_high_filled.

[0102] In step S41, the size of the small block Patch is set according to actual requirements. For example, a 5*5 pixel block can be used as the size of the small block Patch. The boundary texture of the missing area Ω refers to a 5*5 pixel contrast block extending from the boundary of the missing area Ω to the area outside the missing area Ω. In the area outside the missing area Ω of the high-frequency subgraph I_high, search for a small block Patch most similar to the contrast block. The similarity between the contrast block and the small block Patch is evaluated by normalized cross correlation (NCC) or structural similarity index (SSIM).

[0103] The normalized cross-correlation evaluates the similarity between two image blocks by calculating the normalized cross-correlation coefficient, the normalized cross-correlation coefficient ranges from -1 to 1, and the greater the value, the higher the similarity. The structural similarity index evaluates the similarity of image blocks from three dimensions of brightness, contrast and structure, and the value ranges from 0 to 1, and 1 represents complete similarity. Since the structural similarity index (SSIM) is sensitive to the structural information of the image, it can better reflect the similarity of texture, edge and other features, and is more consistent with human visual perception. In this embodiment, the structural similarity index is used to evaluate the similarity between the contrast block and the small block Patch. For example, assuming that the missing area Ω is a 50x50 pixel missing area, the boundary texture block is a 5*5 pixel block, and then a 5*5 window is slid in the effective area (the entire image except the missing area Ω), the SSIM of each window and the boundary texture block is calculated, and the position of the maximum value is the most similar small block Patch.

[0104] In step S42, the main direction of the texture direction field G is obtained by the direction distribution histogram of the direction field G(x, y) of all pixels of the contrast block in the Band area. Specifically, the gradient direction θ of each pixel of the contrast block in the Band area is calculated, and then the main direction of the texture direction field G is obtained by the direction distribution histogram. After determining the main direction of the texture direction field G, the matched small block Patch is rotated and aligned according to the main direction of the texture direction field G, and then filled into the target position in the missing area Ω. For example, the main direction (statistical by the gradient direction θ of all its own pixels) of the most similar small block Patch found in step S41 may be 85°, and the main direction of the texture direction field G is also 70°, so the small block Patch needs to be rotated to the 70° direction and then tiled into the missing area Ω.

[0105] In step S43, during the filling process, after filling a small block Patch, the boundary of the boundary texture will shrink from the boundary of the missing area Ω by the width of a small block Patch, at this time, the dynamic boundary Γ needs to be updated, and then when searching for the next most similar small block Patch, the contrast block used is a 5*5 pixel block extending from the new dynamic boundary Γ to the boundary of the missing area Ω. In this way, in each round, the small block Patch with the most similar texture and the smallest error to the current dynamic boundary Γ can be selected, and when multiple Patches overlap, different weights are given according to the pixel position (such as high weight for pixels close to the boundary) to fuse, so that the overlapping area is smoothly transitioned through weighted averaging.

[0106] In step S44, after the high-frequency subgraph I_high is repaired, the boundary of the missing area Ω is extended by 5*5 pixels, and then the structural similarity index (SSIM) of each window and the boundary texture block is calculated, and the position of the maximum value is the most similar small block Patch. ​​Gaussian weighting is applied, specifically, after the filling of the missing area Ω is completed, along the boundary A region of 2-3 pixels is expanded to both sides, in this region, the patched pixels and the original pixels are mixed using Gaussian weight, the joint is eliminated, and the patched region and the original region are naturally transitioned at the boundary.

[0107] By the patching method of the embodiment, the missing area Ω of the high-frequency subgraph I_high is patched, and a patched high-frequency subgraph I_high_filled is obtained, which has the characteristics of natural texture and avoids false texture.

[0108] In some possible embodiments, step S5 specifically includes the following steps:

[0109] S51: Align the filled low-frequency subgraph I_low_filled with the average value μ and the variance σ of the Band region by formula (4), to obtain a corrected corrected low-frequency subgraph I',

[0110] Formula (4):

[0111] Wherein, μ Band is the average value of the color channel of the Band region, σ Band is the variance of the color channel of the Band region, σ filled is the variance of the filled low-frequency subgraph I_low_filled in the missing area Ω, I low_filled is the patching background color of the filled low-frequency subgraph I_low_filled, μ filled is the average value of the filled low-frequency subgraph I_low_filled in the missing area Ω;

[0112] S52: Input the corrected low-frequency subgraph I' and the patched high-frequency subgraph I_high_filled into the inverse transform by inverse wavelet or pyramid reconstruction, and output the complete map I_out.

[0113] After the filling of the low-frequency subgraph I_low is completed, the average value μ filled of the filled low-frequency subgraph I_low_filled in the missing area Ω is calculated by formula (1), and the variance σ filled of the low-frequency subgraph I_low_filled in the missing area Ω is calculated by formula (2), I low_filled includes all pixel information (including each channel of L / a / b or H / S / V) of the entire filled low-frequency subgraph I_low_filled, and all pixels of the low-frequency subgraph I_low_filled in the missing area Ω can be corrected by formula (4), so that the filled missing area Ω and the surrounding are consistent in tone, matched in contrast, and natural color fluctuations are retained.

[0114] In some embodiments, the low-frequency sub-map I_low and the three high-frequency sub-maps I_high obtained by wavelet decomposition are inverse wavelet reconstructed, the corrected low-frequency sub-map I' and the three repaired high-frequency sub-maps I_high_filled are input into the inverse transform, and the complete map I_out is output. Since the wavelet decomposition retains the high-frequency information in three directions (horizontal, vertical, and diagonal), the edges and texture details in each direction can be restored during reconstruction. In other embodiments, the low-frequency sub-map I_low and the high-frequency sub-map I_high obtained by Gaussian pyramid decomposition are inverse (Gaussian) pyramid, the processed corrected low-frequency sub-map I' and the repaired high-frequency sub-map I_high_filled are fused, and the complete map I_out as shown in Figure 3 is obtained.

[0115] In some possible embodiments, step S6 specifically comprises the following steps:

[0116] S61: Map the complete map I_out back to the 3D dental arch surface according to the original three-dimensional model UV layout;

[0117] S62: Apply small-range Poisson fusion or feathering smoothing at the splicing gap between the complete map I_out and the 3D dental arch surface to eliminate color and texture joints;

[0118] S63: Generate a map file that can be directly used for CAD / CAM rendering and an updated UV coordinate file.

[0119] In dental restoration, in order to obtain a dental arch map, the surface of a 3D dental arch model needs to be unfolded into a 2D plane (i.e., a UV map), and each 3D surface point corresponds to a point on the UV map. The repaired dental arch map (complete map I_out) needs to be re-pasted onto the 3D model through the UV mapping relationship.

[0120] Since the surface of the 3D model is usually divided into multiple UV blocks (also known as UV islands), the blocks are separated in the UV map but adjacent on the 3D model, when the complete map I_out is mapped back to the 3D model, the connecting edges of different UV blocks can produce visible seams (stitching gaps) due to discontinuity of texture or color, therefore, the seams need to be processed, in the embodiment, the stitching gaps are processed by small-range Poisson blending, in the local area of the seam, a Poisson equation is solved to make the color gradients on both sides of the seam as consistent as possible, so as to realize seamless transition, specifically, in a small range on both sides of the seam, the pixel values are adjusted to make the color change at the seam smooth. In other embodiments, the seams can also be eliminated by feathering smoothing processing, that is, weighted average (usually Gaussian weight is used) is used to mix the colors on both sides at the seam to make the transition natural, so as to eliminate the color and texture seams. After the seams are eliminated, the 3D dental arch model will not have obvious texture or color breakage when rendered, improving the realism.

[0121] In some embodiments, the present application also provides a computer system comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of the dental arch map defect repairing method of the above-mentioned embodiments when executing the computer program.

[0122] The computer system of the embodiment can be used in a standard dental workstation device, and the memory is configured to store various types of data to support the operation of the device. The memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0123] The standard dental workstation device can implement the dental arch map defect repairing method of the embodiment by running the computer program in the above-mentioned computer system.

[0124] The above is only the preferred embodiment of the present application and is not used to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method of arch map defect repair, characterized by, The method comprises the following steps: S1: extracting an edge ring band Band region of a missing area Ω of a dental arch map, and obtaining color information and texture direction information of the Band region; S2: performing multi-scale decomposition on the dental arch map to obtain a low-frequency sub-map I_low and a high-frequency sub-map I_high; S3: based on the color information of the Band region, performing filling on the missing area Ω of the low-frequency sub-map I_low by using a gradient-driven filling algorithm to obtain a filled low-frequency sub-map I_low_filled; S4: based on the texture direction information of the Band region, performing repairing on the missing area Ω of the high-frequency sub-map I_high by using a structure-constrained matching algorithm to obtain a repaired high-frequency sub-map I_high_filled; S5: performing color statistical matching on the filled low-frequency sub-map I_low_filled to obtain a corrected low-frequency sub-map I', and inversely reconstructing the corrected low-frequency sub-map I' and the repaired high-frequency sub-map I_high_filled to obtain a complete map I_out; S6: mapping the complete map I_out back to a 3D dental arch model through UV.

2. The dental arch map hole patching method of claim 1, wherein, The step S1 of extracting the edge ring band Band region of the missing area Ω of the dental arch map specifically comprises the following steps: S11: dynamically adjusting the ring band width based on edge curvature to obtain the Band region; S12: performing a closing operation on the extracted Band region to fill edge breaks and smooth the profile.

3. The dental arch map hole patching method of claim 1, wherein, The step S1 of obtaining the color information of the Band region specifically comprises: converting the Band region to Lab or HSV space, calculating the average value μ of the color channel through formula (1), and calculating the variance σ of the color channel through formula (2), Equation (1): Equation (2): where c represents L / a / b or H / S / V channel, N represents the total number of pixels in Band region, represents the value of the i-th pixel in Band region in c channel.

4. The dental arch map hole patching method of claim 1, wherein, The step S1 of obtaining the texture direction information of the Band region specifically comprises: applying a Sobel gradient operator to the Band region to obtain a texture direction field G(x, y) of each pixel, and calculating the gradient direction θ through formula (3), Equation (3): wherein Gx represents the horizontal direction color change intensity, and Gy represents the vertical direction color change intensity; calculating the gradient direction of each pixel in the Band region, and then obtaining the main direction field of the Band region through a direction distribution histogram.

5. The dental arch map hole patching method of claim 1, wherein, The step S2 specifically comprises: performing wavelet decomposition on the dental arch map to obtain a low frequency LL and a high frequency LH / HL / HH, or, performing Gaussian pyramid decomposition on the dental arch map to obtain a low-frequency sub-map I_low and a high-frequency sub-map I_high.

6. The dental arch map hole patching method of claim 1, wherein, The gradient-driven filling algorithm is a Poisson fusion algorithm or a multi-point interpolation algorithm, and the step S3 specifically comprises: based on the color information of the Band region, performing Poisson fusion on the missing area Ω of the low-frequency sub-map I_low, and the Poisson fusion formula is as follows: where u denotes the inpainting color to be generated, I ref denotes the low frequency value of the Band region, denotes the color gradient of the inpainting region, is the gradient field of the Band region, I denotes the original dental arch map, denotes the boundary of the missing region Ω, or using a multi-point interpolation algorithm, selecting a plurality of sampling points in the Band region, and expanding the color to the missing area Ω by distance weight or triangular interpolation.

7. The dental arch map hole patching method of claim 4, wherein, The step S4 specifically comprises the following steps: S41: search for a patch most similar to the boundary texture of the missing region Ω in the region outside the missing region Ω of the high-frequency subgraph I_high; S42: spread the patch into the missing region Ω according to the main direction of the texture direction field G; S43: generate a preliminary high-frequency filling graph through multi-round minimum error block selection and weight fusion iteration optimization; S44: perform Gaussian feathering on the repaired boundary to obtain a repaired high-frequency subgraph I_high_filled.

8. The dental arch map hole patching method of claim 3, wherein, The step S5 specifically includes the following steps: S51: align the filling low-frequency subgraph I_low_filled with the average value μ and the variance σ of the Band region through formula (4) to obtain a corrected low-frequency subgraph I', Equation (4): where μ Band is the mean of the color channel of the Band region, σ Band is the variance of the color channel of the Band region, σ filled is the variance of the inpainted low-frequency sub-image I_low_filled in the missing region Ω, I low_filled is the inpainting base color of the inpainted low-frequency sub-image I_low_filled, μ filled is the mean of the inpainted low-frequency sub-image I_low_filled in the missing region Ω; S52: input the corrected low-frequency subgraph I' and the repaired high-frequency subgraph I_high_filled into inverse transformation through inverse wavelet or pyramid reconstruction, and output a complete map I_out.

9. The dental arch map hole patching method of claim 8, wherein, The step S6 specifically includes the following steps: S61: map the complete map I_out back to the 3D dental arch surface according to the original three-dimensional model UV layout; S62: apply small-range Poisson fusion or feathering smoothing at the splicing gap between the complete map I_out and the 3D dental arch surface to eliminate color and texture joints; S63: generate a map file that can be directly used for CAD / CAM rendering and an updated UV coordinate file.

10. A computer system comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor implements the steps of the method of any one of claims 1 to 9 when executing the computer program.

Images