Dynamic color visualization in three-dimensional models of dental objects

Abstract

The present disclosure relates to computer-implemented methods and intraoral scanning systems. A computer-implemented method of eliminating glare defects in processed image scan data includes receiving a plurality of image scan data of a dental object generated by an intraoral scanner; processing one or more of the plurality of image scan data into processed image scan data, the processed image scan data comprising a plurality of pixels; determining a plurality of defective pixels in the processed image scan data by a glare detection algorithm, wherein the glare detection algorithm is configurable to determine the plurality of defective pixels when a pixel value of each of the plurality of defective pixels satisfies one or more pixel value criteria and when an arrangement geometry of the plurality of defective pixels satisfies one or more geometry criteria; determining non-glare image scan data by analyzing pixels of the plurality of pixels in a vicinity of the plurality of defective pixels by a glare elimination algorithm; and replacing the processed image scan data in the plurality of defective pixels with the non-glare image scan data.

Description

Technical Field

[0001] This invention relates to a method and an intraoral scanning system configured to generate a three-dimensional (3D) model with a dynamic texture that varies according to the viewing orientation of a virtual camera. More specifically, this invention relates to assigning two or more texture values ​​with different viewing orientations to vertices of a 3D model. Background Technology

[0002] Currently, intraoral scanners capture multiple images during scanning, and these images are used to generate a 3D representation of the object. Furthermore, these images are compiled into colors that are applied to the 3D representation. Users can rotate, scale, and translate the 3D representation, but the colors on the 3D representation remain constant. Because a single color is used to represent each point in the 3D representation, it is necessary to average all colors from all captured images covering the same point in the 3D representation. These averaged colors are often unreliable and cannot be used for comparison and measurement because they are acquired from different locations under different lighting conditions and other environmental variables that can affect the true colors. Moreover, some image modalities, such as infrared (IR), are highly dependent on the scanner's view orientation; therefore, images covering the same point may have completely different values ​​depending on the view orientation.

[0003] IR images are suitable for detecting proximal caries, but the problem is that points in a 3D representation cannot be represented by a single IR texture value because this value depends on the scanner's viewing orientation when acquiring the IR image (an IR image acquired from one location may expose a carious lesion, but the same lesion may not be detected from another location). Therefore, a new visualization is needed where colors need to change dynamically in relation to the virtual camera's viewing orientation. Summary of the Invention

[0004] One aspect of this disclosure is overcoming the aforementioned problem regarding how to provide dynamically changing colors in a 3D representation in relation to the viewing direction of a virtual camera. By changing the orientation of the 3D representation (i.e., the 3D model of the dental object) in the graphical user interface, the colors on the 3D model change accordingly with the viewing direction of the virtual camera. This allows for more realistic and accurate coloring of the 3D model.

[0005] According to one aspect, an intraoral scanning system is disclosed. The intraoral scanning system is configurable to assign texture values ​​to a three-dimensional (3D) model of a dental object, and wherein the intraoral scanning system includes an intraoral scanner configured to acquire geometric image data of the dental object and multiple texture image data of the dental object relative to a position on the dental object in different viewing directions.

[0006] Texture values ​​can include color, contrast, brightness, and / or transparency values.

[0007] Geometric image data includes image data with a light pattern, which is used to determine the depth information of the geometric image data. The depth information and the remaining data in the geometric image data are used to generate a 3D model. Texture image data includes one or more of the following: ● The red, green, and blue (RGB) color parameters for dental objects, ●Infrared parameters of dental objects, ● Roughness scalar parameter of dental objects ● Dental resorption parameters, and ● Scattering parameters of dental objects.

[0008] Texture image data can be used to determine the following optical parameters applied to a 3D model: ● Diffuse scattering parameters of dental objects, ● Gloss reflectance parameters of dental objects ● The translucency parameters of dental objects, and ● Internal dental characteristic parameters of dental objects.

[0009] An intraoral scanning system may include one or more processing units configured to generate a 3D model of a dental object based on geometric image data, wherein the 3D model includes a 3D mesh with multiple vertices. The 3D mesh can be provided by segmenting the 3D model, wherein the 3D mesh includes multiple vertices representing a dental object containing at least multiple teeth. In another example, the multiple vertices may also represent a dental object containing multiple teeth and gums. One or more processing units may be configured to project multiple texture rays from at least one pixel in the viewing direction for each of multiple texture image data, wherein at least one pixel corresponds to a texture value. The texture value may be a pixel value that may include the following parameters: ● The red, green, and blue (RGB) color parameters for dental objects, ●Infrared parameters of dental objects, ● Roughness scalar parameter of dental objects ● Dental resorption parameters, and ● Scattering parameters of dental objects.

[0010] Multiple texture rays are virtually generated by one or more processing units in a direction perpendicular to each pixel of the texture image data and toward the 3D model. The one or more processing units are then configured to determine whether a vertex among a plurality of vertices of the 3D model satisfies an intersection criterion relative to each of the multiple texture rays. The intersection criterion could be: a vertex among the vertices intersects with a texture ray, and / or a vertex among the vertices is closest to the intersection point of the 3D mesh and the texture ray. For example, a texture ray may intersect the 3D mesh between multiple vertices, but is closest to a vertex among the vertices, and that vertex will subsequently receive the texture value of the pixel corresponding to the texture ray. In another example, the texture ray may even intersect with that vertex.

[0011] Furthermore, one or more processing units can be configured to assign two or more texture values ​​and the corresponding view directions of the two or more texture rays to a vertex if the vertex satisfies an intersection criterion with respect to two or more texture rays. This means that a vertex may include at least two texture values ​​and a corresponding view direction, which is compared with the virtual camera view direction to determine which of the at least two texture values ​​should be included in the color of the 3D model to be displayed.

[0012] The 3D model can then be rendered by applying texture values ​​from two or more texture values ​​that match the virtual camera's viewing direction to each vertex.

[0013] An intraoral scanning system can be configured to dynamically adjust multiple colors of a 3D model based on the viewing direction of a virtual camera. The intraoral scanning system may include an intraoral scanner configured to acquire geometric image data of a dental object and multiple texture image data of the dental object relative to its position on the object in different viewing directions. The intraoral scanning system may include one or more processing units configured to generate a 3D model of the dental object based on the geometric image data, determine multiple texture values ​​from different viewing directions based on the multiple texture image data, and determine multiple colors of the 3D model based on the multiple texture values ​​according to the virtual camera viewing direction, wherein the viewing directions of the multiple texture values ​​satisfy a deviation angle criterion relative to the virtual camera viewing direction. Dynamic adjustment of multiple colors of the 3D model will provide a more accurate representation of the dental object.

[0014] The 3D model is rendered using a virtual camera that simulates what the human eye would see if the 3D model were physically real. As the user rotates or moves the 3D model, the virtual camera also changes its viewing direction or position. In this example, the virtual camera has a viewing direction that changes based on input from the system user. Rendering the 3D model in different virtual camera viewing directions causes the colors of the 3D model, determined by texture values ​​from multiple vertices, to change. By dynamically changing the colors in the 3D model based on the virtual camera viewing direction, a more realistic 3D representation of the dental object is provided to the user.

[0015] Two or more texture rays have different viewing directions, which means that changing the virtual camera's viewing direction between the viewing directions of the two or more texture rays will cause the color in the 3D model to change based on the relationship between the viewing directions of the two or more texture rays and the virtual camera's viewing direction. This relationship can be the angle between the viewing directions of the two or more texture rays and the virtual camera's viewing direction.

[0016] Geometric image data and multiple texture image data are acquired from the same location relative to the dental object, which can be achieved using a high-speed camera configured to acquire geometric image data and multiple texture image data at a frame rate between 2000 and 3500 images per second. Due to the high frame rate, a scan sequence containing at least geometric image data and multiple texture image data will cover the same area of ​​the dental object.

[0017] Each of the acquired texture image data may have a first timestamp, and the geometric image data may have a second timestamp. One or more processing units may be configured to correlate the multiple texture image data with respect to the geometric image data by comparing the first and second timestamps. To ensure that the multiple texture image data and the multiple geometric image data can be correlated to the same point on the dental object, the time difference between the first and second timestamps should be less than a time difference threshold.

[0018] Each of the plurality of texture image data includes a set of edge pixels that at least partially surround a set of framed pixels in each of the plurality of texture image data. One or more processing units are configured to: for each of the plurality of texture image data, determine a plurality of edge texture rays in the viewing direction of the set of edge pixels, wherein the plurality of edge texture rays are part of a plurality of texture rays. Then, the one or more processing units are configured to determine whether a set of edge vertices among a plurality of vertices satisfies an intersection criterion with respect to the plurality of edge texture rays, wherein the set of edge vertices at least partially surrounds a set of framed vertices among the plurality of vertices, and determine a framed vertex ray in the viewing direction toward each framed vertex in the set of framed pixels. Furthermore, the one or more processing units are also configured to determine whether a framed vertex among the framed vertices satisfies a second intersection criterion between the framed vertex ray of the framed vertex and the framed pixels in the set of framed pixels, and if the framed vertex satisfies the second intersection criterion, assign the texture value of the framed pixel and the viewing direction of the framed vertex ray to the framed vertex. The advantage of combining texture rays and box vertex rays is that assigning texture values ​​to multiple vertices in a 3D model is done much faster than in an example that only uses texture rays.

[0019] One or more processing units are configured to render a 3D model based on the virtual camera's viewing direction, wherein the 3D model comprises multiple faces formed by multiple vertices. Each face may include at least three vertices. One or more processing units are configured to, for each of the at least three vertices, select a texture value from two or more texture values, wherein the corresponding viewing direction of the selected texture values ​​for each of the at least three vertices satisfies a deviation angle criterion relative to the virtual camera's viewing direction, and determine the face's color based on the selected texture values ​​for each of the at least three vertices. The deviation angle criterion may include one or more of the following: the viewing direction of the texture value is closest to the virtual camera's viewing direction, and the viewing direction of the texture value is less than a maximum viewing direction angle relative to the virtual camera's viewing direction. The maximum viewing direction angle may be greater than 90 degrees, between 100 and 180 degrees, or between 120 and 180 degrees.

[0020] One or more processing units are configured to determine the orientation of a 3D model based on the viewing direction of a virtual camera, wherein the virtual camera viewing direction is determined based on user input. The user input may be a rotation of the 3D model, which causes a change in the virtual camera viewing direction.

[0021] It would be impractical to cover all viewing directions of a dental object such that any vertex among multiple vertices would have a texture value whose corresponding viewing direction satisfies a deviation angle criterion from any virtual camera viewing direction. Therefore, a weighting factor is assigned to each texture value of a vertex, and this weighting factor varies based on the angular difference between the texture value's corresponding viewing direction and the virtual camera viewing direction. For example, if a first angular difference between a first viewing direction and the virtual camera viewing direction is less than a second angular difference between a second viewing direction and the virtual camera viewing direction, then the weighting factor for the first texture value corresponding to the first viewing direction will be greater than the weighting factor for the second texture value corresponding to the second viewing direction. By applying dynamically varying weighting factors to each texture value, the faces of the 3D model will be assigned colors during rendering of the 3D model according to any virtual camera viewing direction.

[0022] One or more processing units may be configured to assign a weighting coefficient to each of two or more texture values ​​assigned to a vertex, and to determine the weighting coefficient of each of the two or more texture values ​​based on the difference between the virtual camera's virtual camera viewing direction and the corresponding viewing direction of each of the two or more texture values.

[0023] The weighting factor can be greater than zero when the difference between the virtual camera's viewing direction and the corresponding viewing direction of each of two or more texture values ​​is less than or equal to the maximum deviation angle. The weighting factor can be zero when the difference between the virtual camera's viewing direction and the corresponding viewing direction of each of two or more texture values ​​is greater than the maximum deviation angle. The maximum deviation angle can be greater than 90 degrees or greater than 180 degrees. When the difference between the virtual camera's viewing direction and the viewing direction of the corresponding texture value is greater than the maximum deviation angle, the quality / correctness of the color determined based on the texture value becomes unsuitable for assignment to a facet of the 3D model. This color is unsuitable for application to the 3D model because it does not resemble the color of the dental object at that facet.

[0024] One or more processors are configured to determine color based on a weighted average of selected texture values, wherein each of the selected texture values ​​has a weighting coefficient determined based on the difference between the virtual camera's viewing direction and the viewing direction of each of the selected texture values. For example, if this difference increases, the weighting coefficient of the corresponding texture value decreases, meaning that the corresponding texture value becomes less relevant to determining the color of the patch.

[0025] The difference between the virtual camera's viewing direction and the viewing direction of the corresponding texture value is an angular difference, or a difference between the barycentric coordinates of the viewing direction.

[0026] One or more processing units can be configured to determine color based on the average of selected texture values.

[0027] The difference between the virtual camera's viewing direction and the texture's viewing direction is determined by one or more of the following: the angle between the virtual camera's viewing direction and the texture's viewing direction, the cosine of the angle between the virtual camera's viewing direction and the texture's viewing direction, and the difference between the centroid coordinates of the viewing direction.

[0028] As the viewing direction of the virtual camera differs more from the viewing direction of the texture values, the quality of the color determined based on the texture values ​​decreases in the virtual camera viewing direction. Therefore, feedback regarding the color quality applied to the 3D model needs to be provided to the system user. One or more processing units can be configured to determine a quality score for the color determined for a patch, wherein the quality score can be determined based on the difference between the virtual camera viewing direction and the viewing direction of each of the selected texture values, and wherein the quality score decreases as the difference increases.

[0029] Vertices among multiple vertices may not include texture values ​​in the virtual camera's viewing direction, or may not include texture values ​​that satisfy a deviation angle criterion, such as texture values ​​with a deviation angle smaller than the maximum deviation angle. In this example, to determine the color of the face corresponding to a vertex, one or more processing units may be configured to interpolate texture values ​​that satisfy the deviation angle criterion to that vertex.

[0030] One or more of the at least three vertices of the patch do not include texture values ​​whose corresponding viewing direction satisfies the deviation angle criterion relative to the virtual camera viewing direction. One or more processing units may be configured to interpolate texture values ​​for one or more of the at least three vertices based on texture values ​​whose corresponding viewing direction satisfies the deviation angle criterion relative to the virtual camera viewing direction.

[0031] Applying interpolation to determine texture values ​​negatively impacts the quality of colors on a 3D model compared to not using interpolation. Therefore, it is necessary to provide users with feedback on the quality of colors applied to the 3D model. One or more processing units can be configured to determine a quality score for the color determined for a facet, wherein the quality score can be determined based on the number of at least three vertices of a facet that does not include texture values ​​that satisfy a deviation angle criterion relative to the virtual camera's viewing direction, and wherein the quality score decreases as this number increases. The decrease in quality score when this number increases corresponds to an example where more interpolation is needed to determine the color of the facet.

[0032] One or more processing units can be configured to: assign a feedback color to the patch if the quality score is below a quality threshold; and assign a color to the patch if the quality score is equal to or higher than the quality threshold. The quality threshold can be determined based on the number of interpolations performed to determine the color and / or based on a weighting coefficient level applied to the texture value.

[0033] One or more processing units can be configured to adjust the transparency value of the feedback color based on the quality score. The feedback color can be predefined and stored in the system's memory.

[0034] Multiple texture image data may include one or more of the following wavelengths: ● Visible light wavelengths between 380nm and 750nm ● Ultraviolet wavelengths between 100nm and 380nm, and ● Infrared wavelengths between 750nm and 1350nm.

[0035] The acquired geometric image data may include wavelengths in the visible light wavelength range, and multiple texture image data may include visible light wavelengths, fluorescence wavelengths, and / or infrared wavelengths.

[0036] Multiple texture image data may include wavelength composition, which includes a mixture of visible light wavelengths, fluorescence wavelengths, and / or infrared wavelengths.

[0037] The deviation angle criterion may include one or more of the following: the viewing direction of the texture value is closest to the viewing direction of the virtual camera, the viewing direction of the texture value is closest to the viewing direction of the virtual camera, and the viewing direction of the texture value relative to the viewing direction of the virtual camera is less than the maximum viewing direction angle.

[0038] Intersection criteria may include one or more of the following: vertices among multiple vertices intersect by texture rays, and vertices among multiple vertices are closest to the intersection of the 3D mesh and the texture ray.

[0039] The second intersection criterion may include one or more of the following: the ray from the vertex of the box intersects with a pixel in the box, and the ray from the vertex of the box intersects with the pixel in the box closest to it.

[0040] According to one aspect, a computer-implemented method for assigning texture values ​​to a three-dimensional (3D) model of a dental object is disclosed. The method includes: receiving geometric image data of the dental object and multiple texture image data of the dental object, and corresponding viewing directions relative to the dental object; generating a 3D model of the dental object based on the geometric image data, wherein the 3D model includes a 3D mesh having multiple vertices, and for each of the multiple texture image data, projecting multiple texture rays from at least one pixel in the viewing direction, and wherein at least one pixel corresponds to a texture value; determining whether a vertex among the multiple vertices satisfies an intersection criterion relative to each of the multiple texture rays; and if the vertex satisfies the intersection criterion relative to two or more of the multiple texture rays, assigning two or more texture values ​​and corresponding viewing directions of the two or more texture rays to the vertex. Attached Figure Description

[0041] Various aspects of this disclosure are best understood from the following detailed description taken in conjunction with the accompanying drawings. For clarity, these drawings are schematic and simplified, and they only show details to enhance understanding of the claims, while other details have been omitted. Throughout the drawings, the same or corresponding parts are referred to by the same reference numerals. Individual features of each aspect can be combined with any or all features of other aspects. These and other aspects, features, and / or technical effects will be apparent from and illustrated by the examples described below, in the accompanying drawings: Figure 1 An intraoral scanning system is shown; Figure 2A and 2B Different examples of intersection criteria are shown; Figure 3 This shows another example of how to assign texture values ​​to vertices; Figure 4 An example of rendering a 3D model based on the viewing direction of a virtual camera is shown; Figure 5 An example of one or more processing units is shown; Figure 6A , 6B The 6C illustrates different examples of one or more processing units; Figure 7 Examples of one or more processing units are shown; and Figure 8 An example of interpolated texture values ​​is shown. Detailed Implementation

[0042] The detailed description set forth below, in conjunction with the accompanying drawings, is intended to describe various configurations. The detailed description includes specific details intended to provide a comprehensive understanding of the various concepts. However, those skilled in the art will understand that these concepts can be practiced without these specific details. Various aspects of apparatuses, systems, media, programs, and methods are described by various blocks, functional units, modules, components, circuits, steps, processes, algorithms, etc. (collectively, “elements”). Depending on the specific application, design constraints, or other reasons, these elements may be implemented using electronic hardware, computer programs, or any combination thereof.

[0043] Electronic hardware may include microprocessors, microcontrollers, digital signal processors (DSPs), field-programmable gate arrays (FPGAs), programmable logic devices (PLDs), gated logic, discrete hardware circuits, and other suitable hardware configured to perform the various functions described in this disclosure. Computer programs should be interpreted broadly as instructions, instruction sets, code, code segments, program code, programs, subroutines, software modules, applications, software applications, software packages, routines, subroutines, objects, executable files, threads of execution, procedures, functions, etc., regardless of whether they are referred to as software, firmware, middleware, microcode, hardware description languages, or other names.

[0044] Scanning to provide intraoral scanning data can be performed by a dental scanning system, which may include intraoral scanning devices such as the TRIOS series scanners from 3Shape A / S. The dental scanning system may include wireless functionality provided by a wireless network unit. The scanning device may employ scanning principles such as triangulation-based scanning, confocal scanning, focused scanning, ultrasound scanning, X-ray scanning, stereo vision, motion structure reconstruction, optical coherence tomography (OCT), or any other scanning principle. In one embodiment, the scanning device can obtain surface information by projecting a pattern and translating a focal plane along the optical axis of the scanning device and capturing multiple 2D images located at different focal plane positions, such that each series of captured 2D images corresponding to each focal plane forms a stack of 2D images. The acquired 2D images are also referred to herein as raw 2D images, where "raw" means an image that has not yet undergone image processing. The focal plane position is preferably moved along the optical axis of the scanning system, such that the 2D images captured along the optical axis at several focal plane positions form a stack (also referred to herein as a sub-scan) of the 2D images for a given view of the object (i.e., for a given arrangement of the scanning system relative to the object). After moving the scanning device relative to the object or imaging the object from different views, a new stack of 2D images for that view can be captured. The focal plane position can be changed by at least one focusing element (e.g., a moving focusing lens). During a scanning session, the scanning device is typically moved and angled relative to the dental arch such that at least some sub-scan sets at least partially overlap, enabling the reconstruction of a digital dental 3D model by stitching together the overlapping 3D sub-scans in real time, and displaying the progress of the virtual 3D model on a screen as user feedback. The result of stitching is a digital 3D representation of a surface larger than that that can be captured by a single sub-scan (i.e., larger than the field of view of the 3D scanning device). Stitching, also known as registration and fusion, works by identifying overlapping areas of 3D surfaces in different sub-scans, transforming the sub-scans to a common coordinate system to match the overlapping areas, and ultimately generating a digital 3D model. The Iterative Closest Point (ICP) algorithm can be used for this purpose. Another example of a scanning device is a triangulation scanner, in which a time-varying pattern is projected onto the dental object, and image sequences of different pattern configurations are acquired by one or more cameras positioned at an angle relative to the projector unit.

[0045] Dental scanning systems can be intraoral scanning systems. Scanning devices can be intraoral scanning devices.

[0046] The color texture of a dental object can be obtained by illuminating the object with different monochromatic colors (such as individual red, green, and blue) or by illuminating the object with multicolor light (such as white light). A 2D image can be acquired during the flashing of white light.

[0047] Typically, acquiring real-time surface information of a dental object requires the scanning device to illuminate the surface and acquire a large number of 2D images. High-speed cameras with frame rates of 300-2000 2D frames per second are usually used, depending on the technology and the resolution of the 2D images. The large volume of image data needs to be processed by the scanning device to either directly forward the raw image data stream to an external processing unit or to perform some image processing before transmitting the data to an external device or display. This process requires multiple electronic components within the scanner to operate under high workloads, thus demanding significant current.

[0048] The scanning device includes one or more light projectors configured to generate an illumination pattern to be projected onto a three-dimensional dental object during a scanning session. The light projectors preferably include a light source, a mask signal having a spatial pattern, and one or more lenses (such as collimating lenses or projection lenses). The light source can be configured to produce light of a single wavelength or a combination of wavelengths (monochromatic or polychromatic). Wavelength combinations can be generated by using a light source configured to produce light containing different wavelengths (such as white light). Alternatively, the light projectors may include multiple light sources (such as LEDs) that individually produce light of different wavelengths (such as red, green, and blue light), which can be combined to form light containing different wavelengths. Thus, the light produced by the light source can be defined by wavelengths defining a specific color, or by different wavelength ranges defining color combinations (such as white light). In one embodiment, the scanning device includes a light source configured to excite fluorescent material in the teeth to obtain fluorescence data from the dental object. Such a light source can be configured to produce a narrow wavelength range. In another embodiment, the light from the light source is infrared (IR) light, which is capable of penetrating dental tissue. One or more light projectors may be DLP projectors that use a micromirror array to generate a time-varying pattern, or they may be diffractive optical elements (DOF), or they may be back-illuminated mask signal projectors, wherein a light source is placed after a mask signal having a spatial pattern, thereby patterning the light projected onto the surface of the dental object. The back-illuminated mask signal projector may include a collimating lens for collimating the light from the light source, the collimating lens being placed between the light source and the mask signal. The mask signal may have a checkerboard pattern, such that the generated illumination pattern is a checkerboard pattern. Alternatively, the mask signal may have other patterns, such as lines or dots.

[0049] The scanning device preferably also includes optical components for directing light from the light source onto the surface of the dental object. The specific arrangement of the optical components depends on whether the scanning device is a focusing scanning device, a scanning device using triangulation, or any other type of scanning device. Focusing scanning devices are further described in EP 2 442 720 B1 of the same applicant, the entire contents of which are incorporated herein by reference.

[0050] The optical components of the scanning device guide light reflected from the dental object in response to illumination of the dental object toward one or more image sensors. The image sensors are configured to generate multiple images based on the incident light received from the illuminated dental object. The image sensor units can be high-speed image sensors, such as those configured to acquire images at exposure times of less than 1 / 1000 of a second or frame rates exceeding 250 frames per second (fps). For example, the image sensor can be a rolling shutter (CCD) or a global shutter sensor (CMOS). The image sensors can also be monochrome sensors comprising a color filter array (such as Bayer filters) and / or additional filters configured to substantially remove one or more color components from the reflected light and retain only other unremoved components before converting the reflected light into an electrical signal. For example, such additional filters can be used to remove specific portions of the white light spectrum (such as the blue component) and retain only the red and green components of the signal generated in response to the fluorescent material excitation of the tooth.

[0051] The network unit can be configured to connect a dental scanning system to a network comprising multiple network elements, including at least one network element configured to receive processed data. The network unit can include a wireless network unit or a wired network unit. The wireless network unit is configured to wirelessly connect the dental scanning system to the network comprising multiple network elements, including at least one network element configured to receive processed data. The wired network unit is configured to establish a wired connection between the dental scanning system and the network comprising multiple network elements, including at least one network element configured to receive processed data.

[0052] The dental scanning system preferably also includes a processor configured to generate scan data (such as extraoral and / or intraoral scan data) by processing two-dimensional (2D) images acquired by the scanning device. The processor may be part of the scanning device. For example, the processor may include a field-programmable gate array (FPGA) and / or an advanced RISC machine (ARM) processor located on the scanning device. The scan data contains information related to a three-dimensional dental object. The scan data may contain any of the following forms: 2D images; 3D point clouds; depth data; texture data; intensity data; color data; and / or combinations thereof. For example, the scan data may contain one or more point clouds, where each point cloud contains a set of 3D points describing the three-dimensional dental object. Another example is that the scan data may contain images, each containing image data described, for example, by image coordinates and a timestamp (x, y, t), where depth information can be inferred from the timestamp. The image sensors(s) of the scanning device may acquire multiple raw 2D images of the dental object in response to illumination of the dental object using one or more light projectors. Multiple raw 2D images may also be referred to herein as a 2D image stack. The 2D images can then be provided as input to a processor, which processes the 2D images to generate scan data. Processing the 2D images may include determining which portion of each 2D image is in focus in order to derive / generate depth information from the images. The internal depth information can be used to generate a 3D point cloud, which contains a set of 3D points in a space described, for example, by Cartesian coordinates (x, y, z). The 3D point cloud can be generated by the processor or another processing unit. Each 2D / 3D point may also contain a timestamp indicating when the 2D / 3D point was recorded, i.e., which image in the stack of 2D images the point originated from. The timestamp is associated with the z-coordinate of the 3D point, meaning the z-coordinate can be inferred from the timestamp. Therefore, the processor's output is scan data, which may contain image data and / or depth data, described, for example, by image coordinates and timestamps (x, y, t), or alternatively, the data may be described as (x, y, z). The scanning device can be configured to transmit other types of data in addition to the scan data. Examples of data include 3D information, texture information, such as infrared (IR) images, fluorescence images, reflectance color images, X-ray images, and / or combinations thereof.

[0053] Figure 1An intraoral scanning system 1 is shown, comprising an intraoral scanner 10 configured to acquire geometric image data (3, G) and multiple texture image data (4, T) of a dental object 2. In this example, the dental object 2 is shown as a complete jaw, but it could also be a single tooth, multiple teeth, or the maxilla and / or mandible. In this example, the geometric image data (3, G) and the multiple texture image data (4, T) are acquired separately, i.e., acquired at different timestamps. In another example, the geometric image data (5, G) and the multiple texture image data (5, T) could be acquired simultaneously. System 1 includes one or more processing units 19 configured to generate a 3D model 11 of the dental object 2 based on the geometric image data (3, 5, G), wherein the 3D model 11 includes a 3D mesh with multiple vertices. One or more processing units 19 are further configured to project a plurality of texture rays (13A, 13B) from at least one pixel in a viewing direction (θ1, θ2) for each of a plurality of texture image data (12A, 12B), wherein at least one pixel (P1, P2) corresponds to a texture value (TX1, TX2). One or more processing units 19 are configured to determine whether a vertex 7 among a plurality of vertices satisfies an intersection criterion with respect to each of the plurality of texture rays (13A, 13B), and if vertex 7 satisfies the intersection criterion with respect to two or more texture rays (13A, 13B), then assign two or more texture values ​​(TX1, TX2) and corresponding viewing directions (θ1, θ2) to vertex 7. In this example, vertex 7 is assigned two texture values ​​(TX1, TX2) and corresponding viewing directions (θ1, θ2).

[0054] In addition, system 1 includes a storage unit 14 configured to receive and store a plurality of vertices 7 as well as assigned texture values ​​TX and corresponding viewing directions θ.

[0055] like Figure 1 As shown, two or more texture rays (13A, 13B) have different corresponding viewing directions (θ1, θ2).

[0056] Geometric image data (3, 5, G) and multiple texture image data (4, 5, T) were obtained from the same location relative to dental object 2.

[0057] In the example where geometric image data (3, G) and multiple texture image data (4, T) are acquired separately, each of the acquired texture image data has a first timestamp, and the geometric image data has a second timestamp. One or more processing units 19 are configured to correlate the multiple texture image data with respect to the geometric image data by comparing the first and second timestamps. Furthermore, the geometric image data (3, G) is generated based on green, blue, or red light emitted by the intraoral scanner, while the multiple texture image data are generated based on a combination of green, blue, and red light emitted separately by the intraoral scanner 10. In other examples where geometric image data (5, G) and multiple texture image data (5, T) are acquired simultaneously, the first and second timestamps are identical. In this example, the intraoral scanner 10 is configured to emit white light, which is reflected by the dental object 2 and filtered by a Bayer filter before being acquired by the image sensor. Geometric image data (5, G) and multiple texture image data (5, T) are generated based on the filtered white light, wherein one or more color channels of the Bayer filter are used to generate the geometric image data (5, G) and all color channels of the Bayer filter are used to generate the multiple texture image data (5, T).

[0058] Multiple texture image data (4, 5, T) include one or more of the following wavelengths: visible light wavelengths between 380 nm and 750 nm, ultraviolet wavelengths between 100 nm and 380 nm, and infrared wavelengths between 750 nm and 1350 nm.

[0059] Multiple texture image data (4, 5, T) are synthesized including wavelengths.

[0060] Figure 2A and Figure 2B Different examples of the intersection criteria between vertex 7 and texture ray 13 are shown. In both figures, multiple vertices 7T form multiple patches 21T, where in this particular example, a set of three vertices forms patch 21. Texture ray 12 is projected from pixel P of texture image data 12 toward vertex 7. Figure 2A In this context, the intersection criteria include: vertices intersecting by texture ray 13 will be assigned the corresponding texture value TX of texture ray 13. Figure 2B In this context, the intersection criteria include: the vertex 22 that is closest to the intersection point of texture ray 13A will be assigned the corresponding texture value TX.

[0061] Figure 3 This shows another example of how texture values ​​TX are assigned to vertex 7, and more specifically, to edge vertex 32 and frame vertex 33. Figure 3In the plurality of texture image data 12, each includes a set of edge pixels (P1_edge, P2_edge, P3_edge, P4_edge), which at least partially surround a set of bounding box pixels P of each of the plurality of texture image data 12. One or more processing units 19 are configured to determine a plurality of edge texture rays (13A, 13B, 13C, 13D) in the viewing direction θ of the set of edge pixels (P1_edge, P2_edge, P3_edge, P4_edge) for each of the plurality of texture image data 12. One or more processing units are configured to determine whether a set of edge vertices (32A, 32B, 32C, 32D, 32E) from a plurality of vertices 7T satisfies an intersection criterion with respect to a plurality of edge texture rays (13A, 13B, 13C, 13D), wherein the set of edge vertices (32A, 32B, 32C, 32D, 32E) at least partially encloses a set of bounding vertices (33, 33A, 33B, 33C, 33D) from the plurality of vertices 7T. One or more processing units are configured to determine a bounding vertex ray 31 toward the bounding pixel P in the set of bounding vertices 33 in the viewing direction θ of each bounding vertex (33, 33A, 33B, 33C, 33D). One or more processing units 19 are configured to determine whether the frame vertices (33, 33A, 33B, 33C, 33D) in the frame vertices (33, 33A, 33B, 33C, 33D) satisfy a second intersection criterion between the frame vertex ray 31 of the frame vertex 33 and the frame pixel P in the group of frame pixels P, and if the frame vertex 33 satisfies the second intersection criterion, then the texture value TX of the frame pixel P and the viewing direction θ of the frame vertex ray 31 are assigned to the frame vertex 33.

[0062] The second intersection criterion includes one or more of the following: the vertex ray 33 in the bounding box intersects with the pixel P in the bounding box, and the vertex ray 31 in the bounding box is closest to intersecting with the pixel P in the bounding box. The intersection criterion for texture rays is similar to the second intersection criterion for vertex rays in the bounding box.

[0063] Figure 4An example of rendering a 3D model 11 according to the virtual camera's viewing direction 40 is shown. In this example, each of a plurality of vertices (7A, 7B, 7C) includes at least three texture values ​​TX with corresponding viewing directions (θ1, θ2, θ3). One or more processing units 19 are configured to render the 3D model 11 according to the virtual camera's viewing direction 40, wherein the 3D model 11 includes a plurality of faces 21 formed by a plurality of vertices (7T, 7A, 7B, 7C), such that face 21 of the plurality of faces 21T includes at least three vertices (7A, 7B, 7C) of a plurality of vertices 7T, and for each of the at least three vertices (7A, 7B, 7C), texture values ​​from two or more texture values ​​are selected, wherein the corresponding viewing directions (θ1, θ2, θ3) of the selected texture values ​​TX for each of the at least three vertices (7A, 7B, 7C) satisfy the deviation angle standard θD relative to the virtual camera's viewing direction 40. The deviation angle criterion θD can be that the angular difference between the viewing direction (θ1, θ2, θ3) corresponding to the texture value TX and the virtual camera viewing direction 40 must be less than 90 degrees or less than 180 degrees. One or more processing units 19 are further configured to determine the color of the facet 21 based on the selected texture value for each of at least three vertices. When the 3D model 11 is rotated based on user input 41, the color of each of the plurality of faces 21T of the 3D model changes according to the virtual camera viewing direction 40. One or more processing units 19 are configured to determine the orientation of the 3D model 11 based on the virtual camera viewing direction 40, and wherein the virtual camera viewing direction 40 is determined based on user input 41. The deviation angle criterion θD includes one or more of the following: the viewing direction of the texture value is closest to the virtual camera viewing direction, and the viewing direction of the texture value relative to the virtual camera viewing direction is less than the maximum viewing direction angle.

[0064] Figure 5An example of one or more processing units 19 is shown, configured to render a 3D model 11 according to a virtual camera viewing direction 40 of a virtual camera, wherein the 3D model 11 includes a plurality of faces 21T formed by a plurality of vertices 7T, such that face 21 among the plurality of faces 21T includes at least three vertices (7A, 7B, 7C) of the plurality of vertices 7T. The one or more processing units 19 are configured to select texture values ​​(TX1, TX2, TX3) from two or more texture values ​​TX for each of the at least three vertices (7A, 7B, 7C), wherein the corresponding viewing direction (θ3, θ3, θ3) of the selected texture values ​​(TX1, TX2, TX3) for each of the at least three vertices (7A, 7B, 7C) satisfies the deviation angle standard θ3 relative to the virtual camera viewing direction θD. The one or more processing units 19 are configured to determine the color of face 21 based on the selected texture values ​​(TX1, TX2, TX3) for each of the at least three vertices (7A, 7B, 7C).

[0065] In one example, one or more processing units 19 are configured to determine the color based on the average of selected texture values ​​(TX1, TX2, TX3).

[0066] In yet another example, one or more processing units 19 are configured to determine the color of patch 21 based on a weighted average of selected texture values ​​(TX1, TX2, TX3), wherein each of the selected texture values ​​(TX1, TX2, TX3) has a weighting coefficient that is determined based on the difference between the virtual camera viewing direction 40 of the virtual camera and the viewing direction (θ3, θ3, θ3) of each of the selected texture values ​​(TX1, TX2, TX3).

[0067] Figure 6A , 6B Figures 6C and 6C illustrate different examples of how one or more processing units 19 determine weighted coefficients (W1, W2, W3) of texture values ​​TX(θ1), TX(θ2), and TX(θ3) of vertex 7 relative to the virtual camera viewing direction 40. One or more processing units 19 are configured to assign weighted coefficients (W1, W2, W3) to each of two or more texture values ​​(TX1, TX2, TX3) assigned to vertex 7, and to determine the weighted coefficients (W1, W2, W3) of each of the two or more texture values ​​(TX1, TX2, TX3) based on the difference θD between the virtual camera viewing direction 40 of the virtual camera and the corresponding viewing direction (θ1, θ2, θ3) of each of the two or more texture values ​​(TX1, TX2, TX3).

[0068] The weighting coefficients (W1, W2, W3) are greater than zero when the difference θD between the virtual camera's viewing direction and the corresponding viewing direction of each of two or more texture values ​​is less than or equal to the maximum deviation angle θmax, and the weighting coefficients (W1, W2, W3) are zero when the difference θD between the virtual camera's viewing direction 40 and the corresponding viewing direction (θ1, θ2, θ3) of each of two or more texture values ​​(TX1, TX2, TX3) is greater than the maximum deviation angle θmax.

[0069] The difference between the virtual camera viewing direction 40 and the viewing direction (θ1, θ2, θ3) of the texture value TX is determined based on one or more of the following: the angle between the virtual camera viewing direction 40 and the viewing direction (θ1, θ2, θ3) of the texture value TX, and the cosine of the angle θD between the virtual camera viewing direction 40 and the viewing direction of the texture value TX.

[0070] Figure 6A An example is shown where the virtual camera's viewing direction 40 is closest to the first viewing direction θ1 corresponding to the first texture value TX1, and meets the deviation criterion. In this example, the weighting coefficient W1 of the first texture value TX1 is set to 1, and the remaining weighting coefficients (W2, W3) are set to zero or very small values ​​to reflect the correlation of the texture values. Figure 6B In this example, all observation directions (θ1, θ2, θ3) do not meet the deviation angle criterion. This means that the angular difference between the corresponding observation directions (θ1, θ2, θ3) and the virtual camera's observation direction 40° is greater than the maximum deviation angle θmax. Figure 6C In this context, the virtual camera's viewing direction 40 is located between the first viewing direction θ1 and the second viewing direction θ2 of the corresponding texture values ​​(TX1, TX2), and the corresponding first and second weighting coefficients (W1, W2) are the same. Since the angle difference θD between the third viewing direction θ3 and the virtual camera's viewing direction 40 does not meet the deviation angle standard, the third weighting coefficient W3 is set to zero.

[0071] The deviation angle standard includes the maximum deviation angle, which is between 45 degrees and 180 degrees or between 90 degrees and 180 degrees.

[0072] Figure 7An example of one or more processing units is shown, configured to determine a quality score for a color determined for patch 21, wherein the quality score is determined based on the difference θD between the virtual camera viewing direction 40 and the viewing direction θ1 of each of the selected texture values ​​(TX1, TX2, TX3), and wherein the quality score decreases as the difference increases. In this example, three different virtual camera viewing directions (40A, 40B, 40C) are seen, and each of the virtual camera viewing directions (40A, 40B, 40C) corresponds to a different rendering of the 3D model 11. In all three examples, a second texture value TX2 with a corresponding second viewing direction θ2 is selected, and a second weighting coefficient W2 is set to a value of 1 or close to 1, such as 0.95. A first weighting coefficient W1 is set to a very low value, such as 0.05, and a third weighting coefficient W3 is set to zero. In another example, the second weighting coefficient W2 will vary according to the angular difference θD between the virtual camera viewing direction 40 and the second viewing direction θ2. In the three different examples, the quality scores (Q1, Q2, Q3) vary with the angular difference θD. In this example, the first quality score Q1 is higher than the second and third quality scores (Q2, Q3) because the angular difference θD between the second viewing direction θ2 and the first virtual camera viewing direction 40 is the smallest.

[0073] One or more processing units 19 are configured to: assign a feedback color to patch 21 if the quality score (Q1, Q2, Q3) is lower than the quality threshold; and assign a color to patch 21 if the quality score (Q1, Q2, Q3) is equal to or higher than the quality threshold.

[0074] One or more processing units 19 are configured to adjust the transparency value of the feedback color based on the quality score.

[0075] Figure 8 An example is shown where one or more of at least three vertices (7A, 7B, 7C) of patch 21 do not include texture values ​​TX corresponding to the viewing direction θ3 that satisfy the deviation angle criterion relative to the virtual camera viewing direction 40. In this example, the first vertex 7A does not include texture values ​​TX having a viewing direction θ3 that satisfies the deviation angle criterion relative to the virtual camera viewing direction 40.

[0076] One or more processing units 19 are configured to interpolate the texture value TXI of one or more of the at least three vertices 7A based on one or more (7B, 7C) of at least three vertices that satisfy the deviation angle criterion with respect to the virtual camera's view direction 40, including texture values ​​(TX2, TX3) corresponding to the view direction (θ2, θ3). In this example, one or more processing units 19 are configured to interpolate the texture value TXI having a view direction θi that satisfies the deviation angle criterion. One or more processing units 19 are configured to interpolate the texture value TXI such that the corresponding view direction θi satisfies the deviation angle criterion. The interpolated texture value TXI is determined based on the selected texture value (TX2, TX3) corresponding to the third view direction θ3 that satisfies the deviation angle criterion.

[0077] One or more processing units 19 are configured to determine a quality score of a color determined for a patch 21, wherein the quality score is determined based on the number of one or more vertices of at least three vertices of the patch 21 that do not include texture values ​​TX that satisfy a deviation angle criterion relative to the virtual camera viewing direction 40, and wherein the quality score of the color assigned to the patch is reduced when more interpolation is needed to determine the missing texture values ​​TXI of the corresponding patch 21.

[0078] Many modifications and other embodiments of the present invention will be apparent to those skilled in the art based on the teachings provided in the foregoing description and related drawings. Therefore, it should be understood that the invention is not limited to the specific embodiments disclosed, and that modifications and other embodiments are intended to be included within the scope of the appended claims. Furthermore, although exemplary embodiments have been described in the foregoing description and related drawings in the context of example combinations of certain elements and / or functions, it should be understood that alternative embodiments may provide different combinations of elements and / or functions without departing from the scope of the appended claims. In this regard, for example, combinations of elements and / or functions different from those explicitly described above may also be set forth in some of the appended claims. Although specific terminology is used herein, it is for general and descriptive purposes only and not for limitation.

[0079] project 1A. An intraoral scanning system configured to dynamically adjust multiple colors of a 3D model based on the viewing direction of a virtual camera, wherein the intraoral scanning system includes: The intraoral scanner is configured as follows: Acquire geometric image data of a dental object and multiple texture image data of the dental object from different viewing directions relative to its position on the dental object; One or more processing units, configured as follows: Generating 3D models of dental objects based on geometric image data; Based on multiple texture image data, determine multiple texture values ​​from different viewing directions; Based on the virtual camera's viewing direction and multiple texture values, multiple colors of the 3D model are determined, where the viewing direction of the multiple texture values ​​satisfies the deviation angle standard relative to the virtual camera's viewing direction.

[0080] 1. An intraoral scanning system configured to assign texture values ​​to a three-dimensional (3D) model of a dental object, wherein the intraoral scanning system includes: The intraoral scanner is configured as follows: Acquire geometric image data of a dental object and multiple texture image data of the dental object in different viewing directions relative to its position on the dental object; One or more processing units, configured as follows: Generate 3D models of dental objects based on geometric image data, wherein the 3D model includes a 3D mesh with multiple vertices; For each texture image data in a plurality of texture image data, a plurality of texture rays are projected from at least one pixel in the viewing direction, and at least one pixel corresponds to a texture value; Determine whether a vertex among a plurality of vertices satisfies an intersection criterion with respect to each of a plurality of texture rays; and If a vertex satisfies the intersection criterion with respect to two or more texture rays, then the vertex is assigned two or more texture values ​​and the corresponding viewing directions of the two or more texture rays.

[0081] 2. In the intraoral scanning system described in Project 1, two or more texture rays have different corresponding observation directions.

[0082] 3. The intraoral scanning system according to Project 1, wherein geometric image data and multiple texture image data are acquired from the same position relative to the dental object.

[0083] 4. The intraoral scanning system according to any one of the preceding claims, wherein each of the acquired plurality of texture image data has a first timestamp and the geometric image data has a second timestamp, and wherein one or more processing units are configured to correlate the plurality of texture image data relative to the geometric image data by comparing the first timestamp and the second timestamp.

[0084] 5. The intraoral scanning system according to any one of the preceding claims, wherein each of the plurality of texture image data includes a set of edge pixels that at least partially surround a set of bounding pixels in each of the plurality of texture image data, and one or more processing units are configured to: Determine multiple edge texture rays in the viewing direction of the group of edge pixels, wherein the multiple edge texture rays are part of multiple texture rays; Determine whether a set of edge vertices among a plurality of vertices satisfies an intersection criterion with respect to a plurality of edge texture rays, wherein the set of edge vertices at least partially encloses a set of bounding vertices among the plurality of vertices; Determine the frame vertex ray in the viewing direction of each frame vertex in the set of frame vertices that are directed toward the pixel in the set of frame vertices; Determine whether the vertices within the bounding box satisfy the second intersection criterion between the vertex rays of the vertices within the bounding box and the bounding pixels in the same group of bounding boxes, and If the vertex satisfies the second intersection criterion, then the vertex is assigned the texture value of the pixel in the bounding box and the view direction of the vertex ray in the bounding box.

[0085] 6. The intraoral scanning system according to any one of the preceding claims, wherein one or more processing units are configured as follows: Render a 3D model based on the virtual camera's viewing direction, wherein the 3D model comprises multiple faces formed by multiple vertices, such that each face contains at least three vertices of the multiple vertices; For each of at least three vertices, select two or more texture values, where the corresponding viewing direction of the selected texture values ​​for each of the at least three vertices satisfies a deviation angle criterion relative to the virtual camera's viewing direction; and The color of the face is determined based on the texture value selected for each of at least three vertices.

[0086] 7. The intraoral scanning system according to Item 6, wherein one or more processing units are configured to determine the orientation of a 3D model based on the viewing direction of a virtual camera, and wherein the viewing direction of the virtual camera is determined based on user input.

[0087] 8. The intraoral scanning system according to any one of the preceding claims, wherein one or more processing units are configured as follows: Assign a weighting coefficient to each of the two or more texture values ​​assigned to a vertex, and The weighting coefficients for each of the two or more texture values ​​are determined by the difference between the virtual camera's viewing direction and the corresponding viewing direction of each of the two or more texture values.

[0088] 9. The intraoral scanning system according to Item 8, wherein the weighting coefficient is greater than zero when the difference between the virtual camera observation direction and the corresponding observation direction of each of two or more texture values ​​is less than or equal to the maximum deviation angle; and wherein the weighting coefficient is zero when the difference between the virtual camera observation direction and the corresponding observation direction of each of two or more texture values ​​is greater than the maximum deviation angle.

[0089] 10. The intraoral scanning system according to Item 6, wherein one or more processing units are configured to determine color based on the average of selected texture values.

[0090] 11. The intraoral scanning system according to Item 6, wherein one or more processing units are configured to determine color based on a weighted average of selected texture values, and wherein each of the selected texture values ​​has a weighting coefficient determined based on the difference between the virtual camera viewing direction of the virtual camera and the viewing direction of each of the selected texture values.

[0091] 12. The intraoral scanning system according to any one of items 8 to 11, wherein the difference between the virtual camera viewing direction and the texture value viewing direction is determined based on one or more of the following: The angle between the virtual camera's viewing direction and the texture value's viewing direction; and The cosine of the angle between the virtual camera's viewing direction and the texture value's viewing direction.

[0092] 13. The intraoral scanning system according to Item 6, wherein one or more processing units are configured to determine a quality fraction of a color determined for a patch, and wherein the quality fraction is determined based on the difference between the viewing direction of the virtual camera and the viewing direction of each of the selected texture values, and wherein the quality fraction decreases as the difference increases.

[0093] 14. The intraoral scanning system according to any one of items 6 to 13, wherein one or more of at least three vertices of the patch do not include texture values ​​that satisfy a deviation angle criterion for the corresponding viewing direction relative to the virtual camera viewing direction, and one or more processing units are configured to interpolate the texture values ​​of one or more of the at least three vertices based on the texture values ​​that satisfy the deviation angle criterion for the corresponding viewing direction relative to the virtual camera viewing direction.

[0094] 15. The intraoral scanning system according to item 14, wherein one or more processing units are configured to determine a quality fraction of a color determined for a facet, and wherein the quality fraction is determined based on the number of at least three vertices of a facet that does not include texture values ​​that satisfy a deviation angle criterion relative to the virtual camera's viewing direction, and wherein the quality fraction decreases as the number increases.

[0095] 16. The intraoral scanning system according to item 13 or 15, wherein one or more processing units are configured as follows: • If the quality score is below the quality threshold, assign a feedback color to the patch; and • If the quality score is equal to or higher than the quality threshold, then assign a color to the face.

[0096] 17. The intraoral scanning system according to item 13 or 15, wherein one or more processing units are configured to adjust the transparency value of the feedback color based on the mass fraction.

[0097] 18. The intraoral scanning system according to any one of the preceding claims, wherein the plurality of texture image data includes one or more of the following wavelengths: • Visible light wavelengths between 380nm and 750nm; • Ultraviolet wavelengths between 100nm and 380nm; and • Infrared wavelengths between 750nm and 1350nm.

[0098] 19. The intraoral scanning system according to item 18, wherein multiple texture image data include wavelength composition.

[0099] 20. The intraoral scanning system according to any of the preceding claims, wherein the deviation angle criterion includes one or more of the following: The viewing direction of texture values ​​is closest to that of the virtual camera, and The viewing direction of the texture value is smaller than the maximum viewing direction angle relative to the virtual camera's viewing direction.

[0100] 21. The intraoral scanning system according to any of the preceding claims, wherein the intersection criteria include one or more of the following: Vertices among multiple vertices are intersected by texture rays, and The vertex among multiple vertices is closest to the intersection of the 3D mesh and the texture ray.

[0101] 22. The intraoral scanning system according to Item 5, wherein the second intersection criterion includes one or more of the following: The ray from the vertex of the box intersects with the pixel in the box, and The ray from the vertex of the box intersects the pixel within the box most closely.

[0102] 23. A computer-implemented method for assigning texture values ​​to a three-dimensional (3D) model of a dental object, the method comprising: Receives geometric image data of a dental object and multiple texture image data of the dental object, as well as the corresponding viewing direction relative to the dental object; Generate 3D models of dental objects based on geometric image data, wherein the 3D model includes a 3D mesh with multiple vertices; For each of the multiple texture image data, multiple texture rays are projected from at least one pixel in the viewing direction, and at least one pixel corresponds to a texture value; Determine whether a vertex among a plurality of vertices satisfies an intersection criterion with respect to each of a plurality of texture rays; and If a vertex satisfies the intersection criterion with respect to two or more texture rays, then the vertex is assigned two or more texture values ​​and the corresponding viewing directions of the two or more texture rays.

Claims

1. An intraoral scanning system configured to assign texture values ​​to a three-dimensional (3D) model of a dental object, wherein, The intraoral scanning system includes: An intraoral scanner, wherein the intraoral scanner is configured as follows: Acquire geometric image data of a dental object and multiple texture image data of the dental object in different viewing directions relative to its position on the dental object; One or more processing units, the one or more processing units being configured to: A 3D model of the dental object is generated based on the geometric image data, wherein the 3D model includes a 3D mesh with multiple vertices; For each of the plurality of texture image data, a plurality of texture rays are projected from at least one pixel in the viewing direction, wherein the at least one pixel corresponds to a texture value; Determine whether a vertex among the plurality of vertices satisfies an intersection criterion with respect to each of the plurality of texture rays. If the vertex satisfies the intersection criterion with respect to two or more of the plurality of texture rays, then two or more texture values ​​and the corresponding viewing directions of the two or more texture rays are assigned to the vertex; The 3D model is rendered by applying two or more texture values ​​to each vertex, the texture values ​​corresponding to the viewing direction of the virtual camera being matched with the virtual camera's viewing direction.

2. The intraoral scanning system according to claim 1, wherein, The two or more texture rays have different viewing directions.

3. The intraoral scanning system according to claim 1, wherein, The geometric image data and the plurality of texture image data are obtained from the same position relative to the dental object.

4. The intraoral scanning system according to any one of the preceding claims, wherein, Each of the acquired multiple texture image data has a first timestamp, and the geometric image data has a second timestamp, wherein the one or more processing units are configured to correlate the multiple texture image data with respect to the geometric image data by comparing the first timestamp and the second timestamp.

5. The intraoral scanning system according to any one of the preceding claims, wherein, Each of the plurality of texture image data includes a set of edge pixels, the set of edge pixels at least partially surrounding a set of bounding pixels in each of the plurality of texture image data, and the one or more processing units are configured to, for each of the plurality of texture image data: Determine multiple edge texture rays in the viewing direction of the set of edge pixels, wherein the multiple edge texture rays are a part of the multiple texture rays; Determine whether a set of edge vertices among the plurality of vertices satisfies the intersection criterion relative to the plurality of edge texture rays, wherein the set of edge vertices at least partially encloses a set of box vertices among the plurality of vertices; Determine the frame vertex ray in the viewing direction of each frame vertex in the set of frame vertices toward the set of frame pixels; Determine whether a vertex in a frame satisfies the second intersection criterion between the top ray of the vertex in the frame and a pixel in the frame of the set of pixels in the frame, and If the vertices in the frame satisfy the second intersection criterion, then the texture value of the pixel in the frame and the viewing direction of the ray of the vertex in the frame are assigned to the vertices in the frame.

6. The intraoral scanning system according to any one of the preceding claims, wherein, The one or more processing units are configured as follows: The 3D model is rendered according to the virtual camera's viewing direction, wherein the 3D model includes multiple facets formed by the multiple vertices, such that each facet contains at least three vertices from the multiple vertices; For each of the at least three vertices, a texture value is selected from two or more texture values, wherein the corresponding viewing direction of the selected texture value for each of the at least three vertices satisfies a deviation angle criterion relative to the virtual camera's viewing direction; and The color of the face is determined based on the texture value selected for each of the at least three vertices.

7. The intraoral scanning system according to claim 6, wherein, The one or more processing units are configured to determine the orientation of the 3D model based on the virtual camera's viewing direction, wherein the virtual camera's viewing direction is determined based on user input.

8. The intraoral scanning system according to any one of the preceding claims, wherein, The one or more processing units are configured as follows: Assign a weighting coefficient to each of the two or more texture values ​​assigned to the vertex, and The weighting coefficient for each of the two or more texture values ​​is determined based on the difference between the virtual camera's viewing direction and the corresponding viewing direction of each of the two or more texture values.

9. The intraoral scanning system according to claim 8, wherein, The weighting coefficient is greater than zero when the difference between the virtual camera's viewing direction and the corresponding viewing direction of each of the two or more texture values ​​is less than or equal to the maximum deviation angle; and wherein the weighting coefficient is zero when the difference between the virtual camera's viewing direction and the corresponding viewing direction of each of the two or more texture values ​​is greater than the maximum deviation angle.

10. The intraoral scanning system according to claim 6, wherein, The one or more processing units are configured to determine the color based on the average of selected texture values.

Images