System and method for removing artifacts caused by reflections

Abstract

The present disclosure relates to a 3D scanner system comprising an intraoral scanner comprising: an elongate housing comprising a distal end for insertion into an oral cavity, where the housing comprises an aperture in a sidewall of the distal end of the housing; and a window disposed in the hole of the housing; and / or a sleeve mounted on the exterior of the elongated housing; the 3D scanner system also includes one or more processors operably connected to the intraoral scanner, the processors configured to: receive one or more two-dimensional images comprising a plurality of pattern features, where the images comprise one or more artifacts caused by reflections from the window and / or sleeve; and providing the two-dimensional image as input to a parameterized model trained to determine the location of at least a subset of pattern features in the image while suppressing or ignoring the artifacts.

Description

Technical Field

[0001] The present disclosure relates to a computer-implemented method for improving the accuracy of a three-dimensional (3D) representation. The present disclosure also relates to a 3D scanner system comprising one or more processors configured to perform the steps of the method disclosed herein. The present disclosure also relates to a sleeve for an intraoral scanner. Background Art

[0002] 3D scanning technology is widely used in dentistry for creating digital impressions of teeth. This technology enables dentists to obtain detailed and accurate representations of teeth in a digital format, which can then be used for a variety of applications, including the design and manufacture of dental restorations, orthodontic appliances, and other dental prostheses.

[0003] Typically, the digital impression is generated using an intraoral scanner that is manipulated inside the patient's mouth. Since the intraoral scanner is in contact with the patient, an important aspect thereof is hygiene. Some existing solutions utilize a sterile tip of the scanner, for example sterilized using an autoclave, to ensure a hygienic distal end of the scanner. Other proposed solutions utilize a disposable tip, i.e., the scanning tip is suitable for use only on a single patient. Still other solutions utilize a sheath for the tip of the scanner (such as the sheath disclosed by 3Shape A / S in WO 2020 / 053136 A1).

[0004] However, a problem associated with all existing solutions is that the scanning tip and / or the sheath typically cause many undesired reflections in the image captured by the intraoral scanner. The reflections are typically caused by, for example, specular reflection(s) of the light projected by the scanner. These undesired reflections lead to artifacts in the image. Sometimes, in case the scanner projects a structured pattern, this pattern is reproduced as echoes in the image, which complicates the image processing performed by the 3D scanner system. As a result, the accuracy of the generated digital impression may be affected, or in some cases, erroneous 3D points are generated based on the artifacts.

[0005] Therefore, there is a need for improved methods and systems to remove artifacts caused by reflections, particularly in applications using 3D scanner systems or intraoral scanners.Furthermore, there is a need for new and improved hygiene solutions for intraoral scanning applications. Summary of the invention

[0006] Generally speaking, intraoral scanners utilize structured light projected onto the patient's dentition in order to generate a 3D model of the dentition. Since the intraoral scanner is a medical device used inside the patient's mouth, it is very important to ensure the hygiene and cleanliness of the device. As mentioned above, existing solutions typically use disposable scanning tips, autoclavable scanning tips, or in some cases a sleeve covering the scanner head. All of the mentioned solutions introduce a transparent hygienic barrier in the optical path of the scanner, i.e., in the path of the projection light from the scanner. A problem associated with this is that the barrier causes some unwanted reflections; in particular, specular reflections caused by the projection light from the scanner. Specular reflections from the barrier are undesirable because they complicate the task of determining the 3D topography of the scanned object.

[0007] Furthermore, most scanners rely on capturing reflected light from the scanned object (including the projected pattern on the object). If the pattern or a portion of the pattern is reflected by the sanitary barrier, the scanner captures an "echo" of the reflected pattern, which is undesirable since it has nothing to do with the surface of the scanned object. As a result, the captured image may include artifacts caused by reflections from the sanitary barrier, which likewise complicates the generation of 3D points of the scanned surface.

[0008] The present disclosure solves the above challenges by providing a 3D scanner system, the 3D scanner system comprising:

[0009] Intraoral scanner, which includes:

[0010] an elongated housing including a distal end for insertion into the oral cavity, wherein the housing includes an aperture in a sidewall of the distal end of the housing; and

[0011] a window disposed in the aperture of the housing; and / or

[0012] a sleeve mounted on the exterior of the elongated housing, wherein the sleeve covers the aperture;

[0013] One or more processors operably connected to the intraoral scanner, the processors configured to:

[0014] receiving one or more two-dimensional images comprising a plurality of pattern features, wherein the images include one or more artifacts caused by reflections from the window and / or sleeve; and

[0015] The two-dimensional image(s) are provided as input to a parameterized model that is trained to determine the locations of at least a subset of pattern features in the image(s) while suppressing or ignoring artifacts.

[0016] In a preferred embodiment, one or more parameters of the parameterized model have been selected or adjusted so that artifacts in the image are suppressed or ignored by the parameterized model. The parameterized model can be a machine learning model, such as a neural network. In some embodiments, the parameterized model is a convolutional neural network that is trained to suppress or ignore artifacts and / or ignore portions of the spatial pattern that have been reflected by the window and / or sleeve.

[0017] The inventors have recognized that a trained parameterized model is useful for achieving the desired result of ignoring artifacts, so that 3D points are determined more accurately, and / or fewer erroneous 3D points are generated. In particular, the inventors have recognized how to train a parameterized model to ignore artifacts, for example, how to generate training data, what model and architecture to choose, etc. It is advantageous to split the problem into two, so that the reference true value of some data is known, and other data only represents artifacts. In this case, the reference true value can be understood as the real position of a pattern feature in the image, which pattern feature forms part of the projection pattern of the scanner. The reference true value can be generated by rendering a 3D model of a previously scanned object or by using real 3D data of the scanned object without any windows or sleeves. The rendered image may have been generated using a geometric virtual model of the intraoral scanner, wherein the geometric virtual model contains external parameters and / or internal parameters of the projector unit and camera unit of the scanner.

[0018] The remainder of the training data may advantageously be two-dimensional training images, which ideally contain only artifacts. These training images may have been acquired through the window and / or sleeve, with no objects in focus other than the window and / or sleeve. This ensures that only window / sleeve reflections are recorded in the training images. This approach is an example of how to ensure that the foreground (represented by the window / sleeve surface) can be separated from the background (e.g., the image of the scanned surface).

[0019] The present disclosure also relates to a computer-implemented method comprising the following steps:

[0020] receiving one or more two-dimensional images acquired by one or more camera units forming part of an intraoral scanner, wherein each image comprises at least a portion of the spatial pattern comprising a plurality of pattern features, wherein the image(s) further comprises one or more artifacts caused by reflections from a window and / or sleeve of the intraoral scanner; and

[0021] The (one or more) two-dimensional image is provided as input to a parametric model, which is configured to determine the location of at least a subset of the pattern features in the (one or more) images, wherein one or more parameters of the parametric model are selected so that artifacts caused by reflections from the window and / or sleeve are suppressed or ignored.

[0022] Any implemented computer-implemented method disclosed herein may be performed in whole or in part by a 3D scanner system disclosed herein. The present disclosure also relates to a data processing system (such as a 3D scanner system) comprising one or more processors configured to perform the steps of the disclosed method. The present disclosure also relates to a computer program product comprising instructions that, when executed by a computer, cause the computer to perform the steps of the disclosed computer-implemented method. The computer may form part of a 3D scanner system. The present disclosure also relates to a computer-readable data carrier having stored thereon the computer program product.

[0023] In another aspect, the present disclosure relates to a 3D scanner system comprising:

[0024] Intraoral scanner, which includes:

[0025] an elongated housing including a distal end for insertion into the oral cavity, wherein the housing includes an aperture in a sidewall of the distal end of the housing; and

[0026] a window disposed in the aperture of the housing; and / or

[0027] a sleeve mounted on an exterior of the elongated housing, wherein the sleeve covers the aperture;

[0028] One or more processors operably connected to the intraoral scanner, the processors configured to:

[0029] receiving one or more two-dimensional images including a plurality of pattern features and one or more artifacts caused by reflections from the window and / or sleeve; and

[0030] subtracting a previously acquired background image from each of the images, wherein the images were acquired with no objects in focus other than the window and / or sleeve, thereby obtaining a modified two-dimensional image having a reduced number of artifacts; and

[0031] A digital three-dimensional representation is generated based on the modified two-dimensional image.

[0032] The previously acquired background image may represent a window and / or sleeve without an object behind it (i.e., with a black background). In this way, reflections / artifacts are recorded directly in the background image, which may then be subtracted from the received two-dimensional image of the scanned object to ideally remove all artifacts.

[0033] Thus, the presently disclosed systems and methods are able to suppress or ignore artifacts caused by window and / or sleeve reflections, thereby generating a more accurate 3D model of the scanned object. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 A flow chart of an embodiment of a computer-implemented method according to the present disclosure is shown.

[0035] Figure 2 An embodiment of a 3D scanner system according to the present disclosure is shown.

[0036] Figure 3 A flow chart of an embodiment of a computer-implemented method according to the present disclosure is shown.

[0037] Figure 4 A computer system is shown in which the embodiments of the present disclosure or a part thereof can be implemented as a computer-readable code.

[0038] Figure 5 An intraoral scanner and sleeve are shown according to an embodiment of the present disclosure.

[0039] Figure 6 An intraoral scanner and sleeve are shown according to an embodiment of the present disclosure.

[0040] Fig. 7A Shown is a single sub-scan (3D representation) generated by a scanner if the parametric model has not been trained to ignore or suppress artifacts.

[0041] Figure 7B Several sub-scans of a 3D model fused into noise-displaced elements resembling a noise plane or plane are shown.

[0042] Fig. 8A A raw image of some teeth is shown, wherein the image is captured through a window arranged in a bore of a housing of an intraoral scanner.

[0043] Figure 8B Shows Fig. 7A A digital 3D model of the same tooth shown in , wherein the 3D model is generated based on multiple captured images of the tooth.

[0044] Fig.9A Some raw images of teeth are shown, where the image is captured through a window and a sleeve, where the sleeve covers the window.

[0045] Fig. 9B A reproduction of a digital 3D model is shown, where an image has been captured through both a window and a sleeve covering the window, where a parameterized model has been trained to ignore or suppress artifacts from the window and / or sleeve. DETAILED DESCRIPTION

[0046] Intraoral Scanner

[0047] The scanner disclosed herein may be an intraoral scanner for acquiring intraoral images of a subject. The scanner may be a handheld scanner, i.e., a device configured for holding with a human hand. The scanner may employ any suitable scanning principle, such as focus-based scanning, triangulation-based scanning, stereoscopic vision, structure from motion, confocal scanning, or other scanning principles.

[0048] In some embodiments, the scanner adopts a scanning principle based on triangulation. As an example, the scanner may include a projector unit and one or more camera units for determining points in 3D space based on triangulation. As another example, the scanner includes a projector unit and two or more camera units, wherein the camera units are configured to image the scanned object from separate views (i.e., from different directions). In particular, the camera units can be configured to acquire a set of images, wherein corresponding problems are solved within the set of images based on triangulation. The images in the set of images can be acquired by separate camera units of the scanner.

[0049] The triangulation-based intraoral scanner is further described in the following applications by the same applicant: WO 2023 / 117981 A1 “Systems and methods for generating a digital representation of a 3D object” filed on December 19, 2022, PCT / EP2023 / 058521 “Intraoral 3D scanning device for projecting a high-density light pattern” filed on March 31, 2023, and PCT / EP2023 / 058980 “Intraoral scanning device with extended field of view” filed on April 5, 2023, which patent applications are incorporated herein by reference in their entirety.

[0050] The intraoral scanner may include one or more scanning units, each of which includes a projector unit and one or more camera units. As an example, the scanner may include one scanning unit, the one scanning unit including one projector unit and at least two camera units. As another example, the scanner may include one scanning unit, the scanning unit including one projector unit and four camera units. In yet another example, the scanner may include at least two scanning units, each of which includes a projector unit and two or more camera units. In yet another example, the scanner may include at least two scanning units, each of which includes a projector unit and four camera units. In some embodiments, the scanning units may be arranged in series to achieve an extended or enlarged field of view (FOV) of the intraoral scanner. Thus, each scanning unit may provide a FOV such that the two scanning units provide a combined FOV that is approximately twice the FOV of a single scanning unit.

[0051] case

[0052] The intraoral scanner may include an elongated housing including a distal end for insertion into the oral cavity. The housing may be configured to accommodate a plurality of optical and / or electronic components of the scanner. The housing may constitute a rigid housing of the scanner. The housing may include a hole in a side wall at the distal end of the housing. The hole is preferably arranged so that it is located in the optical path of the light projected by the scanner. In some embodiments, the housing of the scanner includes two or more parts, such as a fixed part and a replaceable part, the replaceable part being configured to be replaceably attached to the fixed part of the housing. As an example, the scanner may include a replaceable scanning end configured to be mounted to the scanner or an elongated housing. In this case, the scanning end may include a hole or window configured to align with a corresponding hole or window of the elongated housing.

[0053] In some embodiments, the intraoral scanner comprises a window arranged in a hole of the housing and / or the scanning end. Thus, the scanner may comprise a window arranged in a hole of the housing, wherein the projector unit and the window are arranged so that a pattern is projected through the window during use of the scanner. In a preferred embodiment, at least one area of ​​the window is made of a transparent material, which is located in the optical path of the projected spatial pattern. The window may be made of a polymer, such as polymethyl methacrylate (PMMA), or a glass material, such as sapphire glass. The window is preferably rigid and planar. In some embodiments, the window is coated with an anti-reflective (AR) coating. The window may be located in the optical path of the scanner, i.e., positioned so that a projection pattern of light is projected through the window during use of the scanner.

[0054] Projector unit

[0055] The intraoral scanner may comprise one or more projector units. Each projector unit may be configured to project a predefined spatial pattern through the aperture onto at least a portion of the surface of the three-dimensional object.

[0056] A projector unit may be understood herein as a device configured to project light onto a surface, such as a surface of a three-dimensional object. In a preferred embodiment, the projector unit is configured to project a pattern of light, such as a spatial pattern, onto a surface of a three-dimensional object, such as a dental object. Preferably, the projector unit is configured to project a pattern of light such that the spatial pattern is focused at a predefined focal length or range within a given tolerance when measured along the projector optical axis. Thus, the spatial pattern may be in acceptable focus within a given focus range. The projector unit may be configured to project unpolarized light.

[0057] The projector unit may be selected from the following group: a digital light processing (DLP) projector using a plurality of digital micromirror devices (DMD) arranged in a matrix or array, or a diffractive optical element (DOE), or a front-lit reflective mask projector, or a micro-LED projector, or a liquid crystal on silicon (LCoS) projector or a backlit mask projector, wherein the light source is placed behind a mask having a spatial pattern. The pattern may be dynamic, i.e. such that the pattern changes over time, or the pattern may be static over time, i.e. such that the pattern remains unchanged over time. The advantage of projecting a static pattern is that it allows all image data to be captured simultaneously, thus preventing distortions due to movement between the scanner and the object.

[0058] The projector unit may include one or more collimating lenses for collimating light from the light source.(One or more) collimating lenses may be placed between the light source and the mask in the backlight mask projector unit. In some embodiments, the one or more collimating lenses are Fresnel lenses. The projector unit may also include one or more focusing lenses configured to focus the projected spatial pattern at a predefined focus range. The projector focusing lens may define the projector optical axis.

[0059] In a preferred embodiment, the projector unit of the scanner comprises at least one light source and a pattern generating element for defining a pattern of light. The pattern generating element is preferably configured to generate a light pattern to be projected onto the surface of the object. As an example, the pattern generating element may be a mask having a spatial pattern. Thus, the projector unit may include a mask configured to define a pattern of light. The mask may be placed between the light source of the projector unit and the one or more focusing lenses so that the light transmitted through the mask is patterned into a spatial pattern. As an example, the mask may define a polygonal pattern comprising a plurality of polygons, such as a checkerboard pattern. In other embodiments, the pattern generating element generates a light pattern based on diffraction and / or refraction, such as a pattern comprising an array of discrete, unconnected small dots.

[0060] Preferably, the projector unit is configured to generate a predefined static pattern which can be projected onto a surface. Alternatively, the projector unit can be configured to generate a dynamic pattern which changes over time, i.e. a time-varying pattern. The projector unit can be associated with its own projector plane, which is determined by the projector optics. As an example, if the projector unit is a backlit mask projector, the projector plane can be understood as a plane containing the mask. The projector plane includes a plurality of pattern features of the projected pattern.

[0061] light source

[0062] The projector unit may include one or more light sources. The projector unit may be configured to project a spatial pattern of light defined by a plurality of projector rays when the (one or more) light sources are turned on / activated. The projector unit may be configured to sequentially turn the light sources on and off at a predetermined frequency, wherein the light sources are turned on for a predetermined period of time. The (one or more) light sources may be configured to generate light of a single wavelength or a combination of wavelengths (monochromatic or polychromatic). The combination of wavelengths may be generated by a light source configured to generate light comprising different wavelengths or wavelength ranges (such as white light). The light source may be configured to generate non-polarized light, such as non-polarized white light.

[0063] In some embodiments, each projector unit includes a light source for generating white light. Its advantage is that white light enables the scanner to simultaneously obtain data or information related to surface geometry and surface color. Therefore, the same set of images can be used to provide both the geometry of the object (e.g., according to 3D data / 3D representation) and the color of the object. Therefore, there is no need to align the data related to the recorded surface geometry and the data related to the recorded surface color in order to generate a digital 3D representation of the object expressing both the color and geometry of the object. Alternatively, the projector unit may include a plurality of light sources (such as LEDs) that individually generate light of different wavelengths (such as red, green, and blue), which can be combined to form light including different wavelengths. Therefore, the light generated by (one or more) light sources can be defined by the wavelength defining a specific color, or by the range of different wavelengths defining a color combination (such as white light). In some embodiments, the light source is a diode, such as a white light diode or a laser diode. In some embodiments, the projector unit includes a laser, such as a blue laser diode or a green laser diode for generating blue light or green light, respectively. This has the advantage that a more efficient projector unit can be achieved, which enables faster exposures than using, for example, white light diodes.

[0064] In some embodiments, the scanner includes a light source configured to excite a fluorescent material to obtain fluorescent data from a dental object (such as a tooth). Such a light source can be configured to produce a narrow range of wavelengths. In other embodiments, the scanner includes one or more infrared light sources configured to generate wavelengths within an infrared range (such as between 700nm and 1.5μm). In some embodiments, the scanner includes one or more light sources selected from the following group: an infrared (IR) light source, a near infrared (NIR) light source, a blue light source, a violet light source, an ultraviolet (UV) light source, and / or a combination thereof. In some embodiments, the scanner includes a first light source forming part of a projector unit, and one or more second light sources (e.g., (one or more) IR-LEDs or (one or more) NIR-LEDs and / or (one or more) blue or violet LEDs) located at a distal portion of the scanner (such as the end of the scanner). Some of the light sources can be used for diagnostic purposes, such as to help detect areas of caries or plaque.

[0065] Light Pattern

[0066] The projector unit can be configured to project a spatial pattern of light defined by a plurality of projector lights when the light source of the projector unit is turned on. The spatial pattern can include a plurality of pattern features. In some embodiments, the pattern features are arranged in a two-dimensional (2D) grid. The terms "illumination pattern", "pattern of light", "spatial pattern" and "projected pattern" are used interchangeably herein.

[0067] The spatial pattern can be generated using a pattern generating element, for example, located in the projector unit. The pattern generating element can be a mask having a spatial pattern, such as a transparent mask or a transmission mask. The mask can be a chrome photomask. In other embodiments, the pattern generating element is configured to generate a light pattern using diffraction and / or refraction. The use of patterns of light can lead to a correspondence problem, in which it is necessary to determine the correspondence between points in the light pattern and points seen by (one or more) camera units viewing the pattern. In some embodiments, the correspondence problem is solved jointly for multiple groups of projector light rays emitted from the projector unit.

[0068] The spatial pattern can be a polygonal pattern comprising a plurality of polygons. The polygon can be selected from the following group: a triangle, a rectangle, a square, a pentagon, a hexagon and / or a combination thereof. Other polygons can also be envisioned. In general, a polygon consists of an edge and an angle. In a preferred embodiment, the polygon is repeated in a pattern in a predefined manner. As an example, the pattern can include a plurality of repeating units, wherein each repeating unit includes a predefined number of polygons, wherein the repeating unit is repeated throughout the pattern. Alternatively, the pattern can include a predefined arrangement, and the predefined arrangement includes any one or more of stripes, squares, dots, triangles, rectangles and / or a combination thereof. In some embodiments, the pattern is non-coded so that any part of the pattern is not unique.

[0069] In some embodiments, the pattern of light generated is a polygonal pattern, such as a checkerboard pattern comprising a plurality of checkerboard squares. Similar to a normal chessboard, the checkerboard in the pattern can have alternating dark and light areas corresponding to areas of low light intensity (dark) and areas of high (higher) light intensity (bright). In some embodiments, the pattern of light is a checkerboard pattern comprising alternating dark and bright light squares. In other embodiments, the pattern of light comprises a distribution of discrete, unconnected light points.

[0070] The spatial pattern preferably includes a plurality of pattern features. The pattern features can be arranged in a regular grid. In some embodiments of the currently disclosed scanner, the total number of pattern features in the pattern is at least 1000, preferably at least 3000, more preferably at least 10000, and even more preferably at least 15000. When a pattern including such pattern features is projected onto the surface of a 3D object, the image of the acquired object will similarly include a plurality of pattern features corresponding to the projected pattern features. Pattern features can be understood as separate well-defined positions in a pattern or image. Examples of pattern features include corners, edges, vertices, points, transitions, dots, stripes, etc. In a preferred embodiment, pattern features include checkerboard corners in a checkerboard pattern. In other embodiments, pattern features include corners in a polygonal pattern (such as a triangular pattern).

[0071] Camera unit

[0072] The intraoral scanner may include one or more camera units configured to acquire one or more two-dimensional images of a three-dimensional object. Preferably, at least a portion of the spatial pattern is present in the image. A camera unit may be understood herein as a device for capturing an image of an object. Each camera unit may include an image sensor for generating an image based on incident light received, for example, from an illuminated 3D object. As an example, the image sensor may be an electronic image sensor, such as a charge coupled device (CCD) or an active pixel sensor (CMOS sensor). In some embodiments, the image sensor is a global shutter sensor configured to simultaneously expose the entire image area (all pixels) and generate an image at a single point in time. In other embodiments, the image sensor is a rolling shutter sensor.

[0073] The (one or more) image sensors may include a pixel array, wherein each pixel is associated with a corresponding camera light. The pixel array may be a two-dimensional (2D) array. Each pixel may be covered by a microlens. In some embodiments, the image area (i.e., the 2D pixel array) is rectangular or square. In some embodiments, the image sensor is a CMOS sensor, which includes an analog-to-digital converter (ADC) for each column of pixels, making the conversion time significantly faster and allowing each camera unit to benefit from greater speed. Each image sensor may define an image plane, which is a plane containing a projected image of an object. Each image obtained by the (one or more) image sensors may include multiple pattern features of a pattern derived from a projection including multiple such pattern features. In some embodiments, one or more camera units include a light field camera. Preferably, each camera unit defines a camera optical axis. The camera unit may also include one or more focusing lenses for focusing light.

[0074] In some embodiments, the image sensor is a monochrome image sensor in which each pixel is associated with a single color channel, such as a grayscale color channel, in which the value of each pixel represents only the amount of light. In other embodiments, the image sensor is a color image sensor or an image sensor comprising a color filter array on a pixel array. As an example, the color filter array can be a Bayer color filter arranged with four color filters (red (R), green (G), green (G), and blue (B)). The Bayer filter may also be referred to as an RGGB filter. When utilizing image sensor data, color pixels may be combined into 2×2 color pixels of monochrome pixels for 3D depth reconstruction. In this case, the resolution of the 3D depth reconstruction is only half the resolution of the image sensor in each direction. When obtaining a texture (color) image, it is preferred to utilize full native resolution (with color filtered pixels).

[0075] According to some embodiments, the projector optical axis and the camera optical axis or axes are non-parallel. As an example, the projector optical axis and the camera optical axis of at least one camera unit can define a camera-projector angle of approximately 5 to 15 degrees (preferably approximately 5 to 10 degrees). All camera units can be similarly angled relative to the projector unit so that each camera optical axis defines approximately the same angle with the projector optical axis. In some embodiments, the camera unit is defocused at the opening of the probe of the scanner and / or at the surface of the optical window in the probe. In a preferred embodiment of the scanner, the camera unit and the projector unit of a given scanning unit are focused at the same distance.

[0076] Each camera unit may include one or more focusing lenses for focusing light on the image sensor of a given camera unit. In some embodiments, each camera unit includes two or more lenses assembled in a camera lens stack. The purpose of the focusing lenses may be to define or ensure a predetermined focal length or working distance of the camera unit. The camera focusing lenses may further define the camera optical axis.

[0077] In some embodiments, the scanner comprises two or more camera units configured to acquire a set of images comprising at least one image from each camera unit, wherein each image comprises at least a portion of the projected pattern. In a preferred embodiment, the images within the set of images are acquired simultaneously. Furthermore, the number of images in the set of images may preferably correspond to the number of camera units, wherein each camera unit contributes one image to the set of images. This has the advantage of improving the light budget; therefore, the light source and the projector unit consume less power. Therefore, since it is generally difficult to remove heat from an intraoral scanner, it is desirable that the components generate less heat.

[0078] The 3D scanner system may include one or more processors configured to generate a 3D representation based on the set of images (e.g., by identifying pattern features in the set of images and determining points in 3D space based on triangulation). Some of the processors may be located on the scanner and / or some may be located on an external computer. The 3D representation may be generated continuously during a scanning session, and / or the 3D representation may be generated in real time. The 3D scanner system may also include a display for displaying the 3D representation. The rendering of the 3D representation and the display of the representation may also be performed in real time, or may be perceived in real time by the user.

[0079] 3D Representation

[0080] The 3D scanner system may include one or more processors operably connected to the intraoral scanner, wherein the processor is configured to generate a 3D representation of the scanned object. A 3D representation may be understood herein as a digital representation of the external geometry and shape of a three-dimensional object. A 3D representation may represent only a portion of the geometry and shape of an object. A 3D representation may include a collection of points and / or polygons that collectively define the surface of an object. As an example, a 3D representation may be selected from the following groups: a point cloud, a signed distance field, a triangulated point cloud, a point cloud collection optionally with additional information such as uncertainty estimates or (one or more) colors, a triangulated point cloud collection, a polygonal mesh, a volume representation such as a voxel model, a parameterized surface, a surface element model, or any other suitable three-dimensional representation model. In a 3D representation, each point may be defined by its three-dimensional coordinates (x, y, z) in a Cartesian coordinate system. These points may form a portion of a surface mesh (such as a polygonal mesh). In some embodiments, a 3D representation is a triangular mesh comprising a set of triangles connected by their common edges or vertices. A collection of points and / or surface meshes may be used to create a visual representation of an object that can be rendered on a computer screen.

[0081] 3D representations can be generated from multiple 3D frames. A 3D frame can be understood herein as a 3D representation generated from a single field of view of an intraoral scanner. In other words, a 3D frame itself can constitute a 3D representation of a portion of the surface of the scanned object, and therefore can fall within the example of a 3D representation given above. As an example, each 3D frame can constitute a point cloud with multiple points in three-dimensional space. During scanning, the scanner can acquire 3D frames at a given frame rate (such as 20 to 35 frames per second). 3D frames can be registered in a common reference system and stitched together to form a 3D representation that is larger than the 3D representation that can be captured in a single field of view of the scanner. Registration of 3D frames can include registering 3D frames to one or more previously captured 3D frames of an object. Registration of 3D frames can be understood as determining the position and / or orientation of a 3D frame relative to another object and / or a collection of objects (such as a collection of 3D frames). Registration can also include the actual positioning of a 3D frame relative to a collection of objects and / or a collection of 3D frames. A collection of 3D frames stitched together may constitute the digital 3D representation described above. For example, a 3D representation of a portion of a given jaw or dental arch may be composed of a large number of 3D frames, such as between 100 and 2500 frames. Other numbers of 3D frames may be envisioned without departing from the scope of the present disclosure.

[0082] Sleeve

[0083] The 3D scanner system may include a sleeve mounted on the scanner (such as mounted on the exterior of the elongated housing). Preferably, the projector unit and the sleeve are arranged such that during use of the scanner a pattern is projected through the sleeve, wherein at least a portion of the spatial pattern is reflected by the window and / or the sleeve. Thus, when mounted on the scanner, the sleeve may be configured to cover at least the aperture and / or window of the scanner.

[0084] In some embodiments, the sleeve is made of a silicone material or a polymer (such as a thin polymer material). As an example, the sleeve can be made of a material selected from the following group: polyurethane (PU), polyethylene (PE), polypropylene (PP), low-density polyethylene (LDPE), polyvinyl chloride (PVC), thermoplastic elastomer (TPE), polyethylene terephthalate (PET), fluoropolymer (e.g., PTFE), polyolefin-based material or silicone material. The selected material can be a material suitable for medical devices. The sleeve can be made of a thin and flexible plastic material.

[0085] The sleeve may be flexible, i.e., it may be made of a flexible material. In some embodiments, the entire sleeve is flexible; in other embodiments, at least a portion of the sleeve is flexible. Such flexibility may ensure a tight fit around the housing of the intraoral scanner. In particular, it is preferred if the sleeve is configured so that the sleeve forms a tight and flat surface at least in an area close to the hole or window of the scanner. Since wrinkles in the sleeve typically result in many reflections, it is desirable to mount the sleeve across the hole or window in a straight and tight manner. In some embodiments, there is no gap between the sleeve and the window; however, a small gap may exist in some cases. In other embodiments, the sleeve is a rigid sleeve obtained, for example, from injection molding. The rigid sleeve may include a transparent area that is configured to align with the hole of the housing when the sleeve is mounted on the intraoral scanner. Similar to the flexible sleeve, the rigid sleeve may be configured to ensure a tight fit around the housing of the scanner. In some embodiments, the first portion of the sleeve is rigid and the second portion is flexible. As an example, the distal portion of the sleeve may be rigid or semi-rigid, with the flexible portion extending from the rigid portion.

[0086] The sleeve is preferably transparent at least in an area that is in the optical path of the projected spatial pattern, for example, the area substantially coincides with the area of ​​the aperture or window of the scanner. In some embodiments, the entire sleeve is transparent, at least to visible light (such as white light). Preferably, the sleeve is transparent at least to wavelengths within the visible, infrared and / or ultraviolet light.

[0087] The sleeve may be made of a thin material, such as a film. The sleeve may have a thickness from about 200 μm to about 800 μm, such as from about 300 μm to about 600 μm, such as from about 400 μm to about 500 μm. When the sleeve is mounted on a scanner, the sleeve may have a substantially uniform thickness everywhere, or it may have a thinner thickness in an area located in the optical path of the projected spatial pattern, where thinner is understood to be relative to other areas of the sleeve.

[0088] The sleeve is preferably closed at the distal end for covering the distal end of the elongated housing. Thus, the sleeve may comprise only one opening for inserting the intraoral scanner. The sleeve may be similar to an elongated bag having substantially the same shape as the housing of the scanner. To ensure hygiene when using the intraoral scanner, the sleeve preferably covers a large portion of the elongated housing when mounted on the intraoral scanner. In some embodiments, the sleeve covers at least 80% of the housing, preferably at least 90% of the housing, and even more preferably at least 95% of the housing when mounted on the scanner. In some cases, the proximal end of the scanner is not covered by the sleeve, since ventilation ducts for cooling the scanner are sometimes placed at this end.

[0089] Method Overview

[0090] The first step of the computer-implemented method disclosed herein may be receiving or acquiring one or more two-dimensional images acquired by one or more camera units forming part of an intraoral scanner. Each of the two-dimensional images may include at least a portion of a spatial pattern, the spatial pattern including a plurality of pattern features. The spatial pattern may be projected onto the scanned object by a projector unit forming part of the intraoral scanner. The images may also include one or more artifacts caused by reflections from a window and / or sleeve of the intraoral scanner.

[0091] The second step of the computer-implemented method disclosed herein can be to provide the (one or more) two-dimensional image as input to a parameterized model, the parameterized model being configured to determine the location of at least a subset of the pattern features in the (one or more) image. In a preferred embodiment, one or more parameters of the parameterized model are selected so that artifacts caused by reflections from the window and / or sleeve are suppressed or ignored by the parameterized model.

[0092] The third step of the computer-implemented method disclosed herein can be to generate a digital three-dimensional representation of the scanned object based on the positions of the determined pattern features. This can be achieved by triangulating the projector light and the camera light corresponding to the pattern features. Thus, a digital 3D representation can be generated based on the triangulation principle.

[0093] Any computer-implemented method disclosed herein may be performed in whole or in part by one or more processors forming part of a 3D scanner system. The processors may also be operably connected to an intraoral scanner forming part of the 3D scanner system. Some of the processors may be located inside the housing of the intraoral scanner, while other processors may be located on a computer forming part of the 3D scanner system.

[0094] Parametric Model

[0095] A parameterized model may be a machine learning model, such as a trained neural network. A parameterized model may refer to a model, such as a machine learning model, that includes a set of learnable parameters. These parameters may be values ​​that the model adjusts during training to capture patterns, relationships, and / or features in input data. The input data may be a captured two-dimensional image acquired by one or more camera units of an intraoral scanner.

[0096] The parameters of a parameterized model can be numerical values ​​that define the behavior of the model (such as output). In the context of neural networks, parameters can include weights and biases. Weights can be used to transform input data, and biases can be used to shift the transformed data. The values ​​of these parameters can be learned from the training process (i.e., based on the training data set). During training, the parameters of the model can be iteratively updated to minimize a specific loss function that measures the difference between the model's predicted values ​​and the true target values. An optimization algorithm based on, for example, gradient descent can be used during the training process to adjust the parameters in order to improve the performance of the model on the training data.

[0097] A parameterized model can be trained to approximate a complex function that maps input data (e.g., an image) to output predictions (e.g., the location of a pattern feature). In other words, the model can learn to extract meaningful features from (one or more) images and use them to make predictions about the content of (one or more) images. The predictions can include the locations of pattern features that form portions of the spatial pattern in the image. The training of a parameterized model can in some cases be understood as adjusting or tuning the aforementioned parameters of the model so that the output of the model is close to the true target value.

[0098] In some embodiments, the parameterized model is a neural network adapted or configured to solve or handle translation invariance problems. Generally speaking, translation invariance problems in the context of computer vision refer to scenarios in which the basic task or goal remains unchanged despite the application of spatial translations or shifts to objects or features within an image. Typically, in translation invariant problems, the exact location or position of objects in an image does not change the problem or its solution. Therefore, the parameterized model of the present disclosure can be trained or configured to recognize patterns and / or pattern features regardless of where they appear within an image. Therefore, in some embodiments, the parameterized model or neural network is translation invariant, so that the model or network produces the same response regardless of how its input is shifted.

[0099] Examples of suitable neural networks or architectures for handling translation invariance problems include: convolutional neural networks (CNNs), twin networks, triple networks, capsule networks (CapsNet), attention-based models, spatial transformer networks (STNs), region-based CNNs (R-CNNs), and graph neural networks (GNNs). Convolutional neural networks can utilize multiple convolutional layers, which enables the network to automatically learn and extract hierarchical features from input data. The neural network can be composed of learnable parameter layers, which include convolutional layers that extract features from the image and fully connected layers that make predictions based on these features. Some examples of suitable architectures for neural networks include: LeNet-5, AlexNet, VGG16, VGG19, GoogLeNet (Inception), ResNet, High Resolution Network (HRNet), DenseNet, MobileNet, and EfficientNet. Other suitable neural networks and architectures can be envisioned without departing from the scope of the present disclosure.

[0100] The architecture of the neural network can be configured to effectively process grid-like data (such as images). The architecture may include an input layer configured to receive input data, such as one or more images, each of which is represented as a pixel grid. The architecture may also include one or more convolutional layers (such as multiple convolutional layers). Each layer may be configured to apply a set of filters (kernels) to the input data to detect specific features (e.g., edges, textures, shapes) through convolution operations. In particular, it is desirable to detect pattern features (such as corners) in spatial patterns present in the image. The network architecture may also include one or more pooling layers, fully connected layers, and output layers. The pooling layer may be placed after the convolutional layer, and they may have the purpose of reducing spatial dimensions and controlling overfitting. The fully connected layer may include multiple neurons, each of which is connected to each neuron in the previous layer and the subsequent layer. These fully connected layers may combine the extracted features and predict using learned weights and biases. The last fully connected layer may be configured to output predictions related to a specific task (e.g., the position of pattern features in the spatial pattern in the image). The output layer may be configured to provide a final prediction based on the task (classification, regression).

[0101] Training Dataset

[0102] The parameterized model or machine learning model may have been trained to suppress or ignore the portion of the spatial pattern that is reflected by the window and / or sleeve. This reflected portion may also be referred to as an artifact in the image, since only the projected pattern on the scanned / imaged object is desired, not the direct reflection(s) from the window or sleeve. To achieve this, the parameterized model or machine learning model may have been trained based on a training data set.

[0103] The training data set may include reference real data, such as the real position of pattern features in the spatial pattern. The training data set may also include multiple two-dimensional training images, which include artifacts caused by reflections from windows and / or sleeves. The training data set may include a large number of said training images (such as at least 50 images, preferably at least 100 images, or even more preferably at least 250 images or more). The training images may have been acquired through sleeves and / or windows. Preferably, the images have been acquired so that there is no focused object in the training images except for the windows and / or sleeves. Advantageously, the training images include only the aforementioned artifacts and nothing else. Therefore, when acquiring the training images, the background should preferably be completely dark and / or so that there is no object in focus in the imaging scene. In some cases, the training images are obtained using the intraoral scanner disclosed herein. Therefore, in some embodiments, the training data set includes reference real data and the training images described above.

[0104] The training data set may also include two-dimensional rendered images of the digital 3D model of the tooth. The training data set may include a large number of the rendered images (such as at least 100 images, preferably at least 1000 images, and even more preferably at least 10000 images). The digital 3D model may be a 3D surface model. Preferably, the rendered image includes at least a portion of the spatial pattern. The digital 3D model may also include texture information of the three-dimensional scanned object (e.g., the color of the surface of the three-dimensional object). The rendered image may have been generated using a geometric virtual model of an intraoral scanner, wherein the geometric virtual model includes external parameters and / or internal parameters of the projector unit and camera unit of the scanner. The training data set may also include data related to the (one or more) postures of the scanner used to generate a given digital 3D model, wherein each posture includes the position and / or orientation of the scanner. As an alternative to the rendered image of the digital 3D model, the training data set may include, for example, real data obtained using the intraoral scanner disclosed herein. In the latter case, data may be obtained without a window or sleeve on the scanner. In some cases, the reference real data is generated based on the rendered image or real data.

[0105] In some embodiments, in addition to the reference real data, the training data set also includes two different sets of two-dimensional images, wherein the first set of images includes artifacts caused by reflections from windows and / or sleeves, and wherein the second set of images does not include the artifacts. The aforementioned training images can constitute the first set of images, and the rendered images of the digital 3D model of the tooth or the real data obtained without windows or sleeves can constitute the second set of images. During the training phase of the parameterized model, the input of the model can be two sets of images, i.e., both the first set including artifacts and the second set without artifacts. The input can also include reference real data, for example, the position of pattern features. Therefore, the training data can include paired images, wherein each pair of images includes a training image and a rendered image or a real image obtained without a sanitary barrier (such as a sleeve or window). In some embodiments, the number of training images is less than the number of rendered images. In such cases, the same training image can be used for multiple rendered images, so that the number of pairs exceeds, such as 10,000 or even 100,000, even if the number of training images is much less (such as less than 1,000 or less than 100).

[0106] In some embodiments, multiple images are provided as input to a parameterized model or neural network, where the images are acquired at different times. In such cases, the parameterized model or neural network can be trained to take into account the past, so that previously acquired images are used as input to improve the output of the neural network.

[0107] Detailed description of the drawings

[0108] Figure 1 A flowchart 100 of an embodiment of a computer-implemented method disclosed herein is shown. In step 102, one or more two-dimensional images are received, for example, by one or more processors, wherein the images have been acquired by one or more camera units forming part of an intraoral scanner. Preferably, each image includes at least a portion of a spatial pattern comprising a plurality of pattern features. The image(s) may also include one or more artifacts caused by reflections from a window and / or sleeve of the intraoral scanner. In step 104, the two-dimensional image(s) are provided as input to a parametric model. Parameters of the parametric model may be set or selected so that artifacts are suppressed or ignored. In step 106, the position of at least a subset of the pattern features in the image(s) is determined while suppressing or ignoring artifacts in the image(s).

[0109] Figure 2 An embodiment of a 3D scanner system 200 according to the present disclosure is shown. The 3D scanner system can be configured to generate a three-dimensional (3D) representation of an object 201 (such as a dental object). As an example, the object 201 can be at least a portion of an oral cavity (including any of the dentition, gums, retromolar triangle, hard palate, soft palate, and floor of mouth, etc.). In this embodiment, the 3D scanner system includes an intraoral scanner 202 for acquiring a set of images of the scanned object (e.g., in a person's oral cavity). The 3D scanner system also includes one or more processors for generating a three-dimensional (3D) representation of the scanned object based on the acquired images. In general, the 3D representation can represent only a portion of the surface of the object, such as the portion captured by the field of view of the intraoral scanner 202. Such 3D representations may also be referred to herein as sub-scans or 3D surfaces. The (one or more) processors may be part of the 3D scanner 202, or they may be external to the intraoral scanner, or a combination of both, i.e., so that some processing is performed on the 3D scanner and further processing is performed on the computer system 204. The intraoral scanner can be configured to continuously (e.g., in real time) acquire a set of images and generate one or more 3D surfaces and / or sub-scans based on the images. It can also be configured to continuously transmit the sub-scans to the computer system 204 in a wired or wireless manner. The sub-scans can be registered and stitched together to form a digital 3D model of the scanned object. The 3D model can be displayed on a display connected to the computer system, for example.

[0110] Figure 3A flowchart 300 of an embodiment of a computer-implemented method disclosed herein is shown. In step 302, one or more two-dimensional images are received, for example by one or more processors of a 3D scanner system, wherein each image includes at least a portion of a spatial pattern, the spatial pattern including a plurality of pattern features and one or more artifacts. In step 304, the (one or more) two-dimensional images are provided as input to a parameterized model, wherein one or more parameters of the parameterized model have been selected such that induced artifacts are suppressed or ignored. In step 306, the positions of at least a subset of the pattern features in the (one or more) images are determined while suppressing or ignoring the artifacts in the (one or more) images. In step 308, a digital 3D representation is generated based on the determined (one or more) positions of the (one or more) pattern features.

[0111] Figure 4 A computer system 400 in which an embodiment of the present disclosure or a part thereof can be implemented as a computer-readable code is shown. The computer system can cover multiple components capable of data processing, storage, communication and user interaction. The computer system described herein may include one or more processors 404, a communication interface 424, a hard disk drive 412, a removable storage drive 414, an interface 420, a main memory 408, a display interface 402 and a display 430. One or more processors 404 may be configured to perform one or more steps of the computer-implemented method disclosed herein. The communication interface 424 may be configured to allow the computer system to communicate with external devices and networks. It may support various communication protocols (such as Ethernet, Wi-Fi or Bluetooth), enable data exchange and facilitate connection with other devices. The hard disk drive 412 may be configured to provide non-volatile storage to the computer system. It may store software programs, operating system files, user data and other files in a medium (such as a magnetic medium). The hard disk drive 412 may ensure persistent storage, allowing data to be retained even when the system is powered off. Removable storage drive 414 (such as a CD / DVD drive or a USB port) can be configured to provide the computer system with the ability to read from and / or write to removable storage unit 418. This enables a user to access an external storage device (such as an optical disk or USB flash drive) and transfer data to or from the computer system. Interface 420 can be configured to connect various external devices (such as a keyboard, mouse, printer, scanner, or audio equipment) to the computer system.

[0112] Figure 5An intraoral scanner 500 according to an embodiment of the present disclosure is shown. The scanner 500 includes an elongated housing 502. The scanner system also includes a sleeve 504, which is suitable for mounting on the exterior of the elongated housing. In this embodiment, the sleeve 504 is a flexible transparent sleeve (such as a flexible polymer sleeve). The sleeve is suitable for ensuring a tight fit on the exterior of the housing. The figure shows the scanner and sleeve before (left) and after (right) the sleeve is installed. The housing includes a hole (not visible in the figure) on the side wall of the distal end of the housing, and when mounted on the scanner, the sleeve covers the hole.

[0113] Figure 6 An intraoral scanner 600 according to an embodiment of the present disclosure is shown. The scanner 600 includes an elongated housing 602. The scanner system also includes a sleeve 604 that is adapted to be mounted on the exterior of the elongated housing. In this embodiment, the sleeve 604 is a rigid sleeve such as manufactured by injection molding. The figure shows the scanner and sleeve before (left) and after (right) the sleeve is installed. The housing includes a hole (not visible in the figure) on the side wall of the distal end of the housing, and when mounted on the scanner, the sleeve covers the hole.

[0114] Fig. 7A A single sub-scan (3D representation) generated by a scanner is shown in the case where a window and a sleeve cover the aperture of the scanner's housing, and where the parameterized model is only trained to recognize the locations of pattern features, and is not trained to ignore or suppress artifacts generated by the window and / or sleeve. It is observed that the sub-scan is completely useless, since there is hardly any visible 3D surface in the 3D representation of the tooth. Furthermore, the 3D scanner is unable to register more 3D data to the "model" due to the small number of 3D points determined. Furthermore, the 3D points may not even relate to the scanned surface of the tooth, but are artifacts, i.e. erroneously determined 3D points belonging to the surface of the hygienic barrier (sleeve / window). This example serves to illustrate the complexity of scanning through a hygienic barrier, such as a transparent sleeve.

[0115] Figure 7B Several sub-scans are shown fused into a 3D model, which resembles a noisy plane or noisy displaced elements of a plane, which constitute a noisy scan of the window and / or sleeve surface itself. Therefore, this 3D model cannot be used for anything, as it does not represent the surface of the intended scan object (i.e., the patient's dentition). This example is used to illustrate the complexity of scanning through a hygienic barrier (such as a transparent sleeve).

[0116] Fig. 8A Shown is an original image of some teeth, where the image is captured through a window arranged in the bore of the housing of an intraoral scanner. Bright spots (white) are visible at the top of the image, which come from specular reflections of the scanner's window. Fig. 8A The image corresponds to a single field of view of the scanner. During the acquisition of this image, no sleeve covered the window.

[0117] Figure 8B Shows Fig. 8A A digital 3D model of the same tooth shown in FIG. 1 , wherein the 3D model is generated based on multiple captured images of the tooth. Thus, the model has been generated from multiple views, wherein Fig. 8A The image of illustrates one of the views. In this case, no sleeve covered the window during the image capture; therefore, the corresponding 3D model looks smooth and accurate, with high quality.

[0118] Fig.9A A raw image of some teeth is shown, where the image was captured through a window and a sleeve, where the sleeve covers the window, and where the window is located in the optical path of an intraoral scanner. A number of bright spots (white circles) are visible in the raw image, which are formed, for example, due to reflections of the projection light by the many folds in the sleeve. The bright spots complicate image processing; in particular, it makes it difficult to identify and determine pattern features in the spatial pattern present on the teeth. This therefore complicates the determination of 3D points, which negatively affects the accuracy of the generated 3D model. Similar to Fig.9A Multiple images of the image in are input to a trained parameterized model for determining pattern features in the image while suppressing or ignoring artifacts from the window and / or sleeve.

[0119] Fig. 9B A rendering of a digital 3D model is shown, where images have been captured both through a window and through a sleeve covering the window (similar to Fig.9A ), where the image has been input to a trained parametric model as described herein. It is observed that even though the 3D model appears to have greater surface roughness, the scanner system is still able to take advantage of the otherwise poor image input (e.g. Fig.9A shown).

[0120] Further details of the invention

[0121] 1. A 3D scanner system, comprising:

[0122] Intraoral scanner, which includes:

[0123] an elongated housing including a distal end for insertion into the oral cavity, wherein the housing includes an aperture in a sidewall of the distal end of the housing; and

[0124] a window disposed in the aperture of the housing; and / or

[0125] a sleeve mounted on an exterior of the elongated housing, wherein the sleeve covers the aperture;

[0126] One or more processors operably connected to the intraoral scanner, the processors configured to:

[0127] receiving one or more two-dimensional images including a plurality of pattern features and one or more artifacts caused by reflections from the window and / or sleeve; and

[0128] The two-dimensional image(s) are provided as input to a parameterized model that is trained to determine the locations of at least a subset of pattern features in the image(s) while suppressing or ignoring artifacts.

[0129] 2. A 3D scanner system according to item 1, wherein one or more parameters of the parametric model have been selected or adjusted so that artifacts are suppressed or ignored by the parametric model.

[0130] 3. A 3D scanner system, comprising:

[0131] Intraoral scanner, which includes:

[0132] an elongated housing including a distal end for insertion into the oral cavity, wherein the housing includes an aperture in a sidewall of the distal end of the housing;

[0133] a projector unit configured to project a predefined spatial pattern through the aperture onto at least a portion of the surface of the three-dimensional object, wherein the spatial pattern comprises a plurality of pattern features;

[0134] one or more camera units configured to acquire one or more two-dimensional images of the three-dimensional object, wherein at least a portion of the spatial pattern is present in the images; and

[0135] a window arranged in the aperture of the housing, wherein the projector unit and the window are arranged such that during use of the scanner the pattern is projected through the window; and / or

[0136] a sleeve mounted on the exterior of the elongated housing, wherein the projector unit and the sleeve are arranged such that during use of the scanner a pattern is projected through the sleeve, wherein at least a portion of the spatial pattern is reflected by the window and / or the sleeve;

[0137] One or more processors operably connected to the intraoral scanner, the processors configured to:

[0138] receiving one or more two-dimensional images acquired by a camera unit; and

[0139] The (one or more) two-dimensional image is provided as input to a parameterized model that is configured or trained to determine the location of at least a subset of pattern features in the (one or more) images, wherein one or more parameters of the parameterized model are selected such that a portion of the spatial pattern that has been reflected by the window and / or sleeve is suppressed or ignored.

[0140] 4. A 3D scanner system according to any of the preceding items, wherein the pattern features in the spatial pattern are arranged into a two-dimensional grid.

[0141] 5. A 3D scanner system according to any of the preceding items, wherein the parameterized model is a machine learning model.

[0142] 6. A 3D scanner system according to any of the preceding items, wherein the parameterized model is a neural network.

[0143] 7. A 3D scanner system according to any of the preceding items, wherein the parameterized model has been trained based on a training dataset.

[0144] 8. A 3D scanner system according to item 7, wherein the training data set comprises two different sets of two-dimensional images, wherein the first set of images includes artifacts caused by reflections from the window and / or sleeve, and the second set of images does not include the artifacts.

[0145] 9. A 3D scanner system according to any one of items 7 to 8, wherein the training data set also includes reference fact data, such as the position of the pattern features.

[0146] 10. A 3D scanner system according to any one of items 7 to 9, wherein the training data set comprises reference real data and two-dimensional training images, the two-dimensional training images including artifacts caused by reflections from the window and / or sleeve.

[0147] 11. A 3D scanner system according to item 10, wherein the two-dimensional training image has been acquired through the window and / or sleeve.

[0148] 12. A 3D scanner system according to any one of items 10 to 11, wherein, apart from the window and / or the sleeve, there are no objects in focus in the two-dimensional training image.

[0149] 13. A 3D scanner system according to any one of items 9 to 12, wherein the reference real data includes the position of the pattern feature.

[0150] 14. A 3D scanner system according to any one of items 7 to 13, wherein the training data set also includes two-dimensional rendered images of the digital 3D model of the tooth.

[0151] 15. The 3D scanner system according to item 14, wherein the digital 3D model is a 3D surface model.

[0152] 16. A 3D scanner system according to any one of items 14 to 15, wherein the digital 3D model contains texture information of the three-dimensional object (e.g., information on the color of the surface of the three-dimensional object).

[0153] 17. A 3D scanner system according to any one of items 14 to 16, wherein the rendered image has been generated using a geometric virtual model of the intraoral scanner, wherein the geometric virtual model includes external parameters and / or internal parameters of the projector unit and camera unit of the scanner.

[0154] 18. A 3D scanner system according to any one of items 14 to 17, wherein the rendered image contains at least a portion of the spatial pattern.

[0155] 19. A 3D scanner system according to any one of items 7 to 18, wherein the training data set also includes data related to the pose of the scanner used to generate a given digital 3D model, wherein each pose includes the position and / or orientation of the scanner.

[0156] 20. A 3D scanner system according to any of the preceding items, wherein the parameterized model is a convolutional neural network (CNN).

[0157] 21. A 3D scanner system according to any of the preceding items, wherein the architecture of the parameterized model or neural network is configured to solve the translation invariance problem.

[0158] 22. A 3D scanner system according to any of the preceding items, wherein the parameterized model or neural network is translation invariant, so that the neural network produces the same response regardless of how its input is shifted.

[0159] 23. A 3D scanner system according to any of the preceding items, wherein the parameterized model or neural network is configured so that a given single pixel in the output is only affected by a given group of pixels in the input.

[0160] 24. A 3D scanner system according to item 23, wherein pixels in the input affect a given single pixel in the output by weighting it according to its proximity to the pixel in the output, such that nearby input pixels are given a higher weight than distant pixels.

[0161] 25. A 3D scanner system according to any of the preceding items, wherein the parameterized model is configured to suppress or ignore a portion of the spatial pattern that has been reflected by the window and / or sleeve.

[0162] 26. A 3D scanner system according to any of the preceding items, wherein the parameterized model is a neural network trained to suppress or ignore a portion of the spatial pattern that has been reflected by the window and / or sleeve.

[0163] 27. A 3D scanner system according to any of the preceding items, wherein a plurality of images are provided as input to a parametric model or a neural network, wherein the images are acquired at different times.

[0164] 28. A 3D scanner system according to any of the preceding items, wherein the parameterized model or neural network is trained to take into account the past so that previously acquired images are used as input to improve the output of the neural network.

[0165] 29. The 3D scanner system according to any one of the preceding items, wherein the sleeve is made of silicone material or polymer.

[0166] 30. The 3D scanner system according to any of the preceding items, wherein the sleeve is made of a thin plastic material.

[0167] 31. A 3D scanner system according to any one of the preceding items, wherein the sleeve is made of a material selected from the following group: polyurethane (PU), polyethylene (PE), polypropylene (PP), low-density polyethylene (LDPE), polyvinyl chloride (PVC), thermoplastic elastomer (TPE), polyethylene terephthalate (PET), fluoropolymer (e.g., PTFE), polyolefin-based material or silicone material.

[0168] 32. A 3D scanner system according to any of the preceding items, wherein the sleeve is flexible.

[0169] 33. A 3D scanner system according to any of the preceding items, wherein the sleeve is transparent.

[0170] 34. A 3D scanner system according to any of the preceding items, wherein the sleeve has a thickness from about 200 μm to about 800 μm (such as from about 300 μm to about 600 μm, such as from about 400 μm to about 500 μm).

[0171] 35. A 3D scanner system according to any of the preceding items, wherein the sleeve is closed at the distal end for covering the distal end of the elongated shell.

[0172] 36. A 3D scanner system according to any of the preceding items, wherein the sleeve comprises only one opening for inserting an intraoral scanner.

[0173] 37. A 3D scanner system according to any of the preceding items, wherein, when mounted on an intraoral scanner, the sleeve covers a majority of the elongated shell, such as at least 90% of the shell.

[0174] 38. A 3D scanner system according to any of the preceding items, wherein the sleeve covers the hole and / or the window such that during use of the scanner the spatial pattern is projected through the window and / or through the sleeve.

[0175] 39. A 3D scanner system according to any of the preceding items, wherein the sleeve is a rigid sleeve obtained by injection molding.

[0176] 40. A 3D scanner system according to item 39, wherein the rigid sleeve includes a transparent area configured to align with the hole of the shell when the sleeve is mounted on the intraoral scanner.

[0177] 41. A 3D scanner system according to any of the preceding items, wherein at least the distal end of the elongated shell is made of a rigid material.

[0178] 42. A 3D scanner system according to any of the preceding items, wherein the elongated shell includes a removable end.

[0179] 43. A 3D scanner system according to any of the preceding items, wherein at least one area of ​​the window is made of a transparent material, said area being located in the optical path of the projected spatial pattern.

[0180] 44. A 3D scanner system according to any of the preceding items, wherein the entire window is made of transparent material.

[0181] 45. A 3D scanner system according to any of the preceding items, wherein the window is made of a rigid polymer such as polymethyl methacrylate (PMMA).

[0182] 46. ​​A 3D scanner system according to any one of the preceding items, wherein the window is made of glass material.

[0183] 47. A 3D scanner system according to any of the preceding items, wherein the window is coated with an anti-reflective coating.

[0184] 48. A 3D scanner system according to any of the preceding items, wherein the window is rigid.

[0185] 49. A computer-implemented method comprising the steps of:

[0186] receiving one or more two-dimensional images acquired by one or more camera units forming part of an intraoral scanner, wherein each image comprises at least a portion of a spatial pattern comprising a plurality of pattern features, wherein the image(s) further comprises one or more artifacts caused by reflections from a window and / or sleeve of the intraoral scanner; and

[0187] The (one or more) two-dimensional image is provided as input to a parametric model, which is configured to determine the location of at least a subset of the pattern features in the (one or more) images, wherein one or more parameters of the parametric model are selected so that artifacts caused by reflections from the window and / or sleeve are suppressed or ignored.

[0188] 50. A computer-implemented method according to item 49, wherein the method further comprises the step of generating a digital 3D representation based on the determined position(s) of the pattern feature(s).

[0189] 51. A computer-implemented method according to any one of items 49 to 50, wherein the digital 3D representation is generated based on the principle of triangulation.

[0190] 52. A computer-implemented method according to any one of items 49 to 51, wherein the pattern features in the spatial pattern are arranged in a two-dimensional grid.

[0191] 53. A computer-implemented method according to any one of items 49 to 52, wherein the parameterized model is a machine learning model.

[0192] 54. A computer-implemented method according to any one of items 49 to 53, wherein the parameterized model is a neural network.

[0193] 55. A computer-implemented method according to any one of items 49 to 54, wherein the parameterized model has been trained based on a training data set.

[0194] 56. A computer-implemented method according to item 55, wherein the training data set comprises two different sets of two-dimensional images, wherein the first set of images includes artifacts caused by reflections from the window and / or sleeve, and the second set of images does not include said artifacts.

[0195] 57. A computer-implemented method according to any one of items 55 to 56, wherein the training data set also contains reference fact data (such as the location of pattern features).

[0196] 58. A computer-implemented method according to any one of items 55 to 57, wherein the training data set comprises reference real data and two-dimensional training images, the two-dimensional training images including artifacts caused by reflections from the window and / or sleeve.

[0197] 59. A computer-implemented method according to item 58, wherein the two-dimensional training image has been acquired through the window and / or sleeve.

[0198] 60. A computer-implemented method according to any of items 58 to 59, wherein, apart from the window and / or the sleeve, there are no objects in focus in the two-dimensional training images.

[0199] 61. A computer-implemented method according to any one of items 57 to 60, wherein the reference real data includes the positions of pattern features.

[0200] 62. A computer-implemented method according to any one of items 55 to 61, wherein the training data set also comprises images of two-dimensional renderings of the digital 3D models of teeth.

[0201] 63. A computer-implemented method according to item 62, wherein the digital 3D model is a 3D surface model.

[0202] 64. A computer-implemented method according to any one of items 62 to 63, wherein the digital 3D model contains texture information of the three-dimensional object (e.g., information on the color of the surface of the three-dimensional object).

[0203] 65. A computer-implemented method according to any one of items 62 to 64, wherein the rendered image has been generated using a geometric virtual model of an intraoral scanner, wherein the geometric virtual model includes external parameters and / or internal parameters of a projector unit and a camera unit of the scanner.

[0204] 66. A computer-implemented method according to any one of items 62 to 65, wherein the rendered image contains at least a portion of the spatial pattern.

[0205] 67. A computer-implemented method according to any one of items 55 to 66, wherein the training data set also includes data related to the poses of a scanner used to generate a given digital 3D model, wherein each pose includes the position and / or orientation of the scanner.

[0206] 68. A computer-implemented method according to any one of items 49 to 67, wherein the parameterized model is a convolutional neural network (CNN).

[0207] 69. A computer-implemented method according to any one of items 49 to 68, wherein the architecture of the parameterized model or neural network is configured to solve the translation invariance problem.

[0208] 70. A computer-implemented method according to any one of items 49 to 69, wherein the parameterized model or neural network is translation invariant so that the neural network produces the same response regardless of how its input is shifted.

[0209] 71. A computer-implemented method as described in any of items 49 to 70, wherein the parameterized model or neural network is configured so that a given single pixel in the output is only affected by a given group of pixels in the input.

[0210] 72. A computer-implemented method as described in item 71, wherein pixels in the input affect a given single pixel in the output by weighting it according to their proximity to the pixel in the output, such that nearby input pixels are given higher weights than distant pixels.

[0211] 73. A computer-implemented method according to any one of items 49 to 72, wherein the parameterized model is configured to suppress or ignore a portion of the spatial pattern that has been reflected by the window and / or sleeve.

[0212] 74. A computer-implemented method according to any one of items 49 to 73, wherein the parameterized model is a neural network trained to suppress or ignore a portion of the spatial pattern that has been reflected by the window and / or sleeve.

[0213] 75. A computer-implemented method according to any one of items 49 to 74, wherein multiple images are provided as input to a parametric model or neural network, wherein the images are acquired at different times.

[0214] 76. A computer-implemented method according to any one of items 49 to 75, wherein the parameterized model or neural network is trained to take into account the past so that previously acquired images are used as input to improve the output of the neural network.

[0215] 77. A 3D scanner system comprising:

[0216] a. Intraoral scanner, which includes:

[0217] an elongated housing including a distal end for insertion into the oral cavity, wherein the housing includes an aperture in a sidewall of the distal end of the housing;

[0218] a projector unit configured to project a predefined spatial pattern through the aperture onto at least a portion of the surface of the three-dimensional object, wherein the spatial pattern comprises a plurality of pattern features arranged in a two-dimensional grid;

[0219] one or more camera units configured to acquire one or more two-dimensional images of the three-dimensional object, wherein at least a portion of the spatial pattern is present in the images; and

[0220] a window arranged in the aperture of the housing, wherein the projector unit and the window are arranged such that during use of the scanner the pattern is projected through the window; and / or

[0221] a sleeve mounted on the exterior of the elongated housing, wherein the projector unit and the sleeve are arranged such that during use of the scanner a pattern is projected through the sleeve, wherein at least a portion of the spatial pattern is reflected by the window and / or the sleeve;

[0222] One or more processors operably connected to an intraoral scanner, the processors being configured to perform the steps of a computer-implemented method as described in any one of items 49 to 76.

[0223] 78. A data processing system comprising one or more processors configured to perform the steps of the method according to any one of items 49 to 76.

[0224] 79. A computer-readable storage medium comprising instructions which, when executed by a computer, cause the computer to perform the steps of the method according to any one of items 49 to 76.

[0225] 80. A computer program product comprising instructions which, when the program is executed by a computer, cause the computer to perform the steps of the method according to any one of items 49 to 76.

[0226] 81. A computer-readable data carrier having stored thereon a computer program product according to item 80.

[0227] Although some embodiments have been described and shown in detail, the present disclosure is not limited to such details, but may also be implemented in other ways within the scope of the subject matter defined in the appended claims. In particular, it should be understood that other embodiments may be utilized and structural and functional modifications may be made without departing from the scope of the present disclosure. In addition, those skilled in the art will find that it is apparent that unless an embodiment is specifically presented only as an alternative, different disclosed embodiments may be combined to achieve a specific implementation, and such specific implementation is within the scope of the present disclosure.

Claims

1. A 3D scanner system, comprising: -Intraoral scanner, comprising: - an elongated housing comprising a distal end for insertion into the oral cavity, wherein the housing comprises an aperture in a side wall of the distal end of the housing; and - a window arranged in a hole in said housing; and / or - a sleeve mounted on the exterior of the elongated housing, wherein the sleeve covers the aperture; - one or more processors operably connected to the intraoral scanner, the processors being configured to: - receiving one or more two-dimensional images comprising a plurality of pattern features, wherein the images include one or more artifacts caused by reflections from the window and / or sleeve; and - providing the one or more two-dimensional images as input to a parameterized model, the parameterized model being trained to determine the locations of at least a subset of the pattern features in the one or more images while suppressing or ignoring the artifacts.

2. The 3D scanner system according to claim 1, wherein: One or more parameters of the parameterized model have been selected or adjusted such that the artifact is suppressed or ignored by the parameterized model.

3. A 3D scanner system according to any one of the preceding claims, wherein: The parameterized model is a machine learning model, such as a neural network.

4. A 3D scanner system according to any one of the preceding claims, wherein: The parameterized model has been trained based on a training data set comprising two different sets of two-dimensional images, wherein a first set of images includes artifacts caused by reflections from the window and / or sleeve and a second set of images does not include said artifacts.

5. The 3D scanner system according to claim 4, wherein: The training data set also includes reference real data, and the reference real data includes the positions of the pattern features.

6. The 3D scanner system according to any one of claims 4 to 5, wherein: The training data set comprises the reference real data and a two-dimensional training image, wherein the two-dimensional training image comprises artifacts caused by reflections from the window and / or sleeve, wherein the training image has been acquired through the window and / or sleeve, wherein there is no object in focus except the window and / or the sleeve.

7. The 3D scanner system according to any one of claims 4 to 6, wherein: The training data set also contains two-dimensional rendered images of the digital 3D model of the teeth, wherein the rendered images have been generated using the geometric virtual model of the intraoral scanner.

8. A 3D scanner system according to any one of the preceding claims, wherein: The parameterized model or neural network is translation invariant, so that the neural network produces the same response no matter how its input is shifted.

9. A 3D scanner system according to any one of the preceding claims, wherein: The intraoral scanner further comprises a projector unit configured to project a predefined spatial pattern through the aperture onto at least a portion of a surface of the three-dimensional object, wherein the spatial pattern comprises a plurality of pattern features.

10. The 3D scanner system according to claim 9, wherein: The intraoral scanner further comprises one or more camera units configured to acquire one or more two-dimensional images of the three-dimensional object, wherein at least a portion of the spatial pattern is present in the images.

Images