Apparatus for implementing a method for analyzing a dental condition

Abstract

The invention relates to a device implementing a method for analyzing the actual dental condition of a patient, the method comprising a cycle of steps A) to C): A) at an update time, acquiring an updated image representative of the actual dental scene; B) determining a virtual dental scene from the actual dental scene; identifying an element of the virtual dental scene that does not meet an evaluation criterion, and highlighting this element in the virtual dental scene and / or adding a message relating to this element to the virtual dental scene; C) presenting the virtual dental scene on a screen and superimposed on the actual dental scene, or displaying the updated image on the screen and presenting the virtual dental scene superimposed on the representation of the actual dental scene; repeating step A) less than five seconds after step C), the device comprising: - a video camera implementing step A); - a processing unit implementing step B); - a screen implementing step C); the video camera, the processing unit, and the screen being in communication with each other to implement the steps.

Description

[0001] This application is a divisional application of the invention patent application filed on May 14, 2019, with application number "201980048831.7" and invention title "Method for Analyzing the Condition of Teeth". Technical Field

[0002] This invention relates to the analysis of a patient's dental condition. Background Technology

[0003] When a dental care professional intervenes in a patient’s teeth or makes a diagnosis, he or she needs to assess the actual dental condition, particularly the position and / or shape and / or size of the teeth, and / or the proper positioning of the orthodontic appliances attached to the teeth.

[0004] This assessment typically requires comparing the actual condition of the teeth with the theoretical condition. For example, to diagnose the detachment of an orthodontic appliance attachment, it may be necessary to compare the actual condition of the teeth (in which the attachment is abnormally separated from the tooth it should be bonded to) with the theoretical condition of the teeth (in which the attachment is actually bonded to the tooth). This need is particularly present in personalized treatment, where attachments are shaped to precisely fit the morphology of the teeth and / or according to a specific prescription from a dental care professional.

[0005] When attempting to assess whether teeth are abnormally positioned or exhibit abnormal shapes, dental professionals also compare the theoretical tooth condition with the actual tooth condition. Specifically, the actual position, shape, or size of the teeth must be compared with the theoretical position, shape, or size the teeth should present when the dental professional does so.

[0006] Furthermore, sometimes it is necessary to analyze the actual condition of the teeth very quickly, or even in real time. For example, when a dental care professional intervenes with tools to change the shape or appearance of teeth, his or her movements with those tools must be precise and adaptable to the changing actual condition of the teeth in a continuous manner.

[0007] Generally, the quality of the comparison between the actual and theoretical condition of teeth has an impact on the quality of treatment, regardless of whether the treatment is therapeutic.

[0008] Therefore, there is a continued need for an analytical method that allows for improved identification of actual dental conditions, especially by dental care professionals.

[0009] Furthermore, the effectiveness of dental treatment is directly related to patient adherence to medical prescriptions. Poor adherence can lead to a deterioration of dental conditions, posing additional risks to patients and incurring costs for social security or care funding organizations.

[0010] Poor treatment adherence also makes clinical research more difficult, especially if patients do not declare the time periods during which non-compliance occurred.

[0011] Therefore, a method is needed that allows for improved compliance.

[0012] One object of the present invention is to at least partially address these needs. Summary of the Invention

[0013] This invention proposes a method for analyzing a patient's actual dental condition, comprising the following sequential steps:

[0014] A) At the update time, preferably by means of a camera, an updated image representing the actual tooth scene is acquired;

[0015] B) Determine the virtual tooth scene based on the representation of the actual tooth scene on the updated image;

[0016] C) The virtual teeth scene is presented as follows:

[0017] - The virtual tooth scene is rendered in transparent mode and overlaid on the actual tooth scene, particularly on a screen through which the actual tooth scene can be seen, or the virtual tooth scene is projected onto the actual tooth scene, or

[0018] - A virtual tooth scene is overlaid on a representation of the actual tooth scene on an updated image displayed on the screen, the virtual tooth scene being overlaid on the representation in either a transparent or opaque mode.

[0019] Then, preferably, return to step A).

[0020] In a particular implementation, step B) includes simulating the actual tooth scene represented on the updated image for a simulation time, and then determining the virtual tooth scene based on the simulation.

[0021] As will be seen in more detail below, the method according to the invention thus allows the operator to observe simultaneously:

[0022] -Actual dental scenes, observed directly or via a screen, or observed on updated images, and

[0023] - A virtual tooth scene that overlaps with either the actual tooth scene or a representation of the actual tooth scene on an updated image.

[0024] This overlay is highly beneficial for operators (preferably in real-time) to analyze the differences between the actual and virtual dental scenes. It also allows for the addition of information related to the actual dental scene and the arrangement of that information based on the representation of the actual dental scene on the updated image. This improves the quality of information transmission to the operator. Finally, it allows for realistic simulations of dental conditions, such as those for aesthetic treatments.

[0025] When this method is implemented by a dental care professional, it allows him or her to improve the quality of his or her interventions with the patient. In particular, when the method is implemented by the patient, it can realistically simulate the effects of dental treatment or modifications to dental treatment, which improves patient compliance.

[0026] The method according to the invention may further include one or more of the following optional features:

[0027] - In step B), the virtual tooth scene includes:

[0028] - A representation of one or more physical elements of an actual dental scene modeled by a reference model, and / or

[0029] - A representation of one or more physical elements of a real tooth scene not modeled by a reference model, and / or

[0030] - Representation of one or more physical elements that are not in the actual tooth scene and / or not in the reference model, and / or

[0031] - Indicators that do not represent physical elements of an actual dental scene, preferably multiple such indicators;

[0032] - In step B), identify elements in the virtual tooth scene that do not meet the evaluation criteria, and highlight the element in the virtual tooth scene and / or add messages related to the element to the virtual tooth scene;

[0033] - The assessment criteria involve the patient's health status and / or the positioning of the orthodontic appliance worn by the patient and / or the condition of the orthodontic appliance worn by the patient and / or the cost of dental treatment;

[0034] - The message is an instruction that the patient and / or the dental care professional responsible for that patient must follow;

[0035] - In step B), the nature and / or position and / or appearance of the elements of the virtual teeth scene, and especially the indicators, are determined according to the following:

[0036] - Context, especially based on distance from another element in the virtual teeth scene and / or based on the desired outcome;

[0037] - In step B), the nature and / or position and / or appearance of the elements of the virtual tooth scene are determined based on the nature and / or position and / or appearance of the elements displayed in the prior virtual tooth scene (that is, the prior period from step A) to C), and / or based on the time interval with the prior virtual tooth scene, and / or based on the difference between the updated images of the consecutive periods from step A) to C).

[0038] - In step B), the virtual teeth scene is determined to include information, particularly textual or graphical information, regarding the following:

[0039] - Update the differences between the image and the reference image, and / or

[0040] - Differences between the reference model and the actual dental scene, and / or

[0041] - Target location, especially for the target location of tools, orthodontic appliances, or decorative items;

[0042] - In step C), the virtual tooth scene is presented on the actual tooth scene or the updated image such that the representation of the actual physical elements of the virtual tooth scene overlaps realistically with the representation of the actual physical elements on the updated image.

[0043] - Accordingly, repeat step A) within 5 seconds after step C);

[0044] - After step C), based on the representation of the virtual dental scene on the actual dental scene or on the representation of the dental scene on the updated image, modify the orthodontic appliance and / or adjust the dental arch and / or position the items or orthodontic appliance.

[0045] According to one aspect of the invention, in step C), a virtual tooth scene is presented on a transparent screen, overlapping with an actual tooth scene that the operator can observe through the screen (especially when facing the screen).

[0046] Preferably, the screen and camera are fixed relative to the operator. Preferably, the screen and camera are integrated into eyeglasses. The method according to the invention is then particularly useful for dental care professionals who wear eyeglasses, enabling them to assess the dental condition of the patient they are examining in real time.

[0047] In particular, according to this aspect of the invention, the method according to the invention can be used for real-time evaluation of peeling or milling operations, or for positioning orthodontic appliances or a portion thereof.

[0048] According to one aspect of the invention, in step C), a virtual tooth scene is presented on an opaque screen, that is, on which the operator cannot see, overlapping a representation of the actual tooth scene on the updated image.

[0049] In one implementation, the screen is the screen of a mobile phone, tablet, laptop, or virtual reality headset. The screen may also be the glass of a mirror, which is preferably equipped with at least one camera.

[0050] Preferably, the operator manipulates a mobile phone, laptop, virtual reality headset, or mirror equipped with at least one camera to obtain the updated images and view the actual and virtual dental scenes.

[0051] According to this implementation, dental care professionals can work on a patient's dental arch by viewing a screen. He or she then simultaneously sees images captured by a camera and information provided through a virtual dental scene on the screen. Specifically, he or she can work "blindly" on areas of the dental arch that are not directly observable but can be seen through the camera (i.e., without directly seeing or viewing the actual dental scene on a screen), especially when the camera is independent of the screen, and particularly when the camera can be introduced into the patient's mouth.

[0052] In one implementation, in step B) or d), a virtual dental scene is determined based on the value of at least one treatment parameter, which can be modified through interaction with the mobile phone, the laptop, the virtual reality headset, the camera, the mirror, or the glasses, preferably with the mobile phone. Preferably, the treatment parameter is related to the patient's wearing of orthodontic appliances and / or the patient's adherence to treatment instructions.

[0053] The method according to the invention, and especially this aspect of the invention, can facilitate patient decision-making, particularly because it allows him or her to see the impact of different possible treatment options on his or her appearance by selecting simulated moments. The method can also be an effective educational tool for improving compliance.

[0054] Use of reference models

[0055] In a preferred embodiment, a virtual dental scene is determined using a three-dimensional digital model that digitally models at least one dental arch of the patient; this model is referred to as a "reference model".

[0056] Preferably, the reference model in step C) is also used to locate the virtual tooth scene relative to the actual tooth scene or its representation.

[0057] Advantageously, the construction of virtual tooth scenes can therefore be fully automated.

[0058] In particular, the method according to the invention may include the following sequential steps:

[0059] a) At the reference time, preferably a reference model for digitally modeling at least one dental arch of the patient is generated using a scanner;

[0060] b) Optionally, modify the reference model;

[0061] c) At the update time, preferably, an updated image representing the actual tooth scene is acquired using a camera;

[0062] d) By observing the optionally modified reference model, search for the reference image that presents the maximum matching degree with the updated image, and then determine the virtual tooth scene based on the reference image;

[0063] e) Presenting a virtual teeth scene.

[0064] - The virtual tooth scene is rendered in transparent mode and overlaid on the actual tooth scene, particularly on a screen through which the actual tooth scene can be seen, or the virtual tooth scene is projected onto the actual tooth scene, or

[0065] - A virtual tooth scene is overlaid on a representation of the actual tooth scene on an updated image displayed on the screen, the virtual tooth scene being overlaid on the representation in either a transparent or opaque mode.

[0066] Then, preferably, return to step c).

[0067] Step c) is a specific case of step A). ​​Therefore, the optional features applicable to step A) also apply to step c).

[0068] Steps a), b), and d) together constitute the specific case of step B). Therefore, the optional features applicable to step B) also apply to steps a), b), and d).

[0069] Step e) is a specific case of step C). Therefore, the optional features applicable to step C) also apply to step e).

[0070] Preferably, in step a), the dental arch or physical model of the dental arch is scanned with or without an orthodontic appliance on the patient's dental arch or the physical model of the dental arch, and particularly when the dental arch or the physical model of the dental arch is not wearing an orthodontic appliance.

[0071] The update time can differ from the reference time, especially if it is more than three days after the reference time. The method described can then be used to examine changes in the dental arch between these two times.

[0072] In one implementation, the virtual teeth scene includes a reference image.

[0073] Preferably, in step d), the nature and / or position and / or appearance of the elements of the virtual teeth scene, and especially the indicators, are determined according to the following:

[0074] -Reference image, and / or

[0075] - Update the differences between the image and the reference image, and / or

[0076] - Context.

[0077] The reference model can provide access to "hidden" information, that is, information inaccessible or difficult for the operator to access. Specifically, this hidden information may depend on the reference image and therefore on the updated image. In one implementation, at least a portion of the hidden information is selected in step d). Thus, step d) enables the creation of a virtual tooth scene containing this information, and step e) allows the virtual tooth scene to be rendered over an actual tooth scene or a representation of an actual tooth scene.

[0078] application

[0079] In particular, the method according to the invention can be used for non-therapeutic purposes, especially for research purposes, such as to evaluate the efficiency of treatment or orthodontic appliances, or for aesthetic purposes, or for educational purposes.

[0080] In one implementation, the virtual tooth scene may relate to an actual tooth scene as simulated at the simulation time, or, in the past, to an actual tooth scene as observed at the simulation time. Specifically, the virtual tooth scene may simulate an actual tooth scene at the simulation time. The simulation is preferably obtained using a computer, and more preferably using artificial intelligence algorithms.

[0081] Therefore, the method according to the invention is particularly useful to operators, especially patients, to be able to view the condition of teeth at an earlier simulation time or preferably after an update time, or to view the evolution of such tooth condition by changing the simulation time.

[0082] Preferably, the method includes a series of cycles of steps A) to C) or steps c) to e), in which a virtual tooth scene is determined at each cycle to simulate the tooth condition at the current simulation moment.

[0083] In one implementation, a virtual tooth scene is determined using a dynamic simulation tool configured to provide at least one element of the virtual tooth scene based on a determined simulation time, particularly based on a simulation time after an update time.

[0084] When the method according to the invention is used for simulation purposes, the method may include one or more of the following optional features:

[0085] - The simulation time is after the update time, for example, more than 1 day, more than 10 days, or more than 100 days after the update time;

[0086] - The simulation time is determined before step A) or before step B), preferably through operator interaction with the screen;

[0087] -The screen:

[0088] - is touching the screen, and the time interval between the updated time and the simulated time is modified through interaction, preferably by dragging a finger on the screen, and / or

[0089] - Includes a region for inputting simulation times;

[0090] - Between the two cycles in steps A) to C), modify the time interval between the update time and the simulation time;

[0091] - Set the simulation time by modifying the position of the cursor displayed on the screen;

[0092] - In step B), the nature and / or position and / or appearance of the elements of the virtual teeth scene, and especially the indicators, are determined according to the following:

[0093] - Context, and in particular based on the distance to another element of the virtual teeth scene and / or based on the goal pursued and / or based on the simulation time;

[0094] - In step B), the color and / or shape and / or position of the visible or invisible portions of the patient's mouth, preferably the jaw, crowns and / or roots and / or gums are determined for the simulation time, and the virtual tooth scene is determined such that the virtual tooth scene reproduces the color and / or shape and / or position;

[0095] - The patient acquires the updated image in step A) and observes the screen in step C).

[0096] In particular, this method according to the invention can be used for simulation:

[0097] - The effect of one or more orthodontic appliances on a patient's teeth, especially in order to select the orthodontic appliance that is best suited to him or her;

[0098] - Temporarily or permanently stop the effects of the current treatment;

[0099] - The effect of the application instructions.

[0100] In particular, the method can be used especially for educational purposes to observe the effects of changing the frequency and / or duration and / or technique of brushing, or the effects of delaying the replacement of orthodontic splints and / or delaying appointments with dental care professionals.

[0101] For example, the method can be used to simulate the color and / or position of a patient's teeth at a simulated time. In particular, the method can be used to observe the effects of treatments for teeth whitening or to observe the effects of smoking.

[0102] When the method includes steps a) to e), in step b), the reference model may be modified to simulate the effect of the time flow between the reference time and the simulation time, particularly the effect of dental treatment (therapeutic or non-therapeutic) between these times.

[0103] The method according to the invention can even be specifically used for:

[0104] -Especially in the context of orthodontic treatment, assess the differences in position and / or shape and / or size of the orthodontic appliance and / or one or more teeth between the reference and update times, or between the simulation and update times, and / or

[0105] - Indicates the target location of the dental arch, specifically the target location of one or more teeth.

[0106] In particular, the method according to the invention can be used to examine the detachment of teeth or the deformation of gingiva between a reference time and an update time or between a simulation time and an update time, or to guide the placement of orthodontic appliances or parts thereof, or dental implants or decorative items.

[0107] In one implementation, the virtual dental scene relates to an actual dental scene. Preferably, the virtual dental scene at least partially represents the actual dental scene that appears at the update time or should appear at the update time according to the simulation. In other words, if the virtual dental scene represents a patient's teeth, it represents the position of the teeth at the update time or their simulated position at the update time. Therefore, the simulation time is the same as or close to the update time, for example, the interval between the simulation time and the update time is less than 1 month, less than 2 weeks, or less than 1 week.

[0108] equipment

[0109] The present invention also relates to apparatus for implementing the method according to the invention.

[0110] This device includes:

[0111] - Optionally, the means for generating and optionally modifying the reference model in steps a) and b) preferably includes a scanner and a computer;

[0112] - A camera, configured to perform step A) or c);

[0113] - A processing unit configured to perform step B) or step d);

[0114] - A presentation device and, preferably, a screen as described above, the presentation device being configured to perform step C) or e);

[0115] The processing unit, camera, and presentation device are configured to communicate with each other to implement the steps described.

[0116] Preferably, the camera is configured to acquire a series of updated images of the actual dental scene and send the updated images to the processing unit.

[0117] Preferably,

[0118] - The processing unit is configured as follows:

[0119] - By observing the reference model, for each updated image received from the camera, a reference image that exhibits the highest degree of matching with the updated image is searched.

[0120] - Determine the virtual teeth scene based on the reference image, and

[0121] - Send the virtual teeth scene to the presentation device;

[0122] - The presentation device is configured to display a virtual tooth scene overlaid on a representation of the actual tooth scene or overlaid on an updated image.

[0123] In one implementation, the virtual tooth scene is presented in a transparent mode over the actual tooth scene or over the representation of the actual tooth scene in an updated image.

[0124] In one embodiment, the device includes glasses that incorporate a camera and a presentation device, preferably incorporating a camera, a presentation device, and a processing unit.

[0125] In one embodiment, the presentation device is configured to present a virtual tooth scene on the lens of glasses, the virtual tooth scene realistically overlapping with an actual tooth scene, or to project the virtual tooth scene onto physical elements constituting the actual scene.

[0126] In one embodiment, the device includes a mobile phone, a camera, a laptop computer, a virtual reality headset, a tablet computer, or a mirror, wherein the device incorporates a camera and a presentation device, preferably incorporating a camera, a presentation device, and a processing unit.

[0127] Preferably, the presentation device includes a screen that allows for the display of updated images and the presentation of a virtual teeth scene on top of a representation of an actual teeth scene in an "augmented reality" manner.

[0128] A representation of an actual dental scene can be an image displayed on the screen of a mobile phone, camera, laptop, virtual reality headset, or tablet.

[0129] A real-world dental scene can be represented by a reflection from a mirror.

[0130] As will be seen in more detail in the description below, the observer can thus observe the actual dental scene or a representation of the actual dental scene on an updated image, and preferably a virtual dental scene on the actual dental scene or a representation of the actual dental scene on an updated image, preferably in a transparent mode. Therefore, he or she can immediately access the information contained in the virtual dental scene presented in the environment of the actual dental scene. For this purpose, integrating the screen into the glasses is particularly advantageous.

[0131] definition

[0132] "Patient" is the person to whom the method according to the invention is applied, whether or not that person is receiving dental treatment.

[0133] "Dental condition" defines a set of characteristics related to a patient's dental arch at a given moment, such as the position of the teeth, the shape of the teeth, and the position of orthodontic appliances at that moment.

[0134] "Dental care professionals" is understood to refer to anyone qualified to provide dental services, including orthodontists and dentists. Operators who wear glasses are preferably orthodontists or dentists.

[0135] "Operator" is understood to refer to the person who implements the method used to present the virtual dental scene to him or her in step C). In particular, the operator can be a patient or a dental care professional.

[0136] "Glasses" is understood to mean a device that can be worn in front of the operator's eyes, preferably placed on the operator's nose and / or ears.

[0137] An orthodontic appliance is an appliance worn or intended to be worn by a patient. Orthodontic appliances can be used for therapeutic or preventative treatment, as well as for aesthetic purposes. Specifically, an orthodontic appliance can be an apparatus with arch-shaped components and attachments, or an orthodontic splint. Such a splint extends along the continuous teeth of the dental arch to which it is fixed. It defines a generally "U"-shaped passage. In particular, the construction of the orthodontic appliance can be determined to ensure its fixation to the teeth, and also according to the desired target positioning of the teeth. More specifically, the form is determined such that, in the service position, the orthodontic appliance applies strain tending to move the treated tooth to its target position (active orthodontic appliance), or applies strain tending to hold the tooth in that target position (passive orthodontic appliance or "restrained" orthodontic appliance).

[0138] "Model" is understood to refer to a digital three-dimensional model.

[0139] "Image" is understood to refer to a two-dimensional image. Images are formed by pixels. "Updated images" are images captured at the so-called "update" moment. They are preferably extracted from images captured by a camera. "Reference images" are views of a "reference" model and exhibit the greatest degree of matching with the updated images.

[0140] "Dental arch" is understood to refer to all or part of the dental arch. Therefore, "image of dental arch" or "model of dental arch" is correspondingly understood to refer to a two-dimensional or three-dimensional representation of all or part of the dental arch.

[0141] A "scene" consists of a set of elements that can be observed simultaneously. A "teeth scene" is a scene that includes the dental arch.

[0142] The “actual scenario” consists of physical elements. Therefore, an actual dental scenario includes the dental arch, but optionally also orthodontic appliances worn on or manipulated by the teeth of the dental arch, and / or tools manipulated by the operator.

[0143] A "virtual scene" consists of representations of elements, especially the following:

[0144] - The physical elements of the corresponding actual tooth scene (that is, the actual tooth scene represented on the updated image from which the virtual tooth scene is determined), or

[0145] - Another information item or "indicator".

[0146] Physical elements can be modeled using a reference model (e.g., a tooth profile) or without a reference model (e.g., using a tool manipulated by an operator).

[0147] The representation of physical elements is not necessarily realistic. For example, tools can be represented by straight lines.

[0148] When the presentation of the virtual tooth scene allows the operator to see the actual tooth scene through the virtual tooth scene, the virtual tooth scene is presented on the actual tooth scene in a "transparent mode".

[0149] When the presentation of the virtual teeth scene allows the operator to see an updated image through the virtual teeth scene, the virtual teeth scene is presented "in a transparent mode" on top of the updated image representing the actual teeth scene.

[0150] When a virtual tooth scene includes a representation of physical elements of an actual tooth scene (this representation is rendered such that the possibly transparent outlines of the representation overlap the outlines of the physical elements or the outlines of the representation of the physical elements in an updated image), the virtual tooth scene is displayed in an "overlapped" or "registered" manner with the representation of the actual tooth scene in the actual tooth scene or the updated image. Therefore, this overlap appears realistic. For example, when an observer observes teeth in a dental arch, it is determined that the representation of that tooth in the virtual tooth scene overlaps with the view of the tooth perceived by the operator.

[0151] When the virtual tooth scene is presented in step C), the “position” of an element in the virtual tooth scene refers to its position relative to the elements of the actual tooth scene or the representation of those elements on the updated image.

[0152] The "fit" (or "closeness") between two objects is a measure of the difference or "distance" between the two objects.

[0153] Preferably, the two images or "views" that exhibit the highest degree of matching present the same elements in essentially the same way. In other words, the representations of elements on the two images can substantially overlap.

[0154] Unless otherwise stated, “including”, “contains”, “presents” or “represents” shall be construed as non-restrictive. Attached Figure Description

[0155] Other features and advantages of the invention will become more apparent from the following detailed description and study of the accompanying drawings, wherein:

[0156] - Figure 1 The different steps of the analysis method according to a preferred embodiment of the present invention are illustrated schematically.

[0157] - Figure 2 An example of a reference image representing a reference model;

[0158] - Figures 3a and 3b, and Figures 4a and 4b, respectively illustrate the processing of the reference image and the updated image to extract the reference image and the updated image representing the contour information, respectively;

[0159] - Figure 5 , Figure 6 and Figure 7 An example of a device according to the invention is shown schematically;

[0160] - Figure 8 This indicates the overlap of the virtual tooth scene on the updated image.

[0161] The same reference numerals are used to indicate the same or similar components in different figures. Detailed Implementation

[0162] equipment

[0163] The device 10 according to the invention includes a camera 16, a computer processing unit 12, and a presentation device 18. The camera 16, the processing unit 12, and the presentation device are provided with means that allow them to communicate with each other.

[0164] like Figure 5 , Figure 6 and Figure 7 As shown, camera 16 and / or processing unit 12 and / or presentation device may be integrated into glasses (which typically include a frame 14 configured to be worn by the operator’s nose and ears), mobile phone, tablet, laptop, virtual reality headset, camera, or mirror 17.

[0165] Camera 16 is designed to capture representations made by the operator through glasses. Figure 5 ), or on the phone ( Figure 6 ), laptops, virtual reality headsets, cameras, or tablet screens, or on the glass of a mirror 19 ( Figure 7 An updated image of the actual dental scene observed on a camera. It can be a conventional camera.

[0166] In one embodiment, the device includes means for projecting a structured light pattern, which is, for example, a group of points, such as laser light, or particularly infrared light. Advantageously, the actual dental scene reveals these patterns in a deformed manner, which allows depth-related information to be inferred from them.

[0167] In the case where the device includes a mirror, the glass 19 can be unplated glass, and the camera can be positioned behind the glass, such as... Figure 7 As shown in the image.

[0168] In one implementation, the device includes multiple cameras, which allows the distance between the observed object and the cameras to be estimated through simple triangulation. Therefore, the variety of cameras enables faster searching for reference images. The cameras also allow for the simulation of a three-dimensional view of the actual dental scene by an observer.

[0169] "Presentation device" is understood to mean any device configured to receive a virtual tooth scene from the processing unit 12 and display the virtual tooth scene on a screen (through which an observer can see the actual tooth scene) or on an updated image, on the actual tooth scene.

[0170] In one embodiment, the presentation device 18 includes a screen and a projector for projecting a virtual teeth scene onto the screen.

[0171] In particular, in the embodiment where the camera is integrated into the glasses, the screen can be transparent. Preferably, the screen is fixed to the frame 14, and a virtual tooth scene is displayed on the screen, so that the operator can see not only the actual tooth scene through the screen, but also the virtual tooth scene projected onto the screen.

[0172] Eyeglasses typically consist of two lenses extending in front of each of the operator's eyes, but there is no limit to the number of lenses. The lenses may or may not correct vision. Multiple lenses may extend in front of the same eye. For example, the first lens may correct the operator's field of vision, and the second lens may serve as a screen.

[0173] Depend on The HoloLens device developed by the company is an example of a presentation device.

[0174] The screen can be opaque, and updated images and virtual tooth scenes are registered on the screen so that the operator can see not only updated images representing the actual tooth scene, but also the virtual tooth scene by observing the screen.

[0175] Specifically, in this embodiment, for example when the camera and screen are combined in a mobile phone ( Figure 6 When in a camera, whether in a laptop, virtual reality headset, camera, tablet, or mirror, the camera can be fixed relative to the screen.

[0176] Alternatively, the camera can be free (or "independent") relative to the screen. Advantageously, the implementation where the camera is independent of the screen allows for great flexibility. For example, in one implementation, the camera can be introduced into the mouth while the screen remains outside the mouth.

[0177] In particular, in embodiments where the camera is integrated into or fixed to a mirror, the screen can be a reflective glass 19 of the mirror, and the virtual tooth scene can be presented on the screen in registration with the reflection R returned through the glass 19, so that the operator can see not only the reflection R representing the actual tooth scene, but also the virtual tooth scene superimposed on the actual tooth scene by observing the screen.

[0178] Therefore, reflections (like an updated image displayed on the screen of glasses, a mobile phone, a laptop, a virtual reality headset, a camera, or a tablet) give the operator a representation of the actual dental scene.

[0179] The processing unit 12 may include any conventional electronic device of a computer, particularly a central processing unit 22, a program, and a memory 24. Preferably, the memory 24 contains a reference model of at least a portion of an actual dental scene, and this model is particularly a model of a dental arch, such as... Figure 2 As shown in the diagram. The program includes code instructions that, when executed, can perform step d), specifically for analyzing the updated image to search for a view (i.e., a reference image) that presents the reference model with the greatest degree of matching to the updated image, and then accordingly determine the virtual tooth scene.

[0180] In one embodiment, the device further includes an interface that allows an operator to parameterize the virtual tooth scene. Preferably, the interface is configured to enable the operator to modify the simulation time at which the virtual tooth scene is determined. In particular, the content of the virtual tooth scene may include a representation of elements of an actual tooth scene simulated for past or future simulation times, or at a known appearance and / or location at past simulation times.

[0181] For example, in one implementation, the screen is a touchscreen, and the operator can modify the simulated time by entering a date in an input area or by moving one or more fingers on the screen. For instance, moving to the right of the screen can move forward to the future, while moving to the left of the screen can move back to the past. Accordingly, the presented virtual teeth scene is preferably adjusted in real time.

[0182] Interfaces can also be mechanical components, such as buttons or wheel.

[0183] In one implementation, the device is configured to continuously change from one cycle to another (i.e., for a continuous cycle of steps A) to C) or steps c) to e) for the time interval between the update moment and the simulation moment, thereby simulating the forward or backward movement of time. In one implementation, the time difference between two consecutive simulation moments is constant, which allows observation of changes in the flow of time at a constant rate.

[0184] The processing unit determines the conditions under which the virtual tooth scene must be presented to the operator in a way that appears to overlap with the actual tooth scene.

[0185] When a virtual teeth scene must be rendered on an updated image displayed on a screen, the operator observes the screen and thus sees what the camera is observing. Simple operations, such as using feature markers, zooming, perspective correction, or realigning the updated image, are sufficient to ensure a realistic overlap of the virtual teeth scene on the updated image, as described below.

[0186] When a virtual dental scene must be presented on a transparent screen through which an operator observes an actual dental scene, it is preferable that the cameras are arranged to capture updated images corresponding to the view of the actual dental scene seen by the observer. It is also preferable that the screen is at a constant distance from the observer and oriented in a fixed manner. Advantageously, the processing unit can thus easily determine from the updated images acquired by the cameras the view of the actual dental scene observed by the operator and the conditions under which the virtual dental scene must appear to overlap with the actual dental scene when presented to the operator.

[0187] method

[0188] The method according to the invention can be implemented using the device according to the invention.

[0189] In step a), at the reference time, a reference model is preferably generated using a scanner.

[0190] The reference model can be a model of the patient's dental arch, or it can consist of a model of the dental arch and a model of an orthodontic appliance placed on the model of the dental arch.

[0191] Preferably, the reference model is created from the measurements taken.

[0192] Preferably, the patient's dental arch is scanned using a 3D scanner to create a reference model while the patient is wearing an orthodontic appliance or a molded dental arch. Preferably, this scan is performed using a conventional 3D scanner.

[0193] The reference model obtained with the aid of a scanner represents the dental arch, but, if necessary, also represents any object scanned simultaneously with the dental arch, such as orthodontic appliances or decorative items attached to the dental arch.

[0194] Alternatively, the reference model may be a generic dental arch model selected from a library of generic models; that is, a dental arch model that is generally applicable to multiple patients or a variation of such a generic model. In particular, the variation may be based on measurements or observations of the patient, such as photographs of his or her dental arch. To enable the variation of the model to correspond to one or more photographs, metaheuristic methods may be implemented, particularly as described in WO 2016 / 066651.

[0195] Specifically, the universal dental arch model can be a model obtained through statistical processing, such as the average of more than 5 historical models, more than 10 historical models, more than 100 historical models, or more than 1000 historical models generated by a scanner. Preferably, the historical model is a model of the dental arch of a "historical" patient who has the same or similar dental features as the patient for whom the reference model was generated, for example, they had the same condition and / or were the same age and / or were the same sex and / or had similar morphology.

[0196] A 3D reference model can be viewed from any angle. The model viewed from a defined angle and distance is called a "view of the model". Figure 2 Figure 3a is an example of a view of the reference model.

[0197] The view of the reference model is designed to be compared with the updated image in order to determine the reference image most similar to the updated image for each updated image, and then to construct a virtual tooth scene with information added about the elements represented on the updated image.

[0198] The reference time can be close to the update time, for example, less than one month, less than two weeks, less than one week, or less than three days after the update time (either after or before the update time). The updated image can then be used to deform the general model until a reference model corresponding to the tooth condition at the update time is obtained. Furthermore, the virtual tooth scene then provides information about the tooth condition specific to the update time. For example, if the reference model models the root of the tooth, the virtual tooth scene can represent the root of the tooth.

[0199] The reference time can be separated from the update time by, for example, more than one month, more than two weeks, more than one week, or more than three days. The reference model may then differ significantly from the actual situation at the update time. The information provided by the virtual dental scene can then be related to changes in the dental condition between the reference and update times. For example, they can provide information about the movement of teeth or orthodontic appliances.

[0200] For example, the method can be used to check whether a passive restraint appliance correctly holds the teeth in the proper position after dental treatment. A reference model can be generated immediately after appliance placement, and updated images can be acquired up to one month after placement. Therefore, the reference images and virtual tooth scenes will focus on the position of the teeth at the end of treatment and the representation of the virtual tooth scene on top of the actual tooth scene at the updated time, thereby allowing dental professionals to detect discrepancies with their actual positions.

[0201] In step b), in one implementation, the reference model is modified.

[0202] The modification can be performed at any time between the reference time and the update time. Preferably, the modification is performed substantially at the reference time.

[0203] Modifications to the reference model can be variations of that model.

[0204] In one implementation, as described in WO 2016 / 066651, the reference model is segmented, and a tooth model is specifically created. The tooth model is then moved or deformed.

[0205] The modifications in step b) can simulate the effects of the time flow between the reference time and the update time, such as the effects of dental treatment. Therefore, the modification of the reference model produces a model representing the estimated position or shape of the tooth at the update time.

[0206] As described in WO 2016 / 066651, the movement or deformation of the dental model can also be determined or refined based on one or more updated images. Preferably, the initial reference model determined in step a) is then obtained by means of measurements performed by scanning at least one dental arch or at least one molded part of the dental arch.

[0207] Therefore, the method can be used to check whether the progress of dental treatment is as expected, with the initial reference model modified to correspond to the expected situation at the update time. In orthodontics, substantial changes may take several days. Preferably, in this embodiment, the time interval between the update time and the reference time is greater than 3 days, or even greater than 1 week, greater than 2 weeks, greater than 4 weeks, greater than 8 weeks, greater than 12 weeks, or even greater than 16 weeks.

[0208] Reference images obtained from observation of the modified reference model will then reveal the expected position or shape of the teeth at the time of update. These reference images constitute, or may be incorporated into, a virtual tooth scene, the presentation of which in step e) allows dental professionals to detect differences between the actual position (or shape) of the teeth and their expected position (or shape) at the time of update.

[0209] Modifications to the initial reference model may include adding another model and / or modifying that other model. For example, a model of an orthodontic appliance mounted on the dental arch may be added to a reference model that initially might only represent the dental arch. Modifications to the reference model may then include changes to the orthodontic appliance to present it in the position or shape expected at the time of update. Its presentation in step e) then allows dental professionals to detect differences between the actual position (or shape) of the orthodontic appliance and the position (or shape) expected at the time of update, and in particular, to detect any separation of attachments.

[0210] In step c), at the update time, an updated image is acquired using a camera that presents the actual teeth scene. In the accompanying drawings, the updated image is represented by a solid line on screen 18.

[0211] The device according to the invention (in which the camera is fixed to the frame of the glasses or incorporated into the mirror) makes it possible to acquire updated images that represent the actual dental scene as observed by an operator through the lenses of these glasses or in the mirror with high precision.

[0212] The device according to the invention (in which the camera is integrated into a mobile phone, laptop, tablet, or camera) enables the acquisition of updated images that accurately represent the actual dental scene as observed by an operator on the screen of the mobile phone, laptop, tablet, or camera.

[0213] In particular, using a virtual reality headset, the camera can be securely fixed to the screen or independent of it. The independence of the camera and screen advantageously allows the operator to work without directly seeing the actual teeth scene; that is, by seeing the actual teeth scene only on the screen. Therefore, he or she can work "blindly."

[0214] Preferably, the updated image is in color, and more preferably in true color.

[0215] In step d), a reference image is searched that shows the greatest match with the updated image.

[0216] In step d), all known methods for generating the maximum matching degree can be envisioned.

[0217] In particular, the maximum degree of matching can be obtained by optimizing to minimize the distance (“best fit”).

[0218] Preferably, a reference image is sought by observing the reference model from different viewing angles and distances until a view of the reference model that substantially corresponds to the updated image (i.e., substantially superimposed on the updated image) is obtained. Thus, for each updated image, a reference image exhibiting the maximum matching degree to that updated image is obtained. Preferably, this search is performed using a metaheuristic method, preferably an evolutionary method, and preferably by simulated annealing.

[0219] Metaheuristic methods are known optimization methods. Preferably, the method selects from a set of the following:

[0220] - Evolutionary algorithms, preferably selected from the following: evolutionary strategies, genetic algorithms, differential evolution algorithms, distribution estimation algorithms, artificial immune systems, path recombination, hybrid complex evolution, simulated annealing, ant colony optimization, particle swarm optimization, tabu search, and GRASP method;

[0221] -Kangaroo Algorithm

[0222] - The Fletcher and Powell method,

[0223] -Sound effects method

[0224] - Random tunneling method

[0225] - Randomly restart the ramp-up.

[0226] - Cross-entropy method, and

[0227] -A hybrid approach to the above metaheuristic methods.

[0228] Exploring a reference model to search for a reference image may include one or more of the features of steps c), d), and e) of WO 2016 / 066651, as long as they are relevant to this exploration.

[0229] Preferably, the updated image is processed to generate an updated map that at least partially represents the recognition information, and the search includes the following steps:

[0230] i) Obtain a view of the reference model;

[0231] ii) Process the view to produce at least one reference diagram that at least partially represents the identification information;

[0232] iii) Compare the updated view and the reference view to determine the distance between them, and if the distance is greater than a threshold, modify the view and then return to step ii).

[0233] The updated image and the reference image represent the same identification information. Identification information is feature information that can typically be extracted from an image (“image features”) through computer processing. Preferably, the identification information is selected from the group consisting of contour information, color information, density information, distance information, brightness information, saturation information, information about glare, and combinations thereof.

[0234] Those skilled in the art know how to process images to reveal identification information and thus create corresponding graphs.

[0235] For example, Figure 4b is an updated view relating to the tooth contour obtained from the updated image of Figure 4a. Figure 3b is a reference view relating to the tooth contour obtained from the view of Figure 3a.

[0236] Preferably, the modification of the view is guided by heuristic rules, for example, by favoring modifications that, based on the analysis of previous distances, appear to be most conducive to reducing the distance.

[0237] Step iii) ultimately yields a reference image that can be substantially superimposed on the updated image.

[0238] The maximum match between two images can also be obtained by identifying the overlap of markers on the reference image and the updated image (e.g., the overlap of fixed markers such as the ends of teeth or the contact points between two teeth).

[0239] Preferably, the virtual tooth scene includes or is composed of reference images. The reference images are specific examples of the virtual tooth scene.

[0240] exist Figure 8 In the image, elements of the virtual teeth scene are represented by dashed lines.

[0241] In a preferred embodiment, the virtual dental scene includes a representation of one or more elements of a corresponding real dental scene, which is preferably a realistic representation. In particular, preferably, the virtual dental scene includes a representation of the dental arch, but optionally, for example, it may include a representation of an orthodontic appliance worn by the teeth of the dental arch or an orthodontic appliance manipulated by an operator in front of the dental arch, or a representation of a tool manipulated by an operator in front of the dental arch.

[0242] like Figure 8 As shown, the virtual tooth scene can, for example, represent the outline 26 of a tooth, and its projection onto the tooth can facilitate the detection of tooth movement or deformation.

[0243] The virtual dental scene may also include representations of one or more physical elements of the actual dental scene that are not modeled by a reference model (e.g., tool 28 manipulated by an operator). To identify such physical elements, updated images may be analyzed, for example, using artificial intelligence algorithms.

[0244] The virtual dental scene may also include representations of one or more physical elements not present in the actual dental scene, such as tools or orthodontic appliances or decorative items 32, which are positioned, for example, on the virtual dental scene so that they appear in the actual dental scene at the location where the tools or orthodontic appliances or decorative items should be placed by a dental care professional or by a patient (“target placement”).

[0245] The representation of physical elements can be realistic (e.g., consisting of the outline of the physical element under consideration) or symbolic. For example, a drill bit can be symbolized by an arrow.

[0246] The physical elements of a virtual dental scene can be represented in two dimensions and / or three dimensions. In particular, the virtual dental scene can be presented as a hologram.

[0247] In addition to or replacing representations of physical elements (such as representations of reference images), a virtual tooth scene may also include one or more indicators 34.

[0248] Indicators are elements in a virtual tooth scene that do not even symbolically represent physical elements of an actual tooth scene.

[0249] Specifically, the indicator can be text or a symbol (dot, line, arrow, color control swatch, etc.), such as the symbol shown below, which indicates the difference between the updated image and the reference image, or provides information related to the action performed by the operator, such as indicating the direction of the tool he or she is manipulating.

[0250] In one implementation, the indicator points to a location that the operator must pay special attention to, such as the brushing area. The indicator can be an image, preferably a playful one, such as a picture of a monster or microorganism, to which the operator must apply a tool (e.g., a toothbrush). For example, such an indicator improves the learning process of brushing teeth.

[0251] exist Figure 8 In the middle, indicator 34 points to the area of ​​the worn-out teeth.

[0252] The nature of the indicator can be determined based on the reference image. In particular, the indicator can be associated with a reference model or a region of the reference model. For example, the indicator is stored in a database in the memory 24 of the processing unit.

[0253] For example, additional information can be associated with different areas of the reference model, such as to indicate brittle or bonded areas of the orthodontic appliance's decay. Additional information can also be associated with the reference model as a whole, such as to specify its occurrence, information about the patient, or information about the orthodontic appliance that may be represented. Indicators make it possible to present such additional information.

[0254] The nature of the indicator can be determined based on the differences between the updated image and the reference image.

[0255] For example, indicators can be added to reveal tooth deformation or movement, or the detachment of orthodontic appliances.

[0256] The nature and / or location and / or appearance of the indicator can be determined based on the context, particularly based on its distance from another element in the virtual dental scene, and / or based on the desired outcome. For example, the appearance of the indicator will change depending on its distance from a specific tooth or on the patient's risk.

[0257] The context is defined by the conditions under which the method is used.

[0258] For example, the operator may have a specific goal: to check the position of the orthodontic appliance worn by the patient. Therefore, indicators unrelated to this check, such as indicators of attenuation areas, are unnecessary.

[0259] In one implementation, prior to step d), the operator enters additional information into a database based on the intended use of the method. For example, he or she specifies the location for attaching decorative items or for milling teeth. Advantageously, corresponding indicators can be added to the virtual teeth scene.

[0260] For example, an arrow can point to an area of ​​a tooth to indicate to the operator where an orthodontic appliance or decorative item must be fixed, or where he or she needs to work on it, for example, with a tool such as a burr.

[0261] Indicators can also realistically or unrealistically represent invisible parts of physical elements in an actual tooth scene (and for parts that are not visible to the operator, they are therefore not part of the actual tooth scene), such as the root of a tooth, bone, or dental nerve.

[0262] The elements constituting the virtual tooth scene are positioned within the scene based on the elements of the actual tooth scene, and therefore, in this particular case, based on a reference image. Specifically, the representations of the actual physical elements are preferably positioned in the virtual tooth scene such that, in step e), they precisely overlap with these actual physical elements or with the representations of the actual physical elements on the updated image.

[0263] Similarly, preferably, the indicator for adding specific information to a physical element, such as a tooth, is positioned such that in step e), the indicator appears to be adjacent to or overlapped with the physical element, for example, adjacent to or overlapped with the tooth.

[0264] Indicators can be arranged to display points where differences have been identified between the reference and updated images in the next step. For example, a red marker could be displayed where the orthodontic appliance has moved away from the tooth.

[0265] In one embodiment, the context changes and the nature and / or position of the indicator is adjusted accordingly in the virtual tooth scene. Specifically, the nature and / or position of the indicator can be determined based on the nature and / or position of the indicator displayed in a prior virtual tooth scene (i.e., the scene determined in previous cycles c)-e (or cycles A) to C)), and / or based on the time interval with said prior virtual tooth scene, and / or based on the differences between updated images in consecutive cycles c)-e (or cycles A) to C)).

[0266] For example, the method can be used to guide an operator's actions using a tool, such as drilling with a power drill. Elements of the virtual tooth scene can be arrows indicating the orientation of the drill and the position of its free end. The database can contain protocol settings, such as the point of entry into the tooth, the drill trajectory, and the optimal forward speed. Then, in each cycle of steps c) to e) (or steps A) to C)), the position of the element in the virtual tooth scene is determined based on the element's position in the previously determined virtual tooth scene, and preferably based on the time interval between the virtual tooth scene and the prior virtual tooth scene.

[0267] Preferably, tools, such as electric drills, appearing in continuously updated images are identified, and the positions of elements in the virtual tooth scene are adjusted to account for the actual position of the tools. Advantageously, the representation of indicators in the virtual tooth scene can therefore be changed according to the actual position of the tools. In particular, it can therefore adapt to the actual rate of progress or actual trajectory of the tools.

[0268] In one implementation, the appearance of elements in the virtual teeth scene changes depending on the context.

[0269] For example, elements in a virtual dental scene that do not meet evaluation criteria, such as exceeding a risk threshold, can be identified and then brought to the attention of that element within the virtual dental scene. For instance, if analysis of the situation indicates that things are becoming dangerous (e.g., because the tool is not following the intended trajectory, or because the orthodontic appliance is causing overcorrection of tooth positioning), the color of the element can be modified.

[0270] In one implementation, the virtual tooth scene does not include a reference image. The reference image may be used to determine the properties and / or positions of elements in the virtual tooth scene. However, it is not presented to the observer in step e)(or step C).

[0271] In one implementation, the virtual tooth scene is determined based on past or future simulation moments. Specifically, it may include representations of one or more elements of the corresponding real tooth scene in past or future constructions, preferably realistic representations.

[0272] In one implementation, the virtual dental scene includes a representation of the dental arch measured or simulated at a past simulation moment and / or a representation of the orthodontic appliances worn on the teeth of the dental arch.

[0273] In one implementation, the virtual dental scene includes a representation of the dental arch as anticipated at future simulation moments and / or a representation of the orthodontic appliances worn on the teeth of the dental arch.

[0274] Simulations of past or future moments can be performed by any simulation software or by an operator (such as a dental care professional).

[0275] A virtual dental scene can, for example, represent the outline of teeth, the positioning of orthodontic appliances, or the color of teeth at a simulation time. Therefore, the presentation of the virtual dental scene on the actual dental scene, preferably in a transparent mode, allows for easy detection of changes between the update time and the simulation time.

[0276] When an indicator represents an invisible portion of a physical element in an actual dental scene (e.g., the root of a tooth), information related to that indicator is preferably included in a reference model. For example, the reference model could be a model of a dental arch, where the crown and root of the teeth are modeled, for example, using a cone-beam scanner. Thus, the method according to the invention allows the operator to view the invisible portions of the patient's dental arch in the actual dental scene. Advantageously, these invisible portions are represented by their actual relative positions to the physical elements of the actual dental scene. For example, the operator sees the roots and nerves of the teeth as observed by him or her in the patient's mouth. Therefore, he or she can avoid interacting with the tool he or she is manipulating or can adjust the orthodontic appliance to limit the risk of collision between the roots of adjacent teeth, and thus limit the risk of root resorption. For example, when administering an injection using a syringe, the operator can precisely introduce the needle to reach the tooth root without touching the dental nerve. This limits the pain felt by the patient.

[0277] In step e), a virtual tooth scene is presented based on a reference image, which is overlaid on the actual tooth scene or on the representation of the actual tooth scene in the updated image.

[0278] Virtual dental scenes can be directly projected onto actual dental scenes. For example, red dots can be projected onto teeth to identify the bonding areas of orthodontic attachments.

[0279] Virtual teeth scenes can be displayed on the screen.

[0280] When the screen is transparent, especially in implementations where the screen is integrated into glasses, the operator can thus observe both the virtual teeth scene on the screen and the actual teeth scene behind the virtual teeth scene in a transparent mode.

[0281] When the screen is opaque, especially in implementations where the screen is that of a mobile phone, laptop, virtual reality headset, or tablet, the operator can thus simultaneously observe updated images on the screen and virtual teeth scenes.

[0282] Virtual tooth scenes can be transparent enough to reveal the actual tooth scene or an updated image underneath.

[0283] Alternatively, the virtual dental scene can be opaque so as not to reveal the actual dental scene or updated images extending behind or below it. The opacity of the virtual dental scene is particularly advantageous for simulating past or future dental conditions, such as viewing the whitening process under the influence of treatment for whitening purposes, or viewing tooth movement under the influence of orthodontic treatment, or viewing periodontal damage of teeth in the absence of orthodontic treatment. When presenting the virtual dental scene on an updated image, it can be added on top of the updated image, or equivalently, the updated image can be replaced with an image reprocessed by the computer to reveal the virtual dental scene.

[0284] Therefore, "the presentation of a virtual tooth scene overlaid on a representation of the actual tooth scene in an updated image" should be understood to include any presentation of a virtual tooth scene shown over a representation of the actual tooth scene, including the display of a reprocessed image.

[0285] When the screen is made of mirrored glass, the operator can simultaneously observe their own reflection on the screen, as well as a virtual scene of teeth superimposed on their reflection. Specifically, the patient can view the effects of dental treatment (therapeutic or non-therapeutic).

[0286] The virtual tooth scene is rendered based on a reference image, and therefore indirectly based on an updated image. In particular, it is positioned or “framed” relative to the actual tooth scene based on the reference image.

[0287] The virtual tooth scene is presented in a way that overlaps or is "registered" with the actual tooth scene or its representation on an updated image, which facilitates comparison. In other words, the elements of the virtual tooth scene corresponding to the elements of the actual tooth scene are essentially overlaid precisely on the actual tooth scene or its representation. Especially when the virtual tooth scene is presented in a transparent mode, the differences between the actual tooth scene or its representation and the virtual tooth scene are thus clearly visible to the operator. The indicator also presents him or her with information that helps in analyzing the situation.

[0288] Preferably, in order to register the virtual tooth scene onto a representation of the actual tooth scene on an updated image, feature markers are identified on the updated image, the position of which relative to the virtual tooth scene is known. For example, the ends of teeth or characteristic shapes of teeth represented in both the updated image and the virtual tooth scene are identified. Specifically, a reference image from which the virtual tooth scene is determined can be used to identify the positions of the feature markers in the virtual scene. In step e)(or step C), the representations of these feature markers are then overlaid on the updated image and the virtual tooth scene, which allows for precise positioning of the virtual tooth scene relative to the feature markers represented in the updated image.

[0289] Preferably, the method then returns to step c) (or step A) after step e) (or step C) (preferably less than 30 seconds, less than 10 seconds, less than 5 seconds, less than 3 seconds, less than 1 second, less than 0.5 seconds, less than 0.2 seconds, and more preferably less than 0.1 seconds. Preferably, the cycle from step c) to e) (or from step A) to C) (is performed continuously in real time.

[0290] Alternatively, in one implementation, the method completes at the end of step e)(or step C)() without returning to step c)(or step A). ​​Therefore, the method is executed "on demand" rather than in real time.

[0291] Application Examples

[0292] Real-time assistance in dental intervention

[0293] In particular, when the image acquisition device is in the form of glasses, the present invention can be used for dental intervention by dental care professionals wearing such glasses.

[0294] In one implementation, the method is used in the context of a tooth stripping or milling operation. Preferably, the virtual tooth scene represents the final shape of the tooth after the stripping or milling to be performed. Preferably, the dental care professional sees the outline of the final-shaped tooth precisely overlap (i.e., registered) with the actual tooth. This information allows him or her to machine the tooth with perfect precision.

[0295] In one embodiment, the method is used to attach a dental attachment or "bracket" to a tooth. Preferably, the virtual dental scene represents the attachment in its final position after it has been attached to the tooth. Preferably, a dental care professional sees a virtual outline of the attachment relative to the actual tooth in its final position. This information allows him or her to precisely overlap the actual attachment with the attachment in the virtual dental scene, thereby positioning the attachment on the actual tooth with perfect accuracy.

[0296] This method can also be used to attach decorative items to teeth.

[0297] During intervention, this invention allows the operator, particularly dental care professionals, access to relevant information. This information is preferably presented to him or her in real time, overlaid with the actual dental scene as directly observed by him or her, or on an updated image. Therefore, this invention enables a significant improvement in the quality of situation analysis and allows for appropriate adjustments to the intervention as needed.

[0298] Improved compliance and use for educational purposes

[0299] In one implementation, the method is used to achieve better patient acceptance of treatment. Specifically, the method can be used to realistically simulate dental treatment. Preferably, the image acquisition device is a mobile phone, laptop, tablet, or mirror.

[0300] The patient uses a mobile phone, laptop, or tablet to photograph the dental arch, much like taking a selfie. The patient uses a mirror, preferably through the surface of the mirror, which allows for updated images that accurately represent the reflections R observed by the patient.

[0301] By overlaying updated images that follow one after another on the screen of a phone, laptop, or tablet, or by overlaying his or her reflection in a mirror, the patient sees corresponding virtual dental scenes representing a step in a dental treatment at a simulated moment. For example, the simulated moment could mark the start of treatment, and the virtual dental scene could represent the orthodontic appliance the patient is considering wearing. The patient can then view his or her face after the appliance is placed in his or her mouth. By turning his or her head or moving the phone, laptop, or tablet, or by moving in front of the mirror, he or she can easily change the direction of observation, thus adjusting the virtual dental scene in real time.

[0302] Preferably, the patient can modify the simulated timeline by interacting with the screen, for example, by moving a virtual cursor. The virtual teeth scene is adjusted accordingly, especially considering the treatment's effects. By acting on the cursor, the patient can then simulate the progress of the treatment, that is, see, for example, how the teeth will move and / or change color (especially in the case of teeth whitening treatments, or, for example, if the patient smokes or drinks large amounts of tea or coffee, the teeth will be affected by the staining agent). Therefore, moving the cursor allows him or her to accelerate the simulated time.

[0303] This simulation helps patients make decisions and effectively motivates them to follow treatment.

[0304] In one implementation, the patient can modify the values ​​of other parameters in the simulation. For example, he or she can modify the values ​​of treatment parameters.

[0305] For example, he or she can simulate:

[0306] - The effects of one or more orthodontic appliances, especially in order to choose the orthodontic appliance that is best suited for him or her;

[0307] - Temporarily or permanently stop the effects of the current treatment;

[0308] - To more or less strictly follow the instructions received, especially to see the effects of changing the frequency and / or duration and / or technique of brushing, or the effects of delaying the replacement of orthodontic splints and / or delaying appointments with dental care professionals.

[0309] Therefore, he or she can simulate the impact of poor adherence to medical prescriptions, which is a factor in improving compliance.

[0310] In one implementation, the virtual teeth scene preferably displays a warning in the form of a message or a highlighting of elements of the virtual teeth scene (e.g., by highlighting areas that should be brushed more carefully in red).

[0311] Typically, this invention can be used to assess a patient’s dental condition, especially to examine the progress of dental treatment, and / or to examine changes in the color and / or shape and / or arrangement of teeth or gums outside the scope of dental treatment.

[0312] Analysis of changing dental conditions

[0313] In one implementation, the method is used to examine the stripping of teeth (e.g., through bruxism). Preferably, the virtual tooth scene represents the prior shape of the tooth before stripping. Preferably, the operator (typically a patient or dental care professional) sees the outline of the tooth in its prior shape over a representation of the actual tooth on an updated image. This information allows him or her to immediately and accurately view the proportion of this wear, especially when the virtual tooth scene is presented in a transparent mode.

[0314] In one implementation, the method is used to observe gingival deformation, particularly gingival concavity. Preferably, the virtual dental scene represents the initial shape of the gingiva before its deformation. Therefore, the operator (typically a patient or dental care professional) will see the outline of the gingiva in its initial shape, superimposed on or over a representation of the actual gingiva. This information allows him or her to view the deformation immediately and precisely, especially when the virtual dental scene is presented in a transparent mode.

[0315] Of course, the present invention is not limited to the embodiments described above and shown.

[0316] In particular, the patients are not limited to humans. The method according to the invention can be used in other animals.

[0317] This invention is not limited to the method including steps a) through e). In particular, searching for a reference image is not essential for the implementation of this invention. If a reference image is unavailable, elements constituting the virtual tooth scene can be located on the actual tooth scene or its representation based on an updated image.

[0318] A virtual tooth scene can be determined based on multiple reference images obtained from multiple corresponding updated images acquired simultaneously or at different (e.g., consecutive) update times.

Claims

1. An apparatus for implementing a method for simulating a virtual dental scene based on a patient's actual dental condition, the method comprising a cycle of steps A) to C), the cycle of steps A) to C) comprising the following consecutive steps: A) At the update time, acquire an updated image, which represents the actual tooth scene as observed by the operator; B) Determine a virtual dental scene based on the actual dental scene represented on the updated image, the virtual dental scene including a representation of the patient's dental arch and / or a representation of the orthodontic appliance expected to be worn by the patient's teeth at a simulation time after the update time; identify elements in the virtual dental scene that do not meet the evaluation criteria, and highlight the elements in the virtual dental scene and / or add messages related to the elements to the virtual dental scene; Then C) Present the virtual tooth scene on the screen in a transparent mode and overlay it on the actual tooth scene, or The updated image is displayed on the screen, and the virtual tooth scene is rendered as an overlay on the representation of the actual tooth scene on the updated image displayed on the screen, the virtual tooth scene being overlaid on the representation of the actual tooth scene in a transparent or opaque mode. Repeat steps A) and C) in real time within five seconds after step C). The device includes: - Camera (16), said camera (16) is configured to perform step A); - Processing unit (12), which is configured to implement step B); - The screen (18), the screen (18) is configured to implement step C); The camera (16), the processing unit (12), and the screen (18) are configured to communicate with each other to implement the steps.

2. The device as claimed in claim 1, wherein, The virtual tooth scene is related to the actual tooth scene simulated at the simulation time, and the simulation time is more than three days apart from the update time.

3. The device as claimed in claim 1, wherein, - The screen (18) is a touchscreen, and the time interval between the update time and the simulation time is modified through interaction with the screen, and / or - The screen (18) includes an area for inputting the simulated time.

4. The device as claimed in claim 1, wherein, Between the two cycles of steps A) to C), the time interval between the update time and the simulation time can be modified.

5. The device as claimed in claim 1, wherein, The camera (16) and / or the screen (18) are integrated into a mobile phone, laptop, virtual reality headset, tablet, camera or mirror (17), the glass (19) of the mirror forming the screen, or the screen and the camera are independent of each other, that is, the screen and the camera are not rigidly fixed to each other.

6. The device as claimed in claim 1, wherein, In step B), the color and / or shape and / or size and / or position of the visible or invisible parts of the mouth are determined such that the virtual tooth scene reproduces the color and / or shape and / or position simulated at the simulation time.

7. The device as claimed in claim 1, wherein, The assessment criteria involve the patient's health status and / or the positioning of the orthodontic appliance worn by the patient and / or the condition of the orthodontic appliance worn by the patient and / or the cost of dental treatment.

8. The device as claimed in claim 1, wherein, In step B), the virtual dental scene is determined based on the value of at least one treatment parameter that can be modified by the operator.

9. The device as claimed in claim 8, wherein, The treatment parameters relate to the patient's wearing of orthodontic appliances and / or the patient's adherence to treatment instructions.

10. The device as claimed in claim 1, wherein, The method comprises the following sequential steps: a) At the reference time, generate a reference model for digitally modeling at least one dental arch of the patient; b) Modify the reference model; c) At the update time, obtain the updated image according to step A); d) By observing the modified reference model, search for a reference image that shows the maximum matching degree with the updated image, and then determine the virtual tooth scene based on the reference image according to step B); e) Present the virtual teeth scene according to step C). The device includes means for generating and modifying the reference model in steps a) and b), the means including a scanner and a computer. The camera is configured to implement step c), the processing unit is configured to implement step d), and the screen is configured to implement step e.

11. The device as claimed in claim 10, wherein, In step a), the reference model is generated as follows: Scan the patient's dental arch or a physical model of the dental arch, or Select a general model for the dental arch from the library of general models, then The general model is modified based on measurements or observations of the patient.

12. The device as claimed in claim 10, wherein, In step b), the reference model is modified to simulate the effect of the time flow between the reference time and the simulated time.

13. The device as claimed in claim 1, wherein, The device is configured to simulate: - The effect of one or more orthodontic appliances; - To temporarily or permanently stop the effects of the current treatment, whether the current treatment is therapeutic or non-therapeutic; - The effectiveness of future treatment, whether the future treatment is therapeutic or non-therapeutic; - The effect of the application instructions; - The effects of changing the frequency and / or duration and / or technique of brushing, or the effects of delaying the replacement of orthodontic splints and / or delaying appointments with dental professionals; - The effectiveness of teeth whitening treatments, or the effects of smoking; or - Assess the position and / or shape and / or size of the teeth, and / or proper positioning of the orthodontic appliances fixed to the teeth.

14. The device as claimed in claim 1, wherein, The message is an instruction that the patient and / or the dental care professional responsible for the patient must follow.

15. The device as claimed in claim 1, wherein, The virtual tooth scene is projected directly onto the actual tooth scene.

16. The device as claimed in claim 15, wherein, Red dots are projected onto the teeth to identify the bonding areas of orthodontic attachments.

Images