Method for acquiring set of images of object in mouth

Abstract

The invention relates to a method for acquiring a set of images covering a target belonging to an object in the mouth of a user, the method comprises the steps of: 1) presenting a multi-dimensional symbol or a set of multi-dimensional symbols (24) to the user on a screen (12) and using spatially augmented reality relative to the object in the mouth observed by an image acquisition device (10), determining the shape and / or position of each symbol in order to indicate to the user at least one predetermined acquisition condition suitable for acquiring such an image; 2) for each symbol, acquiring such an image using the acquisition means when said at least one predetermined acquisition condition associated with said symbol is satisfied, preferably when all predetermined acquisition conditions associated with said symbol are satisfied.

Description

Technical Field

[0001] The present invention relates to a method for acquiring a set of images of an object in the mouth, and in particular a dental object, in particular a dental arch of a user. The present invention also relates to a device for implementing such a method. Background Art

[0002] In order to analyze the dental condition of a user, in particular before establishing orthodontic treatment, a set of images covering an object located in the user's mouth (and in particular the user's dental arch) is usually acquired. An image "covers" an object when the image at least partially represents the object. The set of images "covers" an object when the set of images includes images covering the object from different viewing directions, in particular in order to provide accurate three-dimensional information about the object.

[0003] Acquisition in a dental office or more generally from a dental care professional can involve significant costs and stress to the user.

[0004] Alternatively, the user may acquire dental images himself, for example using his mobile phone. To acquire the images, the user typically looks into a mirror, which makes it difficult to position the phone accurately. More importantly, the user does not know whether the images are of good quality or whether they correctly cover the intended target until he has consulted the gallery where the images are stored. Even when consulting the gallery, he cannot accurately assess whether the coverage of the target by the acquired set of images is sufficient. Finally, acquisitions can be labor-intensive if the user needs to repeat the acquisition. Frustrating as these difficulties are, the user may terminate the acquisition operation prematurely.

[0005] There is a need to facilitate the acquisition of a set of images of the mouth covering a target, in particular all or part of a dental arch, while limiting the risk of incomplete or poor quality acquisition.

[0006] It is an object of the present invention to meet this need. Summary of the invention

[0007] The present invention proposes a method for acquiring a set of images covering a target belonging to an object in a user's mouth (for example, a set of images covering the incisors or teeth (targets) of a user's dental arch (object in the mouth)), wherein the set of images preferably has a coverage rate greater than or equal to a coverage threshold.

[0008] According to a first main aspect of the invention, the method comprises the following steps:

[0009] 1) presenting to the user, on a screen and using spatial augmented reality relative to an object in the mouth observed by an image acquisition device (preferably an image acquisition device operated by the user), a multi-dimensional symbol or a set of multi-dimensional symbols, the shape and / or position of each symbol being determined so as to indicate to the user at least one predetermined acquisition condition suitable for acquiring such an image;

[0010] 2) For each symbol, when at least one predetermined acquisition condition associated with the symbol is met, preferably when all predetermined acquisition conditions associated with the symbol are met, using the acquisition device to acquire such an image.

[0011] As will be seen in more detail later in the description, when the user looks at the screen, the shape of the multidimensional symbol in the space of the real scene observed by the acquisition means makes it possible to inform and guide him towards one or more acquisition conditions associated with said symbol and suitable for acquiring the desired image. Thus, the symbols presented in augmented reality provide particularly effective guidance information.

[0012] The method according to the first main aspect of the invention may also in particular comprise one or more of the following optional features:

[0013] - in step 2), the image is acquired by the acquisition means only if the at least one predetermined acquisition condition associated with at least one symbol is met, preferably only if all predetermined acquisition conditions associated with the symbol are met;

[0014] - At least one symbol qualifier

[0015] a symbol axis, preferably a rotation axis, the acquisition condition being an angular deviation of less than 20° (preferably less than 10°, preferably less than 5°, preferably substantially zero) between the optical axis of the acquisition device and said symbol axis, i.e. the optical axis of the acquisition device is substantially perfectly aligned with said symbol axis, and / or

[0016] a size which is variable on the representation of the symbol on the screen as a function of the distance between the acquisition means and the symbol in the augmented reality, the acquisition condition being for a predetermined specific value of said size or for said size to belong to a predetermined specific range of values, for example said size is less than 1 mm;

[0017] - before step 1), teaching the user, for each symbol, said at least one predetermined acquisition condition associated with said symbol; for example, telling the user that in order to acquire the set of images he must, for each symbol, align the axis of the symbol as closely as possible with the optical axis of the acquisition device and / or try to view the symbol on the screen so that said size is as close as possible to said specific value;

[0018] - said teaching is carried out by means of a tutoring program which preferably makes it possible to progressively teach all the functions of a computer program executed to implement the method, said tutoring program being upgradeable based on updates;

[0019] - in step 2), automatically acquiring said image with the acquisition device if predetermined acquisition conditions defining the spatial position and / or orientation of the acquisition device around its optical axis and comprising said at least one acquisition condition indicated by the symbol are met;

[0020] - In step 2), if such an image is acquired by the acquisition device, then

[0021] - modifying the appearance of the symbol or causing it to disappear, and / or

[0022] - emit an audible signal, and / or

[0023] - modifying a score displayed on the screen and related to the coverage of the target by the image already acquired and / or to the duration of acquisition of the image already acquired and / or to the quality of the image already acquired and / or to the usefulness of the image already acquired;

[0024] - when the set of images has been acquired, presenting to the user on the screen an ordering determined according to said scores;

[0025] - the screen displays these symbols on a preview image representing the real scene observed by the acquisition means or on a view representing at least a model of said object or said target in the mouth like said preview image;

[0026] - the screen displays a reference mark in a fixed position on the screen, the reference mark preferably having a constant shape, such symbol preferably having a shape complementary to the reference mark when a predetermined acquisition condition associated with said symbol is met;

[0027] - the object comprises more than 2, preferably more than 5 and / or less than 32 teeth, and / or the set of images comprises more than 2, more than 5, more than 10 and / or less than 1,000 images taken under different respective acquisition conditions, and / or the set of symbols comprises more than 2, more than 5 and / or less than 100 symbols;

[0028] - the symbols preferably each have an axis oriented towards the center of the user's mouth;

[0029] - the symbols are preferably distributed in one plane, preferably in two planes, preferably in three planes, preferably in order to obtain an image with a viewing axis in the occlusal plane, in a plane inclined to the occlusal plane in order to obtain a top view relative to the occlusal plane and in a plane inclined to the occlusal plane in order to obtain a bottom view relative to the occlusal plane;

[0030] - In the space for objects in the mouth of the augmented reality representation, these symbols are outside the user's mouth, so that the acquired image is extra-oral.

[0031] In one embodiment, the symbol is part of the surface of the target and the preview image or equivalent image displays the aiming point for visualizing the angle of the acquisition device. The aiming point is displayed on the surface of the target on the extension of the optical axis in the same way as the aiming point appears on the object according to the firing direction when the object is aimed by a firearm that projects a laser beam onto the object.

[0032] In order to correctly orient the acquisition device, the user must therefore aim the aiming point at the symbol represented on the surface of the target.

[0033] In this embodiment, the symbol can be a point or a surface. If the symbol is a surface, its outline is deformed according to the shape and distance of the surface of the target onto which it is projected. The symbol can be advantageously used to indicate the distance of the acquisition device from the target and / or the orientation of the acquisition device around its optical axis.

[0034] The shape of the aiming point is not restricted.

[0035] Specifically, the symbol may be a thumbnail, ie a small image, such as a star in a video game, attached to the tooth.

[0036] The set of symbols preferably defines acquisition conditions for acquisition of

[0037] - at least one image taken facing the user and / or at least one image taken toward the right side of the user and / or at least one image taken toward the left side of the user, preferably at least one image taken facing the user and at least one image taken toward the right side of the user and at least one image taken toward the left side of the user; and / or

[0038] - on the one hand, at least one image taken facing the user, and on the other hand, at least one image taken from above the user and / or at least one image taken from below the user; preferably, at least one image taken facing the user and at least one image taken from above the user and at least one image taken from below the user; and / or

[0039] - at least one image taken with the mouth open and / or at least one image taken with the mouth closed.

[0040] The invention also relates to a device for implementing the method according to the first main aspect of the invention, the device comprising

[0041] - an image acquisition device, preferably in the form of a mobile phone;

[0042] a computer, preferably integrated into the acquisition device or in communication with the acquisition device, having a computer program comprising program code instructions for

[0043] - in step 1), one or more multidimensional symbols are arranged in the spatial augmented reality of the object in the mouth and presented to the user on a screen, preferably on a screen of an acquisition device, the shape and / or position of the symbols being determined so as to indicate to the user at least one predetermined acquisition condition suitable for acquiring such an image;

[0044] - Preferably, in step 2), the acquisition of such an image by the acquisition device is authorized only if the at least one predetermined acquisition condition associated with the symbol is met, and / or the acquisition device is commanded to acquire such an image only if the at least one predetermined acquisition condition is met, and / or the coverage level of the acquired image of the target is updated, and preferably the coverage level is compared with a coverage threshold, and information about the coverage level and / or is preferably presented on the screen.

[0045] Or information about the difference between the coverage threshold and the coverage level.

[0046] The acquisition means is preferably

[0047] - a mobile phone, and the screen is integrated into the mobile phone, or

[0048] A device comprising a holder equipped with a camera and designed to rest against a user, in particular against the teeth and / or gums of a user, during image acquisition, the screen being integrated into the holder or being at a distance from the holder.

[0049] According to a second main aspect of the invention, the method aims to quickly cover a target with a coverage rate greater than or equal to a coverage threshold, and the method comprises the following steps:

[0050] a) acquiring at least one image, preferably by a user, using an image acquisition device, preferably a mobile phone;

[0051] b) updating a target coverage level based on the at least one image acquired in step a);

[0052] c) If the coverage level is below the coverage threshold,

[0053] - determining guidance information that directs a person toward suitable acquisition conditions for acquiring additional images of increased coverage level using the image acquisition device;

[0054] - presenting guidance information to the user, and preferably presenting information about the coverage level and / or about the difference between the coverage threshold and the coverage level, i.e. information enabling the user to be aware of the progress of the acquisition of the set of images, and,

[0055] - the position and / or orientation of the acquisition device is modified by the user according to the guidance information and step a) is preferably repeated in real time, the acquired images being preferably extracted from the video viewed by the user on the screen of the acquisition device.

[0056] As will be seen in more detail later in the description, the determination of guidance information based on coverage updated in real time advantageously facilitates the acquisition of images of the user's mouth, and in particular the acquisition of dental images. In particular, the user receives real-time guidance information, which makes the acquisition more efficient, especially when the guidance information is selected to guide people towards optimal acquisition conditions that enable the acquisition of additional images that maximize the increase in coverage levels.

[0057] Under the guidance, the user does not need to visit a dental care professional, or the user does not need to be supervised by a dental care professional to obtain the images. The images can advantageously be obtained under precise acquisition conditions without any special training. In particular, the images can be obtained by the user himself or by one of his relatives. Notably, the method facilitates the acquisition of images of the dental arch of a child by a parent.

[0058] The presentation of information about the coverage level is also particularly advantageous, since it effectively dissuades the user from interrupting the acquisition before it is complete. This makes the acquisition particularly enjoyable, since the user always knows what else he has to do before acquiring the entire set of images. The acquisition can even be fun.

[0059] The method according to the second main aspect of the invention may also in particular comprise one or more of the following optional features:

[0060] - displayed on a screen, preferably on a screen of an acquisition device, preferably a mobile phone:

[0061] - representation of the target,

[0062] - a representation of the part of the object covered by the image or images acquired in step a) or in an earlier step a), and

[0063] - preferably, a representation of an object in the mouth when the object in the mouth comprises a portion of the user's mouth other than the target;

[0064] - guidance information and / or information about the coverage level and / or about the difference between the coverage threshold and the coverage level are presented in augmented reality in a preview image and / or an image equivalent to the preview image, which is displayed on a screen of the acquisition device and represents the real scene observed by the acquisition device, and the image equivalent to the preview image represents a theoretical scene, which symbolically or realistically represents all or some elements of the real scene in the same arrangement as in the real scene;

[0065] - the equivalent image preferably comprises a view of a model of at least a part of said real scene, the model being a digital three-dimensional model;

[0066] - presenting information about the coverage level and / or about the difference between the coverage threshold and the coverage level in a spatial reference frame, so as to constitute so-called guidance information;

[0067] - presenting information about the coverage level and / or about the difference between the coverage threshold and the coverage level on the representation of the object,

[0068] - applying an appearance, preferably a color or a texture, to a region of said representation, which appearance is different depending on whether said region is covered by at least one acquired image, or

[0069] - applying an appearance to a symbol representing a predetermined portion of said object (e.g. a tooth), said appearance being different depending on whether said portion is covered or not, preferably by making the symbol appear or disappear on the screen of the acquisition device, or

[0070] - by applying an appearance to a symbol indicating at least one acquisition condition, said appearance being different depending on whether an image has been acquired using said at least one acquisition condition;

[0071] - the object comprises a set of teeth and / or soft tissues, preferably updating a preview image or equivalent in real time on the screen of the user's mobile phone, the covered teeth being particularly marked by a symbol or by applying an appearance enabling the covered teeth to be distinguished from the uncovered teeth, the teeth being "covered" when their initial surfaces to be covered are covered;

[0072] - in augmented reality, a plurality of symbols are anchored in the real scene observed by the acquisition device so as to appear on the preview image or on an equivalent image, each symbol being able to be anchored, for example, on a corresponding tooth of the user or so as to indicate to the user suitable acquisition conditions for acquiring additional images;

[0073] - the symbols are three-dimensional symbols according to the first main aspect of the invention;

[0074] - in step a), the user aims at such a symbol, preferably a two-dimensional or three-dimensional symbol, with the acquisition device, and

[0075] acquiring at least one image, preferably automatically, i.e. without specific user intervention, when the target symbol is reached, preferably when it is at least partially superimposed on a fixed reference mark displayed on a screen of the acquisition device;

[0076] - in augmented reality, the symbols are anchored on adjacent or non-adjacent teeth on the preview image or on an equivalent image, for example the symbols may be anchored regularly along the dental arch, for example every 2 or 3 teeth;

[0077] - the symbol is anchored and / or shaped so as to define, optionally in cooperation with such reference markings, predetermined acquisition conditions, preferably defining the distance of the acquisition device from the target and / or the orientation of the acquisition device about its optical axis and / or the angle of said optical axis relative to the target;

[0078] - in a step b), analyzing the at least one image acquired in step a) in order to identify a representation of an object on said image, preferably by means of a neural network, and then marking the corresponding area in the preview image, preferably by coloring it with a color associated with said object;

[0079] - in step a), before or after image acquisition, evaluating the quality of the acquired or previewed image, respectively displayed on the screen of the acquisition device, preferably evaluating at least the sharpness and / or the contrast and / or the color balance of the image, and / or the distance of the acquisition device from the target and / or the orientation of the acquisition device about its optical axis and / or the angle of said optical axis relative to the target, and acquiring the image and / or updating the coverage only if this quality exceeds a predetermined quality threshold;

[0080] - in step c),

[0081] testing a plurality of suitable acquisition conditions for acquiring additional images of increased coverage level using the image acquisition device, and then determining guidance information in a manner such that the guidance information guides a person toward optimal acquisition conditions, the optimal acquisition conditions being those conditions suitable for acquiring additional images of increased coverage level to the greatest extent using the image acquisition device;

[0082] - in step b),

[0083] - Each acquired image is submitted to

[0084] at least one first neural network trained to detect object representations in images and / or an image processing algorithm adapted to detect object representations in images, and

[0085] a second neural network, the second neural network being trained to determine the position and orientation of the acquisition device during image acquisition and, therefore, the direction of the optical axis of the acquisition device (i.e., the camera of the acquisition device) relative to the target and the distance of the acquisition device from the target, and then

[0086] - the target representation is projected onto a model or "reference model" of the target or object in the mouth along a projection direction oriented relative to the model in the direction of the optical axis relative to the target, the target representation being virtually (i.e. theoretically) positioned relative to the reference model when the acquisition device is relative to the target, and then

[0087] - determining the coverage from the area of ​​the reference model covered by said projections and from said projections generated in any previous step b);

[0088] - in step b),

[0089] - Each acquired image is submitted to

[0090] a first neural network trained to detect representations of objects or salient points of said objects in an image, and

[0091] a second neural network trained to identify a corresponding region of the reference image from the representation of the object or said salient point in the image, and then - determining the coverage based on the area of ​​the reference image covered by said corresponding region and said corresponding region determined in any previous step b);

[0092] - The acquisition device is

[0093] - a mobile phone with a screen, preferably the user's own mobile phone, or

[0094] - A device comprising

[0095] a holder equipped with a camera and placed against the user during the acquisition of a set of images, preferably allowing the mouth to be opened and closed, preferably partially introduced into the user's mouth, preferably against the gums and / or teeth, and

[0096] - a screen which displays the scene observed by the camera, said screen being integrated in the holder or at a distance therefrom;

[0097] - in step a), the user visualizes in real time on the screen of said mobile phone the actual scene or the corresponding theoretical scene observed by the mobile phone, preferably automatically acquiring images when the mobile phone observes the object under predetermined acquisition conditions, preferably the acquisition conditions determined in step c) of the previous cycle of steps a) to c);

[0098] - in step c), presenting to the user, preferably on a screen of the acquisition device, a counter or meter, preferably in the form of a progress bar, providing information about the coverage level and / or about the difference between the coverage threshold and the coverage level, and / or

[0099] -When the coverage threshold is reached, a score calculated as a function of the time taken to reach the coverage threshold and / or the quality of the acquired images and / or the usefulness of the acquired images is preferably presented to the user on the screen of the acquisition device, and / or a ranking determined according to the score is preferably presented to the user on the screen of the acquisition device.

[0100] In a first main embodiment, the guidance information comprises a set of symbols positioned in augmented reality according to the corresponding images to be acquired. The set of images to be acquired comprises one image for each symbol. In step a), the user has to point at the symbols on his mobile phone and select them, as in a video game. For example, in augmented reality, each symbol can be anchored on the corresponding tooth, the target being constituted by said tooth and / or soft tissue.

[0101] Depending on the desired acquisition conditions, these symbols may be anchored, for example, on non-adjacent teeth, such as every second or third tooth.

[0102] The reference mark may be depicted on the screen of the user's mobile phone. When the reference mark overlaps with a symbol, the latter is selected; an image is then acquired, preferably automatically, and the symbol is marked or made to disappear. The marking or disappearance of the symbol provides information about the coverage. It also provides guiding information, since the user can easily find symbols that have not yet been selected on the preview image displayed on the mobile phone screen. He can arrange the mobile phone accordingly. Once all symbols have been selected, the set of images covers all target teeth and / or soft tissues. For example, the coverage level may be the ratio of the number of symbols reached to the initial number of symbols before the start of the capture.

[0103] In a variant of this first main embodiment, the symbol is anchored and / or shaped so as to define, in cooperation with a reference mark, predetermined acquisition conditions, preferably defining the distance of the acquisition device from the target and / or the orientation of the acquisition device about its optical axis and / or the angle of said optical axis relative to the target.

[0104] In one embodiment, the optional reference mark is shaped to be superimposed on multiple symbols (e.g., two or three symbols) at the same time. For example, it includes multiple basic reference marks, such as circles, which must be superimposed on multiple corresponding symbols at the same time. Advantageously, the superposition corresponds to a predetermined angle and distance between the acquisition device and the target. Preferably, the symbols in the multiple symbols have an appearance specific to the multiple symbols, such as color. For example, the user first aims to place three green symbols in the circle of the reference mark, then three red symbols in the circle of the reference mark, and so on.

[0105] Likewise, in order to set a predetermined distance between the acquisition device and the target, the reference mark and the symbol may have compatible sizes so that when the user accurately superimposes the reference mark and the symbol, the acquisition device and the target are separated by the distance.

[0106] The symbols of the plurality of symbols and the base reference mark are preferably different from each other, for example they have different numbers. For example, the user intends to place three symbols numbered 1, 2 and 3 in circles numbered 1, 2 and 3 on the reference mark, respectively. Advantageously, if the base reference mark is misaligned, such superposition can set a predetermined orientation of the acquisition device around its optical axis.

[0107] Equivalently, in order to set the orientation, the reference marks and symbols may have a non-rotational shape, such as a rectangular shape, preferably a shape without symmetry, so that if the acquisition device is oriented in one or more predetermined orientations around the optical axis, the user can only superimpose the reference marks and symbols in the same orientation.

[0108] In particular, when symbols are used to set angles, distances or orientations on the acquisition device, they are not necessarily associated with a specific tooth, in particular with the tooth to be capped.

[0109] In a second main embodiment, a view of a three-dimensional model of an object (e.g. a set of teeth) is displayed on a mobile phone screen. In one embodiment, the user can modify the view, i.e. change the viewpoint of the model, using conventional model manipulation software. In a preferred embodiment, the view is modified according to the viewing conditions of the object by the mobile phone, preferably following the principles of augmented reality. The template can then be displayed, possibly transparently, superimposed on the preview image, or instead of the preview image. When the preview image covers the object, the image is preferably acquired automatically. The area of ​​the object shown on the acquired image is marked on the model, preferably colored, for example green. The area of ​​the object that remains to be covered is shown differently, for example in red. The user can thus immediately see the coverage rate, for example the ratio between the green area and the total red and green areas. The unmarked areas guide him towards the acquisition conditions that will increase the coverage rate.

[0110] In a variation of the second main embodiment, the reference model view is replaced by a reference image (eg a photograph, such as a panoramic photograph).

[0111] Preferably, the method according to the invention is implemented to obtain

[0112] - at least one image taken facing the user and / or at least one image taken toward the right side of the user and / or at least one image taken toward the left side of the user, preferably at least one image taken facing the user and at least one image taken toward the right side of the user and at least one image taken toward the left side of the user; and / or

[0113] - on the one hand, at least one image taken facing the user, and on the other hand, at least one image taken from above the user and / or at least one image taken from below the user; preferably, at least one image taken facing the user and at least one image taken from above the user and at least one image taken from below the user; and / or

[0114] - at least one image taken with the mouth open and / or at least one image taken with the mouth closed.

[0115] The invention also relates to a device for implementing the method according to the invention, the device comprising:

[0116] - an image acquisition device, a computer equipped with a camera, in particular a mobile phone or a tablet, preferably a mobile phone, or a mirror equipped with a camera, for acquiring a set of images, in particular for implementing step a);

[0117] a computer, which may be an acquisition device, preferably integrated in a mobile phone or in communication with the mobile phone, having a computer program comprising program code instructions for

[0118] According to a first main aspect of the invention, a three-dimensional symbol providing guidance information in augmented reality is presented, and / or

[0119] According to the second main aspect of the present invention

[0120] - updating the coverage level in step b),

[0121] - comparing the coverage level to a coverage threshold, and

[0122] - if the coverage level is less than the coverage threshold, determining in step c) guidance information and presenting the guidance information and preferably information about the coverage level and / or about the difference between the coverage threshold and the coverage level to the user;

[0123] - A screen, preferably a computer screen, preferably for displaying guidance information and according to the second main aspect of the invention, information about the coverage level and / or about the difference between the coverage threshold and the coverage level.

[0124] In a preferred embodiment, the computer program is executed by the image acquisition device, whereby the computer program can be integrated into dedicated software, in particular for a mobile phone or a tablet computer.

[0125] The screen may be integrated into the image acquisition device. Preferably, the screen is the screen of a mobile phone or a tablet computer.

[0126] Preferably, the screen and the computer program are integrated into the image acquisition device.

[0127] The user can then easily acquire an image of the object with good quality and with the entire object covered together by means of a simple mobile phone or tablet without the intervention of a third party, and in particular without the intervention of a dental care professional.

[0128] The device may further comprise communication means, in particular for sending the acquired images and / or receiving the model or reference images.

[0129] Of course, essential or optional features of the various main aspects of the present invention may be combined as long as they are not technically incompatible.

[0130] definition

[0131] A “user” is a person for whom the method according to the invention is carried out.

[0132] The term "dental care professional" refers to anyone qualified to provide dental care, including orthodontists and dentists.

[0133] "Arch" or "dental arch" refers to all or part of a dental arch, preferably including at least 2, preferably at least 3, more preferably at least 4 teeth. According to the international convention of the FDI World Dental Federation, each tooth in a dental arch has a predetermined number.

[0134] "Soft tissues" are those parts of the mouth that are covered by skin, such as the gums, palate, or tongue, opposite the teeth or braces.

[0135] Soft tissue is the extraskeletal supporting tissue such as adipose tissue, tendons, ligaments, fascia, skin, etc. (soft connective tissue) and muscle, blood vessels and nerve tissue (non-connective tissue).

[0136] A "retractor" is a device for rolling up the lips or more generally for moving the lips away from the teeth. The retractor preferably comprises upper and lower flanges and / or right and left flanges extending around the retractor opening and intended to be inserted between the teeth and the lips. In the operating position, the user's lips rest on these edges so that the teeth are visible through the retractor opening. Thus, the retractor allows the teeth to be viewed without being obstructed by the lips. However, the teeth do not rest on the retractor so that by turning the head relative to the retractor, the user can change the teeth visible through the retractor opening. The user can also change the spacing between their dental arches. In particular, the retractor does not press on the teeth to expand the upper and lower jaws apart, but presses on the lips. In one embodiment, the retractor is configured to elastically expand the upper and lower lips apart to expose the teeth visible through the retractor opening. In one embodiment, the retractor is configured so that the distance between the top edge and the bottom edge and / or between the right edge and the left edge is constant. Retractors are described, for example, in PCT / EP2015 / 074896, US 6,923,761 or US 2004 / 0209225.

[0137] "Computer" refers to a computer processing unit, which includes a group of several machines with computer processing capabilities. In particular, the unit may be integrated into a mobile phone, in particular a mobile phone of the user, or may be a PC-type computer or a server, such as a server remote from the user, such as a computer in the "cloud" or located at the dental care professional's premises. The mobile phone and the computer then have means for communicating with each other.

[0138] Typically, a computer includes a processor, a memory, a human-computer interface (usually including a screen), and a computer that can communicate with the Internet, WIFI, or a communication module of a telephone network. Software configured to implement the method of the present invention is loaded into the memory of the computer. The computer may also be connected to a printer.

[0139] The method according to the invention (except for the acquisition operation performed by the image acquisition device and the operation of moving the image acquisition device performed by the user) is implemented by a computer, preferably only by a computer.

[0140] The "real scene" consists of a set of elements observed simultaneously by the acquisition device. The view of the actual scene observed by the acquisition device is usually displayed on the screen of the acquisition device in the form of a "preview image" that is continuously updated in real time, like a movie.

[0141] The preview image may be replaced or supplemented by an equivalent image which symbolically or realistically represents a theoretical scene, which represents all or some elements of the real scene in the same arrangement as in the real scene, i.e. in such a way that the represented elements are arranged relative to each other as in the real scene. The equivalent image is a view of the theoretical scene under the same observation conditions as those used by the acquisition device to observe the real scene and obtain the preview image. Thus, the outlines of the representations of physical elements on the equivalent image can be superimposed on the outlines of said physical elements on the preview image. The equivalent image is selected to match the preview image as closely as possible.

[0142] The preview image is preferably displayed on the screen of the acquisition device, but in one embodiment, the equivalent image replaces the preview image on the screen. However, acquiring an image using the acquisition device at a given time includes recording the preview image when it is displayed, or recording the preview image when it will be displayed when the preview image is replaced by the equivalent image.

[0143] Augmented reality is a form of communication in which visual elements are added to an image to represent a scene either realistically or symbolically.

[0144] In particular, visual elements may be added to a preview image representing the actual scene observed by the acquisition device, or to an equivalent image.

[0145] In a preferred embodiment, augmented reality is used so that when an image representing the target is acquired in step a), the screen represents the target as viewed by the image acquisition device in real time, either literally or symbolically.

[0146] In augmented reality, a symbol is "anchored" when it appears fixed in the real or theoretical scene while the acquisition device is moved relative to the real scene (and thus modifying the image displayed by the acquisition device).

[0147] The symbols appearing in the augmented reality can be points, two-dimensional, i.e. extending in a plane in the space of the object in the mouth, or preferably three-dimensional, i.e. extending virtually in all three dimensions of the space of the object in the mouth. How a two-dimensional or three-dimensional symbol is represented depends on the conditions under which the two-dimensional or three-dimensional symbol is virtually observed. For example, the size of the symbol depends on the observation distance. Preferably, the symbol is not spherical so that its representation also provides information about the angle and / or orientation of the acquisition device around its optical axis. Symbols providing guiding information are preferably displayed on a preview image or equivalent.

[0148] The term "model" refers to a three-dimensional digital model. A model is made up of a set of voxels.

[0149] A "tooth model" is a three-dimensional digital model of teeth. The dental arch model can be cut to define a tooth model of at least some (preferably all) of the teeth represented in the dental arch model. Thus, the tooth model is a model within the dental arch model.

[0150] An "image" refers to a two-dimensional image, such as a photograph or a video frame. An image is made up of pixels. A "video" is considered to be a collection of photographs. The number of pixels in an image is preferably greater than 100, 1,000, 10,000 or 100,000 or 1,000,000 and / or less than 1,000,000,000.

[0151] An image represents a scene, realistically or otherwise.

[0152] Specifically, the image may represent a deformed mask resulting from a projection of the original mask (preferably by an acquisition device). The original mask may be, for example, a grid or a set of points that are generally uniformly distributed. The projection may be in the form of visible or invisible light, preferably in the form of infrared light. The deformation of a portion (e.g., a point) of the original mask resulting from the projection of the original mask carries information about the distance between the region of the scene onto which the portion of the original mask is projected and the image acquisition device. It may also provide information about the orientation of the region in space. The images used by Apple's Face ID software are examples of such images, also referred to as "3D images".

[0153] The image acquired according to the invention is preferably a photograph, possibly extracted from a video, which represents the observed scene realistically, i.e. as perceived by the human eye. The photograph may represent the distorted mask, in particular superimposed on an image realistically depicting the scene. The photograph may represent only the distorted mask. When the original mask is projected onto the real scene observed by the acquisition device, an image representing the deformed mask is also considered to be an image equivalent to the preview image.

[0154] A "match" or "fit" between two objects (e.g., between two images of a dental arch) is a measure of the difference or "distance" between them. A "best fit" is achieved when this difference is minimal, specifically when the two images represent the same elements in substantially the same way (i.e., in such a way that the representations of the elements on the two images are substantially superimposable in alignment).

[0155] The "acquisition conditions" of an image specify the spatial position and / or orientation of the image acquisition device of the image relative to the target, and preferably specify the calibration of the image acquisition device (specifically aperture, exposure time, focus and sensitivity). Symbols may indicate acquisition conditions that are suitable for a single position and orientation of the acquisition device. Symbols may alternatively indicate acquisition conditions corresponding to multiple positions and / or orientations of the acquisition device to guide towards several possible images. For example, the symbol guidance may be towards a predetermined observation axis, rather than guiding towards a specific position along the axis, thereby enabling the user to freely acquire one or more images along the axis at various positions along the axis, such as Figure 3 The symbol can be directed to a predetermined position in space, rather than to a specific orientation of the acquisition device around its optical axis, such as Figure 5 As in the embodiment shown.

[0156] The image acquisition device includes a camera for acquiring images (such as photographs or films). When reference is made to the observation of a scene by the acquisition device, reference is made to the observation of the scene by the camera of the acquisition device. When reference is made to the optical axis of the acquisition device, reference is made to the optical axis of the camera of the acquisition device, and so on.

[0157] The "angle" is the orientation of the optical axis of the acquisition device relative to the target.

[0158] The angle between two lines is the angle formed between two planes perpendicular to the two lines.

[0159] Unless otherwise indicated, "including" or "comprising" or "having" should be interpreted in a non-limiting manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0160] Other features and advantages of the present invention will become apparent from the following detailed description and from an examination of the accompanying drawings, in which:

[0161] [ Figure 1 ] Figure 1 schematically illustrates the steps of a cycle of a method according to a second main aspect of the invention;

[0162] [ Figure 2 ] Figure 2 An example device according to the present invention is shown;

[0163] [ Figure 3 ] Figure 3 An example of implementing the method according to the second main aspect of the invention in a first main embodiment is shown;

[0164] [ Figure 4 ] Figure 4 An example of a method according to the first main aspect of the invention and according to the second main aspect of the invention in a second main embodiment is shown;

[0165] [ Figure 5 ] Figure 5 Another example of a method according to the first main aspect of the invention and according to the second main aspect of the invention is shown in a second main embodiment;

[0166] [ Figure 6 ] Figure 6 shows a message from a user presented to the user when a predetermined acquisition condition associated with a symbol is met Figure 5 Symbols;

[0167] [ Figure 7 ] Figure 7 Another example of a method according to the first main aspect of the invention and according to the second main aspect of the invention is shown in a second main embodiment;

[0168] [ Figure 8 ] Figure 8 A schematic representation of a three-dimensional symbol viewed along its axis is shown;

[0169] [ Fig. 9 ] Fig. 9 schematically illustrates the steps of a cycle of a method according to a second main aspect of the invention;

[0170] [ Fig.10 ] Fig.10 Two examples of images equivalent to the preview image are shown. DETAILED DESCRIPTION

[0171] Figure 1 The acquisition method according to the invention shown is for example based on Figure 2 The device 1 shown is implemented.

[0172] Device 1 includes

[0173] an image acquisition device 10 provided with a screen 12 and into which the computer program is loaded, and

[0174] A computer 14 provided with a screen 18 .

[0175] The computer 14 may be separate from the acquisition device or, preferably, integrated into the acquisition device.

[0176] The computer 14 may also comprise digital communication means for exchanging data 20 , in particular with the image acquisition device 10 or even with a database 22 .

[0177] The database 22 may also be partially or completely integrated into the acquisition device or computer. In particular, the database may contain acquired images, reference models or reference images, definitions of targets and objects in the mouth, or even final models generated from acquired images. The database may also contain information about predetermined acquisition conditions associated with each symbol.

[0178] In a preferred embodiment of the present invention, the image acquisition device 10 is a mobile phone or a tablet computer. The screen 12 of the acquisition device is configured to present guidance information, and preferably presents information about the coverage level and / or about the difference between the coverage threshold and the coverage level. Alternatively or additionally, the information can be presented on a computer screen 18.

[0179] The image acquisition device can also be a mirror equipped with a camera.

[0180] Device 1 is used to implement the method according to the present invention:

[0181] The target has a predetermined "initial coverage area", which is the area that the method intends to cover.

[0182] The "covered area" is that part of the initial surface to be covered which, at a certain point during the method, has been covered, i.e. represented on at least one acquired image. The "area yet to be covered" is that part of the initial area to be covered which, at a given point during the method, has not yet been represented on any acquired image.

[0183] The region of the target may include several elementary regions of corresponding identifiable organs (eg, multiple teeth). An organ may be described as "covered" or "yet to be covered," depending on whether its entire elementary region is covered.

[0184] The object in the mouth may comprise or consist of a tongue and / or a palate and / or one or both gums and / or one or both dental arches and / or one or more teeth. Preferably, the object in the mouth is a dental arch.

[0185] The object in the mouth can also be an orthodontic appliance (e.g., a multi-attachment appliance, a vestibular or lingual orthodontic appliance), an orthodontic aligner (preferably not visible), an auxiliary appliance (e.g., a splint, button, or screw), a functional training appliance (e.g., to modify tongue positioning or treat sleep apnea).

[0186] The target may be an object in the mouth. The target may also be a region of interest of an object in the mouth, specifically an area that has been identified as an area at risk or an area to be monitored, such as an area that scanned poorly during a previous appointment with a dental care professional, or an area that has changed, for example, as part of orthodontic treatment or periodontal follow-up.

[0187] In a preferred embodiment, the object in the mouth is a dental arch and the target is one or preferably a plurality of teeth of said dental arch.

[0188] The target is identified before the method is implemented and the computer is informed. For example, the computer is told to acquire a set of images covering teeth 10 to 14, or is provided with an image or model of the dental arch on which the target representation has been identified.

[0189] The image or model used to identify the target may be generic, i.e. it may be used by multiple users. They may be selected from a database which can be accessed via digital communication. The generic model may be a training mould for orthodontic operations. Preferably, the generic model or generic image is selected such that it represents a target having a shape close to the user's target, which improves the accuracy of the method. If the target belongs to a dental arch, the model may be generated by implementing a tooth model arrangement procedure, for example as described in European application number 18 184486.

[0190] The model or image used to identify the target is preferably a model or image representing the target of the user acquired before the method is implemented.

[0191] Each area of ​​an object represented on an acquired image is said to be "covered" by that image. The union of the areas represented on at least one acquired image is the "covered area".

[0192] When the coverage level of the acquired images reaches the coverage threshold, the set of images is considered sufficient to cover the given target.

[0193] The coverage threshold thus directly or indirectly defines the percentage of the initial area to be covered which is considered sufficient to terminate the acquisition, ie sufficient to consider the acquisition complete.

[0194] For example, a coverage threshold of 100% means that the entire surface of the object must be represented on at least one of the acquired images.

[0195] The coverage threshold may be set so that the set of images is sufficient to view the target from a predetermined angle, specifically any angle.

[0196] The coverage threshold is preferably predetermined before the first step a).

[0197] The coverage threshold may be greater than 50%, 70%, 80%, 90% or 95% of the target area. Preferably, the coverage threshold is greater than 95%.

[0198] The coverage level is a measure of the progress of the acquisition relative to the coverage threshold. Thus, before acquiring the first image, the coverage level is zero. The progress level increases gradually with each cycle of steps a) to c).

[0199] For example, the coverage level may be the ratio of the covered area to the initial area to be covered. If 30% of the initial area to be covered is covered, the coverage level is therefore 30%.

[0200] The coverage level can also be, for example, the ratio of the area covered to the area yet to be covered, which area yet to be covered consists of all areas of the target not represented on any acquired image. If 30% of the initial area to be covered is covered, the coverage level is therefore 30% / 70%.

[0201] If the object in the mouth comprises a set of teeth, the coverage level may also be the ratio of the number of teeth covered to the number of teeth in the set. For example, a tooth may be considered to be covered if at least 90% of the outer surface or occlusal surface or vestibular surface of the tooth is represented on the acquired image, better if at least 95% of the tooth surface is represented on the acquired image, even better if the entirety of the tooth surface is represented on the acquired image.

[0202] A tooth may alternatively be considered covered when one or more salient points of the tooth are represented on at least one acquired image.

[0203] In the above embodiments, it has been assumed that the purpose is to cover the area of ​​the target.

[0204] By extension, in one embodiment, the goal is to acquire a set of images under corresponding acquisition conditions defined by corresponding symbols, preferably multidimensional symbols. Then, at the beginning of the implementation of the method according to the invention, a set of symbols that each define the acquisition conditions for one or more corresponding images can be assimilated as an "initial area to be covered". A set of symbols that define the acquisition conditions under which images have been acquired at one moment can be considered as a "covered area", the number of these symbols defines a "coverage level", and the "coverage threshold" can be the minimum number of these symbols. Finally, a set of symbols that define the acquisition conditions under which images remain to be acquired can be considered as an "area yet to be covered".

[0205] Also by extension, in one embodiment, the goal is to acquire a set of images that partially represent the target, for example representing for each tooth of the target a salient point, such as a mesio-distal point, a cusp, a free edge, a point along the tooth neck, or the center of gravity of the tooth face. At the beginning of the method according to the invention, a set of salient points on the target can then be assimilated as an "initial surface to be covered". A set of salient points of the target that are already represented on the acquired images can be considered as a "covered area", the number of these points defining a "coverage level", the "coverage threshold" being the minimum number of these points. Finally, the set of salient points that have not yet been represented on the already acquired images can be considered as an "area yet to be covered".

[0206] One objective of the method is to guide the user during image acquisition so that a set of acquired images includes as few images as possible, ie the acquisition is efficient, but with enough images so that a coverage threshold is reached.

[0207] A first main aspect of the invention is to guide acquisition by means of multi-dimensional symbols. A second main aspect of the invention is to guide acquisition by informing the user of the progress of the acquisition.

[0208] The first main aspect of the present invention

[0209] exist Fig. 9 In step 1) of the method according to the first main aspect of the invention shown, a multidimensional (i.e. two-dimensional or three-dimensional, preferably three-dimensional) symbol is arranged, i.e. "anchored" and presented to the user in the space of the object in the mouth in augmented reality, and the shape and / or position of the symbol is determined so as to indicate to the user all or part of the acquisition conditions suitable for acquiring the image.

[0210] Of course, the user must be able to interpret the symbols. The information associated with the symbols may be explicit, for example when it takes the form of an arrow. No training is required. Alternatively, the user may be trained so as to provide him with the "rules of the game", i.e. how he is expected to handle the symbols.

[0211] The two-dimensional or three-dimensional symbol appearing in augmented reality extends virtually (i.e. without physical presence) into the real scene observed by the acquisition device, or into a model of at least a part of an equivalent virtual scene, in particular a real scene. The symbol extends in a plane or defines a volume, respectively.

[0212] like Figure 7 As shown, representation in an equivalent virtual scene is advantageous.

[0213] The screen displays a preview image and / or an equivalent image representing the actual scene observed by the acquisition device, specifically a view of the model corresponding to the observation of the actual scene by the acquisition device. The screen also displays symbols as if they had real physical existence and were present in the actual scene.

[0214] Each symbol is associated with one or more acquisition conditions of the corresponding image, specifically

[0215] - a predetermined viewing axis of the object or a predetermined range of viewing axes, and / or

[0216] - a predetermined distance between the target and the acquisition device, or a range of said predetermined distances, and / or

[0217] - A predetermined orientation of the acquisition device around its optical axis, or a range of said predetermined orientations.

[0218] The computer knows the predetermined viewing conditions associated with each symbol.

[0219] The number of symbols is adapted to the number of images in the desired set of images. It is preferably greater than 2, 5, 10 or 100 and / or less than 1,000.

[0220] The symbol (preferably each symbol) is preferably represented in a preview image or equivalent, which is updated in real time as the user moves the acquisition device.

[0221] The symbol, preferably each symbol, is preferably represented in augmented reality on the screen of the acquisition device or communicated with the acquisition device, preferably on the screen of the mobile phone used to acquire the image.

[0222] A symbol (preferably each symbol) is preferably anchored and / or shaped in such a way that the modification of the representation of that symbol on said screen resulting from movement of the acquisition means informs the user of the fact that said movement is away from or towards a predetermined acquisition condition associated with said symbol.

[0223] The predetermined acquisition condition defines a viewing axis of the acquisition device on the target, i.e. a predetermined viewing axis, and / or a predetermined distance of the acquisition device from the target, preferably along the viewing axis, and / or a predetermined orientation of the acquisition device about its optical axis, and

[0224] The modification of the representation of the symbol on the screen preferably informs the user of the fact that the mobile

[0225] - reducing or increasing the angular difference between the optical axis of the acquisition device and the predetermined observation axis, ie improving or deteriorating the angle of the acquisition device; and / or

[0226] - reducing or increasing the difference between the distance between the acquisition device and the target and the predetermined distance; and / or

[0227] - reducing or increasing the difference between the orientation of the acquisition device about its optical axis and a predetermined orientation;

[0228] Each of the three differences mentioned above is preferably reduced or increased.

[0229] The principle of augmented reality is well known. For those skilled in the art, there is no particular difficulty in determining the angle and / or distance and / or orientation difference.

[0230] Preferably, the multi-dimensional symbol (preferably each multi-dimensional symbol) has a shape defining a primary direction or "symbol axis" (eg an axis of rotation) identifiable by a user and indicating a predetermined said viewing axis associated with said symbol.

[0231] When the optical axis of the acquisition device coincides with said main direction, the multidimensional symbol (preferably each multidimensional symbol) has a size which is variable in the representation of the symbol on the screen as a function of the position of the acquisition device along the optical axis (i.e. as a function of the distance between the acquisition device and the symbol).

[0232] Said dimension can be assessed by the user observing the screen and the user knows the value of said dimension defining the position of the acquisition means at a predetermined distance associated with said symbol.

[0233] For example, the multidimensional symbol takes the form of a superposition of rings (preferably of different diameters),

[0234] - a predetermined orientation is obtained when the center of the ring is aligned along the optical axis of the acquisition device,

[0235] That is, the centers are displayed as merged on the screen, and / or

[0236] - A predetermined distance is obtained when the centers of the rings are aligned along the optical axis and the spacing between the rings has a predetermined value when these rings appear on the screen (preferably on a representation of the symbol in a preview image or an equivalent image) (for example when a first ring appears adjacent to a second ring, i.e. when the inner contour of the first ring is in contact with the outer contour of the second ring).

[0237] In step 2), an image is acquired when a predetermined acquisition condition associated with the symbol is met.

[0238] Specifically, the acquisition may be implemented in step a) as described below. The image may be of the type described below for step a).

[0239] The image is then analyzed to determine the acquisition conditions.

[0240] In one embodiment, each acquired image undergoes the following steps:

[0241] A first neural network is trained to detect representations of objects in an image and / or salient points of said objects, and

[0242] The second neural network is trained to recognize the acquisition condition from the representation of the object or the salient point in the image, and in particular the angle and / or distance of the acquisition device to the object.

[0243] In the paragraphs relating to step b2), the first neural network may be specifically selected from an object detection network, and in particular selected from the neural networks listed below. For example, the neural network is trained by presenting 1,000 historical images to the neural network:

[0244] As input, historical images representing historical objects and / or salient points, and

[0245] As output, a representation of said historical objects and / or salient points on the historical image.

[0246] The neural network thus learns to recognize representations of objects and / or salient points in new images.

[0247] The second neural network can specifically be selected from networks called "CNN" ("Convolutional Neural Network") specifically used for image classification, such as AlexNet (2012), ZF Net (2013), VGG Net (2014), GoogleNet (2015), Microsoft ResNet (2015), Cach: BAIR Reference CaffeNet, BAIR AlexNet, Torch: VGG_CNN_S, VGG_CNN_M, VGG_CNN_M_2048, VGG_CNN_M_1024, VGG_CNN_M_128, VGG_CNN_F, VGGILSVRC-2014 16 layers, VGG ILSVRC-2014 19 layers, Network in Network (Imagenet & CIFAR-10), Google: Inception (V3, V4). For example, for more than 1,000 historical images, a neural network is trained by presenting these historical images to the neural network: as input, historical images representing historical objects and / or salient points, and

[0248] As output, the “historical” acquisition conditions of the historical image.

[0249] The neural network thus learns to condition its acquisition for new images.

[0250] Determining the conditions for acquiring the image can also be achieved by searching for views of the model of the dental arch of the user corresponding to the image, for example using an optimization operation, preferably using a metaheuristic method, preferably using an evolutionary method, preferably using simulated annealing. Examples of such searches are described, for example, in PCT / EP2015 / 074859, in European patent application No. 18 184477.0 or in WO2016 / 066651.

[0251] Preferably, when the symbol is observed along a predetermined observation axis, and / or when the acquisition device is at said predetermined distance, and / or when the acquisition device is oriented according to said predetermined orientation,

[0252] Preferably, when the symbol is viewed along a predetermined viewing axis, and when the acquisition device is at said predetermined distance, and when the acquisition device is oriented according to said predetermined orientation,

[0253] The images are preferably acquired automatically, ie without specific user intervention.

[0254] Preferably, the symbol changes appearance (eg color) or disappears when an image is acquired under acquisition conditions associated with the symbol.

[0255] The second main aspect of the present invention

[0256] The method according to the second main aspect of the invention comprises a plurality of cycles of steps a) to c).

[0257] In step b), an image, preferably a photograph, depicting the object in the user's mouth is acquired using an image acquisition device. In one embodiment, a video is acquired using the image acquisition device, and the acquired image is extracted from the video.

[0258] In one embodiment, in step a), a "raw mask" (preferably a point cloud) is projected onto the scene observed by the acquisition device, preferably with the aid of a projector integrated into the acquisition device. The distorted mask resulting from the projection of the raw mask then appears on the preview or equivalent image. In one embodiment, the projection is in infrared light, so that the deformed mask is invisible to the naked eye. In one embodiment, the acquired image is an image representing the deformed mask. The acquisition device preferably uses an infrared camera. However, the nature of the deformed mask is not limited.

[0259] The image is preferably acquired by the user himself.The user may use a mobile phone to acquire the image.

[0260] The images acquired are preferably "extra-oral", ie the lens of the acquisition device is not inserted into the user's mouth.

[0261] The image acquisition device can specifically be a mobile phone, a tablet, a camera or a computer, and the image acquisition device is preferably a mobile phone or a tablet, specifically allowing the user to acquire images anywhere, and specifically outside the dental care professional's clinic, for example more than 1 km away from the dental care professional's clinic.

[0262] In one embodiment, the user uses a mobile phone and a holder to which the mobile phone is removably attached, which rests against the user during the acquisition of at least some of these images, preferably all of the images. In particular, the holder may be of the type described in PCT / EP2021 / 068702, EP17306361, PCT / EP2019 / 079565, PCT / EP2022 / 053847, FR2113577, FR2206750 or FR2206745.

[0263] In a preferred embodiment, the user uses a free mobile phone, i.e. a mobile phone whose position and orientation can be determined freely and in particular a mobile phone which is not attached to a holder. In fact, the method described in the invention enables the user to be guided when taking the image, so that guidance by means of a holder is not necessary. Preferably, the image acquisition device does not come into contact with the user's mouth, either directly or via a holder for the image acquisition device.

[0264] In advance, depending on the target, the computer or acquisition device may ask the user to put orthodontic appliances such as orthodontic aligners, splints or archwire appliances into use or out of use. Alternatively, the user may be asked to move his lips away from his dental arch, preferably using a retractor, in order to better expose the target to the image acquisition device, such as fully exposing at least one tooth, in particular the outer surface of the incisors and / or at least partially the outer surface of the molars. Alternatively, the user may be asked to open his mouth to acquire an occlusal image showing the lingual surfaces and occlusal surfaces of the teeth and palate.

[0265] The number of images acquired in step a) is preferably less than 100, preferably less than 50, most preferably less than 10, so that the guidance information can be updated quickly.

[0266] Before carrying out the first step a) of the method according to the invention, the coverage level is zero.

[0267] In step b), the coverage level is updated to take into account the image acquired in the immediately preceding step a), or the "new image".

[0268] In step b), the computer preferably analyses each new image according to the following steps:

[0269] b1) determine the potential contribution of new images;

[0270] b2) determining the intersection between the potential contribution and previous contributions from previously analyzed images;

[0271] b3) If the intersection is not empty, add the potential contribution to the previous contribution.

[0272] In step b1), the computer determines the potential contribution of the new image. Specifically, it determines whether the new image at least partially represents the target. If not, the new image cannot contribute, and the computer continues to analyze the next new image. If yes, the computer determines the potential contribution of the new image, for example determining the outline of the target representation on the new image, or the number of one or more target teeth at least partially represented in the new image.

[0273] In step b2), the computer then compares the potential contribution to a set of contributions or "previous contributions" derived from analysis of previously analyzed images.

[0274] For example, the potential contribution of a new image is the representation of the object in the new image. The computer evaluates the intersection between this potential contribution and the previous contribution, which consists of the union of all representations of the object on the previously analyzed images.

[0275] If the intersection is empty, the new image cannot make a new contribution and the computer continues to analyze the next new image. If the intersection is not empty, i.e. the new image represents an area of ​​the target that is not in any previously analyzed image, the computer adds the new contribution consisting of the intersection to the previous contributions.

[0276] In another example, the potential contribution of the new image is the number of one or more target teeth identified in the new image. The computer evaluates the intersection between the potential contribution and the previous contribution, which consists of a set of tooth numbers of the target identified in the previously analyzed image. If the intersection is empty, the new image cannot make an additional contribution, and the computer continues to analyze the next new image. If the intersection is not empty, the computer adds the additional contribution to the previous contribution.

[0277] The potential contribution of a new image may be determined by any known means.

[0278] In one embodiment, this involves segmenting the new image in order to identify any full or partial representations of the object. Conventional segmentation methods may be used for the analysis.

[0279] In particular, a new image may be submitted to a neural network which is trained to detect the representation of objects on the new image, for example to determine the number of teeth represented in the image, and / or the contours of said teeth, and / or of the mouth and / or of the lips, and / or of the tongue, as described, for example, in European patent application No. 18 184477.0.

[0280] Networks specialized for localizing and detecting objects in images are well known.The neural network may specifically be selected from object detection networks, such as R-CNN (2013), SSD (Single Shot Multibox Detector: Object Detection Network), Faster R-CNN (Faster Region-based Convolutional Network Approach: Object Detection Network), Faster R-CNN (2015), SSD (2015), RCF (Richer Convolutional Features for Edge Detection) (2017), SPP-Net (2014), OverFeat (Sermanet et al.) (2013), GoogleNet (Szegedy et al.) (2015), VGGNet (Simonyan and Zisserman) (2014), R-CNN (Girshick et al.) (2014), Fast R-CNN (Girshick et al.) (2015), ResNet (He et al.) (2016), Faster R-CNN (Ren et al.) (2016), FPN (Lin et al.) (2016), YOLO (Redmon et al.) (2016), SSD (Liu et al.) (2016), ResNet v2 (He et al.) (2016), R-FCN (Dai et al.) (2016), ResNeXt (Lin et al.) (2017), DenseNet (Huang et al.) (2017), DPN (Chen et al.) (2017), YOL09000 (Redmon and Farhadi) (2017), Hourglass (Newell et al.) (2016), MobileNet (Howard et al.) (2017), DCN (Dai et al.) (2017), RetinaNet (Lin et al.) (2017), Mask R-CNN (He et al.) (2017), RefineDet (Zhang et al.) (2018), CascadeRCNN (Cai et al.) (2018), NASNet (Zoph et al.) (2019), CornerNet (Law and Deng) (2018), FSAF (Zhu et al.) (2019), SENet (Hu et al.) (2018), ExtremeNet (Zhou et al.) (2019), NAS-FPN (Ghiasi et al.) (2019), Detnas (Chen et al.) (2019), FCOS (Tian et al.) (2019), CenterNet (Duan et al.) (2019), EfficientNet (Tan and Le) (2019), or AlexNet (Krizhevsky et al.) (2012).

[0281] The above list is not exhaustive. For example, a neural network is trained by presenting it with 1,000 historical images:

[0282] As input, historical images representing historical targets, and

[0283] As output, a representation of said historical target on a historical image.

[0284] The neural network thus learns representations that recognize objects in new images.

[0285] When the intersection between potential contributions and previous contributions comprises a comparison of representations of targets in the new image and in previously analyzed images, these representations are preferably projected onto a common reference model in order to take into account different acquisition conditions and in particular the orientation of the acquisition device that varies depending on the image considered.

[0286] The reference model preferably represents a reference object in the mouth, preferably similar or even identical to the object in the user's mouth.

[0287] To project the image, the computer analyzes the image to determine its acquisition conditions, i.e., the actual acquisition conditions. Specifically, the computer evaluates the distance between the acquisition device and the object in the user's mouth and the orientation of the acquisition device in space relative to the object in the user's mouth at the time of image acquisition.

[0288] The determination of the actual acquisition conditions may be implemented as described, for example, in European patent application No. 18 184477.0 or in WO 2016 / 066651, or by subjecting the image to a neural network trained to determine the acquisition conditions of the image submitted thereto.

[0289] The actual acquisition conditions are then virtually reproduced relative to the reference model and the target representation of the user in the image is projected onto the reference model.

[0290] A set of projected surfaces obtained from previous images may constitute a previous contribution. A projected surface obtained from a new image is a potential contribution.

[0291] Preferably, each image is submitted to a first detection neural network that identifies objects in the mouth represented in the image, in particular the tongue and / or the number of teeth, and / or the gums and / or the mouth and / or the lips and / or the salient points of these organs, and then each image is submitted to a second neural network that is trained to determine the acquisition conditions of the images submitted thereto. Preferably, the second neural network takes as input the objects in the mouth detected by the first neural network, which improves the determination of the acquisition conditions.

[0292] In one embodiment, the potential contribution of a new image is determined by comparing the image to a "reference" image, (eg, a photograph or panoramic photo of at least one dental arch that is preferably similar or identical to the user's dental arch).

[0293] The object is then typically not represented in the same way on the reference image and on the new image. Preferably, the neural network is trained to match objects represented in both images.

[0294] For example, the neural network is trained by presenting it with the following:

[0295] As input, historical records, each of which includes a historical image representing a historical target under possible acquisition conditions using an acquisition device and an identification of a representation of the historical target in the historical image, and

[0296] As output, an identification of the historical target on the historical reference image.

[0297] The neural network thus learns to recognize representations of objects in the reference image that correspond to representations of objects in new images submitted to the neural network. The neural network thus learns to identify the potential contributions of new images.

[0298] This method can be repeated several times, each time using a new reference image.

[0299] In step b3), if the intersection is not empty, the computer adds the potential contribution to the previous contribution and calculates the coverage level resulting from the increment of the new contribution to the previous contribution.

[0300] In a preferred embodiment, the computer evaluates the quality of the acquired image and adds the potential contribution in step b3) only if the quality is above a predefined quality threshold. In particular, the quality may be an evaluation of the sharpness and / or contrast and / or color balance of the image, and / or the distance between the acquisition device and the user's mouth. Advantageously, only images of satisfactory quality are considered.

[0301] In one embodiment, the image acquisition device, preferably in the form of a mobile phone, is attached to a holder that remains in contact with the user during acquisition, as described above, for example a holder of the type described in PCT / EP2021 / 068702, EP17306361, PCT / EP2019 / 079565, PCT / EP2022 / 053847, FR2113577, FR2206750 or FR2206745. With such a holder, the image quality (in particular brightness, distance of the acquisition device from the target, angle of the acquisition device relative to the target and orientation of the acquisition device about its optical axis) is advantageously well controlled, so that an assessment of the quality of the acquired image or images is optional.

[0302] Preferably, the retainer is not bitten by the user, and even more preferably, allows opening and closing of the dental arches (to a position where the dental arches contact each other in the occlusal plane).

[0303] Quality assessment is particularly advantageous in the absence of such retainers.

[0304] Depending on the results, the computer may decide to take several images, for example changing the focus to obtain a first clear image of the front set of teeth, and then a second clear image of the back set of teeth.

[0305] The acquisition of several images with different calibration conditions may be based on the quality assessment, but may also be programmed to be routine at each acquisition stage.

[0306] In step c), the computer compares the coverage level to a coverage threshold. If the coverage level is greater than or equal to the coverage threshold, step c) is completed. Alternatively, the computer determines guidance information to guide the user to position the image acquisition device towards a "future" acquisition condition suitable for acquiring additional images that increase the coverage level in the next step a).

[0307] Therefore, guidance information is determined to inform the user of the area of ​​the target for which he still needs to acquire one or more images. The guidance information is presented to the user, guiding him to orient and / or position the image acquisition device according to the future acquisition conditions adopted for the next step a).

[0308] In a preferred embodiment, the computer determines future acquisition conditions for the next cycle, such as by random search or using an optimization algorithm, such that image acquisition under these future acquisition conditions maximizes the increase in coverage level.

[0309] The presentation of the guidance information may include visual, auditory and / or tactile, particularly tactile, transmission of the guidance information.

[0310] For example, the guidance information may include audio instructions telling the user to move the image acquisition device toward or away from the teeth, shift the image acquisition device to the right or left, rotate the image acquisition device around the dental arch, open or close the mouth, or open or close the jaw.

[0311] The tactile transmission of the guidance information may be, for example, a vibration that instructs the user to stop moving.

[0312] The tactile transmission of the guidance information may be a vibration, for example to indicate successful passage of a reference mark over the surface of the target or symbol.

[0313] The presentation of guidance information may be adapted to the user.

[0314] Preferably, the presentation of guidance information includes several different types of transmissions that stimulate several different senses to facilitate communication with the user.

[0315] Preferably, the guidance information is presented on a screen, preferably on a screen of the image acquisition device. Displaying the guidance information on the screen may include

[0316] - preferably a reference frame which enables the user to find his orientation when moving the acquisition device in space, i.e. to be able to assess how the acquisition device is positioned relative to the object in the mouth and in particular relative to the target, and

[0317] - an indicator showing the movement relative to the reference frame.

[0318] The reference frame preferably represents at least one object in the mouth as viewed by the acquisition device, at least partially in a symbolic or real manner adapted to enable the user to recognize the represented object. It is determined according to actual acquisition conditions of the acquisition device.

[0319] The reference frame can be, for example, a preview image, i.e. an image observed in real time by an acquisition device (preferably a mobile phone) and displayed on a screen of the acquisition device, and / or an equivalent image representing a part of a user (e.g. the user's head or a part of the head or the user's mouth or dental arch) ("equivalent" means that the image corresponds to an observation of a part of the patient that can be superimposed with the image observed by the acquisition device (and specifically observed along the optical axis of the acquisition device)).

[0320] The equivalent image may be, for example, a line drawing representing the outline of a portion of the user.

[0321] The reference frame may represent a view of a user-specific model or a generic model, which model thus represents exactly or more roughly a part of the user, preferably at least a target, preferably at least an object in the mouth.

[0322] The universal model is common to multiple individuals. Specifically, the universal reference frame can be determined by statistical analysis of historical data representing these individuals. The universal model can be, for example, an orthodontic operation training mold model.

[0323] The user-specific model may be a model of all or part of an object in the user's mouth (specifically, the user's target). Specifically, the user-specific model may be a scan of the user's dental arch. The user-specific model may also include or may be a 3D model of the user's dental arch in a configuration specific to a treatment stage. Specifically, the user-specific model may include or may be a 3D model of the user's dental arch in a configuration suitable for a treatment stage utilizing orthodontic aligners, and specifically a 3D model for the design and manufacture of orthodontic aligners. Such a 3D model may be generated at the beginning of orthodontic treatment or during orthodontic treatment.

[0324] The equivalent image is preferably a view of a generic or specific model. Preferably, a texture is applied to the model to make it more realistic and to make it easier for the user to identify with the model. The texture may be extracted from an image (e.g. the image obtained in step a)) and then applied to the model, which is preferably selected from a database before step a).

[0325] As mentioned above, the model for which the view is used as reference may specifically be a reference model used as a projection medium for the acquired image.

[0326] The equivalent image may be at least partially symbolic. The equivalent image may comprise, for example, a set of geometric shapes representing objects in the mouth, such as a set of disks, each disk representing a tooth of a portion of a dental arch, the objects in the mouth being the dental arch.

[0327] Fig.10 Two examples of equivalent images are shown, representing views of a 3D model of a user's dental arch with and without gums, respectively. The views may also be, for example, wireframe representations.

[0328] In particular, when the equivalent image is a view of a general or specific model, the image acquisition device may preferably display the preview image in a "thumbnail" window, which facilitates the user's spatial positioning.

[0329] Displaying the reference frame on the screen is optional if the indicator provides an indication of the desired movement. For example, the indicator may be an arrow or message recommending a particular movement. However, displaying the reference frame on the screen is preferred because it is quite convenient to obtain the precise positioning of the device.

[0330] The indicator is preferably displayed together with the reference frame on a screen, preferably on a screen of an image acquisition device, preferably on a mobile phone or tablet screen.

[0331] In a preferred embodiment, the reference frame includes a representation of the object in the mouth, and the indicator is a mark indicating an area of ​​the representation that has not been covered. For example, the indicator can be a special outline around the area, or preferably a special color applied to the area, or a symbol superimposed on the area. A "special" outline or color is one that enables the user to distinguish the area from the rest of the representation of the object in the mouth.

[0332] This display helps guide the user quickly and efficiently. This guidance system, which gives the user a lot of freedom, is intuitive and therefore does not require guidance pre-training.

[0333] In particular, the indicator may be displayed transparently or highlighted on the representation of the object in the mouth.

[0334] When the reference frame is a preview image or equivalent, an indicator is preferably displayed in augmented reality.

[0335] Coverage and Thresholds

[0336] Preferably, in step c), the user is informed, preferably in real time, of the achieved coverage level, ie the progress of the acquisition, and preferably of the coverage threshold.

[0337] Preferably, the coverage level information and / or coverage threshold information is presented on a screen, preferably on a mobile phone screen of the user. The information may take the form of a meter or gauge, for example in the form of a progress bar.

[0338] In a preferred embodiment, the coverage level and / or the coverage threshold is represented "graphically" on the screen, in particular in the form of lines and / or areas and / or symbols.

[0339] In one embodiment, the screen displays:

[0340] - the initial area to be covered,

[0341] - the area covered, and

[0342] - Preferably, a representation of elements of the target situation, such as parts of the object in the mouth other than the target, ie parts of the object in the mouth which the method is not intended to cover, such as teeth adjacent to the teeth for which images are to be acquired.

[0343] For example, when the reference frame is a symbolic or preferably real representation of an object or target in the mouth, the initial area to be covered can be presented on the screen in a manner that can be identified by the user to inform him of the coverage threshold. Specifically, the area can be colored with a specific color, or more generally represented with a specific appearance, or delimited by a specific outline. The coverage threshold is thus represented by the area represented with a specific appearance or surrounded by this outline.

[0344] Similarly, the covered area (i.e. the area for which at least one image has been acquired) can be presented on the screen in a manner identifiable by the user, in order to inform him of the level of coverage. Specifically, the area can be colored with a specific color, or more generally represented with a specific appearance, or bounded by a specific outline. The area represented with a specific appearance or surrounded by this outline thus represents the level of coverage.

[0345] For example, the surface of the target may be displayed in green or red, depending on whether acquisition of the surface has been completed or whether acquisition of the surface is still to be performed. Covered areas may be displayed transparently or highlighted.

[0346] In one embodiment, the coverage threshold is graphically represented as a set of symbols that are displayed in close proximity, preferably superimposed on a representation of a corresponding set of teeth. The representation of these teeth belongs to or even constitutes a reference frame. For example, the symbols may be presented in augmented reality on a preview image of a mobile phone or on a view of a dental arch model, preferably on a view of a model of the user's dental arch. The appearance of the symbols related to teeth for which the desired images have been acquired ("covered teeth") may be different from the appearance of the symbols related to teeth for which not all the desired images have been acquired ("teeth yet to be covered"), so that the coverage can be seen graphically.

[0347] In one embodiment, the teeth symbols disappear as soon as the teeth are covered. The user then sees the difference between the coverage threshold (all symbols initially displayed) and the coverage level (symbols that have disappeared).

[0348] Coverage thresholds and "graphical" displays of coverage levels are particularly effective in ensuring that the user acquires all required images.

[0349] It is worth noting that the graphical representations of the coverage threshold and coverage level show the areas of the target that are still to be covered, i.e., the desired images are yet to be acquired for these areas. These graphical representations can be used as indicators to guide the user. For example, coloring the covered areas with a color different from the areas that are yet to be covered highlights the areas that are yet to be covered and thus guides the user.

[0350] The graphic or "visual" marking of the initial area to be covered, the covered area or the area still to be covered is not limited to the application of colours or textures or outlines or the representation of specific symbols.

[0351] Preferably, the coverage threshold and the graphical representation of the coverage level are displayed in augmented reality, preferably on a mobile phone screen.

[0352] In one embodiment, a stopwatch is activated to measure the duration of the image acquisition since the first step a). Displaying this time and the coverage level and / or the difference between the coverage threshold and the coverage level is a motivating factor for the user.

[0353] In one embodiment, a score is calculated based on the time taken to reach a coverage threshold and / or the quality of the acquired images and / or the usefulness of the acquired images, more generally a goal set by the user.

[0354] In one embodiment, the initial area to be covered includes areas to which utility coefficients are assigned, and the score is determined based on the utility coefficients of areas in the covered area. For example, in one embodiment, the initial surface to be covered may consist of a portion that is essential for coverage and a portion that is optional for coverage. For example, the utility coefficient assigned to pixels in the "essential" segment may be 100, and, for example, the utility coefficient assigned to pixels in the "optional" segment may be 10. For example, the score may be a function, or even the sum of the utility coefficients of all pixels in the covered area.

[0355] The score may be compared to scores previously obtained by the user or other users to obtain a ranking for the set of image acquisition operations.

[0356] A ranking can be established for several patients (eg for all patients of the same physician). Information messages and / or gifts (such as rewards) can be sent to the patient according to his ranking order.

[0357] A stopwatch and / or scores and / or rankings may be displayed on the acquisition device screen.

[0358] This makes acquisition interesting. Specifically, acquisition can be presented as a video game, where the goal is to reach the coverage threshold as quickly as possible.

[0359] Guided by the guidance information and motivated by the coverage level information, the user modifies the acquisition conditions so that step a) can be restarted anew, preferably immediately at the end of step c).

[0360] When the user is presented with the level of coverage, in particular when the target becomes colored or when the symbols associated with the teeth change appearance or disappear as the image acquisition progresses, he can immediately visualize the effect of moving the acquisition device. Guidance is highly intuitive.

[0361] In a preferred embodiment, the screen displays a realistic representation of the object in the mouth and the surface covered at that time. The area covered is completed as the cycle progresses, allowing the user to easily identify the area still to be acquired and to position and orient the image acquisition device accordingly.

[0362] The time interval between two consecutive cycles of steps a) to c) is preferably less than 5 minutes, 1 minute, 30 seconds or 1 second. Preferably, the user acquires these acquired images in real time, preferably by photographing the object in the mouth, steps b) to c) being immediately performed for each acquired image.

[0363] At the end of the cycle of steps a) to c), ie when the coverage level is greater than or equal to the coverage threshold, the acquired image may be transmitted to the user and / or preferably to a dental care professional.

[0364] The acquired images may be stored in, for example, a database, preferably accessible by a dental care professional and / or a user. For example, the acquired images may be stored in a user's medical file.

[0365] A set of acquired images typically includes more than 2, more than 5, more than 10, more than 50, more than 100, and / or less than 10,000 images.

[0366] The set of acquired images may in particular be used for:

[0367] -Evaluate the user's dental condition before, during or after orthodontic or dental treatment; and

[0368] / or

[0369] - generating or correcting a model of the user's dental arch; and / or

[0370] -Design of orthodontic appliances, and / or

[0371] - Design and recommend dental treatments such as hygiene, periodontitis or whitening.

[0372] In a particular embodiment, at the end of the cycle of steps a) to c), the method may involve analyzing the acquired images to generate a model of the target, referred to as the "final model". The final model may be sent to a user, preferably a dental care professional. The final model may depict the target with high accuracy. The final model may be generated by a dental care professional's computer to which the acquired images are transferred, or by an image acquisition device. The final model may be stored in, for example, a database, preferably accessible to a dental care professional and / or a user. For example, the final model may be stored in a user's medical file.

[0373] Example

[0374] The purpose of the acquisition is to acquire a set of images of a target consisting of, for example, the teeth of all users. The dental arches constituting the objects in the mouth include teeth and gums. The objects in the mouth are modeled in the form of a reference model 16 accessible to a computer. The reference model 16 may come from a database and may be generic. The objects are identified on the reference model. Thus, the initial area to be covered is the target area in the reference model.

[0375] Figure 3 , Figure 5 , Figure 6 and Figure 7 An example of the first main aspect of the invention is shown, in which multi-dimensional symbols 24 are virtually arranged in space to guide the user to the associated acquisition conditions.

[0376] exist Figure 3 In the example shown, the symbols 24 are two-dimensional. These symbols each take the form of three concentric, coplanar rings. These symbols are represented on a view representing a reference model of the dental arch, the view observing the model when the acquisition device observes the dental arch of the user. This view is preferably presented on the screen of the phone used for acquisition. The user's goal is to position the phone so as to see the target front, that is, so that the rings appear circular. The optical axis then coincides with a predetermined viewing axis suitable for image acquisition.

[0377] Such guidance does not require reference marks. However, reference marks representing, for example, a circular outline of a ring when viewed from the front will improve the positioning accuracy of the acquisition device.

[0378] exist Figure 5 In the example shown, the symbols are three-dimensional. Each of these symbols takes the form of three concentric, superimposed rings. These symbols are represented on a view of a reference model representing the dental arch, the view viewing the model when the acquisition device views the dental arch of the user. This view is preferably presented on the screen of the phone used for acquisition. The user's goal is to position the phone so as to see the target front, i.e. so that the rings appear circular and concentric, as Figure 6 The optical axis is then coincident with a predetermined viewing axis suitable for image acquisition (ie the X-axis of the ring).

[0379] exist Figure 7In the example shown, each symbol 24 consists of two concentric circles extending in planes at 90° to each other. The two circles preferably have the same shape (same inner and outer diameters) and are preferably enclosed in a sphere 26 of color, but transparent to make them easier to locate. The intersection of these two planes defines a predetermined viewing axis X associated with the symbol that can be identified by the user. This form of symbol makes it very easy to determine how to move the acquisition device.

[0380] exist Figure 7 In the embodiment shown, a first horizontal loop is used to determine whether the acquisition device should move up or down, and a second vertical loop in a plane passing through the center of the mouth is used to determine whether the acquisition device should move to the right or left.

[0381] Guidance using three-dimensional symbols is particularly effective.No reference markers are required to precisely position the optical axis of the acquisition device along a predetermined viewing axis suitable for image acquisition.

[0382] Three-dimensional symbols are also used to guide to predetermined distances along predetermined viewing axes. Figure 8 In the example shown in , the distance d between the rings represents the distance between the acquisition device and the symbol along the X-axis of the symbol (i.e. along a predetermined viewing axis). This predetermined distance may correspond, for example, to a view of the symbol in which the rings touch (d=0). The user easily moves the acquisition device to this position.

[0383] When a predetermined acquisition condition associated with the symbol is reached, the image is preferably acquired automatically, and the appearance (eg, color) of the symbol is preferably modified, or the symbol disappears.

[0384] An example illustrating the second main aspect of the invention will now be described:

[0385] In step a), an image is acquired using an acquisition device.

[0386] In step b), the acquired image is analyzed to update the coverage level.

[0387] Specifically, we look for possible representations of objects on the acquired images, i.e., the presence of latent contributions from the acquired images.

[0388] If this potential contribution exists, a view of the reference model showing the greatest agreement with the acquired image is searched for and the area of ​​said view corresponding to the target and hence the corresponding area on the reference model is deduced. If a part of the latter surface has not yet been registered as belonging to the covered area, it is added to the covered area and marked on the reference model, preferably colored in a first color specific to the covered area, such as green, the remainder of the area of ​​the target on said view is preferably colored in a second color, such as red, and the area of ​​the reference model not defining the target is preferably colored in a third different color, such as white.

[0389] A view of the reference model 16 equivalent to the preview image is projected in the direction of the optical axis onto the screen of the image acquisition device 10 , preferably a mobile phone.

[0390] The projected view of the reference model can be

[0391] - superimposed almost exactly on the preview image displayed on the screen, with or without transparency, or

[0392] -Instead of preview images, such as Figure 4 shown.

[0393] Due to the specific color of the areas, this projected view allows the user to see which areas are covered and which areas are yet to be acquired.

[0394] This presentation enables the user to quickly and easily identify the areas of the teeth that are yet to be covered and thus to easily orient the acquisition device.The presentation also informs the user of the coverage level.

[0395] As images are acquired during the cycle of steps a) to c), the area covered increases.

[0396] Figure 4 The images in each represent a view of a reference model representing two dental arches, preferably similar to the dental arches of the user. Each view corresponds to a preview image and is displayed instead of said preview image, preferably on the screen of a mobile phone used for image acquisition. The reference model views are thus "equivalent" to the preview images. In other words, when the user turns the acquisition device around his mouth, the displayed reference model views immediately adapt accordingly. Thus, the user can easily assimilate the reference model to their dental arch. Thus, he can easily position and orient the acquisition device to acquire images of the teeth that are not yet covered.

[0397] exist Figure 4In the embodiment shown, the target consists of all teeth of both dental arches, and the object in the mouth consists of both dental arches (and therefore includes the gums as well as the teeth). The tooth color is specifically dark grey (GF) when the tooth coverage is sufficient, otherwise light grey (GS).

[0398] When the user moves the acquisition device in front of his teeth, the surface area covered increases and thus the number of teeth shown in dark grey that are fully covered increases. The other teeth remain light grey.

[0399] For example, the coverage level corresponds to the percentage of the covered area relative to the initial area to be covered. The coverage threshold can be the percentage of the target area to be covered. For example, the threshold can be 90%, i.e., the coverage threshold is reached when more than 90% of the tooth surface belongs to the covered area.

[0400] A symbol in the form of a dot (possibly one-dimensional) may symbolically represent the tooth to be covered (target). For example, when at least 90% of the surface of the tooth is acquired, better when at least 95% of the surface of the tooth is covered, even better when the entire surface of the tooth is covered, the symbol symbolically representing the tooth is no longer shown on the screen. Alternatively, the symbol is displayed in color or highlighted.

[0401] It is now clear that the device and method according to the invention advantageously make it possible to increase the autonomy of the user and improve the quality and content of the images acquired by a user without special knowledge in the dental field. They can also be used to generate 3D models of objects belonging to or constituting objects in the mouth of the user from a distance. Finally, they make it easier to determine orthodontic treatments at a distance and to follow up any orthodontic treatments without the user having to make an appointment with a dental care professional.

[0402] Of course, the present invention is not limited to the embodiments described above and exemplified.

[0403] In particular, the mobile phone can be replaced by a device comprising a holder equipped with a camera and resting against the user during the acquisition of a set of images, and a screen displaying the scene observed by the camera, the screen being integrated into the holder or at a distance therefrom.

[0404] The shapes of these symbols are not limited. Symbols can be presented in 1, 2 or 3 dimensions simultaneously in augmented reality.

Claims

1. A method for acquiring a set of images covering a target belonging to an object in a user's mouth, the method comprising the steps of: 1) presenting to the user on a screen (12) and using spatial augmented reality relative to the object in the mouth observed by an image acquisition device (10) a multi-dimensional symbol or a set of multi-dimensional symbols (24), determining the shape and / or position of each symbol so as to indicate to the user at least one predetermined acquisition condition suitable for acquiring the image; 2) For each symbol, when at least one predetermined acquisition condition associated with the symbol is satisfied, preferably when all predetermined acquisition conditions associated with the symbol are satisfied, using the acquisition device to acquire the image.

2. The method according to the preceding claim, wherein: In step 2), an image is acquired by means of the acquisition device only if a predetermined acquisition condition associated with at least one symbol is met.

3. A method according to any one of the preceding claims, wherein at least one symbol (24) defines a sign axis (X), preferably an axis of rotation, the acquisition condition being an angular deviation of less than 20° between the optical axis of the acquisition device and the sign axis, and / or - a dimension (d), said dimension being variable in the representation of said symbol on said screen as a function of the distance between said acquisition means and said symbol in the augmented reality, said acquisition condition being for a specific value of said dimension or for said dimension to belong to a predetermined specific range of values.

4. A method according to any of the preceding claims, wherein in step 2), the image is automatically acquired using the acquisition device if predetermined acquisition conditions defining the spatial position and / or orientation of the acquisition device around its optical axis and including at least one acquisition condition indicated by the symbol are met.

5. The method according to any one of the preceding claims, wherein in step 2), the image is acquired by the acquisition device, - modify the appearance of the symbol or make it disappear, and / or - emit an audible signal, and / or - modifying a score displayed on the screen and related to the coverage of the target by the images already acquired and / or to the duration of acquisition of the images already acquired and / or to the quality of the images already acquired and / or to the usefulness of the images already acquired.

6. Method according to the preceding claim, wherein when the set of images has been acquired, an ordering determined according to the scores is presented to the user on the screen.

7. A method according to any of the preceding claims, wherein the screen displays the symbol on a preview image representing the real scene observed by the acquisition device or on a view of a model representing at least the object or the target in the mouth like the preview image.

8. A method according to any one of the preceding claims, wherein the screen displays a reference mark in a fixed position on the screen, the symbol preferably having a shape complementary to the reference mark when a predetermined acquisition condition associated with the symbol is met.

9. Method according to any of the preceding claims, wherein the target comprises more than 5 teeth and / or an orthodontic appliance and / or the set of symbols comprises more than 2 symbols.

10. The method according to any of the preceding claims, wherein in steps 1) and 2) the acquisition device is operated by the user.

11. An acquisition method according to any of the preceding claims, wherein the image is a photograph, preferably a real photograph, and / or represents a deformed mask, wherein the deformed mask is generated by a projection, preferably by the acquisition device, of an original mask, wherein the original mask is preferably in the form of a grid or a set of points.

12. A device for implementing the method according to any one of the preceding claims, comprising - an image acquisition device, preferably in the form of a mobile phone (10); - a computer (14), preferably integrated into the acquisition device or in communication with the acquisition device, having a computer program including program code instructions for - in step 1), one or more multidimensional symbols (24) are arranged in augmented reality in the space of the oral object and presented to the user on a screen (12, 18), preferably the screen (12) of the acquisition device, the shape and / or position of the symbols (24) being determined to indicate to the user at least one predetermined acquisition condition suitable for acquiring the image; -Preferably, in step 2), the acquisition of the image using the acquisition device is authorized only if at least one predetermined acquisition condition associated with the symbol is met, and / or the acquisition device is commanded to acquire the image only if at least one predetermined acquisition condition is met, and / or the coverage level of the acquired image over the target is updated, and preferably the coverage level is compared with a coverage threshold, and information about the coverage level and / or about the difference between the coverage threshold and the coverage level is preferably presented on the screen.

13. The device according to the preceding claim, wherein the acquisition means is - a mobile phone, and the screen is integrated into the mobile phone, or - A device comprising a holder equipped with a camera and which rests against the user during image acquisition, the screen being integrated into the holder or being at a distance from the holder.

Images