Stylus for three-dimensional scanning system

Abstract

The invention relates to a stylus for a three-dimensional scanning system using manual probing and a vision system, comprising a probe and a locator including several unique markers (M), each having a normal orientation different from the others; characterized in that the markers (M) are in number, position, and orientation such that at least two markers (M) are readable by a camera of the vision system, from any viewpoint of the locator in probing condition. Figure for the abstract: Fig. 5

Description

Title of the invention: Stylus for three-dimensional scanning system Technical field

[0001] The present invention relates to three-dimensional digitization, and more particularly to digitization by manual probing using a stylus surmounted by a passive locator. Prior art

[0002] Planar markers are widely used in vision-based spatial localization systems. The operating principle is generally broken down into two main steps: - the detection of the marker on an image; and - the pose estimate describing the spatial position of this marker.

[0003] Markers dedicated to spatial localization are mostly binary in color (black and white), square in shape (“ArUco” contained in the free library “OpenCV”, “AprilTag” developed by the University of Michigan) or circular (“CCTag” contained in the eponymous free library). Such markers consist of a matrix allowing the identification of the different markers of the locator. The deciphering of this matrix is ​​specific to the type of marker used.

[0004] A plane marker can be fixed on a moving object in order to follow the latter's movements.

[0005] In the field of surgical navigation, this marker is positioned on a stylus to the end of which a probe has been fixed. This stylus is used to manually digitize any bone surfaces in three dimensions. It is observed by a camera placed approximately 1 meter away.

[0006] A phenomenon called "pose ambiguity" appears for certain positions of this marker. With reference to [Fig.l], we see that the camera (20) can capture the same image (Cl, Cl') corresponding to two different positions of the stylus, offset by an angle (b). The digitizing system cannot detect whether the stylus (10) is probing a first point (PI) or another point (PI'). Due to the uncertainties of location of this marker, it then becomes complicated, or even impossible, to define by monocular vision the actual pose of the stylus.

[0007] In addition, a partially masked, stained or non-flat marker (if it is stuck on a non-flat surface, or if it comes off) greatly disrupts the spatial position found. The measurement uncertainties obtained are not compatible with the requirements of the digitization, and the stylus must be replaced. Statement of the invention

[0008] The invention therefore aims to overcome the aforementioned drawbacks, and in particular aims to propose a stylus making it possible to avoid ambiguities in placement.

[0009] The invention also aims to provide a stylus that is durable.

[0010] Finally, the invention aims to propose a stylet which is easier to use for the practitioner.

[0011] For this purpose, a stylus has been developed for a three-dimensional digitization system by manual probing and a vision system having a probe and a locator comprising several unique markers and each having a normal of different orientation relative to each other.

[0012] According to the invention, the markers are in number, position and orientation so that at least two markers are readable by a camera of the vision system, from any point of view of the locator in probing condition.

[0013] In this way, regardless of the position or orientation of the probe, it is guaranteed that several markers will be readable by the camera, which avoids ambiguities in positioning when using a monocular camera. In addition, the practitioner does not have to worry about correctly orienting the stylus in relation to the camera: he is therefore freer in his movements during probing, which is facilitated. Furthermore, reading two markers instead of just one increases the resolution of the reading carried out: the digitization is more precise.

[0014] By “unique markers” we mean that the markers are individual, different from each other.

[0015] By camera is meant any optical capture device such as a camera, a video camera or a sensor, configured to obtain an image of the markers.

[0016] The expression "from any point of view of the locator in probing condition" means all reasonably foreseeable arrangements and inclinations of the locator during the probing step, provided that the image of the locator is clear (compatible camera depth of field, and no motion blur during acquisition). Examples of positions are illustrated [Fig.3].

[0017] The locator is passive, that is to say that it does not emit light or radio waves intended to be picked up by the system so that the latter can determine the position of the locator.

[0018] In a preferred embodiment, the markers are in number, position and orientation such that at least three markers are readable by a camera, regardless of the marker position. This mode provides two distinct advantages: - the information captured by the camera is redundant because only two markers would have been sufficient to resolve the pose ambiguity. The redundancy of the information makes the estimation of the stylus pose even more precise, so the three-dimensional digitization will be of better quality. - the durability of the stylus is increased, because if a marker becomes faulty (either due to soiling during probing, or due to deterioration through wear) then the stylus remains functional because two markers are sufficient to avoid ambiguities in placement - the performance of the stylus is increased because an image captured by the camera can be used even if the locator is partially hidden (by the surgeon or by any object): two markers only need to be visible.

[0019] In order to ensure good readability of the markers, the markers are two-dimensional codes, such as a 2-dimensional bar code.

[0020] Still to guarantee good readability, for a given point of view, the normals of the three visible markers have an inclination of less than 75° relative to the optical axis of the point of view, and preferably less than 60°.

[0021] For the resolution of the digitization to be good, the probe has a spherical geometry with a radius between 0.2 mm and 2 mm, and preferably equal to 1 mm.

[0022] In another embodiment, the probe has a geometry in the form of a tip, to palpate certain articular zones by crossing the cartilaginous zone, in order to directly touch the bone.

[0023] In order to ensure a regular distribution of the markers, the normals of the markers are each parallel to the normal of a face of a uniform polyhedron such as a dodecahedron, an icosahedron, a truncated cube or a truncated octahedron. A regular distribution makes it possible to avoid certain orientations of the locator being less efficient than others.

[0024] In one embodiment, the locator has an irregular geometry. This embodiment does not have the aforementioned advantages, however the design and manufacture of the locator are made easier because the geometric constraints are less.

[0025] Advantageously, the markers measure between 8 and 20 mm, preferably between 10 mm and 15 mm, and even more preferably measure 12 mm. By "dimension" is meant the length of the sides of a square marker, or the diameter of a circular marker. These dimensions make it possible to have a good compromise between the size of the locator and the readability of the markers, when the camera is placed at a distance of approximately 1 m from the stylus during probing.

[0026] The invention also relates to a three-dimensional scanning system by manual probing and monocular vision system comprising: - a stylus according to the aforementioned characteristics; - a camera configured to acquire an image of the locator during probing; - a computer running a computer program programmed to: - identify the markers visible on the acquired image; - determine the position of the locator relative to the camera; - determine the position of the probe relative to the locator; - reconstruct a geometry of the probed surface from several determined positions of the probe. Such a system makes it possible to benefit from the aforementioned advantages of the stylus. In particular, the stylus according to the invention makes it possible to use a monocular camera, which reduces the cost of the system, and which frees up the operating field because there is only one camera field of vision to clear. The precision obtained from the localization is nevertheless equivalent and sometimes superior to certain stereoscopic systems of the prior art.

[0027] The invention also relates to a method for calibrating such a vision system, and comprising the following steps: - the camera acquires several images of the locator from different points of view; - the computer program is programmed to: - identify the markers visible on each acquired image; - model in three dimensions the relative positions of the markers on the locator; the position of the probe relative to the locator being predetermined. Such a method makes it easy to use a new stylus with a pre-existing system. The position of the probe relative to the locator can be determined beforehand, by probing a standard of known geometry.

[0028] In order to reduce the memory resources required by the computer, the computer program is programmed to evaluate the readability of a marker, so as to either take the marker into account during acquisitions (during probing) if its readability is sufficient, or to ignore it if its readability is insufficient. Ignoring it makes it possible to avoid errors in identifying the marker and ambiguities in its placement. Ignoring it also makes it possible to save computing resources, and the response times of the computer program are improved.

[0029] Advantageously, the computer program is programmed to issue an alert when the arrangement of the illegible markers is such that there are not at least two markers readable from any point of view of the locator. This method makes it possible to check whether a stylus is still functional, despite its aging or any damage it may have suffered. Description of figures

[0030] [Fig-1] is a diagram illustrating the installation ambiguities that can occur with prior art stylets.

[0031] [Fig.2] is a diagram illustrating a stylus according to the invention.

[0032] [Fig.3] is a diagram illustrating different positions that a stylus can adopt during probing.

[0033] [Fig.4] is a diagram illustrating the normal inclinations of the markers relative to the camera viewpoint.

[0034] [Fig.5] is a diagram illustrating a locator according to the point of view of the camera.

[0035] [Fig.6] is a diagram illustrating an image captured by the camera.

[0036] [Fig.7] is a diagram illustrating an image similar to that of [Fig.6], in which a marker is not readable.

[0037] [Fig.8] is a diagram illustrating a stylus whose locator is of irregular shape. Detailed description

[0038] With reference to Figures 2 to 8, the invention relates to: - a stylus (10) for manual probing allowing three-dimensional scanning; - to a manual probing scanning system; and - to a method of calibrating such a system.

[0039] With particular reference to Figures 2 and 3, a stylus (10) for manual probing digitization comprises: - a probe (11) intended to be positioned on a surface to be scanned of an object (30), for example a part of a joint; - a locator (12) comprising markers (M), such as dimension bar codes, intended to be read by a camera (20); - a rod (13) connecting the probe (11) to the locator (12).

[0040] Depending on the relative arrangement of the markers (M), a computer program is programmed to determine the position of the locator (12), and consequently the position of the point on which the probe (11) is placed. The determination of a sufficient number of points scanned by this stylus makes it possible to digitize the surface of the object (30).

[0041] In order to remove any ambiguity of installation, the locator (12) has at least two markers (M) readable by the camera (20), whatever its position during probing. For this, the locator (12) has markers (M) in a number, position, and location adapted.

[0042] In [Fig.2], the same angle difference (b) is reproduced between the inclinations of the stylus (10). A diagram of the images captured by the camera (20) shows that the capture (Cl) corresponding to the pose on the point (P) cannot be confused with another capture (Cl') corresponding to another inclination.

[0043] Consequently, the surgeon is no longer required to orient the locator (12) to ensure that the markers (M) are facing the camera (20) when placing the stylet (10). He is much freer in his movements, which facilitates palpation.

[0044] [Fig. 3] schematically illustrates different positions that a stylus (10) can adopt during probing. The inclinations of the stylus (10) can be diverse and varied. Compared to the camera (20), there may be a blind spot located behind the object (30) to be probed. In this case, the length of the rod (13) of the stylus (10), generally between 10 cm and 25 cm, makes it possible to probe this area while placing the locator (12) in the field of the camera (20).

[0045] With reference to Figures 2, 4 and 5, the orientations of the markers (M) relative to the camera (20) are illustrated. The locator (12) has a multitude of markers. Although several markers (M) are visible to the camera (20), not all of them are readable by it.

[0046] Indeed, as illustrated [Fig.4], if a marker (M) has too great an inclination with respect to the camera (20), it is not readable. In [Fig.4]: - the markers (M) M1, M2 and M3 are correctly oriented with respect to the camera (20), so that their readability is possible; - markers (M) M4, M5 and M6 are too inclined and are not readable.

[0047] The measurement of the inclination of the markers (M) is made between the optical axis (21) of the camera (20), and the normal (n) of the markers. For example, the normals (ni, n2, n3) respectively of the markers (Ml, M2, M3) are represented in Figures 2 and 4. The corresponding angles (al, a2, a3) are represented in [Fig.4].

[0048] [Fig.6] illustrates a capture (C) carried out by the camera (20), and makes it possible to illustrate that the problem posed is resolved by the orientation of the markers (M), not necessarily by the geometry as such of the locator (12): - the locator (12) can be a convex solid, and in particular a convex uniform polyhedron such as the dodecahedron of [Fig.4], an icosahedron, a truncated cube or a truncated octahedron; - the locator (12) can be a non-convex solid, for example in the form of a star bearing a marker (M) at each of its ends: a regular twelve-pointed star would provide the same capture as that of a dodecahedron; - the locator (12) may be an irregular solid, as illustrated [Fig.8].

[0049] In practice, a convex uniform polyhedron makes it possible to obtain a regular distribution of the markers (M), so that the reading performance is the same, whatever the orientation of the stylus (10) during probing.

[0050] A convex solid does not have retention zones: cleaning of the stylet (10) is facilitated, which is preferable in the medical field, and in particular that of surgery.

[0051] [Fig.7] illustrates a capture (C) on which a marker (M3) is damaged. Depending on the damage, a damaged marker (M) may not be read by the program, or may be misread by the program, i.e. provide: - a good identification of the marker (M), but in the wrong orientation; or - a bad identification of the marker (confusion between two markers).

[0052] In the case of a misreading, the computer program will be misled and the determination of the pose could lead to ambiguity.

[0053] In order to guard against such drawbacks, a method for calibrating the digitization system comprises a step of evaluating the readability of the markers. During this step, the computer program is programmed to evaluate the readability of each marker (M), so as to: - either take into account the marker (M) during acquisitions, if its readability is sufficient; - either ignore the marker (M) if its readability is insufficient.

[0054] Once the system is calibrated according to this method, the computer program is programmed to ignore the defective marker (M). This means that on the images captured by the camera (20), the computer program is programmed to ignore the data likely to correspond to that of the defective marker (M). This data is identified by its position relative to the other markers (M) that are still readable, for example the markers (M) M1 and M2 in [Fig. 6]. The risks of ambiguities in the pose are eliminated.

[0055] The invention also relates to a method for calibrating a three-dimensional scanning system, making it possible to use a new stylus (10) with a pre-existing system. This calibration sequence comprises the following steps: - the camera (20) acquires several images of the locator (12) from different points of view; - the computer program is programmed to: - identify the markers (M) visible on each acquired image; - model in three dimensions the relative positions of the markers (M) on the locator (12).

[0056] Of course, it is necessary to acquire enough images so that each marker (M) has been viewed at least once by the camera (20).

[0057] Thus, for each image captured during probing, the computer program will be able to identify the at least two readable markers (M), then to determine the position (including the orientation) of the locator (12).

[0058] The position of the probe (11) relative to the locator (12) being predetermined in the program, the position of the probe (11) can therefore also be determined.

[0059] In addition, the calibration method may comprise a step of verifying the conformity of the locator (12). For this, the computer program is programmed to issue an alert when the arrangement of the illegible markers (M) is such that there are not at least two markers (M) readable from any point of view of the locator (12). In this way, it is possible to verify that a stylus (10) is still functional (absence of alert), or to replace or repair the defective stylus (10) (in the event of an alert).

[0060] A stylus (10) according to the invention makes it possible to obtain: - the position of the probe (11) with an uncertainty of less than 1 mm3; and - the orientation of the rod (13) of the stylus (10) with an uncertainty of less than 1°.

[0061] The stylus (10) can be located regardless of its position in the field of observation, due to the multitude and arrangement of the markers (M) on the locator (12).

[0062] The stylus (10) is compatible with both systems comprising several cameras (20) and monocular systems.

[0063] Furthermore, the stylus (10) and the locator (12) may be shaped differently from the examples given without departing from the scope of the invention, which is defined by the claims.

[0064] In particular, the markers (M) may be shaped differently from a two-dimensional bar code, may be of any type suitable for the present application. The markers (M) may be of different colors.

[0065] Furthermore, the technical characteristics of the different embodiments and variants mentioned above can be, in whole or in part, combined with each other. Thus, the stylus (10) and the locator (12) can be adapted in terms of costs, functionalities and performance.

Claims

Claims

1. Method for calibrating a three-dimensional scanning system by manual probing and monocular vision system, the three-dimensional scanning system by manual probing and monocular vision system comprising: - a stylus (10) for a three-dimensional scanning system by manual probing and vision system, having a probe (11) and a locator (12) comprising several unique markers (M) and each having a normal of different orientation relative to each other, the markers (M) being in number, position and orientation so that at least two markers (M) are readable by a camera (20) of the vision system, according to any point of view of the locator (12) in probing condition, - a camera (20) configured to acquire several images of the locator (12) during probing;- a computer executing a computer program programmed to: - identify the markers (M) visible on each acquired image; - determine the position of the locator (12) relative to the camera (20); - determine the position of the probe (11) relative to the locator (12); - reconstruct a geometry of the probed surface from the determined positions of the probe (11), the method comprising the following steps: - the camera (20) acquires several images of the locator (12) from different viewpoints; - the computer program is programmed to: - identify the markers (M) visible on each acquired image; - model in three dimensions the relative positions of the markers (M) on the locator (12);the position of the probe (11) relative to the locator (12) being predetermined, characterized in that the computer program is programmed to emit an alert when the arrangement of the illegible markers (M) is such that there are not at least two markers (M) readable from any point of view of the locator (12).;

2. Calibration method according to claim 1, characterized in that the markers (M) are in number, position and orientation so that at least three markers (M) are readable by a camera (20) of the vision system, from any point of view of the locator (12) in probing condition.

3. Calibration method according to claim 2, characterized in that the markers (M) are 2-dimensional codes, such as a 2-dimensional bar code, and in that for a given viewpoint, the normals of the three visible markers (M) have an inclination of less than 75° relative to the optical axis of the viewpoint, and preferably less than 60°.

4. Calibration method according to one of the preceding claims, characterized in that the probe (11) has a spherical geometry with a radius between 0.2 mm and 2 mm, and preferably equal to 1 mm.

5. Calibration method according to one of the preceding claims, characterized in that the normals of the markers (M) are each parallel to the normal of a face of a uniform polyhedron such as a dodecahedron, an icosahedron, a truncated cube or a truncated octahedron.

6. Calibration method according to one of claims 1 to 4, characterized in that the locator (12) has an irregular geometry.

7. Calibration method according to one of the preceding claims, characterized in that the markers (M) measure between 8 and 20 mm, preferably between 10 mm and 15 mm, and even more preferably measure 12 mm.

8. Calibration method according to one of claims 1 to 7, characterized in that the computer program is programmed to evaluate the readability of a marker (M), so as to: - either take the marker (M) into account during acquisitions, if its readability is sufficient; - or ignore the marker (M) if its readability is insufficient.

Images