METHOD, COMPUTER PROGRAM AND SYSTEM FOR IDENTIFYING, DETECTING AND LOCALIZING AN OBJECT INSTANCE IN A 3D SCENE

Description

[0001] The present invention relates to a method, a computer program, and a system for identifying an instance of an object in three-dimensional scene representation data using a descriptor. It also relates to a method for three-dimensional detection and localization of an object instance, including the execution of the aforementioned method for its identification.

[0002] Generally speaking, this falls within the industrial domain of recognizing rigid or quasi-rigid (i.e., non-articulated) objects, or even articulated objects, in three-dimensional computer vision. Such object recognition requires comparing the representation data of the three-dimensional scene with that of at least one rigid or quasi-rigid object model, or even a slightly articulated object model, or with that of another three-dimensional scene, for example, for object detection and localization by registration. This process involves determining at least one set of spatial transformation parameters for translation and rotation to match at least one object instance in the three-dimensional scene with the corresponding object model, or with another corresponding object instance in the other three-dimensional scene.More strongly articulated objects can be broken down into several rigid or quasi-rigid parts, so that the same treatments are applied to each of them, for example by using a set of transformation parameters per rigid or quasi-rigid part of these objects.

[0003] In this three-dimensional context, an "object model" refers to a predetermined representation of a reference object. It can consist of a modeled representation, for example, by a computer, in three spatial dimensions of a real object as a three-dimensional point cloud, a mesh of faces, or any other mathematical representation (explicit, implicit, parametric, etc.) that can be converted into a three-dimensional point cloud. It can also be derived from a real-world scene capture, using one or more sensors, in which the object's representation as a three-dimensional point cloud and its arrangement are sufficiently controlled to serve as a reference.An "object instance" in a three-dimensional scene is understood to be a comparable representation of an object in that three-dimensional scene; that is, a representation modeled as a three-dimensional point cloud, a meshed set of faces, or any other mathematical representation that can be converted into a three-dimensional point cloud. These representations may also contain information other than points, such as color information.

[0004] The practical industrial applications are numerous and extend beyond object detection and localization. They include, but are not limited to: three-dimensional localization, for example in virtual, augmented or mixed reality, of a system such as a vehicle, including an autonomous vehicle, a pedestrian, a robot or a portable device, by simultaneous localization and mapping techniques; three-dimensional reconstruction of a scene by data fusion from complementary partial observations; registration of a model in a scene; classification of objects present in a scene; three-dimensional reconstruction of a scene by three-dimensional registration of one scene onto another, one of these two scenes being a reference; analysis of a three-dimensional scene including recognition and localization of objects in that scene; identification of points of interest in a scene (for example points with a particular curvature); segmentation of surfaces in a scene according to a similarity criterion;Real-time automated inspection of industrially manufactured objects, particularly for defect detection in quality control; robotic gripping and manipulation of objects in space, notably using at least one industrial articulated arm and / or in an assembly line, for example.

[0005] The latest application notably includes bin picking, which consists of recognizing and then moving, successively or simultaneously, using one or more robot arms, bulk objects in a production line or in a storage area.

[0006] The principle of object recognition and localization by rigid registration on a model, or by comparing scene representation data, is extensively studied in the prior art. According to a well-established processing chain, described for example in US patent document 8,830,229 B2, it comprises the following steps: obtaining translation and rotation transformation parameter sets for respective three-dimensional mappings of particular points of each assumed instance of an object in the three-dimensional scene with their arrangements in a corresponding three-dimensional model or in another corresponding object instance in another three-dimensional scene, from these point-to-point parameter sets and if at least one object instance is actually present in the three-dimensional scene, determining, by grouping these point-to-point parameter sets, at least one translation and rotation transformation parameter set for the three-dimensional mapping of at least one object instance from the three-dimensional scene with at least one three-dimensional object model or at least one other object instance from another three-dimensional scene, and optionally,verification and refinement of the determined transformation(s).

[0007] The first step generally involves preprocessing the three-dimensional scene representation data obtained from one or more sensors, selecting specific points of interest to be mapped in the scene and the model or another scene, calculating the values ​​of local or global descriptors to be associated with these selected points to facilitate and improve the mapping, and then performing the mapping itself. This first step can also be implemented in applications other than object detection, recognition, or localization by registration.

[0008] It involves executing a process for identifying an instance of an object in representation data of a three-dimensional scene using a descriptor, comprising the following steps: selection of at least one representative point of the object instance in the representation data; calculation of a plurality of identifying values ​​taken by parameters of the descriptor for each selected representative point; and addition of the identifying values ​​calculated at each selected representative point to the object instance identification data.

[0009] Each representative point of the object instance can be a point on a surface of the object instance. It can also be a representative point of the object instance, and even more precisely of its surface, without itself being on that surface, such as a point on a principal axis or other axis of the object.

[0010] Similarly, it is common practice to calculate the parameter values ​​of the descriptor for each representative point selected in a local coordinate system defined at that point. However, this local coordinate system may be implicit, or even undefined, depending on the definition of the descriptor.

[0011] The choice of a descriptor, whether local or global, as well as the method used to evaluate and compare its values, are crucial to the relevance of the resulting transformation parameters and the success of the matching process. More generally, and beyond the aforementioned application, they also have a major impact on the complexity and processing time of the data. Furthermore, the quality of a descriptor is an even more important criterion, as it may also be desirable to use it for the selection of points of interest themselves, given that these points of interest are advantageously those with easily recognizable and / or highly discriminating descriptor values.

[0012] No descriptor, however complex, can be completely discriminating, firstly because some objects exhibit symmetries or pseudo-symmetries, and secondly because invariances always exist within an object for a given descriptor, whether local or global. Invariances of an object are different areas of that object that can be considered identical from the descriptor's point of view, except for a transformation such as rotation, translation, scaling, or another characteristic described by the descriptor. In particular, it is common for an object to have spherical, cylindrical, or planar portions, which are inherently sources of invariances.

[0013] An example of a descriptor enabling simple, discriminating, and robust surface identification, which can be considered as a local or global descriptor, is given in patent document WO 2020 / 201392 A1. Not necessarily involving statistical calculations, this descriptor is also perfectly suited to comparing its values ​​using Euclidean distance calculations and known rapid search methods. However, like all other descriptors, it is not immune to the ambiguities of matching due to the invariances of the objects to which it applies, in addition to those due to noise, different instances of the same object in a scene, and other difficulties that can disrupt these matching processes.

[0014] Regarding the second and third stages, they are the subject of much research and proposed solutions in the state of the art.

[0015] An ingenious solution for the second step is introduced, for example, in patent document WO 2020 / 065177 A1. It proposes an expression for each set of transformation parameters in a coordinate system that facilitates the calculation of distances and averages for simplified and optimized automatic clustering of these transformation parameter sets into one or more groups (clusters), with the aim of mapping instances and object models. Furthermore, it is particularly well-suited to the descriptor of document WO 2020 / 201392 A1.

[0016] A commonly chosen solution for the third step is the ICP method (from the English "Iterative Closest Point"), taught in the article by Besl et al, entitled "A method for registration of 3-D shapes", published in IEEE Transactions on Pattern Analysis and Machine Intelligence, volume 14, no. 2, February 1992, pages 239 to 256. It leads to very good local convergence, but depends very strongly on the initial conditions, which can therefore advantageously result from the second step when it is well carried out.

[0017] Regardless of the algorithmic methods used for one or more of the three aforementioned steps, the choice of the process used to identify an instance of an object using a descriptor is crucial. The symmetries and other invariances, however inevitable, of objects often lead to the development of descriptors that seek to eliminate them. Consequently, many state-of-the-art documents strive to develop complex descriptors that are highly discriminating in most cases, while neglecting the fact that certain surfaces, such as perfectly spherical, cylindrical, or flat surfaces, remain invariant regardless of the descriptor chosen and its complexity.

[0018] Some documents specifically detect symmetries or invariances, but: either to exclude them from the description of the object and therefore from its identification, as for example in the article by Alexandrov et al, entitled "Leveraging symmetries to improve object detection and pose estimation from range data", published in Proceedings of International Conference on Computer Vision Systems 2019, pages 397-407, or to process and transform the object in order to make it symmetrical, as for example in the US patent document 8,411,081 B2.

[0019] In documents WO 2020 / 065177 A1 and WO 2020 / 201392 A1, although the grouping description and classification methods taught therein cooperate advantageously to detect, identify and locate instances of objects, the symmetries and invariances add an unresolved ambiguity in the processing of the data.

[0020] It may therefore be desirable to provide a method for identifying an instance of an object that allows us to overcome at least some of the aforementioned problems and constraints.

[0021] Therefore, a method is proposed for identifying an instance of an object in representation data of a three-dimensional scene using a descriptor, comprising the following steps: selection of at least one representative point of the object instance in the representation data; calculation of a plurality of identifying values ​​taken by parameters of the descriptor for each selected representative point; addition of the identifying values ​​calculated at each selected representative point to the identification data of the object instance; search for at least one possible invariance by translation and / or rotation transformation of at least one model of the object in representation data of at least one reference scene including this model, to obtain invariance information relating to each invariance found; and addition of said invariance information to the identification data of the object instance.

[0022] Thus, based on the observation that potentially missing information in any descriptor, or the inherent ambiguity of an object's symmetry property, is compensated for by analyzing invariances in a model of that object, and vice versa, the present invention cleverly proposes adding this invariance information to the object instance's identification data, rather than discarding it or using it for another purpose. This results in a more complete identification of the object instance, enabling improved three-dimensional registration, defect detection, and other computer vision applications.

[0023] Consequently, the present invention also makes it possible to use a simple, low-dimensional descriptor such as that in document WO 2020 / 201392 A1 and to add invariance information to it to recover lost information, rather than attempting to develop increasingly complex descriptors. This additional invariance information is also particularly well-suited to the coordinate system proposed in document WO 2020 / 065177 A1 for expressing each set of transformation parameters in a three-dimensional registration application.

[0024] Optionally: The search for at least one possible invariance involves the following steps: obtaining sets of transformation parameters in translation and rotation for a respective three-dimensional mapping of at least one representative point of a first model of the object in representation data of a first reference scene including this first model with at least one representative point of a second model of the object in representation data of a second reference scene including this second model, using said descriptor parameters, and expressing each set of transformation parameters in a multi-dimensional coordinate system including at least one dimension for translation and at least one dimension for rotation, to obtain an invariance pattern of the object; and adding said invariance information involves adding the invariance pattern obtained to the identification data of the object instance..

[0025] Optionally, obtaining the transformation parameter sets in translation and rotation is also done using the said parameters of the descriptor whose values ​​are calculated respectively at each representative point of the first and second models.

[0026] Optionally, the search for at least one possible invariance also includes an automatic grouping of said sets of transformation parameters, each group obtained being representative of an invariance by translation and / or rotation of the object and of a unique set of transformation parameters for matching the first model of the object with the second model of the object.

[0027] Also optional: the representation data of the second model of the object in the second reference scene is obtained by adding predetermined noise to the representation data of the first model of the object in the first reference scene; and / or the second model of the object as represented in the second reference scene results from a deformation and / or a translational and / or rotational transformation of the first model of the object as represented in the first reference scene.

[0028] Optionally, a disambiguating potential score is also calculated in at least one area of ​​said at least one model of the object by combining invariance scores calculated for each invariance found in that area.

[0029] A method for three-dimensional detection and localization of an instance of an object in representation data of a three-dimensional scene is also proposed, by comparison with a model of the object in representation data of a reference scene including this model, comprising the following steps: execution of an identification process as defined above, for the identification of the object instance; obtaining sets of transformation parameters in translation and rotation for respective three-dimensional mappings of each representative point of the object instance in the three-dimensional scene with a possible arrangement of this representative point in the model; automatic grouping of said sets of transformation parameters, each group possibly obtained being representative of a single set of transformation parameters for a mapping of the object instance with its model; and selection of a single group from among the group(s) obtained using the invariance information.

[0030] Optionally, the selection of a single group, from among the group(s) obtained using the invariance information, involves taking into account a disambiguating score for each matching of the object instance with its model.

[0031] Also proposed is a computer program downloadable from a communication network and / or recorded on a computer-readable medium and / or executable by a processor, comprising instructions for executing the steps of an identification process as defined above or for executing the steps of a three-dimensional detection and localization process as defined above, when said program is executed on a computer.

[0032] A system for identifying an instance of an object in three-dimensional scene representation data using a descriptor is also proposed; the system comprises: at least one memory for storing representation data of the three-dimensional scene, descriptor parameters and representation data of at least one reference scene including at least one model of the object; and an object instance identification processor in the three-dimensional scene, programmed to execute the steps of an identification process as defined above.

[0033] The invention will be better understood with the aid of the following description, given solely by way of example and made with reference to the accompanying drawings in which: [ Fig.1 ] there figure 1 schematically represents an example of digital representation in three-dimensional imaging of several object models, [ Fig.2 ] there figure 2 schematically represents an example of a digital representation in three-dimensional imaging of a three-dimensional scene containing several instances of objects, [ Fig.3 ] there figure 3 schematically represents the general structure of a three-dimensional detection and localization system for at least one object instance in three-dimensional scene representation data such as those of the figure 2 , including an object instance identification system according to an embodiment of the invention, [ Fig.4 ] there figure 4 illustrates the successive steps of a three-dimensional detection and localization process for at least one object instance implemented by the system of the figure 3 , including an identification of each object instance present in the three-dimensional scene of the figure 2 and each object model of the figure 1 , according to one embodiment of the invention, [ Fig.5 ] there figure 5 illustrates a first example of an object model and associated invariance pattern as it can be obtained by object identification according to the present invention, [ Fig.6 ] there figure 6 illustrates a second example of an object model and associated invariance pattern such as can be obtained by object identification according to the present invention, [ Fig.7 ] there figure 7 illustrates a third example of an object model and associated invariance pattern such as can be obtained by object identification according to the present invention, [ Fig.8 ] there figure 8 illustrates a preferred embodiment of a pattern grouping step obtained by object identification according to the present invention, [ Fig.9 ] there figure 9 illustrates the invariance pattern of the figure 7 as simplified by grouping according to the preferred embodiment of the figure 8 , [ Fig.10 ] there figure 10 illustrates the object model of the figure 7 on which a disambiguating potential score is calculated at each point thanks to the invariance pattern that can be associated with it, [ Fig.11 ] there figure 11 illustrates the object model of the figure 6 on which a disambiguating potential score is calculated at each point thanks to the invariance pattern that can be associated with it, [ Fig. 12 ] there figure 12 illustrates the result of an example of exploiting an object invariance pattern in the execution of the detection and localization process. figure 4 .

[0034] The set of 20 three-dimensional models represented on the figure 1 is an artificial reference scene consisting of several models of artificial objects. It could also be a real reference scene consisting of models of real objects whose arrangement is known and controlled. Here, it consists of three object models chosen as an illustrative example. A first three-dimensional model 22 represents an inclined T-shaped pipe fitting. A second three-dimensional model 24 represents an elbow pipe fitting. A third three-dimensional model 26 represents a profile with one end wrapped around it. These three object models are arranged and perfectly localizable in a three-dimensional reference frame linked to the assembly 20. They are represented, as previously indicated, as a three-dimensional point cloud, a meshed set of faces, or any other mathematical representation that can be converted into a three-dimensional point cloud.For obvious reasons of clarity, models 22, 24, and 26 are clearly separated from one another in set 20. However, they could advantageously be superimposed by being centered within set 20, while remaining distinguishable using distinctive labels associated with their constituent points. Advantageously, their sampling is the same or at least similar. In a simple case, set 10 could contain only a single three-dimensional model.

[0035] The three-dimensional scene 30 depicted on the figure 2 This illustrates what can result from a three-dimensional capture obtained using a 3D sensor of a real-world scene. It consists of three-dimensional points, or "3D points," in its own reference frame, each indicating, through shades of gray or color, the presence of actual object instances in the real-world scene. As an illustrative example, we can distinguish here two object instances 32 and 34 corresponding to the inclined T-pipe fitting model 22, three object instances 36, 38, and 40 corresponding to the elbow pipe fitting model 24, one object instance 42 of the straight T-pipe fitting type, which has no corresponding model in assembly 20, and no object instances corresponding to the profile model 26. Note that some object instances in scene 30 may only be partially visible.

[0036] In accordance with a three-dimensional detection and localization system / method for object instances 32 to 42 in three-dimensional scene representation data 30, by comparison with object models 22, 24 and 26 in set representation data 20, such as that which will be detailed with reference to figures 3 And 4A correspondence must be established between each object instance that can be identified and reconstructed in the three-dimensional scene 30 and one of the models 22, 24, and 26. This correspondence consists of finding the parameters of a rigid transformation involving translation, rotation, and possibly scaling (enlarging or reducing in size), which allows, where applicable, moving from the positioning of each object instance in scene 30 to that of its model in set 20, or conversely, from the positioning of each model in set 20 to that of a corresponding object instance in scene 30. It should be noted, however, that the example illustrated by the figures 3 And 4relates only to a particular family of possible applications of the present invention, namely the detection and localization of objects in a three-dimensional scene, the invention extending to applications beyond this particular family.

[0037] The 50 system for three-dimensional detection and localization of object instances 32 to 42, as schematically represented on the figure 3 , includes a processing unit 52 associated in a conventional manner with a memory 54 (for example a RAM memory). This memory 54 stores, for example, data 56 representing the scene 30, data 58 representing the set 20 of three-dimensional models and parameters 60 of a descriptor such as, for example, that defined in document WO 2020 / 201392 A1.

[0038] The processing unit 52 can be implemented in a computing device such as a conventional computer with a processor and one or more memories for storing data files and computer programs. The processing unit 52 can then itself be considered as consisting of a processor 62 for mapping instances and object models, associated with a memory 64 for storing the instructions it executes as computer programs. When at least some of the instructions can be executed in parallel, the processor can optionally be adapted for such parallelism, for example, in the form of a GPU (Graphics Processing Unit).

[0039] Among these computer programs, a first program 66 is designed, for example, to perform the identification of object instances in the three-dimensional scene 30 using the descriptor defined at least in part by the parameters 60 stored in memory 54. This identification consists of calculating identifying values ​​that the descriptor takes at several representative points to be selected for each object instance considered. This same first program 66 also allows for a similar identification of object models in the set 20. A second program 68 is designed, for example, to perform a complementary identification of object instances in the three-dimensional scene 30 specific to the present invention.It is its operation, combined with that of the first program 66, which is the main subject of the present invention, through a clever search for possible invariances by translational and / or rotational transformation of the models of assembly 20, for the provision of invariance information. These two programs 66 and 68, associated with the processor 62 in the processing unit 52 and with the memory 54, constitute an object instance identification system according to the present invention.

[0040] A third program 70, for example, is designed to execute a method for obtaining sets of transformation parameters for translation, rotation, and possibly scaling, for the respective three-dimensional mappings of selected representative points of object instances in the three-dimensional scene 30 with their possible arrangements in one of the models of the set 20. Such a method is known and will not be detailed. It is based on mapping the values ​​that the descriptor takes at these points in the two representations. It works all the better the more discriminating the descriptor.Optionally, this third program can also be designed to perform initial processing on the resulting transformation parameter sets, specifically to optimize their expression through transcoding for automatic grouping within a suitable representation space. This could involve expressing each transformation parameter set in a multidimensional coordinate system that includes at least one dimension for translation and at least one dimension for rotation, as described, for example, in document WO 2020 / 065177 A1.

[0041] A fourth program 72, for example, is designed to perform automatic grouping, each group obtained as output being representative of a single object instance in the three-dimensional scene 30 and a single set of transformation parameters for matching with a single model in the set 20. Such grouping is also known. It can be performed in one or more iterations depending on the number of object instances to be located and the method used. The method taught in document WO 2020 / 065177 A1 is, for example, advantageously applicable.Given the possible invariances of the models in set 20 with regard to the descriptor used, or more generally with regard to their possible symmetries or quasi-symmetries, several groups can be associated with the same object instance for the same model: in other words, several sets of transformation parameters involving several different mappings can be found for the same object instance to the same model.

[0042] A fifth program 74 is thus designed, for example, to exploit the invariance information provided by the execution of the second program 68 in order to select, for each object instance, a unique group from among the group(s) obtained by the execution of the fourth program 72. Optionally, this fifth program can also be designed to perform a second operation on each set of transformation parameters representative of each group finally selected in order to transcode it (for example, a reverse transcoding of the first operation) into a parameter set expressed in a desired output format. For example, this option of the fifth program 74 can simply be designed to reverse the operation of the corresponding option of the third program 70.

[0043] A sixth optional program, 76, is designed, for example, to perform a verification and refinement of the transformation(s) determined by automatic grouping. Such an optional verification with refinement is also known and will not be detailed. It involves, for example, the ICP method mentioned previously.

[0044] Finally, a seventh optional program 78 is for example designed to refine or challenge the selection of group(s) of the fifth program 74 by again exploiting the invariance information provided by execution of the second program 68, but according to a slightly different logic.

[0045] The processing unit 52 as illustrated on the figure 3 Thus, it functionally comprises seven computer programs or seven functions of the same computer program. It should be noted that computer programs 66, 68, 70, 72, 74, 76, and 78 are presented as distinct, but this distinction is purely functional. They could just as easily be grouped in any possible combination into one or more software programs. Their functions could also be at least partially microprogrammed or micro-wired into dedicated integrated circuits. Therefore, as an alternative, the computer system implementing the processing unit 52 could be replaced by an electronic device composed solely of digital circuits (without a computer program) to perform the same actions.

[0046] The installation of the figure 3 The system may include, in addition to the three-dimensional detection and localization system 50 for object instances 32 to 42, a 3D sensor 80 for obtaining representation data of the three-dimensional scene 30. Such a 3D sensor is not described in detail because it is well known to those skilled in the art. By generalization, it may consist of a combination of several sensors, for several views or detection modalities. According to one possible embodiment, the three-dimensional detection and localization system 50 is autonomous and separate from the 3D sensor 80. It then includes a (tele)communication interface 82 with this 3D sensor 80, wired or wireless, managed by the object instance and model matching processor 62. According to another possible embodiment, the system 50 could be embedded in the 3D sensor 80 or, conversely, the 3D sensor 80 could be integrated into the system 50.

[0047] A method for the three-dimensional detection and localization of at least one instance of at least one object in representation data of a three-dimensional scene, by comparison with at least one model of said at least one object, implemented by system 50 of the figure 3 , for example using the set of 20 models from the figure 1 for an analysis of scene 30 of the figure 2 will now be detailed with reference to the figure 4 .

[0048] It comprises a first phase 100 implementing a process for identifying an instance or object model, carried out by the processor 62 when it executes the first program 66 and the second program 68. This first phase generally has no real-time computational constraints when it proceeds from a pre-analysis of a reference scene or object models. However, it is constrained in real-time computations when it is applied to the processing of an actual scene containing at least one object instance.

[0049] It includes a second phase 200 of obtaining sets of transformation parameters in translation, rotation and possibly scaling for respective three-dimensional mappings of the selected representative points of object instances in the three-dimensional scene 30 with their arrangements in one of the models of the set 20. This second phase is implemented by the processor 62 when it executes the third program 70.

[0050] It includes a third phase 300 of automatic grouping for matching each object instance identified in the three-dimensional scene 30 with the corresponding object model in the set 20, if applicable. This third phase is implemented by the processor 62 when it executes the fourth program 72.

[0051] It includes a fourth phase 400 of selection, for each object instance, of a single group from among the group(s) obtained by implementing the third phase, advantageously exploiting the invariance information provided by implementing the first phase. This fourth phase is implemented by processor 62 when it executes the fifth program 74.

[0052] It includes an optional fifth phase 500 for verifying and refining the transformation(s) determined by automatic grouping. This fifth phase is implemented by processor 62 when it executes the sixth program 76.

[0053] Finally, it includes an optional sixth phase 600 to refine or question the group selection(s) of the fourth phase 400. This sixth phase is implemented by the processor 62 when it executes the seventh program 78.

[0054] Phases 200 to 600, unlike the first, are generally subject to strong computation time constraints.

[0055] The identification process implemented by the first phase 100 includes a first step 102 of selecting at least one representative point from the surface of an object instance within the representation data of a three-dimensional scene. This representation data is first converted, for example, into a three-dimensional point cloud located on the surface of the object instance under consideration. Preliminary processing of the point cloud to remove a number of artifacts may also be applied, notably through well-known denoising or smoothing techniques. The selection consists of retaining at least some of the points on the surface. If no actual selection is made, all points on the object surface can be retained as selected representative points. This is particularly possible when choosing a simple descriptor such as that of document WO 2020 / 201392 A1.

[0056] The first identification phase 100 includes a second step 104 of determining a three-dimensional local coordinate system at each selected representative point for an object instance. As previously mentioned, there are known methods for defining such a local coordinate system deterministically and automatically. This system advantageously includes an axis representing the normal to the surface of the object instance at the considered representative point. This step is optional if calculating the values ​​of the descriptor parameters at that point does not require the explicit determination of a local coordinate system. It is necessary in the case of the descriptor of document WO 2020 / 201392 A1.

[0057] The first identification phase 100 includes a third step 106 of calculating identifying values ​​taken by the chosen descriptor in the local coordinate system determined for each selected representative point. The identifying values ​​thus calculated for all selected points are added to the object instance identification data.

[0058] In the example of figures 1, 2 And 3 , the first step 102 of selecting representative points, the second step 104 of determining local reference points and the third step 106 of calculating the identifying values ​​taken by the chosen descriptor are carried out by executing the first program 66: online with a real-time calculation constraint for the three-dimensional scene 30 including object instances 32, 34, 36, 38, 40, 42 and offline without a real-time calculation constraint for each object model 22, 24, 26 of the set 20.

[0059] The first identification phase 100 also includes the search for at least one possible invariance by transformation in translation and / or rotation of at least one of the object models 22, 24, 26 of the set 20, in order to obtain invariance information relating to each invariance found.

[0060] For example, for each object model 22, 24, or 26, this search for invariance(s) may include a first step 108 of obtaining sets of transformation parameters for translation and rotation, for respective three-dimensional mappings of points representing a first version of this object model 22, 24, or 26 in representation data of a first reference scene including this first version, with points representing a second version of this same object model 22, 24, or 26 in representation data of a second reference scene including this second version, using said parameters of the chosen descriptor. This first step 108 assumes that the loop of steps 102, 104, and 106 has been applied to each of the two versions of each object model 22, 24, or 26 using the chosen descriptor.As is well known, each set of transformation parameters obtained in translation and rotation is advantageously associated with a confidence index (a positive real number for example between 0 and 1) indicating the quality and / or robustness of the matching.

[0061] Optionally and advantageously, the two aforementioned versions differ by the addition of predetermined noise. The representation data for the second version of object model 22, 24, or 26 in the second reference scene is obtained by adding this noise to the representation data for the first version of object model 22, 24, or 26 in the first reference scene. This prevents step 108 from simply realigning the object model with itself, without any translation or rotation. Furthermore, although step 102 may involve denoising, this does not, a priori, negate the advantage of introducing the predetermined noise, since its signature has no reason to be adapted to the denoising method. This signature is, for example, analogous to that of a sensor observing the model. It is therefore adaptable to a specific type of sensor.

[0062] Alternatively, the two aforementioned versions can differ in the viewing angle of the object model, including by adding predetermined simulated sensor noise, i.e., by a translation and / or rotation transformation between the two versions. This not only prevents step 108 from simply registering the object model on itself without any translation or rotation, but also allows for better modeling of the chosen descriptor's behavior in the presence of occlusions, for example. Optionally, a real sensor observing a reference instance of an object can be used to acquire perspectives of that object as different versions of a model of that object.

[0063] Alternatively, for deformable or articulated objects, or those whose shape and / or scale is only approximately known, the two aforementioned versions may differ in the possible deformation(s) of each object. These deformations are then sampled to potentially represent all possible deformation configurations in two or more versions. In other words, the variability of an object's possible shapes can be taken into account in the invariance information.

[0064] Alternatively, a single version of the object model 22, 24 or 26 could be used and compared to itself, without adding noise, retaining when they exist the n best parameter matches of the chosen descriptor, with n≥2, to avoid falling back on a recalibration of the object model on itself without any translation or rotation when it has at least one other invariance.

[0065] Following step 108, the search for invariance(s) may include a second step 110 involving a change of reference frame. This involves expressing each set of transformation parameters, for example, as a vector, in a representation space with a multidimensional coordinate system that includes at least one dimension for translation and at least one dimension for rotation (e.g., three dimensions for translation and three dimensions for rotation), and obtaining an invariance pattern for the object in the frame of this coordinate system. The teachings in document WO 2020 / 065177 A1 can be advantageously used for this purpose, particularly by executing the loop of steps 104 to 116 in that document. It should be noted that the representation of invariance information can be used in reference frames other than the representation space proposed in document WO 2020 / 065177 A1.In particular, in simpler environments with only two or three degrees of freedom, for example in the case of objects arranged on a plane, the coordinate system can be simplified.

[0066] For the sake of simplification, and optionally, the resulting invariance pattern can be filtered, for example by removing vectors considered outliers because they have fewer than a predetermined number (NV) of neighbors in a predetermined neighborhood of the chosen coordinate system. According to WO 2020 / 065177 A1, the removal of outliers can be accelerated by indexing the vectors in a search tree such as a kd binary, advantageously associated with a Euclidean distance.

[0067] A first example of an invariance pattern that can be obtained by executing steps 108 and 110 is illustrated on the figure 5 for the elbow pipe fitting model 24. Given that the representation space of this invariance pattern is a priori six-dimensional and that the object model 24 essentially exhibits rotational symmetries / invariances, only the rotation subspace is illustrated on the right-hand side of the figure 5 We note in particular, of course, a first grouping R1 of transformation vectors with zero rotation (0°), indicating the realignment of the model on itself, that is to say with a transformation according to the Identity function with zero rotations and translations, but also two other groupings R2 and R3 of transformation vectors with respective rotations of +180° and -180° around the principal axis of symmetry Δ of the object model 24. It should be noted that the grouping R1 is in itself an invariance information since an object whose invariance pattern of the model only presents this grouping would be an object not presenting, in view of the descriptor used, any invariance other than that without transformation implying a simple realignment of the object on itself without rotation or translation.

[0068] A second example of an invariance pattern that can be obtained by performing steps 108 and 110 is illustrated on the figure 6 for a model of an 84-hole, keyed ruler. Given that this particular model exhibits invariances (pseudo-symmetries) in translation and rotation, the two subspaces of translations and rotations are illustrated respectively in the right and lower parts of the figure 6 In the translation subspace, a complex pattern specific to the longitudinal shape of the 84 ruler model is observed. A first grouping R1' of transformation vectors with zero translation (0°), indicating the model's alignment with itself without transformation, is visible. This first grouping R1' is found in the rotation subspace, but surrounded by six other groupings R2', R3', R4', R5', R6', and R7' of transformation vectors with rotations of +180° and -180° respectively around the three principal axes (longitudinal, lateral, and transverse) of the 84 ruler model.

[0069] A third example of an invariance pattern that can be obtained by performing steps 108 and 110 is illustrated on the figure 7 for a model of a notched circular object 86 exhibiting rotational invariances (two symmetries at + / - 180° and 0° and, between these two symmetries, ten pseudo-symmetries in successive 30° steps). Since this particular model essentially exhibits rotational symmetries / invariances, only the rotation subspace is illustrated on the right-hand side of the figure 7 . We note in particular of course a first grouping R1" of transformation vectors with zero rotation (0°), indicating the realignment of the model on itself without transformation, but also eleven other aligned groupings of transformation vectors with successive rotations of 30° of the model 86 on itself.

[0070] Following step 110, the invariance search may optionally include a third step 112 for the automatic grouping of transformation vectors. Each group obtained represents an invariance under translation and / or rotation of the object under consideration and a unique set of transformation parameters for matching the first model of the object with the second model of the object. The teachings in document WO 2020 / 065177 A1 can again be advantageously used for this purpose, particularly by executing the loop of steps 118 to 136 in that document. Any other known automatic grouping method can also be considered. Grouping allows for the discretization and thus simplification of the invariance pattern by providing a concise representation, which can prove very advantageous for the subsequent three-dimensional detection and localization of object instances.A confidence index or invariance score can be associated with each group and result from taking into account, for example a sum, the confidence indices of the transformation vectors that compose it.

[0071] In a preferred embodiment, the grouping as taught in document WO 2020 / 065177 A1 can be improved, particularly because the search for invariance(s) generally does not have real-time computational constraints. Indeed, the groups obtained according to this teaching are of difficult-to-control sizes and can easily stretch due to invariances, split, or regroup depending on noise. Step 112 can then advantageously be broken down into a succession of steps illustrated in the figure 8 It should be noted that the proposed improvement refers to the figure 8 is compatible with the output of steps 118 to 136 of document WO 2020 / 065177 A1 and can therefore be substituted for it in that document. Conversely, the grouping improvement variants proposed in document WO 2020 / 065177 A1, such as the possibility of revising the invariance scores of the groups obtained, are applicable to the sequence of steps in the figure 8 .

[0072] The first step 1120 of the process of the figure 8 This is an initialization step for the enhanced automatic grouping. During this step, the number of groups, N, is initialized to 0. A search radius, RS, is predetermined in the representation space of the transformation vectors to define a priori the size of the groups and their hypersphere shape. A maximum number of groups, Nm, is imposed. Finally, a minimum fraction, Fc, of the transformation vectors whose grouping is desired is also imposed. This fraction, Fc, can be expressed as the minimum number of transformation vectors to be grouped out of the set of calculated transformation vectors, or as the minimum sum of confidence indices of transformation vectors to be grouped out of the total sum of confidence indices of the calculated transformation vectors.

[0073] Step 1120 is followed by a loop of steps 1122 to 1126 carried out to determine a score to be assigned to each vector that has not been considered an outlier.

[0074] The first step 1122 of this loop is a test to determine if there is still at least one transformation vector, not considered an outlier, to be evaluated. If so, such a transformation vector is selected and the process proceeds to a second step 1124 of the loop. If not, it proceeds to an exit step 1128 of the loop.

[0075] Step 1124 consists of determining, in the chosen representation space, which transformation vectors associated with the same object model are located inside the hypersphere of radius RS whose center is the selected vector and storing this neighborhood information in memory 54.

[0076] Next, in step 1126, the sum of the confidence indices of these neighboring transformation vectors is calculated and added to the confidence index of the selected transformation vector to form its score. The process returns to step 1122.

[0077] Step 1128 is a test performed to determine whether the maximum number Nm of groups or the minimum fraction Fc of transformation vectors whose grouping is desired has been reached. If so, the process proceeds to a final step 1130, which exits step 112, in which up to Nm hyperspherical groups of the same size RS are provided with their invariance scores calculated as in document WO 2020 / 065177 A1. If not, it proceeds to step 1132, which consists of selecting the transformation vector with the best score.

[0078] Following step 1132, a new group centered on the selected vector is created during step 1134. The number N of groups is incremented by one and the fraction of transformation vectors classified in these groups is updated to include the selected vector and its neighbors as identified in step 1124.

[0079] Then, in step 1136, for all vectors neighboring the vector selected in step 1132, their score is subtracted from that of their respective neighbors to neutralize any possibility of creating a new hyperspheric group in the neighborhood of the one just created. The process returns to step 1128 to create a new group or stops.

[0080] It should be noted that the parameters RS, Nm and Fc of the process of the figure 8 can be specifically chosen according to the application context concerned. The process of the figure 8 can also be run on several games with these parameters, the one providing the best average score being ultimately selected.

[0081] It should also be noted that in case of insufficient memory, step 1124 can also be performed, if necessary, before each execution of step 1132, with the advantageous use of a cache memory to store neighborhood information. If the cache memory is full, some neighborhood information may be discarded and will need to be recalculated later.

[0082] An example of the result of the improved grouping process of the figure 8 is illustrated on the figure 9 for the notched circular object 86 of the figure 7 in the rotation subspace. A very simplified grouping into twelve hyperspherical groups is observed, corresponding to the twelve 30° rotations of the object about itself. The two groups corresponding to the two rotations of + / - 180° and 0° exhibit the best invariance scores, while the other ten, corresponding to pseudo-symmetries, have lower invariance scores.

[0083] Following step 112, the search for invariance(s) may optionally include a fourth step 114 of calculating a disambiguating potential score in at least one area of ​​the object model under consideration by combining invariance scores calculated for each invariance found in that area.

[0084] This disambiguating potential score calculation is performed on each of several features of the considered area of ​​the object model. These features could be, for example, points, two-dimensional contours, three-dimensional contours or edgelets, values ​​of a descriptor at several points, etc., of the object model. For example, the considered area could encompass the entire object model, and the features could be all the points of the object model.

[0085] A preferred implementation method, for example, is to: calculate, for each feature, a score corresponding to the difference between this feature in the possibly noisy transformed object model and the closest feature in the untransformed model, for each transformation group obtained in step 112, then add, for each feature, the scores calculated for all transformation groups to obtain, for each feature, the desired disambiguating potential score.

[0086] In the case where the features are two- or three-dimensional contours, the score to be calculated for each transformation group is, for example, based on a comparison in terms of distance and direction between each considered contour of the transformed object model and the closest contour of the untransformed object model.

[0087] In the case where the features are surface points, the score to be calculated for each transformation group is, for example, the distance between each point considered of the transformed object model and the nearest point of the untransformed object model, possibly weighted by the absolute value of the scalar product of the normals when each point is oriented and defined in pair with its normal to the object surface.

[0088] In mathematical representation, if we denote {C i}, 1≤i≤N, the set of transformation groups Ti obtained in step 112 and {S i}, 1≤i≤N, their respective invariance scores, if we denote F j = (P j , nj ) the particularities formed by pairs of points and their vector normals in the object model and F j i = P j i n j i the closest respective features in the object model transformed by the transformation Ti, then we note d j i the distance between P j and P j i and a preferred implementation of the calculations proposes the following equation to determine the disambiguating potential score S(P j ) for each point P j: S P j = ∑ i 1 − 1 − min d j i d M d M . abs n j i ⋅ n j ⋅ S i ∑ k S k where the min() function returns the minimum between two values, the abs() function is the absolute value, and the constant dM is the maximum permissible distance given the image spatial sampling and any sensor noise. One possible value for dM is ten times the sampling value.

[0089] This disambiguating potential score is close to 0 (dark) for points with low disambiguation and close to 1 (light) for points that are disambiguating in terms of distance or angle of normals.

[0090] For example, the figure 10 illustrates the value of the disambiguating potential score at each point of the notched circular object 86 of the figure 7 . There figure 11 illustrates the value of the disambiguating potential score at each point of the 84-hole, keyed ruler model. figure 6 This score demonstrates that it is highly revealing of defects, misleading features, or other details that break the ambiguities and symmetries of an object. In a three-dimensional localization context, disambiguation ensures the correct localization of essentially symmetrical objects with only a few distinguishing elements, such as misleading features. It can also be advantageously used for other tasks, particularly defect detection, or assisting an operator working on an object by drawing their attention to important details.

[0091] Following invariance search steps 108, 110, 112 (optional), and 114 (optional), the information from the obtained invariance pattern is associated with the corresponding object model in step 116. This information can be added to the identification data of any object instance once a link is established between that instance and the corresponding model. It thus advantageously complements, in a global and canonical way, the identification that can be made of this object instance by the chosen descriptor, so that the invariance pattern can be considered a global canonical invariance descriptor complementary to the chosen descriptor.

[0092] The complementarity can be explained as follows: If the chosen descriptor describes an unambiguous area of ​​an object instance in a scene, then the correspondences involving this descriptor will be correctly located in the transformation representation space for matching the object instance with a model of that object at the precise location corresponding to the correct transformation; in this case, invariance information is of little use, or even useless; if, on the other hand, the chosen descriptor describes an ambiguous area of ​​the object instance, the correspondences involving this descriptor will not necessarily be located at the location in the representation space corresponding to the correct transformation, but also at at least one other location that is quite predictable according to the invariance pattern; in this case, invariance information proves important for determining the correct transformation;Ultimately, the useful information for identifying the object is present either in the parametric space of the chosen descriptor, or in the invariance pattern; this is all the more true when the invariance pattern is obtained precisely using the chosen descriptor.

[0093] It is important to note that the choice of descriptor is not arbitrary. It can highlight a particular shape or defect. For example, one can choose a descriptor that is invariant everywhere except for a specific part of the object under consideration. The useful information is then retained directly through the complementarity between the ambiguity of the descriptor and the invariance pattern obtained with that descriptor. For instance, a hole in a beverage can can be detected if the descriptor is suitable for describing the local curvature at each point of the can, because the hole manifests as a defect in the curvature, which would otherwise be uniform.

[0094] It should also be noted that the chosen descriptor is not limited to a descriptor of the same type as that in WO 2020 / 201392, nor even more generally to a local or surface descriptor. For example, a number of known methods compare the representation of an object instance to a database of different renderings of a model of that object by template matching, either explicitly or implicitly through a neural network or via any other equivalent artificial intelligence method using statistical learning. The learned representations or the model renderings used can also be considered descriptors within the meaning of the present invention since they describe the object in a certain way and are capable of providing pose hypotheses by matching.In this case, the different versions of an object model used to obtain its invariance pattern are advantageously the different viewpoints mentioned above. Other known methods allow for registration by edge detection and can involve descriptors in a broad sense.

[0095] In the example of figures 1, 2 And 3 , steps 108, 110, 112, 114 and 116 are carried out by executing the second program 68 for each object model 22, 24, 26 of set 20.

[0096] Obtaining transformation parameter sets for translation, rotation, and possibly scaling for respective three-dimensional mappings of selected representative points of object instances in the three-dimensional scene 30 with their arrangements in one of the models of set 20, as implemented by the second phase 200, involves a first step 202 which is executed on the same principles as step 108 and as taught, for example, in document WO 2020 / 065177 A1. This first step 202 assumes that the loop of steps 102, 104, and 106 has been applied online to the three-dimensional scene 30, which includes object instances 32, 34, 36, 38, 40, 42, and offline to each object model 22, 24, 26 of set 20 using the chosen descriptor.Each set of transformation parameters obtained in translation and rotation is advantageously associated with a confidence index (a positive real number for example between 0 and 1) indicating the quality and / or robustness of the matching.

[0097] Following step 202, the second phase of obtaining transformation parameter sets between the three-dimensional scene 30 and the assembly 20 may include a second step 204 of change of reference analogous to step 110 for the expression of each transformation parameter set in the form of a vector in a representation space for example identical to that chosen in step 110 according to the teaching of document WO 2020 / 065177 A1.

[0098] The third phase 300 includes a step 302 for the automatic grouping of the transformation vectors obtained at the end of the second phase 200. This step 302 is similar to step 112 and can be carried out in accordance with the teaching of document WO 2020 / 065177 A1 or that of the figure 8 .

[0099] Given the potential invariances of each of the objects whose instances are to be detected and located in the three-dimensional scene 30, it is quite possible to obtain several transformation vectors far removed from each other in the chosen representation space for the same object instance associated with the same object model from set 20 at the end of phase 200. It is therefore quite possible to obtain several groups of transformations for the same object instance associated with the same object model from set 20 at the end of phase 300.In fact, it is even logical to find the invariance pattern, whether before or after grouping 112, with one exception: a transformation necessary to go from the object instance in the three-dimensional scene 30 to its model in the set 20, in the representation space of the transformation vectors or groups obtained at the end of phase 200 or 300 respectively. This results in a dispersion of information that allows for the detection and localization of object instances.

[0100] The purpose of the fourth phase, 400, is therefore to select, for each object instance, a single group from among the group(s) obtained for that object instance and its model by implementing the third phase, making advantageous use of the invariance information provided by implementing the first phase. This selection involves grouping the different groups resulting from the same invariance pattern into a single meta-cluster.

[0101] To this end, the fourth selection phase 400 includes an initialization step 402 during which N transformation vectors Vi, 1≤i≤N, and N corresponding scores Si, 1≤i≤N, are retrieved from the third phase 300. Each vector Vi, for example, represents one of the N groups obtained at the end of phase 300 and is considered unneutralized. During this step, a search radius is predetermined, for example, the aforementioned RS value.

[0102] In a subsequent step 404, the process tests whether at least one unneutralized transformation vector remains. If so, it proceeds to a first loop of steps 406 to 420 intended to be executed for each of the selected unneutralized transformation vectors one after the other.

[0103] Step 406 consists of testing whether at least one non-neutralized transformation vector remains to be selected. If so, the process proceeds to step 408, which selects a selectable non-neutralized transformation vector V and initializes an aggregate score for this vector V to a=0. Let T be the transformation associated with this vector V. The process then proceeds to a second loop of steps 410 to 418, within the first loop of steps 406 to 420, intended to be executed for each of the non-neutralized transformation vectors other than the selected vector V.

[0104] Step 410 consists of testing whether there is at least one unneutralized transformation vector other than the one already selected. If so, the process proceeds to step 412, which selects an unneutralized transformation vector V' other than the one already selected. Let T' be the transformation associated with this vector V'. During step 412, the transformation U = T - 1 < .T' is also calculated.

[0105] In a subsequent step 414, the transformation vector corresponding to the transformation U is placed in the representation space of the canonical invariance patterns calculated in step 110 (or in step 112 after regrouping, if applicable). Indeed, the transformation T -1< allows the current pattern to be rectified in the canonical representation space of possible transformations of the model onto itself by invariance.

[0106] The following step, 416, is therefore a test step designed to determine whether the transformation vector corresponding to transformation U has a neighbor at a distance less than or equal to RS in the representation space of the invariance pattern of the object model under consideration. If not, the process returns to step 410 for the selection of another transformation vector, V'.

[0107] If, on the contrary, this is the case, the process proceeds to step 418, which updates the aggregate score a of V: ​​the score S' associated with the vector V' is added to a. The process then returns to step 410 for the selection of another transformation vector V'.

[0108] When all the non-neutralized transformation vectors V' other than the already selected vector V have been selected in turn for the execution of the second loop of steps 410 to 418, step 410 directs the process to a step 420 during which the agglomerated score a of V is updated one last time: it is multiplied by the score S associated with the vector V. This vector V is then no longer selectable in the first loop of steps 406 to 420 and the process returns to step 406.

[0109] When all non-neutralized transformation vectors have been selected and have obtained an aggregate score of 'a', step 406 directs the process to step 422, during which the non-neutralized transformation vector with the highest aggregate score of 'a' is selected. A new meta-group is created, and the selected transformation vector V is its representative with the score 'a'.

[0110] In a subsequent step 424, the transformation vector V selected in step 422 is neutralized, and all vectors V' that contributed to increasing its aggregate score a in step 418 are also neutralized. The process then returns to step 404.

[0111] When all transformation vectors are neutralized, step 404 directs the process to a final step 426 of the fourth phase 400. This final step provides the new meta-groups resulting from the regrouping of the groups from the third phase 300 according to the invariance pattern of each model in set 20 and the selection of a single group from each meta-group, the one with the best aggregate score. This aggregate score increases the confidence that can be placed in the detection and localization of the object instance in question.

[0112] The fourth phase has been described as being run on the transformation vectors representing the groups from the third phase (300), but it could also be run on the transformation vectors from the second phase (200), i.e., before grouping. This increases computation time but is likely to produce better results.

[0113] An example of the result of this fourth phase is illustrated on the figure 12 . This is a three-dimensional scene 30' in which several instances of the same object are arranged, corresponding to the notched circular object model 86 represented with its invariance model 88.

[0114] The subspace representing the rotations of the transformation vectors is illustrated in the left part of the figure 12 before grouping 400 groups into meta-groups. However, curved shapes can be distinguished, representing, through deformation of the invariance pattern 88 by transforming the frame of reference of scene 30' towards that of model 86, different invariance patterns, each corresponding to one of the object instances of scene 30'. These curves are clearly illustrated to materialize the groupings on the right side of the figure 12 Each transformation is associated with an object instance and offers several possible transformations, taking into account the invariances of object model 86. For each transformation, a preferred one is selected, i.e., the one whose transformation vector is associated with the highest aggregate score. In the example of object model 86, two transformations are likely to be preferred out of the twelve possible ones for each object instance, since it exhibits true 180° symmetry.

[0115] The fifth phase, 500, includes a step, 502, for verifying and refining the transformation(s) determined by the preceding phases. This is a verification with refinement well-known to those skilled in the art, which will not be detailed here. For example, it implements the method called ICP (Iterative Closest Point), which achieves very good local convergence but is highly dependent on the initialization data. This is why it is often chosen for execution during this optional verification and refinement phase. Optionally, and also as understood by those skilled in the art, this fifth phase can incorporate criteria such as the predicted visibility of certain parts of object instances based on their determined location and the known arrangement of the sensor.

[0116] Finally, the sixth phase 600, which is optionally executed to refine or challenge the group selection(s) of the fourth phase 400, aims to take advantage of the calculation of a disambiguating score, for example, the disambiguating potential score from step 114. It is only really relevant for objects whose models have strong but not total symmetries (such as the notched circular object model 86, for example, with regard to the ten 30° circular pseudo-symmetries), "defects" or misleading features (such as the ruler model 84, for example). In other words, it is really only useful if the invariance pattern of the model under consideration presents several groups at the end of step 112 of grouping of the identification phase and if the execution of step 114 reveals features with a high disambiguating potential score in this model.Thus, in the example of the notched circular object model 86, one of the successive 30° rotations other than the two 180° symmetries could be mistakenly selected at the end of the fourth phase (400) for detecting and locating an instance of this object, given that the descriptor is not necessarily very precise. The sixth phase (600) then corrects this potential error by providing a new score. However, it can also be executed without causing problems on objects whose models are fully symmetrical or, conversely, without symmetries.

[0117] In the first step 602 of this sixth phase 600, a meta-group provided as output from the fourth phase 400 and possibly refined by running the fifth phase 500, or provided as output from the third phase 300 and possibly refined by running the fifth phase 500, and relating to an object instance whose model exhibits symmetry breaks, is selected with the transformation T from the set 20 of models to the three-dimensional scene 30 that represents it, i.e., that of the group with the best aggregated score. During this same step, the transformations T' of the invariance pattern of the model under consideration, as obtained after grouping 112, are listed.

[0118] Next, during a test step 604, it is checked whether there is still a transformation T' to be selected from among the transformations listed in the previous step. If so, the process proceeds to a step 606 applying the transformation TT' to the object model under consideration.

[0119] In a subsequent step 608, the process calculates a score quantifying how closely the model transformed by TT' corresponds to the object instance under consideration in the observed three-dimensional scene. This score is, for example, based on the distance between each point of the model transformed by TT' in the observed scene, or each visible point of the model, and the nearest point of the object instance under consideration in the observed scene, possibly weighted by the absolute value of the cosine of the angle of the normals at these points. More generally, this score can be based on specific characteristics using methods similar to those described for step 114.

[0120] In a subsequent step 610, this score can be amplified in the most disambiguating areas of the object model by weighting these distances with the disambiguating potential scores calculated in step 114. That is, the differences are amplified around the most disambiguating elements of the object, those that break its symmetries or invariances. The process then returns to step 604 for the possible selection of a new transformation T'.

[0121] If, at step 604, no transformation T' remains to be selected, the process proceeds to a final step 612 of selecting the transformation TT' with the highest score. Transformation T is then replaced by transformation T.T'. If T' is the identity transformation, which is always present in the invariance pattern of an object model, this means that the transformation T selected for the meta-group considered at the end of the fourth phase 400 is confirmed. Otherwise, it means that the transformation T is being challenged by exploiting a disambiguating detail that was not correctly used, or perhaps could not be used, during the fourth phase 400 or the third phase 300.

[0122] Phase 600 thus appears as a higher-order analysis than Phase 400 (fourth phase) or Phase 300 (third phase) for object instance localization. In other words, the description scale used for object instance localization in a scene is generally relatively large compared to small details such as alignment marks or other symmetry-breaking "defects." Successive execution of Phases 400 and 600, or 300 and 600, allows us to decouple the description scale used to localize the object instance from the scale used to exploit details that might call this localization into question. This decoupling is all the more valuable because it can be advantageous to choose a detail-insensitive descriptor, particularly to reduce the influence of sensor noise. Executing Phase 6 then allows us to examine details smaller than the sensor noise.

[0123] It is clear that a process / system such as the one described above allows for the automatic integration of an object's invariances and symmetries into its description without prior knowledge of them, without any particular assumptions about their analysis, and without constraints on their nature. Indeed, the invariance information obtained does not need to be analyzed, apart from possible grouping or automatic modeling, to be usable. Contrary to what is known in the prior art, this information is neither processed separately for other purposes nor discarded.

[0124] This improved object identification allows for better localization of object instances in three-dimensional scenes, but also better detection of their defects or misleading details, in asymmetry break, when it is complemented in particular by calculations of disambiguating potential scores such as those proposed previously.

[0125] It also allows for compensating for the weaknesses and imperfections of any descriptor by providing complementary information. For example, a descriptor for which there is uncertainty in the reliability of determining the local reference frames to which it applies will reflect this uncertainty in the resulting invariance information, notably through group duplication in cases of ambiguity regarding the orientation of the normal to a surface. More generally, this improved identification allows for simplifying the descriptor used without any loss of information.

[0126] Furthermore, when possible deformations or articulations of an object model are taken into account in the development of its invariance pattern, this strengthens the robustness of the detection and localization of instances of deformable objects.

[0127] It should also be noted that the invention is not limited to the embodiment described above. Indeed, it will be apparent to those skilled in the art that various modifications can be made to it in light of the instruction just provided. In the detailed presentation of the invention given above, the terms used should not be interpreted as limiting the invention to the embodiment described herein, but rather should be interpreted to include all equivalents that a person skilled in the art can foresee by applying their general knowledge to the implementation of the instruction just provided.

Claims

1. A method (100, 200, 300, 400, 500, 600) for three-dimensional detection and location of an instance (32, 34, 36, 38, 40, 42) of an object in representation data of a real three-dimensional scene (30) by comparison with a model (22, 24, 26) of the object in representation data of a reference scene (20) including this model, this method involving prior capture using a 3D sensor (80) of the real three-dimensional scene (30) including the object instance (32, 34, 36, 38, 40, 42) to be detected and located in order to acquire representation data in the form of a three-dimensional point cloud or data that can be converted into a three-dimensional point cloud, this method being implemented by computer and comprising the following steps: - identification (100), using a descriptor, of the object instance (32, 34, 36, 38, 40, 42) by: • selection (102) of at least one point representative of the object instance (32, 34, 36, 38, 40, 42) in the representation data, • computation (106) of a plurality of identifying values taken by descriptor parameters (60) for each selected representative point, • addition (106) of the computed identifying values at each selected representative point to identification data of the object instance, • search (108, 110, 112, 114) of at least one invariance by translation and / or rotation transformation of at least one model (22, 24, 26) of the object in the representation data of said at least one reference scene (20), to acquire invariance information relating to each invariance found, and • addition (116) of said invariance information to the identification data of the object instance; - acquisition (200) of sets of translation and rotation transformation parameters for respective three-dimensional mappings of each representative point of the object instance in the three-dimensional scene with a possible arrangement of this representative point in the model; - automatic clustering (300) of said sets of transformation parameters, each cluster possibly acquired being representative of a single set of transformation parameters for matching the object instance with its model; and - selection (400, 600) of a single cluster from among the cluster(s) acquired using the invariance information.

2. The method for three-dimensional detection and location (100, 200, 300, 400, 500, 600) according to claim 1, wherein: - the search (108, 110, 112, 114) of at least one invariance comprises the following steps: • acquisition (108) of sets of translation and rotation transformation parameters for a respective three-dimensional mapping of at least one point representative of a first model of the object in representation data of a first reference scene including this first model with at least one point representative of a second model of the object in representation data of a second reference scene including this second model, using said descriptor parameters, and • expression (110) of each set of transformation parameters in a multidimensional coordinate system including at least one dimension for translation and at least one dimension for rotation, to obtain an invariance pattern of the object; and - the addition (116) of said invariance information involves adding the invariance pattern obtained to the object instance identification data.

3. The method for three-dimensional detection and location (100, 200, 300, 400, 500, 600) according to claim 2, wherein acquiring (108) the sets of translation and rotation transformation parameters is performed using said descriptor parameters (60), the values of which are computed respectively at each point representative of the first and second models.

4. The method for three-dimensional detection and location (100, 200, 300, 400, 500, 600) according to claim 2 or 3, wherein the search (108, 110, 112, 114) of at least one invariance further comprises automatic clustering (112) of said sets of transformation parameters, each cluster possibly obtained being representative of an invariance by translation and / or rotation of the object and of a single set of transformation parameters for matching the first model of the object with the second model of the object.

5. The method for three-dimensional detection and location (100, 200, 300, 400, 500, 600) according to any one of claims 2 to 4, wherein: - the representation data of the second model of the object in the second reference scene is obtained by adding predetermined noise to the representation data of the first model of the object in the first reference scene; and / or - the second model of the object as represented in the second reference scene results from a deformation and / or a translation and / or rotation transformation of the first model of the object as represented in the first reference scene.

6. The method for three-dimensional detection and location (100, 200, 300, 400, 500, 600) according to any of claims 1 to 5, wherein a disambiguating potential score is computed (116) in at least one area of said at least one model of the object by combining invariance scores computed for each invariance found in that area.

7. The method for three-dimensional detection and location (100, 200, 300, 400, 500, 600) according to any of claims 1 to 6, wherein the selection (400, 600) of a single cluster from among the cluster(s) obtained using the invariance information involves taking into account (600) a disambiguating score for each mapping of the object instance to its model.

8. A computer program (66, 68, 70, 72, 74, 76, 78) downloadable from a communication network and / or stored on a medium (64) readable by a computer (52) and / or executable by a processor (62), characterized in that it comprises instructions for executing the steps of a method for three-dimensional detection and location (100, 200, 300, 400, 500, 600) according to any of claims 1 to 7, when said program is executed on a computer that has received representation data, in the form of a three-dimensional point cloud or data that can be converted into a three-dimensional point cloud, obtained by prior capture using a 3D sensor (80) of a real three-dimensional scene (30) including an object instance (32, 34, 36, 38, 40, 42) to be detected and located.

9. A system (50) for three-dimensional detection and location of an instance (32, 34, 36, 38, 40, 42) of an object in representation data of a real three-dimensional scene (30) using a descriptor, the system comprising: - a 3D sensor (80) for preliminary capture of the real three-dimensional scene (30) including the object instance (32, 34, 36, 38, 40, 42) to be detected and located in order to obtain the representation data (56) in the form of a three-dimensional point cloud or data that can be converted into a three-dimensional point cloud; - at least one memory (54) for storing data (56) representing the three-dimensional scene (30), descriptor parameters (60), and data (58) representing at least one reference scene (20) including at least one model (22, 24, 26) of the object; and - a processor (62) for three-dimensional detection and location of the object instance in the three-dimensional scene, programmed to execute the steps of a method (100, 200, 300, 400, 500, 600) for three-dimensional detection and location according to any of claims 1 to 7.