METHOD, COMPUTER PROGRAM AND SYSTEM FOR IDENTIFYING AN OBJECT INSTANCE IN A THREE-DIMENSIONAL SCENE

Description

[0001] The present invention relates to a method for identifying an instance of an object in three-dimensional scene representation data using a descriptor. It also relates to a corresponding computer program and system.

[0002] In general, this falls within the industrial domain of recognizing rigid or quasi-rigid (i.e., non-articulated) objects in three-dimensional computer vision. Such object recognition requires comparing the representation data of the three-dimensional scene with that of at least one rigid or quasi-rigid object model, or with that of another three-dimensional scene, for example, for object detection and localization by registration. This process involves determining at least one set of spatial transformation parameters for translation and rotation to match at least one object instance in the three-dimensional scene with the corresponding object model, or with another corresponding object instance in the other three-dimensional scene.

[0003] In this three-dimensional context, an "object model" refers to a predetermined representation of a reference object. It can consist of a modeled representation, for example, by a computer, in three spatial dimensions of a real object as a three-dimensional point cloud, a mesh of faces, or any other mathematical representation (explicit, implicit, parametric, etc.) that can be converted into a three-dimensional point cloud. It can also be derived from a real-world scene capture, using one or more sensors, in which the object's representation as a three-dimensional point cloud and its arrangement are sufficiently controlled to serve as a reference.By "object instance" in a three-dimensional scene, we mean a comparable representation of an object in that three-dimensional scene, that is, a modeled representation in the form of a three-dimensional point cloud, a meshed set of faces, or any other mathematical representation that can be converted into a three-dimensional point cloud.

[0004] The practical industrial applications are numerous and extend beyond object detection and localization. They include, but are not limited to: three-dimensional localization, for example in virtual, augmented or mixed reality, of a system such as a vehicle, including an autonomous vehicle, a pedestrian, a robot or a portable device, by simultaneous localization and mapping techniques; three-dimensional reconstruction of a scene by data fusion from complementary partial observations; registration of a model in a scene; classification of objects present in a scene; three-dimensional reconstruction of a scene by three-dimensional registration of one scene onto another, one of these two scenes being a reference; analysis of a three-dimensional scene including recognition and localization of objects in that scene; identification of points of interest in a scene (for example points with a particular curvature); segmentation of surfaces in a scene according to a similarity criterion;Real-time automated inspection of industrially manufactured objects, particularly for defect detection, robotic gripping and robotic manipulation of objects in space, using at least one industrial articulated arm and / or in an assembly line, for example.

[0005] The latest application notably includes bin picking, which consists of recognizing and then moving, successively or simultaneously, using one or more robot arms, bulk objects in a production line or in a storage area.

[0006] The principle of object recognition and localization by rigid registration on a model, or by comparing scene representation data, is extensively studied in the prior art. According to a well-established processing chain, described for example in US patent document 8,830,229 B2, it comprises the following steps: obtaining translation and rotation transformation parameter sets for respective three-dimensional mappings of particular points of each assumed instance of an object in the three-dimensional scene with their arrangements in a corresponding three-dimensional model or in another corresponding object instance in another three-dimensional scene, from these point-to-point parameter sets and if at least one object instance is actually present in the three-dimensional scene, determining at least one translation and rotation transformation parameter set for the three-dimensional mapping of at least one object instance from the three-dimensional scene with at least one three-dimensional object model or at least one other object instance from another three-dimensional scene, and optionally, verifying the determined transformation(s).

[0007] As for the second and third steps, they are the subject of much research and proposed solutions in the prior art. However, the details of these two steps are not the main focus of the present invention.

[0008] The first step generally involves preprocessing the three-dimensional scene representation data obtained from one or more sensors, selecting specific points of interest to be matched in the scene and the model or another scene, calculating the values ​​of local or global descriptors to be associated with these selected points to facilitate and improve the matching process, and then performing the actual matching. This first step can also be implemented in more general applications than object detection, recognition, or localization by registration.

[0009] But in particular, in the aforementioned object recognition and localization application, the choice of descriptors, as well as the methods for evaluating and comparing them, are crucial to the relevance of the resulting transformation parameters and the success of the matching process. More generally, and beyond the aforementioned application, they also have a major impact on the complexity and processing time of the data. Furthermore, the quality of a descriptor is an even more important criterion, as it may also be desirable to use it for the selection of points of interest themselves, given that these points of interest are advantageously those with easily recognizable and / or highly discriminating descriptor values.

[0010] The proposed solutions generally suffer from limitations, either in terms of the ability to discriminate between different points of the same object instance or between different object instances in the same scene, i.e. the ability to provide sufficiently different values ​​at different points to distinguish them, or in terms of the complexity of the calculations to be performed and therefore of the calculation time.

[0011] For example, the descriptor defined and used in the aforementioned US patent 8,830,229 B2 is applied to each pair of points in a set of selected representative points on the surface of an object instance. This approach presents an inherently difficult drawback: it heuristically selects pairs of points, which is computationally expensive. Furthermore, the dimensions of this descriptor, expressed as distances and angles, are not homogeneous and therefore difficult to index within a search structure.

[0012] More generally, the various state-of-the-art descriptors have equivalent strengths and weaknesses. In seeking increasingly discriminating descriptors and focusing on their robustness, practitioners have made them increasingly complex, notably by using statistical approaches that include histogram calculations, signatures, and moments of distributions. These calculations are slow, difficult to interpret in practice, and suboptimally cover the range of values ​​that the descriptors can take.

[0013] A comparative study on this subject is presented in the article by Guo et al., entitled "3D object recognition in cluttered scenes with local surface features: a survey," published in IEEE Transactions on Pattern Analysis and Machine Intelligence, Volume 36, No. 11, pages 2270 to 2287 (November 2014). Among the descriptors identified in this study, one conceived by the same authors and called RoPS (from the English "Rotational Projection Statistics") involves selecting representative points of an object instance surface in representation data of a three-dimensional scene, determining a three-dimensional local coordinate system at each selected representative point, and calculating a plurality of identifying values ​​taken by the descriptor in the determined local coordinate system for each selected representative point.The descriptor takes the form of a multivalued, or multidimensional, parameter resulting from projections onto different planes defined in the local coordinate system to provide histogram moments expressed in 135 dimensions. In this respect, it is a volumetric descriptor of very high complexity. It exhibits interesting properties and, according to its authors, is highly discriminating and robust to noise, but at the cost of slow computation times and slow comparison of resulting values. Furthermore, the high number of dimensions reduces its robustness to noise and makes the descriptor oversized for most practical applications. Subsampling of the input data and / or the set of calculated descriptor values ​​must then be considered, but this significantly reduces performance because the heuristic used for subsampling is always imperfect.Furthermore, this descriptor is not suitable for comparing its values ​​by calculating Euclidean distances, which is generally used in known rapid search methods.

[0014] Another descriptor, defined in US patent document 8,274,508 B2, also exhibits interesting properties. Like the previous one, it involves selecting representative points of an object instance surface within three-dimensional scene representation data, determining a three-dimensional local coordinate system at each selected representative point, and calculating a plurality of identifying values ​​taken by the descriptor in the determined local coordinate system for each selected representative point. However, it is defined over a much smaller number of dimensions and is limited to surface calculations of far less complexity. More precisely, it takes the form of a multivalued parameter resulting from elevation calculations, relative to the surface of the considered object instance, of concentric zones located in a tangent plane transposed locally at each selected representative point.However, it too involves statistical calculations, making its values ​​slow to compute. Furthermore, it involves nearest neighbor searches, adding to the complexity of the calculations, with an average increase proportional to the square of the descriptor size. Moreover, elevation calculations pose a problem in areas of high curvature of the object instance under consideration, as well as at its edges.

[0015] CHUA CS ET AL: "POINT SIGNATURES: A NEW REPRESENTATION FOR 3D OBJECT RECOGNITION", INTERNATIONAL JOURNAL OF COMPUTER VISION, KLUWER ACADEMIC PUBLISHERS, NORWELL, US, vol. 25, no. 1, October 1, 1997 (1997-10-01), pages 63-85, ISSN: 0920-5691, DOI: 10.1023 / A:1007981719186 proposes a new form of point representation to describe freeform surfaces in 3D. This representation, called "the point signature", allows for a more complete description of the structural neighborhood of a point than simply using the point's 3D coordinates. Invariant under rotation and translation, "the point signature" can be used directly to hypothesize a correspondence with model points having similar signatures. Recognition is achieved by matching the signatures of the data points representing the detected surface to the signatures of the data points representing the surface of the model.The use of point signatures is not limited to recognizing a single-object scene in a small model library. It can naturally be extended to recognizing scenes containing multiple partially overlapping objects (which may also be juxtaposed with each other) in relation to a large model library.

[0016] It may therefore be desirable to provide a method for identifying an instance of an object in representation data of a three-dimensional scene using a descriptor that allows us to overcome at least some of the aforementioned problems and constraints.

[0017] Therefore, a method is proposed for identifying an instance of an object in representation data of a three-dimensional scene using a descriptor, comprising the following steps: selection of at least one representative point of a surface of the object instance in the representation data; determination of a local coordinate system at each selected representative point; calculation of a plurality of identifying values ​​taken by the descriptor in the determined local coordinate system for each selected representative point; characterized in that: the descriptor is defined at least in part by a set of points with predetermined coordinates transposable in each determined local coordinate system; and the calculation of the plurality of identifying values ​​includes, for each selected representative point and each point of the descriptor, the estimation of at least one nearest distance value between that point of the descriptor transposed in the determined local coordinate system of the selected representative point and the surface of the object instance.

[0018] Thus, by defining the descriptor based on the nearest distances between different points of a predetermined set and the surface of the object instance, we obtain a simple, discriminating, and robust surface identification. Since it does not necessarily involve statistical calculations, this descriptor is also perfectly suited to comparing its values ​​using Euclidean distance calculations and known rapid search methods. Experience also shows that a limited number of points, far fewer than the 135 dimensions of the RoPS descriptor, is sufficient for good performance. Furthermore, distance calculations pose fewer problems than elevation calculations in areas of high curvature of the object instance under consideration.

[0019] Optionally: Each selected representative point is a point on the surface of the object instance; the set of points in the descriptor forms a polygon, for example a hexagon; and for each selected representative point, the polygon is transposed into a plane locally tangent to the surface of the object instance at the selected representative point.

[0020] Optionally, the set of points in the descriptor forms a regular transposable polygon with a possible adaptation of the radius of its circumscribed circle to a local curvature parameter of the surface of the object instance at each selected representative point.

[0021] Also optional: Each selected representative point is a point on a principal axis or curve of the object instance; the set of points in the descriptor forms a polygon, for example a hexagon; and for each selected representative point, the polygon is transposed into a plane locally normal to the principal axis or curve of the object instance and containing the selected representative point.

[0022] Optionally, said at least one nearest estimated distance value is a point-to-point distance value, a point-to-plane distance value, or a combination of these distance values.

[0023] Optionally, the transposition of the descriptor's point set also includes a three-dimensional translation, a three-dimensional rotation, and a scaling change. This allows the descriptor's point set to be transposed into a local coordinate system linked to the representative point in the three-dimensional scene.

[0024] Optionally, the calculation of the plurality of identifying values ​​also includes a calculation of the confidence score of the descriptor at each selected representative point, this calculation of the confidence score being a function of the plurality of identifying values.

[0025] Optionally, the descriptor is also defined by at least one of the elements of the set consisting of: a luminance parameter; a chrominance parameter; a roughness parameter; a polarity parameter; a texture parameter; and a reflectivity parameter.

[0026] Also proposed is a computer program downloadable from a communication network and / or recorded on a computer-readable medium and / or executable by a processor, characterized in that it includes instructions for executing the steps of an identification process according to the invention, when said program is executed on a computer.

[0027] A system for identifying an instance of an object in three-dimensional scene representation data using a descriptor is also proposed; the system comprises: at least one memory for storing data representing the three-dimensional scene and predetermined coordinates of the descriptor's point set; and a processor for identifying the object instance in the three-dimensional scene, programmed to execute the steps of an identification process according to the invention.

[0028] The invention will be better understood with the aid of the following description, given solely by way of example and made with reference to the accompanying drawings in which: [ Fig.1 ] there figure 1 schematically represents an example of digital representation in three-dimensional imaging of several object models, [ Fig.2 ] there figure 2 schematically represents an example of a digital representation in three-dimensional imaging of a three-dimensional scene containing several instances of objects, [ Fig.3 ] there figure 3 schematically represents the general structure of a three-dimensional detection and localization system for at least one object instance in three-dimensional scene representation data such as those of the figure 2 , including an object instance identification system according to an embodiment of the invention, [ Fig.4 ] there figure 4 illustrates the adaptation of a descriptor according to an embodiment of the invention at a representative point of a three-dimensional scene, realized by the system of the figure 3 , And [ Fig.5 ] there figure 5 illustrates the successive steps of a three-dimensional detection and localization process for at least one object instance implemented by the system of the figure 3 , including an identification of each object instance present in the three-dimensional scene of the figure 2 and each object model of the figure 1 , according to one embodiment of the invention.

[0029] The set of 10 three-dimensional models represented on the figure 1 is an artificial scene made up of several models of artificial objects. It could also be a real reference scene made up of models of real objects whose arrangement is known and controlled. Here, it consists of three object models chosen as an illustrative example. A first three-dimensional model 12 represents an inclined T-shaped pipe fitting. A second three-dimensional model 14 represents an elbow pipe fitting. A third three-dimensional model 16 represents a profile with one end wrapped around it. These three object models are arranged and perfectly localizable in a three-dimensional reference frame linked to the assembly 10. They are represented, as previously indicated, as a three-dimensional point cloud, a meshed set of faces, or any other mathematical representation that can be converted into a three-dimensional point cloud.For obvious reasons of clarity, models 12, 14, and 16 are clearly separated from one another in set 10. However, they could advantageously be superimposed, all centered within set 10, while remaining distinguishable using distinctive labels associated with their constituent points. Advantageously, their sampling is the same or at least similar. In a simple case, set 10 could contain only a single three-dimensional model.

[0030] The three-dimensional scene 20 depicted on the figure 2 This results from a three-dimensional capture obtained using a 3D sensor of a real-world scene. It consists of three-dimensional points, or "3D points," in its own reference frame, each indicating, through shades of gray or color, the presence of real-world object instances in the scene. Here, as an illustrative example, we can distinguish two object instances 22 and 24 corresponding to the inclined T-pipe fitting model 12, three object instances 26, 28, and 30 corresponding to the elbow pipe fitting model 14, one object instance 32 of the straight T-pipe fitting type, which has no corresponding model in assembly 10, and no object instances corresponding to the profile model 16. Note that some object instances in scene 20 may only be partially visible.

[0031] In accordance with a three-dimensional detection and localization system / method for object instances 22 to 32 in three-dimensional scene representation data 20, by comparison with object models 12, 14 and 16 in set representation data 10, such as that which will be detailed with reference to figures 3 And 5A correspondence must be established between each object instance that can be identified and reconstructed in the three-dimensional scene 20 and one of the models 12, 14, and 16. This correspondence consists of finding the parameters of a rigid transformation involving translation, rotation, and possibly scaling (enlarging or reducing in size), which allows, where applicable, moving from the positioning of each object instance in scene 20 to that of its model in set 10, or conversely, from the positioning of each model in set 10 to that of a corresponding object instance in scene 20. It should be noted, however, that the example illustrated by the figures 3 And 5relates only to a particular family of possible applications of the present invention, namely the detection and localization of objects in a three-dimensional scene, the invention extending to applications beyond this particular family.

[0032] The three-dimensional detection and localization system 40 for object instances 22 to 32, as schematically represented on the figure 3 , includes a processing unit 42 associated in a conventional manner with a memory 44 (for example a RAM memory). This memory 44 stores, for example, data 46 representing the scene 20, data 48 representing the set 10 of three-dimensional models and predetermined coordinates 50 of a set of points defining at least part of a descriptor.

[0033] The processing unit 42 can be implemented in a computing device such as a conventional computer with a processor and one or more memories for storing data files and computer programs. The processing unit 42 can then itself be considered as consisting of a processor 52 for mapping instances and object models, associated with a memory 54 for storing the instructions it executes as computer programs. When at least some of the instructions can be executed in parallel, the processor can optionally be adapted for such parallelism, for example, in the form of a GPU (Graphics Processing Unit).

[0034] Among these computer programs, a first program 56 is, for example, designed to execute a method for identifying object instances in the three-dimensional scene 20 using a descriptor defined at least in part by a set of points at predetermined coordinates 50 stored in memory 44. This identification consists of calculating identifying values ​​that the descriptor takes at several specific points to be selected for each object instance considered. This same first program 56 also allows for a similar identification of object models in set 10. Its operation, through a clever choice of descriptor, is the main focus of the present invention. This first program 56, in conjunction with the processor 52 in the processing unit 42 and the memory 44, constitutes an object instance identification system according to the present invention.A second program 58, for example, is designed to execute a method for obtaining sets of transformation parameters for translation, rotation, and possibly scaling, for the respective three-dimensional mappings of selected specific points of object instances in the three-dimensional scene 20 with their arrangements in one of the models of the set 10. Such a method is known and will not be detailed. It is based on mapping the values ​​that the descriptor takes at these points in the two representations. It works all the better when the descriptor is discriminating.A third program 60, for example, is designed to perform automatic classification. Each class obtained as output represents a single object instance in the three-dimensional scene 20 and a single set of transformation parameters for matching with a single model in the set 10. Such a classification is also known and will not be detailed. It can be performed in one or more iterations depending on the number of object instances to be located and the method used. Finally, a fourth optional program 62 is designed, for example, to perform a verification of the transformation(s) determined by automatic classification. Such an optional verification is also known and will not be detailed.

[0035] The processing unit 42 as illustrated on the figure 3 Thus, it functionally comprises four computer programs or four functions of the same computer program. It should be noted that computer programs 56, 58, 60, and 62 are presented as distinct, but this distinction is purely functional. They could just as easily be grouped in any possible combination into one or more software programs. Their functions could also be at least partially microprogrammed or micro-wired into dedicated integrated circuits. Therefore, as an alternative, the computer system implementing the processing unit 42 could be replaced by an electronic device composed solely of digital circuits (without a computer program) to perform the same actions.

[0036] The installation of the figure 3 The system may include, in addition to the three-dimensional detection and localization system 40 for object instances 22 to 32, a 3D sensor 64 for obtaining representation data of the three-dimensional scene 20. Such a 3D sensor is not described in detail because it is well known to those skilled in the art. By generalization, it may consist of a combination of several sensors, for several views or detection modalities. According to one possible embodiment, the three-dimensional detection and localization system 40 is autonomous and separate from the 3D sensor 64. It then includes a (tele)communication interface 66 with this 3D sensor 64, wired or wireless, managed by the object instance and model matching processor 52. According to another possible embodiment, the system 40 could be embedded in the 3D sensor 64 or, conversely, the 3D sensor 64 could be integrated into the system 40.

[0037] An example of a descriptor conforming to the principles of the present invention and used by the first program 56 will now be detailed with reference to the figure 4 This example is simple and not exhaustive. It illustrates an example of a local descriptor.

[0038] According to a first general principle, this descriptor is defined, at least in part, by a set of points with predetermined coordinates in a three-dimensional coordinate system specific to the descriptor. According to the simplified example of the figure 4 The descriptor, for example, comprises six coplanar points forming a regular hexagon. This results in a descriptor that can be reduced to six dimensions, making it very simple to use because value calculations and comparisons are quick due to the small number of dimensions.

[0039] According to a second general principle, the three-dimensional coordinate system proper to the descriptor and the coordinates of each of its points are transposable into each local coordinate system (x, y, z) respectively linked to each selected representative point of an instance or object model surface. This transposition involves a three-dimensional translation, a three-dimensional rotation, and optionally a scaling, it being understood that each of these transformations can be the identity. It can be performed in a manner known per se using a transformation matrix. Determining a local coordinate system (x, y, z) for each selected representative point is a well-known problem, the solution to which is also known, and is not the subject of the present invention. According to the simplified example of the figure 4 Any representative point on the surface 70 of a model or object instance is a point O on this surface 70, and the descriptor is transposed into the plane (O, x, y) locally tangent to the surface 70 at point O, given that the axis (O, x) is, for example, oriented in the direction of steepest slope and that the axis (O, z) normal to the plane (O, x, y) is, for example, oriented outwards from the object instance bounded by the surface 70. The radius r = OA = OB = OC = OD = OE = OF of the circle circumscribed about the transposed hexagon ABCDEF whose center is at O ​​can be constant or adapted to a local curvature parameter of the surface 70 at O. In particular, this radius r can be proportional to the local radius of curvature of the axis (O, x) according to a predefined proportionality coefficient.The placement of the points of the hexagon ABCDEF in the plane (O, x, y) can be defined deterministically by placing point A at coordinates (r, 0) and rotating in the trigonometric direction to place the other points.

[0040] According to a third general principle, the identifying values ​​that the descriptor takes when transposed into the local frame attached to point O result from an estimation of at least one nearest distance value between each point of the descriptor transposed into the local frame of point O and the surface 70. According to the simplified example of the figure 4 The descriptor transposed into the local coordinate system (O, x, y, z) of point O takes six identifying values: DA, DB, DC, DD, DE, DF, respectively representing the closest distances between points A, B, C, D, E, F and surface 70. It should be noted that the center O of hexagon ABCDEF, which could be considered a point of the descriptor, is in fact not significant since its closest distance to surface 70 is by definition zero, regardless of how this distance is estimated. Therefore, it does not need to be stored. It can also be considered a "free" measurement, always equal to 0.

[0041] Calculating the nearest distance value from any point on the transposed descriptor may involve calculating a point-to-point distance, a point-to-plane distance, or both, for example, by combining them. In a point cloud representation of the surface 70, the point-to-point distance is the Euclidean distance between the point considered X = A, B, C, D, E, or F on the descriptor and the point on the point cloud representing the curve 70 that is closest to X. The point-to-plane distance, on the other hand, is the distance between, on the one hand, the orthogonal projection of point X onto the normal to the surface 70 at the point on the point cloud representing the curve 70 that is closest to X, and, on the other hand, this point on the point cloud that is closest to X.Furthermore, a minus sign can be assigned to the calculated distance value if the point in the point cloud closest to X is below the (O, x, y) plane in the local coordinate system (O, x, y, z), and a plus sign otherwise. The resulting distance value can then be normalized by r to bring it within the interval [-1, 1], since this is the maximum value that a point-to-point or point-to-plane distance can take in this context.

[0042] The search for the point on surface 70 that is closest to a given point in the transposed descriptor is well-suited to a fast search structure such as a kd tree, whether the distance is point-to-point or point-to-plane. Furthermore, it is advantageously independent of the descriptor size, i.e., the scale of description. An adaptation of an algorithm such as one of those taught in the article by Muja et al., entitled "Scalable nearest neighbor algorithms for high dimensional data," published in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 36, no. 11, pages 2227 to 2240 (November 2014), may be suitable.

[0043] Based on this simple search, calculating a point-to-point or point-to-plane distance is also very simple and significantly more relevant than calculating an elevation value, for example. Thus, calculating a point-to-point distance makes the edges of a surface visible but is somewhat too sensitive to surface sampling. Conversely, calculating a point-to-plane distance is particularly robust to sampling irregularities but makes the surface edges less visible. It is also more robust to the nearest neighbor approximation introduced by using a fast search structure. As for the elevation calculation, which is not used in the present invention, it suffers from the two aforementioned drawbacks (sensitivity to sampling and difficulty in making surface edges visible).

[0044] A combination of the two aforementioned distances allows for optimal use of their respective advantages. Optionally, for each point of the descriptor transposed to a selected representative point, the two point-to-point and point-to-plane distance values ​​can be calculated without particular complexity, and then the preferable one determined. This choice can be made automatically by comparing the two multivalued values ​​obtained with the two distances for the descriptor. If the difference between them exceeds a certain threshold, the point-to-point distance value that provides better management of surface edges is retained, without compromising the robustness of the sampling of the rest of the surface. If this difference remains below the threshold, the point-to-plane distance value is retained.

[0045] Alternatively, the general principle of estimating at least one nearest distance value between each point of the descriptor transposed into the determined local coordinate system of the selected representative point and the surface 70 of the object instance can be implemented as an estimation of several distance values ​​between each point of the descriptor and, respectively, several nearest points located on the surface 70 of the object instance. A statistical value (mean, median, etc.) can then be used to estimate the nearest distance between each point of the descriptor and the surface 70 of the object instance. This option improves the robustness of the descriptor, but at the cost of increased complexity. However, this complexity is independent of the descriptor's scale and does not impact other functions such as comparing descriptor values ​​or any other calculation based on these values.

[0046] Alternatively, in addition to the distance(s) between a point in the descriptor and the nearest point(s) on the object instance surface, the distances between this point in the descriptor and points on the object instance surface located in one or more predetermined neighborhoods of the nearest point(s) on the object instance surface can be calculated. Similarly, a statistical value (mean, median, etc.) can then be used to estimate the closest distance between each point in the descriptor and the object instance surface, while maintaining the independence of the complexity at the descriptor level.

[0047] In a different embodiment than that illustrated by the example of the figure 4 , a descriptor conforming to the three general principles mentioned above can be defined globally.

[0048] By adapting the first general principle, the set of points of the descriptor forms a polygon, in particular a hexagon, for example intended to extend beyond the surface 70 of the object instance that it must identify.

[0049] By adapting the second general principle, each selected representative point is a point on a principal axis or curve of the object instance, specifically an axis or curve extending within the object instance's surface 70. For each selected representative point, the descriptor is then transposed into the plane locally normal to the principal axis or curve of the object instance and containing that representative point. The resulting polygon is sized to encompass the object instance's surface 70, thus defining a global descriptor for the object instance. This sizing can be constant. The object description is thus comparable to identification by scanning. As an alternative to this embodiment, not all points of the polygon need to be outside the object instance: points inside can be represented by a negative distance, for example.

[0050] The third general principle can be adapted in the same way for this global descriptor as for the local descriptor of the figure 4 .

[0051] In all cases, the descriptor is ultimately formed as a vector function of the calculated nearest-nearest distance values. In its simplest form, each vector component of the descriptor corresponds to a calculated nearest-nearest distance value for one of the descriptor's points. In this case, the descriptor's identifying values ​​coincide with the distance values ​​at each of its points. However, in a more sophisticated manner, the distance values ​​can undergo processing after their calculation before being distributed into the descriptor's vector components. For example, they can be sorted in ascending or descending order before being distributed to obtain an ordered descriptor.The vector function can also, alternatively, result from a more complex combination of calculated distance values, notably through calculations of average or statistical moments of distance values, the results of these calculations, and no longer the distance values, then forming the identifying values ​​of the descriptor.

[0052] It should be noted that a descriptor such as one of those described above can also be defined by parameters other than the aforementioned set of points: for example, a parameter of luminance, chrominance, roughness, polarity, texture, reflectivity, etc., provided that the input data (i.e., the three-dimensional scene representation data) contains such information on luminance, chrominance, roughness, polarity, texture, reflectivity, etc. Each of these parameters can be estimated at each selected representative point to which the descriptor applies, at each transposed point of the descriptor, or in a neighborhood of these points. For example, chrominance can be one-, two-, or three-dimensional if measured at the selected representative points or in a neighborhood of them.It can have six, twelve, or eighteen dimensions if defined as the relative variation of chrominance between each transposed point of the descriptor and the selected representative points. It should be noted that describing relative variations for each point of the descriptor allows us to at least partially overcome the effects related to scene illumination and the specular properties of certain objects. Adding parameters increases the complexity of the descriptor, not only for the calculation and comparison of its identifying values, but also for memory usage. Therefore, the relevance of each added parameter should be carefully considered in relation to the intended application.For example, using the chrominance parameter can, in some cases, improve the descriptor, particularly for distinguishing objects of the same shape but with different colors or color patterns, but it can also degrade it, especially when the object being described is metallic and / or shiny, in which case the luminance parameter becomes more relevant. In the case of pronounced and visible specular properties, luminance can also degrade the descriptor, leaving only the three-dimensional information reliable.

[0053] A descriptor such as one described previously can be further associated with a confidence parameter defined in such a way as to allow the calculation of a confidence score at each selected representative point. This confidence score calculation is a function of the plurality of identifying values ​​that the descriptor takes at each selected representative point. When one wishes to describe at least two different models, the identifying values ​​that the descriptor takes for each model are calculated independently, but their confidence scores can be calculated in combination. Thus, the most discriminating identifying values ​​with respect to the set of models can be found. This makes it possible to eliminate similar parts of different models from the description.

[0054] In simplified or default version, the confidence score can be a constant (i.e., no confidence score management) or a random number (i.e., random confidence score management).

[0055] According to one version defining confidence as a function of the local curvature of the object instance's surface, the confidence score can be the sum of the squares of the identifying values ​​taken by the points of the descriptor, i.e., DA 2< + DB 2< + DC 2< + DD 2< + DE 2< + DF 2< for the example of the figure 4 (i.e., exploitation of the internal edges of the polygon), or the sum of the squares of the differences in identifying values ​​between vertices of the sides of the polygon formed by the descriptor, that is, (DA-DB)² + (DB-DC)² + (DC-DD)² + (DD-DE)² + (DE-DF)² + (DF-DA)² for the example of the figure 4 (i.e., exploiting the external edges of the polygon), or the sum total of the two previous sums to fully exploit all the internal and external edges of the polygon formed by the descriptor. This version rightly assumes that a descriptor value calculated at a representative point selected on a low-curvature area of ​​the object instance surface is less discriminating than a value calculated on an area of ​​greater curvature variation. Ultimately, if the area is flat, the resulting confidence score is zero, which confirms that a flat object instance surface is not an area that allows for the identification of that object.

[0056] According to a more elaborate version defining confidence based on a singularity of the identifying values ​​taken by the descriptor, the confidence score can be defined as the sum of values ​​taken by a distance function, calculated in the multidimensional space of possible values ​​of the descriptor, of identifying values ​​located in a predetermined neighborhood of a given identifying value. The distance function is then advantageously chosen to penalize the considered identifying value if there are many other identifying values ​​in its neighborhood and if, in general, the identifying values ​​are close to each other in the space of possible values ​​of the descriptor. It is, for example, defined as follows: f d = 1 1 + d 2 , where d is the distance, in the multidimensional space of possible values ​​of the descriptor, between the considered identifying value and another identifying value taken by the descriptor in its predetermined neighborhood.

[0057] Denoting N as the number of other identifying values ​​taken by the descriptor in the predetermined neighborhood of the considered identifying value, the confidence score then takes the following form: ∑ i = 1 N f d i = ∑ i = 1 N 1 1 + d i 2 , where di is the distance between the considered identifying value and the i-th other identifying value taken by the descriptor in its predetermined neighborhood.

[0058] This more elaborate version of the confidence score calculation is also more computationally expensive, even though fast search methods also apply to finding the nearest neighbors in the space of possible values ​​of the descriptor. However, it is particularly well-suited by design to each instance of the described object.

[0059] A method for the three-dimensional detection and localization of at least one instance of at least one object in representation data of a three-dimensional scene, by comparison with at least one model of said at least one object, implemented by system 40 of the figure 3 , for example using set 10 of the models from the figure 1 for an analysis of scene 20 of the figure 2 will now be detailed with reference to the figure 5 .

[0060] It includes a first phase 100 implementing an instance or object model identification process implemented by the processor 52 when it executes the first program 56.

[0061] It includes a second phase 200 of obtaining sets of transformation parameters in translation, rotation and possibly scaling for respective three-dimensional mappings of the particular selected points of object instances in the three-dimensional scene 20 with their arrangements in one of the models of the set 10. This second phase is implemented by the processor 52 when it executes the second program 58.

[0062] It includes a third phase 300 of automatic classification for matching each identified object instance in the three-dimensional scene 20 with the corresponding object model in set 10, if applicable. This third phase is implemented by the processor 52 when it executes the third program 60.

[0063] Finally, it includes an optional fourth phase 400 for verifying the transformation(s) determined by automatic classification. This fourth phase is implemented by processor 52 when it executes the fourth program 62.

[0064] The identification process implemented in the first phase 100 includes a first step 102 of selecting at least one representative point from the surface of an object instance within the representation data of a three-dimensional scene. This representation data is first converted, for example, into a three-dimensional point cloud located on the surface of the object instance under consideration. The selection consists of retaining at least some of the points on the surface. If no actual selection is made, all points on the object surface can be retained as selected representative points.

[0065] The first identification phase 100 includes a second step 104 of determining a three-dimensional local coordinate system at each representative point selected for an object instance. As previously mentioned, there are known methods for deterministically and automatically defining such a local coordinate system.

[0066] The first identification phase 100 includes a third step 106 of transposition of the set of points of the descriptor into the local coordinate system determined for each selected representative point, for example using a transition matrix for a three-dimensional translation, a three-dimensional rotation as well as a possible change of scale.

[0067] Finally, the first identification phase 100 includes a fourth step 108 of calculating the identifying values ​​taken by the descriptor in the local coordinate system determined for each selected representative point. As detailed previously, this calculation involves, for each selected representative point and each point of the descriptor, estimating at least one nearest distance value between that point of the descriptor, transposed into the local coordinate system determined for the selected representative point, and the surface of the object instance. The distance values ​​obtained are then directly used or processed to form the multidimensional identifying value taken by the descriptor at each selected representative point.

[0068] In the example of figures 1, 2 et 3 , the first step 102 of selecting representative points, the second step 104 of determining local reference frames, the third step 106 of transposing the descriptor into the local reference frames and the fourth step 108 of calculating its values ​​after transposition are executed for each instance of object 22, 24, 26, 28, 30, 32 of the three-dimensional scene 20 and for each model of object 12, 14, 16 of the set 10.

[0069] The second, third and fourth phases 200, 300 and 400 are not the subject of the present invention, are moreover well known to the person skilled in the art and will therefore not be detailed.

[0070] It is clear that a descriptor-based identification system / method such as the one described above, in a purely illustrative and non-limiting application context of three-dimensional detection and localization of at least one instance of at least one object in representation data of a three-dimensional scene, allows for simple, fast, discriminating, and robust surface identification using a descriptor that can have a limited number of dimensions, notably six dimensions for the simple variant of a hexagonal descriptor such as that of the figure 4Assuming, for example, that in this simple variant the six dimensions of the descriptor are each expressed in ten possible distinct values, then there are one million possible distinct values ​​for the descriptor. It is thus theoretically possible to describe up to one million representative points with this descriptor without encountering two identical descriptor values. In practice, the point clouds on which a descriptor-based identification method operates rarely exceed a few hundred thousand points. Moreover, certain surfaces, such as planes, are ambiguous by nature, so that the local descriptors of such surfaces necessarily take on ambiguous values, regardless of the number of dimensions of the descriptor. In other words, six dimensions for a descriptor according to the present invention are most often more than sufficient.The result is a compact descriptor forming an optimal compromise between speed and accuracy, linked to the fact that this descriptor is surface-based.

[0071] As previously mentioned, this descriptor is also perfectly suited to comparing its values ​​by calculating Euclidean distances and known rapid search methods, particularly due to the homogeneity of its principal dimensions.

[0072] Thus, the proposed descriptor makes it possible to perform operations previously deemed too costly or tasks previously carried out by other approaches. In particular, the aforementioned selection step 102 can be reduced to its simplest form by retaining all the points in the point cloud of an object instance to be identified, as calculating the values ​​of the proposed descriptor at each of these points is quick and easy. This allows, in particular, the description of object instances of which only a small part is seen by a sensor. Furthermore, given its discriminatory capacity, the descriptor can itself be used for a subsequent selection of representative points of interest for each object instance to be identified, retaining only the points for which the transposed descriptor has a value relatively different from the others.Such a selection is optimal because it uses the same descriptor as the one used to identify object instances. There is indeed synergy between selection and identification.

[0073] The closest distances calculated for this descriptor correspond to the physical reality of the local surfaces to which they are applied. Furthermore, they are perfectly suited to known rapid search structures, and the complexity of calculating the distance values ​​(and consequently the identifying values ​​of the descriptor) does not increase with the chosen scale of description. This allows such a descriptor to be considered at any scale, unlike state-of-the-art descriptors. Finally, in a simplified version of the descriptor, these distances are homogeneous, making comparisons of descriptor values ​​directly relevant and interpretable without the need for weighting or adjustments between dimensions to ensure compatibility.

[0074] The fact that the proposed descriptor is surface-based is also consistent with the fact that current 3D sensors generally only see surfaces and that model representations in computer-aided design are also surface-based.

[0075] Finally, the proposed descriptor incidentally constitutes a good flatness and curvature detector, allowing, for example, the reliable and robust detection of portions of flat surfaces without additional computational cost.

[0076] It should also be noted that the invention is not limited to the embodiments described above.

[0077] An example was detailed in which the descriptor is defined by a set of six points forming a regular hexagon. This is indeed a simple solution for having points equidistant from each other and from a center. But one could also choose a more complex geometric figure, with points at different distances from the center, thus allowing the description of several scales of the three-dimensional scene.

[0078] An example has also been detailed in which object instances are identified and located in a three-dimensional scene by registration against a set of object models, notably for a depalletizing application. However, the present invention applies to many other applications involving the identification of object instances using local or global descriptors, without necessarily including the registration of these instances against models, and without even necessarily involving object detection or location.

Claims

1. A method (100) for identifying an instance of an object in data representing a three-dimensional scene using a descriptor, comprising the following steps: - selection (102) of at least one representative point (O) of a surface (70) of the object instance in the representation data; - determination (104) of a local coordinate system (O, x, y, z) at each selected representative point; - computation (108) of a plurality of identifying values taken by the descriptor in the local coordinate system determined for each selected representative point; characterized in that: - the descriptor is defined at least in part by a set of points with predetermined coordinates in a three-dimensional coordinate system specific to the descriptor, that is, defined for the descriptor independently of each specific determined local coordinate system, but such that these predetermined coordinates can be transposed into each determined local coordinate system by a coordinate system transformation that maps the descriptor's own coordinate system to each determined local coordinate system; and - the computation (108) of the plurality of identifying values involves, for each selected representative point and each point of the descriptor, estimating at least one nearest distance value (DA, DB, DC, DD, DE, DF) between that point of the descriptor transposed (A, B, C, D, E, F) into the determined local coordinate system of the selected representative point and the surface (70) of the object instance.

2. The identification method (100) according to claim 1, wherein: - each selected representative point (O) is a point on the surface (70) of the object instance; - the set of points in the descriptor forms a polygon, such as a hexagon; and - for each selected representative point (O), the polygon is transposed onto a plane (O, x, y) that is locally tangent to the surface (70) of the object instance at the selected representative point (O).

3. The identification method (100) according to claim 2, wherein the set of points of the descriptor forms a transposable regular polygon, with the radius (r) of its circumscribed circle potentially adapted to a local curvature parameter of the surface (70) of the object instance at each selected representative point (O).

4. The identification method (100) according to claim 1, wherein: - each selected representative point is a point on a principal axis or curve of the object instance; - the set of points in the descriptor forms a polygon, such as a hexagon; and - for each selected representative point, the polygon is transposed onto a plane that is locally normal to the principal axis or curve of the object instance and contains the selected representative point.

5. The identification method (100) according to any one of claims 1 to 4, wherein said at least one estimated nearest distance value (DA, DB, DC, DD, DE, DF) is a point-to-point distance value, a point-to-plane distance value, or a combination of these distance values.

6. The identification method (100) according to any one of claims 1 to 5, wherein the transposition of the descriptor's set of points includes a three-dimensional translation, a three-dimensional rotation, as well as a scaling.

7. The identification method (100) according to any one of claims 1 to 6, wherein the computation (108) of the plurality of identifying values further comprises a computation of a confidence score for the descriptor at each selected representative point (O), said confidence score computation being a function of the plurality of identifying values.

8. The identification method (100) according to any one of claims 1 to 7, wherein the descriptor is further defined by at least one element from the set consisting of: - a luminance parameter; - a chrominance parameter; - a roughness parameter; - a polarity parameter; - a texture parameter; and - a reflectivity parameter.

9. A computer program (56) downloadable from a communication network and / or stored on a computer-readable medium (54) and / or executable by a processor (52), characterized in that it comprises instructions for executing the steps of an identification method (100) according to any one of claims 1 to 8, when said program is executed on a computer.

10. A system (44, 52, 56) for identifying an instance of an object in data representing a three-dimensional scene using a descriptor, the system comprising: - at least one memory (44) for storing data (46, 48) representing the three-dimensional scene and predetermined coordinates (50) of the set of points in the descriptor; and - an object instance identification processor (52) in the three-dimensional scene, programmed to perform the steps of an identification method (100) according to any one of claims 1 to 8.