Machine Learning for 3D Object Detection

Abstract

To provide a computer-implemented method of machine learning, for learning a neural network configured for encoding a super-point of a 3D point cloud into a latent vector, a device, a system, and a program.SOLUTION: An improved method for machine learning for 3D object detection comprises a step of providing a dataset of super-points. Each super-point is a set of points of a 3D point cloud. The set of points represents at least a part of an object. The method further comprises a step of learning a neural network based on the dataset of super-points. The learning comprises a step of minimizing a loss. The loss penalizes a disparity between two super-points.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to the field of computer programs and systems, and more particularly to methods, systems, and programs for machine learning for 3D object detection. [Background technology]

[0002] The market offers numerous systems and programs for designing, engineering, and manufacturing objects. CAD is an acronym for Computer-Aided Design, which refers to software solutions for, for example, designing objects. CAE is an acronym for Computer-Aided Engineering, which refers to software solutions for, for example, simulating the physical behavior of future products. CAM is an acronym for Computer-Aided Manufacturing, which refers to software solutions for, for example, defining manufacturing processes and operations. In such computer-aided design systems, graphical user interfaces play a key role in the efficiency of the technology. These technologies can be incorporated into product lifecycle management (PLM) systems. PLM refers to a business strategy that helps companies share product data, apply common processes, and leverage corporate knowledge to develop products from concept to life across the extended enterprise. Dassault Systèmes' PLM solutions (under the trademarks CATIA, ENOVIA and DELMIA) provide an Engineering Hub that organizes product engineering knowledge, a Manufacturing Hub that manages manufacturing engineering knowledge, and an Enterprise Hub that enables enterprise integration and connectivity to the Engineering and Manufacturing Hubs. The combined system provides an open object model that links products, processes and resources to enable dynamic, knowledge-based product creation and decision support that drives the optimization of product definition, manufacturing preparation, production and service.

[0003] In these and other contexts, machine learning and 3D object detection are gaining widespread importance.

[0004] However, there is still a need for improved methods of machine learning for 3D object detection. Summary of the Invention

[0005] Accordingly, a computer-implemented machine learning method is provided for training a neural network configured to encode superpoints of a 3D point cloud into latent vectors. The method includes providing a dataset of superpoints, each of which is a set of points of the 3D point cloud, the set of points representing at least a portion of an object. The method further includes training the neural network based on the dataset of superpoints. The training includes minimizing a loss, which penalizes the error between two superpoints.

[0006] The method may include one or more of the following: The loss is a reconstruction loss, which penalizes the error between a superpoint and the reconstruction of that superpoint. - the error is the distance between the superpoint and the reconstruction of the superpoint, the distance is a Chamfer distance or an Earth-Mover distance, and / or -Learning is unsupervised.

[0007] Additionally, a neural network that can be trained according to the method is provided.

[0008] A computer-implemented method of using a neural network is also provided. The method includes providing one or more first superpoints of a first 3D point cloud, the first 3D point cloud representing a 3D scene. The method further includes providing one or more second superpoints of a second 3D point cloud, the second 3D point cloud representing a 3D object. The method further includes applying the neural network to encode each of the one or more first superpoints into a respective first latent vector and each of the one or more second superpoints into a respective second latent vector. The method further includes determining a similarity between each first superpoint of the one or more first superpoints and each second superpoint of the one or more second superpoints by calculating, for each first superpoint and each second superpoint, a similarity between a respective first latent vector encoding the first superpoint and a respective second latent vector encoding the second superpoint.

[0009] The method of use may include one or more of the following: the step of providing one or more first super points includes: providing one or more initial superpoints of a first 3D point cloud; filtering one or more initial super points by selecting, from among the initial super points, initial super points that satisfy the following respectively; the error between the dimension of the superpoint and the dimension of at least one second superpoint is less than a predefined threshold, and / or The error between the position of the superpoint and the position of at least one second superpoint is smaller than a predefined threshold; The selected super points are the one or more first super points. The filtering step includes a step of selecting, from the initial super points, initial super points that satisfy the following: the distance between each dimension of the superpoint and the corresponding dimension of at least one second superpoint is less than a predefined threshold, and / or the ratio between each dimension of the superpoint and the corresponding dimension of at least one second superpoint is less than the maximum ratio and greater than the minimum ratio; and / or the difference between the relative height of the superpoint from the nearest supporting surface and the relative height of at least one second superpoint from the nearest supporting surface is less than a predefined threshold; the method further comprises determining, from among the first superpoints each of which has been determined to have a similarity to at least one second superpoint greater than a predefined threshold, one or more groups of first superpoints, each group having a shape similar to the second 3D point cloud; The step of determining one or more groups includes: providing a second superpoint graph, the second superpoint graph having a plurality of nodes and a plurality of edges, where each of the plurality of nodes represents a second superpoint, and each of the plurality of edges represents a geometric relationship between the two superpoints represented by the nodes connected by the edge, and has one or more geometric attributes of the geometric relationship; forming one or more graphs each having a plurality of nodes that each represent a first superpoint by constructing a plurality of edges, each edge connecting two nodes and each edge having one or more geometric attributes similar to the geometric attributes of the edges of the graph of the second superpoint, each graph formed corresponding to a respective group; and / or The method further comprises a step of determining, for each group, a similarity score by measuring the similarity between a first superpoint and one or more second superpoints in the group.

[0010] Further provided is a computer program comprising instructions for carrying out the method and / or the method of use.

[0011] Further provided is a device that includes a data storage medium having a computer program and / or a neural network recorded thereon.

[0012] The device may form or function as a non-transitory computer-readable medium, such as in a Software as a Service (SaaS) or other server or cloud-based platform. Alternatively, the device may include a processor coupled to a data storage medium. Thus, the device may form, in whole or in part, a computer system (e.g., the device is a subsystem of the overall system). The system may further include a graphical user interface coupled to the processor.

[0013] Embodiments of the invention will now be described by way of non-limiting examples and with reference to the accompanying drawings, in which: [Brief explanation of the drawings]

[0014] [Figure 1] 1 shows a flowchart of an example of a process incorporating the present method. [Figure 2] 1 illustrates the method. [Figure 3] 1 illustrates the method. [Figure 4] 1 illustrates the method. [Figure 5] 1 illustrates the method. [Figure 6] 1 illustrates the method. [Figure 7]1 illustrates the method. [Figure 8] 1 illustrates the method. [Figure 9] 1 illustrates the method. [Figure 10] 1 illustrates the method. [Figure 11] 1 illustrates the method. [Figure 12] 1 illustrates the method. [Figure 13] 1 illustrates the method. [Figure 14] 1 illustrates the method. [Figure 15] 1 illustrates the method. [Figure 16] 1 illustrates the method. [Figure 17] 1 illustrates the method. [Figure 18] 1 illustrates the method. [Figure 19] 1 shows an example of a system. DETAILED DESCRIPTION OF THE INVENTION

[0015] A computer-implemented machine learning method is proposed for training a neural network configured to encode superpoints of a 3D point cloud into latent vectors. The method includes providing a dataset of superpoints, each of which is a set of points of the 3D point cloud, where the set of points represents at least a portion of an object. The method further includes training the neural network based on the dataset of superpoints. The training includes minimizing a loss, where the loss penalizes the error between two superpoints. The method may also be referred to as a "training method."

[0016] The present learning method constitutes an improved method of machine learning for 3D object detection.

[0017] In particular, the training method enables the detection of 3D objects in 3D point clouds. Neural networks trained by the training method can be used in many applications, particularly those that utilize 3D object detection, including any application that utilizes 3D object detection according to the methods described below. Such applications may include (e.g., automatically) asset inventory and / or management (e.g., identifying all robots in a plant scene and moving or replacing them or parts thereof), (e.g., automatically) object removal and / or replacement (e.g., automatically replacing full scans of similar machines in a plant scene), and / or the generation of virtual twins (e.g., plant scenes). A virtual twin of a plant may consist of a CAD scene or a mix of a CAD scene and a 3D point cloud, where the virtual plant represents a twin of, for example, a scanned plant or a physical, real plant. Finally, the generation of a virtual twin of a plant scene may enable plant simulation and / or various optimizations of the plant.

[0018] Furthermore, the present learning method trains a neural network to take superpoints of a 3D point cloud as input and encode the superpoints into latent vectors. In other words, the present learning method trains a neural network to project any 3D superpoint of a 3D point cloud into a latent representation. The neural network is trained / learned by minimizing a loss that penalizes the error between two superpoints, so that the neural network encodes the input superpoints into latent vectors that are meaningful superpoint representations in terms of geometry information.

[0019] In practice, the loss takes two superpoints as input and calculates the error between them. The loss penalizes the error, so a large error tends to result in a large loss. Thus, during learning, for each superpoint used as a training sample, the superpoint is provided as input to the loss, and another superpoint is also provided as input to the loss, where the other superpoint is obtained from a latent vector encoding the training sample superpoint (i.e., the latent vector resulting from applying a neural network to the superpoint). In other words, the loss penalizes the error between the training sample superpoint and the other superpoint. The other superpoint may be, for example, a reconstruction of the training sample superpoint by decoding the latent vector encoding the superpoint. In this case, the neural network includes an encoder for encoding the superpoint into a latent vector and a decoder for decoding the latent vector into the superpoint, as will be described in more detail below. This is the case, for example, when the neural network is an autoencoder. Next, training calculates a loss value for the training sample superpoint and another superpoint, where the loss receives the paired superpoint as input. This loss value quantifies the geometric error between the superpoint received as input by the loss, i.e., the training sample superpoint and another superpoint. The geometric error is therefore the error between the decoding of the training sample superpoint and the latent vector that encodes it. The loss penalizes such errors, so the value is large when the error is large. To minimize the loss, training modifies the neural network weights when the value is large, because a large value means that the encoding of the training sample superpoint is not satisfactory enough.In other words, if this value is high, the latent vector encoding the training sample superpoint does not capture the geometry of this superpoint accurately enough, and the weights need to be modified to improve accuracy.

[0020] Thus, training by this training method tends to ensure that superpoints with similar geometries are encoded with similar latent vectors, and two superpoints with different geometries are encoded with substantially different latent vectors. In other words, the neural network is trained to project superpoints to latent representations that represent meaningful geometric information; that is, training aims for the distance between the latent vectors encoding two superpoints to be highly correlated with the geometric similarity of these two superpoints. Here, since superpoints of a 3D point cloud are a subset of points of the 3D point cloud that represent a local portion of the 3D point cloud (e.g., a local shape that forms at least a portion of an object within the 3D point cloud), the neural network trained by this training method is able to encode local portions of the 3D point cloud into geometrically meaningful latent vector representations.

[0021] In this manner, the neural network is a neural network that can be used to perform 3D object detection on 3D point clouds. Accordingly, the learning method is a machine learning method for 3D object detection. For example, given a first 3D point cloud including one or more first superpoints and a second 3D point cloud including one or more second superpoints, the neural network can encode the superpoints into latent vectors, which can then be used to calculate the similarity between the latent vectors. This makes it possible to determine which first superpoints are geometrically similar to the second superpoints, thereby detecting local shapes in the first point cloud that are similar to local shapes of the 3D object represented in the second point cloud. Ultimately, these similar local shapes can be used / processed to output 3D objects in the first 3D point cloud that are similar to the 3D object represented by the second 3D point cloud (i.e., 3D objects formed by one or more detected local shapes). For example, if a first 3D point cloud represents a plant / factory and a second 3D point cloud represents a 3D articulated robot, a neural network can be used to detect (e.g., all) 3D articulated robots in the plant that are similar to the 3D articulated robot represented in the second 3D point cloud. Furthermore, thanks to the learning performed by the present learning method, the detection is general: the neural network not only detects exact instances of the second 3D point cloud in the first 3D point cloud, but also detects objects in the first 3D point cloud that are partially similar to the object represented by the second 3D point cloud. That is, given a single 3D object represented in a 3D point cloud, the neural network can identify partially similar 3D objects in a larger 3D point cloud (e.g., a 3D point cloud with 20 times more points), such as geometric transformations (e.g., changes in rotation or position) of the single 3D object instance, occlusions of the single 3D object instance, and other types of 3D objects that are geometrically similar to the single 3D object.

[0022] Furthermore, learning does not require labeling of the training dataset. For example, learning can be unsupervised (unsupervised learning is known per se in the field of machine learning). Therefore, the trained neural network can be agnostic to the class / type of 3D object to be detected and agnostic to the detection context / background (e.g., whether the object to be detected is in an indoor or outdoor scene). For example, once trained, the neural network can be used to detect any object in any situation, including objects that are not visible during learning and / or situations that are not visible during learning. Furthermore, the neural network is agnostic to the type of point cloud to which the input superpoints belong (i.e., there are no constraints on the acquisition of the point cloud for detection to work). This makes the neural network generic and particularly robust for use in 3D object detection on 3D point clouds. For example, the neural network may be trained on a synthetic point cloud and then used to perform 3D object detection on measured 3D point clouds, i.e., 3D point clouds acquired by physical sensors.

[0023] Furthermore, the use of local learning-based representations (i.e., learning trains the neural network to encode superpoints that represent local shapes) makes the neural network robust to noise, occlusion, and point distributions, and invariant to various types of transformations (e.g., rotations) and object positions (e.g., articulated objects such as articulated robots with articulated robotic arms). This makes the neural network more robust for use in 3D object detection in 3D point clouds. In particular, the neural network may be used to detect objects in the 3D point cloud that are partially similar (i.e., not an exact instance) of a given 3D object (i.e., a query, as described below). Furthermore, the neural network detects objects independently of the object's color; the neural network detects similar shapes even when the colors are different.

[0024] Also provided are neural networks trainable according to the present training methods, e.g., neural networks trained by the present training methods, which form a computerized data structure having layers of neurons with weights (also called parameters) whose values ​​are configurable (e.g., set).

[0025] Also provided is a computer-implemented method for using a neural network. The method includes providing one or more first superpoints of a first 3D point cloud. The first 3D point cloud represents a 3D scene. The method further includes providing one or more second superpoints of a second 3D point cloud. The second 3D point cloud represents a 3D object. The method further includes applying the neural network to encode each of the one or more first superpoints into a respective first latent vector and each of the one or more second superpoints into a respective second latent vector. The method further includes determining a similarity between each first superpoint of the one or more first superpoints and each second superpoint of the one or more second superpoints by calculating, for each first superpoint and each second superpoint, a similarity between a respective first latent vector encoding the first superpoint and a respective second latent vector encoding the second superpoint. This method may also be referred to as a "detection method."

[0026] The detection method is a method for 3D object detection in 3D point clouds. The detection method constitutes an improved method for 3D object detection in 3D point clouds. In fact, the method allows for detecting first superpoints in a first 3D point cloud representing a 3D scene, which first superpoints are geometrically similar to second superpoints in a second 3D point cloud representing a 3D object. In other words, each second superpoint represents a local portion of the object, and the detection method detects local portions of the first point cloud (i.e., first superpoints) that are geometrically similar to these local portions of the object. This allows for detecting 3D objects in the 3D scene represented by the first 3D point cloud that are similar to the object represented by the second 3D point cloud.

[0027] To detect similar 3D objects, this detection method uses a neural network to encode superpoints into latent vectors and then determines the similarity between the encoded superpoints. This detection method thus benefits from the robustness of a neural network, which projects input superpoints into geometrically meaningful latent representations, as described above. The determination of similarity between two superpoints is accurate because it encodes the two superpoints and calculates the similarity between the latent vectors, which are geometrically meaningful representations of the two superpoints, as described above. In other words, if similarity is detected between the latent vectors, it means that there is corresponding geometric similarity between the two superpoints. This detection method therefore enables robust detection of similar 3D objects within a 3D point cloud.

[0028] The training method and the detection method may be performed independently, for example, by different users and / or different computers, or the training method and the detection method may be combined into the same computer-implemented process. This process is described with reference to Figure 1, which shows a flowchart of an example process.

[0029] The process includes an offline phase incorporating the training method. The offline phase includes steps S10-S20 of providing a dataset of super points according to the training method. The offline phase further includes step S20 of training a neural network based on the super points according to the training method. The offline phase may further include the step of storing the trained neural network in memory.

[0030] The process also includes an online phase incorporating the present detection method. The online phase includes steps S40-S50 of providing one or more first superpoints and one or more second superpoints according to the present detection method. Prior to this, the online phase may include a step of providing a trained neural network, for example, by retrieving the trained neural network from a memory in which it is stored after being trained in the offline phase. The online phase further includes step S60 of encoding the first and second superpoints according to the present detection method. The online phase further includes step S70 of determining similarities between the one or more first superpoints and one or more second superpoints according to the present detection method.

[0031] This learning method is further explained below.

[0032] This learning method is a machine learning method for 3D object detection in point clouds.

[0033] As known per se, a 3D point cloud is an unordered set of 3D points with (usually 3D) coordinates, optionally with additional properties such as intensity or color (e.g., encapsulated in the additional coordinates). The unordered aspect of this data makes it difficult to analyze, especially compared to a structured grid such as an image. 3D point clouds exist in various formats, depending, for example, on the sensor used to capture the 3D scan that results in the 3D point cloud, but as mentioned above, neural networks are agnostic to these various formats. Currently, prior art sensors provide 3D point clouds of millions of points, providing very high-quality, high-density 3D scans. Any 3D point cloud herein may result from such a high-density 3D scan. Point clouds can also be generated from other 3D representations, such as CAD objects or meshes. Any 3D point cloud herein may result from such generation, as will be further explained below.

[0034] 3D object detection in a 3D point cloud corresponds to the task of detecting / locating a specific 3D shape or 3D object in a 3D scene. The detection may use 3D bounding boxes, e.g., drawing a 3D bounding box around each detected object to indicate that detection has occurred. In the context of this disclosure, a 3D shape may be a 3D object or only a portion thereof. The detection involves detecting one or more shapes that are each similar to an input shape. In the context of this disclosure, similarity between 3D objects consists of both global and local geometric similarity.

[0035] As is known per se, a 3D point cloud is a set of 3D points, each of which is described by three Cartesian coordinates in 3D space. This means that each point cloud herein can be associated with a set of triplets, each of which represents the 3D coordinates of a point in the 3D point cloud. Any 3D point cloud herein may represent a 3D real scene, where a real scene is a slice of the real world (e.g., a plant scene or an office scene), and each point in the 3D point cloud represents a point in that scene. "Representing a real scene" does not necessarily mean that the 3D point cloud is derived from physical measurements performed in the real world. For example, any 3D point cloud herein may be a synthetic representation of a real scene, as discussed further below. Any 3D point cloud herein may represent or include one 3D object, or may include multiple 3D objects. Each object in a 3D point cloud herein is a set of 3D point cloud points, where each point represents a point of the 3D object as in the real world, and the set of points collectively form a representation of the 3D object in the real world.

[0036] Any 3D point cloud herein may be a synthetic point cloud, i.e., a point cloud resulting from a synthetic 3D point cloud generation process. In other words, a synthetic point cloud representing a real scene does not result from physical measurements performed on the real scene to capture a representation of the real scene, but rather forms a numerically generated virtual representation of the real scene. A synthetic point cloud generation process refers to any computer-implemented process that outputs a synthetic 3D point cloud. The generation process may, for example, generate a 3D point cloud from other 3D geometric representations, such as CAD models or 3D meshes. The generation process may, for example, be the process of "Xiangyu Yue et. al., "A LiDAR Point Cloud Generator: from a Virtual World to Autonomous Driving," 2018, which is incorporated herein by reference, and which proposes a virtual scanner simulation for extracting point clouds from 3D CAD scenes (such as video game environments).

[0037] Alternatively, any 3D point cloud herein may originate from physical measurements in a real scene. Providing such a 3D point cloud may include providing one or more physical sensors (e.g., fixed or mobile 3D scanners, such as fixed or mobile LIDARs), each configured to acquire a respective physical signal, and operating the one or more physical sensors in the real scene to acquire the one or more physical signals (i.e., scanning the real scene with each sensor). Thus, providing the 3D point cloud may include automatically determining the 3D point cloud based on measurements according to any known technique. It is noted that prior art sensors can provide point clouds of millions of points, providing very high-quality, dense 3D scans. Any 3D point cloud herein may be acquired by a sensor, such as a 3D scanner (e.g., LIDAR), which is widely used to collect 3D point cloud representations of real environments. Currently, several point cloud datasets are publicly available for outdoors, as discussed, for example, in "Timo Hackel et. al., "SEMANTIC3D.NET: A new large-scale point cloud classification benchmark," ISPRS 2017, and "Andreas Geiger et. al., "Are we ready for Autonomous Driving? The KITTI Vision Benchmark Suite," CVPR 2012 (incorporated herein by reference), and for indoors, as discussed, for example, in "Iro Armeni et. al., "3D Semantic Parsing of Large-Scale Indoor Spaces," CVPR 2016 (incorporated herein by reference). Any 3D point cloud herein may be obtained from any one of these datasets.

[0038] Any 3D point cloud herein may contain more than 10,000, 100,000, 1,000,000, 10,000,000, or 100,000,000 points. For example, a 3D point cloud representing a scene (e.g., a plant / factory scene) may contain more than 10,000,000 or 100,000,000 points. Any 3D point cloud representing a 3D object may contain about 20 times fewer points.

[0039] Any 3D point cloud herein may represent a manufacturing scene, e.g., the interior (or portion thereof) of a plant / factory. Such a scene may include objects from the manufacturing world that are represented in parts / portions of the 3D point cloud. Any 3D point cloud herein may simply represent such objects. Such objects may be (e.g., machine) parts or assemblies of parts (an assembly of parts may be considered the part itself, so a part and an assembly of parts are equivalent), or more generally, any assembly of rigid bodies (e.g., moving mechanisms). Objects may belong to various industrial sectors, such as aerospace, architecture, construction, consumer goods, high-tech equipment, industrial equipment, transportation, marine, and / or offshore oil / gas production or transportation. Thus, the object may be an industrial product, which may be any mechanical part, for example, a part for a land vehicle (including, for example, automobile and light truck equipment, racing cars, motorcycles, trucks and motor equipment, trucks and buses, trains, etc.), a part for an air vehicle (including, for example, airframe equipment, aerospace equipment, propulsion equipment, defense products, aviation equipment, space equipment, etc.), a part for a marine vehicle (including, for example, naval equipment, commercial vessels, offshore equipment, yachts and workboats, marine equipment, etc.), a general mechanical part (including, for example, industrial manufacturing machinery, heavy machinery or equipment, installation equipment, industrial machinery products, metal fabrication products, tire manufacturing products, etc.), an electric or electronic part (including, for example, consumer electronics appliances, security and / or control and / or measurement products, computing and communications equipment, semiconductors, medical devices and equipment, etc.), a consumer product (including, for example, furniture, home and garden products, leisure goods, fashion products, durable goods retailer products, textile retailer products, etc.), a packaging (including, for example, food and beverage and tobacco, beauty and personal care, household goods packaging, etc.). The object may be a manufacturing tool, such as an articulated manufacturing robot.

[0040] This learning method is for machine learning.

[0041] As is known per se in the field of machine learning, processing an input by a neural network involves applying an operation to the input, where the operation is defined by data including weight values. Training a neural network therefore involves determining the values ​​of the weights based on a dataset configured for such training; such dataset is sometimes referred to as a learning dataset or training dataset. As such, a dataset includes pieces of data that each form a training sample. The training samples represent various situations in which the neural network will be used after training. Any training dataset herein may include more than 1,000, 10,000, 100,000, or 1,000,000 training samples. In the context of the present disclosure, "training a neural network based on a dataset" means that the dataset is a learning / training dataset for the neural network, and the values ​​of the weights (also called "parameters") are based thereon.

[0042] The neural network involved in this method may be a deep neural network (DNN). DNNs are a powerful set of techniques for learning in neural networks, as described, for example, in D.E. Rumelhart et al., "Learning internal representations by error propagation, Parallel distributed processing: explorations in the microstructure of cognition," vol. 1: foundations, MIT Press, Cambridge, MA, 1986 (incorporated herein by reference). DNNs are a biologically inspired programming paradigm that enables computers to learn from observed data. DNNs have the ability to learn rich, mid-level representations, as opposed to the manually designed, low-level features (Zernike moments, HOG, Bag-of-Words, or SIFT) used in other methods (min-cut, SVM, boosting, or random forests). Therefore, the use of DNNs is particularly important for object detection / recognition. More specifically, DNNs focus on end-to-end learning based on raw data. That is, DNNs move away from feature engineering to the greatest extent possible, for example, by achieving end-to-end optimization starting from raw features and ending with labels.

[0043] In the context of the present learning method, the training dataset for the neural network is a dataset of provided superpoints, each superpoint forming a training sample of the training dataset. The training dataset is composed of superpoints, each of which is a set of points of a 3D point cloud. The superpoints may originate from one or more 3D point clouds, i.e., each point cloud containing one or more of the superpoints. A superpoint of a 3D point cloud is a set of points of the 3D point cloud, which represents a shape, i.e., at least a part of a 3D object (e.g., the entire object or its local shape). The concept of superpoints is well known. Steps S10-S20 of providing superpoints may, for example, comprise providing already calculated superpoints as they are. Alternatively, steps S10-S20 of providing superpoints may comprise step S10 of providing one or more 3D point clouds and step S20 of extracting superpoints from the provided 3D point clouds.

[0044] Step S10 of providing one or more 3D point clouds may include retrieving the 3D point clouds from a memory. Alternatively, step S10 of providing one or more 3D point clouds may include acquiring at least a portion (e.g., all) of the one or more 3D point clouds. Acquiring the 3D point cloud may include performing a synthetic 3D point cloud generation process, in which case the acquired 3D point cloud is a synthetic 3D point cloud, as described above. The generation process may, for example, generate the 3D point cloud from a CAD scene. Alternatively, acquiring the 3D point cloud may include performing physical acquisition of the 3D point cloud by operating one or more physical sensors (e.g., one or more LIDARs) in a real scene (e.g., the interior of a plant / factory), as described above. Thus, the provided 3D point cloud may consist of only synthetic 3D point clouds, only physically acquired 3D point clouds, or a mix of synthetic and physically acquired 3D point clouds.

[0045] The step S20 of extracting superpoints from the provided 3D point clouds includes, for each provided 3D point cloud, extracting one or more superpoints from the 3D point cloud. The extraction of superpoints from 3D point clouds can be performed by any known method of extracting superpoints from 3D point clouds, and such methods are well known. The method for extracting superpoints from 3D point clouds may be distance-based extraction, in which a point's neighborhood is considered (given within a certain radius or by K-nearest neighbors), as known in the art. Alternatively, the method for extracting superpoints may be geometry-based extraction, in which the point cloud is divided into geometrically homogeneous superpoints, as known in the art.

[0046] The method for extracting superpoints may be, for example, a distance-based method as described in the reference "G. Elbaz et al., "3D Point Cloud Registration for Localization using a Deep Neural Network Auto-Encoder," CVPR 2017, which is incorporated herein by reference. This method is sometimes called the RSCS (Random Sphere Cover Set) method. This method allows for the extraction of superpoints as basic matching units for the registration problem. More specifically, this method consists of an iterative procedure, in which, in each iteration, a random sphere cover set (RSCS) is extracted from a predefined radius R sphere Then, randomly select a point P that does not belong to the super point SP, and create a circle with radius R sphere We define a new SP as the set of points inside the sphere of . Superpoints are allowed to overlap, i.e., a point may be included in multiple superpoints.

[0047] Figure 2 shows the extraction of superpoints from a 3D point cloud 20 using the RSCS method (radius 0.3 m). Superpoints 201 to 209 are extracted.

[0048] Alternatively, the method for extracting superpoints may be a geometry-based method described in L. Landrieu et. al, "Cut pursuit: Fast algorithms to learn piecewise constant functions on general weighted graphs," SIAM Journal on Imaging Sciences, 2017, which is incorporated herein by reference, and which can extract superpoints for semantic segmentation problems. The method distinguishes superpoints by considering geometric features such as linearity, planarity, scattering (also described in J. Demantk et al., "Dimensionality based scale selection in 3D lidar point clouds," International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2011, incorporated herein by reference), and verticality (also described in S. Guinard et al., "Weakly supervised segmentation-aided classification of urban scenes from 3D LiDAR point clouds," ISPRS 2017, incorporated herein by reference). These features describe each point's local neighborhood based on K-nearest neighbors. The method then involves creating an adjacency graph of nearest neighbors, as described in Guinard et al., "Weakly supervised segmentation-aided classification of urban scenes from 3D LiDAR point clouds," ISPRS 2017, incorporated herein by reference.This method uses a cut-pursuit algorithm to perform minimal partitions on geometric features on a graph, as described in L. Landrieu et al., "Cut pursuit: Fast algorithms to learn piecewise constant functions on general weighted graphs," SIAM Journal on Imaging Sciences, 2017, incorporated herein by reference. This algorithm includes regularization weights to control the coarseness of the partitions. The method described in the above-cited Guinard et al., "Weakly supervised segmentation-aided classification of urban scenes from 3D LiDAR point clouds," ISPRS 2017, may first compute linear, planar, and scattering values ​​by computing PCA (principal component analysis) on the neighborhood of each point. The method then computes the eigenvalues ​​of the covariance matrix of the point locations and may use the eigenvalues ​​to estimate surface normals or derive other meaningful vectors.

[0049] Figure 3 shows the superpoint extraction from the 3D point cloud 20 already shown in Figure 2 using the geometry-based method currently described (regularization weight 0.1 m). Superpoints 301 to 308 are extracted.

[0050] Figures 4 and 5 show a comparison between superpoints obtained from the same point cloud using different methods: Figure 4 shows superpoints obtained from a point cloud 40 using a distance-based method, and Figure 5 shows superpoints obtained from the same point cloud 40 using a geometry-based method.

[0051] The linear values ​​may be given by the formula (λ1-λ2) / λ1, the planar values ​​may be given by the formula (λ1-λ3) / λ1, and the scattered values ​​may be given by the formula λ3 / λ1, where λ1, λ2, and λ3 are the eigenvalues ​​of the covariance matrix of the point's location. Figure 6 shows the linear values ​​of a point in a 3D point cloud, Figure 7 shows the planar values, and Figure 8 shows the scattered values.

[0052] The present method may preferably use a geometry-based method for superpoint extraction, such as the method discussed above, to benefit from the advantages of this type of method. In such methods, superpoints tend to correspond to only one uniform part of the point cloud (due to the cuts obtained in the adjacency graph), making them more convenient for subsequent processing, and superpoints tend to be part of a single object, depending on the regularization weights of the cuts. This leads to interesting properties, such as the tendency for superpoint extraction to be consistent between superpoints extracted from isolated point cloud objects and superpoints extracted from the same object (or similar objects) in the scene (i.e., together with other objects). Also, superpoints obtained in this way regroup points with similar geometric properties, simplifying the complexity required for the training step.

[0053] Next, step S30 of training the neural network will be described.

[0054] A trained neural network is a neural network configured to receive superpoints as input and encode them into latent vectors, i.e., vectors that form a latent encoding of the superpoints that is geometrically meaningful, but has a lower dimensionality than the superpoints. The neural network is trained to do so; that is, its weights are modified during training until the neural network performs this task well. The neural network may include an encoder, which is a neural network configured to take superpoints as input and project them into a latent vector space. In other words, the encoder is trained to compress the input superpoints while tending to preserve their properties. The encoder is trained using a loss to project them into a latent representation that is meaningful in terms of their geometric properties. For example, the training is such that a loss correlates the similarity between two superpoints and the similarity of the encodings of these superpoints. The neural network may be or include any neural network that receives superpoints (or more generally, a 3D point cloud) as input and projects the superpoints into a low-dimensional space while being invariant to object transformations such as rotation. For example, the architecture of the neural network may be that described in R. Spezialetti et. al., "Learning an Effective Equivariant 3D Descriptor Without Supervision," ICCV 2019, which is incorporated herein by reference.

[0055] The step S30 of training the neural network may also include training a decoder, e.g., simultaneously with training the encoder, where the decoder is a neural network configured to receive a latent vector as input and decode it into a superpoint. In other words, the decoder performs reconstruction of the superpoint (e.g., encoded by the encoder) based on the input latent vector. Thus, a neural network trained by this method may be comprised of an encoder configured to receive an input superpoint as input and encode it into a latent vector, and a decoder configured to receive a latent vector as input and decode it into a superpoint.

[0056] For example, the neural network may be an autoencoder including an encoder and a decoder. The architecture of an autoencoder is well known in the field of machine learning and is described, for example, in the reference "G. Elbaz et al., "3D Point Cloud Registration for Localization using a Deep Neural Network Auto-Encoder," CVPR 2017, which is incorporated herein by reference. As is known per se in the field of machine learning, an autoencoder compresses the information of its input (i.e., superpoints in the autoencoder of the present disclosure). More specifically, an autoencoder is trained to project the input (herein, the input superpoint) into a latent representation (e.g., several times smaller than the input size) and reconstruct the input from this representation. For multiple different superpoints, the autoencoder compresses the information while attempting to preserve all the characteristics of the input superpoints. Therefore, the learning of an autoencoder implicitly leads to the projection into a latent representation that is meaningful in terms of geometric properties. For example, the learning is such that there is a correlation between the similarity between two superpoints and the similarity of the encoding of those superpoints.

[0057] The training step S30 includes minimizing a loss. The loss penalizes the error between two superpoints. That is, the loss takes two superpoints as input and calculates the error between them, i.e., quantifies their geometric error. The more geometrically different the superpoints are, the higher the loss value for these superpoints. The training step S30 uses the loss to measure whether the input superpoints of the training dataset are encoded accurately enough by the latent vectors when a neural network is applied to the input superpoints. Specifically, the training step S30 may include the following iterations: -Selecting super points in the training dataset, Applying a neural network or neural network encoder to encode the superpoints into latent vectors; For example, reconstructing another superpoint from the latent vector, decoding the latent vector into another superpoint, or obtaining another superpoint from the latent vector by applying a neural network decoder to the latent vector. - giving the super point and another super point as input to the loss and calculating the loss value of the super point and another super point; -Evaluate whether the calculated value is too high, i.e., whether the calculated value is higher than a predefined threshold (e.g., convergence threshold). A high loss value means that the latent vector does not yet encode the superpoint with sufficient accuracy, i.e., the latent vector does not yet have enough geometric meaning as a latent representation of the superpoint. If the calculated value is too high, change the neural network weights, for example by changing the encoder and / or decoder weights.

[0058] The above steps may be repeated as long as a convergence criterion (known per se in the field of machine learning) for the training step S30 is not reached and / or as long as a sufficient (i.e. large enough) number of superpoints of the dataset have not been selected. The selection of superpoints during the iterations may be such that each superpoint of the dataset is selected at least once. Any known training strategy may be used for the selection.

[0059] The loss may be a reconstruction loss, which penalizes the error between a superpoint and the reconstruction of that superpoint. A reconstruction of a superpoint is another superpoint. The reconstruction of a superpoint is obtained from a latent vector encoding that superpoint, i.e., a latent vector obtained by applying a neural network (or its encoder) to that superpoint. The reconstruction of a superpoint may, for example, be the result of applying the decoder of a neural network to the latent vector encoding that superpoint. This is the case, for example, when the neural network is an autoencoder including an encoder and a decoder, as described above. The reconstruction loss can be used to evaluate whether the reconstruction of a superpoint encoded by a neural network is sufficiently similar geometrically to the superpoint, which implicitly evaluates whether the superpoint is encoded into a latent vector that is geometrically sufficiently meaningful. The reconstruction loss also ensures accurate reconstruction by the decoder.

[0060] The error may be any quantification of the geometric error / dissimilarity between two superpoints. For example, the error may be the distance between the superpoints. In other words, the loss may be a reconstruction loss that penalizes the distance between the superpoints and the reconstruction of the superpoints. The distance may be any distance between the superpoints, such as any distance between 3D point clouds. For example, the distance may be the Chamfer distance, which saves the computational resources required to perform the training step S30. The Chamfer distance (d CD ) is calculated using the following formula:

number

number

[0061] Prior to the training step S30, the training method may include a step of adapting the size of the superpoints. That is, prior to the training step S30, the training method may include subsampling and / or upsampling the points of each superpoint in the training dataset so that each superpoint matches the input size of the neural network. For example, after subsampling and / or upsampling, the points of each superpoint may include N points, each with its own 3D coordinate. In such a case, the size of the latent vector may be equal to N / 2. Figure 9 shows a schematic diagram of an autoencoder with an input size N = 128 and a latent vector size N / 2 = 64.

[0062] As mentioned above, the training step S30 involves changing the weights (also called parameters) of the neural network until the loss is sufficiently low for (e.g., all or most) samples (i.e., superpoints) in the training dataset. These concepts are known per se in the field of machine learning. The neural network may be a training model of a deep learning model (also called a DNN), which generally has millions of parameters that characterize these models and whose values ​​cannot be set manually. These parameters are set during model training using a learning algorithm. When the learning algorithm is updating the model parameters, the model is said to be in "training mode." This consists of continuously "correcting" the model according to the output for each input, thanks to the inherent characteristics of each input and, optionally, any associated annotations. Once the model is trained, parameter updates stop. The model is then used only in the online phase to process new inputs (i.e., inputs not seen during training mode) and return results, and is said to be in "testing mode." In the present disclosure, the model (i.e., neural network) may be a DNN, so these considerations may apply to the model.

[0063] The training step S30 may be unsupervised, i.e., the training step S30 may be unsupervised learning. In the unsupervised learning paradigm of 3D point cloud feature extraction, no annotations are provided for the point clouds used during training. This means that feature extraction relies only on the point cloud features (e.g., 3D point coordinates) and is independent of labeling (e.g., extracting similar features for all point clouds from the same labeled category). Using such training / learning, the trained neural network tends to be agnostic to the type of point clouds it receives as input, the class / category of objects or scenes represented by the point clouds, and / or their context (e.g., indoor or outdoor).

[0064] The detection method is further described below. The steps S40-S50 of providing one or more first superpoints and one or more second superpoints may include providing already calculated first superpoints and second superpoints directly (e.g., without calculation). Alternatively, the steps S40-S50 of providing may include a step S40 of providing a first 3D point cloud and a second 3D point cloud, and a step S50 of extracting one or more first superpoints from the first 3D point cloud and one or more second superpoints from the second 3D point cloud. The extracting step S50 may be performed using, for example, the same distance-based or geometry-based method as the extracting step S20 described above. The step S40 of providing a first 3D point cloud and a second 3D point cloud may be performed as the providing step S10 described above.

[0065] The first 3D point cloud represents a 3D scene. The scene is a real scene, i.e., a portion of the real world. The first 3D point cloud may be measured, i.e., result from physical measurements, e.g., as described above, the first 3D point cloud may be obtained by operating one or more physical sensors (e.g., LIDAR) over the scene as in the real world. Alternatively, as described above, the 3D point cloud may be generated from a virtual 3D scene (e.g., a CAD environment). The 3D scene may be a manufacturing scene, e.g., the interior of a plant / factory or a portion thereof. Thus, the 3D scene may include 3D objects from the manufacturing world, e.g., machine parts being manufactured or manufactured, or manufacturing tools such as articulated manufacturing robots.

[0066] The second 3D point cloud represents a 3D object, for example, a 3D object from the manufacturing world. The second 3D point cloud may be synthetically generated as described above, or may result from physical measurements, i.e., a measured 3D point cloud. The 3D object may be similar or partially similar (e.g., up to occlusion and / or movement such as rotation) to one or more 3D objects in the first 3D point cloud. The detection method detects these 3D objects or at least parts of them, as described below.

[0067] The one or more second superpoints may also be referred to as a "query" because they are superpoints used by the detection method to search for similar superpoints in the first 3D point cloud, as described below. This means that the detection method detects 3D shapes / objects similar to the query 3D shape / object. The query may be provided by a user. The first point cloud may also be referred to as a "search area" because the detection method searches the first point cloud for objects similar to objects in the query area. The first 3D point cloud may be a large point cloud, for example, the ratio of the number of points in the 3D point cloud to the memory size of the computer executing the method may be greater than a predefined threshold. For example, the first 3D point cloud may include approximately 100 million or more points for a memory size of 16 gigabytes (corresponding to a standard or inexpensive computer) and / or may include 20 times the number of points of the second 3D point cloud.

[0068] The detection method determines the similarity between the extracted superpoints. To reduce the number of extracted superpoints for which similarity is determined, steps S40-S50 of providing one or more first superpoints may include optional pre-filtering, as described below.

[0069] Specifically, the steps S40-S50 of providing one or more first super points may include providing one or more initial super points of the first 3D point cloud by step S50 of extracting from the first 3D point cloud provided in S40, as described above. Then, the steps S40-S50 of providing may filter the one or more initial super points by respectively selecting, from among the initial super points, initial super points that satisfy the following: the error between the dimension of the superpoint and the dimension of at least one second superpoint is less than a predefined threshold, and / or - the error between the position of the superpoint and the position of at least one second superpoint is smaller than a predefined threshold.

[0070] The selected Super Points constitute one or more first Super Points.

[0071] Therefore, the providing steps S40-S50 may first provide initial superpoints, for example by extracting them from the first 3D point cloud and selecting from among them those that are sufficiently geometrically similar to at least one second superpoint. This allows filtering the superpoints of the first 3D point cloud, which may contain many superpoints (e.g., the initial superpoints may collectively cover all points of the first 3D point cloud). As a result, the detection method only needs to evaluate the similarity between the first and second superpoints of the filtered (i.e., smaller) set.

[0072] The selection of the initial superpoints, each of which is geometrically sufficiently similar to at least one second superpoint, is based on the error between the dimensions and / or positions of the initial superpoint and the second superpoint. Specifically, for each initial superpoint, the detection method evaluates whether the error between a dimension (e.g., each dimension) of the initial superpoint and a dimension (e.g., each dimension) of at least one second superpoint is less than a predefined threshold, and / or the detection method evaluates whether the error between the position of the initial superpoint and the position of at least one second superpoint is less than a predefined threshold. The method then selects, as one or more first superpoints, initial superpoints for which the error between the dimension of the initial superpoint and the dimension of at least one second superpoint is less than a predefined threshold, and / or initial superpoints for which the error between the position of the initial superpoint and the position of at least one second superpoint is less than a predefined threshold. Each error may be any quantification of the geometric error, such as a distance or a ratio.

[0073] The filtering may include selecting, from among the initial superpoints, initial superpoints that respectively satisfy the following: the distance between each dimension of the superpoint and the corresponding dimension of at least one second superpoint is less than a predefined threshold, and / or the ratio between each dimension of the superpoint and the corresponding dimension of at least one second superpoint is less than the maximum ratio and greater than the minimum ratio; and / or The difference between the relative height of the superpoint from the nearest supporting surface and the relative height of at least one second superpoint from the nearest supporting surface is less than a predefined threshold.

[0074] The distance between each dimension, the ratio between each dimension, and the relative height difference are each a quantification / measurement of the geometric error between the superpoints. The dimensions of the superpoints may be the dimensions of the superpoints in a local coordinate reference frame (LRF, as described in J. Yang et. al., “Toward the Repeatability and Robustness of the Local Reference Frame for 3D Shape Matching: An Evaluation,” in IEEE Transactions on Image Processing, vol. 27, no. 8, pp. 3766-3781, Aug. 2018, which is incorporated herein by reference).

[0075] For each initial superpoint, if the distance between each dimension of the superpoint and the corresponding dimension of at least one second superpoint is less than a predefined threshold, the detection method may select that initial superpoint, which may be selected, for example, by determining whether there are second superpoints for which the distance between each dimension of the initial superpoint and the corresponding dimension of the second superpoint is less than a predefined threshold. To do so, the method may determine, for each dimension i of the second superpoint, whether there are second superpoints that satisfy:

number

number

[0076] Additionally or alternatively, if the ratio between each dimension of the superpoint and the corresponding dimension of at least one second superpoint is less than a maximum ratio and greater than a minimum ratio, the detection method may select the initial superpoint, for example by determining whether there is a second superpoint for which the ratio between each dimension of the initial superpoint and the corresponding dimension of the second superpoint is less than a maximum ratio and greater than a minimum ratio. To do so, the method may determine, for each dimension i of the second superpoint, whether there is a second superpoint for which:

number

[0077] The closest supporting surface of a superpoint is a plane having as its normal axis the vertical axis or any vector sufficiently close to the vertical axis (i.e., the norm of its cross product with the vertical axis is lower than a predefined threshold). Thus, the closest supporting surface may represent a floor or other real-world physical support for the shape represented by the superpoint, such as a table, desk, conveyor, or counter. The detection method may calculate or detect the closest supporting surface of the superpoint involved in the detection method to select from among the initial superpoints. The detection of the closest supporting surface may be performed by any known method. For example, the closest supporting surface may be automatically detected during the geometry-based superpoint extraction described above by performing PCA on the neighborhood of each point, performing a minimum cut on an adjacency graph, and detecting superpoints with horizontal plane characteristics (e.g., using the plane value for the horizontal plane of each superpoint).

[0078] That is, for each initial superpoint, the detection method may use one or more of the distance between the dimensions, the ratio between the dimensions, and the difference between the relative heights to select the initial superpoints, which then form one or more first superpoints.

[0079] Next, the detection method encodes each first superpoint into a respective first latent vector by applying a neural network or its encoder to the first superpoint (S60). The method also encodes each second superpoint into a respective second latent vector by applying a neural network or its encoder to the second superpoint.

[0080] Next, the detection method determines (S70) the similarity between each first superpoint of one or more (e.g., all) first superpoints and each second superpoint of one or more (e.g., all) second superpoints. To do so, for each first superpoint of one or more (e.g., all) first superpoints and each second superpoint of one or more (e.g., all) second superpoints, the method calculates the similarity between each first latent vector encoding the first superpoint and each second latent vector encoding the second superpoint. All first and second superpoints may be involved in the similarity determining step S70. In particular, if the detection method considers pre-filtering of the first superpoints, all first and second superpoints may be involved in the similarity determining step S70 because, as mentioned above, the pre-filtering already guarantees the relevance of the superpoints for detecting geometric similarity. Therefore, involving all such superpoints is important for 3D detection while being relatively economical in terms of the computational resources used in the determining step S70.

[0081] Similarity can be any measure / quantification of the similarity between latent vectors. Here, since latent vectors are geometrically meaningful latent representations of the superpoints they encode, calculating the similarity between them amounts to determining the geometric similarity between the superpoints. Similarity can be calculated using the following kind of cosine similarity cs(e i ,e j ) can also be:

number

[0082] The similarity calculation results in respective similarity scores between the first superpoint and the second superpoint, each of which is a similarity value between two latent vectors (i.e., the value calculated in S70). Thus, the determining step S70 results in similarity scores between the first superpoint representing a local portion of the shape of the first 3D point cloud, i.e., a 3D object therein, and the second superpoint representing a local portion of the shape of the second 3D point cloud, i.e., a 3D object represented by the second 3D point cloud. The method may further include evaluating similarity scores that are higher than a predefined threshold among the calculated similarity scores. Each of these similarity scores represents a pair of the first superpoint and the second superpoint for which the similarity value between the two latent vectors to be encoded is higher than the predefined threshold. In this case, the first superpoint is said to match the second superpoint. The predefined threshold may be defined relative to the projection characteristics of the encoder and / or the level of similarity desired by the user. The predefined threshold may be obtained automatically based on a dataset of superpoints that have been manually annotated as similar (e.g., for a target application). In this case, the threshold may be obtained by calculating the minimum similarity obtained between two superpoints labeled as similar. With this automatic determination, the predefined threshold may be [0.7;1] (which can be selected, for example, depending on the desired similarity and / or the architecture used), and may be, for example, equal to 0.8.

[0083] Therefore, the detection method, after the step S70 of determining, determines a set of n first matching super points (SP rz 1 ,…,SP rz n ) to the corresponding matching / similarity score mq with the matched superpoints in the query. i ((mq i ,…,mq m ),…,(mq i,…,mq k )) may be served together.

[0084] The detection method may further include a step S80 of grouping the matched first superpoints together such that the first superpoints are classified into a plurality of groups, each group representing a 3D object of the first 3D point cloud that is similar to a 3D object of the second point cloud formed by the second superpoints (i.e., the query). Thus, after determining the local shapes of the first 3D point cloud (i.e., the matched first superpoints) that are similar to the local shapes of the query (i.e., the second superpoints), by the step S80 of grouping the matched first superpoints, the detection method finds 3D objects in the first 3D point cloud that are geometrically similar to the 3D object represented by the query.

[0085] The grouping step S80 may consist in determining one or more groups of first superpoints from among the first superpoints (i.e., from among the matched first superpoints), each of which has a determined similarity to at least one second superpoint greater than a predefined threshold, each group having a shape similar to the second 3D point cloud. In other words, the determining step S80 may compare geometric properties of the matched first superpoints, such as their orientation and / or position relative to each other and / or a reference point (e.g., the center of the query), with corresponding properties of the query superpoints, thereby grouping the matched first superpoints into groups each geometrically similar to the query. The detection method may perform the determining step S80 using any known grouping method. The grouping may be rotation-invariant, i.e., the grouping detects groups of first superpoints similar to the query even when the groups represent rotations of the 3D object represented by the 3D point cloud.

[0086] Here, an example of the grouping step S80 will be described.

[0087] In one example, the step S80 of determining one or more groups can be performed according to a method for computing a graph from a set of superpoints in a query, which is described next. This method generates a complete graph of the following type: G q =(SPs q ,E), where SPs q are nodes representing superpoints, and E is a set of edges each connecting two nodes. The method includes providing (e.g., determining by any known method) a graph of second superpoints. The graph of second superpoints has nodes and edges, each edge connecting two nodes. Each node represents a second superpoint. Each node may have attributes, which may be one or more of the features and / or dimensions of the encoder. Each edge represents a geometric relationship between the two superpoints represented by the nodes it connects. The geometric relationship may be the relative position and / or orientation of the two superpoints with respect to each other. The edges have one or more geometric attributes of the geometric relationship. The attributes may form data associated with the edges. The attributes may include one or more of the following: the distance between the geometric centers of two superpoints represented by nodes connected by an edge, e.g. a reasonable distance when the superpoints belong to a 3D point cloud representing an articulated object (e.g. an articulated manufacturing robot), and / or - The relative orientation between the surfaces of two superpoints represented by nodes connected by an edge, e.g., a plausible orientation when the superpoints belong to a 3D point cloud representing an articulated object (e.g., an articulated manufacturing robot).

[0088] Providing the graph may include, for example, determining (e.g., calculating) geometric attributes for each edge. For example, the surface of the superpoint may be obtained by local coordinate reference frame (LRF) estimation, as known per se in the art, such as the LRF estimation implemented in J. Yang et. al., “Toward the Repeatability and Robustness of the Local Reference Frame for 3D Shape Matching: An Evaluation”, in IEEE Transactions on Image Processing, vol. 27, no. 8, pp. 3766-3781, Aug. 2018, which is incorporated herein by reference.

[0089] In addition to providing a graph, a method for computing a graph from a set of superpoints in a query includes forming one or more graphs. Each graph has nodes and edges, each connecting two nodes. Each node represents a first superpoint. Each edge represents a geometric relationship between the two superpoints represented by the nodes to which the edge connects. The geometric relationship may be of the same nature as the geometric relationship in the provided graph, e.g., the relative position and / or orientation of the two superpoints. Forming the one or more graphs is performed, e.g., for all graphs simultaneously or graph by graph, by constructing edges for each graph. Each constructed edge connects two nodes (i.e., each representing a matching first superpoint). Each constructed edge has one or more geometric attributes similar to the geometric attributes of an edge in the graph of the second superpoint. For example, constructing edges may include identifying a subgraph of the provided graph and reconstructing the subgraph with respect to the first superpoint, e.g., in a bottom-up manner.

[0090] Specifically, constructing edges may include iteratively searching for matching first superpoints and, for each searched first superpoint, finding one or more valid edges connecting a node representing the searched first superpoint to another node representing another first superpoint. A "valid edge" refers to an edge in the provided graph that has attributes that match those of an edge connecting two nodes representing two second superpoints that match two first superpoints. The step of finding an edge may include evaluating whether, if an edge is constructed, the edge has similar attributes to an edge in the provided graph that connects two nodes representing two second superpoints that match two first superpoints (which would be connected by the edge if constructed). If the attributes are similar, constructing the edge as a result of the discovery. Thus, constructing edges may include calculating those attributes, which may be performed in the same way as for the provided graph. "Similar attributes" or "matching attributes" means, in examples, that the attributes are equal or substantially equal, or that the difference (e.g., measured by distance and / or ratio) is lower than a predefined threshold (e.g., defined with respect to the number of points in the query). The step of finding valid edges can be optimized, for example, by comparing only edges of superpoints that are separated by a distance smaller than the maximum valid distance between two superpoints of the query (or smaller than the expected maximum size of the query).

[0091] Each graph formed results from the construction of edges (i.e., the construction of edges results in the formation of one or more graphs). Each graph corresponds to a respective group of first superpoints, i.e., the nodes of the graph represent the superpoints of the group. The edges of this graph are respectively similar to the edges of the provided graph.

[0092] The above-described example of the grouping step S80, i.e., the example of the grouping step S80 by providing a graph of second superpoints and forming one or more graphs of first superpoints, is particularly robust because it can form groups of first superpoints that each represent 3D objects in the search area that are similar to the 3D object in the query, regardless of whether the 3D objects are non-articulated or articulated objects.

[0093] In another example, step S80 of determining one or more groups may be performed by using a Hough voting method for object centers, as described in reference "F. Tombari et. al., "Hough Voting for 3D Object Recognition under Occlusion and Clutter," IPSJ Transactions on Computer Vision and Application, March 2012, which is incorporated herein by reference. This method attempts to identify possible similar object centers by voting for each superpoint of the search area according to one or more superpoints of the matched query. This may include capturing the relative orientations of the query superpoints and the search area superpoints to vote in the same direction and orientation. Such relative orientation can be obtained via LRF estimation, such as that implemented in J. Yang et al., “Toward the Repeatability and Robustness of the Local Reference Frame for 3D Shape Matching: An Evaluation,” in IEEE Transactions on Image Processing, vol. 27, no. 8, pp. 3766-3781, Aug. 2018, which is incorporated herein by reference.

[0094] As known from the above-cited reference "F. Tombari et. al., "Hough Voting for 3D Object Recognition under Occlusion and Clutter", IPSJ Transactions on Computer Vision and Application March 2012", the Hough Voting method may include a step of defining a 3D grid that decomposes the 3D space of the search area into a set of voxels. The optimal size of the 3D grid can be defined with respect to the size of the query and the accuracy of estimating the LRF votes. In one example, the size of a voxel can be obtained by the following formula:

number

[0095] As known from the above-cited reference "F. Tombari et. al., "Hough Voting for 3D Object Recognition under Occlusion and Clutter," IPSJ Transactions on Computer Vision and Application, March 2012," the Hough voting method may then aggregate the voting voxels, for example, by counting them per voxel against the number of distinct matching superpoints of the query. This may include determining voxels that represent centers of possible objects similar to the query using a threshold defined on the number of superpoints in the query. This threshold controls the number and variability of reasonable occlusions that can be targeted for a particular part of the object. Therefore, this threshold is directly accessible as a parameter that can be modified by the user during inference / detection.

[0096] 10-12 illustrate the Hough voting method. FIG. 10 shows the point cloud already shown in FIG. 2, from which superpoints have been extracted. These superpoints represent the query. The distances from the query center and the center of each superpoint are shown in FIG. 10. For example, reference 100 indicates such distances. FIG. 11 shows a group of superpoints in a search area that form an object similar to the object represented by the point cloud of FIG. 10. FIG. 11 also shows votes cast from the center of each superpoint. For example, reference 110 indicates such a vote. FIG. 12 illustrates how the Hough voting method takes rotation into account when voting.

[0097] It should be understood that the grouping step S80 is not necessarily performed in the present detection method, for example, the second point cloud may consist of only one superpoint, so that no grouping is required.

[0098] The detection method may further comprise a step S90 of determining a bounding box for each determined group. The bounding box encompasses / encloses the group. The step S90 of determining the bounding box may comprise drawing a bounding box around the center of each group and, for example, before determining this center. The size and / or dimensions of each bounding box may be equal to or substantially equal to the size of a (e.g., smallest) bounding box enclosing the second 3D point cloud.

[0099] If the grouping step S80 uses the above-described method of computing a graph of superpoints, the bounding boxes may be drawn to enclose respective groups of first superpoints and may have a size equal to or substantially equal to the size of the (e.g., smallest) bounding box enclosing the query.

[0100] If the grouping step S80 uses Hough voting, then in determining the plausible object centers, a bounding box may be drawn in the search area, centered on the plausible center (with the same dimensions as the query bounding box) and / or around superpoints that voted for voxels representing plausible object centers.

[0101] As mentioned above, it will be appreciated that if the grouping step S80 is not performed in the detection method, the determining step S90 may simply be to determine a bounding box around each matched SuperPoint.

[0102] The determining step S90 may also determine a bounding box around the second 3D point cloud. The detection method may further include displaying each determined bounding box, thereby providing a user of the detection method with visual feedback of the 3D objects detected in the search area.

[0103] The detection method may further include determining a group similarity score for each group by measuring the similarity between a first superpoint and one or more second superpoints in the group. The similarity score for each group is a score of geometric similarity between the group and a group of superpoints formed by the group and one or more second superpoints. The detection method may determine the similarity score by calculating any of such geometric similarity scores. The similarity may vary depending on the application and / or may be based on multiple cues. The similarity score may include one or more of the following: - Superpoint Matching Similarity Score (spms): This is the combined matching scores of the matched superpoints (with the identified superpoints of the query) for a given group of first superpoints, where n is the number of superpoints in the query and m i The matching score of each superpoint with the first group of superpoints (if no matching, m i =0), the score is calculated as follows:

number

[0104] The above similarity scores (also called similarity metrics) may be used together to obtain an overall similarity score, or may be used individually, depending on the intended application.

[0105] In addition to determining a similarity score for each group, the detection method may further include displaying the similarity scores for the groups. For example, the detection method may simultaneously display the similarity scores for all groups, for example, arranged in descending order of similarity score rank. The detection method may also display a bounding box around the group and, optionally, around the second point cloud, simultaneously with displaying the similarity scores for the groups. The detection method may also display the dimensions of the bounding box. The method may also visually indicate, for each group, that the group's similarity score exceeds a similarity score threshold.

[0106] The similarity score threshold may be a percentage of similarity. The similarity score threshold may be user-selectable. For example, the similarity score threshold may be displayed to the user as a slider with a line and a handle (of any shape), where the line represents all possible values ​​of the threshold (e.g., 0% to 100%) and the handle can be visually selected (e.g., by clicking or touching the handle) and slid along the line (e.g., by dragging and dropping) to set the similarity score threshold value. The slider may be part of a widget displayed in a graphical user interface of a computer used by a user of the detection method, where the widget is a software component stored in memory. The user's selection of the threshold may be performed before the detection method is performed, for example, as an initial step of the method, where the user predefines the desired similarity score and the bounding box they wish to display. Additionally or alternatively, the selection may be performed after step S90 of determining the bounding box, so that the user only displays bounding boxes that enclose groups with a sufficient similarity score. Alternatively, the user may continuously select a similarity score threshold while the method is running; for example, once the user selects a similarity score threshold, steps of the detection method are performed in which bounding boxes enclosing groups each having a similarity score above the threshold are detected (i.e., groups are detected and bounding boxes are created) and displayed in real time as a result of the user's selection, wherein the display of the bounding boxes is continuously adapted in real time in response to changes in the user's continuous similarity score threshold.

[0107] FIG. 13 shows a screenshot of a widget 130 displayed on a graphical user interface of a computer used by a user of the present detection method. The widget allows the selection of a similarity score threshold. The widget 130 displays detected groups of first superpoints, each assigned a name, in column 1300. The widget 130 also displays a column 1304 of the bounding box dimensions of each group. The widget 130 also displays a column 1302 of the similarity score of each group. The widget also displays a column 1310 for each group, showing whether the group's similarity score exceeds (or does not exceed) the similarity score threshold. The widget includes a slider consisting of a line 1308 and a handle 1306. A user may change the value of the similarity score threshold by visually selecting the handle 1306 (e.g., by clicking or touching the handle) and sliding it along the line 1308 (e.g., by dragging and dropping). In the screenshot shown in FIG. 13, the threshold is set to 84%.

[0108] Next, an implementation of the process incorporating the learning and detection methods will be described.

[0109] This implementation enables example-based 3D object detection, which consists in identifying shapes / objects in a larger point cloud that are similar to examples from a given point cloud. Identification is based on a 3D bounding box around the detected object. This implementation employs a deep learning approach for detection. The evaluation criteria for this implementation are: -Given a point cloud query (example) and a larger point cloud (search area) as input, The implementation aims to output a 3D bounding box around objects in a search area that are similar to the query, but the number of objects to be found is not known in advance. Note that the search area may not contain any objects similar to the selected query. Example-based object detection must be class-agnostic, meaning that it must be able to find similar objects given any object query. Example-based object detection should be context-agnostic, meaning that detection should be independent of the object's context / background (indoor, outdoor, etc.). It does not use context-specific prior knowledge of the point cloud. The point cloud representation needs to be robust to noise, occlusion, point distribution, and invariant to different kinds of transformations (e.g., rotations) and object positions (e.g., articulated objects such as robotic arms). This is made possible by the use of local, learning-based representations. The implementation aims to identify similar objects that are not necessarily exact instances representing the query. In fact, the query may not even belong to the search area.

[0110] This implementation provides generality, meaning that detection is independent of the object (even for objects / situations not seen during offline training) and the type of point cloud (no special constraints are required for point cloud acquisition). Given a single shape query, this implementation can identify partially similar shapes (i.e., not just identify the exact instance corresponding to a given shape query). This implementation enables 3D shape detection aimed at detecting shapes within point clouds.

[0111] This implementation provides a learning-based framework for performing 3D detection of all shapes within a point cloud that are similar to a given shape query in an efficient and generic manner. The learning part of the framework is based on deep learning techniques (i.e., using deep neural networks).

[0112] This implementation follows the framework described in Figure 14. As shown in Figure 14, this implementation includes an offline phase and an online phase. - Offline Phase: This phase trains the encoder based on the superpoints. It involves two main steps. Note that this phase is transparent to the user. The two main steps are: Superpoint extraction. Superpoints are a subset of points that represent a local part of a given point cloud. Superpoints extracted from isolated point cloud objects are consistent between the same objects in the scene (i.e., together with other objects). ··Encoder learning based on the SuperPoint training dataset, which is a DNN-based model for point clouds. - Online Phase: Given a point cloud shape query and a point cloud scene, superpoints are extracted from both inputs and encoded using the encoder learned in the offline phase. The query superpoint encodings are compared with the scene superpoint encodings. The most similar superpoint encodings are matched.

[0113] Offline Learning Offline learning trains an encoder that can project any 3D superpoint into a latent representation that presents meaningful geometric information, i.e., the distance between the encodings of two superpoints is highly correlated with the geometric similarity of these two superpoints (e.g., with respect to the orientation of the superpoints). The distance is defined as a measure of similarity between two vectors, such as cosine similarity or Euclidean distance. Dataset Generation: Superpoint Extraction

[0114] First, a dataset of superpoints is constructed to train the encoder. The diversity of the superpoints used to train the encoder is directly related to the encoder's ability to construct meaningful projections of arbitrary 3D geometries. The dataset of superpoints can be extracted from any set of one or more point cloud scenes containing a variety of geometries. Superpoint extraction is completely unsupervised, so no point cloud scene annotations are required.

[0115] Point Cloud Scene The point cloud scenes used for training are generated, but no virtual scanner is simulated. Given CAD models of various objects, generation involves converting the CAD models into point clouds and then inserting them into the point cloud background. During object insertion, possible occlusions (object / object occlusion or object / background occlusion) are considered, as well as point density normalization for a realistic synthetic point cloud representation. 100 scenes are used for learning / training.

[0116] Super Point Extraction In our implementation, as mentioned above, we use a geometry-based method for superpoint extraction. Such methods exhibit an interesting feature. The fact that a superpoint usually corresponds to only one uniform part of the point cloud (due to the cuts obtained in the adjacency graph) is more convenient for subsequent processing; this usually means that a superpoint is part of a single object, depending on the regularization weights of the cuts. This gives our implementation at least two interesting properties: - the extraction of superpoints is highly consistent between superpoints extracted from isolated point cloud objects and superpoints extracted from the same object (or similar objects) in the scene (i.e., together with other objects); and -The superpoints obtained in this way regroup points with similar geometric properties, thus simplifying the complexity required for the encoder training step.

[0117] Encoder Training The loss used in this implementation is the reconstruction loss. The loss is a chamfer loss, which provides efficiency in terms of computational cost. The loss is given by the following formula for two sets of points, S1 and S2:

number

[0118] The model is trained using a pre-generated dataset of superpoints. Depending on the number of features considered for the point cloud, the model is trained with the corresponding input size. In this implementation, the number of points in the superpoints is subsampled or upsampled to match the input size of the neural network. In this implementation, the input size is N=128 points per superpoint, the features are 3D coordinates, and the encoding is of size N / 2.

[0119] Online Inference Matching SuperPoint Encoding After obtaining the set of superpoints for the query and search area, the implementation calculates the similarity between each superpoint in the search area and each superpoint in the query. This step can be time-consuming, so to avoid unnecessary comparisons, the implementation may include the following filtering: -Dimensional comparison between superpoints projected onto the LRF, as discussed in J. Yang et. al, "Toward the Repeatability and Robustness of the Local Reference Frame for 3D Shape Matching: An Evaluation", in IEEE Transactions on Image Processing, vol. 27, no. 8, pp. 3766-3781, Aug. 2018, which is incorporated herein by reference. If the ratio between the dimensions per corresponding axis is far from 1 and the difference per dimension is greater than a certain threshold, the superpoint in the search area will not be considered as a possible matching candidate for a given superpoint in the query. In other words, the present implementation aims to satisfy the following for each dimension i:

number

[0120] The similarity measure used in this implementation is the cosine similarity measure given by:

number

[0121] Note that depending on the encoder, a different similarity metric can be chosen (e.g., if regularization of the autoencoder's latent space is enforced).

[0122] In this implementation, a superpoint in the search area is matched with a query superpoint if their similarity (e.g., cosine similarity) exceeds a threshold defined with respect to the encoder's projection characteristics and the user's desired similarity. Using a set of superpoints manually annotated as similar or dissimilar for the target application, such a threshold can be obtained by calculating the minimum similarity obtained between two superpoints labeled as similar. In this implementation, based on such experiments, the threshold is fixed at 0.8.

[0123] At the end of this step, we have a set of n matching superpoints (SP rz 1 ,…,SP rz n ) to the corresponding matching score mq i ((mq i ,…,mqm ),…,(mq i ,…,mq k )) and get it together.

[0124] Superpoint encoding grouping In this step of our implementation, we find a 3D bounding box from the set of matched superpoints. Note that if the object consists of only one superpoint, we draw a bounding box proposal directly around the matched superpoint.

[0125] Up to this step, local descriptors (features of the superpoints) have been computed and compared between the query and the search area. The relative positions of the superpoints within the query (e.g., with respect to a reference point such as the center of the query) are used to infer a set of reasonable bounding boxes.

[0126] This relative position may be obtained by using the Hough voting method described above for the object centers, or alternatively, this relative position may be obtained using the method described above for computing a graph from a set of superpoints in the query.

[0127] An implementation of this process outputs bounding box suggestions. Because similarity can be application or object dependent, this implementation may include adding a similarity indication to the suggestions, allowing the user to control how similar they want the objects found from the query to be. The method may, for example, include displaying widget 130 of FIG. 13, as described above, so that the user may use this widget to set a similarity score threshold, as described above with reference to FIG. 13. Each detection is associated with a similarity score (similarity percentage), and the user can select a threshold to keep only the most favorable ones. In the widget of FIG. 13, a threshold of 84% similarity percentage was selected.

[0128] This similarity varies depending on the application. Secondly, the similarity may be based on multiple cues. The present implementation may use several methods to calculate similarity metrics obtainable from the proposed bounding boxes and / or matched superpoints, as listed below. These similarity metrics may be used together to obtain an overall similarity score or may be used individually, depending on the target application. The metrics are as follows: - Superpoint Matching Similarity Score (spms): is the combined matching score of the matched superpoints (with the identified superpoints of the query) for a given proposal. Let n be the number of superpoints in the query, and m i The matching score of each super point in the proposal (if no matching, m i =0), then:

number

[0129] An example of 3D object detection using the present detection method will now be described with reference to Figures 15 to 18. This example may be achieved, for example, by implementing a process incorporating the training and detection methods described above.

[0130] FIG. 15 shows a screenshot of a 3D object from which a query was extracted. The 3D object is a trash can / dumpster. FIG. 15 shows a bounding box around the trash can. FIG. 16 shows superpoints extracted from the trash can, i.e., the superpoints that form the query. FIG. 17 shows a search area 170. As shown in FIG. 17, as a result of performing the detection method, groups 172, 174, and 176 of superpoints are detected within the search area. These groups represent trash cans similar to the trash can in FIG. 15. FIG. 18 shows a display of a bounding box around one similar trash can detected in search area 170. For illustrative purposes, only one bounding box is shown in FIG. 18, but it will be understood that more bounding boxes may be displayed.

[0131] The learning method, the detection method, and processes incorporating these methods are computer-implemented methods, meaning that the steps (or substantially all steps) of the method are performed by at least one computer or any similar system. Thus, the method steps are performed by a computer, possibly fully automatically or semi-automatically. In examples, initiation of at least some steps of the method may be performed via user-computer interaction. The level of user-computer interaction required depends on the expected level of automation and may be balanced against the need to implement user preferences. In examples, this level may be user-defined and / or predefined.

[0132] A typical example of a computer implementation of the method is running the method on a system adapted for this purpose. The system may include a processor coupled to a memory and a graphical user interface (GUI), the memory having stored thereon a computer program including instructions for carrying out the method. The memory may also store a database. The memory is hardware adapted for such storage, possibly including several physically distinct parts (e.g., one for the program and possibly one for the database).

[0133] FIG. 19 shows an example of a system, where the system is a client computer system, for example a user's workstation.

[0134] The client computer in this example includes a central processing unit (CPU) 1010 connected to an internal communication BUS 1000, and a random access memory (RAM) 1070 also connected to the BUS. The client computer further includes a graphics processing unit (GPU) 1110 and associated video random access memory 1100 connected to the BUS. The video RAM 1100 is also known in the art as a frame buffer. A mass storage controller 1020 manages access to mass storage devices, such as a hard drive 1030. Mass storage devices suitable for tangibly embodying computer program instructions and data include all forms of non-volatile memory, including, by way of example, semiconductor memory devices such as EPROM, EEPROM, and flash memory devices, magnetic disks such as internal hard disks and removable disks, magneto-optical disks, and CD-ROM disks 1040. Any of the foregoing may be supplemented by, or incorporated in, specially designed application-specific integrated circuits (ASICs). A network adapter 1050 manages access to a network 1060. The client computer may also include a haptic device 1090, such as a cursor control device, a keyboard, and the like. A cursor control device is used on the client computer to allow a user to selectively position a cursor at any desired location on the display 1080. Furthermore, the cursor control device allows the user to select various commands and input control signals. The cursor control device includes a number of signal generating devices for inputting control signals into the system. Typically, the cursor control device may be a mouse, with the buttons on the mouse being used to generate the signals. Alternatively or additionally, the client computer system may include a pressure-sensitive pad and / or a pressure-sensitive screen.

[0135] A computer program may include computer-executable instructions, which include means for causing the system to perform one or more of the methods described herein. The program may be recordable on any data storage medium, including the system's memory. The program may be implemented, for example, in digital electronic circuitry, or computer hardware, firmware, software, or a combination thereof. The program may also be implemented as an apparatus, such as an article of manufacture tangibly embodied in a machine-readable storage device for execution by a programmable processor. The method steps may be performed by a programmable processor executing a program of instructions to operate on input data and generate output to perform the functions of the method. Thus, the processor may be programmable or coupled to receive and transmit data and instructions from and to a data storage system, at least one input device, and at least one output device. The application program may be implemented in a high-level procedural or object-oriented programming language, or in assembly or machine language, as appropriate. In either case, the language may be a compiled or interpreted language. The program may be a full installation program or an update program. In either case, application of the program on a system provides instructions for performing the method.

Claims

1. 1. A computer-implemented method for using a neural network configured to encode superpoints of a 3D point cloud into latent vectors, comprising: - providing a data set of superpoints (S10-S20), each superpoint being a set of points of the 3D point cloud, said set of points representing at least a part of an object; - training (S30) the neural network based on the data set of superpoints, the training step (S30) comprising minimizing a loss that penalizes the error between two superpoints; - the following: one or more first superpoints of a first 3D point cloud representing a 3D scene; and one or more second superpoints of a second 3D point cloud representing the 3D object; Steps (S40 to S50) of providing - applying the neural network to encode each of the one or more first superpoints into a respective first latent vector and each of the one or more second superpoints into a respective second latent vector (S60); determining (S70) a similarity between each first superpoint of the one or more first superpoints and each second superpoint of the one or more second superpoints by calculating, for each first superpoint and each second superpoint, a similarity between the respective first latent vector encoding the first superpoint and the respective second latent vector encoding each of the second superpoints; The step of providing one or more first super points (S40 to S50) providing one or more initial superpoints of the first 3D point cloud; and filtering the one or more initial superpoints; Including, The filtering step comprises: the error between the dimension of said superpoint and the dimension of at least one said second superpoint is less than a predefined threshold; and / or - by selecting from among the initial superpoints initial superpoints each satisfying the following: an error between the position of the superpoint and the position of at least one second superpoint is smaller than a predefined threshold; The selected super point is the one or more first super points. method.

2. The method of claim 1 , wherein the loss is a reconstruction loss, and the loss penalizes the error between a superpoint and a reconstruction of that superpoint.

3. The method of claim 2 , wherein the error is the distance between the superpoint and a reconstruction of the superpoint.

4. The method of claim 3 , wherein the distance is a Chamfer distance or an Earth-Mover distance.

5. The method of claim 1 , wherein the learning step is unsupervised learning.

6. The filtering step comprises: the distance between each dimension of said superpoint and the corresponding dimension of at least one said second superpoint is less than a predefined threshold, and / or the ratio between each dimension of said superpoint and the corresponding dimension of at least one said second superpoint is less than a maximum ratio and greater than a minimum ratio; and / or the difference between the relative height of said superpoint from the nearest supporting surface and the relative height of at least one said second superpoint from the nearest supporting surface is less than a predefined threshold value; The method of claim 1 , further comprising the step of selecting initial super points from among the initial super points that satisfy the following:

7. 10. The method of claim 1, further comprising: determining (S80) one or more groups of first superpoints from among each first superpoint that has been determined to have a similarity to at least one second superpoint that is greater than a predefined threshold, wherein each of the groups of first superpoints has a shape similar to the second 3D point cloud.

8. The step of determining one or more groups (S80) - providing a graph of said second superpoints, wherein said graph of second superpoints has a plurality of nodes and a plurality of edges, each of said plurality of nodes representing a second superpoint, each of said plurality of edges representing a geometric relationship between two superpoints represented by the nodes connected by the edge, and having one or more geometric attributes of said geometric relationship; - forming one or more graphs each having a plurality of nodes representing a first superpoint by constructing a plurality of edges, each edge connecting two nodes and each edge having one or more geometric attributes similar to the geometric attributes of the edges of the graph of the second superpoint, each graph formed corresponding to one of the groups.

9. The method of claim 8 , further comprising determining a similarity score for each group by measuring a similarity between a first superpoint and the one or more second superpoints in the group.

10. A computer program comprising instructions for causing a computer to carry out the method of any one of claims 1 to 9.

11. A device comprising a computer-readable data storage medium having recorded thereon the computer program of claim 10.

12. The device of claim 11 , further comprising a processor coupled to the data storage medium.

Images