a method for estimating at least one dimension of an object represented by a point cloud.
The method estimates object dimensions by calculating distances from a cloud of points to a center, ordering points, and using optimization to find the smallest bounding box, addressing complexity and resource issues in existing technologies.
Patent Information
- Application Number
- FR2023004867
- Authority / Receiving Office
- FR · FR
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-16
- Publication Date
- 2024-11-22
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing methods for estimating object dimensions from a cloud of points are complex and require labeled databases or specific hardware for real-time inference, making them inefficient and resource-intensive.
A method that estimates object dimensions by calculating distances from a cloud of points to a center, ordering points based on distance, detecting variations, and estimating dimensions using the smallest or largest distance between points, optionally with rotation and optimization algorithms to find the smallest bounding box.
Provides a simple and efficient way to estimate object dimensions using standard hardware, reducing resource requirements and improving accuracy by filtering noise and finding the smallest bounding box.
Smart Images

Figure 00000000_0000_ABST
Abstract
Description
Title of the invention: A method for estimating at least one dimension of an object represented by a point cloud. Technical field
[0001] The present invention relates to a method and device for estimating the dimensions of an object, represented by a point cloud, such as a 3-dimensional point cloud, and can find application in many fields, such as computer vision, and is applicable to several kinds of object detection applications. Prior art
[0002] Several methods exist for estimating the size of objects from a point cloud representation. Methods from the fields of data analysis and / or artificial intelligence propose constructing models of functions f, g, h: H = f(h), L = g(l), and D = h(d) to estimate the width L, height H, and depth D in meters of objects from these same quantities l, h, and d in pixels. These can be linear models or neural networks. Other approaches use deep neural networks to determine the pixels belonging to the boundary of an object and obtain the object's dimensions by calculating the Euclidean distance between points on the boundary. However, such approaches are complex because they require a labeled database to determine the parameters of the different models.Furthermore, the execution of neural networks requires specific architectures (GPU or TPU) to perform real-time inference and perfect delineation of the object whose dimensions need to be estimated.
[0003] There is therefore a need to propose simple solutions for estimating the dimensions of an object. Statement of the invention
[0004] To this end, the present invention proposes a method for estimating at least one dimension of an object represented by a point cloud relating to a scene comprising said object, the method comprising - obtaining the distances, along at least one direction, of said points of said point cloud relative to a center of said point cloud along said at least one direction, - an ordering of said points according to said distances obtained; - detection of a variation in distance greater than a first value between two successive points of said ordered points; -an estimate of the dimensions of said object, along said at least one direction, taking into account the distance obtained for one of the two successive points
[0005] According to at least one embodiment, the ordering of the points is carried out in ascending order of said distances obtained and said estimation of said dimension of said object, according to said at least one direction, takes into account the smallest distance obtained among the distances obtained from said two points.
[0006] According to at least one embodiment, the ordering of the points is carried out in descending order of said distances obtained and said estimation of said dimension of said object, according to said at least one direction, takes into account the greater distance obtained among the distances obtained from said two points.
[0007] According to at least one embodiment, a coordinate of said center of the region of interest along a dimension is obtained as being the median of the coordinates of the points of the region of interest along said dimension.
[0008] According to at least one embodiment, said distance is normalized.
[0009] According to at least one embodiment, the estimate includes - a representation of said normalized distance for the points of said point cloud, the index of said points being normalized, -a determination of a distance between said normalized representation and the curve y=x in a space of said representation, the dimension being estimated as twice the distance between said center and the point preceding the point whose distance between its normalized representation and the curve y=x is maximum.
[0010] According to at least one embodiment, the estimated dimension is compared to a value that is a function of a measurement noise, and if said estimated value is less than said value that is a function of said measurement noise, then the dimension is equal to twice the distance between the point furthest from the center and said center.
[0011] According to at least one embodiment, said dimension is estimated along three directions corresponding to the width, depth and height of said object, the three estimated directions defining a box encompassing said object.
[0012] According to at least one embodiment, said method comprises a rotation about at least two angles of said point cloud around said center, said dimension being estimated for the at least two angles and the method comprising a selection of the estimated dimension giving the bounding box whose volume is the smallest, among said dimensions estimated for said at least two angles.
[0013] According to at least one embodiment, said rotation angles are sampled according to the Nelder-Mead method.
[0014] The features, presented individually in this application in connection with certain embodiments of the process of this application, can be combined with each other according to other embodiments of this process.
[0015] The invention also relates to a computer program comprising ins instructions for carrying out the steps of the process according to the invention, according to any of its embodiments, when said program is executed by a computer.
[0016] The invention also relates to a computer-readable recording medium on which is recorded a computer program comprising instructions for carrying out the steps of the process according to the invention, according to any one of its embodiments.
[0017] The present invention also relates to a device comprising one or more processors configured together or separately for instructions for executing the steps of the process according to the invention, according to any one of its embodiments. Thus, the present invention relates to a device for estimating at least one dimension of an object represented by a point cloud relative to a scene including said object, the device comprising one or more processors configured together or separately for - to obtain distances, along at least one direction, from said points of said point cloud relative to a center of said point cloud along said at least one direction, - order the said points according to the said distances obtained; - detect a variation in distance greater than a first value between two successive points of said ordered points; - estimate a dimension of said object, according to said at least one direction, taking into account the distance obtained for one of said two successive points. Brief description of the drawings
[0018] Other features and advantages of the present invention will become apparent from the description given below, with reference to the accompanying drawings which illustrate examples of embodiment thereof, without any limiting character.
[0019] [Fig-1] Fig. 1 represents a system implementing certain modes of rea publication of the present invention,
[0020] [Fig.2] Figure [Fig.2] represents objects, each in the form of a point cloud,
[0021] [Fig.3] Fig.3 represents a method according to certain embodiments of the invention,
[0022] [Fig.4a] Fig.4a represents a representation of the distance between points on a point cloud and the center of the cloud,
[0023] [Fig.4b] Fig.4b represents a normalized representation of the distance between points in a point cloud and the center of the cloud,
[0024] [Fig.5] Fig.5 represents an example of a bounding box,
[0025] [Fig.6] Fig.6 represents an example of bounding boxes at two angles of rotation,
[0026] [Fig.7a] Fig.7a represents an example of a point cloud without rotation,
[0027] [Fig.7b] The [Fig.7b] represents a rotation of the point cloud of the [Fig.7a]. Description of the embodiments
[0028] This disclosure relates to the estimation of one or more dimensions of at least one object in a scene, such as a scene captured by a digital image, this object being represented by a point cloud. A scene may be representative of an indoor or outdoor environment and include one or more inanimate objects, animals, people, a background, etc. In the present invention, the term "object" can be understood to mean an everyday object, a part of such an object, but also a living being, such as a person or an animal, a plant, or any other element captured in an image. The term "scene" can be understood to mean a scene representing a virtual image or a real image.
[0029] Fig. 1 represents a system capable of implementing at least some embodiments of the present disclosure.
[0030] In the example of [Fig. 1], a scene is captured by capture means 20. These capture means can take different forms and may in particular consist of: - one or more cameras, such as stereographic cameras, and / or LIDAR (English acronym for "light detection and ranging"), and / or RADAR (English acronym for "radio detection and ranging"), and / or plenoptic cameras, and / or sensors using ultrasound known as ToF (English acronym for "time of flight") devices
[0031] These capture means can be associated with processing means 10. The processing means 10 and the capture means 20 can be contained in a single device 100 or belong to different devices coupled together. The processing means include, in particular, means for obtaining a point cloud of at least a portion of the scene whose image was captured by the capture means. The processing means may include stereographic vision devices, radar vision devices, lidar vision devices, and / or ToF devices.
[0032] As illustrated in [Fig. 1], the processing means may have the hardware architecture of a computer. Thus, the processing means 10 include, in particular, a processor 1, random access memory 2, read-only memory 3 and non-volatile memory 4. They also include communication means 5, for example for communicating with the capture device 20 and / or a user interface 30.
[0033] Read-only memory 3 constitutes a recording medium conforming to at least one An embodiment of this disclosure, readable by processor 1 and on which is stored a computer program PROG conforming to at least one embodiment of this disclosure, comprising instructions for executing steps of the method for estimating at least one dimension of at least one object according to at least one embodiment of this disclosure. The PROG program defines functional modules of the device.
[0034] The user interface 30 may allow a user to interact with the system, for example, in certain embodiments, with the capture and / or processing means. This interaction with the capture and / or processing means may be optional in certain embodiments. The user interface may take various forms, including one or more screens, touchscreen or non-touchscreen, one or more keyboards or styluses, or a tablet, mobile phone, or computer. In [Fig. 1], the user has selected an area encompassing an object whose dimensions are to be estimated by this disclosure.
[0035] The images obtained by the capture means are, for example, images representing the captured scenes in the form of a point cloud as illustrated in [Fig. 2]. [Fig. 2] illustrates, by way of example, images obtained by the capture means representing a box and a football whose dimensions are to be measured. These objects are present in a scene captured by the capture means. For the sake of simplicity, it is assumed here that the objects in the scene whose dimensions are to be estimated using the method of this application have been previously isolated, so as to distinguish (isolate) with a certain degree of precision the points of the point cloud (representing the object) from other points representing the captured scene (for example, other points in a region of interest detected or designated in an image).Depending on the embodiments, this decoupling may have been carried out in different ways, for example according to the process which is the subject of application FR2304863 of the applicant.
[0036] This disclosure allows for the estimation of the dimension(s) of an object captured by a capture means by estimating the size of a bounding box encompassing the point cloud representing that object. According to this disclosure, the smallest bounding box is sought, the dimensions of the object then being considered as the dimensions of the smallest bounding box (also referred to hereafter as the "circumscribed box" of the object).
[0037] Figure 3 represents an embodiment of a process according to this disclosure.
[0038] We consider an object in an image, represented in the form of a point cloud, and whose size we wish to estimate.
[0039] According to the present example, the point cloud comprises three dimensions (x,y,z) but The present invention can be applied to a point cloud in one or more dimensions. The steps of the method described below relate to estimating the dimension along the x-direction (also referred to hereafter as the abscissa axis), and the steps are repeated, sequentially or in parallel, for the other directions so as to obtain all directions. The steps described below are adapted to determine the smallest circumscribed box when the object is aligned with the capture means, and, as explained later in the description, a rotation of the point cloud is considered so as to obtain the smallest circumscribed box.
[0040] During step E1, the position of the center of the point cloud, and therefore its coordinates in the considered direction, can be determined. Depending on the embodiment, the position of the center of the point cloud (and therefore its coordinates) can be obtained or determined using different methods. For example, the coordinates of the center in a direction can be determined as the median of the coordinates of the points in that direction. Using the median can make the process more robust against measurement errors and outliers that the process might otherwise return.
[0041] The coordinates of the points in the cloud in the considered direction are obtained by capture means such as a lidar for example. In a step E2, the distances d; between the point P; and the center C in the x direction are calculated.
[0042] For a point Pi, we obtain the distance d; as the norm
[0043]
[0044] In step E3, the points P are ordered (or, in other words, sorted or classified) according to their distance d from the center coordinate Cx, for example, according to a distance d that increases with the center coordinate Cx. Figure 4a illustrates a representation following such a sorting.
[0045] According to other embodiments, the points P; are ordered according to their distance d; with the coordinate Cx of the center, for example according to a distance d; decreasing with the coordinate Cx of the center.
[0046] The curve in [Fig. 4a] indicates that the points with indices 1 to 7 on the x-axis form a first group and that there is a break between the point with index 7 and the point with index 8. Thus, according to the present disclosure, a variation in distance between two successive points of the ordered points greater than a first value (acting as a threshold) is detected at step E4. In step E5, the dimension in at least one direction is estimated from a representation of the points, taking into account the distance obtained for one of said two successive points.
[0047] When the ordering of the points is carried out according to an increasing order of the distances obtained, the estimation of the dimension of the object, according to at least one
[0048]
[0049]
[0050]
[0051]
[0052]
[0053]
[0054]
[0055]
[0056]
[0057] direction, takes into account the smallest distance obtained among the distances obtained from the two points. When the ordering of points is carried out in descending order of the distances obtained, the estimation of the dimension of the object, according to at least one direction, takes into account the greater distance obtained among the distances obtained from the two points. According to at least one embodiment, the dimension is estimated as twice the distance (from the center of the cloud) associated with the point preceding the first point whose distance is greater than the first value when the points are sorted in order of increasing distance. According to at least one embodiment, the dimension is estimated as twice the distance associated with the point following the first point whose distance is greater than the first value when the points are sorted in order of decreasing distance. This first value corresponds, as illustrated in [Fig.4a], to a break in the point cloud or a break and therefore to a limit of the object considered. This first value can, for example, be a threshold value defined as a function of the average spacing between the distances of points with an index less than i. For example, when we detect that the difference between the distance associated with a point i and that associated with a point i+1 is greater, for example, than 1.5 times this average spacing, then we detect a break. According to at least one other embodiment, in order to determine this break, it is also possible to determine the distance differences between two adjacent points, one at a time, and to select the point i whose distance difference with point i+1 is maximum. The dimension of the object is obtained as twice the distance d from said point i. Point i is subsequently called the break point. According to some embodiments, in step E51, the curve in [Fig. 4a] is normalized to obtain the curve in [Fig. 4b]. The curves in Figures 4a and 4b represent, for the points in the scatter plot, a distance from the center. [Fig. 4b] represents the curve of [Fig. 4a] after normalization as explained below, as well as the curve with the equation y=x. where i is an integer greater than or equal to 1, we write: m = min ( dj )M = max ( d, ) With min (4 ) ct max(d( ), respectively the minimum and maximum values of the distance d;. The normalized curve has the following x-coordinate: X — 4th for ordinate: dj-m y=M^
[0058] In the space of this normalized representation of the distance d; of the points i, we trace the line with equation y=x, step E52, as illustrated in [Fig.4b].
[0059] The estimation of the distance, according to the dimension considered, can then include a step E53 of determining, for the points i, a distance between its normalized representation and its ordinate on the curve y=x in the space of the representation.
[0060] We determine the point on the normalized curve whose distance between its ordinate on the normalized curve and its ordinate on the curve y=x is maximum.
[0061] The dimension along the considered direction can then be estimated as twice the distance between the center and the break point, that is, the point where the distance between its ordinate on the normalized curve and its ordinate on the curve y=x is maximum. Here, as shown in [Fig. 4b], we can therefore see that this distance is that associated with the point indexed 7, as previously shown in [Fig. 4a].
[0062] If we denote r the index of the breaking point, in the example above, r is equal to 7, then the dimension along the direction considered is equal to 2t / r.
[0063] According to certain embodiments, robustness can be added to this estimation by comparing the value M defined previously to a determined value, called the noise value, in step E54. The noise value is determined based on the measurement noise of a point cloud image capture device. If M is greater than this measurement noise value (or threshold), then the dimension is equal to 2dret; otherwise, the dimension is equal to 2M. This can advantageously eliminate unwanted noise. The threshold value can depend on the accuracy of the point cloud capture device. For some capture devices, it can be on the order of a few millimeters to a few tens of millimeters. For example, if a LIDAR has an accuracy of + / - 2 mm, then any difference of this order does not correspond to a break but to noise. This measurement noise can be filtered by considering only breaks that exceed this value.
[0064] The steps of the process can be repeated or carried out in parallel, for at least one other dimension of the object, for example for the other two directions y and z in the detailed examples.
[0065] The dimension can thus be estimated according to three dimensions corresponding to the width, depth and height, the three estimated dimensions defining a box encompassing the region of interest as illustrated in [Fig.5].
[0066] According to some embodiments, the preceding steps can be iterated in order to verify that the resulting box is the smallest box circumscribed about the region of interest, as illustrated in [Fig. 6]. Indeed, the steps of the process as described above make it possible to obtain the smallest circumscribed box when the object is aligned with the image capture means. However, when this is not the case If this is not the case, it is possible to perform one or more rotations of the point cloud and repeat the steps described above, in order to obtain the smallest circumscribed box.
[0067] In Figure 6, the dashed box does not represent the smallest circumscribed box; it is the solid-lined box that represents the smallest circumscribed box. During the iterations, the point cloud can be rotated by a plurality of angles, each new iteration corresponding, for example, to a different angle, as shown in Figures 7a and 7b. The rotation can be performed along at least one of the directions considered (for example, for each of these directions), thus by an angle of √2, √2, √2 in each of the directions. Thus, step E6 can consist of in a rotation of the point cloud.
[0068] According to some embodiments, the rotation can be carried out by incrementing the angle values according to a constant or variable step.
[0069] According to some embodiments, the angles selected for each iteration are sampled using an optimization algorithm. For example, among the optimization algorithms used, the Nelder-Mead method can be used to rotate the point cloud. The Nelder-Mead method makes it possible to find the parameters that minimize a certain criterion. In our case, the parameters are, for example, the rotation angles of the point cloud, and the criterion is, for example, the volume of the bounding box.
[0070] This can help to limit the number of iterations needed to determine the optimal angles (i.e. minimizing the size of the bounding box) and therefore the dimensions of the circumscribed box corresponding to the object and thus to have the dimensions (and therefore the size) of the point cloud of the object.
[0071] One application of this disclosure may be of particular interest in the manufacturing industry, for example, in verifying the conformity of a part produced on production lines. By enabling the estimation of certain dimensions of a part (such as a part detected more or less automatically, for example, by the process described in the applicant's patent application cited above), this disclosure can help to verify its shape, and / or its size, and / or whether it conforms to an expectation or specification, and this in a manner that is at least partially automatic (for example, without human intervention). Obtaining a better understanding of the size (and optionally the location) of objects handled by robots can also help to make robot movements more stable and precise.
[0072] Other applications may relate to the field of logistics. The embodiments described in this disclosure may enable the location of goods in warehouses. Knowledge of the size of the objects can help to limit the storage space required for their storage and the management of flows in warehouses, by optimizing their storage, for example in containers.
[0073] Other applications may relate to autonomous vehicle driving. Estimating the dimensions of obstacles (other vehicles, objects, roadworks, etc.) on the road can help in choosing the appropriate response to their presence in automated driving (deviation from the trajectory, emergency vehicle stop, etc.).
[0074] Other applications may relate to the construction of maps of the environment to enable the navigation of robots, drones and autonomous vehicles in the presence of obstacles.
Claims
Claims
1. Method for estimating at least one dimension of an object represented by a point cloud relating to a scene comprising said object, the method comprising - obtaining the distances, in at least one direction, of said points of said point cloud relative to a center of said point cloud in said at least one direction, - ordering said points as a function of said distances obtained; - detecting a variation in distance greater than a first value between two successive points of said ordered points; - estimating a dimension of said object, in said at least one direction, taking into account the distance obtained for one of said two successive points.
2. Estimation method according to claim 1 where the ordering of the points is carried out according to an increasing order of said obtained distances and said estimation of said dimension of said object, according to said at least one direction, takes into account the smallest obtained distance among the distances obtained from said two points.
3. Estimation method according to claim 1 wherein the ordering of the points is carried out according to a decreasing order of said obtained distances and said estimation of said dimension of said object, according to said at least one direction, takes into account the greatest obtained distance among the distances obtained from said two points
4. Method according to one of claims 1 to 3 wherein a coordinate of said center of the region of interest along one dimension is obtained as being the median of the coordinates of the points of the region of interest along said dimension.
5. Method according to one of claims 1 to 4 in which said distance is normalized.
6. Method according to claim 5 in which the estimation comprises - a representation of said normalized distance for the points of said point cloud, the index of said points being normalized, - a determination of a distance between said normalized representation and the curve y=x in a space of said representation, the dimension being estimated as twice the distance between said center and the point whose distance between its normalized representation and the curve y=x is maximal.
7. Method according to one of the preceding claims in which - the estimated dimension is compared to a value which is a function of measurement noise - if said estimated value is less than said value which is a function of said measurement noise then the dimension is equal to twice the distance between the point furthest from the center and said center.
8. Method according to one of the preceding claims in which said dimension is estimated according to three directions corresponding to the width, depth and height of said object, the three estimated directions defining a box encompassing said object.
9. Method according to claim 8 comprising, a rotation according to at least two angles of said point cloud around said center, said dimension being estimated for the at least two angles and the method comprising a selection of the estimated dimension giving the bounding box whose volume is the smallest, among said dimensions estimated for said at least two angles.
10. Device for estimating at least one dimension of an object represented by a point cloud relating to a scene comprising said object, the device comprising one or more processors configured together or separately to - obtain distances, in at least one direction, of said points of said cloud relative to a center of said point cloud in said at least one direction, - order said points according to said distances obtained; - detect a variation in distance greater than a first value between two successive points of said ordered points; - estimate a dimension of said object, in said at least one direction, taking into account the distance obtained for one of said two successive points.
Citation Information
Patent Citations
BURNER FLAME MONITORING device
FR2304863A1
Caliper-based point cloud geometric dimensioning measurement method and caliper-based point cloud geometric dimensioning measurement device
CN114612547A
Dimensional acquisition of packages
US20170091957A1
