Method and device for removing spurious points from a set of points of a 3D virtual object provided by 3D imaging - Patents.com

Abstract

To provide methods and arrangements for removing erroneous points from a set of points of a three-dimensional, 3D, virtual object provided by 3D imaging of a corresponding real world object by means of a camera with an image sensor.SOLUTION: In a method, for each point obtained from a set of points of a three-dimensional, 3D, virtual object, conflicting points are identified in the set; based on the identification, one or more points of the set that have been involved in conflicts a greater number of times than other points of the set involved in conflicts are removed from the set. Each point of the set is regarded as involved in a conflict each time a point conflicting with the point is identified in the set and / or each time the point itself is identified as a point conflicting with another point of the set.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The embodiments herein relate to a method and apparatus for removing spurious points from a set of points of a three-dimensional, 3D virtual object provided by 3D imaging of a corresponding real-world object using a camera with an image sensor. [Background technology]

[0002] Industrial vision cameras and systems for factory and logistic automation may be based on three-dimensional (3D) machine vision, where 3D images of scenes and / or objects are captured. By 3D image is meant an image that comprises information about pixels in only two dimensions (2D), e.g., intensity and / or color, as in the case of traditional images, but also "height" or "depth" information. That is, each pixel of the image may comprise information that is associated with the pixel's location and maps to the location of what is imaged, e.g., the object. Processing may then be applied to extract information about the object's properties, i.e., the object's 3D properties, from the 3D image and convert, e.g., into various 3D image formats. Such information about height may be referred to as range data, where range data may thus correspond to data from height measurements of the object being imaged, or in other words from range or distance measurements of the object. Alternatively or additionally, the pixels may comprise information about material properties, e.g., regarding the scattering of light in the imaged area, or the reflection of certain wavelengths of light, etc.

[0003] Thus, the pixel value may for example relate to the intensity, and / or range data, and / or material properties of the pixel.

[0004] For example, line-scan image data arises when image data of an image is scanned or provided one line at a time by a camera having a sensor configured to detect and provide image data one pixel line at a time. A special case of line-scan imagery is image data provided by a so-called "sheet of light", e.g., a laser line, 3D triangulation. Although lasers are often preferred, other light sources capable of providing a "sheet of light" may also be used, e.g., light that remains focused and does not diverge much, or in other words, light that is "structured", e.g., light provided by a laser or a light-emitting diode (LED).

[0005] 3D machine vision systems are often based on such sheets of light triangulation. In such systems, there is a light source that illuminates the object with a specific light pattern, such as a sheet of light as a specific light pattern that produces a light or laser line on the object and along which linear 3D characteristics of the object corresponding to the contours of the object can be captured. By scanning the object with such a line, i.e. performing a line scan, with the line and / or object moving, 3D characteristics of the entire object can be captured corresponding to multiple contours.

[0006] A 3D machine vision system or device that uses light sheets for triangulation is sometimes referred to as a system or device for 3D imaging based on light or light sheets, triangulation, or simply laser triangulation when laser light is used.

[0007] Typically, to create a 3D image based on optical triangulation, reflected light from an object to be imaged is captured by an image sensor of a camera, and intensity peaks are detected in the image data. The peaks occur at positions corresponding to locations on the imaged object where there is incident light, e.g., corresponding to a laser line, reflected from the object. The positions of the detected peaks in the image will be mapped to positions on the object from which the light producing the peaks was reflected.

[0008] Laser triangulation camera systems, i.e. imaging systems based on optical triangulation, project a laser line onto an object to create a height curve from the surface of the target object. By moving the object relative to the involved cameras and light sources, information about the height curve from different parts of the object is captured by images, which can then be combined and used with knowledge of the relevant geometry of the system to create a three-dimensional representation of the object, i.e. to provide 3D image data. This technique may be described as acquiring an image of a light, typically a laser line, as it is projected onto and reflected by the object, and then extracting the location of the reflected laser line in the image. This is successfully accomplished, for example, by locating intensity peaks in the image frame using conventional peak-finding algorithms. Typically, but not necessarily, the imaging system is set up such that intensity peaks can be searched for row by row of sensors, and the location in the row maps to height or depth.

[0009] Any measurement system, including a 3D imaging system such as the 3D machine vision system described above, can produce outliers. Outliers generally vary based on the measurement technique used, and therefore depend on the 3D imaging system used, and there can be several different causes for outliers within the same system. Outliers are by definition unwanted, and in industrial measurement applications can make the inspection task more difficult or even impossible.

[0010] Outliers may correspond to image data that does not properly represent and / or correspond to actual points of the imaged real-world object.

[0011] When a 3D imaging system is used to create a 3D virtual object of an imaged real-world object, typically a point cloud, i.e., a set of points, corresponding to surface points of the 3D virtual object, is provided. If the real-world object is in a real-world coordinate system x, y, z, the 3D virtual object may be in a corresponding virtual coordinate system x', y', z'. Outliers may appear in the point cloud as points that do not correspond to actual real points on the surface of the real-world object, i.e., the outliers correspond to incorrect points and may be seen, for example, as "spikes" on the virtual object if the virtual object is drawn based on the point cloud.

[0012] Outliers are generated in different steps during 3D imaging and can also be removed and / or reduced. In the case of laser triangulation as described above, some outliers can be reduced, for example, during intensity peak finding. Outliers that are not removed by previous steps will end up being present in the "point cloud" as false points.

[0013] US2020340800A1 discloses a solution especially for optical triangulation, in which a false detection is determined based on whether a contour is present in a blind spot area in which it is not possible to measure height, and in that case, the part of the contour determined as a false detection is deleted. [Prior art documents] [Patent documents]

[0014] [Patent Document 1] US2020340800A1 Summary of the Invention [Problem to be solved by the invention]

[0015] In view of the above, the objective is to provide one or more improvements or alternatives to the prior art, such as providing an improvement in reducing the adverse effects from erroneous points corresponding to outliers in a point cloud, i.e., a set of points, of a 3D virtual object provided by 3D imaging of a corresponding real-world object. [Means for solving the problem]

[0016] According to a first aspect of an embodiment herein, this object is achieved by a method for removing erroneous points from a set of points of a three-dimensional, 3D virtual object provided by 3D imaging of a corresponding real-world object using a camera with an image sensor. The set of points is obtained. Then, for each point, conflicting points, if any, are identified in the set. Conflicting points are points of the set that cannot validly coexist with the respective point according to one or more predefined criteria. Then, from the set, one or more points of the set are removed that, based on said identification for each point of the set, have been involved in a conflict more times than other points of the set involved in a conflict. Each point of the set is considered to be involved in a conflict each time a conflicting point is identified in the set for the respective point and / or each time the respective point is itself identified as a conflicting point for other points of the set.

[0017] According to a second aspect of an embodiment of the present specification, this object is achieved by a computer program comprising instructions which, when executed by one or more processors, cause one or more devices to perform the method according to the first aspect.

[0018] According to a third aspect of an embodiment of the present specification, this object is achieved by a carrier comprising a computer program according to the second aspect.

[0019] According to a fourth aspect of an embodiment herein, this object is achieved by one or more devices for removing erroneous points from a set of points of a three-dimensional, 3D virtual object provided by 3D imaging of a corresponding real-world object using a camera with an image sensor. The device is configured to obtain the set of points. The device is further configured to identify, for each point, conflicting points, if any, in the set. Conflicting points are points of the set that cannot validly coexist with the respective point according to one or more predefined criteria. Furthermore, the device is configured to remove from the set, based on said identification for each point of the set, one or more points of the set that have been involved in a conflict more times than other points of the set involved in the conflict. Each point of the set is considered to be involved in a conflict each time a conflicting point is identified in the set for the respective point and / or each time the respective point is itself identified as a conflicting point with respect to other points of the set.

[0020] The embodiments herein provide an improvement over deleting or removing each time a conflicting point is identified, which in many circumstances results in correct points also being removed in case of conflicts. In the embodiments herein, more correct points may remain, but still truly incorrect points that do not correspond to real imaged locations on the real-world object, i.e. points corresponding to outliers, may be removed. The embodiments herein may be considered to be based on statistical methods and / or "consensus voting" and may be compared to the solutions described in the background, where the decision for removal is based on conflicts being identified, but does not take into account the total points, and not on how many conflicts each point participated in.

[0021] Example embodiments of the present invention will now be described in more detail with reference to the accompanying schematic drawings, which are briefly described below. [Brief description of the drawings]

[0022] [Figure 1] FIG. 1 shows a schematic diagram of an example of a prior art 3D imaging system that is based on optical triangulation and on which embodiments herein may also be used and / or based. [Diagram 2] FIG. 2 shows a schematic diagram of a simplified example of how a point cloud in the prior art can be generated using a 3D imaging system such as that of FIG. [Figure 3A] FIG. 1 is a schematic diagram of an example of an imaging system and measurement object that may be configured to perform embodiments herein. [Figure 3B] FIG. 3B shows a schematic 3D view of the imaging system and part of the measurement object shown in FIG. 3A. [Figure 3C] FIG. 4 is a schematic diagram showing a side view of the imaging system and a portion of the measurement object shown in FIGS. 3A-B. [Figure 3D] FIG. 3D is a diagram illustrating the points of a point cloud, which corresponds to the virtual view of the measurement object side view shown in FIG. 3C. [Figure 3E] FIG. 3D is a diagram illustrating the points of a point cloud, which corresponds to the virtual view of the measurement object side view shown in FIG. 3C. [Figure 3F] FIG. 3D is a diagram illustrating the points of a point cloud, which corresponds to the virtual view of the measurement object side view shown in FIG. 3C. [Figure 4] 1 is a flow chart to generally illustrate an embodiment of a method according to an embodiment herein. [Figure 5A] FIG. 13 illustrates a virtual 3D object image of a measurement object without application of embodiments herein. [Figure 5B] FIG. 2 illustrates a virtual 3D object image of a measurement object with application of embodiments herein. [Figure 6] 5 is a schematic block diagram to illustrate an embodiment of how one or more devices may be configured to perform the methods and actions described with respect to FIG. 4. [Figure 7]5 is a schematic diagram illustrating some embodiments of a computer program and its carrier for causing a device to perform the methods and actions described with respect to FIG. 4. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0023] The embodiments herein are exemplary embodiments. It should be noted that these embodiments are not necessarily mutually exclusive. Elements from one embodiment may be implicitly assumed to be present in other embodiments, and it will be clear to one skilled in the art how those elements may be used in other exemplary embodiments.

[0024] FIG. 1 shows a schematic representation of an example of an imaging system of the type described in the background, namely an imaging system 105 for 3D machine vision, based on optical triangulation to capture information about 3D properties of a measurement object. The system can be used to provide a 3D image in the form of a set of surface points, i.e. a point cloud, on which the embodiments of the present specification described further below can operate. The system 105 is shown in the figure in a normal operating situation, i.e. typically after a calibration has been performed, and thus the system is calibrated. The system 105 is configured to perform optical triangulation, here in the form of a sheet of optical triangulation, as described in the background. The system 105 further comprises a light source 110, e.g. a laser, for illuminating the measurement object to be imaged with a particular light pattern 111, illustrated and shown in the figure as a sheet of light. The light can be, but does not have to be, a laser light. In the example shown, the target objects are illustrated by a first measurement object 120 in the form of a car and a second measurement object 121 in the form of a gear structure. When the specific light pattern 111 is incident on the object, this corresponds to a projection of the specific light pattern 111 on the object, which may be seen when the specific light pattern 111 intersects with the object. For example, the specific light pattern 111, illustrated in the illustrated example as a sheet of light, gives rise to a light beam 112 on the first measurement object 120. The specific light pattern 111 is reflected by the object, more specifically by a portion of the object at the intersection, i.e., in the illustrated example, the light beam 112. The system 105 further comprises a camera 130 with an image sensor (not shown in FIG. 1). The camera and the image sensor are positioned with respect to the light source 110 and the object to be imaged, such that the specific light pattern, when reflected by the object, becomes incident light on the image sensor. The image sensor is a device, typically implemented as a chip, for converting incident light into image data. The portion of the object, which by reflection causes said incident light on the image sensor, may thereby be captured by the camera 130 and the image sensor, and corresponding image data may be produced and provided for further use.For example, in the illustrated example, a particular light pattern 111 becomes reflected in a ray 112 on a portion of the car roof of the first measurement object 120 towards the camera 130 and image sensor, which may thereby produce image data and provide it with information about said portion of the car roof. With knowledge of the setup, including the geometry of the system 105, e.g., how image sensor coordinates relate to coordinates of a coordinate system 123 associated with the object being imaged and its context, e.g., real world coordinates such as Cartesian, the image data may be converted into information about the 3D properties, e.g., 3D shape or contour, of the object being imaged in a suitable format. The information about the 3D properties, e.g., the 3D shape or contour, may comprise data representing the 3D properties in any suitable format.

[0025] For example, by moving the light source 110 and / or the object to be imaged, such as the first measurement object 120 or the second object 121, so that multiple portions of the object are illuminated and, in practice, typically by scanning the object, image data describing a more complete 3D shape of the object can be produced, corresponding to multiple successive contours of the object, such as the illustrated contour images 141-1 to 141-N of the first measurement object 120, where each contour image shows the outline of the first object 120 from which a particular light pattern 111 was reflected when the image sensor of the camera unit 130 detected the light that gave rise to the contour image. As shown in the figure, a conveyor belt 122 or the like may be used to move the object through the particular light pattern 111, and the light source 110 and camera unit 130 are typically fixed, or the particular light pattern 111 and / or camera 130 may be moved over the object so that all parts of the object, or at least all parts facing the light source 110, are illuminated and the camera is able to receive light reflected from different parts of the object that it is desired to image.

[0026] As can be seen from the above, for example, an image frame provided by the camera 130 and its image sensor of the first measurement object 120 may correspond to any one of the contour images 141-1 to 141-N. As mentioned in the background, each location of the first object's contour shown in any one of the contour images 141-1 to 141-N is typically determined based on identifying intensity peaks in the image data captured by the image sensor and finding the locations of these intensity peaks. The system 105 and conventional peak finding algorithms are typically configured to search for intensity peaks by pixel column in each image frame. If the sensor coordinates are u,v, e.g., u corresponds to a pixel location along a row in the image sensor as shown in the figure, and v corresponds to a pixel location along a column, then for each location, there is u of the image frame searched for peak locations along v, and the identified peaks in the image frame may result in one contour image 141-1 to 141-N as shown in the figure. The contour image is formed by the image points in a sensor-based coordinate system 143, u, v, t. The entire image frame and the contour image may be used to create a 3D image of the first object 120.

[0027] FIG. 2 shows, in order to facilitate understanding of the embodiments herein described further below, a simplified example of how a point cloud, i.e., a set of surface points of a 3D virtual object, as in the prior art, may be generated using a 3D imaging system as in FIG. 1.

[0028] A wedge-shaped measurement object 220 is used in this example. Except for the measurement object, the imaging system 105 can be as in FIG. 1. When an image is captured by the image sensor of the camera 130 at the ray 212 located on the measurement object 220 as shown in the figure and an intensity peak is identified, the result can be a point at the image sensor coordinates u, v as shown in the figure. In this example, the points correspond to samples of the surface of the measurement object 220, i.e. the points correspond to surface points. Note that instead of lines connecting the points as in the case of the contour images 141-1..141-N in FIG. 1, here points are drawn. Each point is at the intensity peak position. Using said knowledge of the geometry of the system 105 and the setup used, of how the image sensor coordinates in u, v and the time t at which the image was captured by the sensor relate to the real world coordinates x, y, z, the points, or rather the positions of the points in the coordinates u, v, t, such as in the sensor-based coordinate system 143, can be transformed into respective positions in the 3D virtual coordinate system x', y', z'. When the complete measurement object 220 is scanned by the light beam 212, the result can thus be a set of points, i.e. a point cloud 260, as shown in the figure. In the shown example, the set of points 260 corresponds to a sample of the surface of the measurement object 220 and can therefore be used, for example, to draw a corresponding virtual object. To facilitate the understanding that the set of points 260 corresponds to the measurement object 220, the outline of the virtual object is indicated in the figure by a dotted line 261.

[0029] 3A illustrates a simplified first example of an imaging system 305 that may be configured to perform, i.e., implement, embodiments of the present disclosure. The imaging system 305 is based on optical triangulation to capture information about 2D and 3D properties of one or more measurement objects.

[0030] The system shown corresponds to a basic configuration and comprises: A light source 310 for illuminating a measurement object 320 with light 311, typically laser light, as part of optical triangulation for 3D imaging of the measurement object 220. A camera 330 with an image sensor 331 arranged to detect reflected light from the measurement object 320 as part of said optical triangulation for 3D imaging.

[0031] The camera 330, image sensor 331, and light source 310 are configured and positioned relative to one another for optical triangulation. System 305 may correspond to system 105 in FIG. 1 for purposes of optical triangulation, but may be further configured to perform in accordance with embodiments herein, as described further below.

[0032] Thus, the measurement object 320 may correspond to the first measurement object 120, or the measurement object 220, and is shown to be located at least partially within the field of view 332 of the camera 330. The light source 310 is typically configured to illuminate the measurement object 320 with light 311, typically in the form of structured light, such as a particular light pattern, e.g., a sheet of light, and / or a beam of light, such as a laser, that is reflected by the measurement object 320, and the reflected first light is captured by the camera 330 and the image sensor 331. Another example of structured light that can be used as the first light is a light edge, i.e., an edge of an area or part where there is illumination. The illumination is in this example vertical, i.e., substantially parallel to the z-axis. However, as will be understood from below, other directions of illumination or directions can of course be used with the embodiments herein, including, for example, a so-called "specular" configuration, using a light source, e.g., a laser, that corresponds to the viewing angle of the camera, but that substantially points at the measurement object with a "specular" angle of incidence from the opposite side.

[0033] The measurement object 320 may be illuminated in this way and images may be captured as in conventional optical triangulation. Said optical triangulation as such may therefore involve, as in the prior art, movement of the light source 310 and / or the measurement object 320 relative to each other, so that at different successive times different successive parts of the measurement object 320 are illuminated by the light source 310 and the light 211. The light 311 after reflection from the measurement object 320 is detected by the image sensor 331. In optical triangulation, typically, but not necessarily, the camera 330 and the first light source 310 are fixed relative to each other and the measurement object 320 moves relative to them. Through said detection by the image sensor 331, each image frame is associated with the respective time when the image frame was detected, i.e. captured, and with the respective part of the measurement object 320 from which the image sensor 331 detected the first light 311 reflected at the respective time.

[0034] The image frames provided by the camera 330 and the image sensor 331 and / or information derived from the image frames, e.g., intensity peak locations, may be transferred to a computing device 333, e.g., a computer or the like, for further processing outside the camera 330. Such further processing may additionally or alternatively be performed by a separate computing unit or device (not shown), e.g., one separate from the image sensor 331 but still included within, e.g., integrated with, the camera 330 or a unit comprising the camera 330. The computing device 333, e.g., a computer or other device (not shown), may be configured to control devices involved in the optical triangulation, such that the optical triangulation and related actions are performed. Furthermore, said computing device 333, e.g., a computer or other device (not shown), may be configured to perform as described above in relation to FIG. 2, based on, e.g., the image frames provided by the camera 330 and the image sensor 331 and / or information derived from the image frames. The computing device 333, e.g., a computer or other device (not shown), may thereby provide a set of points, i.e., a point cloud, of a 3D virtual object corresponding to the measurement object 320. In some embodiments, the computing device 333, e.g., a computer or other device (not shown), is further configured to perform actions related to embodiments herein that are described further below.

[0035] One would like to consider a reasonable practical assumption that the measurement object, e.g., measurement object 320, has a continuous surface with no surface parts or points "floating in the air" without a connection to at least some other parts and points of the object through the surface, i.e., the assumption is that every surface point is connected to one or more adjacent surface points. This is an assumption that can be correct and in principle corresponds to the real situation for all points. If there are special situations, e.g., imaging systems and / or measurement objects used where this is completely or partially not the case due to special setups and / or objects, such situations can typically be identified in advance and treated as special cases. Thus, this assumption does not have undesirable practical consequences or has such consequences that can be ignored. The same should be true for the surface points of a point cloud based on 3D imaging of such a measurement object. That is, that every surface point of the point cloud is connected to the nearest neighboring surface points by a coherent surface and that there are also coherent surfaces between the surface points. This also typically corresponds to the usual assumption when tessellation is used to visualize 3D virtual objects based on point clouds, that in addition to the surface points as such, all surface points are connected to their nearest neighboring surface points by straight lines, together with the assumption that the lines and the surface between the lines correspond to the coherent and opaque surface of the actual measured object, i.e. that there are no holes or transparent areas in the surface between the surface points through which the camera can see.

[0036] FIG. 3B shows a schematic representation of a measurement object 320 shown in a 3D view and in a real-world coordinate system 323 in coordinates x, y, z. The camera 330 is here represented by its camera optical center C, 330b, and a virtual frustum 330a corresponding to the field of view. What will be imaged by the camera 330 and the image sensor 331 are therefore rays incident on the virtual frustum 330a. Three such rays 332 are depicted in the figure, merely by way of example, corresponding to rays from just above and at the most remote edge of the measurement object 320 relative to the camera 330. Also, for this example only, two dummy objects 321a-b are shown in the figure. It should be understood that when the camera is positioned as in this example, the physical object corresponding to the dummy object 321a cannot be imaged by the camera 330 because it is occluded or "shadowed" by the measurement object 320. This corresponds to the dummy object 321a being located under the light beam 332. On the other hand, the dummy object 321b can be imaged because, as seen by the camera 330, the dummy object 321b is not occluded by the measurement object 320 and is positioned such that the light beam from the dummy object 321b can reach and be imaged by the camera 330.

[0037] The surface points of a point cloud, such as point cloud 260, may be evaluated with respect to the camera, e.g. camera 130, or view direction used when capturing the image on which this surface point is based. When the image is captured, there may be parts that are occluded by other parts of the measurement object, e.g. measurement object 220 or 320, based on the above reasonable assumptions. The same principle may be utilized for the evaluation of the surface points of the point cloud. However, in that case the corresponding camera direction should be in the 3D virtual coordinate system x', y', z' of the point cloud, e.g. 3D virtual coordinate system 253 in the case of point cloud 260. That is, the camera direction when capturing the points of the real world object surface points, e.g. measurement object 220 or 320, in the real world coordinate system x, y, z, e.g. real world coordinate system 123 or 323, may be transformed to the corresponding, i.e. virtual, camera direction in the 3D virtual coordinate system x', y', z', e.g. 3D virtual coordinate system 253. Alternatively, the corresponding camera orientation to be used for a point of the point cloud can be determined directly in the 3D virtual coordinate system, i.e. x', y', z', based on known relationships between the coordinate systems involved, e.g. as given by the settings and relationships for the 3D imaging system and the setup used, e.g. as used for optical triangulation when that type of 3D imaging system is used.

[0038] One way to take advantage of the above is to identify, for each surface point of the point cloud, e.g., point cloud 260, one or more conflicting points of the point cloud. That is, for each point, identify and / or investigate whether there are any other points or points that cannot be validly present at the same time as the point under investigation, e.g., because at least one of them is occluded by the other. Two surface points may be considered to be in conflict if one of them is "in the way" of the other or each other, and thus it is not possible for the camera to see both of them at the same time. In other words, if that is the case, one, or at least one of the conflicting points must be wrong or incorrect. In that case, both points may be removed from the point cloud to ensure that the bad points are not used. However, it should be appreciated that then many correct points will also be removed, which may be undesirable or even critical in some situations. With some further assumptions that may be valid in some situations, namely that the correct one in case of a conflict is always the point closest to the camera, some improvement may be achieved without the removal of correct points, or at least with less removal. However, there are situations when this method also results in the correct point being removed, i.e. when the correct one is not the closest point to the camera. This can happen, for example, when the bad point is due to noise or reflection, which can result in an incorrect point being closer to the camera and / or further away from the camera relative to the correct point. In that case, always removing the closest point to the camera can even have an aggravating effect, since the correct point may be removed and the incorrect point may be kept.

[0039] The embodiments herein provide an improvement on the above. In short, the embodiments herein may be described as being based on statistical methods and / or consensus voting. All or many points of the point cloud may be checked for conflicts with other points of the group, but rather than considering only those points that are in conflict with each other and then removing all or some of the conflicting points, each point may be attributed a score each time it is identified as being involved in a conflict with another point. That is, when a point of the point cloud is involved in a conflict, its score is increased, and this is done for all points of the point cloud. As a result, the total score of each point will indicate, and even be proportional to, the number of conflicts in which it has been involved in total. Thus, it is possible to compare and evaluate points not only whether they are involved in a conflict or not, but also by their total score, i.e., the total number of conflicts in which they are involved. Then, all points involved in conflicts that exceed a certain score corresponding to a threshold value may be removed. In other words, points involved in the most conflicts are removed using this approach, without making any removal decision based on the mere identification of the conflicts as such. Truly incorrect points, i.e. points that are not in a position that corresponds to an imaged real surface location on a real-world measurement object, e.g., measurement object 220 or 320, will typically, at least on average, be involved in more conflicts than other points and will therefore be removed to a greater extent using techniques according to embodiments of the present specification.

[0040] Said threshold value may be predetermined, but in a flexible manner. For example, a slider bar for changing the threshold value may be implemented and the result may be visualized directly in the virtual 3D object drawn based on the remaining surface points of the point cloud. This allows the user to find, for a given setup, a suitable threshold level that removes points that would otherwise produce artifacts when the 3D object is drawn based on the remaining surface points of the point cloud. Typical artifacts are spikes that occur in the 3D virtual object when it is drawn. At the same time, the user can ensure that a threshold value that is too low is not used, i.e. a threshold value that would cause the removal of real object details that are desired to be preserved.

[0041] FIG. 3C is a side view of the measurement object 320 as shown in FIG. 3B. In this view, an exemplary surface 325, excluded in FIG. 3B, that supports the measurement object 320 and on which the measurement object 320 resides is also shown. The surface 325 may be, for example, part of a conveyor band or the like, as may be used, for example, during optical triangulation as described above, or may be a surface that is fixed, for example, part of a table or the like. In the latter case and in optical triangulation, the camera and light source may move to achieve a scan of the measurement object 320. Note that the surface 325 does not need to be imaged, and may be, for example, outside the field of view or excluded in the point cloud. In some embodiments, there may be a surface or other structure that is located, for example, on the side or above the measurement object 320, rather than below, to which the measurement object is connected, for example, to ensure that the measurement object remains in a location, orientation, and / or is moved in a desired way during 3D imaging.

[0042] Figure 3D is a virtual view corresponding to the real world view in Figure 3C. Figure 3D shows diagrammatically some surface points P1-P15 (only some surface points are explicitly marked) of the point cloud in a 3D virtual coordinate system 353 in x', y', z', which corresponds to a virtual version of the real world coordinate system 323 in x, y, z. The point cloud thus now comprises surface points of a virtual object corresponding to the measurement object 320. The measurement object, and how its side view would look if fully modelled, is indicated by a thin dotted outline 361 in Figure 3D, which is shown for reference only.

[0043] Compare for example Figures 3C-3D with the situation illustrated in Figure 2 described above regarding how a point cloud 260 in the 3D virtual coordinate system 253, i.e. in x', y', z', can be formed by 3D imaging, in this example by optical triangulation and line scanning, of the measurement object 320 in the real world coordinate system 323, i.e. in x, y, z. Thus, the surface points P1-P15 in Figure 3D can be of a point cloud resulting from 3D imaging of the measurement object 320, e.g. resulting from optical triangulation and line scanning using the imaging system 305.

[0044] To illustrate the above-mentioned situation where surface points are assumed to be connected to their nearest neighbors, surface points P1-P15 are shown as being connected to each other. This is visualized in the figure to facilitate understanding regarding the conflicting points described next. The connected points may generally be points that would have been connected if tessellation or the like were to be applied to generate a 3D virtual object from the points, as described above.

[0045] Figure 3D, and also Figures 3E-F, described further below, are used below to diagrammatically illustrate and explain and illustrate the principles behind embodiments herein, such as with regard to determining which surface points are in conflict with each other and which are not, and are thus intended to illustrate the case when a setup such as that of Figure 3A is used.

[0046] Figure 3D is for illustrating an example when the point under investigation, i.e. the point for identifying a conflict for Pi, is point P4, i.e. in this example Pi=P4. Figure 3E is for the corresponding situation, but when Pi=P12 and Figure 3F when Pi=P13. As mentioned above, checking for a conflict for a point is to identify which other points, if any, are in conflict with said point, or in other words which other points, if any, cannot validly exist at the same time as the point being investigated, i.e. the point for which a conflict is being identified.

[0047] In Figures 3D-3F, point C'(i) corresponds to the camera location for point Pi in the 3D virtual coordinate system 353, i.e. in the same coordinate system as the points of the point cloud, here P1-P15. The location of C' may be referred to as the virtual camera location. The camera location may suitably correspond to and / or be represented by the camera center C. Those skilled in the art will be able to determine the camera center location given the type of camera and the setup used, or if other representations of the camera position are more suitable, e.g. as explained further below.

[0048] It should be noted that depending on the 3D imaging system and setup used for the 3D imaging of the measurement object 320, i.e. the imaging on which the surface points of the point cloud give rise to the image on which they are based, the camera location C' in the 3D virtual coordinate system x', y', z' may vary from surface point to surface point. In the case of optical triangulation using line scanning, the camera location C' is the same for the light rays illuminating the measurement object and is therefore the same for each contour image, e.g. for each contour image 141-1..141-N. However, it should be appreciated that the camera position C' varies with the position in y', since the time dimension, i.e. the time when each image frame is captured, is utilized to provide information on the third dimension for the 3D virtual coordinate system, more specifically on y' in the examples presented herein. In these examples, i denotes the position along y', which means that the camera position C' depends on i, i.e. C'(i). In other words, the camera position C' in x', y', z' is the same for the same y' and i. However, if z' changes for point P, as for example the case for P13 compared to P3-P12, C'(i) will still be at the same position as if P13 were at the same z' as P3-P12, which means that the camera orientation will be different for P13. In these examples, this can be seen in FIG. 3F. The camera orientation, i.e. the direction in x', y', z' from surface point Pi to camera position C' for that surface point Pi, can be represented in this example by a virtual ray 352, d, that depends on C' and Pi, and thus can be represented as d(C',Pi). Since d and the camera orientation are in a 3D virtual coordinate system, here 3D virtual coordinate system 353, i.e. x', y', z', 353, such a camera orientation may be referred to as a virtual camera orientation.

[0049] It should be noted that the dependence of C' on the position of P does not necessarily always hold for all relevant 3D imaging systems, i.e., 3D imaging systems that may be used to provide image data on which a point cloud may be based and which may then be improved by the embodiments herein, and other dependencies beyond the examples herein are of course possible.

[0050] The embodiments herein may be generally applied to surface points of a point cloud regardless of how the image, i.e., the image on which the point cloud is based, is provided, but there should be a predefined mapping from the real measured object, via the camera and image sensor, to the points of the point cloud, and thereby also to the virtual 3D object based on the point cloud.

[0051] In some cases, the camera location C' may be the same or approximated to be the same for all surface points P of the point cloud or part of the point cloud to which the embodiments herein are applied. In general, however, to cover all possible situations, the camera location C' for a surface point P may be considered to depend on the position of the surface point P in the point cloud, i.e., C'(x',y',z'), but the camera position C' is expected to be the same at least for surface points of a point cloud based on one and the same image frame captured by the camera. The camera position C' in the 3D virtual coordinate system may be considered to be predefined or even predetermined when the embodiments herein are applied at the surface points of the point cloud. The camera position C' may be considered to be given or predetermined by the 3D imaging system and setup used when capturing the image from which the surface points of the point cloud are derived.

[0052] Note that while Figures 3D-3F only show one side view, i.e., a "slice" or plane along y' for some x', i.e., the z'-y' plane, C' does not, of course, have to lie in the same plane, although this may be assumed for simplicity in subsequent examples illustrating the general principles behind how a score may be calculated for each surface point P. In practice, the virtual ray d from point P to camera position C' is typically a line in 3D space, i.e., in all dimensions x', y', z', rather than only in y', z'.

[0053] The same, of course, applies to sets of points that do not just lie in a single z'-y' plane, such as points P1-P15 in this example, because these points are of a 3D virtual object and therefore comprise points at different positions in all three coordinates of the 3D virtual coordinate system 353, e.g. in all x', y', z'.

[0054] Even when embodiments herein operate, for example, on a surface point P per z'-y' plane among several different such planes with the approximation that the camera position C' is located where the actual camera position is projected in that plane, improvements based on embodiments herein due to removal of surface points have been observed compared to keeping all surface points of the point cloud. However, further improvements are possible by using the actual or more accurate camera position C'.

[0055] As already indicated above, other representations of the camera position C' are possible other than the camera center. For example, if an orthographic camera is used, e.g., the camera 330 is orthographic, such as a camera with a telecentric lens, or if the camera used is approximated by an orthographic camera, i.e., a camera with parallel rays, rather than from the camera center, the result in the example of Figures 3D-3F will be that all camera orientations will have the same, i.e., parallel, virtual rays from each point P, regardless of x' and z'. Thus, the virtual rays from P13 will then be at angles such as from P12 and from P4. Although a truly orthographic camera was not used, improvements from application of the embodiments herein were observed even when the camera orientations were approximated as if this were the case.

[0056] If the camera and camera position C' are placed further apart, the effect on the camera orientation in the virtual coordinate system x', y', z', i.e. on the virtual ray d from the surface point P to the camera, will be similar to that of an orthogonal camera, i.e. the camera orientations for all surface points P approach one and the same direction.

[0057] In practice, if an approximation regarding the camera position C′ and / or the camera orientation is considered, e.g., to facilitate and / or simplify practical implementation, routine testing and experimentation, e.g., involving simulation, can provide information on whether the approximation is sufficient, e.g., in view of the requirements to be met. For example, a person skilled in the art may apply the embodiments herein in testing and experimentation both when using the true camera position C′ and when using a desired approximation, and then compare the results, e.g., comparing a 3D virtual object based on surface points when the real camera position C′ is used with another 3D virtual object based on surface points when the approximated camera position C′ is used instead.

[0058] Next, with reference to Figures 3D-3F, an example follows to explain the general principles behind how a score can be calculated for each surface point Pi.

[0059] 3D-3F it can be seen that surface points P1-P15, except for P13, follow the contour 361 well, at least as well as might be expected with a surface point density such as that in this example. Point P13 should therefore be understood as an example of an incorrect surface point, i.e. one that does not correspond to a real surface point on the real-world measurement object 320.

[0060] As already mentioned above, the example of Fig. 3D is specifically intended to illustrate the identification or investigation of surface points that are in conflict with surface point P4. This figure shows two "shadowing" or occlusion areas A and B for point Pi=P4. It should be appreciated that with the camera position C' shown in this figure and the assumptions made above that the measurement object 320 has no "hanging in the air" parts and therefore no surface points, the following conclusions can be made: If there is any other surface point P in area A, P4 will be occluded, i.e. hidden, from view by the camera. Thus, any point in area A is a point that is in conflict with P4, i.e. cannot effectively coexist with P4, but as can be seen, there is none in this example. The situation with "shadowed" or occluded points can be compared to a similar situation in the "real world" described above with respect to FIG. 3C, where the exemplary dummy object 321a is "shadowed" by the measurement object 320 and cannot be seen by the camera, but the exemplary dummy object 321b is not.

[0061] A corresponding situation as in the case of area A occurs in area B of FIG. 3D. P4 becomes occluding any point in area B, and therefore each point in area B also becomes a conflict point with P4. As can be seen in this example, only P13 is in area B, and therefore is a conflict point with P4. Thus, both P4 and P13 may be temporarily given a respective score of 1, or any existing scores for these points may be increased by 1. For example, if P4 and P13 started at 0, and this was the first identified conflict, P4 and P13 may both have a score of 1 resulting from the investigation of the conflicting points with P4. Based on this principle, it can be seen that P1-P3 are not involved in conflicts in the illustrated example.

[0062] It should be noted that areas A and B may also be influenced by the direction of the lighting used, as will be appreciated by those skilled in the art, and are shown in the examples herein with lighting as in FIG. 3A. A virtual lighting direction 354 is shown in FIG. 3D, and it should be understood that, as in the case of the camera, there may be a virtual light source position that depends on the position in x', y', z'. However, in the example shown, the virtual lighting direction from each point P is always vertical, i.e., along z'. Areas A and B may differ if such or corresponding ones apply in other cases, and therefore depending on the imaging system and setup used for these cases. In general, for point P, its area A is within the area in the direction towards the camera, and its area B is the area away from the camera. Both areas are such that they are conflicting points, i.e. points that cannot effectively coexist with point P, if any other point of the set is located within them. Where the points that cannot effectively coexist are located depends on the imaging system and setup used for 3D imaging. This, and thereby what corresponding areas A and / or B should be applied, can be determined by one skilled in the art based on the imaging system and setup being used in a particular case.

[0063] FIG. 3E shows a corresponding exemplary situation as in FIG. 3D, but areas A and B are shown for Pi=P12, i.e., i=12. Here again, only P13 is a conflicting point in B for P12. In other words, P12 and P13 may have their scores increased by 1 since they are both involved in conflicts with other points. Furthermore, it can be seen that if all points P1-P12 would have been investigated correspondingly, the result after conflicting points have been identified and scores increased for up to and including P12 would be as follows:

[0064] [Table 1]

[0065] 3F shows yet another example situation where there are areas A and B for Pi=13, i.e., i=13. Here, points P9-P12 are within area A and are therefore conflicting points for P13. From this figure, it can be further seen that P14-P15 are not involved in any conflict with each other or with any of the other points P1-P13 in this example. Thus, in the shown example, the resulting scores after investigating and identifying conflicting points for all of the shown points P1-P15 are as follows:

[0066] [Table 2]

[0067] In other words, using a threshold in the range 3-13 and removing all points with a score equal to or greater than this threshold will result in the removal of point P13 from point cloud P1-P15. Thus, the incorrect point P13 will be removed. It should also be appreciated that the newly created point cloud without P13 will better match the outline 361 and therefore will be a better representation of the real world object than if P13 were still part of the point cloud.

[0068] In this example, it is typically not required to look for conflict points in both areas A and B, i.e. both towards and away from the camera. For implementation reasons, it may be preferable to look for each point P only in one direction, e.g. towards or away from the camera position C', i.e. either in area A or B. If checking only in area B, the result in the shown example would be:

[0069] [Table 3]

[0070] Again, it should be appreciated that P13 is responsible for the most conflicts. Thus, this principle works if, for each surface point, we look for conflicting points in either area A or B. However, actual incorrect points will typically get a higher score when both areas are used, and therefore in some situations more incorrect points may be removed by the threshold.

[0071] It may not be necessary to use area A and / or area B that extend to the end of the point cloud for each surface point. Instead, it may be sufficient to only check a certain distance along the camera direction, i.e. along the virtual ray d of each surface point, towards and / or away from the camera. For example, with reference to the example shown in Figures 3D-3F, assume that only the five closest consecutive y' positions in area B are checked: i.e. for P4 with i=4, points P5-P9 are checked for conflicts, i.e. using area A that extends from P4 to P9 along y'. Applying this for all points P1-P15 and otherwise the same principle as above, the result is:

[0072] [Table 4]

[0073] Thus, even in this situation and example, by using a suitable threshold, it is possible to identify P13 as the point to be removed.

[0074] Fig. 4 is a flow chart for illustrating in a schematic manner an embodiment of a method according to an embodiment of the present specification. The following actions may form a method for removing erroneous or potentially erroneous points, corresponding to outliers as described in the background art, from a set of points, e.g. the set 355-1..355-15 exemplified by point P, i.e. from a point cloud. In practice, the point cloud in question for applying the embodiments of the present specification is of course much larger than only 15 as in the case of the simplified example above. The number of points of the set may be in the order of millions of points, just to give one example. The points are points of a 3D virtual object provided by 3D imaging of a corresponding real-world object, e.g. the measurement object 320, using a camera with an image sensor, e.g. the camera 330 and the image sensor 331.

[0075] Furthermore, the points of the set are typically surface points of a 3D virtual object, which is typically the surface of the real-world object being imaged, but may be, for example, an image through the actual outermost surface of a partially transparent object, and thus the virtual object and the set of points may possibly correspond to an interior surface of the real-world object.

[0076] In some embodiments, the 3D imaging is based on optical triangulation comprising illumination of a real-world object, e.g., measurement object 320, by a light source, e.g., light source 310, where the light emission is reflected light from the surface of the real-world object resulting from the illumination.

[0077] 4 may be performed by one or more devices, such as the camera 330 and / or the computing device 333, or by the imaging system 305 and / or a suitable device therein or connected thereto. Methods and devices for performing the actions thereof are further described below.

[0078] Please note that the following actions may be taken in any suitable order and / or may overlap in whole or in part, when this is possible and suitable.

[0079] Action 401 The set of points is obtained. The set of points may be obtained, for example, completely or partially, by the same device or devices involved in performing the method, from measurements and calculations by a 3D imaging system and / or by being received from other devices or units that may be part of the 3D imaging system, but alternatively, may be received from other, for example external, devices or units that store the set of points. Furthermore, the set of points may correspond to a point cloud and may be points corresponding to, i.e. covering, a complete real-world object or partially thereto, i.e. the points may be of a 3D virtual object that corresponds to only a partial 3D view of the real-world object.

[0080] Action 402 For each point, e.g., for each one of points 355-1..355-15, a respective camera direction may be obtained, e.g., corresponding to camera direction 352. Each camera direction corresponds to a direction of light emission from a corresponding point of a real-world object toward the camera, the light emission of which is detected by an image sensor during the 3D imaging. The virtual light ray 352 above is an example of such a camera direction. As mentioned above, the obtained camera directions may be referred to as virtual camera directions, because they are directions for the points of the set, and thus with respect to the coordinates of the 3D virtual coordinate system used for the set. A conflicting point may be a point of the set that cannot usefully coexist with the respective point, at least based on its camera direction.

[0081] The camera orientation may be obtained by being determined as indicated above, such as being calculated and / or estimated based on knowledge of the 3D imaging system and setup used when the image based on the points was sensed. The set of points is typically associated with a predefined mapping of how the positions in a real-world coordinate system, e.g., x, y, z, in which the 3D imaging system and real-world object, e.g., imaging system 305 and measurement object 320, are located, relate to the coordinates of a corresponding 3D virtual coordinate system, x', y', z', in which the virtual object and said set of points are. The mapping is typically provided by a calibration performed for each 3D imaging system and setup used for 3D imaging, and is therefore typically pre-determined when the embodiments herein are performed, and at least the mapping may be considered to be pre-defined or given by the 3D imaging system and setup used when the image was sensed. Thus, with knowledge of the camera positions in x, y, z, the corresponding positions in x', y', z' may be determined, and from this the camera orientation for the points may also be determined.

[0082] Camera orientations may be predetermined for various positions in, for example, x', y', z' for the imaging system and setup used, and the predetermined camera orientations may then be used to determine camera positions for the points of the set of points.

[0083] In some embodiments, a pre-determined camera orientation, or even a camera orientation for the set of points, may be obtained by being received fully or partially from another device or unit, e.g., one that determines, e.g., calculates, the camera orientation.

[0084] As indicated above, the determination of whether a point is a conflicting point, i.e., whether a point can usefully coexist with each other, may also take into account a virtual light direction for the point, i.e., a virtual direction of incident light, which corresponds to the direction of incident light on the real-world object, e.g., from light source 310, that is reflected by the real-world object and becomes the light detected by the camera and image sensor, such as in the case of optical triangulation. The example virtual lighting direction 354 described above corresponds to such a virtual light direction.

[0085] Thus, in some embodiments, the action further includes obtaining, for each point, a respective virtual light direction, which corresponds to the direction of incident light, e.g., from a light source, on a corresponding point on the real-world object. In these embodiments, the conflicting points may be a set of points that cannot usefully coexist with the respective point based further on its virtual light direction.

[0086] Action 403 For each point, e.g., for each one of points 355-1..355-15, a set of conflicting points, if any, are identified. A conflicting point is a set of points that cannot validly coexist with the respective point according to one or more predefined criteria. One such criterion may be based on the camera orientation obtained for the point, as described above under action 402. It should be appreciated that the one or more criteria may further be based on in principle anything that may be used to determine what can be simultaneously seen and imaged and what cannot be for the 3D imaging system and setup used. For example, in the case of optical triangulation and as illustrated above, when there is, e.g., optical direction involved, the one or more criteria may further be based on a virtual optical direction, e.g., to determine areas A and / or B, or the like, as in the above example. If light cannot reach a point, this may be an additional reason why the point cannot be seen by the camera, and thus the virtual light direction may be used together with the virtual camera orientation to identify points that cannot be validly present with other points, for example to achieve improved predefined criteria for application. However, since it is implicit that some light must have reached the camera by correct imaging of the real-world object, a criterion based on the camera orientation, and the assumption that lighting has reached all points, may be sufficient.

[0087] In some embodiments, conflicting points are points of the set, e.g., points 355-1..355-15, that are connected to their nearest neighboring surface points, or at least some of their nearest neighboring surface points, by a coherent surface and cannot validly coexist with the respective points based on the assumption that the coherent surface would block light, e.g., light emission as described under action 402. The points to which the respective points are connected by a coherent surface may correspond to points to which the respective points were or will be connected in order to generate a 3D virtual object from points, e.g., by tessellation and / or according to any prior art method for generating a 3D virtual object from a point cloud.

[0088] Further, in some embodiments, the identification of conflicting points for each point is limited to a set of points that are within a certain distance from the respective point. This is illustrated and described in some detail above in connection with Figures 3D-3F and may be particularly important for applications to complex real-world objects and / or large sets of points, where it is undesirable and / or impractical to search through the entire point cloud to identify conflicting points for each point. For example, it may be sufficient to identify conflicting points that are within a distance that includes the nearest neighboring point location, e.g., pixel, or rather voxel location since in 3D, where there is a potentially conflicting point, if any. The certain distance may be predefined or predetermined, and in a particular application may be based on routine experimentation and testing to find a suitable and / or acceptable distance for the application in question.

[0089] Limiting identification to over a distance reduces processing and allows for implementations that can run faster.

[0090] Furthermore, conflicting points located far away may be unlikely to occur, and it may not be efficient to expend resources, if any, on identifying very distant conflicting points. Also, if remote conflicting points exist, such are likely to be identified as points with many conflicts anyway when conflicts are identified for points closer to such points.

[0091] For example, a similar effect to limiting over a distance can be achieved by dividing a larger set of points or applying the embodiments herein on a partial set of the complete set of virtual objects, e.g., the partial set output during line scanning and optical triangulation.

[0092] For example, for each point in a point cloud to which an embodiment of the present disclosure is applied, the number of conflicts identified for each point may be kept counted, e.g., by incrementing a score for each point as described above.

[0093] Information such as a count of the identified conflicts for each point may be stored. For a set of points, for example, after conflicting points for each point in the set have been identified, the points are involved in various numbers of conflicts, a value and / or identifier may be stored for each point that identifies or indicates the total number of identified conflicts in which the point was involved.

[0094] Action 404 Based on said identification for each point, i.e., based on action 403, one or more points of the set that have been involved in a conflict more times than other points of the set that are involved in a conflict are removed from the set. Each point of the set, e.g., point 355, which is any one of points 355-1..355-15, is considered to be involved in a conflict whenever a conflicting point is identified for it in the set, i.e., when and / or whenever the respective point is itself identified as a conflicting point with respect to other points of the set. This has been illustrated and described in some detail above in connection with Figures 3D-3F.

[0095] In some embodiments, points involved in conflicts more than a predefined threshold number of times are removed from the set. The predefined threshold, typically a threshold or number, may be as described above and, in some embodiments, may be predetermined.

[0096] In some embodiments, the one or more predefined criteria are based on what a camera, e.g. camera 330, can see virtually, i.e. theoretically, from its corresponding position in the coordinate system of the set of points, i.e. the coordinate system used for the positions of the set of points, e.g. virtual coordinate system 353. The corresponding position for the camera can be found from a predefined or predetermined mapping between the real world coordinate system of the measurement object, e.g. x, y, z, and the virtual coordinate system of the virtual object and set of points, e.g. x', y', z', as described above. The camera position can be represented by the camera center, or by other representations, also as described above.

[0097] Thus, what the camera can virtually, i.e., theoretically, see may be based on the set of points and the camera position in the same coordinate system as the set of points, e.g., at x', y', z'. Note that this may differ from what the camera could actually see in the real world. The advantage of leveraging what the camera can or could theoretically, i.e., virtually, see is that if what the camera could actually see needed to be determined, all calculations can be done in x', y', z' using information and data that is typically already available, including the set of points, without the need for any additional and often complex real-world measurements and then further mapping to x', y', z'.

[0098] As already mentioned above, the embodiments herein provide an improvement over deleting or removing each conflicting point identified, which in many circumstances results in correct points also being removed in case of conflicts. In the embodiments herein, more correct points may remain, but still truly incorrect points, i.e. points that are not at a position corresponding to an imaged real location on the real-world object, e.g., measurement object 220 or 320, may be removed.

[0099] The embodiments herein can be compared with solutions as in the prior art described in the background art, as explained above, where the decision for the removal, i.e. removal, of a part of the contour is made based on the presence of a blind spot area for the contour, i.e. based on the fact that there is a conflict, and thus the removal is made for all identified conflicts, in other words, in the prior art, the removal is based on the fact that a conflict has been identified, not on how many conflicts a point has been involved in.

[0100] Furthermore, embodiments herein may be efficient for removing false points caused by noise or reflections. In the case of 3D imaging based on optical triangulation, reflections may be reduced or removed as part of intensity peak finding, but in embodiments herein, false points due to reflections may additionally or alternatively be removed from the point cloud.

[0101] 5A-5B show virtual 3D object images of a measurement object without and with application of embodiments herein, respectively.

[0102] 5A shows an image of a 3D virtual object based on a set of acquired points, i.e., a point cloud, without application of the embodiments herein. The point cloud is here provided conventionally from prior art optical triangulation. The point cloud may correspond to the one acquired in action 401 and on which the embodiments herein operate. As can be seen, the point cloud comprises outliers, i.e. incorrect points, which cause distortions visible as "spikes" on the object shown in the image.

[0103] FIG. 5B shows another image of the corresponding 3D virtual object based on the same acquired set of points, but after application of an embodiment of the present specification, i.e., after removal of points involved in conflicts above a certain threshold. In the example shown, an embodiment of the present specification has been applied to a set of about 30000 points, and for each point, conflicting points, if any, within a distance of 25 pixels apart were identified. The score for each point was initially 0, and each identified conflict resulted in an increase in the score by 1. A total score threshold of 3 was used, and all points with a total score above this threshold were removed from the set. The result was the removal of 1804 points. The improvement can be clearly seen, the "spikes" are no longer present on the image shown.

[0104] Fig. 6 is a schematic block diagram for illustrating an embodiment of one or more devices 600, i.e. device 600, which may correspond to the devices already described above for performing the embodiments of the present specification, such as for performing the methods and / or actions described above with respect to Fig. 4. Thus, device 600 is for removing false points from a set of points of a 3D virtual object provided by 3D imaging of a corresponding real-world object using a camera with an image sensor, such as provided by 3D imaging of a measurement object 320 using a camera 330 with an image sensor 331. Device 600 may correspond for example to computing device 333 and / or camera 330 or a suitable device included in and / or forming a 3D imaging system comprising a camera and an image sensor, e.g. imaging system 305, or may be a device, e.g. a computer, external to the 3D imaging system providing the set of points.

[0105] The schematic block diagram is intended to illustrate an embodiment of how device 600 may be configured to perform the methods and actions described above with respect to FIG.

[0106] The device 600 may comprise a processing module 601, such as a processing means, e.g., one or more processing circuits such as a processor, one or more hardware modules including circuitry, and / or one or more software modules for performing the methods and / or actions.

[0107] The device 600 may further comprise a memory 602, which may comprise, include, store, or otherwise comprise a computer program 603. The computer program 603 comprises "instructions" or "code" directly or indirectly executable by the device 600 to perform the methods and / or actions. The memory 602 may comprise one or more memory units and may further be arranged to store data, such as configurations, data, and / or values ​​involved in or for performing the functions and actions of the embodiments herein.

[0108] Additionally, device 600 may comprise, as an illustrative hardware module, processing circuitry 604 involved in processing and, e.g., encoding, data, and may comprise or correspond to one or more processors or processing circuits. Processing module 601 may comprise, e.g., be "embodied in" or "implemented by," processing circuitry 604. In these embodiments, memory 602 may comprise a computer program 603 executable by processing circuitry 604, such that device 600 is operable or configured to perform the methods and / or actions thereof.

[0109] Typically, device 600, e.g., processing module 601, comprises an input / output (I / O) module 605 configured to participate in, e.g., by executing, any communication to and / or from other units and / or devices, such as transmitting information to and / or receiving information from other devices. I / O module 605 may be exemplified by an acquisition module, e.g., a receiving module, and / or a providing module, e.g., a transmitting module, when applicable.

[0110] Further, in some embodiments, device 600, e.g., processing module 601, comprises one or more of an acquisition module, an identification module, and a removal module as illustrated hardware and / or software modules for performing actions of embodiments herein, which may be implemented fully or partially by processing circuitry 604.

[0111] Therefore, the result is as follows:

[0112] The device 600, and / or the processing module 601, and / or the processing circuitry 604, and / or the I / O module 605, and / or the acquisition module are operable or configured to acquire said set of points.

[0113] The device 600, and / or the processing module 601, and / or the processing circuitry 604, and / or the I / O module 605, and / or the acquisition module may be operable or configured to acquire the respective camera orientation for the respective point.

[0114] The device 600, and / or the processing module 601, and / or the processing circuitry 604, and / or the identification module are operable or configured to identify, for said respective points, said conflicting points.

[0115] The device 600, and / or the processing module 601, and / or the processing circuitry 604, and / or the removal module are operable or configured to remove from the set, the one or more points of the set that have been involved in a conflict more times than other points of the set that have been involved in a conflict based on the identification for the respective points of the set.

[0116] FIG. 7 is a schematic diagram illustrating some embodiments of a computer program and its carrier for causing the device 600 to perform the methods and actions described above.

[0117] The computer program may be a computer program 603, which when executed by the processing circuitry 604 and / or the processing module 601, comprises instructions that cause the device 600 to execute as described above. In some embodiments, a carrier, or more specifically a data carrier, for example a computer program product, comprising the computer program is provided. The carrier may be one of an electronic signal, an optical signal, a radio signal, and a computer-readable storage medium, for example a computer-readable storage medium 701 as shown diagrammatically in the figure. Thus, the computer program 603 may be stored on the computer-readable storage medium 701. By carrier, a transitory propagating signal may be excluded, and the data carrier may be correspondingly referred to as a non-transitory data carrier. Non-limiting examples of data carriers that are computer-readable storage media are memory cards or memory sticks, disk storage media such as CDs or DVDs, or mass storage devices typically based on hard drives or solid-state drives (SSDs). The computer-readable storage medium 701 may be used to store data accessible over a computer network 702, for example the Internet or a local area network (LAN). The computer program 603 may further be provided as a pure computer program or may be included in one or more files. The one or more files may be stored on the computer-readable storage medium 701 and available, for example, through download, for example, via a server, for example, on a computer network 702 as shown in the figure. The server may be, for example, a web or file transfer protocol (FTP) server. The one or more files may be, for example, executable files, for direct or indirect download to and execution on the device, for example, by execution by the processing circuitry 604, to cause the device to perform as described above.One or more files may also, or alternatively, be for intermediate download and compilation, involving the same or other processor causing the device 600 to perform as described above to make the files executable before further download and execution.

[0118] It should be noted that any of the processing modules and circuits described above may be implemented as software and / or hardware modules, e.g., in existing hardware and / or as an application specific integrated circuit (ASIC), field programmable gate array (FPGA), etc. It should also be noted that any of the hardware modules and / or circuits described above may be included in, for example, a single ASIC or FPGA, or may be distributed among several separate hardware components, whether individually packaged or assembled into a system on a chip (SoC).

[0119] Those skilled in the art will also appreciate that the modules and circuitry described herein may refer to combinations of hardware modules, software modules, analog and digital circuitry, and / or one or more processors configured with software and / or firmware stored, for example, in memory, that when executed by one or more processors, may configure and / or cause a device, sensor, etc. to perform the methods and actions described above.

[0120] Identification by any identifier herein may be implicit or explicit. Identification may be unique in a certain context, for example, for a certain computer program or program provider.

[0121] As used herein, the term "memory" may refer to a data memory for storing digital information, typically a hard disk, a magnetic storage, medium, a portable computer diskette or disk, a flash memory, a random access memory (RAM), etc. Additionally, the memory may be an internal register memory of a processor.

[0122] It should also be noted that any enumerated terminology, such as first device, second device, first surface, second surface, etc., should be considered as such to be open-ended, and that the terminology, as such, does not imply any hierarchical relationship. In the absence of any explicit information to the contrary, the enumerated naming should be considered merely a way of achieving distinct names.

[0123] As used herein, the phrase "configured to" may mean that a processing circuit is configured or adapted to perform one or more of the actions described herein, using a software or hardware configuration.

[0124] As used herein, the term "number" or "value" may refer to any kind of number, such as a binary, real, imaginary, or rational number. Additionally, a "number" or "value" may be one or more characters, such as a letter or a string of letters. Additionally, a "number" or "value" may be represented by a string of bits.

[0125] As used herein, the phrases "may" and "in some embodiments" are typically used to indicate that the described features can be combined with any of the other embodiments disclosed herein.

[0126] In the drawings, features that may only be present in some embodiments are typically depicted using dotted or dashed lines.

[0127] When the word "comprise" or "comprising" is used, the word should be construed as being open-ended, i.e., meaning "consist at least of."

[0128] The embodiments of the present specification are not limited to the embodiments described above. Various alternatives, modifications, and equivalents may be used. Therefore, the above embodiments should not be interpreted as limiting the scope of the present disclosure, which is defined by the appended claims. [Explanation of symbols]

[0129] 105 Imaging system, system 110 Light source 111 Specific Light Patterns 112 Rays of light 120 First measurement object, first object 121 second measurement object, second object 122 Conveyor Belt 123 Coordinate Systems, Real World Coordinate Systems 130 Cameras, camera units 141-1~141-N Contour image 143 Sensor-based coordinate system 211 The First Light 212 Ray of light 220 Wedge-shaped measuring object, measuring object 253 3D Virtual Coordinate System 260 Point clouds, point sets 261 Dotted Line 305 Imaging system, system 310 First light source, light source 311 First light, reflected first light, light 320 Measurement object, real world measurement object, corresponding real world object, real world object 321a, 321b Dummy objects, exemplary dummy objects 323 Real World Coordinate System 325 Example Surface, Surface 330 Cameras, Devices 330a Virtual Frustum 330b Camera optical center C 331 Image Sensor 332 Field of view, light rays 333 Computing Devices 352 Virtual rays, camera directions, and respective camera directions 353 3D virtual coordinate system, x', y', z', virtual coordinate system, coordinate system 354 Virtual Lighting Direction 355 points, each point, each point itself 355-1..355-15 set of points, points in the set 361 Thin dotted outline, outline 600 devices 601 Processing Module 602 Memory 603 Computer Programs 604 Processing circuit configuration, processor 605 Input / Output (I / O) Module, I / O Module 701 Computer-readable storage medium

Claims

1. 1. A method for removing false points from a set of points (355-1..355-15) of a three-dimensional, 3D virtual object provided by 3D imaging of a corresponding real-world object (320) using a camera (330) with an image sensor (331), said method comprising: - obtaining (401) said set of points (355-1..355-15); - for each point (355), identifying (403) conflicting points, if any, in said set, where conflicting points are points of said set that cannot validly coexist with said respective point (355) according to one or more predefined criteria and that cannot validly coexist with said respective point (355) based on the assumption that said points (355-1..355-15) of said set are connected to nearest neighboring surface points by a coherent surface, which would block light; - removing (404) one or more points of the set that have been involved in a conflict more times than other points of the set based on the identification for each point (355) of the set, wherein each point (355) of the set is considered to be involved in a conflict each time a conflicting point is identified in the set for the respective point and / or each time the respective point (355) is identified as a conflicting point with respect to other points of the set; A method comprising:

2. The method of claim 1 , wherein points that are involved in a conflict more than a predefined threshold number of times are removed from the set.

3. 3. The method according to claim 1 or 2, wherein the one or more predefined criteria are based at least on what the camera (330) can virtually see from its corresponding position in a coordinate system (353) of the set of points (355-1..355-15).

4. The method comprises: - obtaining (402) for each point (355) a respective camera direction (352) corresponding to the direction of a light emission from the corresponding point of the real-world object (320) towards the camera (330), the light emission of which is detected by the image sensor (331) during the 3D imaging; Further comprising: The method of claim 1 , wherein a conflicting point is a point of the set that cannot usefully coexist with the respective point based at least on its camera orientation.

5. The method of claim 1 , wherein the identification of competing points for each point is limited to the set of points that are within a certain distance from the respective point.

6. 5. The method of claim 4, wherein the 3D imaging is based on optical triangulation comprising illumination of the real-world object (320) by a light source (310), and the light emission is reflected light from a surface of the real-world object (320) resulting from the illumination.

7. A computer program (603) comprising instructions that, when executed by one or more processors (604), cause one or more devices (600) to perform the method of claim 1.

8. A carrier comprising the computer program (603) according to claim 7, said carrier being a computer readable storage medium (701).

9. One or more devices (600) for removing false points from a set of points (355-1..355-15) of a three-dimensional, 3D virtual object provided by 3D imaging of a corresponding real-world object (320) using a camera (330) with an image sensor (331), said one or more devices comprising: obtaining (401) the set of points (355-1..355-15); identifying (403) for each point (355), conflicting points, if any, in the set, where conflicting points are points of the set that cannot validly coexist with the respective point (355) according to one or more predefined criteria and that cannot validly coexist with the respective point (355) based on the assumption that the points (355-1..355-15) of the set are connected to nearest neighboring surface points by a coherent surface, which would block light; and removing (404) from the set one or more points that have been involved in a conflict more times than other points of the set based on the identification for each point (355) of the set, wherein each point (355) of the set is considered to be involved in a conflict each time a conflicting point is identified for the respective point in the set and / or each time the respective point (355) is identified as a conflicting point with respect to other points of the set. One or more devices (600) configured to perform the steps of:

Images