Method and apparatus for removing false points from a set of points of a 3D virtual object provided by 3D imaging

Abstract

The present disclosure relates at least to a method and apparatus for removing erroneous points from a set of points of a 3D virtual object provided by 3D imaging. A method and apparatus for removing erroneous points from a set of points (355-1... 355-15) of a 3D virtual object provided by 3D imaging of a corresponding real-world object (320) by means of a camera (330) having an image sensor (331) is disclosed. The set of points (355-1... 355-15) is obtained (401). For a respective point (355) a conflicting point, if any, in the set is identified (403). A conflicting point is a point in the set that cannot coexist with the respective point (355) effectively according to one or more predefined criteria. Based on the identification, one or more points in the set that are involved in more conflicts than other points in the set are removed (404) from the set. Each time a conflicting point is identified in the set for a respective point and / or each time the respective point itself (355) is identified as a conflicting point of another point in the set, the respective point (355) in the set is considered to be involved in a conflict.

Description

TECHNICAL FIELD

[0001] Embodiments herein relate to a method and apparatus for removing false points from a set of points of a 3D virtual object provided from a 3D imaging of a corresponding real-world object by means of a camera pair having image sensors. BACKGROUND

[0002] Industrial vision cameras and systems for factory and logistics automation can be based on three-dimensional (3D) machine vision, where 3D images of scenes and / or objects are captured. A 3D image refers to an image that also contains "height" or "depth" information, and not only or at least not only information about pixels in only two dimensions (2D) as in regular images, e.g. intensity and / or color. That is, each pixel of the image can comprise information associated with the position of the pixel, and which maps to a position of an object (e.g. an object) that has been imaged. Processing can then be applied to extract information about characteristics of the object (i.e. 3D characteristics of the object) from the 3D image, and e.g. convert it into various 3D image formats. Such information about height can be referred to as range data, where the range data can thus correspond to data from a height measurement of the imaged object, or in other words data from a range or distance measurement of the object. Alternatively or additionally, the pixels can comprise information about e.g. material properties, such as information related to scattering of light or reflection of light of a certain wavelength in the imaged area.

[0003] Thus, a pixel value can e.g. relate to intensity and / or range data and / or material properties of the pixel.

[0004] Line scan image data is produced when the image data of an image is scanned or provided one line of pixels at a time, e.g. by a camera having a sensor configured to sense and provide the image data one line of pixels at a time. One special case of line scan images is image data provided by so-called "sheet of light" (e.g. laser line) 3D triangulation. Laser is often preferred, but other light sources capable of providing a "sheet of light" can also be used, e.g. light sources capable of providing light that remains focused and does not spread out too much, or in other words "structured light" (e.g. light provided by a laser or a light emitting diode (LED)).

[0005] 3D machine vision systems are often based on such sheet of light triangulation. In such systems, there is a light source that illuminates an object with a certain light pattern, such as a sheet of light as the certain light pattern, e.g. producing a light or laser line on the object, and along this line 3D characteristics of the object corresponding to the profile of the object can be captured. By scanning the object using such a line, i.e. performing line scanning, involving movement of the line and / or the object, 3D characteristics of the entire object corresponding to multiple profiles can be captured.

[0006] A 3D machine vision system or device using light sheets for triangulation can be referred to as a system or device for 3D imaging based on light, or light sheets, triangulation or simply laser triangulation when using lasers.

[0007] Generally, to produce a 3D image based on light triangulation, reflected light from an object to be imaged is captured by an image sensor of a camera and intensity peaks are detected in the image data. The peaks occur at positions in the image corresponding to positions on the imaged object having incident light (e.g. corresponding to a laser line) reflected from the object. The position of the detected peaks in the image will map to the position on the object of the light that produced the peaks.

[0008] A laser triangulation camera system, i.e. an imaging system based on light triangulation, projects a laser line onto an object to produce a height profile from the surface of the target object. By moving the object relative to the camera and light source involved, information about the height profile from different parts of the object can be captured by the image, which is then combined and used together with knowledge of the relevant geometry of the system to produce a three-dimensional representation of the object, i.e. to provide 3D image data. This technique can be described as a grab of the image when a light line, typically a laser line, is projected onto an object and reflected by the object, and then extracting the position of the reflected laser line in the image. This is typically done by identifying the position of intensity peaks in the image frame, e.g. using a conventional peak finding algorithm. The imaging system is generally, but not necessarily, set up so that intensity peaks can be searched for in each column of the sensor and the position within the column mapped to a height or depth.

[0009] Any measurement system, including 3D imaging systems such as the 3D machine vision system mentioned above, can produce outliers. Outliers are generally different based on the measurement technique used and thus depending on the 3D imaging system used, and there can be several different sources of outliers within the same system. By definition, outliers are unwanted and make the inspection task more difficult and sometimes even impossible in industrial measurement applications.

[0010] Outliers can correspond to image data that does not correctly represent and / or correspond to actual points of the real-world object being imaged.

[0011] When a 3D imaging system is used to generate 3D virtual objects from real-world objects being imaged, it typically provides a point cloud, i.e., a set of points corresponding to the surface points of the 3D virtual object. If the real-world object is in a real-world coordinate system x, y, z, then the 3D virtual object can be in a corresponding virtual coordinate system x', y', z'. Outliers can be represented in the point cloud as points that do not correspond to actual real points on the surface of the real-world object; that is, outliers correspond to erroneous points, and if the virtual object is drawn based on the point cloud, they can be considered, for example, as "peaks" on the virtual object.

[0012] Outliers can be generated, removed, and / or reduced at different steps during 3D imaging. In the case of laser triangulation as described above, some outliers can be reduced, for example, during intensity peak detection. Outliers not removed in earlier steps will eventually appear as erroneous points in the "point cloud".

[0013] U2020340800A1 discloses a solution, particularly for optical triangulation, that determines erroneous detection values ​​based on the presence of a contour in a blind spot region where height cannot be measured. In that case, portions of the contour identified as erroneous detection values ​​are removed. Summary of the Invention

[0014] In view of the above, the aim is to provide one or more improvements or alternatives to the prior art, such as providing improvements regarding reducing the negative impact of erroneous points corresponding to outliers in a point cloud (i.e., a set of points of a 3D virtual object provided by a 3D imaging of a corresponding real-world object).

[0015] According to a first aspect of the embodiments herein, this object is achieved by a method for removing erroneous points from a set of points of a 3D virtual object provided by a three-dimensional (3D) image of a corresponding real-world object obtained by means of a camera with an image sensor. Then, conflicting points (if any) in the set are identified for each corresponding point. A conflicting point is a point in the set that cannot coexist effectively with the corresponding point according to one or more predefined criteria. Subsequently, based on the identification of the corresponding points in the set, one or more points from the set that involve conflicts more frequently than other points in the set involved in conflicts are removed from the set. Each time a conflicting point is identified in the set for a corresponding point and / or each time the corresponding point itself is identified as a conflicting point of another point in the set, the corresponding point in the set is considered to be involved in a conflict.

[0016] According to a second aspect of the embodiments herein, this objective is achieved by a computer program comprising instructions, when executed by one or more processors, to cause one or more devices to execute the method according to the first aspect.

[0017] According to a third aspect of embodiments herein, the object is achieved by a carrier comprising a computer program according to the second aspect.

[0018] According to a fourth aspect of embodiments herein, the object is achieved by one or more devices for removing false points from a set of points of a three-dimensional (3D) virtual object provided by means of 3D imaging of a corresponding real-world object by means of a camera pair having image sensors. The device(s) are configured to obtain the set of points. The device(s) are further configured to identify, for a respective point, conflicting points, if any, in the set. A conflicting point is a point in the set that cannot coexist with the respective point effectively according to a predefined criterion or criteria. Moreover, the device(s) are configured to remove, from the set, one or more points in the set that are involved in more conflicts than other points in the set based on said identification of conflicting points for respective points of the set. A respective point in the set is considered to be involved in a conflict each time a conflicting point is identified in the set for the respective point and / or each time the respective point itself is identified as a conflicting point for another point in the set.

[0019] Embodiments herein provide an improvement compared to the deletion or removal each time a conflicting point is identified in that it in many cases results in that also correct points are removed in case of a conflict. For embodiments herein, more correct points can be kept while still removing truly incorrect points, i.e. points corresponding to outliers, not in positions corresponding to actual positions on the real-world object that is imaged. Embodiments herein can be based on statistical methods and / or a “consensus voting” and can be compared to the solution mentioned in the background, where the decision to remove is based on identifying a conflict, not considering the total number of points and not based on how many conflicts a respective point is involved in. BRIEF DESCRIPTION OF DRAWINGS

[0020] Examples of embodiments are described in more detail below with reference to the attached drawings, which are briefly described below.

[0021] Fig. 1 schematically illustrates an example of a prior art 3D imaging system based on light triangulation and embodiments herein can also use and / or be based on light triangulation.

[0022] Fig. 2 schematically illustrates how a simplified example of a point cloud in the prior art is generated by means of the 3D imaging system in Fig. 1.

[0023] FIG. 3A An imaging system and an example of a measurement object that can be configured to perform embodiments herein are schematically illustrated.

[0024] FIG. 3B A 3D view of a part of the imaging system and the measurement object shown in Fig. 3 is schematically illustrated. FIG. 3A A 3D view of a part of the imaging system and the measurement object shown in Fig. 3 is schematically illustrated.

[0025] FIG. 3C Fig. 1 schematically illustrates FIG. 3A an imaging system and a side view of a measured object as shown in -B.

[0026] FIG. 3D Fig. 1 schematically illustrates FIG. 3C points of a point cloud and corresponds to a virtual view of the measured object side view as shown in -B.

[0027] FIG. 4 is a flow chart for schematically illustrating embodiments of a method according to embodiments herein.

[0028] FIG. 5A -B shows virtual 3D object images of a measured object without and with application of embodiments herein, respectively.

[0029] FIG. 6 is a schematic block diagram for illustrating how one or more devices can be configured to perform embodiments of the methods and actions discussed in relation to FIG. 4

[0030] FIG. 7 is a schematic diagram illustrating some embodiments in relation to a computer program and carrier thereof, to cause a device(s) to perform the methods and actions discussed in relation to FIG. 4 DETAILED DESCRIPTION

[0031] Embodiments herein are example embodiments. It should be noted that the embodiments are not necessarily mutually exclusive. It can be assumed by default that components from one embodiment exist in another embodiment, and it can be obvious to a person skilled in the art how those components can be used in other example embodiments.

[0032] ​​Fig. 1 schematically illustrates an example of an imaging system of the type referred to in the background, i.e. an imaging system 105 for 3D machine vision based on light triangulation for capturing information about 3D characteristics of a measurement object. The system can be used to provide a 3D image in the form of a set of surface points (i.e. a point cloud) on which the embodiments herein can operate as further described below. The system 105 shown in the figure is in a normal operating situation, i.e. typically after calibration has been performed and thus the system is calibrated. The system 105 is configured to perform light triangulation, here in the form of light sheet triangulation as referred to in the background. The system 105 further comprises a light source 110, e.g. a laser, for illuminating a measurement object to be imaged with a specific light pattern 111 (illustrated and shown as a light sheet in the figure). The light can but need not be a laser. In the example shown, the target object is illustrated as a first measurement object 120 in the form of a car and a second measurement object 121 in the form of a gear configuration. When the specific light pattern 111 is incident on the object, which corresponds to a projection of the specific light pattern 111 on the object, this can be observed when the specific light pattern 111 intersects the object. For example, in the example shown, the specific light pattern 111, illustrated as a light sheet, results in light rays 112 on the first measurement object 120. The specific light pattern 111 is reflected by the object, more specifically by the part of the object at the intersection, i.e. at the light rays 112 in the example shown. The system 105 further comprises a camera 130, which comprises an image sensor (not shown in Fig. 1). The camera and the image sensor are arranged relative to the light source 110 and the object to be imaged such that the specific light pattern, when reflected by the object, becomes incident light on the image sensor. The image sensor is a device, typically implemented as a chip, for converting the incident light into image data. The part of the object that by the reflection causes said incident light on the image sensor can thereby be captured by the camera 130 and the image sensor, and corresponding image data can be produced and provided for further use. For example, in the example shown, the specific light pattern 111 will reflect the light rays 112 on a part of the roof of the first measurement object 120 towards the camera 130 and the image sensor, whereby image data can be produced and provided that has information about said part of the roof. With knowledge of the setup (including the geometry) of the system 105, e.g. how the image sensor coordinates relate to real world coordinates such as coordinates of a coordinate system 123 (e.g. Cartesian) related to the object to be imaged and its context, the image data can be converted into information about 3D characteristics (e.g. 3D shape or profile) of the object imaged in an appropriate format. The information about said 3D characteristics, e.g. said 3D shape(s) or profile(s), can comprise data describing the 3D characteristics in any appropriate format.

[0033] By moving, e.g. the light source 110 and / or the object to be imaged, such as the first measurement object 120 or the second object 121, so that multiple portions of the object are illuminated and cause reflected light on the image sensor, in practice, typically by scanning the object, it is possible to produce image data describing a more complete 3D shape of the object, e.g. corresponding to multiple consecutive profiles of the object, such as the profile images 141-1-141-N of the first measurement object 120 shown, where each profile image shows the outer shape of the first object 120 reflecting the specific light pattern 111 when the image sensor of the camera unit 130 senses the light producing the profile image. As shown, a conveyor belt 122 or similar can be used to move the object through the specific light pattern 112, where the light source 110 and the camera unit 130 are typically fixed, or the specific light pattern 111 and / or the camera 130 can be moved over the object so that all portions of the object, or at least all portions facing the light source 110, are illuminated, and the camera can receive light reflected from different portions of the object desired to be imaged.

[0034] As is clear from the above, the image frames provided by the camera 130 and its image sensor of, e.g. the first measurement object 120, can correspond to any one of the profile images 141-1-141-N. As mentioned in the background section, each position of the profile of the first object shown in any one of the profile images 141-1-141-N is typically determined based on identifying intensity peaks in the image data captured by the image sensor and finding the positions of these intensity peaks. The system 105 and conventional peak finding algorithms are typically configured to search for intensity peaks of each pixel column in each image frame. If the sensor coordinates are u, v, and e.g. as indicated in the figure, u corresponds to the pixel position in the image sensor along the rows, and v corresponds to the pixel position along the columns, then for each position u of the image frame the peak position along v is searched for, and the peaks identified in the image frame can produce the profile images 141-1-141-N as shown. The profile images are formed by the image points u, v, t in a sensor-based coordinate system 143. The sum of the image frames and the profile images can be used to create a 3D image of the first object 120.

[0035] Fig. 2 schematically illustrates how a point cloud in prior art, i.e. a set of surface points of a 3D virtual object, is generated by means of the 3D imaging system in Fig. 1, to facilitate the understanding of embodiments to be further discussed herein below.

[0036] A wedge-shaped measurement object 220 is used in the example. In addition to the measurement object, the image system 105 can be as shown in Fig. 1. When the image sensor of the camera 130 positioned as shown at the light ray 212 on the measurement object 220 captures an image and identifies an intensity peak, the result can be a point at image sensor coordinates u,v as shown. In the example, the point corresponds to a sample of the surface of the measurement object 220, i.e. the point corresponds to a surface point. Note that here a point is plotted instead of a line connecting the points as in the case of the contour image 141-1..141-N in Fig. 1. Each point is at an intensity peak location. Using the knowledge of the geometry of the system 105 and the settings used, the image sensor coordinates in u,v and the time t at which the image was captured by the sensor (related to the real-world coordinate x,y,z point), or more precisely, how the location of a point in coordinates u,v,t such as in a sensor-based coordinate system 143 can be converted into a corresponding location in a 3D virtual coordinate system x',y',z'. When the complete measurement object 220 has been scanned by the light ray 212, the result can thus be such a collection of points as shown, i.e. a point cloud 260. In the shown example, the collection of points 260 corresponds to samples of the surface of the measurement object 220 and can thus be used for e.g. rendering a corresponding virtual object. The contour of the virtual object is indicated in the figure with a dotted line 261 to facilitate understanding that the collection of points 260 corresponds to the measurement object 220.

[0037] FIG. 3A A first example of an imaging system 305 is schematically illustrated, which can be configured to perform embodiments herein, i.e. for implementing embodiments herein. The imaging system 305 is based on light triangulation for capturing information about 2D and 3D features of one or more measurement objects.

[0038] The shown system corresponds to a basic configuration and comprises a light source 310 for illuminating a measurement object 320 with light 311 (typically laser light) as part of a light triangulation for 3D imaging of the measurement object 220. A camera 330 with an image sensor 331 is arranged for sensing reflected light from the measurement object 320 as part of said light triangulation for 3D imaging.

[0039] The camera 330, the image sensor 331 and the light source 310 are configured and positioned relative to each other for light triangulation. The system 305 can correspond to the system 105 in Fig. 1 for the purpose of light triangulation, but can additionally be configured to perform in accordance with embodiments herein as further described below.

[0040] Thus: the measurement object 320 can correspond to the first measurement object 120 or the measurement object 220 and is shown to be at least partially located within a field of view 332 of the camera 330. The light source 310 is configured to illuminate the measurement object 320 with light 311, typically in the form of structured light, such as a specific light pattern, e.g. a light sheet and / or a light, such as a laser, line, which is reflected by the measurement object 320 and the reflected first light is captured by the camera 330 and the image sensor 331. Another example of structured light that can be used as the first light is a light edge, i.e. an edge of a region or portion that is illuminated. The illumination is in this example in a perpendicular direction, i.e. substantially parallel to the z-axis. However, as will be realized from the following, other illumination direction(s) can of course be used with the embodiments herein, including e.g. a so-called “mirror” configuration, where the light source, e.g. a laser, is directed substantially towards the measurement object with a “mirror” angle of incidence, corresponding to the angle of view of the camera but from the opposite side.

[0041] The measurement object 320 can thus be illuminated and images can be captured as in a conventional light triangulation. Such a light triangulation can thus be as in the prior art and involve movement of the light source 310 and / or the measurement object 320 relative to each other, such that at different consecutive time instances different consecutive portions of the measurement object 320 are illuminated by the light source 310 and the light 211. The light 311 after reflection from the measurement object 320 is sensed by the image sensor 331. In the light triangulation, typically but not necessarily, the camera 330 and the first light source 310 are fixed in relation to each other and the measurement object 320 is moved relative to these. By the sensing by the image sensor 331, the respective image frames are associated with the respective time instances at which the image frames are sensed, i.e. captured, and with the respective portions of the measurement object 320 from which the image sensor 331 senses the reflected first light 311 at the respective time instances.

[0042] The image frames provided by the camera 330 and the image sensor 331 and / or information derived from the image frames (e.g. intensity peak positions) can be transferred to, for example, a computing device 333 (such as a computer or similar device) for further processing outside the camera 330. Such further processing can additionally or alternatively be performed by a separate computing unit or device (not shown), for example, separate from the image sensor 331 but still comprised in the camera 330 (e.g. integrated with the camera 330 or a unit comprising the camera 330). The computing device 333 (e.g. a computer) or the other device (not shown) can be configured to control the devices involved in the light triangulation in order to perform the light triangulation and related actions. Also, the computing device 333 (e.g. a computer) or the other device (not shown) can be configured to perform as described above in connection with Fig. 2, for example, based on the image frames provided by the camera 330 and the image sensor 331 and / or information derived from the image frames. The computing device 333 (e.g. a computer) or the other device (not shown) can thereby provide a point set, i.e. a point cloud, of a 3D virtual object corresponding to the measurement object 320. In some embodiments, the computing device 333 (e.g. a computer) or the other device (not shown) is further configured to perform actions related to the embodiments herein as further described below.

[0043] It is considered a reasonable and practical assumption that a measurement object (e.g. the measurement object 320) has a continuous surface, no surface parts or points “floating in the air”, no points not connected to at least some other parts and points of the object by a surface, i.e. that all surface points are connected to one or more neighboring surface points. This is a correct assumption and corresponds in principle to the actual situation of all points. If there are special cases, e.g. using the imaging system and / or the measurement object in a situation where this is not fully or partially the case due to special settings and / or the object, this can typically be identified in advance and treated as a special case. Thus, this assumption has no negative practical consequences or its negative practical consequences are negligible. This shall equally apply to surface points of a point cloud of a 3D imaging based on such a measurement object. I.e. all surface points of the point cloud are connected to the nearest neighborhood surface points by a coherent surface and there is also a coherent surface between the surface points. This is also typically a normal assumption when a surface subdivision is used to visualize a 3D virtual object based on a point cloud and corresponds to all surface points being connected to their nearest neighborhood surface points by a straight line(s) assuming that the lines and the surface between the lines correspond to a coherent and opaque surface of the actual measurement object, i.e. no holes or transparent areas in the surface between the surface points that the camera can see through, except for such surface points themselves.

[0044] FIG. 3BThe diagram schematically illustrates the measured object 320 in a 3D view, shown in coordinates x, y, z in the real-world coordinate system 323. The camera 330 is represented here by its camera optical center C 330b and a virtual frustum 330a corresponding to the field of view. Therefore, what the camera 330 and image sensor 331 image is the light rays incident on the virtual frustum 330a. Only three such light rays 332 are shown as examples, corresponding to rays emitted from directly above the top and farthest edge of the measured object 320 relative to the camera 330. Moreover, two dummy objects 321a-b are shown as examples only. It can be understood that, with the camera positioned as in the example, the physical object corresponding to the dummy object 321a cannot be imaged by the camera 330 because it is occluded or "masked" by the measured object 320. This corresponds to the dummy object 321a being located below the light ray 332. On the other hand, the dummy object 321b can be imaged because, from the perspective of the camera 330, it is not obstructed by the measured object 320 and is positioned such that light from the dummy object 321b can reach the camera 330 and be imaged by the camera 330.

[0045] Surface points of a point cloud (such as point cloud 260) can be evaluated relative to a camera (e.g., camera 130) or from the viewpoint and orientation used when capturing the image on which these surface points are based. When capturing the image, based on the reasonable assumptions described above, there may be occlusions from other parts of the measured object (e.g., measured object 220 or 320). The same principle can be applied to the evaluation of surface points in a point cloud. However, in that case, the corresponding camera orientation should be in the 3D virtual coordinate system x', y', z' of the point cloud, for example, in the case of point cloud 260, in the 3D virtual coordinate system 253. That is, the camera orientation when imaging points of a real-world object surface (e.g., of measured object 220 or 320) in the real-world coordinate system x, y, z (e.g., real-world coordinate system 223 or 323) can be converted into the corresponding (i.e., virtual) camera orientation in the 3D virtual coordinate system x', y', z' (e.g., 3D virtual coordinate system 253). Alternatively, based on the known relationships between the coordinate systems involved (e.g., given by the settings and relationships used for the 3D imaging system and setup, such as those used for optical triangulation when using that 3D imaging system), the corresponding camera orientation of the points used in the point cloud can be directly determined in the 3D virtual coordinate system (i.e., in x', y', z').

[0046] One way to utilize the foregoing is to, for each surface point of the point cloud (e.g., of the point cloud 260), identify one or more colliding points of the point cloud. That is, for each point, identify and / or investigate whether there is any other one or more points that cannot exist simultaneously with the investigated point, e.g., due to at least one of them being occluded by the other. If one of the points “blocks” the other or each other, then the two surface points can be considered colliding, and thus it is impossible for the camera to have seen them simultaneously. In other words, if that is the case, then one or at least one of the colliding points is necessarily false or incorrect. In that case, the two points can be removed from the point cloud to ensure that no false points are used. However, it can be recognized that many correct points will also be removed, which is undesirable and even critical in some cases. With the further assumption that can be reasonable in some scenarios, namely that in cases of collision it is always the point closest to the camera that is correct, some improvement can be achieved such that fewer correct points are removed or at least removed. However, there are cases where this approach also results in correct points being removed, namely when the point closest to the camera is not the correct point. This can happen, for example, if the false point is due to noise or reflections, which can cause the false point to be closer and / or further away from the camera relative to the correct point. In that case, always removing the point closest to the camera can even have a worse effect, as it removes the correct point and keeps the false point.

[0047] Embodiments herein provide improvements with respect to the foregoing. Briefly, they can be described as being based on statistical methods and / or consensus voting. Rather than just considering points that collide with each other and then removing all or some of the colliding points, all or a large number of points of the point cloud can be investigated for collisions with other points of the group, and then each point can be attributed a score each time it is identified as being involved in a collision with another point. That is, when a point of the point cloud is involved in a collision, its score is increased, and this is done for all points of the point cloud. Thus, the total score of each point will then indicate how many collisions in total the point is involved in, and even be proportional to it. It is thus possible to compare and evaluate points by their total score, i.e., the total number of collisions they are involved in, and not just whether they are involved in a collision or not. Then, all points involved in collisions above a certain score corresponding to a threshold can be removed. In other words, with this approach, points involved in the most collisions are removed, rather than any removal decisions being made based on the identification of collisions alone. Points that are truly incorrect, i.e., those that are not in positions corresponding to imaged actual surface positions on a real-world measured object, such as the measured object 220 or 320, will typically be involved in more collisions than other points, at least on average, and will thus be removed to a greater extent by the method according to embodiments herein.

[0048] The threshold can be predetermined in a flexible manner. For example, a slider to change the threshold can be implemented, and the result can be visualized directly in a virtual 3D object drawn from the remaining surface points of the point cloud. This allows users to find a suitable threshold level for a given setting that removes points that would otherwise cause artifacts when drawing a 3D object from the remaining surface points of the point cloud. Typical artifacts are spikes that appear in a 3D virtual object when drawing it. At the same time, users can ensure that the threshold is not used too low, i.e., a threshold that causes the removal of the actual object details that are expected to be preserved.

[0049] FIG. 3C Is it like this? FIG. 3B The side view of the measuring object 320 shown in the figure. This figure also shows... FIG. 3B An exemplary surface 325 is excluded from the diagram, which supports the measurement object 320 and on which the measurement object 320 resides. Surface 325 may be, for example, part of a conveyor belt or the like, as may be used during optical triangulation as mentioned above, or it may be a fixed surface, for example, part of a table or the like. In the latter case and in optical triangulation, the camera and light source can be moved to complete the scanning of the measurement object 320. Note that surface 325 does not need to be imaged and may be, for example, outside the field of view or excluded from the point cloud. In some embodiments, there may be a surface or other structure not located below the measurement object 320 but, for example, located to the side or above the measurement object 320, and the measurement object is attached to this surface or other structure, for example, to ensure that the measurement object remains in a specific position, orientation, and / or moves in a desired manner during 3D imaging.

[0050] FIG. 3D Is with FIG. 3C A virtual view that corresponds to the real-world view. FIG. 3D The diagram schematically illustrates some surface points P1-P15 (only some surface points are explicitly labeled) of the point cloud in a 3D virtual coordinate system 353 (in x', y', z') corresponding to a virtual version of the real-world coordinate system 323 (in x, y, z). Therefore, the point cloud here includes surface points of the virtual object corresponding to the measured object 320. The appearance of the measured object and its side view when perfectly modeled is determined by... FIG. 3D The fine dotted outline 361 in the figure is indicated for reference only.

[0051] For example, FIG. 3C-3DIn comparison with the situation illustrated in Fig. 2 as explained above, regarding how the point cloud 260 in the 3D virtual coordinate system 253, i.e. in x', y', z', can be formed by 3D imaging, in the example by phototriangulation and line scanning of the measured object 320 in the real world coordinate system 323, i.e. in x, y, z. Thus, FIG. 3D The surface points P1-P15 in Fig. 3A can be points of a point cloud resulting from 3D imaging of the measured object 320, e.g. from phototriangulation and line scanning using the imaging system 305.

[0052] The surface points P1-P15 are shown connected to each other to illustrate the above mentioned situation, assuming that the surface points are connected to their nearest neighbor surface points. This is visualized in the figure to facilitate the understanding of the collision points described next. The connected points can generally be those that would be connected if a surface subdivision or similar would be applied as mentioned above in order to generate a 3D virtual object from the points.

[0053] FIG. 3D and further described below FIG. 3E - schematically illustrate and will be used in the following to explain and exemplify the principles behind the embodiments herein, such as regarding identifying which surface points collide with each other, and which do not. Thus, these figures are used to exemplify the situation with the settings as FIG. 3A

[0054] FIG. 3D to illustrate the example when the point under investigation, i.e. the point for which a collision is identified, is point P4, i.e. in the example Pi = P4. FIG. 3E for the corresponding situation, but with Pi = P12, and FIG. 3F when Pi = P13. As mentioned above, the checking of collision of a point is regarding identifying which other points, if any, collide with said point, or in other words, which other points, if any, cannot exist simultaneously valid with the point under investigation, i.e. the point for which a collision is identified.

[0055] In FIG. 3D In Fig. 3A, the point C'(i) corresponds to the position of the camera for the point Pi in the 3D virtual coordinate system 353, i.e. in the same coordinate system as the points of the point cloud, here P1-P15. The position of C' can be referred to as a virtual camera position. The position of the camera can suitably correspond to and / or be represented by the camera center C. Given the type of camera and the settings used, the skilled person is able to determine the camera center position, or whether another representation of the camera position is more suitable, e.g. as discussed further below.

[0056] ​It is noted that depending on the 3D imaging system and settings used for the 3D imaging of the measurement object 320, i.e. the imaging of the images on which the surface points of the point cloud are based, the camera position C' in the 3D virtual coordinate system x', y', z' can be different for different surface points. In case of light triangulation with line scanning, the camera position C' is the same with respect to the light rays illuminating the measurement object, and thus, for example, with respect to each profile image, e.g. with respect to each profile image 141-1...141-N. However, since the time dimension, i.e. the time at which the respective image frame is captured, is used to provide information about the third dimension, more specifically about y' in the example shown herein, it is recognized that the camera position C' is different for different positions in y'. In the example, i denotes the position along y', which means that the camera position C' depends on i, i.e. C'(i). In other words, for the same y' and i, the position C' of the camera in x', y', z' is the same. However, if the z' of the point P changes, e.g. P13 in comparison to P3-P12, C'(i) is still the same position as if P13 would be located at the same z' as P3-P12, which means that the camera direction would be different for P13. In the example, this can be seen in FIG. 3F . The camera direction, i.e. the direction in x', y', z' from the surface point Pi to the camera position C' for that surface point Pi, can in the example be represented by a virtual light ray 352d, which depends on C' and Pi, and thus, can be denoted as d(C', Pi). Since d and the camera direction are in the 3D virtual coordinate system, here 3D virtual coordinate system 353, i.e. in x', y', z', 353, such a camera direction can be referred to as virtual camera direction.

[0057] It is noted that C' depending on the position of P does not need to be like this for all relevant 3D imaging systems, i.e. 3D imaging systems that can be used to provide image data on which the point cloud can be based and then improved by the embodiments herein. Also, other dependencies than in the example herein are of course possible.

[0058] The embodiments herein can in general be applied to surface points of a point cloud, independent of the way the images, i.e. the images on which the point cloud is based, are provided, but there should be a predefined mapping from the actual measurement object via the camera and the image sensor to the points of the point cloud and thus also to the virtual 3D object based on the point cloud.

[0059] In some cases, the camera position C’ can be or be approximated to be the same for all surface points P of the point cloud or for the part of the point cloud to which the embodiments herein are applied. However, in general, in order to cover all possible cases, the camera position C’ for a surface point P can be considered to depend on the position of the surface point P in the point cloud, i.e. C’(x’, y’, z’), but the camera position C’ is expected to be the same at least for surface points of the point cloud based on one and the same image frame captured by the camera. When the embodiments herein are applied to surface points of a point cloud, the camera position C’ in the 3D virtual coordinate system can be considered to be predefined, or even predetermined. The camera position C’ can be considered to be given or predetermined by the 3D imaging system and the settings used at the time of capturing the image(s) from which the surface points of the point cloud originate.

[0060] It is noted that, FIG. 3D - only one side view along y’ is shown for a certain x’, i.e. a “slice” or plane, namely the z’-y’ plane, but C’ does not need to be located in the same plane, of course, although for simplicity this can be assumed in the examples below, which are about the general principles behind illustrating how to calculate the score for each surface point P. In practice, the virtual light ray d from a point P to the camera position C’ is typically a line in 3D space, i.e. in all dimensions x’, y’, z’, not only in y’, z’.

[0061] Similar considerations apply, of course, to points in the set which are not only located in a single z’-y’ plane as the points P1-P15 in the example, since these points belong to a 3D virtual object and thus comprise points at positions which differ in all three coordinates of the 3D virtual coordinate system 353, e.g. all x’, y’, z’.

[0062] Even when the embodiments herein operate on each z’-y’ plane of several different such planes, e.g. approximating that the camera position C’ is located at the position in which the actual camera position is projected in that plane, for the removal of surface points, improvements based on the embodiments herein have been observed compared to retaining all surface points of the point cloud. However, using the actual or more accurate camera position C’ enables further improvements.

[0063] As already indicated above, other representations of the camera position C’ are possible in addition to the camera center. For example, if an orthogonal camera is used, e.g. the camera 330 is orthogonal, such as a camera with a telecentric lens, or if the camera used approximates an orthogonal camera, i.e. a camera with parallel light rays instead of light rays from the camera center, FIG. 3DThe result in the -F example would be that all camera orientations are the same, i.e., parallel virtual rays from each point P, independent of x' and z'. Therefore, in that case, the virtual ray from P13 has the same angle as those from P12 and P4. Even when a true orthogonal camera is not used, but the camera orientation is approximated as if this were the case, improvements have been observed in the applications of the embodiments described herein.

[0064] If the camera and camera position C' are placed further apart, then the effect of the camera orientation in the virtual coordinate system x, y, z (i.e., the effect of the virtual ray d from the surface point P toward the camera) becomes similar to that of an orthogonal camera, that is, the camera orientation for all surface points P approaches the same direction.

[0065] In practice, considering approximations regarding camera position C' and / or camera orientation can provide information about whether the approximation is sufficient, for example, to facilitate and / or simplify practical implementation, routine testing, and experiments (e.g., involving simulations), taking into account the requirements to be met. For example, those skilled in the art can apply the embodiments described herein in tests and experiments using both the actual camera position C' and the desired approximation, and then compare the results, for example, comparing a 3D virtual object based on surface points when using the actual camera position C' with another 3D virtual object based on surface points when instead using an approximate camera position C'.

[0066] The following example will refer to FIG. 3D -F explains the general principles behind how to calculate a score for each surface point Pi.

[0067] exist FIG. 3D In -F, it can be seen that surface points P1-P15, except for P13, follow contour line 361 very well, at least as well as one could expect using the surface point density in the example. Point P13 should therefore be understood as an example of an incorrect surface point, i.e., it does not correspond to an actual surface point on the real-world measured object 320.

[0068] As mentioned above, FIG. 3DThe example of Fig. 6 is particularly useful to explain the identification or investigation of surface points that conflict with surface point P4. This figure shows two "shadows" or occluded areas A and B for point Pi = P4. It should be recognized that with the camera position C' shown in the figure and the above-mentioned assumptions, the part of measured object 320 that is not "hanging in the air" and thus does not have such surface points, it can be concluded that if there are any other surface points P in area A, then P4 will be occluded, i.e. hidden from view by the camera. Thus, any point in area A is a conflict point with P4, i.e. cannot effectively coexist with P4, but as can be seen, there are none in this example. The case of a point with "shadowing", i.e. being occluded, can be compared to the similar situation in the "real world" discussed above with respect to Fig. 5, where example dummy object 321a is "shadowed" by measured object 320 and not visible through the camera, but example dummy object 321b is not. FIG. 3C

[0069] In area B of Fig. 6, the corresponding situation for area A occurs. P4 will occlude any point in area B, and thus each point in area B will also be a conflict point with P4. As can be seen from the example, only Pi 3 is in area B, and thus is a conflict point with P4. Thus, P4 and Pi 3 can temporarily get a respective score of 1, or any existing score of these points can be increased by 1. For example, if P4 and Pi 3 start with 0 and this is the first conflict identified, then they can both have a score of 1 as a result of the investigation of conflict points with P4. Based on this principle, it can be recognized that Pi - P3 are not involved in a conflict in the example shown. FIG. 3D

[0070] It is noted that areas A and B can also be influenced by the direction of the lighting used, as recognized by the person skilled in the art, and are shown in the example herein with lighting in Fig. 6. FIG. 3A FIG. 3D ​​​The diagram illustrates a virtual illumination direction 354, and it is understood that, similar to a camera, a virtual light source position can exist that depends on the location in x', y', z'. However, in the example shown, the virtual illumination direction from the corresponding point P is always vertical, i.e., along z'. Regions A and B, if such or corresponding regions are applied in other cases, can therefore vary depending on the imaging system and setup already used in those cases. Generally, for point P, its region A is the region in the direction toward the camera, and its region B is the region away from the camera. These two regions make it such that if any other point in the set is located in that region, then those points are conflicting points, i.e., points that cannot coexist effectively with point P. The location of points that cannot coexist effectively depends on the imaging system and setup used for 3D imaging. This region, and what corresponding regions A and / or B are applied, can be determined by those skilled in the art based on the imaging system and setup used in the particular case.

[0071] FIG. 3E It shows the relationship with FIG. 3D The corresponding example is shown, but regions A and B are represented as Pi = P12 (i.e., i = 12). Furthermore, here, only P13 is a conflict point in B against P12. In other words, the scores for P12 and P13 can be increased by 1 because they both involve a conflict with another point. Additionally, it can be recognized that if a corresponding investigation were conducted on all points from P1 to P12, then after identifying the conflict points and increasing their scores, up to and including P12, the result would be:

[0072] Surface point Score Each of P1-P3 0 Each of P4-P12 1 P13 9

[0073] FIG. 3F Another example case is shown, where regions A and B are relative to Pi = 13, i.e., i = 13. Here, points P9-P12 are in region A, and therefore are conflict points of P13. It can also be recognized from the figure that P14-P15 do not involve any conflict with each other or with any other points P1-P13 in the example. Therefore, in the example shown, the score obtained after investigating and identifying all conflict points of the shown points P1-P15 is:

[0074]

[0075]

[0076] In other words, using a threshold in the range of 3-13 and removing all points with scores equal to or greater than this threshold will result in the removal of point P13 from the point cloud P1-P15. Therefore, the incorrect point 13 will be removed. It is also recognized that the new resulting point cloud without P13 will better fit contour line 361, and thus better represent real-world objects compared to P13 remaining part of the point cloud.

[0077] In an example, it is typically not necessary to look for collision points in both regions A and B, i.e. towards and away from the camera. For implementation reasons, it can be preferred to look in only one direction, e.g. towards or away from the camera position C, i.e. for the respective point P in either region A or B. If only region B is checked, the result in the shown example will be:

[0078] Surface point Score Each of P1-P3 0 Each of P4-P8 1 Each of P9-P12 1 P13 9 Each of P14-15 0

[0079] It is recognized that in this case P13 also relates to most of the collisions. Thus, if collision points are looked for in either region A or B for the respective surface point, the principle works. However, if both regions are used, often the points that are not correct will get a higher score, and thus in some cases more points that are not correct can be removed by a threshold.

[0080] For each surface point, it can not be necessary to use region A and / or region B that extends to the end of the point cloud. Instead, it can be sufficient to only check a certain distance along the camera direction, i.e. along a virtual ray d for each surface point, towards and / or away from the camera. For example, with reference to the example shown in FIG. 3D - F, it is assumed that only the closest 5 consecutive y’ positions in region B are checked. I.e. for P4 with i = 4, it is checked if points P5-P9 exist with a collision, i.e. region A is used that extends along y’ from P4 to P9. Applying this to all points P1-P15, otherwise the same principles as above are applied, the result is:

[0081] Surface point Score Each of P1-P7 0 Each of P8-P12 1 P13 5 Each of P14-15 0

[0082] Thus, in this case and example, it is also possible to identify P13 as a point to be removed, e.g. by using a suitable threshold.

[0083] FIG. 4 is a flow chart for an embodiment for schematically illustrating a method according to embodiments herein. The following actions can constitute the method for removing erroneous or possibly erroneous points from a set of points, e.g. the set exemplified by points P355-1...355-15, i.e. from a point cloud, which correspond to outliers as mentioned in the background art. In practice, the point cloud for which embodiments herein are applied is of course much larger than the mere 15 points in the simplified example above. The number of points of the set can be in the order of several million points, just to mention some examples. The points are points of a 3D virtual object, which are provided by 3D imaging of a corresponding real world object, e.g. the measurement object 320, by means of a camera of an image sensor, e.g. the camera 330 and the image sensor 331.

[0084] In addition, the points in the set are typically surface points of the 3D virtual object. It is typically an imaged surface of a real-world object, but for example, for partially transparent objects, it can be through the image of the actual outermost surface, thus the virtual object and the set of points can in some cases correspond to an inner surface of the real-world object.

[0085] In some embodiments, the 3D imaging is based on light triangulation, including illumination of the real-world object (e.g., the measurement object 320) by a light source (e.g., the light source 310), wherein the light emission is from a reflection of a surface of the real-world object resulting from the illumination.

[0086] The methods and / or actions indicated below and in FIG. 4 The methods and / or actions indicated below and in

[0087] Note that the following actions can be performed in any suitable order and / or executed in time completely or partially overlapping, where possible and suitable.

[0088] Action 401

[0089] The set of points is obtained. For example, it can be obtained from measurements and calculations of the 3D imaging system, all or in part by one or more same devices involved in performing the present method, and / or by receiving from another device or unit that can be part of the 3D imaging system, but can alternatively be received from another (e.g., external) device or unit of a stored set of points. In addition, the set of points can correspond to a point cloud and can be points corresponding to (i.e., covering) a complete real-world object or a part, i.e., the points can be points of a 3D virtual object corresponding to a partial 3D view of the real-world object only.

[0090] Action 402

[0091] For a respective point, e.g., for a respective point of the points 355-1...355-15, a respective camera direction can be obtained, e.g., corresponding to the camera direction 352. The respective camera direction corresponds to a direction of light emission from the corresponding point of the real-world object towards the camera that was sensed by the image sensor during the 3D imaging. The above virtual light ray 352 is an example of such a camera direction. As mentioned above, the obtained camera directions can be referred to as virtual camera directions, as they are directions of the points in the set and thus related to coordinates of a 3D virtual coordinate system for the set. A conflicting point can be a point in the set that cannot effectively coexist with a respective point based on at least its camera direction.

[0092] The camera direction can be obtained as indicated above by determining based on knowledge of the 3D imaging system and settings used when the image the sensing point is based on, such as calculating and / or estimating. The set of points is typically associated with a predefined mapping of how positions in a real world coordinate system, e.g. x, y, z, in which the 3D imaging system and real world object, e.g. imaging system 305 and measured object 305, are located, relate to coordinates of a corresponding 3D virtual coordinate system, x', y', z', in which the virtual object and said set of points are located. The mapping is typically provided by a calibration performed for each 3D imaging system and settings used for 3D imaging, thus the mapping is typically determined in advance when performing the embodiments herein, and at least the mapping can be considered predefined or given by the 3D imaging system and settings used when sensing said image. Thus, by knowing the position of the camera in x, y, z, the corresponding position in x', y', z' can be determined, and from this also the camera direction for the point can be determined.

[0093] The camera direction can have been determined in advance for the imaging system and settings used, e.g. for various positions in x', y', z', and then the pre-determined camera direction can be used to determine the camera position for the point in said set of points.

[0094] In some embodiments, the pre-determined camera direction, or even the camera direction for said set of points, can be obtained by receiving it completely or partially from another device or unit, e.g. a device or unit having determined, such as calculated, the camera direction.

[0095] As indicated above, determining whether a point is a collision point, i.e. whether it can effectively co-exist with the corresponding point, can also take into account a virtual light direction for the point, i.e. a virtual direction of incident light corresponding to the direction of incidence of light from the light source 310 on the real world object, and which would be the light reflected by the real world object and sensed by the camera and image sensor, such as in the case of light triangulation. The exemplary virtual illumination direction 354 discussed above corresponds to such a virtual light direction.

[0096] Thus, in some embodiments, the present action additionally comprises obtaining for the corresponding point a respective virtual light direction corresponding to the direction of incidence of light from the light source on the corresponding point on the real world object. In these embodiments, a collision point can be a point in the set which cannot effectively co-exist with the corresponding point further based on its virtual light direction.

[0097] Action 403

[0098] For a respective point, e.g., for a respective one of the points 355-1...355-15, conflicting points (if any) in the set are identified. A conflicting point is a point in the set that cannot coexist with the respective point effectively according to one or more predefined criteria. One such criterion can be based on the camera direction obtained for the point, as mentioned above under action 402. It is recognized that in principle the one or more criteria can also be based on anything that can be used to determine what can be seen and imaged at the same time and what cannot for the 3D imaging system and setup used. For example, when it comes to light directions, e.g., in the case of light triangulation and as exemplified above, the one or more criteria can also be based on virtual light directions, e.g., to determine regions A and / or B or similar regions as in the example above. If light cannot reach a certain point, this can be another reason why the point cannot be seen by the camera, and thus virtual light directions can be used together with virtual camera directions to identify points that cannot coexist with another point effectively, e.g., to implement an improved predefined standard to apply. However, since the criterion based on camera direction and the assumption that lighting has reached all points can be sufficient due to the implication that some light must have reached the camera by passing through the correctly imaged real-world object.

[0099] In some embodiments, a conflicting point is a point that cannot coexist with the respective point effectively based on the assumption that it (or at least some of its nearest neighbor surface points) is connected by a coherent surface to a point in the set (e.g., the points 355-1...355-15) and that the coherent surface would block light (e.g., such light emission mentioned under action 402). The point to which the respective point is connected by the coherent surface can correspond to the point to which the respective point would be connected in order to generate a 3D virtual object from the points, e.g., by a surface subdivision and / or according to any existing technique for generating a 3D virtual object from a point cloud.

[0100] In addition, in some embodiments, the identification of conflicting points for a respective point is limited to points in the set that exist within a certain distance from the respective point. This has been exemplified and discussed in some detail above in connection with FIG. 3D -F, and can be of particular interest for applications to complex real-world objects and / or extensive point sets where it is not desirable and / or practical to search the entire point cloud to identify conflicting points for a respective point. For example, it can be sufficient to identify conflicting points (if any) that are within a distance that relates to the location of the nearest neighbor points (e.g., pixel or more precisely voxel locations, since it is 3D) that have potential conflicting points. The certain distance can be predefined or predetermined and can be found for a particular application based on routine experimentation and testing to find a suitable and / or acceptable distance for the application in question.

[0101] Limiting the identification to within a certain distance can reduce processing and make possible implementations that can be performed faster.

[0102] In addition, conflict points located far away can be less likely to occur, and it can be inefficient to spend resources on identifying very far conflict points, if any. Also, if there are far conflict points, then when a conflict is identified for a point closer to the point, it can be identified as a point with many conflicts anyway.

[0103] Similar effects to limiting to a certain distance can be achieved by, for example, splitting larger sets of points, or applying the embodiments herein to partial sets of the complete set of virtual objects, for example, partial sets output during line scanning and light triangulation.

[0104] The number of conflicts identified for respective points, for example, for each point of a point cloud to which the embodiments herein are applied, can be kept count of, for example, by increasing the score for each point as mentioned above.

[0105] Information such as counts of identified conflicts for respective points can be stored. When conflict points for points of a set, for example, for each point of a set, have been identified, the points have been involved in different numbers of conflicts and a value and / or identifier identifying or indicating the total number of identified conflicts that a point has been involved in can be stored for each point.

[0106] Action 404

[0107] Based on said identifying of respective points, i.e., based on action 403, one or more points of the set that are involved in conflicts more times than other points of the set are removed from the set. Respective points of the set, for example, points 355 are any of points 355-1... 355-15, are considered to be involved in conflicts each time, i.e., when, a conflict point for the respective point is identified in the set and / or each time the respective point itself is identified as a conflict point for another point of the set. The above in connection with FIG. 3D - Some detailed examples and explanations of this are given above.

[0108] In some embodiments, points that are involved in conflicts more times than a predefined threshold are removed from the set. The predefined threshold, typically a threshold of values or numbers, can be as discussed above and can in some embodiments be predetermined.

[0109] In some embodiments, the one or more predefined criteria are based on what the camera (e.g., camera 330) can virtually (i.e., theoretically) see from its corresponding position in the coordinate system of the set of points (i.e., coordinate system of the locations of the points of the set, e.g., virtual coordinate system 353). The corresponding position for the camera can be found from a predefined or predetermined mapping between the real-world coordinate system (e.g., x, y, z) of the measured object and the virtual coordinate system (e.g., x', y', z') of the virtual object and set of points, as mentioned above. The camera position can be represented by the camera center, or by other representations as mentioned above.

[0110] Thus, what the camera can virtually (i.e., theoretically) see can be based on the set of points and the position of the camera in the same coordinate system as the set of points (e.g., in x', y', z'). Note that this can be different from what the camera actually sees in the real world. One advantage of utilizing what the camera can or could theoretically (i.e., virtually) see is that all calculations can be done in x', y', z' using information and data that is typically already available (including the set of points), without the need for any additional and often complex real-world measurements and further mapping to x', y', z' (if it is necessary to determine what the camera actually can see).

[0111] As already mentioned above, embodiments herein provide an improvement compared to deleting or removing each time a conflicting point is identified, which in many cases results in also removing correct points in case of a conflict. With embodiments herein, more correct points can be kept while still removing truly incorrect points, i.e., points that are not in a position corresponding to a real position of imaging on a real-world object (e.g., measured object 220 or 320).

[0112] As mentioned above, embodiments herein can be compared to the solution in the prior art mentioned in the background, where a part of the contour is decided to be deleted (i.e., removed) based on the existence of a blind spot region of the contour (i.e., based on the fact that a conflict exists), thus removing for all identified conflicts. In other words, in the prior art, the removal is based on identifying a conflict, and not on how many conflicts a point is involved in.

[0113] In addition, embodiments herein can be efficient for removing false points caused by noise or reflections. In case of 3D imaging based on light triangulation, reflections can be reduced or removed as part of the intensity peak finding, but for embodiments herein, false points caused by reflections can additionally or alternatively be removed from the point cloud.

[0114] FIG. 5A- B shows a virtual 3D object image of a measurement object without applying and with applying the embodiments herein, respectively.

[0115] FIG. 5A An image of a 3D virtual object is shown, which is based on an obtained set of points (i.e. a point cloud) to which the embodiments herein are to be applied. The point cloud is provided as a matter of course, here from a light triangulation of the prior art. The point cloud can correspond to the point cloud obtained in action 401 and to which the embodiments herein are to be applied. As can be seen, the point cloud comprises outliers, i.e. incorrect points, which cause distortions, visible as "spikes" on the object shown in the image.

[0116] FIG. 5B Another image of a 3D virtual object is shown, which is based on the same obtained set of points but after applying the embodiments herein (i.e. after removing points that are involved in conflicts exceeding a certain threshold). In the shown example, the embodiments herein are applied to a set of approximately 30000 points, and for the respective points, conflicting points (if any) within a distance of 25 pixels are identified. The score for the respective point is 0 at the beginning, and each identified conflict causes the score to increase by 1. A total score threshold of 3 is used, and all points with a total score higher than this threshold are removed from the set. The result is that 1804 points are removed. The improvement is clearly visible, where the "spikes" are no longer present on the shown image.

[0117] FIG. 6 is a schematic block diagram of an embodiment of one or more devices 600 (i.e. the device(s) 600) that can correspond to the device(s) already mentioned above for performing the embodiments herein, such as for performing the above described methods and / or actions with respect to FIG. 4 The device(s) 600 are thus for removing erroneous points from a set of points of a 3D virtual object provided by means of 3D imaging of a corresponding real-world object with a camera pair having an image sensor, e.g. as provided by means of 3D imaging of the measurement object 320 with the camera 330 having the image sensor 331. The device(s) 600 can for example correspond to or be comprised in and / or form a suitable device of a 3D imaging system comprising a camera and an image sensor, e.g. the imaging system 305, or can be a device (e.g. a computer) external to the 3D imaging system providing the set of points.

[0118] The schematic block diagram is for illustrating how the device(s) 600 can be configured to perform embodiments of the above discussed methods and actions with respect to FIG. 4

[0119] ​The device(s) 600 can comprise a processing module 601, such as a processing component, one or more hardware modules, including for example one or more processing circuits, circuitry, such as a processor, and / or one or more software modules for performing the described methods and / or actions.

[0120] The device(s) 600 can further comprise a memory 602, which can comprise, such as contain or store, a computer program 603. The computer program 603 comprises "instructions" or "code" executable by the device(s) 600, directly or indirectly, to perform the described methods and / or actions. The memory 602 can comprise one or more memory units and can further be arranged to store data, such as configurations, data and / or values, which are relevant or used for performing the functions and actions of the embodiments herein.

[0121] Also, the device(s) 600 can comprise processing circuitry 604 involved in processing and for example encoding data, as an example hardware module(s), and can comprise one or more processors or processing circuits or correspond thereto. The processing module(s) 601 can comprise for example be "implemented in the form of" or "be realized by" the processing circuitry 604. In these embodiments, the memory 602 can comprise the computer program 603 executable by the processing circuitry 604, whereby the device(s) 600 is operable or configured to perform the described methods and / or actions thereof.

[0122] Generally, the device(s) 600, e.g. the processing module(s) 601, comprise input / output (I / O) module(s) 605 configured to be involved in, e.g. by performing, communication with other units and / or devices, such as sending and / or receiving information to and / or from other devices. The I / O module(s) 605 can be exemplified by obtaining, e.g. receiving, and / or providing, e.g. sending, the module(s), as applicable.

[0123] Further, in some embodiments, the device(s) 600, e.g. the processing module(s) 601, comprise one or more of obtaining module(s), identifying module(s), removing module(s), as example hardware and / or software module(s) for performing the actions of the embodiments herein. These modules can be realized fully or partly by the processing circuitry 604.

[0124] Thus:

[0125] The device(s) 600 and / or processing module(s) 601 and / or processing circuitry 604 and / or I / O module(s) 605 and / or obtaining module(s) can be operable or configured to obtain the set of points.

[0126] The device(s) 600 and / or processing module(s) 601 and / or processing circuitry 604 and / or I / O module(s) 605 and / or obtaining module(s) can be operable or configured to obtain the respective camera direction for the respective point.

[0127] The device(s) 600 and / or processing module(s) 601 and / or processing circuitry 604 and / or identifying module(s) can be operable or configured to identify, for the respective point, a conflicting point.

[0128] The device(s) 600 and / or processing module(s) 601 and / or processing circuitry 604 and / or removing module(s) can be operable or configured to remove, from the set, one or more points in the set that involve more conflicts than other points in the set based on the identification of the respective points in the set.

[0129] FIG. 7 FIG. 7 is a schematic diagram illustrating some embodiments related to a computer program and carrier thereof, to make the above-mentioned device(s) 600 perform the described methods and actions.

[0130] The computer program can be a computer program 603 and comprise instructions that, when executed by the processing circuitry 604 and / or processing module(s) 601, cause the device(s) 600 to carry out any of the steps discussed above. In some embodiments, a carrier, or more specifically a data carrier, e.g. a computer program product, comprising the computer program is provided. The carrier can be one of an electronic signal, optical signal, radio signal, and computer readable storage medium, e.g. a computer readable storage medium 701 as schematically depicted in the figure. The computer program 603 can thus be stored on the computer readable storage medium 701. The carrier can not be a transitory, propagating signal and the data carrier can correspondingly be named a non-transitory data carrier. Non-limiting examples of data carriers that are computer readable storage medium are a memory card or stick, a disk storage medium such as a CD or DVD, or a mass storage device typically based on a hard drive or solid state drive(s) (SSD). The computer readable storage medium 701 can be used to store data accessible via a computer network 702, e.g. the Internet or a local area network (LAN). The computer program 603 can also be provided as (one or more) pure computer programs or contained in one or more files. The file(s) can be stored on the computer readable storage medium 701 and obtained, e.g. downloaded, e.g. via a server, e.g. via a computer network 702 as depicted in the figure. The server can for example be a web or file transfer protocol (FTP) server. The file(s) can for example be executable files for direct or indirect download to and execution on the device(s) in question to make them perform as described above, e.g. by being executed by the processing circuitry 604. The file(s) can also or instead be used for intermediate download and compilation involving the same or another processor(s) to make them executable before further download and execution, so that the device(s) 600 perform as described above.

[0131] Note that any of the processing module(s) and circuitry mentioned in the foregoing can be implemented as software and / or hardware modules, e.g. in existing hardware and / or as application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), etc. Note also that any of the hardware modules and / or circuitry mentioned in the foregoing can for example be comprised in a single ASIC or FPGA, or be distributed across several separate hardware components, whether individually packaged or assembled into a system-on-chip (SoC).

[0132] Those skilled in the art will also appreciate that modules and circuitry discussed herein can refer to a combination of hardware and / or software in one or more processors, e.g., stored in memory, that, when executed by one or more processors, can cause a device, sensor(s), etc. to be configured to and / or perform the methods and actions described above.

[0133] Identifications made herein by any identifier can be implicit or explicit. The identification can be unique in a particular context, e.g., for a particular computer program or program provider.

[0134] As used herein, the term “memory” can refer to a data storage for storing digital information, typically a hard disk, magnetic storage, media, portable computer floppy or diskette, flash memory, random access memory (RAM), etc. Further, the memory can be an internal register memory of the processor.

[0135] It is also noted that any recited term (such as first device, second device, first surface, second surface, etc.) should be considered non-limiting and such terms do not imply a certain hierarchical relationship. Rather, recited naming should be considered merely a way of distinguishing different names without any explicit information.

[0136] As used herein, the expression “configured to” can mean that the processing circuitry is configured to or adapted to perform one or more of the actions described herein by means of software or hardware configuration.

[0137] As used herein, the term “numeric value” or “value” can refer to any kind of number, such as binary, real, imaginary, or rational number, etc. Further, the “numeric value” or “value” can be one or more characters, such as a letter or a string of letters. In addition, the “numeric value” or “value” can be represented by a bit string.

[0138] As used herein, the expressions “may” and “in some embodiments” have generally been used to indicate that there are alternative implementations of a feature that can be used in conjunction with other features disclosed herein.

[0139] In the drawings, features which can only be present in some embodiments are generally drawn using dotted or dashed lines.

[0140] When the word “comprise” or “comprising” is used it should be interpreted as non-limiting, i.e. meaning “including but not limited to”.

[0141] The embodiments herein are not limited to the above-described embodiments. Various alternatives, modifications and equivalents can be used. Therefore, the above embodiments should not be taken as limiting the scope of the disclosure, which is defined by the appended claims.

Claims

1. A method for removing erroneous points from a set (355-1..355-15) of points of a 3D virtual object provided by 3D imaging of a corresponding real-world object (320) by means of a camera (330) having an image sensor (331), wherein the method comprises: - obtaining (401) the set (355-1..355-15) of points; - identifying (403) for a respective point (355) of the set a conflicting point, if any, which is a point of the set that cannot coexist with the respective point (355) effectively according to one or more predefined criteria; and - removing (404) from the set one or more points of the set that are involved in more conflicts than other points of the set that are involved in conflicts, based on said identification for the respective points (355) of the set, wherein the respective point (355) of the set is considered to be involved in a conflict each time a conflicting point is identified in the set for the respective point and / or each time the respective point itself is identified as a conflicting point of another point of the set, wherein the one or more predefined criteria are based at least on what the camera (330) virtually sees from its corresponding position in a coordinate system (353) of the set (355-1..355-15) of points.

2. The method of claim 1, wherein points that are involved in more conflicts than a predetermined threshold are removed from the set.

3. The method of any of claims 1-2, wherein the method further comprises: - obtaining (402) for a respective point (355) a respective camera direction (352) corresponding to a direction of a light emission from a corresponding point of the real-world object (320) towards the camera (330) that is sensed by the image sensor (331) during the 3D imaging; wherein a conflicting point is a point of the set that cannot coexist with the respective point (355) effectively based at least on its camera direction (352).

4. The method of any of claims 1-2, wherein, A conflicting point is a point of the set that cannot coexist with the respective point (355) effectively based on an assumption that points (355-1..355-15) of the set are connected by a coherent surface with nearest neighbor surface points and that the coherent surface would block light.

5. The method of any of claims 1-2, wherein the identification of conflicting points for a respective point is limited to points of the set that exist within a certain distance from the respective point.

6. The method of claim 3, wherein the 3D imaging is based on light triangulation, which comprises illumination of the real-world object (320) by a light source (310), wherein the light emission is reflected light from a surface of the real-world object (320) caused by the illumination.

7. A computer-readable storage medium comprising a computer program (603) comprising instructions which, when executed by one or more processors (604), cause one or more devices (600) to perform the method according to any of claims 1-6.

8. A device (600) for removing false points from a point set (355-1..355-15) of a 3D virtual object provided by 3D imaging of a corresponding real-world object (320) by means of a camera (330) having an image sensor (331), wherein the device is configured to: obtain (401) the point set (355-1..355-15); identify (403) for a respective point (355) of the set, if any, a conflicting point of the set, which is a point of the set that cannot coexist validly with the respective point (355) according to a predefined criterion or criteria; and remove (404) from the set one or more points of the set that are involved in more conflicts than other points of the set that are involved in conflicts, based on the identification of the respective points (355) of the set, wherein the respective point (355) of the set is considered to be involved in a conflict each time a conflicting point of the set is identified for the respective point and / or each time the respective point itself is identified as a conflicting point of another point of the set, wherein the predefined criterion or criteria being based at least on what is virtually visible from a corresponding position of the camera (330) in a coordinate system (353) of the point set (355-1..355-15).

Images