Determining a transformation having a deskew effect for a 3D image from a scan by a 3D imaging system

Abstract

To provide methods and devices for determining a transformation having a deskew effect on 3D image date resulting from a scan by a 3D imaging system.SOLUTION: The method includes obtaining 3D image data-resulting from a scan by a 3D imaging system-of actual surfaces of planes that are non-parallel and have a constant angular relationship or relationships between one another at least during the scan. The transformation is determined based on said one or more angular relationships such that the angular relationships between the detected imaged surfaces will match said angular relationships between the planar real surfaces.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] Embodiments herein relate to determining a transformation that has a deskewing effect on 3D image data resulting from a scan by a 3D imaging system, such as a machine vision system based on optical triangulation. [Background technology]

[0002] Industrial vision cameras and systems for factory and logistics automation may be based on three-dimensional (3D) machine vision, where a 3D image of an object, etc. is captured. A 3D image refers to an image that does not contain information such as intensity and / or color related to pixels in only two dimensions (2D) as in traditional images, or at least that information, but also contains "height" or "depth" information.

[0003] Generally, each pixel of an image captured by a camera has a position in image sensor coordinates that corresponds to a location that the camera and image sensor imaged in the real world, or more specifically, information about light sensed by an image sensing element of the image sensor from a location in the real world that the image sensing element corresponds to the pixel. Typically, what is sensed is light reflected from what is being imaged, e.g., an object. Depending on the camera and system, the light used, and the manner in which illumination by the light is provided, the sensed light may include various information about the location from which the light was reflected, such as information about the location on the object being imaged. Thus, a pixel of a captured image has a position in image sensor coordinates that corresponds to a location in the real world, such as a location on an object. The sensed light may also include additional information about the location, such as its intensity, color, reflectance, etc.

[0004] Many 3D machine vision cameras or systems, or generally 3D imaging systems for 3D imaging, such as imaging of an object, are based on multiple 2D images captured by the camera's image sensor, typically sequentially during scanning of the object. Each such 2D image may contain 3D information about the object's 2D profile, and thus the sum of such 2D images may contain 3D information about the entire object, from which a 3D image of the entire object can be formed. The 3D image may be represented by a "point cloud," in which each point corresponds to a location on the object and is associated with coordinates in 3D for that point. Each point may also be associated with further information about the point, such as color or other features associated with the corresponding object point.

[0005] When a pixel has a 3D location, rather than "only" a 2D location, it may be called a voxel.

[0006] Line scan image data typically results when image data for an image is scanned or provided one line at a time by scanning the object to be imaged using a light plane projected as a line of light onto the object and measuring the reflected light from the object.

[0007] A special case of 3D imaging by scanning is 3D imaging based on optical triangulation, in which structured light, typically a light plane or "sheet of light," is used, and an object is scanned through and / or by this light plane. During scanning, a light line is projected onto the object, and this light line corresponds to the position where the sheet or plane intersects with the object. While lasers are often preferred, other light sources can also provide structured light, such as a light plane; for example, the light source can provide light that remains focused and does not spread too much, such as light provided by a laser or light-emitting diode (LED). Instead of a light plane corresponding to a "sheet of light," for example, a light plane corresponding to an edge of illumination, i.e., a light edge, can be used.

[0008] 3D machine vision systems are often based on optical triangulation. In such systems, a light source illuminates an object with a specific light pattern, typically structured light such as the light plane described above. This type of 3D machine vision system or device is sometimes referred to as a system or device for 3D imaging based on optical triangulation, light plane triangulation, or simply laser triangulation if laser light is used. A line of light projected onto an object is imaged by a camera, i.e., the light reflected from the object is imaged. Along the line of light, optical triangulation captures 3D characteristics corresponding to the object's profile, including height information. In this way, by scanning the entire object, corresponding to a line scan and involving line and / or object movement, 3D characteristics of the entire object corresponding to multiple 2D profiles of the object can be captured, based on which a 3D image of the object can be formed as discussed above. To generate a profile image of the object during scanning, the reflected light from the object is captured by the camera's image sensor, and in particular, its intensity peaks are detected in the image data. The peaks occur at positions corresponding to the locations on the object where incident light corresponding to the line of light was reflected from the object, and the locations of the detected peaks in the image are mapped to the locations on the object where the light resulting in the peak was reflected according to the optical triangulation that the system is configured and set to perform.

[0009] Scanning-based 3D imaging systems, such as those based on optical triangulation, often cause skew effects. In 3D imaging systems based on optical triangulation and using a light plane for scanning, this occurs when the scanning direction, e.g., the object's movement direction, is not perpendicular to the light plane. As a result, the scanned object and the 3D image of the object, i.e., the imaged object, may appear distorted compared to the actual object. For example, for some system configurations with specific mounting positions of measurement units such as cameras, there is no or minimal skew effect. However, in general, scanning an object may result in, or be expected to result in, a skew effect, which should be eliminated or at least reduced.

[0010] If skew effects exist, some correction corresponding to the deskew should be performed.

[0011] The skew effect corresponds to the fact that real-world scanning results in a non-orthogonal coordinate system for the 3D image data: when the 3D image data, and therefore the scanned object, are visualized in a conventional orthogonal coordinate system, the imaged object appears distorted.

[0012] In other words, deskewing 3D image data corresponds to transforming a non-Cartesian coordinate system into a Cartesian coordinate system, or at least approximating such a coordinate system. To deskew 3D image data, a transformation can be determined and then applied to the 3D image data, which corresponds to reducing the non-orthogonality of the non-Cartesian coordinate system and preferably approximating it to orthogonality. The transformation may be referred to as a deskew transformation or skew transformation. In principle, deskewing is achieved by a skew that is opposite to the skew resulting from the scan.

[0013] The skew effect can be described by its projection onto one or more axes, which can be modeled using the angle of the sheet of light relative to the direction of travel. These angles correspond to how much one or two axes of a non-Cartesian coordinate system are skewed compared to the corresponding orthogonal axis, and therefore how much they should be corrected to become orthogonal. Each angle can therefore be considered to correspond to an angular offset relative to the respective orthogonal direction.

[0014] Thus, a deskew transform may be described as a transform that removes or reduces angular offsets, thereby removing or reducing the skew effects of image data.

[0015] Furthermore, with the transformation determined and therefore known, other operations that may be desirable or even necessary to be performed on the image data become easier, such as multi-camera alignment.

[0016] Conventionally, to determine the transformation, a predetermined object having point-referenced features is used and scanned. The object and point-referenced features have known geometric shapes and dimensions, and how they relate to each other in the object, and therefore in the real world, is known. The imaged point-referenced features are then identified in the image data resulting from the scan. The transformation can be determined based on how the imaged point features relate to each other and to their counterparts, i.e., point-referenced features in actual objects in the real world.

[0017] US Patent Application No. 2022290978A1 discloses a solution based on this approach.

[0018] However, conventional approaches based on point features of the target object typically have problems with accurately capturing the point features, identifying them in the image data, and / or there may be discrepancies between them during estimation, such that the transformations determined based on them may not provide sufficient deskewing effect. [Prior art documents] [Patent documents]

[0019] [Patent Document 1] U.S. Patent Application No. 2022290978A1 Summary of the Invention [Problem to be solved by the invention]

[0020] In view of the above, it is an object of the present invention to provide one or more improvements or alternatives to the prior art, which provides one or more improvements regarding the determination of a transformation having a deskew effect on 3D image data. [Means for solving the problem]

[0021] According to a first aspect of an embodiment herein, the object is achieved by a method for determining a transformation having a deskew effect on 3D image data resulting from a scan by a 3D imaging system, such as one based on optical triangulation. The 3D image data resulting from a scan by the 3D imaging system is acquired of actual surfaces of planes that are non-parallel at least during the scan and have one or more fixed angular relationships between each other. As a result, the 3D image data includes imaged surfaces of planes that correspond to the actual surfaces of the planes. The imaged surfaces of the planes are detected in the 3D image data. A transformation is determined based on the one or more angular relationships between the detected imaged surfaces of the planes and the actual surfaces of the planes, such that when the transformation is applied to a description of the detected imaged surfaces in the coordinates of the 3D data, the resulting angular relationship between the detected imaged surfaces will match the angular relationship between the actual surfaces of the planes.

[0022] According to a second aspect of an embodiment herein, the object is achieved by one or more devices for determining a transformation having a deskew effect on 3D image data resulting from a scan by a 3D imaging system, such as one based on optical triangulation. The device is configured to acquire 3D image data resulting from a scan by the 3D imaging system of planar actual surfaces that are non-parallel at least during the scan and have one or more fixed angular relationships between each other. As a result, the 3D image data includes planar imaged surfaces that correspond to the planar actual surfaces. The device is further configured to detect the planar imaged surfaces in the 3D image data. Furthermore, the device is configured to determine a transformation based on the one or more angular relationships between the detected planar imaged surfaces and the planar actual surfaces, such that when the transformation is applied to a description of the detected imaged surfaces in the coordinates of the 3D data, the resulting angular relationship between the detected imaged surfaces coincides with the angular relationship between the planar actual surfaces.

[0023] According to a third aspect of embodiments herein, the object is achieved by one or more computer programs comprising instructions which, when executed by one or more processors, cause one or more devices to perform the method according to the first aspect.

[0024] According to a third aspect of embodiments herein, the object is achieved by a carrier comprising a computer program according to the third aspect.

[0025] Thanks to the embodiments herein, it is possible to avoid using multiple point reference features in 3D as in the conventional case, and therefore also to avoid the associated problems mentioned in the background.

[0026] Additionally, embodiments herein enable positional and rotational invariance with respect to the arrangement of a set of planar actual surfaces: only one fixed relative angular relationship of the planar actual surfaces needs to be known, corresponding to the constant angular relationship or relationships between the planar actual surfaces during scanning.

[0027] Furthermore, embodiments herein enable scale invariance: simple objects with planar real surfaces can be used. Such objects can be presented at different scales for different sized fields of view, and the same algorithm will work the same way without changes in parameterization as traditionally required for point-referenced features.

[0028] Furthermore, embodiments herein allow for simpler algorithms for surface detection and calculation than in the conventional case using point-referenced features: plane estimation is generally simpler than sphere or cone fitting typically used in the conventional case.

[0029] Furthermore, embodiments herein enable more robust estimation than in conventional cases. For example, compared to fitting a cone or the like to estimate a cone to find point-referenced features corresponding to the apex of the cone, embodiments herein can use plane fitting to estimate a planar imaged surface, which can take into account more data corresponding to the points forming the respective surface, thereby providing more robust results.

[0030] Thus, the embodiments herein provide improvements over the prior art.

[0031] Example embodiments herein will be described in more detail with reference to the accompanying schematic drawings, which are briefly described below. [Brief explanation of the drawings]

[0032] [Figure 1] 1 is a diagrammatic illustration of an example of a prior art type 3D imaging system based on optical triangulation; [Figure 2A] 1 is a schematic diagram for visualizing, illustrating and explaining the skew effect; [Figure 2B] 1 is a schematic diagram for visualizing, illustrating and explaining the skew effect; [Figure 2C] 1 is a schematic diagram for visualizing, illustrating and explaining the skew effect; [Figure 2D] 1 is a schematic diagram for visualizing, illustrating and explaining the skew effect; [Figure 3A] FIG. 1 is a schematic diagram for visualizing, illustrating, and explaining the principles and ideas of the embodiments herein. [Figure 3B] FIG. 1 is a schematic diagram for visualizing, illustrating, and explaining the principles and ideas of the embodiments herein. [Figure 4] 1 is a schematic diagram illustrating and visualizing two real objects that can be used to provide 3D image data according to embodiments herein. FIG. [Figure 5] 1 is a schematic diagram illustrating and visualizing two real objects that can be used to provide 3D image data according to embodiments herein. FIG. [Figure 6] FIG. 1 is a schematic diagram illustrating an example of a 3D imaging system that may be used with and / or configured to perform embodiments herein. [Figure 7] 1 is a flow chart that schematically illustrates an embodiment of a method according to embodiments herein. [Figure 8A] 10A and 10B illustrate examples from reality without application of a transformation determined according to embodiments herein. [Figure 8B] 10A-10C illustrate examples from reality with application of transformations determined in accordance with embodiments herein. [Figure 9] 8 is a schematic block diagram illustrating an embodiment of how one or more devices may be configured to perform the methods and actions discussed in connection with FIG. 7. [Figure 10] 1 is a schematic diagram illustrating some embodiments of a computer program and its carrier; DETAILED DESCRIPTION OF THE INVENTION

[0033] The embodiments herein are exemplary embodiments. It should be noted that these embodiments are not necessarily mutually exclusive. Elements from one embodiment may be implicitly assumed to be present in another embodiment, and it will be clear to one skilled in the art how those elements can be used in other exemplary embodiments.

[0034] To enable a better understanding of the related art and as an advancement towards the embodiments herein, the state of the art and problems outlined above in the background will be further detailed before describing the embodiments herein.

[0035] FIG. 1 schematically illustrates an example of an imaging system 105 for 3D machine vision, or simply 3D imaging, based on optical triangulation, as discussed in the background. The system may be referred to as a machine vision system. The imaging system 105 is shown in the figure in a normal operating state. The imaging system 105 is an example of a system in connection with which embodiments herein may be used and applied, as described further below. The system 105 is configured to perform optical triangulation, here using a light sheet, or light plane 111. The system 105 further includes a light source 110, e.g., a laser, for illuminating a measurement object to be imaged with a specific light pattern, illustrated and shown in the figure as light plane 111. The light may be, but is not necessarily, laser light. In the illustrated example, the objects to be imaged are illustrated by a first measurement object 120 in the shape of an automobile and a second measurement object 121 in the shape of a gear structure. When a particular light pattern, here a light plane 111, is incident on an object, this corresponds to the projection of the light plane 111 onto the object, and can be seen as the light plane 111 intersecting the object. For example, in the illustrated example, the light plane 111 results in a light line 112 on the first measurement object 120. The light plane 111 incident on the object is reflected by the object, more specifically, by a portion of the object at an intersection line, i.e., the light line 112 in the illustrated example. The system 105 includes a camera 130 including an image sensor (not shown in FIG. 1 ). The camera and image sensor are positioned relative to the light source 110 and the object to be imaged such that the light plane 111, when reflected, is incident on the image sensor. The image sensor, typically implemented as a chip, is for converting the incident light into image data. The portion of the object that reflects the incident light onto the image sensor is thereby captured by the camera 130 and the image sensor, and corresponding image data can be generated and provided for further use.For example, in the illustrated example, light plane 111 may be reflected at light line 112 at a portion of a car roof of first measurement object 120 toward camera 130 and the image sensor, thereby generating and providing image data having information about the portion of the car roof. Using knowledge of the configuration, including the geometry, of system 105, e.g., knowledge of how the coordinates of the image sensor relate to a coordinate system 123, e.g., a Cartesian coordinate system, associated with the imaged object and its context, the image data may be converted into information about 3D properties, e.g., 3D shape or profile, of the imaged object in an appropriate format. The information about the 3D properties, e.g., 3D shape or profile, may include data describing the 3D properties in any appropriate format.

[0036] In practice, typically by scanning the object, for example by moving the light source 110 and / or the object to be imaged, such as the first measurement object 120 or the second object 121, so that multiple portions of the object are illuminated and produce reflected light on the image sensor, image data can be generated that describes a more complete 3D shape of the object, corresponding to multiple successive profiles of the object, such as the illustrated profile images 141-1 to 141-N of the first measurement object 120, where each profile image shows the outline of the first object 120 from which the light surface 111 was reflected when the image sensor of the camera 130 sensed the light resulting in the respective profile image. As shown in the figure, a conveyor belt 122 or the like may be used to move the object through the light plane 111, while the light source 110 and camera unit 130 are typically fixed, or the light plane 111 and / or camera 130 may be moved over the object so that all parts of the object, or at least all parts facing the light source 110, are illuminated and the camera 130 can receive light reflected from different parts of the object that it is desired to image.

[0037] As can be seen from the above, for example, the image frames provided by camera 130 and its image sensor imaging first measurement object 120 can result in any one of profile images 141-1 through 141-N. As noted in the background, the location of each of the first object's contours shown in any of profile images 141-1 through 141-N is typically determined based on identifying intensity peaks in the image data captured by the image sensor and locating these intensity peaks, for example, by one or more intensity peak-finding algorithms. System 105 and conventional peak-finding algorithms are typically configured to search for intensity peaks for each pixel column in each image frame. Sensor coordinates are u, v as shown in the figure. The u coordinate can be along the image sensor rows, and u can be used to indicate a position within each row, for example, corresponding to an image sensor column. Correspondingly, the v axis can be along the image sensor columns, and can be used to indicate a position within such a column, for example, corresponding to an image sensor row.

[0038] For each position u in the image frame, a peak location along v may be searched for, for example, by a peak finding algorithm as described above, and an identified peak in the image frame may result in one of profile images 141-1 to 141-N, as shown in the figure. The profile image is formed by image points in a sensor-based coordinate system 143, such as u, v, and t. The sum of the image frame and the profile image may be used to create a 3D image of the first object 120.

[0039] For example, after a system is installed in a location where it will be used, calibration may be used to more accurately determine how image sensor coordinates relate to real-world coordinates for the system. Typically, a specific, typically predetermined, object, i.e., a calibration object, is used for calibration. The calibration object has specific known features, measurements, etc., and is scanned by the system for calibration purposes. Knowledge and information about the object and knowledge and information from scanning the calibration object are then used to calibrate the system so that it can more accurately provide 3D image data of the scanned object. As shown in the figure, an object scanned as part of the system's normal intended use and operation, such as imaging and measuring the object, may be referred to as a product object or measurement object, as opposed to a calibration object. The deskewing discussed in the background may be part of a general calibration procedure or may be separate from calibrations performed for other reasons other than (deskew) skew.

[0040] The skew discussed in the background occurs, for example, when the scan direction s is not orthogonal to the light plane 111, i.e., when the scan direction s deviates from the y direction in the figure, where the light plane 111 lies in the zy plane. Such a deviation may be intentional, for example, if the system and / or its configuration provides some benefit, e.g., is tailored to extract specific information about the object, and requires or benefits from the scan direction being non-orthogonal to the light plane 111. Alternatively or additionally, there may be some deviation due to the system being imperfectly configured or not configured accurately enough, resulting in some measurable offset from what should be, such as from an orthogonal relationship or from what a previous skew-related calibration requires. In either case, it may be desirable to be able to eliminate or reduce the skew effect, for example, by using a transform as described in the background, which has a deskew effect on the resulting 3D image data. After peak detection, the transform is applied to the 3D image data for several profiles, which typically corresponds to a "point cloud", a height map, or a depth map, which may result from a scan of the entire object or become available after a period of time when the image data for several profiles has been provided by the system.

[0041] It should be understood that the scan direction may be non-orthogonal to the light plane 111 in any direction s parallel to the xy plane but non-orthogonal to the x axis, and / or in any direction s parallel to the zy plane but non-orthogonal to the x axis. From the diagram, this can be described as a new coordinate system, e.g., a sheared coordinate system sheared relative to the illustrated x, y, z coordinate system, such that x', y', z' have a y' direction along the scan direction s, and x' and / or z' are non-orthogonal to y'. Each non-orthogonal axis of the sheared coordinate system x', y', z' can be considered to be "offset" from orthogonality by a respective angle. The skew effect results from sampling during the scan being performed in such a sheared coordinate system. Deskewing corresponds to transforming the (skewed) 3D data from the scan, and therefore in the sheared coordinate system, into a Cartesian coordinate system. As should be understood, the angle or angles can be used for this purpose and can therefore be used to determine the appropriate transformation for deskewing. However, these angles are not generally known.

[0042] 2A-2D are schematic diagrams used to visualize, illustrate and explain the skew effect.

[0043] 2A-2B show the situations of a cubic object 220 without and with the skew effect, respectively. Here, we assume that the light plane used in the scan is in the rx-plane or the r'-x-plane, and the object 220 is scanned in the y-direction. In each figure, parallel dotted squares schematically show how the light plane "samples" the profile of the cube 220 during scanning, i.e., this illustrates how the profile of the cube 220 is captured. In FIG. 2A, there is a light plane 211a (in the rx-plane) that is orthogonal to the scanning direction (y), and there is no skew effect. On the other hand, in FIG. 2B, there is a light plane 211b (in the r'-x-plane) that is non-orthogonal to the scanning direction (y), and there is a skew effect. More specifically, in Fig. 2B, the light plane (in the r'-x plane) is tilted with respect to the scanning direction (y), which corresponds to the light plane (in the rx plane), which is orthogonal to the scanning direction (y) as in Fig. 2A, being rotated about the x-axis by a certain angle α, which results in r' instead of r. The coordinate system x, y, r' therefore corresponds to the shear coordinate system, as mentioned above.

[0044] The skew effect can be seen from the fine dotted line representation of how light surface 211b "samples" the profile of cube 220 compared to light surface 211a.

[0045] 2C-2D each show a schematic representation of the resulting 3D image of the same object when the object is imaged instead of cube 220 in the setup of FIGS. 2A-2B. FIG. 2C shows imaged object 225-1, which resembles the actual object "sampled" by light plane 211a instead of object 220 in the scan direction y. Imaged object 225-1 has no visible skew effect. FIG. 2D, on the other hand, shows imaged object 225-2, which has a clearly visible skew effect. The imaged object results from the actual object "sampled" by light plane 211a instead of object 220 in the scan direction y. The actual object scanned to produce the resulting imaged objects 225-1 and 225-2 is an example of an object mentioned in the context of having point-referenced features, which may be used for calibration and, as in the prior art, may use information from the calibration to determine a transformation to remove or reduce skew effects from 3D image data.

[0046] On the other hand, embodiments herein are based on the insight that point-referenced features do not need to be used, but instead planar surfaces of real objects with known angular relationships between the surfaces can be used. Planar surfaces are imaged as planar imaged surfaces even in the case of skew. After scanning a real object, planar imaged surfaces are detected in the resulting 3D image data instead of point-referenced features as in the past, and from this, angular relationships between imaged surfaces can be found, for example, by fitting a plane to each detected surface and calculating the angular relationship based thereon. The detected image surfaces are preferably each described by their normals, which allows each surface to be easily described with less data and may also simplify calculations. A deskew transformation is then determined, for example, by estimation. This can be done in a manner similar to the prior art, for example, by iteratively applying and testing adjustable transformations, such as by using a nonlinear solver, until a transformation that provides a sufficient skew effect according to testing is found. However, in the prior art, this is done based on point-referenced features, the actual point-referenced features in the real object, and the imaged point-referenced features affected by the transformation. The embodiments herein are instead based on considering how the adjusted transformation, when applied, affects the angular relationships between planar imaged surfaces, which, for deskewing effects, should be the same as, or at least close to, the corresponding angular relationships of the real object, i.e., the angular relationships between the transformed imaged surfaces are compared to the corresponding known angular relationships between the actual planar surfaces of the real object.

[0047] Preferably, an actual object having at least three planar, non-parallel surfaces is used, with the three surfaces having the above-mentioned angular relationship between them. This is because, as discussed above, three planar surfaces can provide sufficient information about skew in any skew direction that occurs. Therefore, using three surfaces can provide more generally useful information. In some skew situations, two surfaces may be sufficient, but this is insufficient for some cases that may occur because two surfaces can only provide information about skew in one dimension, and the skew effect may exist in two different dimensions simultaneously.

[0048] 3A-3B are schematic diagrams used to visualize, illustrate, and explain the principles and ideas of embodiments herein. FIG. 3A schematically illustrates a real object 320, a cube, being scanned by a 3D imaging system according to a specific configuration that results in a skew effect. The real object 320 is a known, e.g., predetermined, object with planar surfaces 322a-322c and a known angular relationship between the surfaces, which in this example, for a cube, is 90 degrees. Each planar surface 322a, 322b, 322c is described and represented by its normal 324a, 324b, 324c, e.g., represented by a respective vector. For example, two surfaces that are at a 90-degree angle to each other mean that the scalar product between the surface normal vectors is zero. For planar surfaces with a non-zero angular relationship, the angular relationship corresponds to how the normals relate to each other. FIG. 3B shows an imaged object 325 based on 3D image data from a scan of the skewed real object 320. As can be seen in the example, the result is that the imaged object 325 corresponds to a skewed version of the cube that is the real object 320. There are still imaged planar surfaces 327a, 327b, and 327c that correspond to the real planar surfaces 322a, 322b, and 322c of the real object 320. These can be detected in the 3D image data corresponding to the imaged object 325. Planes can be fitted to the target planar surfaces 327a-327c, and their normals 329a-329c can be determined. Then, for example, the normals 329a-329c can be used to compare the angular relationships between the real planar surfaces 322a-322c of the real object 320 with the corresponding angular relationships between the corresponding planar imaged surfaces 327a-327c. As already indicated above, an adjustable transformation can be applied and adjusted until the angular relationships of the imaged surfaces 327a-327c result in the angular relationships being preferably the same as, or at least desirably close to, or closer to, the corresponding angular relationships of the planar surfaces 322a-322c of the actual object 310.This means that the transformation will have a deskew effect even if it is subsequently applied to other imaged objects scanned with the same imaging system and settings.

[0049] While a simple cube as in the above example is sufficient, the principle will of course also work for other shapes of real objects with planar surfaces, which may be more advantageous for practical and specific use cases. As already mentioned above, as with 3D image data, at least two angles may be needed to describe skew in 3D, so preferably a real object with at least three non-parallel planar surfaces should be used. In general, for N angles, at least N+1 surfaces are needed.

[0050] 4-5 are schematic diagrams illustrating and visualizing two further real objects that can be used to provide 3D image data according to embodiments herein.

[0051] 4 shows a real object 420 having three planar real surfaces 422a-422c corresponding to the corners of a cube. It has been found that a cube corner contains three surfaces and is sufficient to provide information about the angular relationship corresponding to two skew angles. Also, a cube corner occupies less space and there is less risk that the scan will be positioned to insufficiently sample the cube than the entire cube.

[0052] FIG. 5 illustrates a real object 520 having three planar real surfaces 522a-522c with angular relationships of less than 90 degrees. This can be advantageous because, in some imaging system configurations, obtaining good sampling of three surfaces with 90-degree angular relationships can be difficult because it requires both a light and a camera to simultaneously illuminate and observe each surface. This may be easier if the angular relationships are less than 90 degrees. However, if the surfaces can be sampled sufficiently well, there are benefits from a large difference in the angular relationships between the surfaces, such as at or near 90 degrees, and disadvantages if the difference is too small or very small, such as near 0 degrees. A larger angular difference typically leads to a clearer distinction between the surfaces and better numerical stability of solutions based on embodiments herein, which may make it easier to determine the transformation more accurately.

[0053] Furthermore, the real object 420b is not symmetrical, i.e., asymmetrical, which is advantageous because it can be used to ensure that the imaged surfaces are not confused and can be used to detect the configuration of a mirrored system. Correcting for mirroring can be beneficial in certain applications, such as reading text. In this way, a correctly oriented Cartesian coordinate system can be ensured.

[0054] Note that, in principle, it is not necessary for the actual object to have non-parallel planar actual surfaces; actual surfaces with known angular relationships to each other would suffice. For example, a support surface for the object to be scanned, such as the conveyor belt mentioned above, could be scanned and adjusted during the scan, capturing several profile images of each at two or more different angular relationships. This can be achieved by tilting the support surface. The resulting 3D data would include imaged planar surfaces of a virtual imaged object formed from the scan, including imaged planar surfaces corresponding to support surfaces with different inclinations. This would enable, for example, a system that can "self-calibrate" for a particular setup, at least with respect to skew, and automatically eliminate or reduce skew effects. However, for simplicity and robustness, using a separate, single object is likely to be preferred in most practical situations.

[0055] FIG. 6 schematically illustrates an example of a 3D imaging system 605 that may be used with and / or configured to implement embodiments herein. The system is for 3D imaging of an object based on a scan; more specifically, the illustrated system is based on optical triangulation. The imaging system 605, how profile images are captured from the scan, the optical triangulation, peak detection, and how 3D image data corresponding to a 3D image of the scanned object can be formed, may be similar to the prior art; in this respect, the imaging system 605 may correspond to the imaging system 105 described above. Accordingly, the imaging system 605 correspondingly includes a camera 630 having an image sensor 631 and a light source 610 for illuminating an object to be imaged with a particular light pattern, such as a light plane 611. The object to be imaged is illuminated by the light plane 611 in a field of view 625 of the camera 630. Such an object is schematically illustrated in the figure as actual object 620. The real object 620, in the case of embodiments herein, may be, for example, any of the real objects 320, 420, 520 discussed above, or any other real object having planar real surfaces that are non-parallel to each other and have one or more fixed angular relationships between them during scanning and provision of 3D image data.

[0056] In addition to the image sensor 631, the camera 630 may further include one or more processors, memories, etc., and / or the image sensor 631 may have some integrated processing capabilities of its own. Of course, there are typically also input and / or output interfaces for controlling the camera 630 and / or for inputting and / or outputting information, such as information extracted from captured images and / or 3D image data based on found peaks, etc. One or more processors may be used to integrate the same or similar capabilities as a computer into the camera, i.e., within the same unit, making the camera more autonomous and capable of performing more of its own processing and / or control. That is, obtaining corresponding functionality without the need for additional and separate units, such as a separate computer, and / or enabling a simpler setup process and / or faster processing, for example, by reducing the need for communication between separate units. In such cases, the camera 630 may solely perform embodiments herein, as described below. In some embodiments, system 605 corresponds to a camera 630, i.e., system 605 corresponds to or at least resembles a camera and may be in the form of or implemented as a single unit including camera functionality. Network capabilities, light sources, etc. may also be integrated into such a unit.

[0057] However, in some embodiments, camera 630 is a camera unit having only some traditional image-specific capabilities; for example, in addition to imaging involving image sensor 641, there may be some image processing capabilities within camera 630, e.g., for single image and peak detection. Camera 630 may be connected to one or more other parts of the system, such as computing device 640 as illustrated in the figure, having additional processing capabilities. Computing device 640 may be configured to control camera 630 and / or other parts of system 605, if present, and / or to perform embodiments herein based, for example, on 3D image data resulting from image data captured by the camera during scanning of real object 620 and input information regarding the angular relationship between the planar real surface of real object 602. Computing device 640 may receive 3D image data corresponding to a profile image of real object 620 from camera 630. Computing device 640 may comprise one or more processors, memories, etc., and may correspond to a computer. To illustrate that the computing power and the like for carrying out embodiments herein may alternatively or additionally be remote in time and / or space from where the scan is performed, the figure also shows a schematic computer cloud 650, representing one or more servers or remote computers. These may be part of system 605 or separate from system 605, but may be configured to operate on 3D image data from imaging system 605 or a similar system and use information about angular relationships between actual surfaces of planes scanned by the system to provide 3D image data. Thus, embodiments herein may be performed remotely, for example, as a computer cloud service and / or on one or more servers that may be communicatively connected to imaging system 605.

[0058] 7 is a flow chart for outlining an embodiment of a method according to some embodiments herein, the following actions that may form a method for determining a transformation having a deskew effect on 3D image data resulting from a scan by a 3D imaging system that may be based on optical triangulation, such as 3D imaging system 605, which is used below as an example of a 3D imaging system.

[0059] 7 may be performed by one or more devices, i.e., 3D imaging system 605, or some device that is part of or external to the system, such as camera 630 and / or computing device 640 and / or computer cloud 650. Devices for performing methods and their actions are also described in more detail below.

[0060] The following actions may, where possible and appropriate, be performed in any suitable order and / or with full or partial overlap in time.

[0061] Action 701 The 3D image data resulting from the scanning by the 3D imaging system 505 of non-parallel planar actual surfaces that have one or more constant or fixed angular relationships with each other at least completely or partially during scanning is acquired, whereby the 3D image data includes imaged planar surfaces corresponding to the actual planar surfaces. Examples of planar actual surfaces are the planar actual surfaces 322a-322c, 422a-422c, and 522a-522c, and the planar actual surfaces 322a-322c will be primarily used as an example below. Examples of imaged surfaces are the imaged surfaces 327a-327c, which will be used as an example below. It should be understood that "constant during scanning" in this context means constant so that the scan can adequately sample, or in other words, capture a sufficient number of profile images of each planar actual surface to form its corresponding planar imaged surface. As already mentioned above, planar scanned actual surfaces will result in planar imaged surfaces even when skew effects are present, but the angular relationships between such surfaces are affected.

[0062] The actual surfaces of said planes preferably have a fixed relationship to each other during the entire scan, for example by being part of the same object that does not change.

[0063] Alternatively, the actual surfaces of the plane may be part of different objects, but these surfaces are in a fixed relationship to each other at least during scanning; for example, one surface may correspond to a support surface for the object being scanned, and the other surface or surfaces may correspond to surfaces of the object.

[0064] It should be noted that, as already discussed elsewhere above, in some particular embodiments, the same physical surface may be adjusted during a scan, and in that case may be adjusted during a portion of a scan, to achieve two or more of the actual surfaces being non-parallel planes having a fixed angular relationship between each other during the scan.

[0065] The actual surfaces of the planes and their relationships to each other, and therefore their angular relationship(s), are preferably predetermined and therefore known in advance at the time of scanning, although the actual surfaces and the angular relationships between them may alternatively be determined, for example, by measurement after scanning.

[0066] When specific objects with planar real surfaces are used, they are preferably predetermined, with predetermined real surface and angular relationships, and may be referred to as calibration objects. Note that such objects may be for skew purposes only, or may additionally include other calibration-related features, or there may be separate objects for other calibrations that may be performed wholly or partially apart from the embodiments herein.

[0067] It may be easier to achieve planar real surfaces having a predetermined and / or precise angular relationship between one another if the planar real surfaces are part of the same object and can be manufactured as part of the same object. Thus, in some embodiments, the planar real surfaces, such as planar real surfaces 322a-322c, are part of the same real object, such as real object 320. In some of these embodiments, the real object, e.g., real object 520, is asymmetric. Why this is beneficial was also discussed above.

[0068] Action 701a In some embodiments, for example, when the method is performed by a 3D imaging system 605, the 3D image data is acquired by scanning the actual plane of the plane with the 3D imaging system 605.

[0069] In other embodiments, when the method is performed by a single device, for example part of or external to the 3D imaging system 605, such device may acquire the 3D image data by receiving the 3D image data from the 3D imaging system 605, typically its camera 630.

[0070] Action 702 Planar imaged surfaces 327a-327c are detected in the 3D image data. The imaged surfaces can be captured by more samples and therefore described by more points in the 3D image data than point-referenced features imaged as in the prior art solutions described above. Thereby, detection of planar imaged surfaces in the 3D image data may be more robust and simpler than in the case of point-referenced features, and / or may allow for a more accurate representation of their real-world counterparts.

[0071] The detected planar imaged surface is preferably described as a planar surface in the coordinate system of said 3D image data.

[0072] This action preferably includes fitting each surface to each imaged surface, which may be done according to any known method for doing so, for example, using a robust least-squares method. Thus, each fitted plane is perfectly planar, i.e., completely planar, and the fitting may involve minimizing the median or average distance to points forming each imaged surface. Thus, fitting a plane can be used to estimate a planar imaged surface, both for detection and to provide a simple description of each imaged surface, for example, by its normal, a plane equation, or any method by which a planar surface can be described. Each normal of each planar imaged surface may be simply described by a vector in the coordinate system of the 3D image data.

[0073] It should be noted that instead of fitting a plane, the imaged surface of each plane can be detected more directly by finding multiple points of the plane's surface in the 3D image data and using them as a representation of the detected plane's surface and / or by using these points to form a description of the plane's surface.

[0074] Action 703 A transformation is determined based on the one or more angular relationships between the detected planar imaged surfaces 327a-327c and the planar actual surfaces 322a-322c such that when the transformation is applied to a description of the detected planar imaged surfaces 327a-327c in the coordinate system of the 3D data, the transformation results in angular relationships between the detected imaged surfaces 327a-327c that match the angular relationships between the planar actual surfaces 322a-322c. In other words, the transformation is determined such that when used on the 3D image data, angular relationships between the planar imaged surfaces 327a-327c that result in the planar imaged surfaces 327a-327c being imaged match the corresponding angular relationships between the planar actual surfaces 322a-322c.

[0075] As already indicated above, the embodiments herein are based on the realization that one or more angular relationships between a detected planar imaged surface and a corresponding planar actual surface contain all the information necessary to be able to determine a deskew transformation. Therefore, there is no need to use point reference features as in the prior art. The imaged surface is planar even in the case of skew, and the transformation can be determined by considering and comparing the angular relationships between the imaged surface and the corresponding actual surface.

[0076] The angular relationships between the detected planar imaged surfaces 327a-327c required to match the angular relationships between the planar actual surfaces 322a-322c may be according to one or more criteria, which may be predetermined. Additionally or alternatively, match occurs when the angular relationships between the planar imaged surfaces 327a-327c are the same as, and / or closer to, and / or sufficiently close to, the angular relationships between the planar actual surfaces 322a-322c. Match should at least involve the angular relationships between the planar imaged surfaces 327a-327c being closer to the corresponding angular relationships from the application of the transformation than they would be if the transformation were not applied, and thus the transformation providing some deskewing effect.

[0077] The angular relationship corresponds to and can be represented by the angle between the surfaces of the relevant planes, but does not necessarily have to be represented as or by the angle itself, other ways of representing the relationship are possible, for example by the normals of the surfaces and how the normals relate to each other, which contains information about the angle between the surfaces. This is of course the case for both the actual surfaces of the planes and the imaged surfaces of the detected planes.

[0078] Determining a transformation based on the detected planar imaged surface may correspond to or include determining a transformation based on or using a description and / or representation of the detected planar imaged surface, e.g., using a plane fitted to said planar imaged surface and / or normals to the detected planar imaged surface, as discussed above. When normals are used, this matching of the angular relationships of the imaged surfaces and the angular relationships of the actual surfaces is typically based on the normals of those surfaces, e.g., matching angles between normals or how the normals relate to each other, rather than directly matching angles between the surfaces.

[0079] Determining the transformation may include testing one or more determined candidate transformations until said match occurs.

[0080] Action 703a In some embodiments, an adjustable transform is provided for operations on input 3D image data coordinates, thereby transforming the input 3D image data coordinates into output 3D image data coordinates. The transform is adjustable such that adjusting the transform results in a change in the deskew effect provided by the transform.

[0081] In practice, the transformation can introduce, and be adjusted to introduce, a skew, which corresponds to a deskew if the skew due to the transformation is opposite to or otherwise cancels the skew present in the 3D image data, as also discussed above.

[0082] Action 703b In embodiments where action 703a is performed, a transformation is iteratively applied and adjusted to the imaged surfaces 327a-327c of the detected plane described in the coordinate system of the 3D image data until the match occurs.

[0083] The application of the transformation to the detected planar image surfaces is typically to a description or representation of the detected surfaces, such as their normals, as discussed above.

[0084] Actions 703a-703b correspond to the basic and typically preferred numerically based approach to determining the transformation, for example, a suitable non-linear solver may be utilized and adapted for the purpose, as already mentioned above.

[0085] In some embodiments, iteratively applying and adjusting the transformation as in actions 703a-703b involves applying a nonlinear solver to a function based on the difference between each angular relationship of the actual surfaces and each corresponding transformed angular relationship between the imaged surfaces.

[0086] Finding the parameters for deskewing can be done using a nonlinear least-squares solver and an appropriate 3D transformation. For example, the Ceres solver is one such solver, available as open source for modeling and solving large, complex optimization problems. This and other similar types of optimization problems can be solved using an auto differentiated cost function. For application in this context, the Ceres solver can be used to take the observed plane normal and form a cost function that directly minimizes the dot product difference between the observed normal and the expected dot product in an orthogonal, real-world coordinate system.

[0087] Using a nonlinear solver is typically efficient and simple. However, other methods can also be used, which may require more computation, but this may not be a problem if there is sufficient computing power in the near future. Essentially, methods based on simple iterative guessing can be used. For example, if a guess results in a reduction in error, i.e., has some deskewing effect, further guesses closer to that guess can be made.

[0088] As already indicated above, in some embodiments, for example, as part of the iterative numerical method according to actions 703a-703b, the planar actual surfaces 322a-322c and the planar imaged surfaces 327a-327c are each represented by their normals, and each angular relationship is represented by a respective relationship between the normals of the surfaces to which the respective angular relationship pertains. Using normals instead of plane equations or two or more vectors in each plane allows for simpler and more efficient calculations.

[0089] An alternative to the iterative numerical method according to actions 703a-703b may be to determine the transformation analytically.

[0090] Thanks to the embodiments herein as described above in relation to actions 701-703, it is possible to avoid using multiple point reference features in 3D as is the conventional case, and therefore avoid the associated problems as mentioned in the background.

[0091] Additionally, embodiments herein enable positional and rotational invariance with respect to the arrangement of a set of planar actual surfaces: only one fixed relative angular relationship of the planar actual surfaces needs to be known, corresponding to the constant angular relationship or relationships between the planar actual surfaces during scanning.

[0092] Furthermore, embodiments herein enable scale invariance. Simple objects with planar real surfaces, such as those exemplified above, can be used. Such objects can be presented at different scales for different sized fields of view, and the same algorithm will work the same way without changes in parameterization as traditionally required for point-referenced features.

[0093] Furthermore, embodiments herein allow for simpler algorithms for surface detection and calculation than in the conventional case using point-referenced features: plane estimation is generally simpler than sphere or cone fitting typically used in the conventional case using point-referenced features.

[0094] Furthermore, embodiments herein enable more robust estimation than in conventional cases. For example, with reference to the skew example object of FIG. 2D , compared to fitting a cone, thereby estimating the cone, and finding point-referenced features corresponding to the apex of the cone, embodiments herein can use plane fitting to estimate a planar imaged surface, which can consider more data corresponding to the points forming the respective surface, thereby providing more robust results.

[0095] Thus, the embodiments herein provide improvements over the prior art.

[0096] 8A-8B show examples from reality without and with application of a transformation determined in accordance with embodiments herein. FIG. 8A shows a first imaged object 835a resulting from a scan, having the skew effect of the real object, and thus without application of embodiments herein. FIG. 8B shows a second imaged object 835b resulting when 3D image data corresponding to the first imaged object 835a is transformed using a transformation determined in accordance with embodiments herein, thereby having a deskew effect on the 3D image data, as can be seen from a comparison of the figures. The second imaged object 835b corresponds to a deskewed version of the first transformed object 835a.

[0097] The actual object that was scanned and resulted in the original 3D image data, visualized here as first imaged object 835a, was scanned as discussed above by a system corresponding to system 605, such that it would be scanned like actual object 220 in FIG. 2B and would have the skew effect present. However, the actual object behind that shown in FIGS. 8A-8B is the same actual object that was scanned to result in the schematic example, since the skew effect is clearly visible for such an object. As noted above, this object is the object with point reference features traditionally used to determine the deskew transformation, but it should be noted that the actual object used for the object in FIGS. 8A-8B is used only to visualize skew and deskew, and not to determine the deskew transformation, with or without the use of embodiments herein. The transformation that was used and resulted in the deskew effect shown by FIGS. 8A-8B was determined in accordance with embodiments herein, and thus was determined using an actual object corresponding to, for example, actual object 420. The real object was then scanned with the same system and settings used to scan other real objects, which resulted in the 3D image data visualized by imaged object 835a in Figure 8A.

[0098] FIG. 9 is a schematic block diagram illustrating an embodiment of one or more devices 900, i.e., device 900, which may correspond to the devices already discussed above for performing embodiments herein, such as for performing the methods and / or actions described in connection with FIG. 7 .

[0099] The device 900 may correspond, for example, to any one of the 3D imaging systems 605, or to a part thereof or a device external to the system, such as the camera 630 and / or the computing device 640 and / or the computer cloud 650. As will be appreciated by those skilled in the art, some embodiments of the present invention include actions that may be performed in a distributed manner by multiple devices configured to perform the actions. It should also be understood that the device 900 performing a method according to some embodiments may perform the method at a time later than the time at which any imaging system is operated and provides 3D image data. Thus, the method may be performed by a device that is remote in time and space from the imaging system and the imaging itself. Output from the imaging system may be uploaded, for example, to a server or computer cloud, such as the computer cloud 650, that includes a computing device configured to perform the method, or from where the computing device for performing the method obtains the image data. However, it may be preferable for the device configured to perform the method to be part of the imaging system and imaging involved, or at least to perform the method in conjunction with it, as this may allow for more reliable, simpler and faster execution, and the transformation determined by the method is then used on subsequent 3D image data resulting from a scan with the same imaging system.

[0100] The schematic block diagram is intended to illustrate an embodiment of how device 900 may be configured to perform the methods and actions discussed above in relation to Figure 7. Thus, device 900 is for determining a transformation that has a deskew effect on 3D image data resulting from a scan by a 3D imaging system, as described above for the method.

[0101] The device 900 may comprise a processing module 901, such as a processing means, e.g., one or more hardware modules including one or more processing circuits, circuits such as a processor, and / or one or more software modules for performing the methods and / or actions.

[0102] The device 900 may further comprise a memory 902 that may comprise, e.g., contain or store, a computer program 903. The computer program 903 comprises "instructions" or "code" that are executable directly or indirectly by the device 900 to perform the methods and / or actions, respectively. The memory 902 may comprise one or more memory units and may further be arranged to store data, such as configurations, data, and / or values, involved in or for performing the functions and actions of the embodiments herein.

[0103] Further, each device 900 may comprise processing circuitry 904, illustrative of a hardware module, involved in processing, e.g., encoding, data, and may comprise or correspond to one or more processors or processing circuits. Processing module 901 may, for example, comprise such processing circuitry 904, for example, be embodied in the form of, or be "implemented" by, such processing circuitry 904. In these embodiments, memory 902 may comprise computer programs 903, respectively executable by processing circuitry 904, such that each device 900 is operable or configured to perform the methods and / or actions thereof.

[0104] Typically, device 900 includes an input / output (I / O) module 905 configured to participate in any communication to and / or from other units and / or devices, such as transmitting information to and / or receiving information from other devices, e.g., by performing methods thereof. I / O module 905 may be exemplified by an obtaining module, e.g., a receiving module, and / or a providing module, e.g., a transmitting module, where applicable.

[0105] Additionally, in some embodiments, device 900, e.g., processing module 901, includes one or more of an acquisition module, a detection module, a scanning module, an application module, a conversion module, an adjustment module, a determination module, a scanning module, as illustrative hardware and / or software modules for performing actions of embodiments herein, which may be implemented in whole or in part by processing circuitry 904.

[0106] therefore, The device 900, and / or the processing module 901, and / or the processing circuitry 904, and / or the I / O module 905, and / or the acquisition module are operable or configured to acquire said 3D image data resulting from a scan of the actual surface of said plane by a 3D imaging system.

[0107] The device 900, and / or the processing module 901, and / or the processing circuitry 904, and / or the I / O module 905, and / or the detection module are operable or configured to detect the imaged surface of said plane in the 3D image data.

[0108] The device 900, and / or the processing module 901, and / or the processing circuit 904, and / or the I / O module 905, and / or the determination module are operable or configured to determine a transformation based on the one or more angular relationships between the imaged surface of the detected plane and the actual surface of the plane.

[0109] In some embodiments, the device 900, and / or the processing module 901, and / or the processing circuitry 904, and / or the I / O module 905, and / or the acquisition module, and / or the application module are operable or configured to obtain the adjustable transformation and iteratively apply the transformation to the imaged surface of the detected plane described in the coordinate system of the 3D image data and adjust the transformation until the match occurs.

[0110] In some embodiments, the device 900, and / or the processing module 901, and / or the processing circuitry 904, and / or the I / O module 905, and / or the acquisition module, and / or the scanning module are operable or configured to scan the actual surface of said plane by a 3D imaging system.

[0111] FIG. 10 is a schematic diagram illustrating some embodiments of a computer program 903 and its carrier for causing the device 900 to perform the methods and actions.

[0112] The computer program 903 comprises instructions that, when executed by the processing circuit 904 and / or the processing module 901, cause the device 900 to perform the operations described above. In some embodiments, one or more carriers, i.e., carriers, or more specifically, data carriers such as computer program products, are provided that comprise the computer program. Each carrier may be one of an electrical signal, an optical signal, a radio signal, and a computer-readable storage medium, such as the computer-readable storage medium 1001 shown schematically in the figure. Thus, the computer program 903 may be stored on the computer-readable storage medium 1001. A carrier may exclude a transitory propagating signal, and the data carrier may accordingly be termed a non-transitory data carrier. Non-limiting examples of computer-readable storage media and data carriers are memory cards or memory sticks, disk storage media, or mass storage devices typically based on hard drives or solid-state drives (SSDs). The computer-readable storage medium 1001 may be used to store data accessible via a computer network 1002, such as the Internet or a local area network (LAN). The computer program 903 may further be provided as a pure computer program or may be contained within a file. The file may be stored on a computer-readable storage medium 1001 or may be available by download, such as via a server, for example, via a computer network 1002 as shown in the figure. The server may be a web server, a File Transfer Protocol (FTP)-based server, or similar server. The file may be an executable file for direct or indirect download to the device and execution on the device, for example, by execution by the processing circuitry 904 to cause the device to perform as described above. The file may also or alternatively be for intermediate download and compilation, with the same or another processor being executable before further download and execution, causing the device 900 to perform as described above.

[0113] It should be noted that any of the processing modules and circuits described above may be implemented as software modules and / or hardware modules, e.g., within existing hardware and / or as an application specific integrated circuit (ASIC), field programmable gate array (FPGA), etc. Also, any of the hardware modules and / or circuits described above may be included, for example, within a single ASIC or FPGA, or may be distributed across several separate hardware components, whether individually packaged or incorporated into a system on a chip (SoC).

[0114] Those skilled in the art will also understand that the modules and circuits discussed herein may refer to hardware modules, software modules, analog and digital circuits, and / or one or more processors configured with software and / or firmware stored, for example, in memory, that, when executed by the one or more processors, may cause a device, sensor, etc. to configure and / or perform the methods and actions described above.

[0115] Identification by any identifier herein may be implicit or explicit. Identification may be unique in a particular context, for example, to a particular computer program or program provider.

[0116] The term "memory" as used herein may refer to a data memory for storing digital information, typically a hard disk, magnetic storage, medium, portable computer diskette or disk, flash memory, random access memory (RAM), etc. Additionally, the memory may be an internal register memory of a processor.

[0117] Also, note that any enumerated terminology, such as first device, second device, first surface, second surface, etc., should be considered open-ended and no particular hierarchical relationship is implied by such terminology. Absent explicit information to the contrary, enumerated naming should be considered merely a way of achieving distinct names.

[0118] As used herein, the phrase "configured to" may mean that a processing circuit is configured or adapted, by software or hardware configuration, to perform one or more of the actions described herein.

[0119] As used herein, the term "number" or "value" may refer to any type of number, such as a binary number, a real number, an imaginary number, or a rational number. Furthermore, a "number" or "value" may be one or more symbols, such as a character or a string of characters. Additionally, a "number" or "value" may be represented by a string of bits.

[0120] As used herein, the phrases "may" and "in some embodiments" are typically used to indicate that the described features can be combined with any of the other embodiments disclosed herein.

[0121] In the drawings, features that may be present in only some embodiments are typically depicted using dotted or dashed lines.

[0122] When the word "comprises" or "comprising" is used, it shall be construed as meaning open-ended, i.e. "consisting of at least".

[0123] The embodiments herein are not limited to the embodiments described above. Various alternatives, modifications, and equivalents may be used. Therefore, the above embodiments should not be construed as limiting the scope of the present disclosure, which is defined by the appended claims.

Explanation of Symbols

[0124] 220 Cubic object, object, cube 211a Smooth surface 211b Smooth surface 225-1 Imaged object 225-2 Imaged object 320 Actual object 322a Flat surface, actual flat surface 322b Flat surface, actual flat surface 322c Flat surface, actual flat surface 324a Normal line 324b Normal line 324c Normal line 325 Imaged object 327a Imaged flat surface, target flat surface, imaged surface of the flat surface 327b Imaged flat surface, target flat surface, imaged surface of the flat surface 327c Imaged flat surface, target flat surface, imaged surface of the flat surface 329a Normal line 329b Normal line 329c Normal line 420 Actual object 420b Actual object 422a Actual flat surface 422b Actual flat surface 422c Actual flat surface 520 Actual object 522a Actual flat surface 522b Actual flat surface 522c Actual flat surface 605 3D Imaging System 610 Light source 611 Smooth surface 620 Actual object 625 Field of view​​​​​​ 650 Computer Cloud 835a First Imaged Object 835b Second Imaged Object 900 devices 901 Processing Module 902 memory 903 Computer Programs 904 Processing Circuit 905 Input / Output (I / O) Module 1001 Computer-readable storage medium 1002 Computer Network

Claims

1. 1. A method for determining a transformation having a deskew effect on 3D image data resulting from a scan by a 3D imaging system (605), said method comprising: a step (701) of acquiring 3D image data resulting from a scan by the 3D imaging system (605) of planar actual surfaces (322a-322c, 422a-422c, 522a-522c) that are non-parallel at least during the scan and have one or more fixed angular relationships between each other, whereby the 3D image data includes planar imaged surfaces (327a-327c) that correspond to the planar actual surfaces (322, 422, 522); detecting (702) the planar imaged surface (327) in the 3D image data; determining (703) the transformation based on the one or more angular relationships between the detected imaged surfaces (327a-327c) of the planes and the planar actual surfaces (322a-322c, 422a-422c, 522a-522c) such that, when the transformation is applied to a description of the detected imaged surfaces (327a-327c) in the coordinates of the 3D data, the result is that the angular relationships between the detected imaged surfaces (327a-327c) match the angular relationships between the planar actual surfaces (322a-322c, 422a-422c, 522a-522c); A method comprising:

2. The determining action is: obtaining an adjustable transformation for operation on input 3D image data coordinates, thereby transforming the input 3D image data coordinates into output 3D image data coordinates (703a), wherein adjusting the transformation results in a change in the deskew effect provided by the transformation; iteratively applying said transformation to the imaged surface (327a-327c) of said detected plane described in said coordinate system of said 3D image data and adjusting said transformation (703b) until said match occurs; 2. The method of claim 1, comprising:

3. 3. The method of claim 2, wherein the step of iteratively applying and adjusting the transformation involves application of a nonlinear solver to a function based on the difference between each angular relationship of the actual surfaces and each corresponding transformed angular relationship between the imaged surfaces.

4. 4. The method of claim 1, wherein the planar actual surfaces (322a-322c) and the planar imaged surfaces (327a-327c) are each represented by their normals, and respective angular relationships are represented by respective relationships between the normals of the surfaces to which the respective angular relationships relate.

5. The action to be obtained is: a step (701a) of scanning the actual surface (322a-322c) of the plane by the 3D imaging system (605); 5. The method according to any one of claims 1 to 4.

6. The method according to any one of claims 1 to 5, wherein the planar real surfaces (322a-322c, 422a-422c, 522a-522c) are part of one and the same real object (320, 420, 520).

7. The method of claim 6, wherein the real object (520) is asymmetric.

8. One or more devices (605, 630, 640, 650, 900) for determining a transformation having a deskew effect on 3D image data resulting from a scan by a 3D imaging system (605), said one or more devices comprising: acquiring (701) 3D image data resulting from a scan by the 3D imaging system (605) of planar actual surfaces (322a-322c, 422a-422c, 522a-522c) that are non-parallel at least during scanning and have one or more fixed angular relationships between each other, whereby the 3D image data includes planar imaged surfaces (327a-327c) that correspond to the planar actual surfaces (322, 422, 522); detecting (702) the planar imaged surface (327) in the 3D image data; determining (703) the transformation based on the one or more angular relationships between the detected imaged surfaces (327a-327c) of the planes and the actual surfaces (322a-322c, 422a-422c, 522a-522c) of the planes, such that when the transformation is applied to a description of the detected imaged surfaces (327a-327c) in the coordinates of the 3D data, the result is that the angular relationships between the detected imaged surfaces (327a-327c) match the angular relationships between the actual surfaces (322a-322c, 422a-422c, 522a-522c) of the planes; configured to: One or more devices (605, 630, 640, 650, 900).

9. One or more computer programs (1003) comprising instructions that, when executed by one or more processors, cause the one or more devices of claim 8 to perform the method of any one of claims 1 to 7.

10. 10. One or more carriers comprising one or more computer programs (1003) according to claim 9, wherein the one or more carriers are one or more of an electrical signal, an optical signal, a radio signal, or a computer-readable storage medium (1101).

Images