Determination of transformations with de-skew effects on 3D image data generated by 3D imaging system scans

Abstract

The invention relates to determination of transformations having a de-skew effect on 3D image data generated by a 3D imaging system scan. A method and apparatus (605; 606) for determining a transformation having a de-skew effect on 3D image data resulting from a scan by a 3D imaging system (605). 630; 630; 640: 650; 900) for a vehicle. Obtaining (701) a planar real surface (322a-c; 422a-c, 422a-c; 522a-c) resulting from a scan by the 3D imaging system (605), the planar real surfaces being non-parallel at least during the scan, having a constant one or more angular relationship between each other, whereby the 3D image data includes a 3D image data associated with the planar real surfaces (322; 422; 422; 522) corresponding to the planar imaging surface (327a-c). A planar imaging surface (327) is detected (702) in the 3D image data. On the basis of the detected planar imaging surface (327a-c) and the planar real surface (322a-c; 422a-c, 422a-c; 522a-c), such that when the transformation is applied to a description of the detected imaging surfaces (327a-c) in the coordinates of the 3D data, the transformation results in a relationship between the detected angular relationships between the imaging surfaces (327a-c) and the planar real surface (322a-c; 422a-c, 422a-c; 522a-c) between the first and second terminals (522a-c).

Description

Technical Field

[0001] The embodiments described herein relate to the determination of a transformation that has a deskewing effect on 3D image data generated by scanning through a 3D imaging system such as a machine vision system based on optical triangulation. Background Technology

[0002] Industrial vision cameras and systems used for factory and logistics automation can be based on capturing 3D images, such as 3D images of objects, in three-dimensional (3D) machine vision. A 3D image refers to an image that also contains "height" or "depth" information, rather than, or at least not only, information about pixels such as intensity and / or color in a two-dimensional (2D) form as in a conventional image.

[0003] Typically, each pixel in an image captured by a camera has a position in the image sensor coordinate system that corresponds to the location of the camera and image sensor in the real world, or more specifically, information about the light from a certain location in the real world sensed by the image sensing elements of the image sensor, and which image sensing elements correspond to the pixel. Typically, this light originates from the object being imaged, such as reflected light from the sensed object. Depending on the camera and system, what light is used, and how the illumination is provided, the sensed light may contain various information about the location where the light is reflected, such as about the location on the object being imaged. Therefore, the pixels that capture the image have a position in the image sensor coordinate system corresponding to their real-world location, such as their location on the object. The sensed light may also contain additional information about that location, such as information related to its intensity, color, reflectivity, etc.

[0004] Many 3D machine vision cameras or systems used for 3D imaging, such as imaging objects, or 3D imaging systems in general, are based on multiple 2D images captured sequentially by the camera's image sensor during the scanning of an object. Each such 2D image can contain 3D information about the object's 2D contours, so the sum of these 2D images can contain 3D information about the entire object, thereby forming a 3D image of the whole object. The 3D image can be represented by a "point cloud," where each point corresponds to a position on the object and is associated with 3D coordinates about that point. Each point can also be associated with further information about that point, such as color or other properties associated with the corresponding object point.

[0005] When a pixel has a 3D location instead of "just" a 2D location, it can be named a voxel.

[0006] Line scan image data is generated when the object to be imaged is scanned one line at a time by scanning the object as a plane of light projected onto the object and measuring the reflected light from the object.

[0007] A specific case of 3D imaging via scanning is 3D imaging based on optical triangulation, in which structured light, typically a light plane or "light sheet," is used to scan the object through and / or by means of this light plane. During the scanning process, light rays are projected onto the object at locations where the light sheet or light plane intersects with it. Lasers are generally preferred, but other light sources capable of providing structured light, such as light planes, can also be used, for example, light sources that provide focused light without excessive dispersion, such as light provided by lasers or light-emitting diodes (LEDs). Instead of a light plane corresponding to a "light sheet," a light plane corresponding to an illumination edge (i.e., a light edge) can be used, for example.

[0008] 3D machine vision systems are typically based on optical triangulation. In such systems, a light source illuminates the object with structured light, such as a specific light pattern, generally a light plane as described above. This type of 3D machine vision system or device can be called a system or device for 3D imaging based on optical or light plane triangulation, or simply laser triangulation when using lasers. The light rays projected onto the object are imaged by a camera; that is, the light reflected from the object is imaged. Along the light rays, 3D characteristics corresponding to the contours of the object with height information are captured by optical triangulation. By scanning the entire object in this way, corresponding to line scanning, and involving the movement of lines and / or the object, the 3D characteristics of the entire object corresponding to multiple 2D contours of the object can be captured, and based on this, a 3D image of the object can be formed as described above. To generate an image of the object's contours during scanning, the image sensor of a camera captures the reflected light from the object, specifically detecting its intensity peaks in the image data. The peaks appear at positions on the object corresponding to where the incident light of the light rays is reflected from the object. The location of the detected peak in the image will be mapped to the location of the light reflected from the object, based on the optical triangulation performed by the system and configured.

[0009] Scanning-based 3D imaging systems, such as those based on optical triangulation, typically introduce a skew effect. This occurs in optical triangulation systems that use an optical plane for scanning if the scanning direction, such as the object's direction of travel, is not orthogonal to the optical plane. Therefore, the scanned object and its 3D image—the imaged object—may appear skewed compared to the real object. For some system setups, such as certain mounting locations with a camera-equipped measurement unit, there is no or only a very small skew effect. However, in general, scanning an object causes or is likely to cause a skew effect, and this effect should be eliminated or at least reduced.

[0010] When a skew effect exists, some corrections should be made to address the skew effect, which corresponds to deskewing.

[0011] The skew effect corresponds to the non-orthogonal coordinate system of 3D image data resulting from scanning real-world objects. When 3D image data is visualized in a traditional orthogonal coordinate system, thereby making the scanned object visible, the imaged object appears skewed.

[0012] In other words, deskewing 3D image data corresponds to transforming a non-orthogonal coordinate system into an orthogonal coordinate system, or at least closer to an orthogonal coordinate system. To deskew 3D image data, a transformation can be determined and then applied to the 3D image data, corresponding to making the non-orthogonal coordinate system less non-orthogonal, preferably orthogonal. This transformation can be called deskewing or skew transformation. In principle, deskewing is achieved by creating a skew that is the opposite of the skew produced by the scanning.

[0013] The skew effect can be described by projections onto one or two axes that can be modeled using the angles of the light sheet relative to the direction of movement. These angles correspond to the degree to which one or two axes of the non-orthogonal coordinate system are skewed compared to their corresponding orthogonal axes, and therefore also to the degree to which they should be corrected to make said one or two axes orthogonal. Each angle can thus be considered to correspond to an angular offset relative to the corresponding orthogonal direction.

[0014] The deskew transformation can therefore be interpreted as a transformation that eliminates or reduces angular offset, thereby eliminating or reducing the skew effect of image data.

[0015] Furthermore, when the transformation is determined and thus known, other operations on the image data, such as multi-camera alignment, may become simpler, which may be desirable or sometimes even required.

[0016] To determine the transformation, traditionally, a predetermined object with point reference features is used and scanned. The object and the point reference features have known geometry and dimensions, and it is known how they relate to each other on the object and, consequently, in the real world. The imaged point reference features are then identified in the resulting image data. Based on how the imaged point features relate to each other and to their counterparts—that is, how point reference features on real objects in the real world relate—the transformation can be determined.

[0017] US2002290978A1 discloses a solution based on this method.

[0018] However, traditional methods based on point features of the target object generally have problems with the accurate capture of point features, the identification of point features in image data, and / or the estimation of point features. Therefore, transformations based on these deterministic transformations may not provide sufficient deskewing effects. Summary of the Invention

[0019] In view of the above, one object of the present invention is to provide one or more improvements or alternatives to the prior art, such as providing one or more improvements or alternatives to the determination of transformations that have a deskewing effect on 3D image data.

[0020] According to a first aspect of the embodiments herein, the objective is achieved by a method for determining a transformation that has a deskewing effect on 3D image data scanned by a 3D imaging system, such as a 3D imaging system based on optical triangulation. 3D image data generated by the 3D imaging system is obtained from planar real surfaces, which are not parallel at least during the scanning and have one or more constant angular relationships with each other. As a result, the 3D image data includes planar imaging surfaces corresponding to the planar real surfaces. Planar imaging surfaces are detected in the 3D image data. The transformation is determined based on the detected planar imaging surfaces and the one or more angular relationships between the planar real surfaces, such that when the transformation is applied to a description of the detected imaging surfaces in the coordinates of the 3D data, the transformation results in the angular relationships between the detected imaging surfaces matching the angular relationships between the planar real surfaces.

[0021] According to a second aspect of the embodiments herein, the objective is achieved by one or more devices for determining a transformation that has a deskewing effect on 3D image data scanned by a 3D imaging system, such as a 3D imaging system based on optical triangulation. The one or more devices are configured to obtain 3D image data scanned by the 3D imaging system of planar real surfaces, which are not parallel at least during the scanning and have one or more constant angular relationships with each other. As a result, the 3D image data includes planar imaging surfaces corresponding to the planar real surfaces. The one or more devices are also configured to detect planar imaging surfaces in the 3D image data. Furthermore, the one or more devices are configured to determine the transformation based on the one or more angular relationships between the detected planar imaging surfaces and the planar real surfaces, such that when the transformation is applied to a description of the detected imaging surfaces in the coordinates of the 3D data, the transformation results in the angular relationships between the detected imaging surfaces matching the angular relationships between the planar real surfaces.

[0022] According to a third aspect of the embodiments herein, the objective is achieved by one or more computer programs containing instructions that, when executed by one or more processors, cause one or more devices to perform the method according to the first aspect.

[0023] According to a fourth aspect of the embodiments herein, the objective is achieved by a carrier including a computer program according to the third aspect.

[0024] Because of the embodiments described herein, the use of multiple point reference features in 3D as in the conventional case can be avoided, and thus the related problems mentioned in the background art can also be avoided.

[0025] Furthermore, the embodiments described herein allow for positional and rotational invariance relative to the placement of this set of planar real surfaces. It is only necessary to know the fixed relative angular relationships between the planar real surfaces, which correspond to the constant one or more angular relationships between the planar real surfaces during scanning.

[0026] Furthermore, scale invariance can be achieved due to the embodiments described herein. Simple objects with planar, realistic surfaces can be used. Such objects can be provided at different scales for different sizes of field of view, and the same algorithm will work in the same way without any changes to the parameterization required when traditionally using point reference features.

[0027] Furthermore, the algorithms used in the embodiments described herein for surface detection and computation are simpler than those employed in the traditional case using point reference features. Plane estimation is generally simpler than the spherical or conical fitting typically used in the traditional case.

[0028] Furthermore, the embodiments described herein enable more robust estimations than conventional methods. For example, instead of, for instance, fitting a cone or similar object to estimate the cone and find point reference features corresponding to the cone's vertices, the embodiments described herein can use plane fitting to estimate a planar imaging surface. This can take into account more data corresponding to the points forming the corresponding surface, resulting in more robust results.

[0029] Therefore, the embodiments described herein provide improvements over the prior art. Attached Figure Description

[0030] Examples of embodiments in this document are described in more detail with reference to the accompanying schematic diagrams briefly described below.

[0031] Figure 1 illustrates an example of a prior art type of 3D imaging system based on optical triangulation.

[0032] Figures 2A-2D It is a schematic diagram used to visualize, illustrate, and explain the skew effect.

[0033] Figures 3A-3B These are schematic diagrams used to visualize, illustrate, and explain the principles and ideas behind the embodiments in this article.

[0034] Figures 4-5 These are schematic diagrams illustrating and visualizing two real objects that can be used to provide 3D image data for the embodiments described herein.

[0035] Figure 6 The schematic diagram illustrates examples of 3D imaging systems that can be used with and / or configured to perform the embodiments described herein.

[0036] Figure 7 This is a flowchart illustrating an embodiment of the method according to the embodiments described herein.

[0037] Figures 8A-8B Real-world examples of transformations not applied and not applied according to the embodiments described herein are shown.

[0038] Figure 9 It is used to illustrate how one or more devices can be configured to perform about Figure 7 A schematic block diagram illustrating an embodiment of the methods and actions discussed.

[0039] Figure 10 These are schematic diagrams illustrating some embodiments related to computer programs and their carriers. Detailed Implementation

[0040] The embodiments described herein are exemplary embodiments. It should be noted that these embodiments are not necessarily mutually exclusive. Components from one embodiment may be assumed to exist in another embodiment by default, and it will be apparent to those skilled in the art how these components can be used in other exemplary embodiments.

[0041] In order to enable a better understanding of the relevant technology and as a development toward the embodiments described herein, the prior art situation and problems pointed out above in the background section will be further explained before describing the embodiments herein.

[0042] Figure 1 schematically illustrates an example of an imaging system 105 for 3D machine vision based on optical triangulation, or simply 3D imaging, as mentioned in the background section. This system may be referred to as a machine vision system. The imaging system 105 is shown in the figure in a normal operating condition. The imaging system 105 is an example of a system that can be used and applied relative to the embodiments described below. System 105 is configured to perform optical triangulation, here using a light sheet, i.e., an optical plane 111. System 105 also includes a light source 110, such as a laser, for illuminating the object to be imaged with a specific light pattern (illustrated and illustrated in the figure as optical plane 111). The light can be a laser, but is not required to be one. In the example shown, the objects to be imaged are exemplified by a first measuring object 120 in the form of a car and a second measuring object 121 in the form of a gear structure. When the specific light pattern (here, optical plane 111) is incident on the object, this corresponds to the projection of optical plane 111 onto the object, which can be seen when optical plane 111 intersects the object. For example, in the illustrated example, the light plane 111 generates light ray 112 on the first measuring object 120. The light plane 111 incident on the object is reflected by the object, more specifically, by the various portions of the object at their intersections, i.e., at light ray 112 in the illustrated example. System 105 also includes a camera 130, which includes an image sensor (not shown in FIG. 1). The camera and image sensor are arranged relative to the light source 110 and the object to be imaged, such that the light plane 111 becomes incident light on the image sensor when reflected. The image sensor, typically implemented as a chip, is used to convert the incident light into image data. The portion of the object that results in the incident light on the image sensor through reflection is thus captured by the camera 130 and the image sensor, and corresponding image data can be generated and provided for further use. For example, in the illustrated example, at light ray 112 on a portion of the roof of the first measuring object 120, the light plane 111 will be reflected to the camera 130 and the image sensor, which can then generate and provide image data having information about that portion of the roof. By understanding the setup of system 105, including its geometry, such as how image sensor coordinates relate to real-world coordinates relevant to the imaged object and its environment, such as coordinates in coordinate system 123 (e.g., a Cartesian coordinate system), image data can be converted in a suitable format into information about the 3D characteristics of the imaged object, such as its 3D shape or contour. Information about these 3D characteristics, such as the 3D shape or contour, can include data describing the 3D characteristics in any suitable format.

[0043] By moving, for example, the light source 110 and / or the object to be imaged, such as the first measuring object 120 or the second object 121, so that multiple parts of the object are illuminated and cause reflected light on the image sensor (in practice, this is generally done by scanning the object), image data describing a more complete 3D shape of the object can be generated, such as image data corresponding to multiple consecutive contours of the object (e.g., contour images 141-1 to 141-N of the first measuring object 120 shown), wherein the corresponding contour images show the shape of the first object 120 at the location of the reflecting light plane 111 when the image sensor of the camera 130 senses the light that causes the corresponding contour image. As shown, a conveyor belt 122 or the like can be used to move the object through the light plane 111 while the light source 110 and camera unit 130 are generally stationary, or the light plane 111 and / or camera 130 can be moved above the object to illuminate all parts of the object, or at least all parts facing the light source 110, and the camera 130 can receive light reflected from the different parts of the object that need to be imaged.

[0044] As described above, the image frames provided by camera 130 and its image sensor, such as those provided by imaging the first measurement object 120, can generate any one of contour images 141-1 to 141-N. As described in the background art, each position of the shape of the first object shown in any one of the contour images 141-1 to 141-N is generally determined based on the identification of intensity peaks in the image data captured by the image sensor, and based on finding the positions of these intensity peaks, for example, by means of one or more intensity peak lookup algorithms. System 105 and conventional peak lookup algorithms are generally configured to search for intensity peaks column-by-column in each image frame. As shown, the sensor coordinates are u and v. The u-axis can be along an image sensor row, and u can be used to indicate a position in such a row, for example, corresponding to an image sensor column. Correspondingly, the v-axis can be along an image sensor column and is used to indicate a position in such a column, for example, corresponding to an image sensor row.

[0045] For each position u in the image frame, a peak position can be searched along v, for example, using the peak-finding algorithm mentioned above, and the peaks identified in the image frame can result in one of the contour images 141-1 to 141-N as shown in the figure. The contour image is formed by image points in the sensor based on coordinate system 143, such as u, v, and t. The sum of the image frame and the contour image can be used to create a 3D image of the first object 120.

[0046] Calibration can be used to more accurately determine how the coordinates of an image sensor relate to the real-world coordinates of the system, for example, after the system has been set up in its place of use. Generally, a specific object (usually a pre-determined object) is used for calibration; this is called the calibration object. The calibration object has certain known characteristics, measurements, etc., and is scanned by the system for calibration. Knowledge and information about the object, as well as knowledge and information from the scan of the calibration object, are used to calibrate the system so that it can subsequently provide more accurate 3D image data of the scanned object. In contrast to the calibration object, an object scanned as part of the system's normal, intended use and operation (such as imaging and measuring the object shown in the figure) can be referred to as a production object or a measurement object. The deskipation discussed in the background art can be part of the usual calibration process or can be separate from calibration performed for reasons other than (de)skipation.

[0047] For example, when the scanning direction s is not orthogonal to the light plane 111, i.e., if the scanning direction s deviates from the -y direction in the figure, where the light plane 111 lies in the zx plane, the skewness discussed in the background art occurs. This deviation may be intentional, for example, if the system and / or the system setup is performed in a manner that brings some benefit (e.g., extracting certain information about the object), and this requires or benefits from the scanning direction not being orthogonal to the light plane 111. Alternatively or additionally, there may be some deviations caused by imperfections or inaccuracies in the system setup, resulting in some non-negligible deviations relative to what should be, such as deviations relative to orthogonality or deviations relative to the state required by early calibration regarding skewness. In both cases, it is desirable to be able to eliminate or reduce the skewness effect, for example, by using a transformation on the generated 3D image data that has a deskewness effect, as mentioned in the background art. This transformation is applied after peak detection and is typically used on 3D image data corresponding to several contours of a “point cloud,” height map, or depth map. This 3D image data may be the result of scanning the entire object or available some time after the system has provided image data of multiple contours.

[0048] It should be recognized that the scanning direction can be in any direction s parallel to the xy plane and not orthogonal to the x-axis and / or in any direction s parallel to the zy plane and not orthogonal to the x-axis, and not orthogonal to the light plane 111. As can be understood from the figure, this can be described as a sheared coordinate system being sheared relative to the shown x, y, z coordinate system, such that the new coordinate system, such as x', y', z', will have a y' direction along the scanning direction s, and x' and / or z' will be not orthogonal to y'. The corresponding non-orthogonal axes of the sheared coordinate system x', y', z' can be considered as being "offset" by the corresponding angle relative to orthogonality. The skew effect is caused by the sampling during scanning being performed in this sheared coordinate system. Deskewing corresponds to transforming the (skewed) 3D data obtained from the scan (which is therefore in the sheared coordinate system) into an orthogonal coordinate system. It should be recognized that the one or two angles mentioned can be used for this purpose, and thus can be used to determine the appropriate deskewing transformation. However, these angles are often unknown.

[0049] Figures 2A-2D It is a schematic diagram used to visualize, illustrate, and explain the skew effect.

[0050] Figures 2A-2B The figures show the cube object 220 under both skew-free and skew-effect conditions. It is assumed here that the light plane used in the scanning is in the rx plane or the r′-x plane, and that object 220 is scanned in the y direction. In the corresponding figures, parallel thin dashed squares schematically indicate how the light plane will "sample" the contour of cube 220 during scanning; that is, this illustrates how the contour of cube 220 will be captured. Figure 2A In this context, there exists a light plane 211a orthogonal to the scanning direction (y) (in the rx plane), therefore there is no skew effect. On the other hand, in Figure 2B In the process, there exists a light plane 211b that is not orthogonal to the scanning direction (y) (in the r'-x plane), thus causing a skew effect. More specifically, in Figure 2B In this context, the light plane (in the r'-x plane) is tilted relative to the scanning direction (y), corresponding to what will make... Figure 2A The light plane orthogonal to the scanning direction (y) (in the rx plane) is rotated about the x-axis by an angle α, thus producing r' instead of r. The coordinate system x, y, r' therefore corresponds to the shear coordinate system as described above.

[0051] The skew effect can be understood by the fine dashed line indication of how the light plane 211b will “sample” the contour of the cube 220 during scanning, compared to the light plane 211a.

[0052] Figures 2C-2D The diagrams illustrate the situation if, according to... Figures 2A-2BThe settings are used to image one of the same objects instead of cube 220, resulting in a 3D image of that object. Figure 2C An imaging object 225-1 is shown, which is similar to an actual real object “sampled” by the light plane 211a in the scanning direction y, replacing object 220. There is no visible skew effect for the imaging object 225-1. On the other hand, Figure 2D An imaged object 225-2 with a clearly visible skew effect is shown. This imaged object is the result of an actual object “sampled” by a light plane 211a in the scanning direction y, replacing object 220. The actual objects scanned to produce imaged objects 225-1 and 225-2 are examples of objects with point reference features mentioned in the background art, which can be used for calibration, and, as in the prior art, the calibration-derived information is used to determine transformations to eliminate or reduce skew effects in 3D image data.

[0053] On the other hand, the embodiments in this paper are based on the insight that instead of using point reference features, planar surfaces of a real object with known angular relationships between surfaces can be used. In the case of skew, the planar surfaces will also be imaged as planar imaging surfaces. After scanning the real object, planar imaging surfaces are detected in the generated 3D image data, instead of point reference features as in the conventional case. This allows the angular relationships between image surfaces to be found, for example, by fitting a plane to the corresponding detected surface and calculating the angular relationship based on this. The detected imaging surfaces are preferably described separately by their normals, which makes it easy to describe each surface with very little data and also simplifies the computation. The deskew transformation is then determined (e.g., estimated). This can be done in a manner similar to the prior art, for example by applying and testing an adjustable transformation in an iterative manner, such as by using a nonlinear solver, until a transformation that provides sufficient deskew effect according to the test is found. However, in the prior art, this is done based on and using point reference features, real point reference features on the real object, and imaging point reference features affected by the transformation. Instead, the embodiments in this document are based on observing how the adjusted transformation affects the angular relationships between planar imaging surfaces when applied, and in order to achieve deskewness, these angular relationships should be the same as or at least close to the corresponding angular relationships of the real object. That is, the angular relationships between the transformed imaging surfaces are compared with the corresponding known angular relationships between the real planar surfaces of the real object.

[0054] Preferably, a real object with at least three non-parallel planar surfaces is used, wherein the three surfaces have the aforementioned angular relationship. This is because, as mentioned above, three planar surfaces can provide sufficient information about the skew in any skew direction that occurs. Therefore, using three surfaces provides more generally useful information. In some cases where skew exists, two surfaces may be sufficient, but two surfaces can only provide information about the skew in one dimension, which is insufficient for some situations where the skew effect may exist simultaneously in two different dimensions.

[0055] Figures 3A-3B These are schematic diagrams used to visualize, illustrate, and explain the principles and ideas behind the embodiments in this article. Figure 3A A real object 320, assuming the form of a cube, is schematically shown. This real object 320 will be scanned by a 3D imaging system in a configuration that produces a skew effect. The real object 320 is a known (e.g., predetermined) object with planar surfaces 322a-c and known angular relationships between these surfaces, which, in this example, are 90 degrees, are cubes. Each planar surface 322a;b;c can be described and represented by its normals 324a;b;c, for example, by corresponding vectors. For example, two surfaces at 90 degrees relative to each other mean that the scalar product between the normals of these two surfaces is zero. Similarly, in the case of planar surfaces with non-zero angular relationships, the angular relationships correspond to how the normals are related to each other. Figure 3B An image of an object 325 with the aforementioned skew is schematically illustrated based on 3D image data from a scan of a real object 320. As can be seen from the example, the result is that the imaged object 325 corresponds to a cube, i.e., a skewed version of the real object 320. Nevertheless, there are still imaged plane surfaces 327a;b;c corresponding to the real planar surfaces 322a;b;c of the real object 320. These can be detected in the 3D image data corresponding to the imaged object 325. Planes can be fitted to the target planar surfaces 327a-c, and their normals 329a-c can be determined. Then, for example using the normals 329a-c, the angular relationships between the real planar surfaces 322a-c of the real object 320 can be compared with the corresponding angular relationships between the corresponding planar imaged surfaces 327a-c. As described above, an adjustable transformation can be applied and adjusted until the angular relationship of the imaging surfaces 327a-c preferably becomes the same as, or at least preferably close to or closer to, the corresponding angular relationship of the planar surfaces 322a-c of the real object 310. This means that the transformation will have a deskewing effect, even if it is later applied to other imaging objects scanned through the same imaging system and with the same setup.

[0056] While the simple cube in the example above is sufficient, this principle certainly applies to other real-world objects with planar surfaces, some of which may be more advantageous in practice and under certain circumstances. As mentioned above, it is preferable to use a real-world object with at least three non-parallel planar surfaces, as at least two angles may be needed to describe skew in 3D, as is the case with 3D image data. Generally, for N angles, at least N+1 surfaces are required.

[0057] Figures 4-5 These are schematic diagrams illustrating and visualizing two other real objects that can be used to provide 3D image data for the embodiments described herein.

[0058] Figure 4 A real object 420 is shown corresponding to one corner of a cube and having three planar real surfaces 422a-c. It is recognized that a corner of a cube contains three surfaces, sufficient to provide information about the angular relationships corresponding to the two skew angles. Furthermore, a corner of a cube occupies very little space, and its placement, compared to a complete cube, reduces the risk of the scan improperly sampling the cube's surfaces.

[0059] Figure 5 A real object 520 with three planar real surfaces 522a-c is shown, these planar real surfaces having an angular relationship of less than 90 degrees. This can be advantageous because in some imaging system settings, it can be difficult to obtain good sampling of three surfaces with a 90-degree angular relationship, as this requires that both light and the camera can simultaneously illuminate and observe each surface. If the angular relationship is less than 90 degrees, this may be easier. However, if the surfaces can be sampled well enough, a larger difference in the angular relationship between the surfaces (e.g., 90 degrees or close to 90 degrees) can be advantageous, while a difference that is too small or very small (e.g., close to 0 degrees) can be disadvantageous. Larger angular differences generally result in clearer distinctions between the surfaces and better numerical stability for solutions based on the embodiments described herein, which can facilitate more accurate determination of transformations.

[0060] Furthermore, the real object 420b is asymmetric, which is advantageous because it can be used to ensure that the imaging surface is not confused and can be used to detect mirror system settings. Correcting mirrors can be beneficial in certain applications such as reading text. This method ensures correct orientation and an orthogonal coordinate system.

[0061] Note that, in principle, it does not need to be a single real object with non-parallel planar surfaces; real surfaces with known angular relationships to each other are sufficient. For example, one can imagine scanning a support surface of the object to be scanned, such as a conveyor belt as described above, and adjusting it during the scan so that it is scanned while having two or more different angular relationships to capture several contour images of each. This can be achieved by tilting the support surface. The resulting 3D data will contain the imaging plane surfaces of the virtual imaged object formed by the scan, and also the imaging plane surfaces corresponding to the support surfaces with different tilts. For example, this would allow the system to "self-calibrate" for a certain setting, at least in terms of skew, and automatically eliminate or reduce skew effects. However, for simplicity and robustness, in most practical situations, using a single object may be better.

[0062] Figure 6 The schematic diagram illustrates an example of a 3D imaging system 605 that can be used and / or configured to perform the embodiments described herein. This system is used for 3D imaging of a scanned object, and more specifically, the illustrated system is based on optical triangulation. The imaging system 605, how contour images are captured from the scan, optical triangulation, peak detection, and how 3D image data corresponding to a 3D image of the scanned object can be as in the prior art; in this respect, the imaging system 605 can correspond to the imaging system 105 described above. Therefore, the imaging system 605 correspondingly includes a camera 630 having an image sensor 631 and a light source 610 for illuminating the object to be imaged with a specific light pattern (e.g., a light plane 611). The object to be imaged is illuminated by the light plane 611 within the field of view 625 of the camera 630. In the figures, such an object is schematically illustrated as a real object 620. For the embodiments described herein, the real object 620 may be, for example, any of the real objects 320, 420, 520 discussed above, or it may be any other one or more real objects having non-parallel planar real surfaces with one or more constant angular relationships between them during scanning and the provision of 3D image data.

[0063] In addition to the image sensor 631, the camera 630 may also include one or more processors, memory, etc., and / or the image sensor 631 itself may have some integrated processing capabilities. Of course, input and / or output interfaces are generally present, such as input and / or output for control and / or information of the camera 630, such as captured images and / or information extracted from captured images, 3D image data based on found peaks, etc. One or more processors can be used to integrate the same or similar capabilities as a computer in the camera, i.e., in the same unit, making it more autonomous and capable of more processing and / or self-control. That is, no further and / or separate units (such as a separate computer) are needed to obtain the corresponding functions and / or to achieve, for example, simple setup procedures and / or faster processing, because the communication requirements between separate units are reduced. In this case, the camera 630 can operate independently in the embodiments described herein. In some embodiments, system 605 corresponds to camera 630, i.e., system 605 can be in the form of a single unit, or can be implemented as a single unit that corresponds to or is at least similar to a camera and includes camera functions. Network capabilities, light sources, etc., can also be integrated into such a unit.

[0064] However, in some embodiments, camera 630 is a camera unit with only more conventional image-specific capabilities, such as some image processing capabilities related to, for example, individual images and peak detection, in addition to imaging involving image sensor 631. Camera 630 may be connected to one or more other parts of the system with further processing capabilities, such as computing device 640 illustrated in the figures. Computing device 640 may be configured to control camera 630 and other parts of system 605 (if any), and / or be configured to perform the embodiments herein, for example, based on 3D image data generated from image data captured by the camera during scanning of real object 620, and input information based on angular relationships between planar real surfaces of real object 620. Computing device 640 may receive 3D image data corresponding to a contour image of real object 620 from camera 630. Computing device 640 may include one or more processors, memory, etc., and may correspond to a computer. To illustrate that the computational capabilities, etc., of the embodiments described herein can be alternatively or additionally performed far from the location where the scanning is performed in time and / or space, the figure also illustrates a schematic computer cloud 650 representing one or more servers or remote computers. These may be part of or separate from system 605, but are configured to operate on 3D image data from imaging system 605 or a similar system, and provide 3D image data using information about angular relationships between planar real surfaces scanned by the system. Thus, the embodiments described herein may, for example, be performed as a computer cloud service, and / or may be performed remotely on one or more servers communicatively connected to imaging system 605.

[0065] Figure 7 This is a flowchart illustrating an embodiment of a method according to some of the embodiments described herein. The following actions of the method can be formed to determine a transformation with a deskipation effect on 3D image data generated by a 3D imaging system that can be based on optical triangulation (such as 3D imaging system 605, which will be used as an example of a 3D imaging system hereinafter).

[0066] Below and Figure 7 The methods and / or actions indicated herein may be performed by one or more devices (such as a 3D imaging system 605, or one or more devices belonging to the system or one or more devices outside the system, such as a camera 630 and / or a computing device 640 and / or a computer cloud 650). The one or more devices used to perform the methods and actions will be described in slightly more detail below.

[0067] The following actions may be performed in any suitable order, and / or, where possible and appropriate, may be performed in a fully or partially overlapping manner in time.

[0068] Action 701

[0069] A 3D imaging system 505 obtains 3D image data generated by scanning planar real surfaces, which are at least completely or partially non-parallel during the scanning process and have a constant, or in other words, fixed, angular relationship with each other, thereby the 3D image data contains an imaging planar surface corresponding to the real planar surfaces. Examples of planar real surfaces are planar real surfaces 322a-c; 422a-c; 522a-c, of which planar real surface 322a-c will be primarily used as an example below. Examples of imaging surfaces are imaging surfaces 327a-c, which will be used as examples below. It should be understood that, in this context, "constant during scanning" means "remaining constant" so that the scan can adequately sample (or in other words, capture) enough contour images of each planar real surface to form its corresponding planar imaging surface. As mentioned above, when a skew effect is present, the scanned planar real surfaces will also produce planar imaging surfaces, but the angular relationships between these surfaces will be affected.

[0070] The planar real surfaces preferably maintain a fixed relationship with each other throughout the scanning process, for example, as part of the same object that does not change.

[0071] Alternatively, the planar real surfaces can be part of different objects, but maintain a fixed relationship with each other at least during the scan. For example, one surface may correspond to the supporting surface of the scanned object, while another one or more surfaces correspond to the surface of the object.

[0072] Note that, as discussed separately above, in some specific embodiments, the same physical surface can even be adjusted during scanning to achieve two or more planar real surfaces that are not parallel during scanning (in this case, during a portion of the scanning) and have a constant angular relationship with each other.

[0073] The planar real surfaces and their relationships with each other, and therefore one or more of their angular relationships, are preferably predetermined and thus known in advance during scanning. However, the real surfaces and their angular relationships may be determined, for example, after scanning, such as by measurement.

[0074] When using one or more specific objects with a planar real surface, it / they are preferably predetermined, have a predetermined real surface and angular relationship, and may be referred to as calibration objects. Note that such objects may be used solely for skew purposes, or may additionally include features related to other calibrations, or may exist as separate objects for other calibrations that can be performed entirely or partially separately from the embodiments described herein.

[0075] If the planar real surfaces are part of the same object and can be manufactured as part of the same object, then it is easier to achieve planar real surfaces with predetermined and / or precise angular relationships to each other. Therefore, in some embodiments, the planar real surfaces, such as planar real surfaces 322a-c, are part of the same real object (e.g., real object 320). In some of these embodiments, the real object, such as real object 520, is asymmetrical. Why this may be advantageous has also been discussed above.

[0076] Action 701a

[0077] In some embodiments, such as when the method is performed by a 3D imaging system 605, 3D image data is obtained by scanning a planar real surface using the 3D imaging system 605.

[0078] In other embodiments where the method is performed, for example by a single device (part of or outside the 3D imaging system 605), such a device can acquire 3D image data by receiving 3D image data from the 3D imaging system 605 (typically its camera 630).

[0079] Action 702

[0080] Detection of planar imaging surfaces 327a-c in 3D image data. Compared to imaging point reference features in the prior art solutions mentioned above, imaging surfaces can be captured by more samples, and thus can be described by more points in the 3D image data. Therefore, the detection of planar imaging surfaces in 3D image data may be more robust, simpler, and / or able to represent their true counterparts more accurately than in the case of point reference features.

[0081] The detected planar imaging surface is preferably described as a planar surface in the coordinate system of the 3D image data.

[0082] This action preferably involves fitting the corresponding plane to the corresponding imaging surface, which can be done using any known method for fitting, such as robust least squares. The fitted plane can therefore be an absolute plane, i.e., a perfect plane, and the fitting may involve minimizing the median or mean distance to the points forming the corresponding imaging surface. Thus, the fitting of the plane can be used to estimate the planar imaging surface and can be used both for detection and to enable a simple description of the corresponding imaging surface, such as by its normal, plane equation, or any other means that can describe the planar surface. The corresponding normal of the corresponding planar imaging surface can be simply described by a vector in the coordinate system of the 3D image data.

[0083] Note that, as an alternative to fitting a plane, the corresponding planar imaging surface can be detected more directly by finding multiple points belonging to the planar surface in the 3D image data, and then these points can be used as a representation of the thus detected planar surface and / or used to form a description of the planar surface.

[0084] Action 703

[0085] The transformation is determined based on the one or more angular relationships between the detected planar imaging surfaces 327a-c and the planar real surfaces 322a-c, such that when the transformation is applied to the description of the detected planar imaging surfaces 327a-c in the coordinates of the 3D data, the transformation causes the angular relationships between the detected imaging surfaces 327a-c to match the angular relationships between the planar real surfaces 322a-c. In other words, the transformation is determined so that when the transformation is applied to the 3D image data, the angular relationships between the planar imaging surfaces 327a-c will match the corresponding angular relationships between the planar real surfaces 322a-c that are imaged and produce the planar imaging surfaces 327a-c.

[0086] As stated above, the embodiments described herein are based on the understanding that one or more angular relationships between the detected planar imaging surface and the corresponding planar real surface contain all the information needed to determine the deskew transformation. Therefore, point reference features are not required as in the prior art. In the case of skew, the imaging surface will also be planar, and the transformation can be determined by examining and comparing the angular relationships between the imaging surface and the corresponding real surface.

[0087] The conditions required for the detected angular relationship between planar imaging surfaces 327a-c to match the angular relationship between planar real surfaces 322a-c can be determined according to one or more predetermined criteria. Alternatively or additionally, a match occurs when the angular relationship between planar imaging surfaces 322a-c becomes the same as and / or closer to and / or sufficiently close to the angular relationship between planar real surfaces 322a-c. The match should at least involve the angular relationship between planar imaging surfaces 322a-c obtained by applying the transformation becoming closer to the corresponding angular relationship between the planar real surfaces compared to when the transformation was not applied, thus providing a certain degree of de-skewness effect.

[0088] Angular relationships correspond to the angles between the involved planar surfaces and can be expressed using the angles between the involved planar surfaces, but do not need to be expressed as angles themselves. Other ways of expressing this relationship are also possible, for example, by means of the surface normals and how the normals relate to each other (which contain information about the angles between the surfaces). Of course, this applies to both real planar surfaces and detected planar imaging surfaces.

[0089] Determining a transformation based on a detected planar imaging surface can correspond to or include determining the transformation based on or using a description and / or representation of the detected planar imaging surface, such as, as described above, using a plane already fitted to the planar imaging surface and / or the normals of the detected planar imaging surface. When using normals, the matching of the current action between the angular relationship between the imaging surface and the real surface is generally based on the normals of these surfaces, such as matching the angles between normals rather than directly matching the angles between the surfaces, or matching how the normals are related to each other.

[0090] Determining the transformation may include testing one or more specific candidate transformations until a match is found.

[0091] Action 703a

[0092] In some embodiments, an adjustable transformation is obtained for operating on the input 3D image data coordinates to transform them into output 3D image data coordinates. The transformation is adjustable such that adjustments to the transformation result in variations in the deskewing effect provided by the transformation.

[0093] In practice, transformations can induce skew and can be adjusted to induce skew. When the skew induced by the transformation is opposite to, or in other words, cancels out, the skew present in the 3D image data, this skew corresponds to deskew. This has already been discussed above.

[0094] Action 703b

[0095] In an embodiment of action 703a, the transformation is iteratively applied to the detected planar imaging surfaces 327a-c described in the coordinate system of the 3D image data, and the transformation is adjusted until the match occurs.

[0096] Applying a transformation to a detected planar imaging surface is generally done to the description or representation of the detected surface as described above, such as applying it to their normals.

[0097] Actions 703a-b correspond to the basic and generally preferred numerical methods for determining the transformation, wherein, for example, suitable nonlinear solvers can be utilized and adapted for the purposes mentioned above.

[0098] In some embodiments, iteratively applying and adjusting the transformation as in actions 703a-b involves applying a nonlinear solver to a function of the difference between the corresponding angular relationship based on the real surface and the corresponding corresponding angular relationship after transformation between the imaging surface.

[0099] Obtaining the deskew parameters can be accomplished using a nonlinear least squares solver and appropriate 3D transformations. For example, the Ceres solver is such a solver, available as open-source software, for modeling and solving large, complex optimization problems. This optimization problem, and others of similar types, can be solved using an automatically differentiating cost function. For the application in this context, the Ceres solver can be used to formulate a cost function that takes the observed plane normals and directly minimizes the difference between the dot product of these observed normals and the expected dot product in the orthogonal real-world coordinate system.

[0100] Using nonlinear solvers is generally efficient and simple. However, other methods can be used, although they may require more computation, but this may not be a problem in practice if sufficient computing power is available. Essentially, a simple iterative guessing-based approach can be used. For example, when a guess leads to a reduction in error, i.e., exhibits some deskewing effect, further guesses can be made to get closer to that guess.

[0101] As described above, in some embodiments, for example, as part of the iterative numerical method according to actions 703a-b, the planar real surfaces 322a-c and the planar imaging surfaces 327a-c are respectively represented by their normals, and the corresponding angular relationships are represented by the corresponding relationships between the normals of the surfaces involved in the corresponding angular relationships. By using normals instead of, for example, plane equations or two or more vectors in the corresponding planes, simpler and more efficient calculations can be achieved.

[0102] An alternative to the iterative numerical method based on actions 703a-b could be to determine the transformation analytically.

[0103] Because of the embodiments described herein, such as those described above with respect to actions 701-703, the use of multiple point reference features in 3D as in the conventional case can be avoided, and thus the related problems mentioned in the background art can also be avoided.

[0104] Furthermore, the embodiments described herein allow for positional and rotational invariance relative to the placement of this set of planar real surfaces. It is only necessary to know the fixed relative angular relationships between the planar real surfaces, which correspond to the constant one or more angular relationships between the planar real surfaces during scanning.

[0105] Furthermore, scale invariance can be achieved due to the embodiments described herein. Simple objects with planar, realistic surfaces, such as those illustrated above, can be used. Such objects can be provided at different scales for different field-of-view sizes, and the same algorithm will work in the same way without any changes to the parameterization required when traditionally utilizing point reference features.

[0106] Furthermore, the algorithms used in the embodiments described herein for surface detection and computation are simpler than those employed in the traditional case using point reference features. Plane estimation is generally simpler than the spherical or conical fitting typically used in the traditional case employing point reference features.

[0107] Furthermore, the embodiments described herein enable more robust estimations than conventional methods. For example, refer to... Figure 2D For skewed objects, the embodiments described herein can use plane fitting to estimate a planar imaging surface, compared to fitting a cone and thus estimating the cone and finding a point reference feature corresponding to the cone's vertex. This can take into account more data corresponding to the points forming the corresponding surface, thus producing more robust results.

[0108] Therefore, the embodiments described herein provide improvements over the prior art.

[0109] Figures 8A-8B Real-world examples of transformations not applied and not applied according to the embodiments described herein are shown. Figure 8A The first imaged object 835a produced from a scan of a real object with a skew effect is shown, therefore the embodiments described herein are not applied. Figure 8B The diagram illustrates a second imaging object 835b obtained when the 3D image data corresponding to the first imaging object 835a is transformed using the transformations determined according to the embodiments herein. Figure 8A and Figure 8B The comparison shows that this transformation has a deskewing effect on 3D image data. The second imaging object 835b corresponds to the deskewed version of the first imaging object 835a.

[0110] The real object that is scanned and thus visualized here as the first imaging object 835a is scanned by the system corresponding to system 605 in the manner described above, and therefore will have a skew effect, such as... Figure 2B The real objects in the image were scanned in 220. However, Figures 8A-8B The real object behind the content shown is for obtaining Figure 2D The illustration shows the same real-world object being scanned, because the skew effect is clearly visible for such objects. As mentioned above, this object has point reference features traditionally used to determine the deskew transformation, but note that... Figures 8A-8BThe actual objects used are only for visualizing skew and deskew with and without the embodiments described herein, and not for determining any skew transformation. Figures 8A-8B The deskewing transformation shown is determined according to the embodiments described herein, and is therefore determined by using, for example, a real object corresponding to real object 420. This real object is obtained by subsequently scanning another real object, thereby obtaining the transformation from... Figure 8A The same system and settings are used to visualize the 3D image data of the 835a imaging object.

[0111] Figure 9 This is a schematic block diagram illustrating an embodiment of one or more devices 900, which may correspond to the embodiments mentioned above for carrying out the embodiments described herein, such as those mentioned above regarding... Figure 7 The device used to describe the method and / or action.

[0112] One or more devices 900 may correspond, for example, to a 3D imaging system 605, or any of one or more devices belonging to that system or one or more devices outside of that system, such as a camera 630 and / or a computing device 640 and / or a computer cloud 650. As those skilled in the art will recognize, some embodiments of the method include actions that can be distributed among multiple devices configured to perform the actions. It should also be recognized that one or more devices 900 performing the method according to some embodiments may perform the method at a later time than when any imaging system is operating and 3D image data is being provided. Thus, the method can be performed by one or more devices that are temporally and spatially distant from the imaging system and the imaging itself. Output from the imaging system may, for example, be uploaded to a server or computer cloud (e.g., computer cloud 650) containing one or more computing devices configured to perform the method, or the one or more computing devices performing the method may obtain image data from the server or computer cloud. However, for one or more devices configured to perform the method, it may be preferred that the one or more devices be part of, or at least combined with, the imaging system and imaging involved in performing the method, as this allows for more reliable, simpler, and faster execution, and the transformations determined by the method will subsequently still be used for subsequent 3D image data generated by scanning with the same imaging system.

[0113] This schematic diagram illustrates how one or more devices 900 can be configured to perform the functions mentioned above. Figure 7 Examples of methods and actions discussed. Therefore, one or more devices 900 are used to determine a transformation that has a deskewing effect on 3D image data generated by scanning from a 3D imaging system, as described above with respect to the method.

[0114] One or more devices 900 may include one or more processing modules 901, such as processing devices, one or more hardware modules, such as one or more processing circuits, circuit systems, such as processors, and / or one or more software modules for performing the methods and / or actions.

[0115] One or more devices 900 may also include one or more memories 902, which may include (e.g., contain or store) one or more computer programs 903. The one or more computer programs 903 include instructions or code, each executable directly or indirectly by one or more devices 900, to perform the methods and / or actions. The one or more memories 902 may include one or more storage units and may also be arranged to store data, such as configurations, data, and / or values, that participate in or are used to perform the functions and actions of the embodiments herein.

[0116] Furthermore, the corresponding device 900 may include processing circuitry 904 involved in processing and, for example, encoding data, as one or more exemplary hardware modules, and may include or correspond to one or more processors or processing circuitry. One or more processing modules 901 may include such processing circuitry 904, for example, 'embodied in the form of such processing circuitry 904' or 'implemented by such processing circuitry 904'. In these embodiments, memory 902 may include one or more computer programs 903 executable by processing circuitry 904, thereby enabling the corresponding device 900 to operate or be configured to perform the methods and / or their actions.

[0117] Typically, one or more devices 900, such as one or more processing modules 901, include one or more input / output (I / O) modules 905. These I / O modules 905 are configured to participate in (e.g., by performing) any communication to and from other units and / or devices, such as sending information to and / or receiving information from other devices. Where applicable, one or more I / O modules 905 may be exemplified as one or more acquiring (e.g., receiving) modules and / or one or more providing (e.g., sending) modules.

[0118] Furthermore, in some embodiments, one or more devices 900, such as one or more processing modules 901, including one or more acquisition modules, one or more detection modules, one or more scanning modules, one or more application modules, one or more transformation modules, one or more adjustment modules, one or more determination modules, and one or more scanning modules, serve as one or more exemplary hardware and / or software modules for performing the actions of the embodiments herein. These modules may be implemented in whole or in part by the processing circuitry 904.

[0119] therefore:

[0120] One or more devices 900 and / or one or more processing modules 901 and / or one or more processing circuits 904 and / or one or more I / O modules 905 and / or one or more acquisition modules are operable or configured to acquire the 3D image data generated by the 3D imaging system scanning the true surface of the plane.

[0121] One or more devices 900 and / or one or more processing modules 901 and / or one or more processing circuits 904 and / or one or more I / O modules 905 and / or one or more detection modules are operable or configured to detect the planar imaging surface in 3D image data.

[0122] One or more devices 900 and / or one or more processing modules 901 and / or one or more processing circuits 904 and / or one or more I / O modules 905 and / or one or more determining modules are operable or configured to determine a transformation based on the one or more angular relationships between the detected planar imaging surface and the planar real surface.

[0123] In some embodiments, one or more devices 900 and / or one or more processing modules 901 and / or one or more processing circuits 904 and / or one or more I / O modules 905 and / or one or more acquisition modules and / or one or more application modules are operable or configured to obtain the adjustable transformation, iteratively apply the transformation to the detected planar imaging surface described in the coordinate system of the 3D image data, and adjust the transformation until the match occurs.

[0124] In some embodiments, one or more devices 900 and / or one or more processing modules 901 and / or one or more processing circuits 904 and / or one or more I / O modules 905 and / or one or more acquisition modules and / or one or more scanning modules are operable or configured to scan the planar real surface by a 3D imaging system.

[0125] Figure 10 These are schematic diagrams illustrating some embodiments relating to one or more computer programs 903 and their carriers that cause the one or more devices 900 discussed above to perform the methods and actions.

[0126] One or more computer programs 903 include instructions that, when executed by processing circuitry 904 and / or one or more processing modules 901, cause one or more devices 900 to operate as described above. In some embodiments, one or more carriers including one or more computer programs are provided, or more specifically, one or more data carriers, such as one or more computer program products. The corresponding one or more carriers may be one of electronic signals, optical signals, radio signals, and computer-readable storage media (e.g., computer-readable storage media 1001 illustrated in the figure). One or more computer programs 903 may therefore be stored on computer-readable storage media 1001. The carrier may exclude transient propagation signals, and the data carrier may be correspondingly named a non-transitory data carrier. Non-limiting examples of data carriers as computer-readable storage media are memory cards or memory sticks, disk storage media, or generally mass storage devices based on one or more hard disk drives or one or more solid-state drives (SSDs). Computer-readable storage media 1001 may be used to store data accessible via computer network 1002, such as the Internet or a local area network (LAN). One or more computer programs 903 may also be provided as one or more pure computer programs or contained in one or more files. The one or more files may be stored on computer-readable storage medium 1001 and may be obtained, for example, via a server, through download on a computer network 1002 as shown in the figure. The server may be a web server or a file transfer protocol (FTP) based server, or a similar server. The one or more files may be executable files, to be downloaded directly or indirectly to and executed on the one or more devices to enable the one or more devices to function as described above, for example by execution by processing circuitry 904. The one or more files may also be used, or alternatively, for intermediate download and compilation involving the same or other processors to make them executable prior to further download and execution, thereby enabling the one or more devices 900 to function as described above.

[0127] Note that any of the processing modules and circuits mentioned above can be implemented as software and / or hardware modules, such as in existing hardware, and / or implemented as application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), etc. Also note that any of the hardware modules and / or circuits mentioned above can be contained within a single ASIC or FPGA, or distributed across several independent hardware components, whether individually packaged or assembled into a system-on-a-chip (SoC).

[0128] Those skilled in the art will also recognize that the modules and circuits discussed herein may refer to hardware modules, software modules, analog and digital circuits and / or combinations of one or more processors configured with, for example, software and / or firmware stored in memory, which, when executed by the one or more processors, can configure one or more devices, one or more sensors, etc., to perform the methods and actions described above, and / or cause one or more devices, one or more sensors, etc., to perform the methods and actions described above.

[0129] Identification by any identifier here can be implicit or explicit. In a certain context, such as for a particular computer program or program provider, the identification can be unique.

[0130] As used herein, the term "memory" can refer to data storage used to store digital information, typically a hard disk, magnetic storage device, medium, portable computer floppy disk or hard disk, flash memory, random access memory (RAM), etc. Alternatively, memory can refer to the processor's internal register memory.

[0131] Furthermore, it should be noted that any enumerative terms (such as first device, second device, first surface, second surface, etc.) should be considered non-restrictive, and such terms do not imply any hierarchical relationship. In the absence of any explicit information to the contrary, naming by enumeration should only be considered as a way of achieving different names.

[0132] As used herein, the expression “configured to” can mean that the processing circuitry is configured, or adapted, to perform one or more of the actions described herein, through software or hardware configuration.

[0133] As used herein, the term “number” or “value” can refer to any kind of number, such as a binary number, a real number, an imaginary number, or a rational number. Furthermore, a “number” or “value” can be one or more characters, such as a single letter or a string of letters. Additionally, a “number” or “value” can be represented by a bit string.

[0134] As used herein, the expressions “may” and “in some embodiments” have been generally used to indicate that the described features may be combined with any other embodiments disclosed herein.

[0135] In the accompanying drawings, features that may only exist in some embodiments are typically drawn using dotted lines or dashed lines.

[0136] When the word “includes” or “contains” is used, it will be interpreted as non-restrictive, meaning “consisting of at least…”.

[0137] The embodiments described herein are not limited to those described above. Various alternatives, variations, and equivalents may be used. Therefore, the above embodiments should not be construed as limiting the scope of this disclosure as defined by the appended claims.

Claims

1. A method for determining a transformation having a deskewing effect on 3D image data generated by scanning from a 3D imaging system (605), wherein the method comprises: - Obtain (701) planar true surface (322a-c); 422a-c; 3D image data generated by scanning by a 3D imaging system (605) of 522a-c), wherein the planar real surfaces are not parallel at least during the scanning and have a constant angular relationship with each other, thereby the 3D image data includes planar imaging surfaces (327a-c) corresponding to the planar real surfaces (322; 422; 522); - Detect (702) planar imaging surfaces (327) in 3D image data; - The transformation (703) is determined based on the one or more angular relationships between the detected planar imaging surfaces (327a-c) and the planar real surfaces (322a-c; 422a-c; 522a-c), such that when the transformation is applied to the description of the detected imaging surfaces (327a-c) in the coordinates of the 3D data, the transformation causes the angular relationships between the detected imaging surfaces (327a-c) to match the angular relationships between the planar real surfaces (322a-c; 422a-c; 522a-c).

2. The method according to claim 1, wherein the determining action includes: - Obtain (703a) an adjustable transform for manipulating input 3D image data coordinates to transform them into output 3D image data coordinates, wherein adjusting the transform results in a change in the deskewing effect provided by the transform, and - Iteratively (703b) the transformation is applied to the detected planar imaging surface (327a-c) described in the coordinate system of the 3D image data, and the transformation is adjusted until the match occurs.

3. The method according to claim 2, wherein iteratively applying and adjusting the transformation involves applying a nonlinear solver to a function of the difference between the corresponding angular relationship based on the real surface and the corresponding corresponding angular relationship after transformation between the imaging surface.

4. The method according to any one of claims 1-3, wherein the planar real surface (322a-c) and the planar imaging surface (327a-c) are respectively represented by their normals, and the corresponding angular relationship is represented by the corresponding relationship between the normals of the surfaces involved in the corresponding angular relationship.

5. The method according to any one of claims 1-4, wherein the obtaining action comprises: - The planar real surface (322a-c) is scanned (701a) by a 3D imaging system (605).

6. The method according to any one of claims 1-5, wherein the planar real surface (322a-c; 422a-c; 522a-c) is part of the same real object (320; 420; 520).

7. The method according to claim 6, wherein the real object (520) is asymmetrical.

8. One or more devices (605; 630; 640; 650; 900) for determining a transformation having a deskewing effect on 3D image data generated by scanning from a 3D imaging system (605), wherein said one or more devices are configured to: 3D image data generated by scanning (701) planar real surfaces (322a-c; 422a-c; 522a-c) by a 3D imaging system (605) is obtained, wherein the planar real surfaces are not parallel at least during the scanning and have one or more constant angular relationships with each other, thereby the 3D image data includes planar imaging surfaces (327a-c) corresponding to the planar real surfaces (322; 422; 522); Detect (702) planar imaging surfaces (327) in 3D image data; and The transformation (703) is determined based on the one or more angular relationships between the detected planar imaging surfaces (327a-c) and the planar real surfaces (322a-c; 422a-c; 522a-c), such that when the transformation is applied to the description of the detected imaging surfaces (327a-c) in the coordinates of the 3D data, the transformation causes the angular relationships between the detected imaging surfaces (327a-c) to match the angular relationships between the planar real surfaces (322a-c; 422a-c; 522a-c).

9. One or more computer programs (1003) containing instructions that, when executed by one or more processors, cause one or more devices according to claim 8 to perform the method according to any one of claims 1-7.

10. One or more carriers comprising one or more computer programs (1003) according to claim 9, wherein the one or more carriers are one or more of the following: electronic signals, optical signals, radio signals, or computer-readable storage media (1101).

Images