Arrangement and method for providing information indicating intensity peak width for use in image sensors for 3D imaging systems based on optical triangulation.

Abstract

In a 3D imaging system based on optical triangulation, this provides indicators such as measured and / or estimated values ​​of intensity peak width in image data from an image sensor. [Solution] An arrangement and method performed by one or more devices for providing information indicating the intensity peak width of an intensity peak along a pixel line of a pixel containing an intensity peak. The intensity peak is formed from such reflected light sensed by an image sensor. A first value is calculated, including the magnitude of a lower-order Discrete Fourier Transform (DFT) coefficient for the pixel intensity value of the pixel along the pixel line containing the intensity peak (601). A second value is calculated, including the magnitude of another higher-order DFT coefficient for the pixel intensity value of the pixel along the pixel line containing the intensity peak (602). Information indicating the intensity peak width is provided based on the first and second values ​​(603).

Description

Technical Field

[0001] Embodiments of the present specification relate to an arrangement and method for providing information indicating an intensity peak width for use in an image sensor for a 3D imaging system based on optical triangulation.

Background Art

[0002] Industrial vision cameras and systems for factory and logistics automation may be based on 3D machine vision in which a three-dimensional (3D) image, such as an object, is captured. By a 3D image, it is meant an image that includes information not only about intensities and / or colors of pixels that are only two-dimensional (2D) like a normal image, or at least not only such information, but also information about "height" or "depth".

[0003] Generally, each pixel in an image captured by a camera has a position in image sensor coordinates corresponding to the position of what the camera and image sensor are capturing in the real world, or more specifically, information about the light from the real-world position sensed by the image sensing elements of the image sensor, and which image sensing element corresponds to the pixel. Typically, what is sensed is reflected light from what is being captured, for example, from an object. Depending on the camera and system, what kind of light is used, and how the illumination is provided, the sensed light may contain various information about the position from which the light was reflected, such as the position on the object being captured. Thus, each pixel in the captured image has a position in image sensor coordinates corresponding to a real-world position, such as the position on the object. The sensed light may also contain additional information about the position and the properties of the object at that position, such as information from the light's intensity, color, reflectivity, and scattering. Many 3D machine vision cameras or systems, or 3D imaging systems in general, for 3D imaging are based on the fact that multiple 2D images are captured, usually sequentially, by the camera's image sensor during the scanning of an object. Each such 2D image may contain 3D information about the object's 2D profile, and therefore the entirety of such 2D images may contain 3D information about the entire object, from which a 3D image of the entire object may be formed. The 3D image may be represented by a "point cloud" where each point corresponds to a position on the object and is associated with a 3D coordinate relative to that point. Each point may also be associated with further information about the point, such as color or other features associated with the point on the corresponding object.

[0004] When a pixel has a 3D position, not just a 2D position, that pixel is sometimes called a voxel.

[0005] Line scan image data is generally obtained when an object is scanned using a light plane onto which lines of light are projected, and the reflected light from the object is measured, resulting in image data of the image being scanned or provided line by line at a time. A special case of 3D imaging by scanning is 3D imaging based on optical triangulation, where structured light, e.g., a specific pattern, typically a light plane or "sheet of light," is used, and the object is scanned through and / or by this light plane. Lines of light are projected onto the object during the scan, corresponding to the locations where the sheet or plane intersects the object. Lasers are often preferred, but other light sources that can provide structured light, such as a light plane, may also be used, e.g., light sources that can provide light that remains focused and does not spread out too much, such as light-emitting diodes (LEDs). Instead of a light plane corresponding to a "sheet of light," a light plane corresponding to the edge of illumination, i.e., a light edge, may be used.

[0006] 3D machine vision systems are often based on optical triangulation. In such systems, there is a light source that illuminates an object with a specific pattern of light, typically structured light such as the optical plane described above. This type of 3D machine vision system or device is sometimes called a system or device for 3D imaging based on optical or optical plane triangulation, or simply laser triangulation when laser light is used. A line of light projected onto an object is imaged by a camera, i.e., the light reflected from the object is imaged. Along the line of light, 3D features are captured by optical triangulation, corresponding to a profile of the object with height information. By scanning the entire object in this way, corresponding to line scanning and including the movement of the line and / or the object, it is possible to capture 3D features of the entire object, corresponding to multiple 2D profiles of the object, on which a 3D image of the object can be formed as discussed above. To generate a profile image of the object during scanning, reflected light from the object is captured by the camera's image sensor, and in particular, peaks in the intensity of the reflected light are detected in the image data. The peak occurs at a position corresponding to the location on the object where the incident light corresponding to the light line is reflected from the object. The position of the detected peak in the image is mapped to the location on the object where the light that produced the peak was reflected, according to the optical triangulation configured and set up by the system.

[0007] Peak detection algorithms typically operate in the digital domain, and their goal is to find the central location of the peak light distribution at subpixel resolution. This means that an image with intensity peaks containing analog pixel values ​​corresponding to the light sensed by the sensor elements corresponding to the pixels of the image sensor must first be read from the image sensor and then converted from analog to digital (A / D).

[0008] As can be understood from the above, intensity peaks contain information of interest. However, this is not simply because the location of the peak can be found at sub-pixel resolution, as discussed above. In applications of optical triangulation, intensity peak width also conveys information about the surface that reflected the light that produced the intensity peak. More specifically, when intensity peaks in optical triangulation are used to determine the 3D position of a point on the surface of an imaged object, intensity peak width contains information about how this point influenced the light that produced the intensity peak, for example, how the point scattered the light. Thus, intensity peak width is, for example, a measure of light scattering of interest and / or is used in many applications of optical triangulation.

[0009] Therefore, it is interesting that indicators such as measured and / or estimated intensity peak widths can be obtained, for example, since it is a measure of light scattering. Furthermore, it is interesting that such indicators can be provided in a robust and effective manner. Note below that measured intensity peak widths of imaged light lines and measured light scattering may be used interchangeably.

[0010] An example of a conventional method for estimating intensity peak width is to sum the number of pixels on either side of the peak intensity, at a specific distance from the maximum peak intensity, for each column of an image capturing the intensity lines obtained as a result of optical triangulation. Such a method is sometimes called "offset scatter." This method is sensitive to noise contained in the light lines, such as speckle noise when the light is laser. Tuning this method to obtain "good" data can be difficult, including setting the distance and the number of pixels to sum. Also, the method relies on the availability of the pixels to sum within the image dataset surrounding the maximum intensity.

[0011] On the other hand, in EP1985969, scattering data indicating the intensity peak width is estimated by using zero- and second-order moments. This method is sometimes called "moment scatter." The method is useful for 3D data, especially when the location of intensity peaks at sub-pixel resolution is calculated using a centroid algorithm. In this case, the zero-order moment is already available as it has been calculated for use in 3D data calculations. Only the second-order moment is calculated to estimate the scattering data, and then it needs to be used together with the zero-order moment. The method is not very effective when centroid calculations are not used.

[0012] Furthermore, it is desirable to eliminate or reduce latency in general, thereby facilitating or enabling higher throughput and / or lower latency in 3D imaging systems, providing improvements in how quickly, for example, a 3D imaging system based on optical triangulation can provide a 3D image of an object along with relevant information, such as intensity peak width estimates and scattering data, and / or supporting or better supporting high-speed or faster applications. [Overview of the project] [Problems that the invention aims to solve]

[0013] In view of the foregoing, an object of the present invention is to provide one or more improvements or alternatives to the prior art, particularly to provide further alternatives and / or more effective and / or robust methods for providing indications such as measured and / or estimated values ​​of intensity peak widths with respect to intensity peak lines in image data from an image sensor of a camera used in a 3D imaging system based on optical triangulation. [Means for solving the problem]

[0014] According to a first aspect of the embodiments of this specification, the objective is achieved by a method performed by the device to provide information indicating the intensity peak width of an intensity peak along a pixel line of a pixel having a pixel intensity value that includes an intensity peak and is part of the intensity peak. The pixel corresponds to a pixel element in the image sensing area of ​​an image sensor, and obtains the pixel intensity value of the pixel from exposure of the image sensing area as part of image sensing by the image sensor of light reflected from the surface. The intensity peak is formed from such reflected light sensed by the image sensor. The device calculates a first value including the magnitude of a lower-order Discrete Fourier Transform (DFT) coefficient with respect to the pixel intensity value of the pixel along the pixel line containing the intensity peak. The device calculates a second value including the magnitude of another higher-order DFT coefficient with respect to the pixel intensity value of the pixel along the pixel line containing the intensity peak. Higher-order is a DFT coefficient of higher order than the lower-order DFT coefficient. The device provides the information indicating the intensity peak width based on the first and second values.

[0015] According to a second aspect of the embodiments of this specification, the objective is achieved by a device for providing information indicating the intensity peak width of an intensity peak along a pixel line of a pixel having an intensity peak and a pixel intensity value that is part of the intensity peak. The pixel corresponds to a pixel element in the image sensing area of ​​an image sensor, and obtains the pixel intensity value of the pixel from exposure of the image sensing area as part of image sensing by the image sensor of light reflected from the surface. The intensity peak is formed from such reflected light sensed by the image sensor. The device is configured to calculate a first value including the magnitude of a lower-order Discrete Fourier Transform (DFT) coefficient with respect to the pixel intensity value of the pixel along the pixel line containing the intensity peak. The device is further configured to calculate a second value including the magnitude of another higher-order DFT coefficient with respect to the pixel intensity value of the pixel along the pixel line containing the intensity peak. Higher-order is a DFT coefficient of higher order than the lower-order DFT coefficient. Furthermore, the device is configured to provide the information indicating the intensity peak width based on the first and second values.

[0016] According to a third aspect of the embodiments of this specification, the objective is achieved by one or more computer programs that include instructions causing a device to perform the method according to the first aspect when executed by one or more processors.

[0017] According to a fourth aspect of the embodiments of this specification, the objective is achieved by one or more carriers comprising one or more computer programs according to the third aspect.

[0018] Embodiments of this specification are based on the understanding that the magnitudes of lower-order and higher-order DFT coefficients, or the corresponding first and second values, and more specifically, how those values ​​relate to each other, include information indicating the width of the intensity peak. Furthermore, the width may indicate the scattering of light from a point on the surface that caused the reflection resulting in the intensity peak, as described in the "Background Art." Estimation of the intensity peak according to embodiments of this specification has demonstrated the ability to provide results at least as good as other existing methods, such as "moment scattering" mentioned in the "Background Art." In some situations and / or with respect to some applications where the intensity peak width is of interest, a particular method for estimating the intensity peak width may be preferred and / or more advantageous than another method, depending, for example, the information required, the information already available, the type of device involved, other operations being performed, etc. Thus, estimation of the intensity peak according to embodiments of this specification is generally advantageous as a further alternative method for providing information indicating the intensity peak width, such as by being an estimate.

[0019] Furthermore, the first and second values ​​may be obtained by calculations that can be performed by relatively simple operations that can be performed directly on the image sensor, either entirely or partially, without the need to first read image data from the image sensor. If the image sensor performs calculations to calculate the first and second values ​​for each pixel column of the image sensing area, for example, such that the pixel lines correspond to pixel columns, this can be particularly advantageous because it enables fast, column-by-column parallel processing on the image sensor, typically "on-chip," and as a result, for example, the information is also provided column by column. This means that the information indicating the intensity peak width can already be provided efficiently and quickly in connection with the provision of image data from the image sensor, and / or, for example, if only the information is of interest, image data may not be transferred from the image sensor at all.

[0020] Examples of embodiments of this specification will be described in more detail with reference to the accompanying schematic drawings briefly described below.

Brief Description of the Drawings

[0021] [Figure 1] A diagram schematically showing an example of a prior art 3D imaging system based on optical triangulation. [Figure 2] A diagram schematically showing an example of an imaging system for 3D imaging based on optical triangulation that can execute embodiments of this specification and / or can be configured to execute embodiments of this specification. [Figure 3A] A diagram schematically showing an example of an image sensor that can execute embodiments of this specification and / or can be configured to execute embodiments of this specification. [Figure 3B] A diagram schematically showing the peak of intensity along the column of the image sensing area of FIG. 3A. [Figure 4A] A diagram regarding a first example of the peak of intensity included by a sequence of pixels. [Figure 4B] A diagram regarding a first example of the DFT coefficients calculated for a sequence of pixels. [Figure 5A] A diagram regarding a second example of the peak of intensity included by a sequence of pixels. [Figure 5B] A diagram regarding a second example of the DFT coefficients calculated for a sequence of pixels. [Figure 6] A flowchart schematically showing and exemplifying a first method according to embodiments of this specification. [Figure 7A] An example of a scale of how lower-order DFT coefficients and higher-order DFT coefficients are related to each other and how this is mapped to the intensity peak width. [Figure 7B] An example of a scale of how lower-order DFT coefficients and higher-order DFT coefficients are related to each other and how this is mapped to the intensity peak width. [Figure 7C] An example of a measure of how the lower-order DFT coefficients and the higher-order DFT coefficients are related to each other and how this is mapped to the intensity peak width. [Figure 8A] An example of a measure of how the lower-order DFT coefficients and the higher-order DFT coefficients are related to each other and how this is mapped to the intensity peak width. [Figure 8B] An example of a measure of how the lower-order DFT coefficients and the higher-order DFT coefficients are related to each other and how this is mapped to the intensity peak width. [Figure 8C] An example of a measure of how the lower-order DFT coefficients and the higher-order DFT coefficients are related to each other and how this is mapped to the intensity peak width. [Figure 9A] An example of a measure of how the lower-order DFT coefficients and the higher-order DFT coefficients are related to each other and how this is mapped to the intensity peak width. [Figure 9B] An example of a measure of how the lower-order DFT coefficients and the higher-order DFT coefficients are related to each other and how this is mapped to the intensity peak width. [Figure 9C] An example of a measure of how the lower-order DFT coefficients and the higher-order DFT coefficients are related to each other and how this is mapped to the intensity peak width. [Figure 10] A schematic block diagram for showing an embodiment of how one or more devices may be configured to perform the methods and operations described in relation to FIG. 6. [Figure 11] A schematic diagram showing an embodiment relating to a computer program and its carrier.

MODE FOR CARRYING OUT THE INVENTION

[0022] The embodiments described herein are exemplary. It should be noted that these embodiments are not necessarily mutually exclusive. Components from one embodiment may be implicitly assumed to be present in another embodiment, and how those components may be used in other exemplary embodiments will be apparent to those skilled in the art.

[0023] Figure 1 schematically shows an example of an imaging system of the type described in the background art, namely, an imaging system 105 for 3D machine vision or simply 3D imaging based on optical triangulation, i.e., imaging for capturing information about the 3D features of an object using a camera 130. System 105 is shown in the figure in normal operation, i.e., typically after calibration has been performed and therefore after the system has been calibrated. System 105 is configured here to perform optical triangulation in the form of triangulation of a sheet of light, as described in the background art. System 105 further includes a light source 110, for example, a laser, for illuminating the object to be imaged with a specific pattern of light corresponding to the sheet of light, typically structured light such as the optical plane 111 shown in the figure. The light is typically laser light, but alternatively, it may be light from, for example, one or more light-emitting diodes (LEDs). An alternative to, but with a similar effect, an optical edge, i.e., the edge of the illuminated area. The generated light and illumination are typically provided through one or more lenses of the camera 130, for example, to focus the light. Furthermore, the camera 130 is typically configured and positioned to have a focal plane aligned with the light plane 111, in other words, aligned with the light plane 111, based on the so-called Scheimpflug principle or Scheimpflug focusing. In this way, object reflections occurring in the light plane are focused on the image sensor. In the example shown, the objects to be imaged are exemplified by a first object 120 in the shape of an automobile and a second object 121 in the shape of a gear structure. When the light plane 111 is incident on an object, this corresponds to the projection of the light plane 111 onto the object, which may be seen when the light plane 111 intersects with the object. For example, in the example shown, the light plane 111 produces a line of light 112 on object 120. The light is reflected by the object, more specifically by a part of the object at the intersection, i.e., at the line of light 112 in the example shown. Camera 130 includes an image sensor (not shown).The camera and image sensor are positioned relative to the light plane 111 such that the light plane 111, when reflected by an object, becomes incident light on the image sensor through the camera 130. The image sensor is typically implemented as a chip to sense the incident light and convert it into image data as intensity values ​​resulting from the light sensed by, for example, sensing elements corresponding to pixels in the image sensing area of ​​the image sensor. For example, in the shown example, the light plane 111 is reflected toward the camera 130 and image sensor in a line of light 112 over a portion of the roof of the car object 120, and the camera 130 and image sensor thereby generate and provide image data containing information about the portion of the car roof. Using a setup including the geometric arrangement of the system 105, for example, knowledge of how the image sensor coordinates relate to real-world coordinates such as x, y, z in a coordinate system 123 related to the object being imaged and its context, the image data may be converted into information about the 3D features of the object being imaged, such as its 3D shape or profile, in a preferred format. The 3D features, for example, information regarding the 3D shape or profile, may include data describing the 3D features in any preferred format.

[0024] By moving the object to be imaged, such as a light source 110 and / or a first object 120 or a second object 121, so that multiple parts of the object are illuminated and reflected light is produced on the image sensor, for example, by scanning the object, image data describing the more complete 3D shape of the object may be generated, corresponding to multiple consecutive profiles of the object, such as shown profile images 141-1 to 141-N of the first object 120, each profile image showing the contour of the first object 120 as reflected by the light plane 111 when the image sensor of the camera 130 senses the light that gives rise to the profile image. As shown in the figure, the light source 110 and camera 130 are usually stationary, and a conveyor belt 122 or the like may be used to move the object across the light plane 111. Alternatively, the light plane and / or camera 130 may be moved over the object so that all parts of the object, or at least all parts facing the light source 110, are illuminated, and the camera can receive light reflected from different parts of the object that are desirable to image.

[0025] As can be understood from the above, for example, an image frame of a first object 120, provided by the camera 130 and its image sensor, may result in one of the profile images 141-1 to 141-N. As described in the background art, each position of the contour of the first object shown in one of the profile images 141-1 to 141-N is typically determined based on the identification of intensity peaks in the image data captured by the image sensor and finding the locations of these intensity peaks by, for example, one or more peak-finding algorithms. System 105 and typical peak-finding algorithms are typically configured to search for intensity peaks pixel by pixel in each image frame. As shown in the figure, if the sensor coordinates are u, v, then u may be used to indicate a position in such a row that lies along the image sensor row, for example, corresponding to an image sensor column. Correspondingly, v may be used to indicate a position in such a column that lies along the image sensor column, for example, corresponding to an image sensor row.

[0026] For each position u in the image frame, for example, that position u may be searched for along v by a peak discovery algorithm as described above, and the identified peak in the image frame may result in one of the profile images 141-1 to 141-N as shown in the figure. The profile image is formed by image points in coordinate system 143, u, v, t based on sensors related to real-world coordinates such as x, y, z in coordinate system 123, as described above. The entire image frame and profile image may be used to create a 3D image of the first object 120 in the form of such a "point cloud" as described in the background art, where each point corresponds to a position on the object and is associated with 3D coordinates relative to that point.

[0027] As a development toward embodiments of this specification, the circumstances described in the background art will first be explained in more detail.

[0028] A reasonable assumption, and supported by practical evidence, is that the cross-section of the intensity peak line produced by the light used in optical triangulation, typically a laser, is Gaussian or Gaussian in shape, and therefore follows a normal distribution. It can then be mathematically demonstrated that the Discrete Fourier Transform (DFT) applied to the pixels surrounding the maximum intensity peak also has such a shape. In other words, the DFT was applied to the pixels covering the intensity peak, and therefore to the pixels with image data containing the intensity peak. Furthermore, mathematically, there exists a relationship between the width of such a Gaussian-shaped light line, the "sigma of the light," and the width of a Gaussian-shaped DFT, the "sigma of the DFT." Furthermore, if we divide the magnitude of the lowest-order DFT coefficient (DFT0 or zero-order DFT coefficient) by the magnitude of the second-lowest-order DFT coefficient (DFT1 or first-order DFT coefficient), i.e., DFT0 / DFT1, it can be shown that this is a measure corresponding to an estimate of the intensity peak width, the "sigma of light." Moreover, this is a measure of the scattering of light caused by the point of the object that reflected the light resulting in the intensity peak, as explained above in "Background Techniques." In general, the relationship between any higher-order and lower-order DFT coefficients contains information that can be used as an indicator of the intensity peak width when both exist as substantial coefficients. For example, alternative methods may involve calculating one of the following instead: DFT0 / DFT2, DFT1 / DFT2, DFT0-DFT1, DFT0-DFT2, etc. However, using lower-order coefficients is usually preferred and can provide a better estimate of a wider range of widths. Since DFT0 is a real value that actually always exists and can be calculated simply by adding up the pixel values ​​of the pixels surrounding the maximum intensity peak, it is always beneficial to use DFT0 in any case, and the pixels surrounding the maximum intensity peak are therefore the pixels that the DFT is involved in and calculated.

[0029] The above has been carried out in actual tests with good results, and some examples of this and the mathematical relationships mentioned are given further below. Also, several tests have been performed on 3D imaging of objects with wooden surfaces based on optical triangulation. It may be of particular interest that wood has a surface with natural variations that affect light scattering, and therefore light scattering can be estimated by estimating the width of the intensity peaks obtained as a result of optical triangulation. Light scattering includes information such as the type and quality of the wood, as well as how and at what location on the object this may vary with respect to a wooden object.

[0030] The above-described DFT-based method for obtaining indicators such as intensity peak width and, consequently, measured and / or estimated values ​​of light scattering is sometimes called "DFT scattering (DFT scatter)" to distinguish it by name from other methods mentioned in the background, such as "offset scattering" and "moment scattering." DFT-based methods, i.e., "DFT scattering," have several advantages over other methods, for example, • DFT scattering handles laser speckles better, for example, when lasers are used as the light source for optical triangulation, thus enabling more robust peak width estimation compared to offset scattering. • DFT scattering does not require the use of a fixed distance from the highest intensity pixel detected. • Peak width estimation can be performed based on experimental results with at least the same level of accuracy as "moment scattering," without requiring any "centroid" calculations. "DFT scattering" can be performed using relatively simple calculations that can be computationally efficient on or closely associated with the image sensor.

[0031] The DFT-based methods described above and disclosed herein, or in other words, “DFT scattering,” enable the estimation of intensity peak widths and the measurement of corresponding light scattering, which in at least some situations may be computationally more beneficial to use than “moment scattering.”

[0032] Figure 2 schematically shows an example of an imaging system 205 for optical triangulation-based 3D imaging in which embodiments of the Specified herein may be performed and / or configured to perform embodiments of the Specified herein. The imaging system 205 may structurally correspond to the imaging system 105, but is shown here in a more simplified form. The figure also shows the context in which an image sensor 231 configured to work according to embodiments of the Specified herein may be used, as will be further described below. The image sensor 231 is part of a camera 230 included in the imaging system 205 and used for optical triangulation-based 3D imaging, and which may perform embodiments of the Specified herein in that context.

[0033] Therefore, the diagram shows the following:

[0034] Object 220 may correspond to object 120 and is shown to be at least partially located within the field of view 232 of camera 230.

[0035] Typically, the light source 210 of the imaging system 205 is configured to illuminate object 220 with light 211 for 3D imaging based on optical triangulation, in the form of structured light such as a specific pattern of light that produces lines of light on object 220, is reflected by object 220, and is captured by camera 230 having an image sensor 231. For example, illumination by light 211 may be a “sheet of light,” i.e., an optical plane, which consequently produces lines of light on the object corresponding to where the optical plane intersects the object. Another example of structured light that can be used as the first light is an optical edge, i.e., the edge of an area or part of the illumination. For example, light from an LED rather than a laser is possible. The illumination in this example is perpendicular, i.e., substantially parallel to the z-axis shown in the figure, and therefore the optical plane lies in the zx plane. However, one or more other directions of illumination are of course available in the embodiments herein.

[0036] A computing device 233, such as a computer, may be part of the imaging system 205. The computing device 233 may be connected to the camera 230 and / or the light source 210 and may be configured to receive and process image-related data and information from and / or via the camera 230 to the image sensor 231. Furthermore, the computing device 233 may be configured to control the imaging system 205 and / or the camera 230 and / or its image sensor 231 and / or the light source 210. The computing device 233 may further be configured to perform and / or participate in some of the operations related to the embodiments herein, as further described below. Alternatively or additionally, the computing device 233 or its corresponding function may be fully or partially integrated into the same unit as the camera 230, in which case the camera 230 may be considered to have this function.

[0037] Thus, similar to conventional 3D imaging based on optical triangulation, the object 220 may be illuminated by light 211, and the image may be captured by a camera 230 having an image sensor 231.

[0038] Accordingly, similar to conventional optical triangulation-based imaging systems, the imaging system 205 may be configured to move the light source 210 and / or the light 211 provided by the light source 210, and / or move the object 220 to illumination by the light 211 provided by the light source 210, so that different consecutive parts of the object 220 are illuminated by the light 211 at different consecutive moments. To achieve this, the object 220 may be moved as part of the transport of the object 220, for example, by a conveyor belt. After reflection from the object 220, the light is sensed by the image sensor 231 via the camera 230. Each image frame is associated with each moment when the image frame was sensed, i.e., captured, and each part of the object 220 from which the image sensor 231 sensed the reflected light 211 at each moment.

[0039] Information derived from the image frame, such as the image frame and / or other information about the intensity peaks, including the intensity values ​​of pixels that are part of the intensity peaks of the light lines captured as part of optical triangulation, the location of the intensity peaks, and / or the width of the intensity peaks, may be provided by the camera 230 and / or the image sensor 231 and may be transferred to, for example, a computing device 233 for further processing within and / or outside the camera 230.

[0040] Regions of interest (ROIs), particularly windows around maximums or windows at maximums (WAMs), may be usefully used in embodiments herein. WAMs are described and discussed further below. Image sensors included in embodiments herein, such as image sensor 231, may be implemented as disclosed in EP4266673A1 and / or have corresponding functions to facilitate the use of WAMs as ROIs.

[0041] Before going into further details and examples relating to embodiments of this specification, the following are some examples and considerations in the context of images and image sensors to enable a deeper understanding. Images may originate from the camera 230 or the image sensor 231 of the imaging system 205. More specifically, the examples and considerations relate to captured lines of light, their intensity peaks, and the WAM described above.

[0042] Figure 3A schematically shows an example of an image sensor 331 having an image sensing area 351 formed by the entirety of pixels shown as squares. The image sensor 331 is an image sensor on which embodiments of the herein can be performed and / or configured to perform embodiments of the herein. The pixels of the image sensor 331 correspond to light sensing elements and may be of a conventional type. In the example, the entirety of pixels is arranged in N rows and M columns, with N=12 in the shown example. Thus, there are 352-1...352-M columns and 354-1...354-12 rows. The shown figure is divided along the rows.

[0043] An intensity peak line 361 is drawn in the image sensing area 351, where the darker pixels are formed by 5 pixels per column, and therefore have a WAM height 362 which is 5 pixels in the example, indicating an ROI which is a WAM centered on the intensity peak line 361. Of course, if such a thing is used, such a line or WAM would not be visible by visual inspection of the corresponding real image sensing area 351. The intensity peak line 361 and WAM are drawn solely to facilitate understanding how the intensity peaks and WAMs may relate to each other and to the image sensor. The intensity peak line 361 corresponds to a location where the intensity peak may be located at subpixel resolution and may occur as a result of exposure of the image sensor 331 when the image sensor 331 is part of a camera in a 3D imaging system based on optical triangulation, such as imaging system 205. If an image based on the entire pixels is read from the image sensor 331 after the image sensor 331 has been exposed to a line of light intensity having a peak in intensity in a light distribution having a peak according to the intensity peak line 361, the line of light intensity may become visible in the image and the intensity peak line 361 may be detected therefrom. Thus, such lines of intensity and the intensity peak line 361 may be caused by reflected light from an object, for example, object 220, when the object is illuminated by a line of light during optical triangulation.

[0044] In practice, each pixel can typically hold only a single pixel value corresponding to an intensity value. The WAM for each column 352 is centered on and therefore surrounds the pixel detected within that column so as to hold the maximum pixel value corresponding to the maximum intensity. Such pixels are marked with thick lines in the figure. The local WAM, i.e., the entirety of the column-by-column WAMs for all columns M, forms the overall WAM centered on and therefore surrounding the intensity peak line 361.

[0045] In this example, there are N=12 pixel rows 0...11 corresponding to image sensor rows 354-1...354-12, and M pixel columns 1...M corresponding to image sensor columns 352-1...352-M.

[0046] Typically, pixel values ​​are read out in parallel from the image sensor 331, row by row; that is, the pixel values ​​of pixels in the same row are read out in parallel and then converted from analog to digital (A / D). As shown in the figure, M columns 352 may be connected to M parallel A / D converters (ADCs) 356. An alternative technique may instead be to read out pixels with the same WAM height position in parallel; that is, all the dark pixels in Figure 3A, i.e., all the pixels corresponding to the WAM, may be read out by five consecutive parallel readouts. A detailed example of such an implementation is disclosed, for example, in EP4266673A1.

[0047] After A / D conversion, and therefore in the digital domain, there may be some processing and / or computational circuits, such as computational circuit 357 schematically shown in the figure, that operate on the resulting digital image data, for example, on pixels from each column and / or from multiple columns and the values ​​of those pixels. Therefore, there may be parallel computations. Each parallel computation may be related to the pixel values ​​of pixels from each column. For example, a calculation on DFT coefficients may be performed in parallel for all columns. During such processing and / or computations, there may be further memory elements included to temporarily store pixel values ​​and intermediate data. Such memory elements may allow the image sensor to perform processing and computations on pixel values ​​from several pixels in each column covering column-by-column intensity peaks, such as the column-by-column intensity peaks around the intensity peak line 361, for example, all pixels of the WAM, or several other pixels.

[0048] Furthermore, the image sensor 331 includes, for example, some input / output (I / O) circuit 358 for which inputs are involved for control of the image sensor, enabling the reading of image data from the image sensor 331 after A / D conversion, and / or enabling the reading of information obtained as a result of calculations performed by, for example, the calculation circuit 357. When the image sensor provides something for reading from the image sensor, this is via an I / O circuit such as the I / O circuit 358.

[0049] The figure also shows the control circuit 359 involved in controlling the image sensor 331 and parts thereof, for example, for addressing and / or selecting columns and / or rows and / or pixels to be A / D converted. The control circuit 359 may also be involved in temporarily storing information such as ROIs such as WAM in use, and which pixels are included in WAM.

[0050] Figure 3B schematically shows an intensity peak 363 with a light distribution 364. The intensity pixel value of a pixel containing an intensity peak corresponding to the cross-section of the intensity peak line 361 thus captures the light distribution as it might appear when sampled. The intensity peak lies along the pixel line corresponding to column 352-m, which may be any of columns 352-1...352-M. The light distribution 364 is shown along axis v, i.e., in the direction along the column, as in the case of column 352-m.

[0051] The intensity values ​​of pixels along column 352-m are shown in the figure as small black squares. As can be seen, the intensity values ​​represent intensity I, belong to different rows, i.e., are at different row coordinates v, and therefore vary along the column. The intensity values ​​occur according to the distribution of light 364, which has a Gaussian "bell" shape as described above.

[0052] What is shown are pixels from a single column, column 352-m, and WAM 366a of this column is drawn, and therefore corresponds to the darker pixels in the column in Figure 3A. Thus, WAM 366a has a WAM height 362 of 5 pixels. As can be seen, with respect to the example light distribution 364, 5 pixels do not provide very good coverage of the intensity peak 363. When looking at the entire peak following the shown light distribution 364, WAM 366a barely covers the full width at half maximum 371. Another sequence of pixels from column 352-m, namely pixels corresponding to WAM 366b covering 15 pixels, is also shown in the figure. With respect to a particular application of the embodiments herein, it may be preferable to use a WAM or corresponding sequence of pixels covering the intensity peak that covers with some margin the widest intensity peak expected for the application and which the embodiments herein should be able to handle for that application. In practice, it is desirable to be able to handle the application, and therefore it is sometimes recommended to cover at least twice the FWHM corresponding to the maximum FWHM of the application. For example, a number of pixels corresponding to the maximum FWHM are covered on both sides, i.e., around, the highest intensity pixel of the intensity peak.

[0053] The WAM for each column and the overall WAM formed by multiple WAMs for each column are not only used to select pixels used in the embodiments herein. As already shown above, a more common use is to select pixels around intensity peaks and use those pixels to find the peak location at subpixel resolution. For that purpose, a WAM height should be used that provides sufficient coverage of the light distribution 364 so that the peak discovery algorithm can utilize the light distribution 364 to find the actual center location of the light distribution 364 at subpixel resolution. The peak discovery algorithm is generally used outside the image sensor after the digital image data has been read out from the image sensor. As shown in the figure, as shown in Figure 3A, the actual peak location has the maximum intensity value and is some offset from the center location of the pixel around which the WAM is centered. In practice, for this purpose, a WAM height that covers about 75% or more of the energy of the intensity peak should typically be used. With respect to a typical sensor and setup, this may mean a WAM height in the range of 7 to 30 pixels. However, there may be situations in which WAM heights less than 7 and greater than 30 may also be used. For the purposes of the embodiments described herein, it is usually sufficient to cover 75% of the energy. For applications where using a WAM for both purposes is of interest, it is usually advantageous to use one same whole WAM and WAM height that is sufficient for both purposes.

[0054] As can be understood from the foregoing, when calculating the magnitude of the DFT, or more precisely, some DFT coefficients, as in the embodiments herein, this may be done on each pixel in each column corresponding to a WAM or a corresponding sequence of pixels that adequately cover the intensity peaks. A WAM, by definition, covers the intensity peaks in each column and is therefore of interest to the pixels for "DFT scattering" and the embodiments herein. A WAM where the number of pixels belonging to each column of the WAM corresponding to a WAM height of 362 is the same means that, in a corresponding way, the same number of calculations can be performed for each column to calculate the DFT coefficients. This is advantageous to the embodiments herein, even when the available WAM functionality is not explicitly used.

[0055] While not mandatory, using WAM may be advantageous in the embodiments herein when WAM is available and / or the image sensor has support for providing such a feature. It is possible to select a sequence of pixels that cover the intensity peaks and calculate the magnitude of these DFT coefficients without using the image sensor's WAM and / or WAM functionality. For example, the magnitude of the DFT coefficients included in the embodiments herein may be calculated with respect to pixels that cover or include the intensity peaks but do not belong to what is normally called WAM. Even a different number of pixels per column may be used to calculate the corresponding DFT coefficients and / or for different DFT coefficients, but this does not offer any obvious advantage as it requires unnecessary additional calculations to be performed in order to achieve consistency and to make it easier to compare results between columns.

[0056] Figures 4A and 4B relate to a first example of the magnitude of the intensity peak 463 contained by a sequence of pixels 466 and the DFT coefficient 473 calculated with respect to the sequence of pixels. The example is based on embodiments of this specification and is intended to illustrate the relationship between the magnitude of the DFT coefficient used in the following further examples related to embodiments of this specification.

[0057] Figure 4A shows the intensity peak 463 in the corresponding figure as Figure 3B, with the normalized intensity values ​​of pixel 466 containing the intensity peak 463. Pixel 466 is, in this example, 31 pixels located along v from integers -15 to +15, including 0. As can be seen from the diagram, in this example, of pixel 466, only five have pixel values ​​that actually contribute to peak 463, which appears quite narrow and "pointed" when a line is drawn connecting the values ​​of adjacent pixels, and the peak does not have an obvious Gaussian shape distribution. The width of intensity peak 463 is indicated by the full width at half maximum (FWHM) 471 of the peak. FWHM is the width of the peak at half the peak's maximum intensity. While FWHM may be a suitable width measure for use in the embodiments herein, other width measures at peak heights or any other measure indicating width may be used. However, to facilitate comparisons of different widths, which may, for example, correspond to different degrees of light scattering from the positions of different objects, and for such comparisons, it is preferable to use the same type of width scale. In the following, as a non-limiting example, the type of width scale used is FWHM. FWHM 471 corresponds to a width of 2 pixels and is an example of a narrow width.

[0058] Figure 4B shows the magnitude of the DFT coefficients of the DFT calculated for intensity peak 463, i.e., pixel 466. As can be seen, the DFT has a length corresponding to the number of samples for which the DFT was calculated, which corresponds here to the number of pixels 466, and thus the number of coefficients. The first three DFT coefficients, starting from the lowest order, are explicitly drawn: DFT coefficient 473-0 is the lowest-order DFT coefficient DFT0, DFT coefficient 473-1 is the second lowest-order DFT coefficient DFT1, and DFT coefficient 473-2 is the third lowest-order DFT coefficient DFT2.

[0059] Figures 5A and 5B relate to a second example of the magnitude of the intensity peak 563 captured by a sequence of pixels 566 and the DFT coefficient 573 calculated with respect to the sequence of pixels. The example is based on embodiments of this specification and is intended to illustrate the relationship between the magnitude of the DFT coefficient used in the following further examples relating to embodiments of this specification.

[0060] Figure 5A shows the intensity peak 563 in the normalized intensity value of pixel 566, which contains the intensity peak 563, in the corresponding figures as Figure 3B and Figure 4A. Pixel 566 is the same number of pixels as in Figure 4A, i.e., 31 pixels located along v from integers -15 to +15, including 0, in the second example as well. It appears to be the same pixel involved, but this should be considered an example effect. In fact, different peaks generally occur in different locations, especially when the peaks originate from 3D imaging by optical triangulation. What is shown in Figures 4A and 5A could further be peaks from different columns occurring simultaneously, for example, on the image sensing area 351. In any case, as should be understood from what is shown, in this example there are 31 more pixels that actually contribute to peak 563, which has a typical Gaussian "bell shape" that can be seen by connecting the values ​​of adjacent pixels in a straight line. The width of intensity peak 563 is indicated by peak FWHM 571. FWHM 571 corresponds to a width of 12 pixels and may be compared to FWHM 471 of intensity peak 463, which has a significantly smaller and narrower width.

[0061] Figure 5B shows the magnitude of the DFT coefficients of the DFT calculated for intensity peak 563, i.e., pixel 566. As can be seen, here again, the DFT has a length corresponding to the number of samples for which the DFT was calculated, which corresponds here to the number of pixels 566, and thus the number of coefficients. The first three DFT coefficients, starting from the lowest order, are explicitly drawn: DFT coefficient 573-0 is the lowest-order DFT coefficient DFT0, DFT coefficient 573-1 is the second lowest-order DFT coefficient DFT1, and DFT coefficient 573-2 is the third lowest-order DFT coefficient DFT2.

[0062] DFT can generally be thought of as corresponding to the "frequency" content of the signal for which it is calculated. The fairly narrow and "sharp" intensity peak 463 corresponds to a signal with more "higher frequency" components, so as can be seen and expected, there are more higher-order DFT coefficients compared to the broader and "blunter," broader light distribution of the intensity peak 563.

[0063] Figure 6 is a flowchart illustrating a method according to embodiments of this specification, which has already been shown above but is disclosed in some more detail here. The following operations which may form the method are for providing information indicating the intensity peak width, such as one of the FWHMs 371, 471, and 571 of the intensity peaks, such as one of the intensity peaks 363, 463, and 563. The intensity peaks are along a column, such as being in or part of a column, such as a column 352-m of pixels that contain the intensity peaks and have intensity pixel values ​​that are part of the intensity peaks. For example, the pixels may be one of the sequences of pixels 366, 466, and 566.

[0064] The aforementioned pixels further correspond to pixel elements in an image sensing area, such as an image sensing area 351 of an image sensor 231 or 331. The pixels acquire their intensity pixel values ​​from the exposure of the image sensing area as part of image sensing by the image sensor of structured light reflected from a surface, such as lines of light, when imaging is 3D imaging based on optical triangulation. The intensity peaks are formed from such reflected light sensed by the image sensor, i.e., the reflected light is sensed as part of image sensing. Thus, the intensity peaks correspond to the cross-section of the imaged lines of light. A similar relationship to that in this paragraph is illustrated and explained above in relation to Figures 3A and 3B.

[0065] The methods and / or operations described below may be performed by one or more devices. As will be understood by those skilled in the art, methods involving operations as described below may be performed in a distributed manner by multiple devices configured to perform the operations and / or involved in performing the operations. However, the one or more devices are preferably the image sensors, for example, image sensors 231 or 331, or include such image sensors and / or are part of an imaging system such as imaging system 205, or correspond to such an imaging system. For example, the one or more devices may be computing device 233 and / or camera 230, or include computing device 233 and / or camera 230. In the following, image sensor 331 may be used as an image sensor and a non-limiting example of a device performing the method. As will be understood from further consideration above, the image sensors in the embodiments herein should, in any case, be image sensors suitable for use as image sensors for cameras in an imaging system for 3D imaging of objects based on optical triangulation.

[0066] Therefore, in some embodiments, image sensing by the image sensor 331 is part of an imaging system for 3D imaging of an object having a surface, such as the object 220, for example, 3D imaging based on optical triangulation by an imaging system 205. The imaging system includes a camera, such as a camera 230 having an image sensor 231 which may correspond to the image sensor 331, and a light source, such as a light source 210, for providing illumination that brings the light on the surface as part of the 3D imaging based on optical triangulation. Therefore, the light on the surface, for example, lines of light, may be part of the 3D imaging based on optical triangulation. In some of these embodiments, an intensity peak, such as one of the intensity peaks 363, 463, or 563, is associated with a location on the surface where the light that brings the intensity peak is reflected. In these embodiments, the information indicating the intensity peak width may be provided for use as an indication of the scattering of light from the location.

[0067] In particular, when an image sensor is involved in carrying out the embodiments of this specification, it may be preferable from an implementation and / or computational standpoint that the pixel line is, for example, a pixel row corresponding to a row of pixel elements of the image sensor. However, it should be noted that, in principle, the pixel line can be in any direction as long as the pixel contains an intensity peak, and / or can correspond to any one-dimensional line of a pixel, for example, any pixel line in any direction on the image sensing area 351 of the image sensor 331.

[0068] Furthermore, as should be understood, the pixel elements in the image sensing area correspond to the light-sensing elements that sense light during exposure. Each pixel value acquired by each pixel element during exposure typically corresponds to the cumulative amount of light received by each image sensor pixel during exposure. The amount and pixel value typically correspond to the measured intensity of the light incident on the pixel element, or more specifically, its photosensitive portion, during exposure.

[0069] Please note that the following actions may and / or may be performed in any preferred order, and may overlap entirely or partially in time, where preferable.

[0070] Operation 601 The image sensor 331 calculates a first value, for example, that includes the magnitude of a lower-order DFT coefficient relating to the pixel intensity value of a pixel along a pixel line containing an intensity peak. The first value may include or correspond to a complex-valued lower-order DFT coefficient, where the complex-valued lower-order DFT coefficient includes its magnitude, magnitude itself, or any other value from which the magnitude can be derived or which specifies the magnitude.

[0071] For example, refer to the magnitudes of DFT coefficients 473 and 573 in Figures 4B and 5B. DFT coefficients in general, how they relate to DFT, and how they may be calculated will be discussed separately below. Lower-order DFT coefficients are further discussed below in operation 602.

[0072] Pixels related to lower-order DFT coefficients are thus associated with the intensity peaks in question. The pixels could be, for example, a sequence of 366 pixels if the intensity peak is intensity peak 363, a sequence of 466 pixels if the intensity peak is intensity peak 463, a sequence of 566 pixels if the intensity peak is intensity peak 563, but of course, they could also be other shorter or longer sequences of pixels containing the intensity peaks.

[0073] operation 602 The image sensor 331 calculates a second value, for example, that corresponds to a certain magnitude, which includes, for example, the magnitude of another higher-order DFT coefficient relating to the pixel intensity value of a pixel along a pixel line containing an intensity peak. The second value may include or correspond to a complex-valued higher-order DFT coefficient, which includes its magnitude, magnitude itself, or any other value from which the magnitude can be derived or which specifies the magnitude.

[0074] For example, refer to the magnitudes of the DFT coefficients 473 and 573 in Figures 4B and 5B. The DFT coefficients in general, how they relate to DFT, and how they may be calculated will be discussed separately below.

[0075] The pixels related to higher-order DFT coefficients are thus related to the intensity peak in question, and therefore to the same intensity peak to which lower-order DFT coefficients are related. Preferably, but not necessarily, the pixels are the same pixels to which the magnitude of the lower-order DFT coefficients is calculated. Thus, the pixels may be a sequence of pixels 366 if the intensity peak is intensity peak 363, or a sequence of pixels 466 if the intensity peak is intensity peak 463, or a sequence of pixels 566 if the intensity peak is intensity peak 563, but of course, they may be other shorter or longer sequences of pixels containing each respective peak.

[0076] Preferably, the lower-order and higher-order DFT coefficients include one or more of the following: • The lowest-order DFT coefficient. This coefficient is sometimes called DFT0 and may be, for example, either DFT0 473-0 or DFT0 573-0. • The second lowest-order DFT coefficient. This coefficient is sometimes called DFT1 and may be, for example, either DFT1 473-1 or DFT1 573-1. • The third lowest DFT coefficient. This coefficient is sometimes called DFT2 and may be, for example, either DFT2 473-2 or DFT2 573-2.

[0077] Preferably, the lowest-order DFT coefficient is the lowest-order DFT coefficient, DFT0. This coefficient exists, has a substantial magnitude with respect to all peaks, and can be calculated relatively easily compared to the other coefficients. The magnitude of DFT0 may be calculated simply and favorably as the sum of the pixel intensity values ​​of the pixels containing the intensity peaks, for example, pixels 366, 466, or 566.

[0078] Preferably, the higher-order DFT coefficients are the second lowest DFT coefficient DFT1 or the third lowest DFT coefficient DFT2.

[0079] The DFT coefficients in general, how they relate to DFT, and how they may be calculated will be discussed separately below. Further examples and explanations below may be found, for example, of how the magnitudes of the two DFT coefficients, the lower-order DFT coefficient and the higher-order DFT coefficient, can actually be calculated by the image sensor 331.

[0080] operation 603 The image sensor 331 provides the information indicating the intensity peak width, such as using the first value and the second value, or based on the first value and the second value, indicating one of the FWHM values ​​371, 471, or 571. Therefore, the information should be provided as an indication of the intensity peak width, and / or rather, for use as a measured value and / or estimate of the intensity peak width.

[0081] The information indicating the width may include the first value and the second value, or include, correspond to, or be based on, a measure of how the first value and the second value relate to each other, such as how the magnitudes of the DFT coefficients relate to each other.

[0082] The scale may include, correspond to, or be based on, how a first and second value relate to each other, preferably how they differ from each other, such as the magnitudes of the lower-order DFT coefficients and the higher-order DFT coefficients. The scale may include, correspond to, or be based on, a calculated quotient or difference between the first and second values, such as between the magnitudes of the lower-order DFT coefficients and the higher-order DFT coefficients. Further, as should be recognized, the quotient discloses how the values ​​being divided differ from each other.

[0083] If the information indicating the width includes the first value and the second value, another device outside the image sensor 331 may provide and typically calculate a scale on behalf of the image sensor 331, for example, based on how the first value and the second value relate to each other. This device may be, for example, a computing device 233 or a camera 230.

[0084] For example, if the first and second values ​​are lower-order complex-valued DFT coefficients and higher-order complex-valued DFT coefficients, the information indicating the width may include, correspond to, or be based on these complex-valued values. The magnitude of the DFT coefficients, and / or the complex-valued values ​​or the difference between their magnitudes, and / or the quotient between the magnitudes may be calculated outside the image sensor 331.

[0085] In another example, each first and second value is formed by adding the square of the real part of its DFT coefficient to the square of the imaginary part of its DFT coefficient. As can be understood from Equation 2 below, such values ​​include magnitude. Information indicating width may include, correspond to, or be based on these values, and may include the scale as a calculated quotient between the first and second values, for example. The quotient may be calculated outside of the image sensor 331.

[0086] How such quotients and differences mentioned above relate to, for example, the width of the intensity peak, such as the FWHM, and may be mapped, is illustrated in the following figures and will be further explained in relation to these figures.

[0087] The measure calculated by another device based on the first and second values ​​provided by the image sensor, or calculated by the image sensor itself, may therefore correspond to a calculated estimate of the peak width of the intensity.

[0088] Embodiments of this specification are based on the understanding that the magnitudes of lower-order and higher-order DFT coefficients, or the corresponding first and second values, and more specifically, how those values ​​relate to each other, include information indicating the width of the intensity peak. Furthermore, the width may indicate the scattering of light from a point on the surface that caused the reflection resulting in the intensity peak, as described in the "Background Art." Estimation of the intensity peak according to embodiments of this specification has demonstrated the ability to provide results at least as good as other existing methods, such as "moment scattering" mentioned in the "Background Art." In some situations and / or with respect to some applications where the intensity peak width is of interest, a particular method for estimating the intensity peak width may be preferred and / or more advantageous than another method, depending, for example, the information required, the information already available, the type of device involved, other operations being performed, etc. Thus, estimation of the intensity peak according to embodiments of this specification is generally advantageous as a further alternative method for providing information indicating the intensity peak width, such as by being an estimate.

[0089] Furthermore, the first and second values ​​may be obtained by calculations that can be performed entirely or partially on the image sensor, for example, on the image sensor 331, by relatively simple operations that can be performed directly on the image sensor, without the need to first read image data from the image sensor. If the image sensor performs calculations to calculate the first and second values ​​for each pixel column of the image sensing area, for example, such that the pixel lines correspond to pixel columns, this may be particularly advantageous because it enables fast, column-by-column parallel processing on the image sensor, typically "on-chip," and as a result, for example, the information is also provided column by column. This means that the information indicating the intensity peak width can already be provided efficiently and quickly in connection with the provision of image data from the image sensor, and / or, for example, if we are interested only in knowing the information, image data may not be transferred from the image sensor at all.

[0090] In some embodiments, pixels containing intensity peaks, for example, intensity peak 363, are pixels of an ROI, preferably a WAM, that partially covers an image sensing area, such as an image sensing area 351, as discussed above in relation to Figure 3. The ROI is determined to cover each intensity peak, such as intensity peak 363, in each column of the image sensing area, for example, columns 352-m. ROIs such as WAMs should have the same number of pixels in each column, and the image sensor should preferably support providing parallel readouts for pixels having the same ROI or WAM location, such as the same WAM height position.

[0091] The embodiments herein are advantageously combined with WAMs discussed above in relation to Figure 3A, i.e., ROIs or WAMs that cover the captured lines of light and their intensity peaks. These are intensity peaks of interest to the embodiments herein, and the embodiments herein can therefore be applied in combination with the use of such ROIs or WAMs to read out image data relating to intensity peaks.

[0092] With respect to a given imaging system and / or setup of such an imaging system that produces lines of light in an image, those skilled in the art can find, for example, how the quotient between the magnitudes of the DFT coefficients, as in the embodiments herein, relates to the intensity peak width, for example, how the quotient maps to the intensity peak width. This can be done, for example, in a calibration procedure that uses a given range of intensity peak widths and calculates the quotient with respect to them. Relationships such as the mapping between the quotient and the peak width can be given therein. A system implementing the embodiments herein, for example, its image sensor and / or other device, may, during normal operation, calculate the quotient between the magnitudes of the DFT coefficients, as in the embodiments herein, and provide an estimate of the intensity peak width using the mapping from the calibration. In some applications, for example, it may be sufficient to use only the quotient as a measure and be mindful of the variation, for example, the relative deviation from the “nominal” quotient. A quotient much larger than the nominal quotient may mean a much larger peak width and, for example, increased light scattering. And furthermore, this may indicate, for example, differences in the material at the location of the surface reflecting the light that gives rise to the peak.

[0093] For example, when a line of light is part of a 3D image based on optical triangulation, it should be further noted that, as in the embodiments herein, the intensity peaks resulting from the imaged line of light, for example, a line of reflected laser, will actually have a “peak shape” with a Gaussian or at least Gaussian-like light distribution. Furthermore, DFT will give narrower peaks more high DFT coefficients with substantial size compared to wider peaks. It should be understood that this will result in the principle of the embodiments herein always working to provide an indication of at least the intensity peak width, regardless of the exact light distribution of the peak.

[0094] Figures 7–9 are examples of different scales of how lower-order and higher-order DFT coefficients relate to each other and how this maps to intensity peak width. Thus, the DFT coefficients may correspond to the first and second values ​​in the embodiments herein, and the scales may correspond to the information indicating the intensity peak width in the embodiments herein. The width is represented by FWHM in the examples. The different widths used are similar to those in Figures 4A and 5A, but with different FWHMs for intensity peaks, and the magnitude of the DFT coefficients was calculated for each peak and used according to the embodiments herein. Further details follow below for each figure.

[0095] Figures 7A to 7C show an example where the scale is the calculated quotient between the first and second values, i.e., the first value divided by the second value, and how the quotient changes with intensity peak width. More details are as follows:

[0096] Figure 7A shows the case where the first value is calculated as the magnitude of the lowest-order DFT coefficient DFT0, and the second value is calculated as the magnitude of the second lowest-order DFT coefficient DFT1.

[0097] Figure 7B shows the case where the first value is calculated as the magnitude of the lowest-order DFT coefficient DFT0, and the second value is calculated as the magnitude of the third-lowest-order DFT coefficient DFT2.

[0098] Figure 7C shows the case where the first value is calculated as the magnitude of the second lowest-order DFT coefficient DFT1, and the second value is calculated as the magnitude of the third lowest-order DFT coefficient DFT2.

[0099] Figures 8A to 8C show an example from Figure 7, but instead of the quotient, the square root (sqrt) of the natural logarithm (ln) of the quotient is shown. The reason is to demonstrate that this yields a good linear relationship that may be useful in some applications of the embodiments described herein. Also, for example, in the calibration procedure discussed above, the result may be a simple linear equation that can then be used during normal operation to obtain an estimate of the width.

[0100] Figures 9A to 9C show an example where the scale is the calculated difference between a first value and a second value, i.e., the first value minus the second value, and how the difference changes with intensity peak width. More details:

[0101] Figure 9A shows the case where the first value is calculated as the magnitude of the lowest-order DFT coefficient DFT0, and the second value is calculated as the magnitude of the second lowest-order DFT coefficient DFT1.

[0102] Figure 9B shows the case where the first value is calculated as the magnitude of the lowest-order DFT coefficient DFT0, and the second value is calculated as the magnitude of the third-lowest-order DFT coefficient DFT2.

[0103] Figure 9C shows the case where the first value is calculated as the magnitude of the second lowest-order DFT coefficient DFT1, and the second value is calculated as the magnitude of the third lowest-order DFT coefficient DFT2.

[0104] The Discrete Fourier Transform (DFT) itself is well known and defined from a mathematical standpoint and has long been used in practical applications, but it is briefly summarized below to set the context for the DFT coefficients on which the embodiments herein are based. Some explanation will also be given as to how the DFT coefficients may be calculated in the context of the embodiments herein.

[0105] The Discrete Fourier Transform typically transforms a sequence of values ​​x[n] corresponding to a sample into another sequence of complex values ​​X[k].

number

[0106] From this and Equation 1, as already stated above, it can be understood that X0, i.e., DFT0, is simply the sum of x[n] for all N values, and therefore the sum of the pixel values ​​of the pixels containing the peak.

[0107] Generally, DFT coefficients are complex values, and their magnitudes are...

number

[0108] For DFT coefficients above X0, i.e., above DFT0, weighting coefficients must be considered and can be described as follows:

number

[0109] With respect to a given DFT coefficient, e.g., X1, i.e., DFT1, the essence of the real part of X1 is to multiply each x[n] by its cosine weighting coefficient and accumulate the results, and so it is understood that this is done for all N x[n] and corresponds to doing it for all pixels containing intensity peaks, such as for all 31 pixels in a sequence of 466 pixels. With respect to the imaginary part of X1, the essence of the imaginary part is to multiply each x[n] by its sine weighting coefficient and accumulate the results, corresponding to this. The cosine and sine weighting coefficients may be pre-calculated and stored on a chip, for example, on the image sensor 331. The image sensor may be configured with the pre-calculated weighting coefficients, for example, programmed during the initialization phase of the image sensor. The pre-calculated weighting coefficients may then be used in several multiplication and accumulation calculations to estimate the width of several intensity peaks from multiple exposures of the image sensor to which the embodiments herein should be applied.

[0110] As should be understood, multiplication and accumulation operations may, advantageously, be performed in parallel on the image sensor, such as column by column, in relation to the reading of image data from the image sensing area of ​​the image sensor, for example, the image sensing area 351, either pixel line by pixel or WAM by height position.

[0111] To calculate the first and second values ​​of the embodiments herein corresponding to the magnitudes of the two DFT coefficients, this can be done by four multiplication and accumulation operations, twice for each DFT coefficient for the real and imaginary parts of the DFT coefficient, respectively. If DFT0 is calculated, an accumulation operation is sufficient to calculate the magnitude of DFT0, in which case, therefore, two multiplication and accumulation operations, and one accumulation operation in total for the two DFT coefficients, are sufficient.

[0112] The first and second values ​​include the real and imaginary parts of the corresponding DFT coefficients and may therefore be complex values. The values ​​of the DFT coefficients, and / or their real and imaginary parts, may be transmitted off-chip, for example, outside the image sensor 331, to an external location where the magnitude of the DFT coefficients and / or, for example, the width estimation may be performed. This method of operation has already been shown above. For example, a computing device 233 or a camera 230 with computing capabilities may be configured to perform calculations for the magnitude calculation and / or the final width estimation. Such other devices may be better suited to performing those calculations than the image sensor itself.

[0113] Figure 10 is a schematic block diagram illustrating embodiments of how one or more devices 1000 may be configured in general to perform the methods and operations described in relation to Figure 6, in other words, how one or more devices 1000 may correspond to the devices already mentioned above. There may be multiple devices configured to be involved in and / or perform the methods and / or operations, i.e., multiple devices configured to be involved in and / or perform the methods and / or operations in a distributed manner. The device 1000 preferably corresponds to or includes the image sensor, for example, image sensor 231 or 331, and / or is part of or corresponds to an imaging system such as imaging system 205. For example, one or more devices may be a computing device 233 and / or camera 230, or include the computing device 233 and / or camera 230. If a device includes or is an image sensor, what is shown in Figure 10 may be considered a different diagram of how image sensor 331 may be configured.

[0114] Therefore, the device 1000 may include processing means, such as one or more hardware modules including one or more processing circuits, such as a processor, and / or one or more software modules for performing the method and / or the operation, such as the processing module 1001.

[0115] Device 1000 may further include memory 1002 which may contain or store computer program 1003. Computer program 1003 includes “instructions” or “code” that can be directly or indirectly executed by device 1000 to perform the method and / or operation described herein. Memory 1002 may include one or more memory units which may further be arranged to store data such as configurations, data, and / or values ​​that are involved in or for performing the functions and operations of the embodiments herein.

[0116] Furthermore, the device may include a processing circuit 1004, as an example of a hardware module, which is involved in processing and, for example, encoding data, and may include or correspond to one or more processors or processing circuits. The processing module 1001 may include a processing circuit 1004, for example, "embodied in the form of" the processing circuit 1004 or "realized by" the processing circuit 1004. In these embodiments, the memory 1002 may include a computer program 1003 that can be executed by the processing circuit 1004, thereby enabling or configuring the device 1000 to operate to perform the method and / or the operation.

[0117] Device 1000, for example, processing module 1001, may include an input / output (I / O) module 1005 configured to engage in any communication to and from other units and / or devices, such as sending and / or receiving information to and from other devices, for example by performing such actions. The I / O module 1005 may be exemplified, where applicable, by an acquisition module, for example, a receiving module, and / or a providing module, for example, a transmitting module.

[0118] In addition, device 1000 may include a sensing circuit 1006 as an example of a hardware module involved in sensing light and / or images, which fully or partially corresponds to or fully or partially includes those considered above with respect to the image sensor 331, including an image sensing area such as an image sensing area 351.

[0119] Furthermore, in some embodiments, the device 1000, for example, the processing module 1001, includes one or more of the following, as illustrative hardware and / or software modules for performing the operations of the embodiments herein: an acquisition module, an exposure module, a sensing module, a calculation module, and a providing module. These modules may be fully or partially implemented by the processing circuit 1004.

[0120] Therefore, device 1000, and / or processing module 1001, and / or processing circuit 1004, and / or calculation module are operable or configured to calculate the first value, which includes the magnitude of the lower-order DFT coefficients.

[0121] Device 1000, and / or processing module 1001, and / or processing circuit 1004, and / or calculation module are operable or configured to calculate the second value, which includes the magnitude of higher-order DFT coefficients.

[0122] Device 1000, and / or processing module 1001, and / or processing circuit 1004, and / or providing module, and / or I / O module 1005 are operable or configured to provide the information indicating the intensity peak width based on the first value and the second value.

[0123] Figure 11 is a schematic diagram showing several embodiments relating to a computer program 1003 and its carrier that causes the device 1000 discussed above to perform the method and the operation, or to cause the method and the operation to be performed.

[0124] Each computer program 1003 includes instructions that cause the corresponding device 1000 to operate as described above when executed by a suitable processing circuit, such as a processing circuit 1004, and / or a processing module, such as a processing module 1001. In some embodiments, one or more carriers are provided, i.e., carriers containing computer programs, or more specifically, data carriers such as computer program products. Each carrier may be one of electronic signals, optical signals, radio signals, and computer-readable storage media, for example, the computer-readable storage media 1101 schematically shown in the figure. Thus, computer programs may be stored in the computer-readable storage media 1101. Depending on the carrier, transient propagating signals may be excluded, and accordingly, the data carrier may be called a non-transient data carrier. Non-limiting examples of data carriers that are computer-readable storage media include memory cards or memory sticks, disk storage media, or mass storage devices generally based on hard drives or solid-state drives (SSDs). The computer-readable storage medium 1101 may be used to store data accessible via a computer network 1102, for example, the Internet or a local area network (LAN). Computer programs may further be provided as pure computer programs or contained in one or more files. One or more files may be stored in the computer-readable storage medium 1101 and made available, for example, through download via the computer network 1102, as shown in the figure, such as via a server. The server may be a web or file transfer protocol (FTP) based server, etc. One or more files may be executable files for direct or indirect download to device 1000 and execution on device 1000, for example, by execution by suitable respective processing circuits.One or more files may, further or alternatively, be for intermediate downloads and compilations involving the same or a different processor to make those files executable before further downloads and executions that cause the device 1000 to function as described above.

[0125] It should be noted that any of the aforementioned processing modules and circuits may be implemented as software and / or hardware modules, for example, on existing hardware and / or as application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), etc. It should also be noted that any of the aforementioned hardware modules and / or circuits may be distributed across several separate hardware components, whether, for example, they are contained in a single ASIC or FPGA, individually packaged, or assembled into a system-on-a-chip (SoC).

[0126] Furthermore, those skilled in the art will understand that the modules and circuits discussed herein may be hardware modules, software modules, analog and digital circuits, and / or one or more processors that, when executed, configure devices, sensors, etc., to perform the methods and operations described above, and / or may refer to a combination of one or more processors configured, for example, using software and / or firmware stored in memory, which may cause devices, sensors, etc., to perform the methods and operations described above.

[0127] Identification by any identifier in this specification may be implicit or explicit. Identification may be unique, for example, in a specific context relating to a particular computer program or program provider.

[0128] As used herein, the term “memory” may refer to data memory for storing digital information, typically such as hard disks, magnetic storage, media, portable computer diskettes or disks, flash memory, random access memory (RAM), etc. Furthermore, memory may also refer to the internal register memory of a processor.

[0129] It should be noted that all enumerative terms, such as "first device," "second device," "first surface," and "second surface," should be considered non-exclusive in themselves, and that the terms themselves do not imply any specific hierarchical relationship. Conversely, unless there is some explicit information, enumerative naming should be considered simply a way of realizing different names.

[0130] When used herein, the expression "configured to..." may mean that a processing circuit is configured or adapted by software or hardware configuration to perform one or more of the operations described herein.

[0131] As used herein, the terms “number” or “value” may refer to any type of digit, such as a binary number, a real number, an imaginary number, or a rational number. Furthermore, a “number” or “value” may be one or more characters, such as a letter or string of characters. Also, a “number” or “value” may be represented by a bit string.

[0132] When used herein, the expressions “may” and “in some embodiments” are generally used to indicate that the described features may be combined with any other embodiments disclosed herein.

[0133] Features that may be present in only some embodiments are typically depicted using dotted or dashed lines in the drawings.

[0134] When the words "comprise" or "comprising" are used, they are interpreted as non-restrictive, meaning "consist at least of".

[0135] The embodiments described herein are not limited to those described above. Various alternatives, modifications, and equivalents may be used. Accordingly, the embodiments described above should not be construed as limiting the scope of the disclosure as defined by the appended claims. [Explanation of Symbols]

[0136] 105 Imaging System 110 Light source 111 Light plane 112 Light Lines 120 The first object 121 The second object 122 Conveyor Belt 123 Coordinate System 130 Cameras 141-1~141-N Profile Images 143 Coordinate System 205 Imaging System 210 Light source 211 light 220 Object 230 Cameras 231 Image Sensor 232 Field of view 233 Computing Devices 331 Image Sensor 351 Image Sensing Area 352, 352-1~352-M, 352-m row Lines 354-1~354-12 356 M parallel ADCs 357 Calculation circuit 358 I / O circuit 359 Control circuit 361 Intensity Peak Line 362 WAM height 363 peak intensity 364 Light Distribution 366a WAM 366b WAM 371 Half-value full width 463 Peak intensity 466 pixels 471 FWHM 473 DFT coefficients 473-0 DFT coefficients 473-1 DFT coefficients 473-2 DFT coefficients 563 Peak intensity 566 pixels 571 FWHM 573 DFT coefficients 573-0 DFT coefficients 573-1 DFT coefficients 573-2 DFT coefficients 1000 devices 1001 Processing Module 1002 memory 1003 Computer Programs 1004 Processing Circuit 1005 I / O Module 1006 Sensing circuit 1101 Computer-readable storage medium 1102 Computer Networks

Claims

1. A method performed by one or more devices (205, 231, 331, 1000) for providing information indicating the intensity peak width (371, 471, 571) of a pixel (366, 466, 566) along a pixel line (352-m) of the intensity peak (363, 463, 563), the pixel being an image Corresponding to the pixel elements of the image sensing area (351) of the image sensor (231, 331), the pixel intensity value of the pixel is obtained from the exposure of the image sensing area (351) as part of the image sensing by the image sensor (231, 331) of light reflected from the surface, and the peak of the intensity (363, 463, 563) is formed from such reflected light sensed by the image sensor (231, 331), and the method is Step (601) of calculating a first value including the magnitude of lower-order discrete Fourier transform (DFT) coefficients (473, 573) for the pixel intensity values ​​of pixels (366, 466, 566) along the pixel line (352-m) containing the intensity peaks (363, 463, 563), Step (602) of calculating a second value including the magnitude of another higher-order DFT coefficient (473, 573) relating to the pixel intensity value of a pixel (366, 466, 566) along the pixel line (352-m) containing the intensity peaks (363, 463, 563), wherein higher-order is a higher-order DFT coefficient (473, 573) than lower-order DFT coefficient (473, 573), A method comprising the step (603) of providing information indicating the intensity peak widths (371, 471, 571) based on the first and second values.

2. The method according to claim 1, wherein the image sensing by the image sensors (231, 331) is part of 3D imaging based on optical triangulation by an imaging system (205) for three-dimensional (3D) imaging of an object (220) having a surface, and the imaging system (205) includes a camera (230) having the image sensors (231, 331) and a light source (210) for providing illumination to bring the light onto the surface as part of the 3D imaging based on optical triangulation.

3. The method according to claim 1 or 2, wherein the intensity peaks (363, 463, 563) are associated with the positions on the surface that reflect the light giving rise to the intensity peaks (363, 463, 563), and the information indicating the intensity peak width is provided for use as an indicator of light scattering from the positions.

4. The method according to any one of claims 1 to 3, wherein the lower-order DFT coefficients and higher-order DFT coefficients (473, 573) include one or more of the following: the lowest-order DFT coefficients (473-0, 573-0), the second lowest-order DFT coefficients (473-1, 573-1), and the third lowest-order DFT coefficients (473-2, 573-2).

5. The method according to any one of claims 1 to 4, wherein the lower-order DFT coefficient is the lowest-order DFT coefficient (473-0, 573-0).

6. The method according to claim 5, wherein the magnitude of the lowest-order DFT coefficient (473-0, 573-0) is calculated as the sum of the pixel intensity values ​​of the pixels (366, 466, 566) that include the intensity peak (363, 463, 563).

7. The method according to claim 5 or 6, wherein the higher-order DFT coefficients are the second-lowest-order DFT coefficients (473-1, 573-1) or the third-lowest-order DFT coefficients (473-2, 573-2).

8. The method according to any one of claims 1 to 7, wherein the information indicating the intensity peak width includes the first value and the second value.

9. The method according to any one of claims 1 to 7, wherein the information indicating the intensity peak width includes or is based on a scale of how the first value and the second value relate to each other.

10. The method according to claim 9, wherein the scale is based on a calculated quotient or difference between the first value and the second value.

11. The method according to any one of claims 1 to 10, wherein the pixels containing the intensity peaks are selected by a region of interest (ROI) determined to partially cover the image sensing area (351) and to cover the respective intensity peaks (363) in each row (352-m) of the pixel elements of the image sensing area (351).

12. The method according to any one of claims 1 to 11, wherein the method is performed by one or more devices which are the image sensor described in claim 1, or include the image sensor, and / or are part of the imaging system described in claim 2 or 3, or correspond to the imaging system.

13. One or more devices (205, 231, 331, 1000) for providing information indicating the intensity peak width (371, 471, 571) of a pixel (366, 466, 566) along a pixel line (352-m) of the intensity peak (363, 463, 563), the pixel being an image sensor (231, 331) Corresponding to the pixel elements of the sensing area (351), the pixel intensity values ​​of the pixels are obtained from the exposure of the image sensing area (351) as part of image sensing by the image sensor (231, 331) of light reflected from the surface, and the peaks of the intensity (363, 463, 563) are formed from such reflected light sensed by the image sensor (231, 331), and one or more devices (205, 231, 331, 1000) Calculating a first value (601) that includes the magnitude of lower-order discrete Fourier transform (DFT) coefficients (473, 573) for the pixel intensity values ​​of pixels (366, 466, 566) along the pixel line (352-m) containing the intensity peaks (363, 463, 563), Calculating a second value (602) that includes the magnitude of another higher-order DFT coefficient (473, 573) relating to the pixel intensity values ​​of pixels (366, 466, 566) along the pixel line (352-m) containing the intensity peaks (363, 463, 563), wherein higher-order is a higher-order DFT coefficient (473, 573) than lower-order DFT coefficient (473, 573), One or more devices (205, 231, 331, 1000) configured to provide (603) the information indicating the intensity peak width (371, 471, 571) based on the first and second values.

14. One or more computer programs (1003) that, when executed by one or more processors, include instructions causing one or more devices to perform the method described in any one of claims 1 to 12.

15. One or more carriers comprising one or more computer programs (1003) as described in claim 14, wherein the carriers are one or more of the following: electronic signals, optical signals, wireless signals, or computer-readable storage media (1101).

Images