Apparatus and methods for use with image sensors in 3D imaging systems based on optical triangulation, providing information indicating the width of intensity peaks.

Abstract

An apparatus and a method performed by one or more devices (205; 231; 331; 1000) for providing information indicating the width (371; 471; 571) of the intensity peak (363; 463; 563) of an intensity peak (366; 466; 566) along a pixel line (352-m) including an intensity peak (363; 463; 563). The pixels correspond to pixel elements of an image sensing region (351) of an image sensor (231; 331). The intensity peak (363; 463; 563) is formed by reflected light sensed by the image sensor (231; 331). Calculate (601) a first value, which includes the magnitude of a low-order Discrete Fourier Transform (DFT) coefficient (473; 573) relating to the pixel intensity values ​​of pixels (366; 466; 566) along a pixel line (352-m) including intensity peaks (363; 463; 563). Calculate (602) a second value, which includes the magnitude of another high-order DFT coefficient (473; 573) relating to the pixel intensity values ​​of pixels (366; 466; 566) along a pixel line (352-m) including intensity peaks (363; 463; 563). Based on the first and second values, provide (603) the information indicating the width of the intensity peaks (371; 471; 571).

Description

Technical Field

[0001] The embodiments described herein relate to apparatus and methods for providing information indicating the width of intensity peaks and for use with image sensors in 3D imaging systems based on optical triangulation. Background Technology

[0002] Industrial vision cameras and systems used for factory and logistics automation can be based on capturing 3D images, such as 3D images of objects, in three-dimensional (3D) machine vision. 3D images refer to images that also contain "height" or "depth" information, rather than, or at least not only, information about pixels such as intensity and / or color in a two-dimensional (2D) form as in conventional images.

[0003] Typically, each pixel in an image captured by a camera has a position in the image sensor coordinate system that corresponds to the location of the camera and image sensor in the real world, or more specifically, information about the light sensed by the image sensing elements of the image sensor from a location in the real world and which image sensing elements correspond to the pixel. Typically, this is light reflected from the object being imaged, such as the object being sensed. Depending on the camera and system, what light is used, and how the illumination is provided, the sensed light may contain various information about the location where the light is reflected, such as about the location on the object being imaged. Therefore, the pixels that capture the image have a position in the image sensor coordinate system corresponding to their real-world location, such as their location on the object. The sensed light may also contain additional information about the location and the properties of the object at that location, such as information from light intensity, color, reflectivity, scattering, etc. Many 3D machine vision cameras or systems, or 3D imaging systems generally used for 3D imaging, are based on multiple 2D images captured by the camera's image sensor, typically captured sequentially during the scanning of an object. Each such 2D image can contain 3D information about the 2D outline of an object, and therefore the sum of such 2D images can contain 3D information about the entire object, thus forming a 3D image of the entire object. The 3D image can be represented by a "point cloud," where each point corresponds to a position on the object and is associated with 3D coordinates about that point. Each point can also be associated with further information about that point, such as color or other properties associated with the corresponding object point.

[0004] When a pixel has a 3D location instead of "just" a 2D location, it can be named a voxel.

[0005] Line scan image data is generated when the object to be imaged is scanned one line at a time by scanning the object as a plane of light that is projected onto the object and measuring the reflected light from the object.

[0006] A specific case of 3D imaging via scanning is 3D imaging based on optical triangulation, where structured light, such as a specific pattern, typically a light plane or "light sheet," is used to scan the object through and / or by means of this light plane. During the scanning process, light rays are projected onto the object at locations where the light sheet or light plane intersects with it. Lasers are generally preferred, but other light sources capable of providing structured light, such as light planes, can also be used, for example, light sources that provide focused light without excessive dispersion, such as light-emitting diode (LED) based sources. Instead of a light plane corresponding to a "light sheet," a light plane corresponding to an illumination edge (i.e., a light edge) can be used, for example.

[0007] 3D machine vision systems are typically based on optical triangulation. In such systems, a light source illuminates the object with structured light, such as a specific light pattern, generally a light plane as described above. This type of 3D machine vision system or device can be called a system or device for 3D imaging based on optical or light plane triangulation, or simply laser triangulation when using lasers. The light rays projected onto the object are imaged by a camera; that is, the light reflected from the object is imaged. Along the light rays, 3D characteristics corresponding to the contours of the object with height information are captured by optical triangulation. By scanning the entire object in this way, corresponding to a line scan, and involving the movement of the lines and / or the object, the 3D characteristics of the entire object corresponding to multiple 2D contours of the object can be captured, and based on this, a 3D image of the object can be formed as described above. To generate an image of the object's contours during scanning, the image sensor of the camera captures the reflected light from the object, specifically detecting its intensity peaks in the image data. The peaks appear at positions on the object corresponding to where the incident light of the light rays is reflected from the object. The location of the detected peak in the image will be mapped to the location of the light reflected from the object, resulting in the peak, based on the optical triangulation performed by the system and configured.

[0008] Peak finding algorithms typically operate in the digital domain and aim to find the center location of the light distribution peak within a sub-pixel resolution. This means that an image with intensity peaks must first be read from the image sensor, which includes analog pixel values ​​corresponding to the light sensed by sensor elements that correspond to the pixels of the image sensor, and the image must be converted from analog to digital (A / D).

[0009] As recognized above, intensity peaks contain information of interest. However, this is not solely due to the ability to locate peak positions at sub-pixel resolution, as described above. In optical triangulation applications, the intensity peak width also carries information about the surface from which the light causing the intensity peak is reflected. More specifically, when using intensity peaks in optical triangulation to determine the 3D location of a point on the surface of an imaged object, the intensity peak width contains information about how that point influences the light it reflects and the light that causes the intensity peak—for example, how the point scatters light. Therefore, the intensity peak width is, for example, a measure of light scattering, which is of interest to know and / or used in many optical triangulation applications.

[0010] Therefore, for example, since it is a measure of light scattering, it is of interest to be able to obtain an indication of the intensity peak width, such as a measure and / or estimate. Furthermore, it is of interest to be able to provide such an indication in a robust and efficient manner. Note that in the following text, the measure of the intensity peak width of the imaging ray and the measure of light scattering can be used interchangeably.

[0011] An example of an existing technique for estimating intensity peak width is to sum the number of pixels at a certain distance from the maximum peak intensity on both sides of each column of an image being imaged from an intensity line obtained from optical triangulation. This method can be called "offset scattering." This method is sensitive to noise in the light, such as speckle noise when the light is laser. Adjusting the method to obtain "good" data, including setting the distance and number of pixels to be summed, is also difficult. The method also relies on the availability of the pixels to be summed in the image dataset around the maximum intensity.

[0012] On the other hand, in EP1985969, scattering data indicating the width of intensity peaks is estimated using the zeroth and second moments. This method can be called "moment scattering." This method is useful when calculating 3D data using the centroid algorithm, especially when determining the location of intensity peaks with sub-pixel resolution. In this case, the zeroth moment is already available because it has been calculated for 3D data computation. Only the second moment needs to be calculated and then used together with the zeroth moment to estimate the scattering data. This method is less effective when centroid calculation is not used.

[0013] Furthermore, it is generally desirable to eliminate or reduce latency, thereby promoting or enabling higher throughput and / or lower latency in 3D imaging systems, for example, by providing improvements on how quickly optical triangulation-based 3D imaging systems can provide 3D images of objects and related information, including, for example, estimates of intensity peak width and scattering data, and / or enabling or better supporting high-speed or higher-speed applications. Summary of the Invention

[0014] In view of the foregoing, one object of the present invention is to provide one or more improvements or alternatives to the prior art, such as providing further alternatives and / or more efficient and / or robust methods, to provide, for example, an indication, measurement and / or estimation, of the intensity peak width of an intensity peak line in image data, particularly from image sensor data of a camera used in a 3D imaging system based on optical triangulation.

[0015] According to a first aspect of the embodiments herein, this objective is achieved by a method performed by (a plurality of) devices, the method being used to provide information indicating the width of an intensity peak along a pixel line comprising an intensity peak and having intensity pixel values ​​as part of the intensity peak. The pixels correspond to pixel elements of an image sensing region of an image sensor, and their intensity pixel values ​​have been obtained from exposure of the image sensing region as part of image sensing performed by the image sensor on light reflected from a surface. The intensity peak is formed by this reflected light sensed by the image sensor. The devices calculate a first value comprising a magnitude of a low-order Discrete Fourier Transform (DFT) coefficient relating to the pixel intensity values ​​of the pixels along the pixel line comprising the intensity peak. The devices calculate a second value comprising a magnitude of another high-order DFT coefficient relating to the pixel intensity values ​​of the pixels along the pixel line comprising the intensity peak. The high-order DFT coefficient is a DFT coefficient of a higher order than the low-order DFT coefficient. The devices provide the information indicating the width of the intensity peak based on the first and second values.

[0016] According to a second aspect of the embodiments herein, this objective is achieved by means of a plurality of devices for providing information indicating the width of an intensity peak along a pixel line of a pixel, the pixel line of which includes an intensity peak and has an intensity pixel value as part of the intensity peak. The pixels correspond to pixel elements of an image sensing region of an image sensor and have their intensity pixel values ​​obtained from exposure of the image sensing region as part of image sensing performed by the image sensor on light reflected from a surface. The intensity peak is formed by this reflected light sensed by the image sensor. The devices are configured to calculate a first value comprising a magnitude of a low-order Discrete Fourier Transform (DFT) coefficient of the pixel intensity value along the pixel line including the intensity peak. The devices are also configured to calculate a second value comprising a magnitude of another high-order DFT coefficient of the pixel intensity value along the pixel line including the intensity peak. The high-order is a DFT coefficient of a higher order than the low-order DFT coefficient. Furthermore, the plurality of devices are configured to provide the information indicating the width of the intensity peak based on the first value and the second value.

[0017] According to a third aspect of the embodiments herein, this objective is achieved by one or more computer programs including instructions that, when executed by one or more processors, cause (a plurality of) devices to perform the method according to the first aspect.

[0018] According to a fourth aspect of the embodiments herein, this objective is achieved by one or more carriers including one or more computer programs according to the third aspect.

[0019] The embodiments herein are based on the understanding that the first and second values, which include or correspond to the magnitudes of the lower-order and higher-order DFT coefficients, and more particularly how they relate to each other, include information indicating the width of the intensity peak. As explained in the background art, the width can in turn indicate light scattering from a point on the surface that causes reflection leading to the intensity peak. Intensity peak estimation based on the embodiments herein has shown the ability to provide results at least as good as other existing methods, such as "moment scattering" mentioned in the background art. In some cases and / or for some applications where the width of the intensity peak is of interest, a particular method for estimating the intensity peak width may be preferred and / or more advantageous than another method, for example, depending on the information required, the information already available, the type of equipment involved, other actions being performed, etc. Therefore, it is generally advantageous to utilize an alternative method to provide information indicating (e.g., by way of estimation) the width of the intensity peak.

[0020] Furthermore, the first and second values ​​can be obtained through computation, which can be performed by a relatively simple operation that can be performed directly, either entirely or partially, on the image sensor without first reading the image data from the image sensor. This can be particularly advantageous if the image sensor performs the computation to calculate the first and second values, for example, by calculating the corresponding pixel columns for each image sensing region, where the pixel lines correspond to pixel columns, because it enables fast and parallel processing per column on the image sensor (typically "on-chip") and provides the results for each column, such as the aforementioned information. This means that, already combined with providing image data from the image sensor, the information indicating the width of the intensity peak can be provided efficiently and quickly, and / or if, for example, only the information is desired, there is no need to transmit image data from the image sensor at all. Attached Figure Description

[0021] Examples of embodiments described herein are illustrated in more detail with reference to the accompanying schematic diagrams, which are briefly described below.

[0022] Figure 1 An example of a prior art 3D imaging system based on optical triangulation is illustrated schematically.

[0023] Figure 2An example of an imaging system for 3D imaging based on optical triangulation is illustrated schematically. The embodiments described herein can be performed using this imaging system, and / or the imaging system can be configured to perform the embodiments described herein.

[0024] Figure 3A Examples of image sensors are illustrated schematically. The embodiments described herein may be performed in conjunction with an image sensor and / or the image sensor may be configured to perform the embodiments described herein.

[0025] Figure 3B Schematic illustration along Figure 3A The intensity peaks of the columns in the image sensing region.

[0026] Figure 4A -B refers to the first example of intensity peaks composed of pixel sequences and DFT coefficients calculated for pixel sequences.

[0027] Figure 5A -B relates to a second example of intensity peaks formed by a sequence of pixels and DFT coefficients calculated for the pixel sequence.

[0028] Figure 6 This is a flowchart used to schematically illustrate and exemplify a first method according to embodiments herein.

[0029] Figure 7-9 shows an example of how low-order and high-order DFT coefficients are correlated with each other and how this will map to different measures of intensity peak width.

[0030] Figure 10 It is used to illustrate how one or more devices can be configured to perform actions related to... Figure 6 A schematic block diagram illustrating an embodiment of the described method and action.

[0031] Figure 11 This is a schematic diagram illustrating an embodiment related to a computer program and its carrier. Detailed Implementation

[0032] The embodiments described herein are exemplary embodiments. It should be noted that these embodiments are not necessarily mutually exclusive. Components of one embodiment may be implicitly assumed to exist in another embodiment, and how these components can be used in other exemplary embodiments will be apparent to those skilled in the art.

[0033] Figure 1An example of this type of imaging system mentioned in the background art is schematically shown, namely, imaging system 105 for 3D machine vision, or simply 3D imaging based on optical triangulation, i.e., imaging for capturing information about the 3D characteristics of an object using camera 130. In the figure, system 105 is in normal operating condition, i.e., typically after calibration has been performed, and the system is thus calibrated. System 105 is configured to perform optical triangulation, here in the form of an optical triangulation sheet mentioned in the background art. System 105 also includes a light source 110, such as a laser, for illuminating the object to be imaged with structured light, said structured light being, for example, a specific light pattern, typically a light plane 111 corresponding to the light sheet as shown. The light is typically a laser, but can alternatively be, for example, light from one or more light-emitting diodes (LEDs). An alternative to the light plane is a light edge, i.e., the edge of an illuminated area, but with a similar effect. The generated light and illumination are typically provided by one or more lenses of camera 130, for example, to focus the light. Furthermore, camera 130 is typically configured and positioned such that it, based on the so-called Scheimfler principle or Scheimfler focusing, will have a focal plane located at the same position as the light plane 111, in other words, a focal plane aligned with the light plane 111. Thus, object reflections occurring in the light plane will be focused in the image sensor. In the example shown, the objects undergoing imaging are exemplified by a first object 120 in the form of a car and a second object 121 in the form of a gear. When the light plane 111 is incident on the object, this corresponds to the projection of the light plane 111 onto the object, which can be viewed as the light plane 111 intersecting the object. For example, in the example shown, the light plane 111 results in light ray 112 on the object 120. The light is reflected by the object, more specifically, partially reflected by the object at the intersection point, i.e., light ray 112 in the example shown. Camera 130 includes an image sensor (not shown). The camera and image sensor are arranged relative to the light plane 111 such that the light plane 111, when reflected by the object, becomes incident light on the image sensor via camera 130. An image sensor is a device, typically implemented as a chip, used to sense incident light and convert it into image data, such as intensity values ​​of light sensed by sensing elements corresponding to pixels in the image sensing area of ​​the image sensor. For example, in the example shown, a light plane 111 will reflect light rays 112 on a portion of the roof of an object 120 toward a camera 130 and an image sensor, thereby generating and providing image data with information about said portion of the roof.Knowing the setup of system 105 (including its geometry), such as how image sensor coordinates relate to real-world coordinates (e.g., x, y, z coordinates in a coordinate system 123 (e.g., a Cartesian coordinate system) associated with the imaged object and its context), image data can be converted into information about the 3D characteristics (e.g., 3D shape or contour) of the imaged object, in an appropriate format. Information about these 3D characteristics (e.g., the 3D shape or contour) can include data describing the 3D characteristics in any suitable format.

[0034] By moving, for example, light source 110 and / or the object to be imaged, such as first object 120 or second object 121, so that multiple parts of the object are illuminated and reflected light is induced on the image sensor, image data describing a more complete 3D shape of the object can be generated, such as multiple consecutive contours of the object, such as the outline images 141-1–141-N of the first object 120, where each contour image shows the contour of the first object 120, wherein the contour images are generated when the image sensor of camera 130 senses light, and light plane 111 is reflected. As shown, conveyor belt 122 or the like can be used to move the object through light plane 111, while light source 110 and camera 130 are typically fixed. Alternatively, light plane and / or camera 130 can be moved over the object so that all parts of the object, or at least all parts facing light source 110, are illuminated, and the camera can receive light reflected from different parts of the object to be imaged.

[0035] As understood above, image frames of, for example, the first object 120 provided by camera 130 and its image sensor can generate any one of contour images 141-1–141-N. As described in the background section, each position of the contour of the first object shown in any of the contour images 141-1–141-N is typically determined based on the identification of intensity peaks in the image data captured by the image sensor and based on finding the positions of these intensity peaks, for example, through one or more intensity peak finding algorithms. System 105 and conventional peak finding algorithms are typically configured to search for intensity peaks in each pixel column in each image frame. If the sensor coordinates are u, v, then as shown, they can be along an image sensor row and used to indicate the position in that row, for example, corresponding to an image sensor column. Accordingly, v can be along an image sensor column and used to indicate the position in that column, for example, corresponding to an image sensor row.

[0036] For each position u in the image frame, a peak position along v can be searched, for example, by means of a peak-finding algorithm as described above, and the identified peaks in the image frame can generate one of the contour images 141-1 to 141-N as shown in the figure. The contour image is formed by image points u, v, t in a sensor-based coordinate system 143, which, as described above, relate to real-world coordinates, such as coordinates x, y, z in coordinate system 123. The sum of the image frame and the contour image can be used to create a 3D image of the first object 120, for example, in the form of a “point cloud” as mentioned in the background art, where each point corresponds to a position on the object and is associated with 3D coordinates about that point.

[0037] As an improvement to the embodiments herein, the situation described in the background art will first be described in further detail.

[0038] A reasonable assumption, supported by actual findings, is that the light used in optical triangulation, typically laser light, produces an intensity peak line with a Gaussian shape, and therefore a normally distributed cross-section. It can then be mathematically shown that the Discrete Fourier Transform (DFT) applied to the pixels around the maximum intensity of the intensity peak also has this shape. In other words, the DFT is applied to each pixel covering the intensity peak, and thus to pixels with image data including the intensity peak. Furthermore, mathematically, there is a relationship between the width of this Gaussian-shaped ray, “the sigma of the light,” and the width of the Gaussian-shaped DFT, “the sigma of the DFT.” Moreover, it can be seen that if the magnitude of the lowest-order DFT coefficient (DFT0 or zero-order DFT coefficient) is divided by the magnitude of the second lowest-order DFT coefficient (DFT1 or first-order DFT coefficient), i.e., DFT0 / DFT1, this is a measure that actually corresponds to an estimate of the intensity peak width, “the sigma of the light.” As described in the background section above, this is again a measure of light scattering caused by the object point reflecting the light that leads to the intensity peak. Typically, the relationship between any higher-order and lower-order DFT coefficients, when both exist as substantial coefficients, will contain information that can be used as an indication of the intensity peak width. For example, alternatives could include calculating any one of DFT0 / DFT2, DFT1 / DFT2, DFT0-DFT1, DFT0-DFT2, etc. However, using lower-order coefficients is generally preferred and can provide a better width estimate over a wider range. In any case, using DFT0 is advantageous because it always exists as a substantial value, which can be simply calculated by adding the pixel values ​​of the pixels surrounding the maximum intensity of the intensity peak, which is therefore the pixel involved in and being calculated by the DFT.

[0039] The above has been conducted in actual tests with good results, and some examples of the mathematical relationships mentioned are further provided below. Some tests have also been performed on 3D imaging based on optical triangulation of objects with wooden surfaces. Wood has a surface that naturally varies in influencing light scattering; therefore, it is of particular interest to be able to estimate light scattering by estimating the width of the intensity peak produced by optical triangulation. Light scattering contains information, for example, about the type and quality of the wood and how it might vary for a wooden object, and at what location on the object.

[0040] The method for obtaining an indication of intensity peak width, such as measurement and / or estimation, based on the aforementioned DFT, and thus an indication of light scattering, can be called "DFT scattering" to distinguish it, for example, by name, from other methods mentioned in the background art, such as "offset scattering" and "moment scattering." The DFT-based method, i.e., "DFT dispersion," has several advantages over other methods, such as:

[0041] Compared to offset scattering, it enables more robust peak width estimation, for example because it handles laser speckle better when / if the laser is used as the light for optical triangulation.

[0042] It does not require using a fixed distance from the pixel with the highest detected intensity.

[0043] Based on experimental results, peak width estimation can be at least as good as that obtained from "moment scattering" and does not require any "centroid" calculation.

[0044] It can be performed using relatively simple calculations and can be implemented computationally efficiently on or closely related to image sensors.

[0045] Therefore, the DFT-based, or in other words, “DFT scattering” method disclosed above and herein enables the estimation of intensity peak width and the corresponding light scattering measurement, which is paired with “moment scattering” and, at least in some cases, may be computationally more advantageous than “moment scattering”.

[0046] Figure 2An example of an imaging system 205 for 3D imaging based on optical triangulation is schematically illustrated, which can be used to perform the embodiments described herein, and / or can be configured to perform the embodiments described herein. Imaging system 205 may structurally correspond to imaging system 105, but is shown herein in a more simplified manner. The figure also illustrates the context in which an image sensor 231 configured to perform according to embodiments described herein can be used, as further described below. Image sensor 231 is part of camera 230 included in imaging system 205 and can be used for 3D imaging based on optical triangulation, and in this context, to perform the embodiments described herein.

[0047] Therefore, the diagram shows:

[0048] Object 220, which may correspond to object 120, is shown as being at least partially within the field of view 232 of camera 230.

[0049] The light source 210 of the imaging system 205 is configured to illuminate the object 220 with light 211 for 3D imaging based on optical triangulation, typically in the form of structured light, such as a specific light pattern, causing ray(s) to be formed on the object 220 and reflected therefrom, and then captured by a camera 230 having an image sensor 231. For example, the illumination of light 211 can be in the form of a "light sheet," i.e., a light plane, thereby producing light rays on the object corresponding to the locations where the light plane intersects with the object. Another example of structured light that can be used as the first light is a light edge, i.e., the edge of an area or portion that is illuminated. For example, light from an LED can be used instead of light from a laser. In this example, the illumination is in the vertical direction, i.e., substantially parallel to the z-axis shown in the figure, so the light plane is in the zx plane. However, one or more other illumination directions can of course be used with the embodiments herein.

[0050] A computing device 233, such as a computer or similar device, may be part of the imaging system 205. It may be connected to the camera 230 and / or the light source 210, and may be configured to receive and process image-related data and information via the camera 230 and / or from the image sensor 231. Furthermore, the computing device 233 may be configured to control the imaging system 205 and / or its camera 231 and / or image sensor 231 and / or light source 210. The computing device 233 may also be configured to perform some actions relevant to the embodiments herein and / or be included in some actions relevant to the embodiments herein, as further described below. Alternatively or additionally, the computing device 233 or its corresponding functionality may be wholly or partially integrated into the same unit as the camera 230, and the camera 630 may then be considered to have this functionality.

[0051] The object 220 can therefore be illuminated by light 211, and the image is captured by camera 230 with image sensor 231, as in conventional 3D imaging based on optical triangulation.

[0052] Accordingly, as in an imaging system based on conventional optical triangulation, imaging system 205 can be configured to move light source 210 and / or its provided light 211 and / or object 220 relative to illumination by light 211 provided by light source 210, such that different consecutive portions of object 220 are illuminated by light 211 at different consecutive moments. To achieve this, object 220 can be moved as part of transport, for example by a conveyor belt or the like. After reflection from object 220, image sensor 231 senses the light via camera 230. A corresponding image frame is associated with the corresponding moment when the image frame is sensed, i.e., captured, and with a corresponding portion of object 220 from which image sensor 231 senses reflected light 211 at the corresponding moment.

[0053] Image frames and / or information derived from image frames (e.g., intensity values ​​of pixel portions of intensity peaks of light imaged as part of light triangulation, intensity peak positions, and / or other information about intensity peaks (e.g., about their width)) may be provided by camera 230 and / or image sensor 231 and may be transmitted for further processing within and / or outside camera 230, such as to computing device 233.

[0054] A region of interest (ROI), particularly a window at or around its maximum value (WAM), can be advantageously used with the embodiments herein. WAMs are further explained and discussed below. The image sensor involved in the embodiments herein, such as image sensor 231, can be implemented and / or have the corresponding functionality disclosed in EP4266673A1 to facilitate the use of a WAM as an ROI.

[0055] Before proceeding with further details and examples of embodiments described herein, some examples and discussions in the context of images and image sensors are provided for better understanding. Images may be generated by camera 230 or image sensor 231 in imaging system 205. More specifically, these examples and discussions relate to the aforementioned imaging rays, their intensity peaks, and WAM.

[0056] Figure 3AAn example of an image sensor 331 is schematically shown, having an image sensing area 351 formed by all pixels, displayed as squares. Image sensor 331 is an image sensor that embodiments of this document can use to perform and / or can be configured to perform embodiments of this document. The pixels of image sensor 331 correspond to photosensitive elements and can be of a conventional type. In this example, all pixels are arranged in N rows and M columns, where N = 12 in the illustrated example. Therefore, there are 352-1...352-M columns and 354-1...352-12 rows. The view shown is broken along the rows.

[0057] In the image sensing region 351, an intensity peak line 361 has been drawn, and the shaded pixels represent the ROI as a WAM formed by 5 pixels per column. Therefore, the WAM height 362 in this example is 5 pixels and centered on the intensity peak line 361. Of course, if such a line or WAM is used, it cannot be seen by visual inspection of the corresponding real image sensing region 351. The intensity peak line 361 and WAM are drawn only to facilitate understanding how intensity peaks and WAMs can be correlated with each other and how they relate to the image sensor. The intensity peak line 361 corresponds to the location where the intensity peak can be located at sub-pixel resolution and can be generated by the exposure of the image sensor 331 when the image sensor 331 is part of a camera in a 3D imaging system based on optical triangulation (such as imaging system 205). If, after exposing the image sensor 331 to a light intensity line with an intensity peak, an image based on all pixels is read from the image sensor 331, where the light intensity line has a light distribution according to the peak value of the intensity peak line 361, then the light intensity line can be seen in the image, and the intensity peak line 361 can be detected therein. Therefore, when the object has been illuminated during optical triangulation, such an intensity line and intensity peak line 361 can be generated by reflected light from the object (e.g., object 220).

[0058] In practice, each pixel typically retains only a single pixel value corresponding to the intensity value. The WAM in each column 352 is centered around, and thus around, the pixel detected in that column, to retain the maximum pixel value corresponding to the maximum intensity. Such pixels are marked with thick lines in the diagram. All local WAMs, i.e., the WAMs of each column of all columns M, form the total WAM centered around and thus around the intensity ray 361.

[0059] In the example, there are N=12 pixel rows 0..11, which correspond to image sensor rows 354-1..354-12, and M pixel columns 1..M, which correspond to image sensor columns 352-1..352-M.

[0060] Typically, pixel values ​​are read out in parallel row by row from image sensor 331; that is, the pixel values ​​of pixels in the same row are read out in parallel and then converted to analog-to-digital (A / D) converters. As indicated in the figure, M columns 352 can be connected to M parallel A / D converters (ADCs) 356. An alternative could be to read out pixels with the same WAM height position in parallel, which means... Figure 3A All the shadow pixels, i.e., the pixels corresponding to WAM, can be read out in five consecutive parallel readouts. A detailed example of this implementation is disclosed, for example, in EP4266673A1.

[0061] Following the A / D conversion, and thus in the digital domain, there may be processing and / or computational circuitry, such as computational circuitry 357 schematically shown in the figure, which operates, for example, on the obtained digital image data, such as on each column and / or on pixels and their values ​​from multiple columns. Therefore, parallel computation can be present. The corresponding parallel computation can be with respect to the pixel values ​​of pixels from the corresponding columns. For example, computations related to DFT coefficients can be performed in parallel on all columns. There may also be memory elements involved in temporarily storing pixel values ​​and intermediate data during such processing and / or computation. Such memory elements enable the image sensor to perform processing and computation on pixel values ​​from several pixels in a corresponding column, such as all pixels of WAM, or another number of pixels covering the intensity peaks of each column, such as the intensity peaks of each column around intensity peak line 361.

[0062] Image sensor 331 also includes, for example, some input / output (I / O) circuitry 358 involved in the input for controlling the image sensor, so as to enable the reading of image data from image sensor 331 after A / D conversion and / or to enable the reading of information generated by calculations, such as those performed by computing circuitry 357. When the image sensor provides something for reading from the image sensor, this is via I / O circuitry, such as I / O circuitry 358.

[0063] The figure also shows control circuitry 359, which relates to the control of image sensor 331 and its components, such as addressing and / or selecting columns and / or rows and / or pixels to be converted by A / D. Control circuitry 359 may also involve temporarily storing information about the ROI, such as the WAM in use, which pixels are involved in the WAM, etc.

[0064] Figure 3BAn intensity peak 363 with a light distribution 364 is schematically shown. The light distribution, captured by the intensity pixel values ​​of the pixels including the intensity peak, can be viewed as a sampled light distribution, the intensity peak thus corresponding to a cross-section of the intensity peak line 361. The intensity peak runs along the pixel line corresponding to column 352-m, which can be any of columns 352-1...352-M. The light distribution 364 is shown along axis v, i.e., in the direction corresponding to along the column, as in the case of column 352-m.

[0065] The intensity values ​​of the pixels along column 352-m are shown as small black squares in the image. It can be seen that the intensity values ​​represent intensity I and belong to different rows, i.e., at different row coordinates v, and therefore vary along the column. The intensity values ​​appear according to the light distribution 364, exhibiting a Gaussian "bell shape" as described above.

[0066] The image shown is from a single column (column 352-m), and the WAM 366a of this column is marked, therefore WAM366a corresponds to Figure 3A The shadow pixels in the column. Therefore, WAM 366a has a WAM height 362 of 5 pixels. As can be seen, for the light distribution 364 in the example, 5 pixels do not provide such good coverage of the intensity peak 363. If the entire peak is viewed according to the light distribution 364 shown, WAM 366a hardly covers the full width 371 of the peak at its half maximum. Another pixel sequence in column 352-m is also shown in the figure, corresponding to the pixels of WAM 366b covering 15 pixels. For the specific application of the embodiments herein, it may be preferable to use a WAM or corresponding pixel sequence that covers the intensity peak, which will cover the widest intensity peak excluded by a certain margin for an application, and the embodiments herein should be able to handle the application. In practice, it can be suggested to ensure coverage of at least twice the FWHM, which is the FWHM that is expected to be handled, and therefore corresponds to the maximum FWHM for the application. For example, such that on each side of the maximum intensity pixel of the intensity peak, i.e., around it, several pixels corresponding to the maximum FWHM are covered.

[0067] Each column of WAMs and the total WAM formed by multiple WAMs in each column are used not only to select pixels to be used with the embodiments herein. As mentioned above, a more common use is to select pixels around intensity peaks to use them to find peak locations with sub-pixel resolution. For this purpose, a WAM height should be used that results in sufficient coverage of the light distribution 364 so that the peak-finding algorithm can use the light distribution 364 to find the actual center location of the light distribution 364 at sub-pixel resolution. After the digital image data has been read from the image sensor, the peak-finding algorithm is typically used outside the image sensor. As shown in the figure, and Figure 3AAs shown, the actual peak position is offset from the center position of the pixel with the maximum intensity value, and the WAM is centered around this center position. In practice, for this purpose, a WAM height of approximately 75% or more covering the energy in the intensity peak should generally be used. For typical sensors and settings, this can mean a WAM height in the range of 7-30 pixels. However, there may be cases where less than 7 and greater than 30 pixels can also be used. Covering 75% of the energy is generally sufficient for the purposes of the embodiments described herein. For applications interested in using WAMs for both purposes, using a single, consistent total WAM and a WAM height sufficient for both purposes is generally advantageous.

[0068] As understood from the above, when calculating the DFT, or more precisely, calculating the magnitude (or “quantity”) of some DFT coefficients, as in the embodiments herein, this can be done for each column of pixels corresponding to the WAM or for a sequence of corresponding pixels that fully cover the intensity peaks. By definition, the WAM will involve pixels covering the intensity peaks of each column, and is therefore of interest to “DFT scattering” and the embodiments herein. A WAM having the same number of pixels belonging to the WAM in each column, corresponding to a WAM height of 362, means that the same number of calculations can be performed on each column and in a corresponding manner to calculate the DFT coefficients. This is advantageous for the embodiments herein, and also advantageous if the available WAM functionality is not explicitly used.

[0069] While not mandatory, using WAM can be advantageous for the embodiments described herein, when WAM is available and / or supported by the image sensor. A sequence of pixels covering the intensity peak can be selected, and the magnitudes of the DFT coefficients for these pixels can be calculated without using the image sensor's WAM and / or WAM functionality. For example, the magnitudes of the DFT coefficients involved in the embodiments herein can be calculated for pixels that cover or include the intensity peak but are not typically referred to as WAM pixels. Even different numbers of pixels in each column can be used to calculate the corresponding DFT coefficients, and / or for different DFT coefficients, although this offers no significant advantage, as it would require performing unnecessary additional computations, such as for consistency and to make it easier to compare results between columns.

[0070] Figure 4A -B relates to a first example of the magnitudes of an intensity peak 463 composed of a pixel sequence 466 and DFT coefficients 473 calculated for the pixel sequence. This example is used to illustrate the relationship between the magnitudes of the DFT coefficients on which the embodiments herein are based, and will be used in the following further examples relating to the embodiments herein.

[0071] Figure 4A In Figure 3BThe corresponding view shows an intensity peak 463 with normalized intensity values ​​for pixel 466, including an intensity peak 463. In the example, pixel 466 is 31 pixels located along v at integers -15 to +15 (inclusive). As can be appreciated from the illustration, in this example, only 5 pixels of pixel 466 have pixel values ​​that actually contribute to peak 463, which appears rather narrow and "sharp," with straight lines binding the values ​​of adjacent pixels together, and the peak does not have a distinct Gaussian shape distribution. The width of intensity peak 463 is represented by the full width at half maximum (FWHM) 471 of the peak. FWHM is the peak width at half the maximum intensity of the peak. FWHM can be a suitable width measurement for the embodiments herein, but width measures at other peak heights or some other measure indicating width can also be used. However, for ease of comparison, and for example, for comparison between different widths that may correspond to different degrees of light scattering from different object locations, it is preferable to use the same type of width measure. Hereinafter, by way of non-limiting example, the type of width measure used is FWHM. FWHM 471 corresponds to a width of 2 pixels and is an example of a narrow width.

[0072] Figure 4B The size of the DFT coefficients calculated for intensity peak 463, i.e., for pixel 466, is shown. It can be seen that the DFT has a length and therefore a number of coefficients corresponding to the number of samples computed, here corresponding to the number of pixels 466. The first three DFT coefficients, starting from the lowest order, are explicitly labeled: DFT coefficient 473-0 is the lowest-order DFT coefficient, DFT0; DFT coefficient 473-1 is the second-lowest-order DFT coefficient, DFT1; and DFT coefficient 473-2 is the third-lowest-order DFT coefficient, DFT2.

[0073] Figure 5A -B relates to a second example of the magnitudes of the intensity peak 563 captured by the pixel sequence 566 and the DFT coefficients 573 calculated for the pixel sequence. This example is used to illustrate the relationship between the magnitudes of the DFT coefficients on which the embodiments herein are based, and will be used in other examples below related to the embodiments herein.

[0074] Figure 5A In such Figure 3B and Figure 4A The corresponding view shows intensity peak 563, which includes the normalized intensity value of pixel 566, with intensity peak 563. In the second example, pixel 566 is also 31 pixels located along v at integers from -15 to +15, including 0, i.e., with... Figure 4AThe number of pixels is the same. It also appears to involve the same pixels, but this should be considered an example effect. In reality, especially if the peaks originate from 3D imaging measured via optical triangulation, different peaks typically appear at different locations. Figure 4A and 5A The peaks shown could also be from different columns and appear simultaneously on the image sensing area 351. In any case, as should be appreciated from the illustrations, there are more pixels in this example, approximately all 31, with pixel values ​​that actually contribute to peak 563, which here has a typical Gaussian "bell shape," even visible using a straight line that binds the values ​​of adjacent pixels together. The width of intensity peak 563 is represented by the peak's FWHM 571. FWHM 571 corresponds to a width of 12 pixels and can be compared to the significantly smaller and narrower FWHM 471 of intensity peak 463.

[0075] Figure 5B The values ​​of the DFT coefficients calculated for intensity peak 563, i.e., for pixel 566, are shown. It can be seen that the DFT here also has a length, and therefore a number of coefficients corresponding to the number of samples calculated, here corresponding to the number of pixels 566. The first three DFT coefficients, starting from the lowest order, are explicitly labeled: DFT coefficient 573-0 is the lowest-order DFT coefficient, DFT0; DFT coefficient 573-1 is the second lowest-order DFT coefficient, DFT1; and DFT coefficient 573-2 is the third lowest-order DFT coefficient, DFT2.

[0076] The DFT can generally be considered to correspond to the “frequency” content of the signal it is calculating. Due to the rather narrow, “sharp” intensity peak 463 corresponding to a signal with more “high-frequency” components, as can be seen and as expected, there are more higher-order DFT coefficients compared to the wider and “duller” light distribution of the intensity peak 563.

[0077] Figure 6 This is a flowchart illustrating and demonstrating a method according to embodiments of those described above, but which will now be disclosed in some further details. The following actions can form a method for providing information indicating the width of an intensity peak (e.g., any one of intensity peaks 363, 463, 563) of an intensity peak (e.g., any one of intensity peaks 363, 463, 563). The intensity peak is along a pixel line (e.g., a pixel column, such as column 352-m), for example, in or part of a pixel line having pixels that include the intensity peak and have intensity pixel values ​​as part of the intensity peak. For example, the pixel can be any one of the sequence of pixels 366, 466, 566.

[0078] The pixels also correspond to pixel elements of an image sensing region (e.g., image sensing region 351) of an image sensor (e.g., image sensor 231 or 331). The pixels have obtained their intensity pixel values ​​from exposure of the image sensing region as part of the image sensor's image sensing of light reflected from a surface, such as structured light (e.g., light rays), as in the case of 3D imaging based on optical triangulation. Intensity peaks are formed by this reflected light sensed by the image sensor, i.e., the reflected light is sensed as part of the image sensing. Therefore, the intensity peaks correspond to the cross-section of the imaging light rays. The relationships in this paragraph are combined above. Figure 3A -B provides examples and explanations.

[0079] The following methods and / or actions can be performed by one or more devices. As those skilled in the art will recognize, methods having the following actions can be performed in a distributed manner by multiple devices configured to perform the actions and / or involved in performing the actions. However, the one or more devices are preferably or include the image sensor, such as image sensor 231 or 331, and / or are part of or correspond to an imaging system, such as imaging system 205. For example, the one or more devices may be or include computing device 233 and / or camera 230. In the following, image sensor 331 can be used as a non-limiting example of an image sensor and a device performing the method. As understood from the further discussion above, the image sensor of the embodiments herein should in any case be an image sensor in a camera suitable for use as an imaging system for 3D imaging of an object based on optical triangulation.

[0080] therefore:

[0081] In some embodiments, image sensing by image sensor 331 is part of optical triangulation-based 3D imaging performed by an imaging system (e.g., imaging system 205) for 3D imaging of an object (e.g., object 220) having the surface. The imaging system includes a camera, such as camera 230 having an image sensor 231 that may correspond to image sensor 331, and a light source, such as light source 210, for providing illumination to generate the light on the surface as part of the optical triangulation-based 3D imaging. Therefore, the light on the surface, such as light rays, can be part of the optical triangulation-based 3D imaging. In some of these embodiments, intensity peaks, such as any one of intensity peaks 363, 463, and 563, are associated with a location on the surface that reflects light, thus generating the intensity peak. In these embodiments, the information indicating the width of the intensity peak can be provided as an indication of light scattering from that location.

[0082] Note that although it may be preferred from an implementation and / or computational perspective, particularly when an image sensor is involved in performing the embodiments described herein, that the pixel line is a column of pixels, such as a column corresponding to the pixel elements of an image sensor, the pixel line can in principle be any one-dimensional line in any direction and / or corresponding to any pixel, such as any pixel line in any direction on the image sensing area 351 of the image sensor 331, as long as the pixel includes an intensity peak.

[0083] Furthermore, if necessary, the pixel elements of the image sensing region correspond to the photosensitive elements that sense light during exposure. The corresponding pixel value obtained by the corresponding pixel element during exposure typically corresponds to the cumulative amount of light received by the corresponding image sensor pixel during exposure. This amount and pixel value typically correspond to a measure of the light intensity of incident light on the pixel element, or more specifically on its photosensitive portion, during exposure.

[0084] Note that the following actions may be taken in any suitable order and / or performed in a fully or partially overlapping manner in time, as is possible and appropriate.

[0085] Action 601

[0086] Image sensor 331 calculates a first value, which includes, for example, magnitudes corresponding to low-order DFT coefficients relating to pixel intensity values ​​along a pixel line including an intensity peak. The first value may include or correspond to complex-valued low-order DFT coefficients, which include their magnitudes, the magnitudes themselves, or other values ​​that can be derived from them or identify the magnitudes.

[0087] For example, see Figure 4B , 5B The values ​​of the DFT coefficients 473 and 573 are given. The general DFT coefficients, their relationship to the DFT, and how they can be calculated are discussed below. In action 602, the lower-order DFT coefficients are further commented on below.

[0088] Therefore, the pixels involved in the low-order DFT coefficients are related to the intensity peak in question. If the intensity peak is intensity peak 363, then the pixels can be, for example, the pixel sequence 366; if the intensity peak is intensity peak 463, then the pixel sequence 466; if the intensity peak is intensity peak 563, then the pixel sequence 566; but of course, it can be another shorter or longer pixel sequence that includes the intensity peak.

[0089] Action 602

[0090] Image sensor 331 calculates a second value, which includes, for example, a magnitude corresponding to a higher-order low-order DFT coefficient with respect to the pixel intensity value along a pixel line including an intensity peak. The second value may include or correspond to a complex-valued higher-order DFT coefficient, which includes its magnitude, the magnitude itself, or some other value from which the magnitude can be derived or identify the magnitude.

[0091] For example, see Figure 4B , 5B The values ​​of DFT coefficients 473 and 573 are given below. The general DFT coefficients, their correlation with the DFT, and how they can be calculated are discussed below.

[0092] Therefore, the pixels involved in the higher-order DFT coefficients are related to the intensity peak in question, and thus are the same as the intensity peak involved in the lower-order DFT coefficients. Preferably, the pixel is the same as the pixel for which the magnitude of the lower-order DFT coefficients is calculated, but this may not be necessary. Thus, if the intensity peak is intensity peak 363, the pixel could be pixel sequence 366; or if the intensity peak is intensity peak 463, the pixel could be pixel sequence 466; or if the intensity peak is intensity peak 563, the pixel could be pixel sequence 566, but of course, it could be another shorter or longer pixel sequence including the corresponding peak.

[0093] Preferably, the low-order and high-order DFT coefficients include one or more of the following:

[0094] The lowest-order DFT coefficient. This coefficient can be named DFT0, and can be, for example, either DFT0 473-0 or DFT0 573-0.

[0095] The second lower-order DFT coefficient. This coefficient can be named DFT1, and can be, for example, either DFT1 473-1 or DFT1573-1.

[0096] The third lowest DFT coefficient. This coefficient can be named DFT2, and can be, for example, either DFT2 473-2 or DFT2573-2.

[0097] Preferably, the lower-order DFT coefficients are the lowest-order DFT coefficients, DFT0. This coefficient exists and has a considerable magnitude for all peaks, and can be calculated relatively simply compared to others. The magnitude of DFT0 can be easily and advantageously calculated as the sum of the pixel intensity values ​​of the pixels including the intensity peaks, for example, the sum of the pixel intensity values ​​of pixels 366, 466, or 566.

[0098] Preferably, the higher-order DFT coefficients are the second-lowest DFT coefficient DFT1 or the third-lowest DFT coefficient DFF2.

[0099] The general DFT coefficients and how they relate to the DFT, as well as how they can be calculated, are discussed below. Examples and notes can also be found below on how the magnitudes of the corresponding DFT coefficients (lower and higher) can be calculated in practice, for example, via image sensor 331.

[0100] Action 603

[0101] Image sensor 331 provides information indicating the intensity peak width based, for example, using the first and second values, such as information indicating any one of FWHM 371, 471, 571. Therefore, this information should be provided as an indication of the intensity peak width, such as as an indication of the intensity peak width and / or even as a measure and / or estimate of the intensity peak width.

[0102] The information indicating the width may include the first value and the second value, or may include or correspond to or be based on a measure of how the first value and the second value are related to each other (such as how the magnitudes of the DFT coefficients are related to each other).

[0103] The metric may include, correspond to, or be based on how the first and second values ​​(e.g., the magnitudes of the lower and higher-order DFT coefficients) are related to each other, preferably how they differ from each other. The metric may include, correspond to, or be based on a calculated quotient or difference between the first and second values, such as the magnitudes of the lower and higher-order DFT coefficients. It should be appreciated that the quotient also discloses how the values ​​being divided differ from each other.

[0104] In cases where the information representing the width includes both the first and second values, another device outside the image sensor 331 can typically provide (typically, compute) a metric based, for example, on how the first and second values ​​relate to each other, rather than the image sensor 331. This device could be, for example, a computing device 233 or a camera 230.

[0105] For example, if the first and second values ​​are complex-valued low-order and high-order DFT coefficients, the information indicating the width may include these complex values, corresponding to them or based on them. The magnitudes of the DFT coefficients, and / or their complex values, or the differences between their magnitudes, and / or the quotients between their magnitudes, may be calculated outside of the image sensor 331.

[0106] In another example, the corresponding first and second values ​​are formed by adding the squares of the real parts of their DFT coefficients to the squares of the imaginary parts of their DFT coefficients. As can be recognized from Equation 2 below, such values ​​include magnitudes. Information representing the width may include these values, corresponding to or based on them, and may include, for example, the metric calculated as a quotient between the first and second values. This quotient may be calculated outside of the image sensor 331.

[0107] How such quotients and differences can be correlated with (e.g., mapped to) the width of intensity peaks (such as FWHM) is illustrated in the following figures, and further commentary is provided on these figures.

[0108] The metric, calculated by another device based on the first and second values ​​provided by the image sensor or calculated by the image sensor itself, can therefore correspond to a calculated estimate of the width of the intensity peak.

[0109] The embodiments herein are based on the understanding that the first and second values, which include or correspond to the magnitudes of the lower-order and higher-order DFT coefficients, and more particularly how they relate to each other, include information indicating the width of the intensity peak. As explained in the background art, the width can in turn indicate light scattering from a point on the surface that causes reflection leading to the intensity peak. Intensity peak estimation based on the embodiments herein has shown the ability to provide results at least as good as other existing methods, such as "moment scattering" mentioned in the background art. In some cases and / or for some applications where the width of the intensity peak is of interest, a particular method for estimating the intensity peak width may be preferred and / or more advantageous than another method, for example, depending on the required information, the information already available, the type of equipment involved, other actions being performed, etc. Therefore, it is generally advantageous to utilize an alternative method to provide information indicating the width of the intensity peak (e.g., by means of its estimation).

[0110] Furthermore, the first and second values ​​can be obtained through computation, which can be performed by a relatively simple operation that can be performed directly, either entirely or partially, on the image sensor, such as image sensor 331, without first reading the image data from the image sensor. This can be particularly advantageous if the image sensor performs the computation to calculate the first and second values, for example, per corresponding column of pixels in the image sensing region, where rows of pixels correspond to columns of pixels, because it enables fast and parallel processing column-by-column on the image sensor (typically "on-chip"), and provides the result for each column, such as the aforementioned information. This means that, already combined with providing image data from the image sensor, the information indicating the width of the intensity peak can be provided efficiently and quickly, and / or if, for example, only the information is desired, there is no need to transmit image data from the image sensor at all.

[0111] In some embodiments, the pixel including the intensity peak (e.g., identity peak 363) is a pixel of the ROI, preferably a WAM, as discussed above with respect to FIG3, which partially covers the image sensing region, such as image sensing region 351. The ROI has been identified as the corresponding intensity peak, such as intensity peak 363, in the corresponding column (e.g., column 352-m) covering the image sensing region. The ROI, such as the WAM, should have the same number of pixels in the corresponding column, and the image sensor should preferably support providing parallel readout of pixels having the same ROI or WAM location (e.g., the same WAM height location).

[0112] As mentioned above Figure 3A The embodiments discussed herein are advantageously combined with WAM, i.e., ROI or WAM, which covers the imaging rays and their intensity peaks. These are intensity peaks of interest to the embodiments herein, and therefore the embodiments herein can be applied in conjunction with the use of such ROI or WAM to read out image data about the intensity peaks.

[0113] For a given imaging system and / or the setup of the imaging system that generates the image light, a person skilled in the art can discover, for example, how the quotients between the magnitudes of the DFT coefficients, as in the embodiments herein, relate to the intensity peak width (e.g., how the quotients map to the intensity peak width). This can be done, for example, during a calibration process using a predetermined range of intensity peak widths and calculating the quotients of these. In this way, a relationship, such as a mapping, can be provided between the intensity peak width and the peak width. Using a system implementing the embodiments herein, such as its image sensor and / or other device, the quotients between the magnitudes of the DFT coefficients can be calculated during normal operation as in the embodiments herein, and the mapping from the calibration can be used to provide an estimate of the intensity peak width. In some applications, for example, using the quotient only as a metric may be sufficient, and variations, such as relative deviations from the “nominal” quotient, can be observed. A quotient much larger than the nominal value may mean a much larger peak width and, for example, increased light scattering. This, in turn, can indicate, for example, material differences at the surface location where reflected light causes the peak.

[0114] It should also be noted that, as in the embodiments herein, for example when the light is part of a 3D imaging based on optical triangulation, the intensity peak obtained from the imaging light (e.g., from a reflected laser line) will have a “peak shape” whose light distribution will actually be Gaussian or minimal Gaussian. Furthermore, DFT will result in narrow peaks having higher DFT coefficients, which are considerably larger in magnitude compared to wider peaks. It should be understood that this should mean that the principles of the embodiments herein should always work to at least provide an indication of the width of the intensity peak, however, provide an accurate light distribution of the peak.

[0115] Figures 7-9 illustrate how low-order and high-order DFT coefficients correlate with each other and how this maps to different measures of intensity peak width. The DFT coefficients thus correspond to the first and second values ​​described in the embodiments herein, and the measures can correspond to the information indicating the intensity peak width in the embodiments herein. In the examples, the width is represented by FWHM. Different widths have been used for, for example... Figure 4A , 5A The peaks in the figure have different intensities but different FWHM values, and the magnitudes of the DFT coefficients have been calculated for the corresponding peaks and used according to the embodiments described herein. Further details are given in each figure below.

[0116] Figure 7A -C is an example of how the metric is a calculated quotient between the first and second values, i.e., the first value divided by the second value, and how the quotient varies with the intensity peak width. More specifically: Figure 7A This is the case when the first value is calculated as the magnitude of the lowest-order DFT coefficient DFT0, and the second value is calculated as the magnitude of the second lowest-order DFT coefficient DFT1.

[0117] Figure 7B This is the case when the first value is calculated as the value of the lowest-order DFT coefficient DFT0 and the second value is calculated as the value of the third lowest-order DFT coefficient DFT2.

[0118] Figure 7C This is the case when the first value is calculated as the magnitude of the second lowest-order DFT coefficient DFT1 and the second value is calculated as the magnitude of the third lowest-order DFT coefficient DFT2.

[0119] Figure 8A -C is an example from Figure 7, but instead of the quotient, the square root (sqrt) of the natural logarithm (ln) of the quotient is shown. This is to illustrate that this results in a good linear relationship, which may be beneficial in some applications of the embodiments described herein. Furthermore, for example, in the case of a calibration process as described above, the result can be a simple linear equation that can then be used during normal operation to obtain a width estimate.

[0120] Figure 9A -C is an example of how the metric is the calculated difference between the first and second values, i.e., the first value minus the second value, and how that difference varies with the intensity peak width. More specifically:

[0121] Figure 9A This is the case when the first value is calculated as the magnitude of the lowest-order DFT coefficient DFT0, and the second value is calculated as the magnitude of the second lowest-order DFT coefficient DFT1.

[0122] Figure 9BThis is the case when the first value is calculated as the value of the lowest-order DFT coefficient DFT0 and the second value is calculated as the value of the third lowest-order DFT coefficient DFT2.

[0123] Figure 9C This is the case when the first value is calculated as the magnitude of the second lowest-order DFT coefficient DFT1 and the second value is calculated as the magnitude of the third lowest-order DFT coefficient DFT2.

[0124] Although the Discrete Fourier Transform (DFT) itself is well-known, mathematically defined, and has been used in practice for a long time, it is briefly outlined below to set the context for the DFT coefficients on which the embodiments herein are based. Within the context of the embodiments herein, some comments are also made on how the DFT coefficients can be computed.

[0125] The Discrete Fourier Transform transforms a sequence of values, typically corresponding to a sample x[n], into another sequence of complex values ​​X[k], and is defined as:

[0126]

[0127] (Formula 1)

[0128] Where N is the number of values ​​(e.g., samples) in the sequence. In the context of this embodiment, the sequence of values ​​x[n] corresponds to the pixel values ​​of pixels including intensity peaks, for example, having values ​​corresponding to... Figure 4A , 5A The sequence of pixels with intensity I indicated in the text is 466 or 566. Therefore, N is the number of pixels involved, and thus 31 in these examples. The DFT coefficients correspond to X[k], so the lowest DFT coefficient, which has been named DFT0 above, corresponds to X[0] or X0, DFT1 corresponds to X1, DFT2 corresponds to X2, and so on.

[0129] Based on this and Equation 1, as mentioned above, it can be recognized that X0, i.e. DFT0, is simply the sum of x[n] for all N values, and therefore the sum of the pixel values ​​of the pixels including the peak.

[0130] Typically, DFT coefficients are complex values, and their magnitudes can be calculated as follows:

[0131] (Formula 2)

[0132] Where Re refers to the real part and Im refers to the imaginary part.

[0133] For DFT coefficients above X0, i.e., DFT0, a weighting factor must be considered, and this can be written as:

[0134]

[0135] (Formula 3)

[0136] For a given DFT coefficient, say X1, i.e., DFT1, the real part of X1 is implemented with respect to multiplying each x[n] by a cosine weighting factor and summing the results, such that summation is performed for all N x[x], which corresponds to summation for all pixels including the intensity peaks, such as for all 31 pixels of pixel sequence 466. For the imaginary part of X1, it is correspondingly implemented with respect to multiplying each x[n] by its sine weighting factor and summing the results. The cosine and sine weighting factors can be pre-computed and, for example, stored on a chip, such as on image sensor 331. The image sensor can be configured with pre-computed weighting factors, for example, during the image sensor initialization phase, such as being programmed with pre-computed weighting factors. The pre-computed weighting factors can then be used in several multiplication and summation calculations for estimating the widths of several intensity peaks and in multiple exposures of the image sensor according to the embodiments to which this document is applied.

[0137] It should also be recognized that multiplication and accumulation operations can be advantageously performed in parallel, for example, on each column of the image sensor, and, for example, in conjunction with reading image data from the image sensing area of ​​the image sensor (e.g., image sensing area 351), performed per pixel line at each WAM height position.

[0138] To calculate the first and second values ​​of this embodiment, corresponding to the magnitudes of the two DFT coefficients, this can be accomplished by four multiplications and accumulations, two for each DFT coefficient and two for each DFT coefficient, respectively. If DFT0 is calculated, it is sufficient to calculate the magnitude of DFT0 using the accumulation operation, and in this case, therefore, for the two DFT coefficients, there are a total of two multiplications and accumulations, and one accumulation operation.

[0139] The first and second values ​​may include the real and imaginary parts of the corresponding DFT coefficients, and are therefore complex values. The values ​​of the DFT coefficients and / or their real and imaginary parts can be sent off-chip, for example, outside the image sensor, such as outside image sensor 331, where DFT coefficient value calculations and / or, for example, width estimation can be performed. This mode of operation has already been described above. For example, computing device 233 or camera 230 with computing capabilities can be configured to perform value calculations and / or calculations for the final width estimation. Another device like this might be more suitable than such an image sensor for performing those calculations.

[0140] Figure 10 This describes how one or more devices 1000 (in other words, device 1000) can typically be configured to perform actions related to... Figure 6A schematic block diagram illustrating embodiments of the described methods and actions. The device 1000 preferably corresponds to or includes the image sensor, such as image sensor 231 or 331, and / or is part of or corresponds to an imaging system (e.g., imaging system 205). For example, the one or more devices may be or include computing device 233 and / or camera 230. Where the device includes or is an image sensor, Figure 10 The images shown can be considered as different views of how the image sensor 331 can be configured.

[0141] therefore:

[0142] Device 1000 may include a processing module 1001 such as a processing device, one or more hardware modules including, for example, one or more processing circuits, circuitry such as a processor, and / or one or more software modules for performing the methods and / or actions.

[0143] One or more devices 1000 may further include a memory 1002, which may include (such as containing or storing) a computer program 1003. The computer program 1003 includes “instructions” or “code” that can be executed directly or indirectly by one or more devices 1000 to perform the methods and / or actions. The memory 1002 may include one or more memory cells and may also be arranged to store data, such as configurations, data, and / or values, related to or used in performing the functions and actions of the embodiments described herein.

[0144] Furthermore, the devices may include processing circuitry 1004 involved in processing and, for example, encoding data, as exemplary hardware modules, and may include or correspond to one or more processors or processing circuitry. One or more processing modules 1001 may include, for example, "embodied in" or "implemented by" processing circuitry 1004. In these embodiments, memory 1002 may include one or more computer programs 1003 executable by processing circuitry 1004, thereby enabling or configuring the devices 1000 to perform the methods and / or actions.

[0145] Device 1000 (e.g., processing module 1001) may include input / output (I / O) module 1005, which is configured to participate, for example, by performing any communication to and / or from other units and / or devices (e.g., sending and / or receiving information to and / or from other devices). One or more I / O modules 1005 may be exemplified, where applicable, by obtaining (e.g., receiving) modules and / or providing (e.g., sending) modules.

[0146] Additionally, device 1000 may include sensing circuitry 1006 as one or more example hardware modules that relate to sensing light and / or images, such as those that fully or partially correspond to or include what has been discussed above with respect to image sensor 331, including, for example, image sensing regions, such as image sensing region 351.

[0147] Furthermore, in some embodiments, device(s) 1000 (e.g., processing modules(s) 1001) includes one or more of the following: acquisition modules, exposure modules, sensing modules, computing modules, and providing modules, as exemplary hardware and / or one or more software modules for performing the actions of the embodiments herein. These modules may be implemented wholly or partially by processing circuitry 1004.

[0148] therefore:

[0149] One or more devices 1000 and / or one or more processing modules 1001 and / or one or more processing circuits 1004 and / or one or more computing modules are operable or configured to calculate the first value, which includes the magnitude of the lower-order DFT coefficients.

[0150] One or more devices 1000 and / or one or more processing modules 1001 and / or one or more processing circuits 1004 and / or one or more computing modules are operable or configured to calculate the second value, which includes the magnitude of the higher-order DFT coefficients.

[0151] One or more devices 1000 and / or one or more processing modules 1001 and / or one or more processing circuits 1004 and / or one or more providing modules and / or one or more I / O modules 1005 are operable or configured to provide the information indicating the intensity peak width based on the first and second values.

[0152] Figure 11 This is a schematic diagram illustrating some embodiments related to computer program 1003 and its carrier, which are used to cause the device 1000 discussed above to perform the methods and actions or to have them performed.

[0153] The corresponding computer program 1003 includes instructions that, when executed by suitable processing circuitry such as processing circuitry 1004 and / or processing module such as processing module 1001, cause the corresponding device(s) 1000 to perform as described above. In some embodiments, one or more carriers comprising one or more computer programs are provided, or more specifically, one or more data carriers, such as one or more computer program products. The corresponding one or more carriers may be one of electronic signals, optical signals, radio signals, and computer-readable storage media (e.g., computer-readable storage media illustrated in the figure). One or more computer programs can therefore be stored on computer-readable storage medium 1101. The carrier may exclude transient propagation signals, and the data carrier may be correspondingly named a non-transitory data carrier. Non-limiting examples of data carriers as computer-readable storage media are memory cards or memory sticks, disk storage media, or generally mass storage devices based on one or more hard disk drives or one or more solid-state drives (SSDs). Computer-readable storage medium 1101 can be used to store data accessible via computer network 1102, such as the Internet or a local area network (LAN). One or more computer programs may also be provided as one or more pure computer programs or contained in one or more files. The one or more files may be stored on computer-readable storage medium 1101 and may be obtained, for example, via a server, through download on a computer network 1102 as shown in the figure. The server may be a web server or a file transfer protocol (FTP) based server, or a similar server. The one or more files may be executable files, to be downloaded directly or indirectly to and executed on the one or more devices 1000 to enable the one or more devices 1000 to function as described above, for example by execution by suitable corresponding processing circuitry. The one or more files may also be used, or alternatively, for intermediate download and compilation involving the same or other one or more processors to make them executable prior to further download and execution, thereby enabling the one or more devices 1000 to function as described above.

[0154] Note that any of the processing modules and circuits mentioned above can be implemented as software and / or hardware modules, such as in existing hardware, and / or implemented as application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), etc. Also note that any of the hardware modules and / or circuits mentioned above can be contained within a single ASIC or FPGA, or distributed across several independent hardware components, whether individually packaged or assembled into a system-on-a-chip (SoC).

[0155] Those skilled in the art will also recognize that the modules and circuits discussed herein may refer to hardware modules, software modules, analog and digital circuits and / or combinations of one or more processors configured with, for example, software and / or firmware stored in memory, which, when executed by the one or more processors, can configure one or more devices, one or more sensors, etc., to perform the methods and actions described above, and / or cause one or more devices, one or more sensors, etc., to perform the methods and actions described above.

[0156] Identification by any identifier here can be implicit or explicit. In a certain context, such as for a particular computer program or program provider, the identification can be unique.

[0157] As used herein, the term "memory" can refer to data storage used to store digital information, typically a hard disk, magnetic storage device, medium, portable computer floppy disk or hard disk, flash memory, random access memory (RAM), etc. Alternatively, memory can refer to the processor's internal register memory.

[0158] Furthermore, it should be noted that any enumerative terms (such as first device, second device, first surface, second surface, etc.) should be considered non-restrictive, and such terms do not imply any hierarchical relationship. In the absence of any explicit information to the contrary, naming by enumeration should only be considered as a way of achieving different names.

[0159] As used herein, the expression “configured to” can mean that the processing circuitry is configured, or adapted, to perform one or more of the actions described herein, through software or hardware configuration.

[0160] As used herein, the term “number” or “value” can refer to any kind of number, such as a binary number, a real number, an imaginary number, or a rational number. Furthermore, a “number” or “value” can be one or more characters, such as a single letter or a string of letters. Additionally, a “number” or “value” can be represented by a bit string.

[0161] As used herein, the expressions “may” and “in some embodiments” have been generally used to indicate that the described features may be combined with any other embodiments disclosed herein.

[0162] In the accompanying drawings, features that may only exist in some embodiments are typically drawn using dotted lines or dashed lines.

[0163] When the word “includes” or “contains” is used, it will be interpreted as non-restrictive, meaning “consisting of at least…”.

[0164] The embodiments described herein are not limited to those described above. Various alternatives, variations, and equivalents may be used. Therefore, the above embodiments should not be construed as limiting the scope of this disclosure as defined by the appended claims.

Claims

1. A method performed by one or more devices (205; 231; 331; 1000) for providing information indicating the width (371; 471; 571) of an intensity peak (363; 463; 563) of an intensity peak (363; 463; 563) along a pixel line (352-m) of pixels (366; 466; 566), the pixel line including the intensity peak (363; 463; 563) and having an intensity pixel value as part of the intensity peak (363; 463; 563), wherein the pixels correspond to pixel elements of an image sensing region (351) of an image sensor (231; 331) and their intensity pixel values ​​have been obtained from exposure of the image sensing region (351) as part of image sensing of light reflected from a surface by the image sensor (231; 331), the intensity peak (363; 463; 563) being... 463； 563) by the image sensor (231; 331) The formation of such reflected light, wherein the method includes: - Calculate (601) a first value, the first value including the magnitude of the low-order discrete Fourier transform (DFT) coefficients (473; 573) of the pixel intensity values ​​of the pixels (366; 466; 566) along the pixel line (352-m) including the intensity peaks (363; 463; 563); - Calculate (602) a second value, the second value comprising a magnitude of another higher-order DFT coefficient (473; 573) relating to the pixel intensity values ​​of pixels (366; 466; 566) along a pixel line (352-m) including the intensity peaks (363; 463; 563), the higher-order being a higher-order DFT coefficient (473; 573) than the lower-order DFT coefficient (473; 573), and - Based on the first and second values, the information indicating the intensity peak width (371; 471; 571) is provided (603).

2. The method according to claim 1, wherein, Image sensing by image sensors (231; 331) is part of a three-dimensional 3D imaging based on optical triangulation performed by an imaging system (205) for 3D imaging of an object (220) having the surface, wherein the imaging system (205) includes a camera (230) having image sensors (231; 331) and a light source (210) for providing illumination, which causes light on the surface to be part of the optical triangulation-based 3D imaging.

3. The method according to any one of claims 1-2, wherein the intensity peak (363; 463; 563) is associated with a position on the surface that reflects the light to produce the intensity peak (363; 463; 563), wherein the information indicating the width of the intensity peak is provided as an indication of light scattering from the position.

4. The method according to any one of claims 1-3, wherein, The low-order DFT coefficients and high-order DFT coefficients (473; 573) include one or more of the following: lowest-order DFT coefficients (473-0; 573-0), second lowest-order DFT coefficients (473-1; 573-1), and third lowest-order DFT coefficients (473-2; 573-2).

5. The method according to any one of claims 1-4, wherein, The lower-order DFT coefficients are the lowest-order DFT coefficients (473-0; 573-2).

6. The method according to claim 5, wherein, The magnitude of the lowest-order DFT coefficient (473-0; 573-0) is calculated as the sum of the pixel intensity values ​​of the pixel (366; 466; 566) including the intensity peak (363; 463; 563).

7. The method according to any one of claims 5 to 6, wherein, The higher-order DFT coefficients are the second lowest DFT coefficients (473-1; 573-1) or the third lowest DFT coefficients (473-2; 573-2).

8. The method according to any one of claims 1-7, wherein, The information indicating the width includes the first value and the second value.

9. The method according to any one of claims 1-7, wherein, The information indicating the width includes or is based on a measure of how the first and second values ​​relate to each other.

10. The method of claim 9, wherein the metric is based on a calculated quotient or difference between the first value and the second value.

11. The method according to any one of claims 1-10, wherein, The pixels including the intensity peak are selected by the region of interest (ROI), which partially covers the image sensing region (351), and are determined to be the corresponding intensity peak (363) in the corresponding column (352-m) of the pixel element covering the image sensing region (351).

12. The method according to any one of claims 1-11, wherein, The method is performed by one or more devices, which are or include the image sensor of claim 1, and / or are part of or correspond to the imaging system of any one of claims 2-3.

13. One or more devices (205; 231; 331; 1000) for providing information indicating the width (371; 471; 571) of an intensity peak (363; 463; 563) of an intensity peak (363; 463; 563) along a pixel line (352-m) of pixels (366; 466; 566), the pixel line including an intensity peak (363; 463; 563) and having an intensity pixel value as part of the intensity peak (363; 463; 563), wherein the pixels correspond to pixel elements of an image sensing region (351) of an image sensor (231; 331), and their intensity pixel values ​​have been obtained from exposure of the image sensing region (351) as part of image sensing by the image sensor (231; 331) of light reflected from a surface, the intensity peak (363; 463; 563) being formed by such reflected light sensed by the image sensor (231; 331), wherein the one or more devices are configured to: Calculate (601) a first value, the first value including the magnitude of the low-order discrete Fourier transform (DFT) coefficients (473; 573) of the pixel intensity values ​​of the pixels (366; 466; 566) along the pixel line (352-m) including the intensity peaks (363; 463; 563); Calculate (602) a second value, which includes a magnitude of another higher-order DFT coefficient (473; 573) relating to the pixel intensity values ​​of pixels (366; 466; 566) along a pixel line (352-m) including intensity peaks (363; 463; 563), wherein the higher-order DFT coefficient (473; 573) is a higher-order DFT coefficient than the lower-order DFT coefficient (473; 573), and The information indicating the intensity peak width (371; 471; 571) is provided (603) based on the first and second values.

14. One or more computer programs (1003), including instructions that, when executed by one or more processors, cause one or more devices to perform the method according to any one of claims 1-12.

15. One or more carriers, including one or more computer programs (1003) according to claim 14, wherein the one or more carriers are one or more of the following: electrical signals, optical signals, radio signals, or computer-readable storage media (1101).

Images