Method and mechanism for providing pixel values ​​for readout from an image sensor - Patents.com

Abstract

To provide a laser triangulation based 3D imaging system, device and method allowing optical triangulation based 3D imaging throughput enhancement and high-speed applications.SOLUTION: By comparing pixel values in each column with a threshold pixel value, a window width (WAM width) around a pixel which is on an intensity line 251 when an object is illuminated by a light line during optical triangulation and which has the highest intensity in each column (for example, a WAM width 257 around a pixel Pm,n in column m) is identified and only pixels included in the WAM are read out, so that the number of readouts is reduced.SELECTED DRAWING: Figure 2A

Description

[Technical field]

[0001] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiments herein relate to methods and mechanisms for providing pixel values ​​for readout from an image sensor, in particular pixel values ​​derived from exposure to light reflected from an object as part of 3D imaging of the object based on optical triangulation. [Background technology]

[0002] Industrial vision cameras and systems for automating factories and logistics may be based on three-dimensional (3D) machine vision, where 3D images of scenes and / or objects are captured. A 3D image refers to an image that includes "height" or "depth" information, which is not, or at least not only, information such as intensity and / or color for pixels that are only two-dimensional (2D) in a normal image. That is, each pixel of the image may include information related to the pixel's location, mapping to the location of the imaged object, for example an object. Processing may then be applied to extract information about the object's properties, i.e., its 3D properties, from the 3D image and convert it, for example, into various 3D image formats. Such information about height may be referred to as range data, and thus range data may correspond to data from a height measurement of the object being imaged, or in other words from a measurement of the object's range or distance. Alternatively or in addition, the pixel may include information about material properties, such as, for example, light scattering or reflection of certain wavelengths of light in the imaged area.

[0003] Thus, the pixel value may for example relate to the intensity, and / or range data, and / or material properties of the pixel.

[0004] Line scan image data results when image data of an image is scanned or provided one line at a time, for example by a camera using a sensor configured to sense one line of pixels at a time to provide the image data. A special case of line scan imagery is image data provided by so-called "sheets of light", for example 3D triangulation of laser lines. Lasers are often preferred, but other light sources capable of providing a "sheet of light" can also be used, for example light sources providing light that can remain focused and does not diverge too much, in other words "structured light", which is light provided by, for example, a laser or a light emitting diode (LED).

[0005] 3D machine vision systems are often based on such light sheet triangulation. In such systems, a light source illuminates an object with a specific light pattern, such as a light sheet, to produce, for example, a light or laser line on the object along which 3D characteristics of the object corresponding to the object's profile can be captured. By scanning the object with such a line, i.e. by performing a line scan encompassing the line and / or object movement, 3D characteristics of the entire object corresponding to multiple profiles can be captured.

[0006] A 3D machine vision system or device that uses a light sheet for triangulation may be referred to as a system or device for 3D imaging based on light or light sheet triangulation, or simply laser triangulation when a laser beam is used.

[0007] Typically, to generate a 3D image based on optical triangulation, reflected light from an object being imaged is captured by an image sensor in a camera, and intensity peaks are detected in the image data. The peaks occur at positions that correspond to locations on the object imaged with incident light, e.g., corresponding to a laser line reflected from the object. The image positions of the detected peaks are then mapped to the positions on the object from which the light resulting in the peak was reflected.

[0008] Laser triangulation camera systems, i.e. imaging systems based on optical triangulation, project a laser line onto an object to generate a height profile from the surface of the object of interest. By moving the object relative to the containing camera and light source, information about the height profile from various parts of the object is captured by the image, which can then be combined and used with knowledge of the relevant geometry of the system to provide a three-dimensional representation of the object, i.e. 3D image data. This technique can be described as capturing an image when light, typically a laser line, is projected onto the object and reflected, and then extracting the location of the reflected laser line in the image. This is usually achieved by identifying the location of intensity peaks in the image frame, for example using conventional peak detection algorithms. Typically, the imaging system is set up so that intensity peaks for the reflected light occur and are inferred for each sensor row, mapping the location in the row to height or depth, although this is not required.

[0009] Peak detection algorithms generally operate in the digital domain and aim to detect the location of the center of the light distribution of the peak with a sub-pixel resolution. This means that an image with intensity peaks, containing analog pixel values ​​corresponding to the light sensed by the sensor elements corresponding to the pixels of the image sensor, must first be read out from the image sensor and analog-to-digital (A / D) converted. Conventionally, pixel values ​​of the same row are read out simultaneously from the image sensor, i.e. the pixels of the same row are read out in parallel, then A / D converted in parallel and then digitally processed and / or stored. Once the pixels of one row have been read out, a readout of another pixel, for example the next row, is performed, etc. That is, the pixels of the same row may be read out in parallel, and the readout and A / D conversion of such pixel rows are performed sequentially, i.e. one after the other.

[0010] In general, it is desirable to eliminate or reduce delays to facilitate or enable higher throughput from 3D imaging systems, for example, it is desirable to determine how quickly and / or how quickly a 3D imaging system based on optical triangulation can provide 3D images of an object or to better support higher speed applications.

[0011] The solution disclosed in US2021 / 0006738A1 is to use a first readout step, which is performed by reading out the entire pixel array corresponding to the entire image from the image sensor at a low accuracy and high speed using a separate, fast readout channel. The readout image is then used to calculate the position of the image ROI. The position of the ROI is then fed back to the image pixel array of the image sensor in order to select the pixels to be read out as a separate image in a second readout step. The second readout step involves reading out the pixels of the ROI with the highest accuracy using a separate, slower readout channel, so that fewer pixels need to be read out by that channel. [Prior art documents] [Patent documents]

[0012] [Patent Document 1] US Patent Application Publication No. 2021 / 0006738 Summary of the Invention [Problem to be solved by the invention]

[0013] In view of the above, it is an object of the present invention to provide one or more improvements or alternatives to the prior art, in particular to reduce the delays involved in the operation of a 3D imaging system based on or for such laser triangulation-based 3D imaging, and / or an image sensor. [Means for solving the problem]

[0014] According to a first aspect of an embodiment herein, this object is realized by a method performed by one or more devices for providing pixel values ​​for readout from at least a region of pixels of an image sensor. The device obtains information indicating a window (WAM) width at or around a maximum value corresponding to a plurality of pixels and a threshold pixel value. The device exposes pixels of at least said region of the image sensor such that the exposed pixels achieve analog pixel values ​​corresponding to light sensed during the exposure, respectively. The device identifies a respective pixel group of one or more pixels in a respective image sensor column of the region whose analog pixel values ​​during the exposure first reach or exceed the threshold pixel value in the respective column. Moreover, the device associates the identified respective pixel groups with a respective specific WAM position of a respective WAM having the WAM width in the respective image sensor column of the region, whereby each WAM in the respective column identifies a plurality of pixels corresponding to a WAM width covering the identified respective pixel group and neighboring pixels. Furthermore, the device reads out each WAM position in parallel and provides pixel values ​​associated with the same WAM position in the respective WAM of pixels based on the exposure.

[0015] According to a second aspect of an embodiment herein, this object is realized by one or more devices for providing pixel values ​​for readout from at least a region of pixels of an image sensor. The device is configured to obtain information indicating a WAM width and a threshold pixel value corresponding to a plurality of pixels. The device is configured to expose pixels of at least said region of the image sensor, whereby the exposed pixels achieve analog pixel values ​​corresponding respectively to light sensed during exposure. The device is configured to identify a respective pixel group of one or more pixels in each image sensor column of the region, where an analog pixel value during said exposure reaches or exceeds said threshold pixel value for the first time in said respective column. Moreover, the device is configured to associate said identified respective pixel group with a respective specific WAM position of a respective WAM having said WAM width in said respective image sensor column of the region, whereby each WAM in each column identifies a plurality of pixels and neighboring pixels corresponding to a WAM width covering the identified respective pixel group. Still further, the device is configured to read out each WAM position in parallel and provide pixel values ​​associated with the same WAM position in each WAM of pixels based on said exposure.

[0016] According to a third aspect of an embodiment of the present specification, this object is achieved by a computer program comprising instructions which, when executed by one or more devices, cause the devices to perform the method according to the first aspect and / or which, when executed by a hardware synthesis device, synthesizes and / or configures hardware as said one or more devices according to the second aspect.

[0017] According to a fourth aspect of an embodiment of the present specification, this object is achieved by a carrier comprising a computer program according to the third aspect.

[0018] According to a fifth aspect of an embodiment herein, this object is achieved by an imaging system comprising one or more devices according to the second aspect and a light source configured to illuminate an object with a specific pattern of light, the light source and the image sensor being arranged relative to each other in the imaging system such that the specific pattern of light is at least partially incident on the image sensor when reflected from the object to expose said pixel, As a result, said pixel value for readout comprises information convertible into information of a three-dimensional property of the object relating to the position of the object that reflected light to cause said incident light on the image sensor.

[0019] Thus, in the embodiments herein, it is the pixel position in or relative to each WAM that determines the pixels to be provided in parallel for readout, and not the actual image sensor row position as in the past. In other words, the readout here is not relative to the actual row position, but to the position in the respective WAM. In this way, it is not necessary to read out all the rows of pixels required to cover all the rows or all the columns of WAMs in the area of ​​the image sensor. Thus, the number of readouts that need to be performed to obtain the relevant pixel values, i.e. image data, covering all the WAMs can be reduced compared to how this is conventionally handled. Thus, faster readouts and faster performance are possible. It will be appreciated that the method is of particular interest to implement applications involving image sensors that are interested in readouts from around the light intensity maximum. This is the case, for example, when the image sensor is used as an image sensor of light in laser triangulation, such as 3D imaging based on optical triangulation, or in other words, when the image sensor is used as an image sensor of a camera, for example, of a 3D imaging system based on optical triangulation.

[0020] Example embodiments of the present invention will now be described in more detail with reference to the accompanying schematic drawings outlined below. [Brief description of the drawings]

[0021] [Figure 1] FIG. 1 is a schematic diagram illustrating an example of a prior art 3D imaging system based on optical triangulation with which some embodiments herein may be used. [Figure 2A] FIG. 2 is a schematic diagram illustrating an example of an image resulting from exposure of an image sensor. [Figure 2B] 1A-1C are schematic diagrams showing examples of light distributions around an intensity peak. [Figure 2C] FIG. 1 is a schematic diagram showing an example of a window at or around a maximum (WAM). [Figure 3A] FIG. 2 is a schematic diagram illustrating an example of intensity lines on pixels of an image sensor. [Figure 3B] FIG. 3B is a schematic diagram illustrating an example of which pixel in each column of FIG. 3A is first detected as exceeding a threshold pixel value during exposure. [Figure 3C] FIG. 1 is a schematic diagram showing a WAM with a WAM width of 5 pixels used in column m. [Figure 3D] FIG. 3D is a schematic diagram showing an example when the WAM of FIG. 3C is applied around a pixel detected as in FIG. 3B. [Figure 3E] FIG. 1 is a schematic diagram showing that pixels having the same WAM location are read out in parallel, and that such parallel readout of different WAM locations is performed sequentially. [Figure 3F] FIG. 1 is a schematic diagram showing an example pixel that is part of a first identical WAM location corresponding to a pixel in a first WAM row for parallel readout, where indexes m, n are for columns and rows are used to identify pixel P. [Figure 4] FIG. 1 is a schematic diagram illustrating an example of an imaging system in which embodiments herein may be implemented and / or may be configured in whole or in part to implement some embodiments. [Diagram 5] 1 is a flow chart that generally illustrates an embodiment of a method according to an embodiment herein. [Figure 6A]FIG. 13 is a schematic diagram showing the WAM for a 2-pixel pixel group and the 3-pixel group WAM width for column m. [Figure 6B] FIG. 6B is a schematic diagram showing an example when the WAM of FIG. 6A is applied around a pixel detected as in FIG. 3B. [Figure 7] 4 is another flow diagram that generally illustrates some embodiments of methods involving double exposure according to embodiments herein. [Figure 8] FIG. 2 is a schematic block diagram illustrating an example of how each pixel of an image sensor may be implemented to support and perform certain embodiments herein. [Figure 9] FIG. 1 is a schematic block diagram to illustrate how arbiter functionality for some embodiments may be implemented by logic in hardware. [Figure 10] 6 is a schematic block diagram illustrating an embodiment of how one or more devices may be configured to perform the methods and actions discussed with respect to FIG. 5. [Figure 11] 1 is a schematic diagram illustrating some embodiments of a computer program and its carrier. [Figure 12] 1 is a schematic diagram illustrating some embodiments of a computer program and its carrier. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0022] The embodiments herein are exemplary embodiments. It should be noted that these embodiments are not necessarily mutually exclusive. Components of one embodiment may be implicitly assumed to be present in another embodiment, and how to use those components in other exemplary embodiments will be apparent to one of ordinary skill in the art.

[0023] FIG. 1 shows a schematic representation of an example of such a type of imaging system mentioned in the background, namely an imaging system 105 for optical triangulation, i.e. 3D machine vision or simply 3D imaging based on imaging to capture information about 3D properties of a measurement object. In the figure, the system 105 is shown in a state of normal operation, i.e. generally after a calibration has been performed and the system has been calibrated. The system 105 is configured to perform optical triangulation, here in the form of light sheet triangulation as mentioned in the background. The system 105 further comprises a light source 110, for example a laser, for illuminating the measurement object to be imaged with a specific light pattern 111, illustrated and shown in the figure as a light sheet. The light can be, but does not have to be, a laser beam. In the illustrated example, the objects of interest are illustrated by a first measurement object 120 in the form of a car and a second measurement object 121 in the form of a gear structure. When the specific light pattern 111 is incident on the object, this corresponds to a projection of the specific light pattern 111 on the object, which can be observed when the specific light pattern 111 intersects with the object. For example, in the illustrated example, the specific light pattern 111 is illustrated as a sheet of light, resulting in a light line 112 on the first measurement object 120. The specific light pattern 111 is reflected by the object, more specifically by the intersecting part of the object, i.e., in the illustrated example, the part of the light line 112. The system 105 further comprises a camera 130 with an image sensor (not shown in FIG. 1), the camera and the image sensor being arranged in relation to the light source 110 and the object to be imaged such that the specific light pattern, when reflected by the object, becomes incident light on the image sensor. The image sensor is generally implemented as a chip and is a mechanism for converting incident light into image data. The part of the object that, by reflection, results in said incident light on the image sensor may thereby be captured by the camera 130 and the image sensor, and corresponding image data may be generated and provided for further applications.For example, in the illustrated example, a particular light pattern 111 may be reflected in a light line 112 at a portion of a vehicle roof of a first measurement object 120 towards the camera 130 and the image sensor to generate and provide image data having information about said portion of the vehicle roof. The image data may be converted into information about 3D characteristics, e.g., a 3D shape or profile of the object being imaged, in a suitable format, using the system's 105 knowledge of the object being imaged and its related settings, including geometry, e.g., how the coordinates of the image sensor relate to real-world coordinates, e.g., coordinates of the coordinate system 123, which are Cartesian coordinates. The information about the 3D characteristics, e.g., the 3D shape or profile, may include data describing the 3D characteristics in any suitable format.

[0024] By moving, e.g., the light source 110 and / or the object to be imaged, such as the first measurement object 120 or the second object 121, in fact typically by scanning the object, so as to illuminate multiple portions of the object and provide reflected light to the image sensor, image data describing a more complete 3D shape of the object can be generated, e.g. corresponding to multiple successive profiles of the object, such as the illustrated profile images 141-1 to 141-N of the first measurement object 120, each profile image showing the contour of the first object 120, and it is when a particular light pattern 111 is reflected that the image sensor of the camera unit 130 senses light and provides the profile image. As indicated in the figure, a conveyor belt 122 or the like may be used to move the object through the particular light pattern 112, and the light source 110 and camera unit 130 may typically be fixed, or alternatively, the particular light pattern 111 and / or camera 130 may be moved across the object so that all parts of the object, or at least all parts facing the light source 110, are illuminated and the camera can receive light reflected from various parts of the object that it is desired to image.

[0025] As can be seen from the above, an image frame of, for example, the first measurement object 120 provided by the camera 130 and its image sensor can result in any of the profile images 141-1 to 141-N. As mentioned in the background, the respective positions of the contours of the first object shown in any of the profile images 141-1 to 141-N are typically determined based on identifying intensity peaks in the image data captured by the image sensor and detecting the positions of these intensity peaks, for example by one or more intensity peak detection algorithms. The system 105 and conventional peak detection algorithms are typically configured to search for intensity peaks by pixel columns in each image frame. If the sensor coordinates are u,v as indicated in the figure, u may be along an image sensor row and may be used to indicate a position in such a column, for example corresponding to an image sensor row. Correspondingly, v may be along an image sensor column and may be used to indicate a position in such a column, for example corresponding to an image sensor row.

[0026] For each position u in the image frame, a peak position may be searched for along v, for example by a peak detection algorithm as described above, and the peaks identified in the image frames may result in profile images 141-1 to 141-N as shown in the figure. The profile images are formed by the image points u, v, t in a sensor-based coordinate system 143. The ensemble of image frames and profile images may be used to generate a 3D image of the first object 120.

[0027] As a development towards the embodiments of this specification, first the situation indicated in the background is further elaborated.

[0028] At the time of reading out the pixel values ​​from the image sensor, the location of the intensity peak in the image sensor area is unknown. To ensure that the intensity peak is captured, i.e., it may be necessary to read out the entire image including all rows and columns of the image sensor and A / D convert the read out pixel values. An option for this could be a partial readout if the image sensor supports such a readout, but it usually needs to be known that the intensity peak can only occur in a certain part of the total image sensor area, for example a sub-area corresponding to a region of interest (ROI) of the image sensor, and even if known, it is not known exactly where in the ROI the peak will occur, as mentioned above. Such a sub-area can be predefined based on knowledge of the imaging system, settings and / or the object being imaged. However, even a partial readout usually means that it is necessary to read out essentially the total number of image sensor rows to ensure that all positions in the column where the intensity peak can occur are captured.

[0029] The time that a profile image can be provided as described above based on the identified intensity peak location can be significantly affected by the height of the measurement area, i.e. the area where the intensity peak can occur, which corresponds to the number of rows used on the image sensor, and proportionally. Thus, the performance of a 3D imaging system based on optical triangulation is substantially limited by the delay resulting from this readout. A solution related to reducing the readout delay should facilitate and / or enable a higher throughput from the 3D imaging system and support high-speed applications. It is noted that if time can be saved, not only is there interest in increasing the throughput itself, but the time can also be used to perform multiple exposures, for example with different exposure times, for example to increase the dynamic range of the imaging.

[0030] For a peak detection algorithm to be able to identify the location of an intensity peak, image data in pixels needs to be available with at least some margin around the location of the intensity peak, e.g., to enable the peak detection algorithm to detect the location of the center of the light distribution with sub-pixel resolution. The margin should substantially cover the light distribution around the intensity peak.

[0031] For example, for a row of the image sensor above which it is known that no intensity peaks occur, and another row below which it is known that no intensity peaks occur, it would be sufficient to read out only between the top and bottom rows, preferably with the margin, so that the light distribution is captured for peaks in the "border" rows as well. When the image sensor is exposed and the pixels contain pixel values ​​for reading corresponding to the image, the image sensor is thereby provided with the functionality to identify, for example based on a threshold, the top and bottom rows of pixels associated with the intensity peaks. This should provide some flexibility compared to the case where the readout portion has to be determined in advance, but it is still necessary to read out all rows with intensity values ​​between the "top row" and the "bottom row". If the intensity values ​​are linear or have small variations along a single row, this may reduce or even minimize the number of rows to be read out. However, if the variation in height or depth captured by the image is larger and the dynamic range in this respect is larger, i.e. the intensity values ​​span many rows, then the intensity peaks that need to be read out will also cover many rows. Such variations are unavoidable since the variations themselves are fundamental to capturing height and depth information about the object.

[0032] Figure 2A shows diagrammatically one example of an image resulting from exposure of an image sensor, with the location of the highest intensity pixel in each column shown as an intensity line 251 corresponding to the captured reflected light from the object when the object is illuminated by a line of light during optical triangulation, e.g., as described above with respect to Figure 1. The image is in sensor coordinates u, v, with N rows of pixels 1,...,N corresponding to image sensor rows 254-1,...,254-N, and M columns of pixels 1,...,M corresponding to image sensor columns 252-1,...,252-M.

[0033] A window at or around a maximum (WAM) is a window that covers the intensity peak centre with some margin, preferably substantially and / or fully covers the light distribution around the intensity peak and / or a window around the intensity peak centre. It will be sufficient to have what is needed for the peak detection algorithm used and the required or preferred light distribution magnitude, for the algorithm to output a centre location with the desired sub-pixel accuracy, which can be used to represent the location of the intensity peak.

[0034] 2B shows a schematic light distribution 261 as an example of how the light distribution looks around an intensity peak, in this example along axis v or column. The actual values ​​available are discrete for each pixel, depicted in the figure as black boxes. The example shown is for the highest intensity pixel P m,n The actual center position 263, for example at sub-pixel resolution, of the light distribution is initially unknown and is determined, for example, by a peak detection algorithm after A / D conversion. m,nNote that the center position 263 is typically (and in this example) offset from the location of the highest intensity pixel in the column, i.e., .times. ...

[0035] 2C illustrates an example of a WAM 258 having a typical WAM width 257. In this example, the WAM is aligned with pixel P m,n , centered around row n and extending along column m. In the case of the WAM 258, the WAM is preferably a pixel window, i.e., a window corresponding to an integer number of pixels, e.g., the highest intensity pixel P at pixel location 255. m,n etc. However, the WAM may of course be somewhat offset with respect to such pixels and / or not perfectly symmetric around the central pixel of the WAM, such as if the WAM were instead made up of an even number of pixels rather than an odd number of pixels as in the example shown.

[0036] As explained above, each WAM around the highest intensity pixel of each intensity line 251 results in upper and lower margins of intensity line 251, as seen in FIG. 2A, that correspond to the dotted lines shown above and below intensity line 251 in FIG. 2A.

[0037] Thus, as already indicated above, to read out all the pixel rows contained in all the WAMs, i.e. all the rows of captured light, processed by the peak detection algorithm, a number of rows corresponding to a total width 259, which is usually much larger than the WAM width 257, is required. This is because, as mentioned above and made clear by the example, the intensity lines vary across the columns. Such variations are fundamental in themselves, since height and depth information about the object is captured, and therefore it is not desirable to remove or avoid them. However, at the same time, this means that it is necessary to read out all or a large number of rows from the image sensor, if not all, then at least for a large number of images. Still, since the pixels of the WAM contain the real information of interest, i.e. information about the light distribution, it would be sufficient to read out the pixels of the WAM in each column.

[0038] The embodiments herein are based on the concept of reading out only pixels in the WAM, thereby significantly reducing the number of readouts that would normally need to be performed otherwise, thereby enabling and facilitating increased throughput and high speed applications of 3D imaging, especially those based on optical triangulation.

[0039] Below, various embodiments of the method are shown and discussed to enable this, and examples are also given of how the method may be implemented in hardware.

[0040] Next, using Figures 3A to 3F, we will explain embodiments of the present specification that are based on the concept of a WAM and parallel readout of pixels in a WAM that have the same WAM position, i.e., pixels that can be considered to be in the same WAM row, rather than parallel readout of pixels in the same pixel row of an image sensor as in conventional pixel readout.

[0041] 3A shows a schematic diagram of one example of intensity lines 351 on pixels of image sensor 331, with individual pixels shown as squares. The pixels corresponding to the light sensing elements may be of a conventional type. N=12 pixel rows 0,...,11 correspond to image sensor rows 354-1,...,354-12, and M pixel columns 1,...,M correspond to image sensor columns 352-1,...,352-M.

[0042] This example shows the general situation where pixel values ​​are read out one column at a time; conventionally, this would be one row of pixels of an image sensor at a time, i.e., one row of pixel values ​​are read out in parallel and then analog-to-digital (A / D) converted; i.e., as indicated in the figure, M columns may be connected to M A / D converters 356 configured to operate in parallel.

[0043] Note that the intensity lines 351 shown here are merely an indication of where the actual intensity peaks are, and of course a pixel cannot sense the lines with the resolution shown. The intensity lines could be the reflected light from an object when the object is illuminated by a line of light during light triangulation, for example as described above in connection with FIG.

[0044] For illustrative purposes only, several pixels are individually selected. For example, pixel 355a is pixel P at column m=0, row n=5, and thus pixel P m,n =P 0,5 Pixel 355b may be referred to as pixel P m,n =P 5,8 and pixel 355c is P m,n =P M-1,8 It is.

[0045] All of these are also examples of pixels where the majority of the intensity line 351 lies in that column.

[0046] The concept of reducing the number of rows of readout has the problem that the location of the intensity peak is not known or obvious before readout, as explained above in relation to FIG. 2. The location of the intensity line 351 is not known, and the peak detection algorithm operates in the digital domain after readout. It would be desirable to be able to roughly identify the peak center, for example at normal pixel resolution, and apply the WAM around it. To complicate matters further, since it is actually several pixels per column that sense light, not just one, it is not enough to identify only the pixels that have sensed light, but rather the pixels per column that sensed the most light.

[0047] The embodiments herein are based on this solution, a type of solution that can be called "winner takes all". Rather than waiting until after exposure, during exposure, the first pixel in each column to reach or exceed a threshold pixel value is examined, and this pixel becomes the "winner" in that column, meaning for example to be in the center position of the WAM in that column, or at least to be covered by the WAM in that column. The first to reach or exceed the threshold in a column indicates that this pixel is the first to be strong enough, i.e., to sense a certain amount of light. This concept utilizes the assumption that the pixel exposed to the most intense light in the column during exposure should be the first to reach or exceed the threshold. For example, in the illustrated example, pixels 355a-355c are examples of pixels in their respective columns that are first to exceed such a threshold.

[0048] The light pattern on the image sensor, for example the intensity line 351, is assumed to be substantially stationary during exposure and not move over several pixels. Any movement is so small that it should have no practical adverse effect. This is not a practical issue since any significant movement would normally result in blurring of the image, and in principle for all imaging, at least for machine vision and industrial applications, and especially for 3D imaging based on optical triangulation, everything is constructed and operated to provide a crisp, unobscured image, i.e. there is no or only a small change in the position of the light pattern during exposure.

[0049] The threshold should be set so that at least one pixel reaches or exceeds the threshold so that all actual intensity peaks of interest can be identified in the column. At the same time, it may be desirable not to use a threshold that is too small, since a threshold that is too small may introduce other light or noise that may cause other "wrong" pixels to register some light sooner. It is within the capabilities of one of ordinary skill in the art to find appropriate thresholds to use, for example, by routine testing and experimentation with the image sensor, imaging system, and their settings.

[0050] Figure 3B, as mentioned in the preceding paragraph, shows a schematic example of each pixel in each column of Figure 3A that first reaches or exceeds a common threshold pixel value during exposure. The example pixels 355a-355c are selected as examples of such "winner" pixels. From Figure 3A, it can be seen that the "winner" of each column is the single pixel that receives or "sees" the most light, e.g., receives the most intensity, during exposure, as determined in this example by the position of the intensity line 351 in that column.

[0051] Thus, by utilizing thresholding during exposure, the pixels in each row that should be covered by the WAM, and preferably the center pixel of the WAM, can be identified. Using a threshold to identify the first pixel in a pixel row of the image sensor to reach or exceed the threshold can be accomplished in a number of different ways, as will be appreciated by those skilled in the art. Specific examples are provided separately below.

[0052] The row number or row address of the identified pixel in each column may be stored, e.g., one such row identifier per column having an identified pixel. It is sufficient to identify the row of the identified pixel in each column in order to identify the pixel. The reason for storing the row number is to allow the location of the WAM to be later related or mapped to the actual sensor row location, as will be explained further below, so that the actual location on the sensor is known with respect to the location of an intensity peak detected, e.g., by a peak detection algorithm.

[0053] FIG. 3C shows a schematic of a WAM 358-m, where m can be m=0,...,M-1, with a WAM width of 5 pixels. Thus, as shown, there are five WAM positions or WAM rows 364-1,...,364-5 per WAM. WAM 358-m will cover, and preferably be centered around, a pixel identified as a "winner" pixel in a column, i.e., a pixel in the column that has a pixel value that first reaches or exceeds a threshold during exposure. In other words, the center position of each WAM may be located on a pixel in the respective column that has been identified as a "winner" pixel as described above in connection with FIG. 3B. If the "winner" is at the edge of the image sensor, the handling of this is left to and within the capabilities of a person skilled in the art based on the circumstances, e.g., application, requirements, etc. One solution in such a case may be to adapt the WAM to start at an edge pixel and spread from there along the column to the WAM width. For such pixels, the WAM will not be centered around the winner, but will at least cover it as well as possible, and then the processing of this is left to a peak detection algorithm, and image data from such a WAM is for example used or discarded to detect the peak position. It may also be that in some columns there are no pixels reaching the threshold, in which case the position of the WAM may be set to a predefined position, for example one edge of the imager. In this way, subsequent processing, such as readout with the WAM for each column, can be aligned. In this case, subsequent processing, for example by a peak detection algorithm, cannot generate a valid peak position, since there is anyway no valid peak with sufficient strength, and therefore it becomes meaningless which pixel value to output for that column.

[0054] Figure 3D shows a schematic example of the WAM of Figure 3C when it is centered on a pixel detected as in Figure 3B. In other words, Figure 3D shows one WAM per column, i.e., each WAM 358-m centered on an identified "winner" pixel in each column m, m=1,...,M. Thus, there is one such WAM 358 per column.

[0055] 3E shows diagrammatically that pixels with the same WAM location are read out in parallel, and that such parallel readouts for different WAM locations are performed sequentially. For example, the WAM width in the example is 5 pixels, so the readout time here consists of 5 sequential readouts, e.g., for each WAM, the first readout is to the same WAM location, and thus the WAM location for each column is exactly the same. Note that for pixels that are read out in parallel, both the WAM location and the WAM row are the same. However, as will be understood from the following description, identical WAM locations do not necessarily map to the same image sensor row.

[0056] In the example, during the first 1 / 5 of the total readout time, all pixels in each column covered by the first WAM position, i.e., first WAM row 364-1, are read out in parallel, etc. Referring to the example of Figure 3D, this means that pixels of all columns 352-1, ..., 352-M in the top WAM position, i.e., WAM row 364-1, are provided for parallel readout and A / D conversion, for example, by ADC 356.

[0057] Figure 3F shows, by way of example only, a schematic representation of an example pixel that is part of a first identical WAM location, i.e., first WAM row 364-1, and thus corresponds to a pixel in the first WAM row for parallel readout from columns 352-1, ..., 352-M. By examining Figure 3D, where the first WAM location of each WAM is located, it can be seen that the pixel of the image sensor in Figure 3F is a pixel in the first WAM row 356-1.

[0058] Thus, the embodiments herein described above involve identifying the "winning" analog pixel value during exposure, in other words the analog pixel value in the respective column that is the first to reach or exceed a threshold pixel value. Thus, compared to such prior art solutions mentioned in the background, there is no need to first (pre-) read out the entire image from the sensor at high speed and low accuracy using a separate high-speed readout channel, and then process the read out digital image to detect the location of the pixels forming the ROI, and then read out with detailed normal accuracy in a second readout step. The embodiments herein eliminate the need to read out the same pixels from the image sensor in two stages, or the need for a separate readout channel for the first readout step as in the prior art, by looking at the analog pixel value of one or more pixels per column that is the first to reach or exceed a threshold pixel value in the respective column during exposure. The analog pixel value that is the first to reach or exceed a threshold value also means that no more time needs to be spent for the pixel or group of pixels per column that reach or exceed a threshold value to identify the location of the WAM. The sum of the WAM, which in the prior art may be considered to correspond to the ROI, is thus determined in a different manner.

[0059] FIG. 4 illustrates generally one example of an imaging system 405 in which embodiments herein may be implemented and / or may be configured in whole or in part to implement some embodiments.

[0060] An imaging system that may be configured to perform or implement embodiments of the present specification is shown in a simplified manner. The imaging system 405 is based on optical triangulation for capturing information about 2D and 3D properties of one or more measurement objects, for example the measurement object 420 shown diagrammatically in the figure. The shown system may be considered to correspond to a basic configuration and comprises a light source 410 for illuminating the measurement object 420 with light 411, typically a laser beam, as part of the optical triangulation for 3D imaging of the measurement object 420. The camera 430 has an image sensor 431 arranged to sense reflected light from the measurement object 420 as part of said optical triangulation for 3D imaging.

[0061] The camera 430, image sensor 431 and light source 410 are configured and positioned relative to one another for optical triangulation. System 405 may correspond to system 105 of FIG. 1 for optical triangulation, but may additionally be configured to operate with and / or according to embodiments herein, as described further below.

[0062] Thus, the measurement object 420 may correspond to the first measurement object 120 or the measurement object 220 and is shown at least partially located within the field of view 432 of the camera 430. The light source 410 is configured to illuminate the measurement object 420 with light 411, typically in the form of structured light, e.g. a specific light pattern, e.g. a light sheet and / or a beam of light, e.g. a laser, which is reflected by the measurement object 420 and the reflected first light is captured by the camera 430 and the image sensor 431. Another example of structured light that may be used as the first light is a light edge, i.e. an edge of an area or part with illumination. In this example, the direction of illumination is vertical, i.e. substantially parallel to the z-axis. However, as will be understood from the following description, one or more other illumination directions can of course be used with the embodiments herein, including, for example, a so-called "specular" configuration, using a light source, e.g. a laser, substantially directed at the measurement object at a "specular" angle of incidence corresponding to the viewing angle of the camera but from the opposite side.

[0063] The measurement object 420 may be illuminated in this way and images may be captured in the same way as in conventional optical triangulation. The optical triangulation may thus involve the movement of the light source 410 and / or the measurement object 420 relative to one another, so that at different successive times different successive parts of the measurement object 420 are illuminated by the light source 410 and the light 411. The light 411 after being reflected from the measurement object 420 is sensed by the image sensor 431. In optical triangulation, the camera 430 and the first light source 410 are usually fixed relative to one another and the measurement object 420 is moved relative to them, although this is not essential. Each image frame is associated by the sensing by the image sensor 431 with the time at which the image frame was respectively sensed or captured and with the respective part of the measurement object 420 at which the image sensor 431 sensed the reflected first light 411 at the respective time.

[0064] Image frames and / or information derived from the image frames, e.g., intensity peak locations, provided by the camera 430 and image sensor 431 may be transferred to a computing device 434, e.g., a computer or similar, for further processing outside the camera 430. Such further processing may additionally or alternatively be performed by a separate computing unit or computing device (not shown), e.g., separate from the image sensor 431 and integrated into the camera 330, or a unit comprising the camera 430. The computing device 434, e.g., a computer or another device (not shown), may also be configured to control devices involved in the optical triangulation to perform optical triangulation and related actions. Furthermore, said computing device 434, e.g., a computer, or another device (not shown) may be configured to perform and / or be included in some of the actions described above and / or related to the embodiments of the present specification, as described further below. Furthermore, to carry out the embodiments of the present specification, as shown in the figure, the image sensor 431 or what may correspond to a conventional image sensor may be provided, for example, incorporated with an image sensing device or circuit 433, e.g., a circuit, having further or additional functionality. A detailed example of this is discussed separately below. It is noted that the image sensing circuit 433 may correspond to an image sensor having more advanced functionality and circuitry than a conventional image sensing circuit, and when this circuitry and image sensor are incorporated into one and the same device and unit, e.g., a chip, together with the image sensor 431, may also be named an image sensor and configured to operate according to the embodiments of the present specification. In the case that the image sensor 431 and / or the image sensing device 433 may correspond to a conventional image sensor implemented, from a hardware point of view, e.g., as a chip, it may also be configured and / or programmed by software to operate according to the embodiments of the present specification.

[0065] 5 is a flow chart that generally illustrates an embodiment of a method according to embodiments herein, the following actions that may form the method are at least for providing pixel values ​​for readout from a region, e.g., a ROI of pixels of an image sensor:

[0066] A region that is a ROI, as used in the context of this specification, may in principle correspond to any region or contiguous group of pixels of the image sensor 431, for example a region comprising a first number of image sensor rows and a second number of image sensor columns that may be the same as or different from the first number. The region may comprise all or some of the pixels of the image sensor 431. Thus, the special case when the region includes all pixels of the image sensor 431 corresponds to the general case of applying an implied region that is the entire image sensor area rather than an explicit region.

[0067] 5 may be performed by an apparatus, i.e., one or more devices, such as image sensing device 433, camera 430, and / or computing device 434, and / or imaging system 405 and / or other suitable devices. Apparatus for performing the methods and actions thereof are further described below. To simplify the description of the methods, the methods and actions are exemplified below as being performed by image sensing device 433.

[0068] It should be noted that the following actions may be taken in any suitable order and / or may be performed with full or partial overlap in time, where possible and appropriate.

[0069] Action 501 The image sensing device 433 obtains information indicating a threshold pixel value corresponding to a number of pixels and a window (i.e., WAM) width at or around the maximum value. The WAM width may correspond to a WAM width of 257 or 357 as in the above example. In the following, a WAM width of 357 is used as an example.

[0070] The information may be obtained by receiving externally, e.g., by setting and / or by providing pre-configured values ​​to inputs of the device that performs the action, or may be obtained internally by the device, e.g., after being programmed, and / or stored in the memory of the device. In some embodiments, the information may be predetermined or predefined. WAM widths and threshold pixel values ​​may correspond to such WAM widths and threshold pixel values, as in various examples herein.

[0071] Action 502 An image sensing device 433 exposes, i.e., exposes, pixels of at least said region of the image sensor 431 to light, e.g., as part of optical triangulation, so that each exposed pixel achieves an analog pixel value respectively corresponding to the light sensed during exposure.

[0072] In some embodiments, as a partial 3D imaging of the object 420 based on optical triangulation, during exposure, each pixel is exposed to light reflected from the object, e.g., the object 420. That is, in the context of the optical triangulation discussed above, e.g., in relation to FIG. 4, the embodiments herein are applied. Thus, the 3D imaging can be performed by a 3D imaging system, e.g., the 3D imaging system 405, comprising an apparatus, e.g., the image sensing apparatus 433, for performing the method, and a light source, e.g., the light source 410, configured to illuminate the object with a specific light pattern, e.g., light that may correspond to the light 411 or a laser beam, e.g., a laser line. Thus, as the 3D imaging system is based on optical triangulation, the light source and the image sensor are arranged in relation to each other such that the specific light pattern, when reflected by the object, at least partially becomes the incident light on the image sensor, e.g., the image sensor 431, or at least becomes the incident light on said area of ​​the image sensor, resulting in said pixel value during exposure. In these embodiments, the pixel values ​​for readout contain information that can be converted into information about the three-dimensional properties of an object, e.g. object 420, with respect to a position on the object that reflected light, e.g. light 411, resulting in said incident light on the image sensor.

[0073] Action 503 The image sensing device 433 identifies a respective group of one or more pixels in each image sensor column, e.g., a respective one of columns 252-1, ..., 252-M or 352-1, ..., 352-M of the region, whose analog pixel value during the exposure first reaches or exceeds the threshold pixel value in the respective column. Thus, in some embodiments, the group may be just one pixel corresponding to either pixel 255 or 355 in the above example, but in some other embodiments may be a group of several pixels, which is further described below in a separate example. In the remainder of the description of FIG. 5, each identified group of pixels is illustrated by pixel 355, e.g., one of the example pixels 355-c above.

[0074] Action 504 In some embodiments, the image sensing device 433 stores a respective image sensor row position identifier that identifies an image sensor row position, e.g., one of image sensor rows 254-1, ..., 254-N or 354-1, ..., 354-12, associated with an identified respective group of pixels, e.g., one of pixels 255, 355-1, ..., 355-M.

[0075] Action 505 Image sensing device 433 associates each of the identified pixels with a respective specific WAM location, e.g., each WAM 358-1, ..., 358-M, in the respective image sensor column of the region, e.g., each image sensor column 352-1, ..., 352-M. Each of the WAMs has a WAM width, e.g., WAM width 357. As a result, each WAM identifies a number of pixels in each column corresponding to the WAM width that covers each of the identified pixels and neighboring pixels. That is, the WAM in a column covers each of the identified pixels and some of its neighboring pixels, e.g., the pixels in the column, and also covers some light distribution associated with each of the identified pixels.

[0076] In some embodiments, pixels corresponding to the WAM width are positioned around, ie, on either side of, each group of pixels identified in each column.

[0077] In some embodiments, the respective particular locations of the respective WAMs with which the respective identified pixel groups are associated are the same, e.g., centres, of the respective WAM locations of the respective WAMs.

[0078] It should be understood from the examples that the WAM location as used herein refers to the location of or within the WAM itself, i.e., the location relative to the WAM independent of which pixel of the image sensor the WAM is associated with. For example, if the WAM width is 7 pixels, there are 7 different WAM locations. Thus, associating each of the identified pixel groups with each identical WAM location of each WAM means that there must be an identical WAM location, e.g., the first or center location in each WAM, to name a few, with which each of the identified pixel groups is associated.

[0079] As also described in the example above, the association in action 505 results in, for each column, a respective WAM covering and / or surrounding the pixel identified as the first pixel in the column to reach or exceed the threshold pixel value (i.e., the "winner"). Thus, a number of pixels in each column corresponding to the WAM width will cover and / or surround the identified pixel or "winner." Thus, the "winner" will be associated with, covered by, and / or included in the WAM for the column, such that the WAM preferably surrounds, i.e., covers, both sides of, the "winner."

[0080] Action 506 The image sensing device 433 provides pixel values ​​of pixels associated with the same WAM position, e.g., the same WAM position or row 364 in each WAM, based on the exposure, for parallel readout for each WAM position, e.g., pixel values ​​of pixels in WAM position or row 364-1 of WAMs 358-1, ..., 358-M are provided for parallel readout.

[0081] Thus, the image sensor columns involved, ie the image sensor columns of said area or contained in said area, for example image sensor columns 352-1, . . . , 352-M, are read out in parallel.

[0082] Thus, the WAM location of a pixel in a pixel group of a pixel is the location in the WAM of the pixel itself relative to the WAM, e.g. the first or top position or column of the pixel, the top WAM row, which may be referred to as the first WAM location or first WAM row. When the group is several pixels, the WAM location of the group may be at the group resolution, i.e. the pixel value provided for readout may be the value of the group of pixels associated with the same WAM location, in this case e.g. the value of a binned single pixel. The use of pixel groups and binning, where each group is two or more pixels, is discussed further below.

[0083] It should be noted that in different image sensor columns, pixels having the same WAM location, e.g., the same WAM row location, may still be in different image sensor rows. For example, pixels at the same location, e.g., the first WAM location, may belong to two or more different image sensor rows. Such situations are specifically illustrated and discussed above, see, e.g., FIG. 3F.

[0084] Thus, the pixel values ​​provided for readout are those resulting from exposure of the image sensor, i.e., action 502. In some embodiments, as in the example above, the pixel values ​​and / or sensor row positions, such as those in action 504, may be provided for readout as analog values ​​and then A / D converted. In other embodiments, they are first A / D converted, e.g., by an A / D converter, column by column to provide digital pixel values ​​and provided for readout. In some embodiments, e.g., in the case of proximity sensor imaging (NSIP) type image sensors and image sensor circuits, they are A / D converted pixel by pixel to provide digital pixel values ​​for readout.

[0085] Thus, in the embodiments herein, as will be understood from the action, it is the group and / or pixel positions in the respective WAMs and / or relative to the respective WAMs that determine the pixel groups and / or pixels to be provided in parallel for readout, and not the actual image sensor row positions as in the past. In other words, the readout here is not to the actual row positions, but to the positions in the respective WAMs. Thus, in contrast to some of the prior art solutions discussed above, it is not necessary to read out all the rows of pixels required to cover all the rows or all the columns of WAMs of the image sensor area. Thus, the number of readouts that must be performed to obtain pixel values ​​or image data covering all the WAMs can be reduced compared to how this is conventionally handled. Thus, faster readouts and faster performance are possible. It is further understood that this method may be of particular interest for implementing applications that are interested in readouts from around the light intensity maximum. This is the case, for example, when the image sensor is used as an optical image sensor in laser triangulation, e.g. 3D imaging based on optical triangulation, in other words when the image sensor is used as an image sensor of a camera of a 3D imaging system based on optical triangulation.

[0086] Action 507 In embodiments in which action 504 is performed to store a sensor row position identifier, image sensing device 433 may further provide the stored respective sensor row position identifier associated with each WAM for readout associated with the respective WAM, e.g., respective WAM 358. In this manner, the provided respective WAM analog pixel values ​​may be mapped in association with an image sensor row position, e.g., any of image sensor rows 354-1, ..., 354-12 or 254-1, ..., 254-N, in addition to a sensor column position, e.g., any of image sensor columns 352-1, ..., 352-M or 252-1, ..., 252-M with which the WAM is already associated.

[0087] As can be seen from action 503, the stored respective image sensor row positions are for respective image sensor columns of the region, and the respective image sensor row positions are stored, and by action 505, the identified row positions are associated with respective particular WAM positions of respective WAMs, for example the center positions of respective WAMs, and thus associated with respective WAMs.

[0088] The stored sensor row position identifier may be in the form of information, e.g., a value, that identifies the sensor row position, such as an identifier stored for each column of the region, in other words, a row position identifier for each WAM and therefore for each included column.

[0089] Said provision for readout associated with each WAM may be the provision of one identifier per WAM, e.g. one identifier per contained image sensor column or one identifier per image sensor column of the region. The provision may be by separate or shared output with provision of pixel values ​​for readout, e.g. per image sensor column. For example, in case of a shared output, the provision of identifiers may be after the actual readout of pixel values ​​of all WAM row positions and before the readout of pixel values ​​related to the next exposure of the image sensor, see e.g. action 508. See below for a specific example encompassing several exposures.

[0090] There may be, for example, an image sensor row address associated with each pixel, e.g., in an associated memory, e.g., a "bit memory" or the like, which is used as an image sensor row location identifier. For example, in the case of a pixel group of multiple pixels, such as in the case of binning described below, this may be pre-determined / defined, e.g., that it is always the bottom / top pixel of the group that is used to represent the group.

[0091] Action 508 In some embodiments, an image sensing device 433 or other device reads the pixel values ​​provided for parallel readout, i.e., the device implementing the embodiment not only provides the pixel values ​​to enable parallel readout, but also performs the actual readout.

[0092] That is, the pixel values ​​provided for parallel readout are now read out in parallel, i.e. via image sensor columns, using the same means as for a conventional readout, used for reading out pixels having the same physical image sensor row location.

[0093] The action may be performed by the same or a different device than the one that performed the previous action. As mentioned above, the previous action may be performed by an image sensing device or circuit, e.g., image sensing device 433, or an image sensor device or circuit, which itself comprises an image sensor, e.g., image sensor 431 integrated in a single device or unit such as a chip. The action may be performed by the same device or a separate device for operating and / or with such an image sensor device, e.g., a chip. The separate device may comprise an A / D converter. It should be noted that a single device having further functionality in addition to an image sensor by itself may still sometimes be referred to as an image sensor in the art.

[0094] As indicated above, in the sequence of parallel readouts, this action of reading out the WAM positions or rows one at a time may be repeated, whereby the pixel values ​​of all WAM pixels are read out. Any A / D conversion may also be performed sequentially, so that the analog pixel values ​​for the pixel values ​​of one and the same first WAM position in a column of the image sensor are A / D converted in parallel to the corresponding digital pixel values, and then the pixel values ​​of the second WAM position, for example the pixel values ​​of the next WAM position, are read out in parallel. This means that the pixel values ​​from each parallel readout may be stored digitally, and when the parallel readouts of all the number of WAM widths are performed sequentially, only after the parallel readouts of the number of WAM widths, as described above, there are digital pixel values ​​for all WAM pixels available for further processing. The digital pixel values ​​may then be further processed by some prior art methods suitable for this, for example to identify the light intensity center position with sub-pixel resolution by a peak detection algorithm.

[0095] In some embodiments, each pixel group corresponds to a binned group of two or more pixels, and the analog pixel values ​​of each pixel group are the binned pixel values ​​of the pixels of the pixel group, as illustrated and described in some further detail below and in relation to Figures 6A-6B.

[0096] With 2 pixel binning groups, a WAM width of 31 such groups covers 62 pixels. Each WAM in the case of binning is a "binning resolution". With binning and the resulting "combined pixels", fewer reads of the same area or image sensor coverage are required as each binning group is read out. For example, assuming there is an 11 pixel WAM without binning, 11 sequential reads are required. With 3 pixel binning, only 5 sequential reads are required for a WAM width of 5 binning groups covering a sensor area larger than 11 pixels, i.e. 15 pixels.

[0097] Binning can also speed up the identification of each group of pixels, i.e., the identification of a "winner" in each image sensor column, because binning allows the analog pixel values ​​of each group of pixels to reach or exceed the threshold pixel value more quickly.

[0098] In practice, we are interested in binning groups of 2, 4, and perhaps up to 8; anything larger than that risks providing too coarse a resolution for identifying valid "winner" groups.

[0099] In some embodiments, parallel reads to different WAM locations, e.g., different WAM rows, such as those in action 506, are performed sequentially in a sequence, the length of the sequence corresponding to the WAM width, with each such parallel read being associated with a respective WAM location. For example, as shown in FIG. 3E, a first parallel read from all columns 352-1, ..., 352-M is performed to a first WAM row 364-1, then a second parallel read from all columns 352-1, ..., 352-M is performed to a second WAM row 364-2, etc. Thus, the actual reads in action 508 may be performed sequentially.

[0100] In some embodiments, after the parallel readout sequence, the pixels of at least the region of the image sensor 431 are reset and exposed a second time during a second exposure. As a result, the exposed pixels each achieve a new second analog pixel value corresponding to the light sensed during the second exposure. The respective WAMs, for example the respective WAMs 358, are referred to as respective first WAMs. In these embodiments, based on the respective first WAMs, respective new second WAMs in the respective image sensor columns of the region are provided. Then, based on the second exposure, pixel values ​​of pixels associated with the same WAM position in the respective second WAMs are provided for parallel readout. This is illustrated and explained in some further detail below in relation to FIG. 7.

[0101] Thus, in these embodiments there is a double exposure, but the second WAM is not made a new first WAM, instead to some extent the first WAM of the first exposure can be "reused". This can be interesting to save time and / or apply when it is known that the change in the position of an object, for example object 420, between the exposures was not or is sufficiently small and / or known, for example when using a different light than during the first exposure, and it is desirable to obtain information about this same or nearly the same position on the object from the second exposure.

[0102] The respective second WAMs, in some embodiments, may be identical in width and position on the image sensor as the respective first WAMs, i.e., cover the same image sensor pixels as the first WAMs, or in other words may not change between exposures in this respect. In some embodiments, instead, there is a predefined or predetermined offset of the image sensor rows for the respective second WAMs compared to the respective first WAMs, and / or a predefined or predetermined change in WAM width. For example, the second WAMs may be offset to compensate for movement of the object between the first and second exposures. In another example, a second WAM with a wider WAM width may be placed in the same central position as the first WAM, providing some margin so that any new respective intensity centers and the first WAM widths around them are certainly covered, or likely to be covered, by the respective second WAMs. This means that although several more WAM positions are provided for readout, there is no need to identify any new pixels or groups of pixels that reach threshold first during exposure as in action 503, or to provide or read out any new sensor row position identifiers (that are identical to those of each first WAM) associated with each second WAM as in action 507. The actual center pixels per column of the wider second WAM can later be digitally identified, and the digital pixel values ​​of the pixels within the first WAM width can be used to detect the actual peak positions with sub-pixel resolution by a peak detection and identification algorithm.

[0103] Another example is where a first WAM covers more image sensor rows than a second WAM, such that the WAM width of the second WAM covers fewer image sensor rows. For example, if binning as described above is used for the first WAM, then the second WAM, e.g., without binning, may be placed at the binning group position of the first WAM that first reached the threshold during the first exposure.

[0104] In some embodiments, if multiple pixel groups in the same image sensor column, for example any of columns 352-1, ..., 352-M of the region, each having analog pixel values ​​that indicate reaching or exceeding the threshold pixel value simultaneously, only one of the multiple pixel groups is selected as the respective pixel group identified for that image sensor column by one or more predefined criteria.

[0105] For example, it may be predetermined or predefined that the lowest or highest image sensor row location is selected as the "winner," or some other criteria may be applied that results in only one "winner" per column of the region. While it is theoretically highly unlikely that two pixels will reach the same threshold at exactly the same time, practical implementation of the embodiments herein as described above may indicate that two or more pixels reach the threshold at the same time and need to be handled. When this problem is realized, handling it as described above is a simple solution and may be relatively easy to implement. How this may be implemented in hardware is illustrated below in connection with FIG. 9.

[0106] FIG. 6A shows a schematic of a WAM 658-m for column m for use with pixel groups of 2 pixels, e.g., binning groups. The WAM width 657 corresponds to a group of 3 pixels. The example of FIG. 6A and WAM 658-m can be compared to the example of FIG. 3C and WAM 358-m. Thus, the difference is that instead of a pixel group of 1 pixel, here it is 2 pixels, and there is another WAM width. Since there are 2 pixels per group, the WAM width 657 for the 3 groups corresponds to a width of 6 pixels.

[0107] The WAM locations are, for example, in binned pixel groups, and since the resolution is here two pixels, there are three WAM locations, or WAM rows 664-1, . . . , 664-3.

[0108] 3C, WAM 658-m will be centered about the pixels in column m identified as the "winner," i.e., the pixels in column m that have binned pixel values ​​that reach or exceed a threshold value first during exposure. In other words, the center position of each WAM is located, in this example, at the pixels identified as the "winner" pixels in their respective columns.

[0109] Figure 6B shows a schematic example of the WAMs of Figure 6A when they are centered on the pixels identified as in Figure 3B. Figure 6B can be compared to Figure 3D to find differences in the WAMs even though the intensity lines 351 and image sensors 331 etc. are identical. Figure 6B also shows one WAM per column, i.e., each WAM 658-m for m=1,...,M, but centered on the pixels of the identified "winner" rather than on the pixels of each column m. In this case there is again one WAM 658 per column.

[0110] FIG. 7 is another flow diagram that generally illustrates an embodiment of a method including double exposure according to some embodiments herein. An example of some embodiments already mentioned and discussed in relation to FIG. 5 above is shown. This example may be considered as an extension of the method and actions shown in FIG. 5 and may be applicable to some embodiments. For example, actions 701-702 may be or may correspond to actions 501-502. In the context of the embodiment related to FIG. 7, the exposure related to action 502 is considered as the first exposure, i.e. exposure #1. Action 703 may correspond in whole or in part to actions 503, 505.

[0111] The WAM with actions 503, 505 is considered the first WAM, i.e., WAM#1, in the context of the embodiment associated with Figure 7. Although not shown in Figure 7, there may also be actions corresponding to one or more of actions 504, 507, 509. Actions 704-705 may be or may correspond to actions 506, 508, respectively.

[0112] The actual dilation involves actions 706-709. As in the case for some embodiments discussed above in relation to FIG. 5, the first action of dilation is usually action 706 for the second exposure #2 which deletes pixel values ​​originating from the first exposure #1 of the image sensor, so all pixel values ​​of the pixels of WAM#1 should be read before performing dilation with actions as described below. That is, first, the parallel readouts for the separate WAM positions, e.g. each of 364-1, ..., 364-5 or 664-1, ..., 664-3, each associated with a position of WAM#1, may be performed sequentially in a sequence of such parallel readouts to read values ​​of all pixels or groups of pixels respectively associated with WAM#1. The length of the sequence corresponds to the width of WAM#1, e.g. 5 for WAM width 357 and 3 for WAM width 657. Thus, each such parallel read is associated with a respective WAM#1 location, e.g., a first parallel read is associated with WAM#1 location 364-1 or 664-1, a second parallel read is associated with WAM#1 location 364-2 or 664-2, etc.

[0113] It should be noted that the actions relating to FIG. 7, as well as the actions of FIG. 5, may be taken in any suitable order and / or may be performed with full or partial overlap in time, where this is possible and appropriate.

[0114] Action 706 After performing the parallel readout sequence, the pixels of at least the region of the image sensor are reset and exposed a second time during a second exposure, i.e., exposure #2, such that each exposed pixel achieves a new second analog pixel value corresponding to the light sensed during the second exposure. The second exposure may be of a second exposure time, which may be the same as the first exposure time, or may be longer or shorter than the first exposure time, depending on the circumstances and / or reasons for the second exposure.

[0115] Action 707 Based on the respective first WAM, a respective second WAM in each of the respective image sensor columns of the region, eg, columns 352-1, . . . , 352-M, is provided.

[0116] That is, in this case, there is no second WAM provided as the first WAM, i.e., there is no new pixel or group of pixels identified as a "winner" or the like, instead, to some extent, the WAM of the first exposure is "reused." As already mentioned above, the second WAM may be identical to the first WAM, or the WAM width may vary and / or the location of the WAM center on the image sensor column, for example, may vary.

[0117] Action 708~709 These actions may correspond to actions 506, 508, with the only change being that, for example, a second WAM is used instead of the first WAM, and that it is the second exposure rather than the first exposure that results in the pixel value. Thus, in some embodiments, the same pixel of the sensor may be read out from another second exposure, for example when the second WAM is at least as wide as the first WAM and covers the first WAM. The pixel value from the second exposure may come from the reflection of light from the same or substantially the same location on the object, for example object 420, as during the first exposure. In this way, new additional information from other light about the object may also be provided, and the embodiments herein should require fewer readouts than before.

[0118] 8 is a schematic block diagram illustrating an example of how a respective pixel element 870 of an image sensor, e.g., image sensor 331 or 431, may be implemented to support and perform some embodiments herein. Thus, pixel element 870 may be, e.g., pixel P of image sensor 331 shown in FIG. m,n It may correspond to any one of the following:

[0119] A light sensing element 871 of pixel element 870 is shown diagrammatically along with associated additional circuitry. The light sensing element 871 and connected pixel bus 874, which may correspond, for example, to a column of image sensor 331, may be of the prior art. The following focuses on the additional circuitry, its functionality, and how it is involved in supporting and / or performing the actions discussed above.

[0120] The additional circuitry comprises a comparator 877, the first input of which is connected to a threshold line 875, which may be shared with other pixel elements, and the second input of which is connected to the output of the light sensing element 871, which outputs an analog pixel value corresponding to the sensed light in the form of a pixel output voltage. The threshold may be obtained in action 501 and may be available via the threshold line 875. It should be understood that the comparator is an example of a way of detecting that the pixel value reaches or exceeds the threshold during exposure and inverting the comparator output 884, for example to H or "1". Thus, the comparator 877 and its use here is an example of an implementation related to action 503. The comparator 877 also has an input connected to a comparator reset line 879, which may be shared with other pixel elements, and is controlled to reset by the reset line. The comparator output 884 connects to a global OR net, which in FIG. 8 comprises a global OR line 885 and a global OR switch 886. The global OR line 885 is shared with other pixel elements in the same column. The comparator output 884 also connects to a data input of a D flip-flop 887, which is also described further below.

[0121] The additional circuitry further comprises a D flip-flop 887 interconnected with the corresponding D flip-flop of the adjacent pixel element in the same column. These are included in the implementation of placing and reading out the WAM, for example, in connection with actions 505, 506, 508. This is further described below. A D flip-flop clock line 889 is connected to the clock input of the D flip-flop 887 for providing a clock signal to the D flip-flop 887, which may be shared with the D flip-flops of other pixel elements. Furthermore, the D flip-flop 887 has a data input connected to the comparator output 884 for setting the D flip-flop to a value, for example, setting the D flip-flop to "1", when the pixel value reaches or exceeds a threshold value. Furthermore, the D flip-flop 887 has a first data input / output 891 connected to a corresponding D flip-flop (not shown) of one of the two pixel elements adjacent to the pixel element 870 in the column. The D flip-flop 887 further has a second data input / output 892 connected to a D flip-flop (not shown) corresponding to the other second of the two adjacent pixel elements in the column. It should be noted that when one of the first data input / output 891 and the second data input / output 892 is operating as an input, the other acts as an output, and vice versa. This is controlled via the D flip-flop shift direction line 895, which is used when the D flip-flop 887 is operating as a shift register.

[0122] A D flip-flop reset line 893 is further connected to the reset input of the D flip-flop 887 for resetting the D flip-flop 887. The D flip-flop reset line 893 may be shared with corresponding D flip-flops of other pixel elements. The D flip-flop 887 also has a lock control input 894 connected to the global OR net, so that the global OR net "locks" the value that the D flip-flop 887 has on its data input, i.e. the comparator output 884, when a pixel in the column reaches or exceeds a threshold value, meaning that the D flip-flop 887 will not be affected by the value of its data input, i.e. the comparator output 884, thereafter, until, for example, the next reset.

[0123] Also shown is the D flip-flop shift direction line 895, which may be shared with other pixel elements in the same column, for example, to control the shift direction of how the D flip-flops in the column operate as shift registers, for example up or down in the column (left-right in the figure), i.e. to switch the input-output direction, thereby shifting and moving the positions in the WAM to indicate one WAM position for readout at the time. This is explained further below. The D flip-flop 887 in the column and the corresponding D flip-flops of the other pixels are intended to operate as a shift register after the entire OR net of the column has locked the data inputs of the D flip-flops, as will be understood from the following description. In practice, this is after exposure and is used to provide pixel values ​​for readout.

[0124] Essentially, the comparator output 884 connected to the data input of a D flip-flop 887 acts as an input to set the value of the D flip-flop 887 during exposure. After exposure, the D flip-flop 887 acts as a shift register shifting data from input to output using a first data input / output 891 and a second data input / output 892, independent of the comparator output 884.

[0125] At the start of exposure in action 502, the light sensing element 871, the comparator 877, the D flip-flop 887 may be reset to precharge the global OR net (precharge not shown). During exposure, the pixel voltage output from the light sensing element in the same column as the light sensing element 871 is monitored by a comparator corresponding to the comparator 877. In the following, the pixel in the column that first reaches the threshold corresponding to the voltage on the threshold line 875 is assumed to be the pixel corresponding to the pixel element 870 shown in the figure, which flips the comparator 877 to generate a "1" on the global OR net, which causes the global OR switch 886 to generate a lock signal to the D flip-flop 887 through its lock control input 894. The global OR net "locks" the current output values ​​for the D flip-flops of all the comparators to, for example, "0" or "1", where only one "1" should be "1" corresponding to one row position per column, indicating the row of the pixel that first reached the threshold in the column. In other words, this is an example of an implementation of action 503. As an example for handling the case where two or more "1"s indicating a "winner" occur in the same column, please also refer to the example of Figure 9 discussed below.

[0126] After some time, the exposure time ends and all light sensing elements including light sensing element 871 achieve their respective pixel values ​​after exposure. Thus, at this time, the only "1" in each column is the pixel element, e.g. pixel element 870, that first reaches or exceeds the threshold in the column, and thus indicates the row position of this pixel in the column. For example, the first input / output 891 of the D flip-flop may hold a "1". As explained above, the pixel that first reaches or exceeds the threshold is further associated with a position (e.g. the center position of the WAM used). Thus, said only "1" may be appropriately shifted by the D flip-flop of the column by "half" the WAM width, i.e. half the WAM size in the column, in the up direction using the D flip-flop shift direction line 895, the D flip-flop clock line 889, the first data input / output 891 and the second data input / output 892. In practice, "half" may mean a shift that is actually (WAM width-1) / 2 if symmetric WAM widths are used around a central position, and thus an odd WAM width. Thus, after the shift, a "1" may indicate the first WAM position in this column, for example, the WAM position of the WAM in WAM row 364-1.

[0127] In the example shown, a transmission gate 898 connected to the second data input / output 892 is configured such that if the second data input / output 892 is a "1", such as after a "1" has been shifted therein, it is time to read out the pixel value of the WAM location associated with the pixel element 870, so that the light sensing element receives a control signal so that the pixel value can be provided to the pixel bus 874, read out and then A / D converted, etc. Appropriate ones of the control lines 873 to the light sensing element may be controlled by the transmission gate 898, although typically there would also be other overall control lines, e.g. for reset, activation of readout, etc.

[0128] Assume that a "1" is shifted into the first WAM position in the column, and a pixel element at that position similar to pixel element 870 provides its pixel value for readout onto pixel bus 874. Then the "1" may be shifted "down" one position again, so that the next WAM position and the pixel value of the pixel at this WAM position are available to be read out, and so on until the pixels of the entire WAM are read out, i.e., for all positions of the WAM, depending on the WAM width. A similar procedure continues in parallel for other columns, so that pixel values ​​of the same WAM position, e.g., the same WAM row, of different image sensor columns and different image sensor rows may be read out in parallel, as described with respect to the embodiments herein.

[0129] From the above, it should be understood that the WAM width should determine how the D flip-flops are controlled and shifted using the D flip-flop shift direction line 895 and the D flip-flop clock line 889.

[0130] Of course, it is not necessary to start from the "top" of each WAM and shift down as in the above example, but rather one may start from the "bottom" and shift up, or read from the WAM positions shifted in any other order, but the order used should be predetermined or predefined.

[0131] Still further, as discussed in connection with, for example, actions 504, 507 and 509, what is of interest is to identify and read out, for each WAM, the pixel in the image sensor row location, indeed at least one particular WAM location, such as the center location, i.e., column, that first reaches or exceeds the threshold. FIG. 8 provides an example of how this may be accomplished for pixel element 870. Pixel element 871 has access to a row location memory 897, or row location memory, which may be a read-only memory and may correspond to a hard-coded, e.g., N-bit, number of bits that identifies the image sensor row location of pixel element 870. Row location memory 897 is connected to a row number bit bus 883, where the number of bits may be output bit by bit, i.e., continuously, using a clock signal connected to row location memory 897 via row number clock signal line 881, which may be shared with other pixel elements. The output of the bit number can be controlled to occur for the pixel indicated by the "1" in the column, e.g., for one pixel of each WAM, e.g., the pixel in the column that first reaches or exceeds the threshold, and thus corresponds to the central position of the WAM in that column.

[0132] An alternative embodiment of row position memory 897 may be based on providing address bits to the pixel elements from row lines.

[0133] Fig. 9 is a schematic block diagram to show how the arbiter functionality for some embodiments can be implemented by logic in hardware. This is an example of a proportional simple solution where, for example, per column, there may be multiple pixels corresponding to pixel element 870 of Fig. 8, which are the pixels designated as the "winner" in the column, for example by a "1", i.e. the multiple pixels designated as the first to reach or exceed the threshold in the same column, as in the example of Fig. 8. Both Fig. 9 and this example are quite self-explanatory with the logic elements and transitions shown, and the solution is based on the serial connection of AND gates in a sequence. As can be seen in this example, what is assumed is that two of the four input lines 992a-1, ..., 992a-4 on the left side are set to "1", and therefore both are designated as the first to reach or exceed the threshold. The input value at input lines 992a-1, ..., 992a-4, which in this example is "0101", may correspond to the value output by a D flip-flop, such as D flip-flop 887, on a D flip-flop output line, such as D flip-flop output line 892. Control line 999 may be used by the end of the exposure, such as after a triggered exposure, to activate the function by being set to "1". As shown in Figure 9, the control line 999 switching from "0" to "1" results in the output of the first AND gate in the sequence also switching from "0" to "1" prior to the first of the input lines having a "1", here input line 992a-2, thereby preventing any input lines occurring later in the sequence from outputting a value other than "0". As a result, input lines 992a-1, ..., 992a-4 have only one "1" and the output value at output lines 992b-1, ..., 992b-4 will be "0100" in this example, indicating only one "winner."As will be appreciated, the corresponding functionality may be achieved with slight modifications and / or other logic elements, and if the functionality does not need to be implemented in hardware, there are many more possibilities for selecting which of the multiple row instructions in a column to select so that the readout is based on only one. Alternatives may include, for example, selecting the first "winner" or last "winner" instruction in a column, or the closest "winner" instruction among adjacent columns, or randomizing the "winner" instructions in a column.

[0134] Fig. 10 is a schematic block diagram for illustrating an embodiment of how one or more devices 1000, i.e. device 1000, may be configured to perform the methods and actions discussed in relation to Fig. 5 and / or Fig. 7. The device 1000 may correspond to the devices already mentioned above for performing the embodiments herein, such as for performing the methods and / or actions described above in relation to Fig. 5 and Fig. 7. Thus, the device 1000 is for said provision of pixel values ​​for reading out from at least said region of pixels of an image sensor, e.g. image sensor 431.

[0135] The apparatus 1000 may comprise one or more hardware modules including a processing module 1001 such as a processing means, one or more processing circuits, e.g., circuits such as a processor, and / or one or more software modules for performing the methods and / or actions described above.

[0136] The apparatus 1000 may further comprise a memory 1002 that may contain, store, or otherwise include a computer program 1003. The computer program 1003 includes "instructions" or "code" directly or indirectly executable by the apparatus 1000 to perform the methods and / or actions described above. The memory 1002 may comprise one or more memory units and may be further configured to store data, such as configurations, data and / or values, that are involved in or for performing the functions and actions of the embodiments herein.

[0137] Furthermore, the device 1000 may include a processing circuit 1004 involved in processing and, e.g., encoding, data as an example hardware module, which may include or correspond to one or more processors or processing circuits. The processing module 1001 may, for example, be "embodied in" or "implemented by" the processing circuit 1004. In these embodiments, the memory 1002 may include a computer program 1003 executable by the processing circuit 1004 such that the device 1000 is operated or configured to perform the methods and / or actions described above.

[0138] The device 1000 may comprise an input / output (I / O) module 1005, configured for example for the processing module 1001 to be involved by performing some communication between other units and / or devices, e.g., exchanging information with other devices. The I / O module 1005 may be embodied, when applicable, by obtaining, e.g., a receiving module and / or by providing, e.g., a transmitting module.

[0139] Further, in some embodiments, the apparatus 1000 includes one or more of an acquisition module, an exposure module, an identification module, an association module, a supply module, a storage module, a reset module, a read module, and a selection module as example hardware and / or software modules for performing the actions of the embodiments herein, such as by the processing module 1001. These modules may be implemented in whole or in part by the processing circuit 1004. Thus, as follows:

[0140] The apparatus 1000, and / or the processing module 1001, and / or the processing circuit 1004, and / or the I / O module 1005, and / or the acquisition module are operative or configured to acquire said information indicating the threshold pixel value and the WAM width.

[0141] The apparatus 1000, and / or the processing module 1001, and / or the processing circuit 1004, and / or the exposure module are operative or configured to expose said pixels of at least said area of ​​the image sensor, whereby each exposed pixel achieves said analog pixel value.

[0142] The apparatus 1000, and / or the processing module 1001, and / or the processing circuit 1004, and / or the identification module are operative or configured to identify the respective group of one or more pixels whose analog pixel values ​​during the exposure first reach or exceed the threshold pixel value in the respective image sensor columns of the region.

[0143] The apparatus 1000, and / or the processing module 1001, and / or the processing circuit 1004, and / or the association module are operative or configured to associate each of the identified pixel groups with the respective specific WAM positions of the respective WAMs in the respective image sensor columns of the region.

[0144] The apparatus 1000, and / or the processing module 1001, and / or the processing circuit 1004, and / or the I / O module 1005, and / or the supply module are operative or configured to supply the pixel values ​​associated with the same WAM position in each WAM of a pixel based on the exposure for parallel readout per WAM position.

[0145] In some embodiments, the apparatus 1000, and / or the processing module 1001, and / or the processing circuit 1004, and / or the storage module are operative or configured to store the respective image sensor row position identifiers that identify the image sensor row positions of the respective identified groups of pixels.

[0146] In some embodiments, the device 1000, and / or the processing module 1001, and / or the processing circuit 1004, and / or the I / O module 1005, and / or the supply module are operative or configured to supply the stored respective sensor row position identifiers associated with the respective WAMs for readout associated with the respective WAMs.

[0147] In some embodiments, the apparatus 1000, and / or the processing module 1001, and / or the processing circuit 1004, and / or the I / O module 1005, and / or the supply module are operative or configured to perform the parallel readouts of separate WAM positions sequentially in the sequence, the length of the sequence corresponding to the WAM width, and each such parallel readout being associated with a respective WAM position.

[0148] In some embodiments, the apparatus 1000, and / or the processing module 1001, and / or the processing circuit 1004, and / or the reset module, and / or the exposure module are operative or configured to reset the pixels of at least the region of the image sensor after performing the parallel readout sequence, and then perform the second exposure during the second exposure, thereby causing the exposed pixels to achieve the new second analog pixel values, respectively.

[0149] In some embodiments, the apparatus 1000, and / or the processing module 1001, and / or the processing circuit 1004, and / or the I / O module 1005, and / or the supply module are operative or configured to supply, based on the respective first WAM, a new second WAM in the image sensor column of the region, and to supply pixel values ​​of pixels associated with the same WAM position in the respective second WAM based on the second exposure for parallel readout.

[0150] In some embodiments, the apparatus 1000, and / or the processing module 1001, and / or the processing circuitry 1004, and / or the exposure module are operative or configured to expose pixels during exposure to the light reflected from an object as part of 3D imaging of the object based on optical triangulation.

[0151] In some embodiments, the apparatus 1000, and / or the processing module 1001, and / or the processing circuit 1004, and / or the I / O module 1005, and / or the readout module are operative or configured to read out (508) the pixel values ​​provided for parallel readout.

[0152] In some embodiments, the device 1000, and / or the processing module 1001, and / or the processing circuit 1004, and / or the I / O module 1005, and / or the selection module are operative or configured to select only one of the multiple pixel groups as the respective pixel group identified for the image sensor row by the predefined one or more criteria if the analog pixel values ​​of the multiple pixel groups in the same image sensor row of the region each indicate that they simultaneously reach or exceed the threshold pixel value.

[0153] 11-12 are schematic diagrams illustrating some embodiments of a computer program and its carrier.

[0154] FIG. 11 shows a schematic diagram of a hardware synthesis device 1101 and a second computer program 1103, which loads and executes the second computer program 1103 to synthesize and / or configure hardware, typically a chip, for example, i.e., as the device 1000 and / or the image sensing device or circuit 433 as some embodiments described herein above. The chip to be synthesized may be, for example, or based on Field Programmable Gate Array (FPGA) technology, or similar, or may be, for example, an Application Specific Integrated Circuit (ASIC). The hardware synthesis device 1101 may be conventional for synthesizing hardware of the type of hardware to be synthesized. The second computer program 1103 may be based on some traditional programming language for hardware synthesis, for example, a suitable hardware description language (HDL), such as Very High Speed ​​Integrated Circuit Hardware Description Language (VHDL), or some other programming language suitable or specific to the type of hardware to be synthesized. In some embodiments, the second computer program 1103 may be or include binary files and / or files in some proprietary format that describe the design of an ASIC executable by some of the equipment of the system, for example when an ASIC is synthesized, for its manufacturer to use in manufacturing the ASIC. In such a case, the second computer program 1103 may include both a program, for example in VHDL, that is executed by the computer and the files thereby generated for use in manufacturing the ASIC.

[0155] The hardware synthesis device 1101 may, for example, comprise a computer running a second computer program 1103 and one or more computer-controlled devices connected to the computer that perform the hardware synthesis themselves, for example by running on an FPGA chip.

[0156] As used herein, synthesis of hardware refers to the existence of hardware formed by physically creating or generating, e.g., electronic circuitry, and / or the making of generally permanent physical changes to synthesizable hardware units or parts, e.g., FPGA chips. This is distinct from more conventional computer programs that run on existing hardware, e.g., utilizing electronic circuitry already implemented. Such computer programs control the hardware, e.g., to perform certain actions, but do not generate the hardware.

[0157] The first computer program 1003 is described above and may thus include instructions that cause the device 1000, e.g., the processing module 1001, and / or the processor 1004, and / or the image sensing device 433, to execute such instructions to perform the methods, such as those described above in relation to Figure 5 and / or Figure 7. In other words, the first computer program 1003 may be of a conventional type relating to software.

[0158] One or more carriers, or more specifically, data carriers, which are for example computer program products, as provided in some embodiments, as shown in Fig. 12, include a first computer program 1003 and / or a second computer program 1103, hereinafter referred to as computer programs 1003 and / or 1103. The carrier or each of said carriers may be one of electronic signals, optical signals, radio signals, and one or more computer readable storage media, such as for example one or more computer readable storage media 1201, as shown in the figure. Thus, the computer programs 1003 and / or 1103 may be stored in one or more computer readable storage media 1201. Transient propagating signals may be excluded from the carriers, and the data carriers may be named accordingly as non-transitory data carriers. Non-limiting examples of data carriers that are computer readable storage media include memory cards or memory units, such as memory sticks, disk storage media, or mass storage devices, for example based on hard disks or solid-state drives (SSDs). The one or more computer-readable storage media 1201 may be used to store data accessible over a computer network 1202, such as the Internet, or a local area network (LAN). The computer programs 1003 and / or 1103 may be provided as pure computer programs as well, or may be included in one or more files. The files may be stored in the one or more computer-readable storage media 1201 and may be made available, for example, by downloading to the computer network 1202, for example, via a server, as indicated in Fig. 12. The server may be, for example, a web server or a file transfer protocol (FTP) server.The one or more files may be executable files that are, for example, downloaded directly or indirectly to and executed by the device 1000, and / or the image sensing device 433, and / or the hardware synthesis device 1101 to configure or operate some or all of them as described above. The one or more files may additionally or instead be for intermediate download and compilation involving the same or another processor to make them executable prior to further download and execution.

[0159] It should be noted that any of the processing modules referred to above, e.g., in connection with Figure 10, may be implemented as software and / or hardware modules, e.g., existing hardware and / or as an ASIC or FPGA or the like. It should also be noted that any of the hardware modules and / or circuits referred to above may be included in, e.g., a single ASIC or FPGA, or may be distributed among several individual hardware components, individually packaged or assembled into a system on a chip (SoC).

[0160] As indicated above, for example, device 1000 and / or image sensing device 433 may be implemented as, for example, an implementation of a chip of an NSIP type processor integrated with an image sensor, as an NSIP architecture or the like, and / or implemented on an NSIP architecture or the like. At least some embodiments herein may be realized by programming, such as installing a computer program 1003 into a programmable image sensor chip, e.g., an NSIP chip, or for execution of controlling such a chip.

[0161] Furthermore, for ease of implementation, chips for realizing high speed image sensing device 433 supporting some state-of-the-art CMOS imager technologies, based on and / or implementing multiple silicon chips stacked on top of each other, for example based on ASICs, can be fabricated and used. This allows more logic circuitry to be used close to the pixel without compromising the pixel's performance by occupying useful space and making the active area of ​​each pixel small compared to the total pixel area. Such stacking allows one die to be optimized for analog pixel design and one die for digital operation, improving efficiency and allowing cost reduction by using two smaller area dies.

[0162] Those skilled in the art will also recognize that the modules and circuits discussed herein may refer to combinations of hardware modules, software modules, analog and digital circuits, and / or one or more processors configured with software and / or firmware stored, for example, in memory, which software and / or firmware executes to cause, for example, device 1500 to be configured and / or execute the actions described above in the embodiments herein.

[0163] Any identification by any identifier herein may be implicit or explicit. Identification may be unique in a particular context, for example, to a particular computer program or program provider.

[0164] The term "memory" as used herein may refer to a data memory for storing digital information, typically a hard disk, a magnetic storage device, a portable computer diskette or disk, a flash memory, a random access memory (RAM), or the like. Additionally, the memory may be an internal register memory of a processor.

[0165] It should also be noted that any enumerated terminology, such as first value, second value, first device, second device, etc., should itself be considered non-limiting, and the terminology itself does not imply any particular hierarchical relationship. Instead, enumerated naming should be considered simply a way to achieve discrete names in the absence of explicit information.

[0166] As used herein, the term "number" or "value" generally refers to any kind of binary, real, imaginary or rational number, etc. Furthermore, a "number" or "value" may be one or more characters, such as a character or a string of characters. A "number" or "value" may also be represented by a bit string.

[0167] As used herein, the phrase "configured to" may mean that a processing circuit is configured or adapted, by a software or hardware configuration, to perform one or more of the actions described herein.

[0168] As used herein, the terms "may" and "in some embodiments" are typically used to indicate that the described features may be combined with any of the other embodiments disclosed herein.

[0169] In the figures, features that may only be present in some embodiments are typically depicted using dotted or dashed lines.

[0170] When the words "comprise" or "comprising" are used, they are to be construed in an open sense, i.e., "consisting of at least".

[0171] The embodiments of the present specification are not limited to the embodiments described above. Various alternatives, modifications and equivalents can be used. Therefore, the above embodiments should not be interpreted as limiting the scope of the present disclosure, which is defined by the appended claims. [Explanation of symbols]

[0172] 105 Imaging System 110 Light source 111 Light Pattern 112 Light Line 120 First measurement object 121 Second measurement object 122 Conveyor Belt 123 Coordinate System 130 Camera, camera unit 141-1~141-N Profile image 143 Sensor-Based Coordinate Systems 251 Strength Line 252-1~252-M Image sensor array 254-1~254-N Image sensor rows 255 pixel positions 257 WAM width 258 WAM 259 Overall width 261 Light distribution 263 Center position 331 Image Sensor 351 Strength Line 352-1~352-M Image sensor array 354-1~354-12 Image sensor row 355 pixels 355-6 pixels 355a pixels 355b pixels 355c pixels 355-1~355-M pixels 356 A / D Converters, ADCs 356-1 First WAM Line 357 WAM width 358-1~358-M WAM 358-6 WAM 358-m WAM 364-1 WAM position 364-2 WAM position 364-3 WAM position 364-4 WAM position 364-5 WAM position 405 3D Imaging System 410 Light source 411 light 420 Object 430 Camera 431 Image Sensor 432 Field of view of camera 430 433 Image sensing device, image sensing circuit 434 Computing Equipment 657 WAM width 658 WAM 658-m WAM 664-1 WAM position 664-2 WAM position 664-3 WAM position 870 pixel element 871 Light-sensing elements 873 Control Line 874 pixel bus 875 Threshold Line 877 Comparator 879 Comparator Reset Line 881 rows of clock signal lines 883 row number bit bus 884 Comparator Output 885 Overall OR Line 886 Overall OR Switch 887 D Flip Flop 889 D Flip-Flop Clock Line 891 1st Data Input / Output 892 Second Data Input / Output 893 D Flip-Flop Reset Line 894 Lock Control Input 895 D Flip-Flop Shift Direction Line 897 Row Position Memory 898 Transmission Gate 992a-1~992a-4 Input lines 992b-1~992b-4 Output lines 999 Control Line 1000 devices 1001 Processing Module 1002 Memory 1003 Computer Programs 1004 Processing circuit 1005 Input / Output (I / O) Module 1101 Hardware synthesis device 1103 Second Computer Program 1201 Computer-readable storage medium 1202 Computer Networks

Claims

1. 1. A method performed by one or more devices (433, 1000) for providing pixel values ​​for readout from at least one region of pixels of an image sensor (431), comprising: - obtaining (501) information indicating a window (WAM) width (257, 357, 657) at a maximum value corresponding to a number of pixels and a threshold pixel value; - exposing (502) pixels of at least said area of ​​said image sensor (431), whereby each exposed pixel attains an analog pixel value corresponding to the light sensed during said exposure; - identifying (503) a respective group of one or more pixels (255, 355, 655) in a respective image sensor column (252, 352) of said region, the analog pixel value of said respective group of pixels during said exposure being the first in said respective column (252, 352) to reach or exceed said threshold pixel value; - associating (505) each of the identified pixel groups (255, 355, 655) with a respective specific WAM location of a respective WAM (258, 358, 658) having said WAM width (257, 357, 657) in said respective image sensor column (252, 352) of said region, whereby said respective WAM identifies a number of pixels in each column corresponding to said WAM width covering said respective identified pixel group and neighbouring pixels; providing (506) pixel values ​​of pixels associated with the same WAM position (364, 664) in said respective WAM (258, 358, 658) for parallel readout per WAM position, based on said exposure; The method includes:

2. 2. The method of claim 1, wherein the pixels corresponding to the WAM width are positioned around each of the identified pixel groups (255, 355, 655) in each column.

3. 3. The method of claim 1 or 2, wherein the respective specific WAM location of the respective WAM (258, 358, 658) with which the respective identified pixel group (255, 355, 655) is associated is a respective center location of the respective WAM (258, 358, 658).

4. - storing (504) respective image sensor row position identifiers identifying the image sensor row positions (254, 354) associated with each of said identified pixel groups (255, 355, 655); providing (507) the stored respective sensor row position identifiers associated with each WAM (258, 358, 658) for readout associated with said respective WAM (258, 358, 658), whereby the provided analog pixel values ​​of each WAM (258, 358, 658) can be mapped to and associated with image sensor row positions (254, 354) in addition to sensor column positions (252, 352); The method of claim 1 , further comprising:

5. 5. The method of claim 1, wherein parallel reads for separate WAM locations (364, 664) are performed sequentially in a sequence, the length of the sequence corresponding to the WAM width (257, 357, 657), and each such parallel read is associated with a respective WAM location (364).

6. 6. The method of claim 5, wherein after the parallel readout sequence is performed, the pixels of at least the region of the image sensor (431) are reset and exposed a second time during a second exposure, such that the exposed pixels each achieve a new second analog pixel value corresponding to light sensed during the second exposure, and the respective WAMs (258, 358, 658) are respective first WAMs (258, 358, 658), and respective new second WAMs (258, 358, 658) in the respective image sensor columns (252, 352) of the region are provided based on the respective first WAMs (258, 358, 658), and pixel values ​​of pixels associated with the same WAM positions (364, 664) in the respective second WAMs (258, 358, 658) are provided for parallel readout and based on the second exposure.

7. 7. The method of claim 1, wherein each pixel group corresponds to a binned group of two or more pixels, and the analog pixel values ​​of the respective pixel group are binned pixel values ​​of the pixels of the pixel group.

8. 8. The method of claim 1, wherein during the exposure, the pixels are exposed to light reflected from an object (420) as a partial 3D imaging of the object (420) based on optical triangulation.

9. The method of claim 1 , further comprising the step of reading (508) the pixel values ​​provided for parallel readout.

10. 10. The method of claim 1, wherein if multiple pixel groups in the same image sensor column (252, 352) of the region have analog pixel values ​​that each indicate reaching or exceeding the threshold pixel value simultaneously, then only one of the multiple pixel groups is selected as the respective identified pixel group for that image sensor column (252, 352) by one or more predefined criteria.

11. One or more devices (433, 1000) for providing pixel values ​​for readout from at least one region of pixels of an image sensor (431), comprising: Obtaining (501) information indicating a threshold pixel value and a window around a maximum (WAM) width (257, 357, 657) corresponding to a number of pixels; exposing (502) pixels of at least said region of said image sensor (431) such that said exposed pixels each attain an analog pixel value corresponding to light sensed during said exposure; identifying (503) one or more groups of pixels (255, 355, 655) in each image sensor column (252, 352) of said region, wherein said analog pixel values ​​during said exposure are the first in said respective columns (252, 352) to reach or exceed said threshold pixel value; Associating (505) each of the identified pixel groups (255, 355, 655) with a respective specific WAM location of a respective WAM (258, 358, 658) having the WAM width (257, 357, 657) in the respective image sensor column (252, 352) of the region, thereby causing the respective WAM in each column to identify a plurality of pixels and neighboring pixels corresponding to the WAM width that cover the respective identified pixel group; providing (506) pixel values ​​for pixels associated with the same WAM position (364, 664) in the respective WAMs (258, 358, 658) based on the exposure for parallel readout per WAM position; One or more devices (433, 1000) configured to:

12. said one or more devices (433, 1000) storing (504) respective image sensor row position identifiers identifying the image sensor row positions (254, 354) of the respective identified groups of pixels (255, 355, 655); providing (507) the stored respective sensor row position identifier associated with each WAM (258, 358, 658) for a readout associated with the respective WAM (258, 358, 658), such that the provided analog pixel value of each WAM (258, 358, 658) can be mapped to and associated with an image sensor row position (254, 354) in addition to a sensor column position (252, 352); The one or more devices (433, 1000) of claim 11, further configured to:

13. One or more computer programs (1003, 1103) comprising instructions which, when executed by one or more devices (433, 1000), cause the one or more devices (433, 1000) to perform the method of any one of claims 1 to 10, and / or instructions which, when executed by a hardware synthesis device (1101), synthesize and / or configure hardware as one or more devices (433, 1000) as described in claim 11 or 12.

14. One or more carriers comprising one or more computer programs (1003, 1103) according to claim 13, each of which is a computer readable storage medium (1201).

15. An imaging system (405) comprising one or more devices (433, 1000) according to claim 11 or 12 and a light source (410) configured to illuminate an object (420) with light (411) of a specific light pattern, wherein the light source (410) and the image sensor (431) are configured relative to each other in the imaging system (405) such that the light (411) of the specific light pattern, when reflected by the object (420), becomes incident light on the image sensor (431) when the pixel is at least partially exposed, whereby the pixel value for readout contains information convertible into information of three-dimensional characteristics of the object (420) relating to the position on the object (420) that reflected the light (411) and brought the incident light onto the image sensor (431).

Images