Image sensors, cameras and imaging systems with two or more focal planes

Abstract

To provide improvements regarding focus for cameras of imaging systems, particularly for 3D imaging based on light triangulation.SOLUTION: An image sensor 331 comprises an optical plate 337 arranged to cover an image sensing area 333 with at least two optical plate portions 334-1, 334-2 having different refractive properties. As a result, the image sensing area portions covered by the at least two optical plate portions 334-1, 334-2 become associated with different focus planes.SELECTED DRAWING: Figure 3A

Description

[Technical Field]

[0001] Embodiments herein relate to an image sensor for use with a lens, a camera comprising an image sensor and a lens, and an imaging system comprising a camera and configured for three-dimensional imaging based on optical triangulation. More particularly, embodiments herein relate to focal planes associated with the image sensor and lens, the camera, and the imaging system. [Background technology]

[0002] Industrial vision cameras and systems for factory and logistics automation may be based on three-dimensional (3D) machine vision, thereby capturing 3D images of scenes and / or objects. By 3D image, we mean an image that includes not only pixel-related information, such as intensity and / or color, but also "height" or "depth" information, as in traditional images. That is, each pixel of the image may include information that maps to the location of the imaged entity, e.g., the object, associated with the pixel's location. Processing may then be applied to extract information about the object's properties from the 3D image, thereby providing 3D information about the object that can be converted into various 3D image formats. Data with information about "height" or "depth" may be referred to as range data, which may therefore correspond to data from a height measurement of the object being imaged, or, in other words, from a range or distance measurement of the object. Alternatively or additionally, the pixel may include information about material properties, such as light scattering or reflection of specific wavelengths of light in the imaged area.

[0003] Hence, the pixel value may relate to the intensity and / or to the range data and / or to the material properties of the pixel.

[0004] Line-scan image data occurs when image data of an image is scanned or provided one line at a time, for example by a camera whose sensor is configured to sense and provide image data one line of pixels at a time. A special example of a line-scan image is image data provided by 3D triangulation of a so-called "sheet of light," such as a laser line. While lasers are often preferred, other light sources capable of providing a "sheet of light," i.e., a plane of light, can also be used, for example light sources that can provide less divergent light, or in other words, light that is "structured," such as light provided by a laser or light-emitting diode (LED).

[0005] 3D machine vision systems are often based on optical triangulation. In such systems, a light source illuminates an object with a specific light pattern, such as a sheet of light, resulting in a light or laser line on the object along which 3D characteristics of the object can be captured, corresponding to the object's profile. By scanning the object with such a line, i.e., line scanning, with movement of the line and / or object, multiple profiles can be captured, and a 3D image of the object can be formed.

[0006] A 3D machine vision system or device that uses a light plane for triangulation may be referred to as a system or device for 3D imaging based on light or light plane triangulation, or simply laser triangulation when laser light is used.

[0007] Typically, to generate a 3D image based on optical triangulation, the reflected light from the object to be imaged is acquired by the image sensor of the camera, and intensity peaks are detected in the image data. The peaks occur at positions corresponding to the locations on the imaged object where the incident light, e.g., corresponding to a laser line, is reflected from the object. The positions of the detected peaks in the image are then mapped to the positions on the object from which the light that led to the peaks was reflected.

[0008] Optical triangulation camera systems, i.e., imaging systems based on optical triangulation, therefore typically project a line of light onto an object to create a height profile from the object's surface. By moving the object relative to the associated camera and light source, information about the height profile from different parts of the object is acquired by imaging, which can then be used in combination with the system's knowledge of the relevant geometry to generate a three-dimensional representation of the object, i.e., provide 3D image data. This technique is described as capturing an image of a line of light as it is projected onto the object and reflected thereby toward the camera, and the location of the reflected laser line can then be extracted from the image. This is usually achieved by identifying the location of intensity peaks in the image frame, for example, using conventional peak-finding algorithms. Typically, although not necessarily, the imaging system is set up so that intensity peaks for the reflected light should be expected to occur for each row of sensors, and the location within the row maps to height or depth.

[0009] Imaging systems for 3D imaging based on optical triangulation such as the above are typically configured to have a focal plane aligned with the optical plane, so that reflections from objects will occur at different "heights" in the optical plane. For such cameras, the image plane of the image sensor would typically be perpendicular to the optical axis of the lens, and the focal plane would be parallel to the image plane. This would require the camera to be oriented toward the optical plane so that the image plane and focal plane are also parallel to the optical plane. However, for optical triangulation purposes, it is desirable to position the camera and line of sight at an angle relative to the optical plane. To enable optical triangulation systems to align the focal plane with the optical plane, the so-called Scheimpflug principle is typically utilized. Therefore, in such systems, the image plane of the image sensor in the camera and the camera lens are configured according to the Scheimpflug principle with respect to how the camera is positioned to view the optical plane so that the focal plane is aligned with the optical plane. For traditional optical triangulation applications, this works well, as all reflections occurring at the optical plane during scanning of the object can be focused onto the image sensor. Summary of the Invention [Problem to be solved by the invention]

[0010] In view of the above, it is an object to provide one or more improvements or alternatives to the prior art, such as providing focus improvements to cameras of imaging systems, particularly for 3D imaging based on optical triangulation. [Means for solving the problem]

[0011] According to a first aspect of an embodiment herein, the object is achieved by an image sensor for use with a lens, the lens being arranged to focus light onto an image sensing area of ​​the image sensor. The image sensor comprises an optical plate, equivalent to an optical window or an optical filter, arranged to cover the image sensing area with at least two optical plate portions. The optical plate portions have different refractive indices such that light incident from the lens and refracted by the optical plate portions towards the image sensing area travels different distances to be focused onto the image sensing area depending on which of the optical plate portions the light is refracted by. As a result, the portions of the image sensing area covered by the at least two optical plate portions are associated with different focal planes.

[0012] According to a second aspect of embodiments herein, the object is achieved by a camera comprising an image sensor according to the first aspect. The camera also comprises the lens described above. In some embodiments of the camera, separate optical plate layers are stacked on a transparent protective layer, with spaces separating the layers to prevent the layers from coming into direct physical contact with each other. The separate optical plate layers may in these embodiments be part of a separate lens section or unit of the camera that comprises the lens and is removably attached and / or configured to be removably attached to a housing section of the camera, the housing section comprising the image sensing area.

[0013] According to a third aspect of embodiments herein, the object is achieved by an imaging system for three-dimensional imaging of an object based on optical triangulation, comprising a camera according to the second aspect. The imaging system may comprise a first light source for providing first light in the form of a first light plane for illuminating the object as part of the optical triangulation. A camera having an image sensor may be disposed in the imaging system for acquiring first light reflected from the object as part of the optical triangulation. The imaging system may further be configured such that at least a first focal plane of the at least two focal planes is co-located with the first light plane. [Effects of the Invention]

[0014] Due to the additional focal plane or planes provided by the embodiments herein, different portions of an object can be simultaneously focused in the same image at different depths within the camera's field of view. This can be exploited, for example, in imaging systems based on optical triangulation, where the additional focal point can be used to perform focal imaging of other portions of the object simultaneously with conventional optical triangulation imaging. The additional focal point imaging can relate to the acquisition of focal 2D data of the object, or to further optical triangulation acquisition of image data from the object, for example, to enable faster full scans of the object and / or to detect unexpected movements that may occur between scans of the same position on the object, thereby avoiding erroneous 3D representations of the scanned object.

[0015] Furthermore, the embodiments herein allow for easy implementation, for example, with existing image sensors.

[0016] Example embodiments herein will now be described in more detail with reference to the accompanying schematic drawings, which are briefly described below. [Brief explanation of the drawings]

[0017] [Figure 1] FIG. 1 schematically illustrates an example of a prior art imaging system on which some embodiments herein may be based. [Figure 2A] FIG. 1 schematically illustrates a simplified example of a simplified prior art imaging system on which embodiments herein may be based. [Figure 2B] FIG. 1 schematically illustrates a cross-section of a prior art image sensor on which some embodiments herein may be based. [Figure 2C] 2C is a diagram schematically illustrating a top view of the prior art image sensor in FIG. 2B with cross sections marked; [Figure 3A]FIG. 2 is a diagram that schematically illustrates a cross-section of a first exemplary image sensor according to some embodiments herein. [Figure 3B] 3B is a diagram schematically illustrating a top view of the image sensor in FIG. 3A with cross sections marked. [Figure 3C] FIG. 4A is a diagram illustrating a first simplified example of an imaging system, according to some embodiments herein, comprising a camera with the image sensor of FIGS. 3A-3B. [Figure 3D] FIG. 4A is a diagram illustrating a simplified example of how two focal planes may relate to the image sensor and lens in FIGS. 3A-3B, for example, as part of a camera. [Figure 3E] FIG. 10 is a diagram illustrating schematically how light rays are affected by thick versus thin optical plate portions. [Figure 4A] FIG. 1 illustrates a schematic diagram of an imaging system setup with a camera according to some embodiments herein for imaging an object having a dot-patterned surface; [Figure 4B] FIG. 4B is a diagram schematically illustrating a front view of the object shown in FIG. 4A, showing the dot-patterned surface. [Figure 4C] 4A-4B show real images taken by a corresponding real imaging system, camera, and object as illustrated in FIGS. 4A-4B. [Figure 5A] FIG. 10 is a diagram that schematically illustrates a cross-section of a second exemplary image sensor according to some embodiments herein. [Figure 5B] 5B is a diagram schematically illustrating a top view of the image sensor in FIG. 5A with cross sections marked. [Figure 5C] 1A and 1B are diagrams illustrating schematically a camera system having a camera housing portion and a camera lens portion attached together to form a camera with an image sensor according to some embodiments herein. [Figure 6A]10A-10C are diagrams illustrating different examples of alternatives on how different refractive indexes of optical plate portions can be achieved for image sensors according to embodiments herein. [Figure 6B] 10A-10C are diagrams illustrating different examples of alternatives on how different refractive indexes of optical plate portions can be achieved for image sensors according to embodiments herein. [Figure 6C] 10A-10C are diagrams illustrating different examples of alternatives on how different refractive indexes of optical plate portions can be achieved for image sensors according to embodiments herein. [Figure 6D] 10A-10C are diagrams illustrating different examples of alternatives on how different refractive indexes of optical plate portions can be achieved for image sensors according to embodiments herein. [Figure 7A] FIG. 10 is a diagram schematically illustrating a second simplified example of an imaging system according to some embodiments herein. [Figure 7B] FIG. 10 is a diagram schematically illustrating a third simplified example of an imaging system according to some embodiments herein. DETAILED DESCRIPTION OF THE INVENTION

[0018] The embodiments herein are illustrative embodiments. It should be noted that these embodiments are not necessarily mutually exclusive. Components from one embodiment may be implicitly assumed to be present in another embodiment, but it will be clear to one skilled in the art how those components may be used in other illustrative embodiments.

[0019] Before going into details regarding various embodiments, the situation presented in the background will be further elaborated upon and the main principles behind the embodiments herein will be presented.

[0020] For conventional optical triangulation applications, a single focal plane aligned with the light plane, as described in the background art, works well, based on the Scheimpflug principle, allowing all reflections occurring at the light plane during object scanning to be focused onto the sensor. However, situations and potential applications have been identified where it would be desirable to be able to additionally focus outside the light plane to improve imaging capabilities and, for example, obtain focal 2D information, such as color, from the surface of the object being 3D imaged. It has been found difficult to generate good focus for both optical triangulation and separate 2D image data with conventional systems. While optical triangulation setups typically use the Scheimpflug principle to have maximum focus only around the light plane, it is typically desirable to use a large aperture to allow more light to reach the image sensor, narrowing the depth of focus.

[0021] Solutions that can still focus outside the light plane may involve additional cameras and / or lenses, but this adds complexity: the additional cameras also require imaging from different angles and separate images to capture the same thing, which may be undesirable in some cases.

[0022] Instead, the solution on which the embodiments herein are based is to create one or more additional focal planes that can be simultaneously used by the same camera and image sensor. According to the embodiments herein, this is achieved using an optical plate disposed between the lens and the image sensing area of ​​the image sensor, e.g., over the image sensing area, having at least two portions with different refractive indices such that light incident from the lens and refracted by the optical plate portions toward the image sensing area travels different distances before being focused on the image sensing area. As a result, the image sensing area portions covered by the at least two optical plate portions are each associated with a different focal plane. The different refractive indices may be achieved by different thicknesses of the optical plate portions and / or by different refractive indices of the materials from which the optical plate portions are made. This allows for simple implementation with existing image sensors, such as those in the form of image sensor chips. Image sensors typically have a transparent protective layer, typically made of glass, disposed over their image sensing (and therefore light-sensitive) area to protect the light-sensitive pixel elements and electronic circuitry. An example of a simple implementation of embodiments herein based on a conventional image sensor is to add an additional separate optical plate layer on top of the protective layer, with the composite layer forming one of the optical plate portions, while the other of the optical plate portions corresponds to the portion of the protective layer that is not covered by the separate optical plate layer. With knowledge of the lens, how the image sensing area is disposed relative to the lens, and knowledge of, for example, where the first focal plane would be located without the separate optical plate layer, it is possible to calculate and / or find by routine testing and experimentation the appropriate material and / or thickness of the separate optical plate layer to achieve an additional focal plane at a desired distance further away from the first focal plane for a camera with an image sensor. It may be preferable to use an additional optical plate layer made of the same material as the protective layer, for example, glass with the same refractive index, and to use the thickness of the additional optical plate layer to control where the additional focal plane will be located.

[0023] As is clear from the background art, since the application scope of the embodiments herein is in imaging systems for 3D imaging based on optical triangulation, such systems and the state of the prior art will be described and explained in some detail before further details about the embodiments herein are described in the optical triangulation context. This will also facilitate understanding of the benefits of the embodiments herein when used with imaging systems for 3D imaging based on optical triangulation. Therefore:

[0024] FIG. 1 schematically illustrates an example of an imaging system 105 for 3D imaging based on optical triangulation, as known from the prior art, and will be used to explain the basic principles on which such systems are based. The imaging system 105 may alternatively be named, for example, an imaging system for 3D machine vision based on optical triangulation for obtaining information about the 3D characteristics of a target object. The imaging system 105 is shown in the figure in normal operation, i.e., after a typical calibration has been performed, and thus the system is calibrated. The system 105 is configured to perform optical triangulation, here in the form of sheet-like light triangulation, i.e., optical triangulation in which a light plane is used. The imaging system 105 further comprises a light source 110, such as a laser, for illuminating the object to be imaged with a specific light pattern, illustrated and exemplified in the figure as a light plane 111. The light may be laser light, but is not necessarily so. The camera is typically configured and installed to have a focal plane that is co-located, or in other words, aligned with, the light plane 111 based on the Scheimpflug principle, so that object reflections occurring at the light plane are focused on the image sensor. In the illustrated example, the target object is exemplified by a first object 120 in the form of a car and a second object 121 in the form of a gear structure. The object to be imaged may be referred to as a measurement object. When the specific light pattern 111 is incident on the object, this corresponds to the projection of the specific light pattern 111 on the object, and may be observed when the specific light pattern 111 intersects with the object. For example, in the illustrated example, the specific light pattern 111, exemplified as a light plane, leads to a light line 112 on the first measurement object 120. The specific light pattern 111 is reflected by the object, or more specifically, by a portion of the object, at the intersection, i.e., in the illustrated example, at the light line 112. The imaging system 105 further includes a camera 130 having an image sensor (not shown in FIG. 1 ). The camera and image sensor are positioned with respect to the light source 110 and the object to be imaged such that a particular light pattern, when reflected by the object, becomes incident on the image sensor.An image sensor is a device, typically implemented as a chip, for converting incident light into image data. The portion of the object causes the incident light to be reflected onto the image sensor, which may be captured by the camera 130 and image sensor, and corresponding image data may be generated and provided for further use. For example, in the illustrated example, a particular light pattern 111 may be reflected toward the camera 130 and image sensor in a light line 112 on a portion of the car roof of the first object 120, thereby generating and providing image data with information about the portion of the car roof. According to the principles of optical triangulation, with knowledge of the geometry of the measurement system 105, e.g., knowledge of how image sensor coordinates relate to world coordinates, such as coordinates of a coordinate system 123, e.g., Cartesian coordinates, associated with the object being imaged and its context, the image data may be converted into information about 3D characteristics of the object being imaged in an appropriate format, e.g., in the form of a 3D shape or profile. The information about the 3D characteristics may comprise data describing the 3D characteristics in any appropriate format.

[0025] By moving the light source 110 and / or the object to be imaged, such as the first object 120 or the second object 121, in practice typically by scanning the object, so that multiple portions of the object are illuminated and cause reflected light to reach the image sensor, image data may be generated that depicts a more complete 3D shape of each object, corresponding to multiple successive profiles of each object, such as the illustrated profile images 140-1 to 140-N of the first object 120, where each profile image shows the outline of the first object 120 from which a particular light pattern 111 was reflected when the image sensor of the camera unit 130 sensed light leading to the profile image. As shown in the figure, a conveyor belt 122 or similar may be used to move the object through the particular light pattern 111 while the light source 110 and camera unit 130 typically remain fixed, or the particular light pattern 111 and / or camera 130 may be moved over the object so that all parts of the object, or at least all parts facing the light source 110, are illuminated and the camera receives light reflected from all parts of the object that are desired to be imaged.

[0026] As can be seen from above, for example, an image frame provided by camera 130 capturing an image of first object 120 and its image sensor may correspond to one of profile images 140-1 through 140-N. The location of each of the first object's contours shown in one of profile images 140-1 through 140-N is typically determined based on identifying intensity peaks in the image data acquired by the image sensor and locating these intensity peaks. Imaging system 105 and conventional peak-finding algorithms are typically configured to search for intensity peaks pixel-by-pixel column in each image frame. If the sensor coordinates are u, v, and, for example, u corresponds to a pixel location along a row in the image sensor and v corresponds to a pixel location along a column, as shown in the figure, then for each position u in the image frame searched, there is a peak location along v. The identified peaks in the image frame may be combined into a single "clean" profile image, as shown in the figure, and the sum of the image frames and profile image can be used to create a 3D image of first object 120.

[0027] FIG. 2A schematically illustrates a simplified example of a prior art imaging system 205 on which embodiments herein may be based. The depicted system may correspond to imaging system 105, but is shown in a more schematic and simplified diagram. The details shown in the figure are for facilitating understanding of and subsequent comparison with embodiments herein, which are presented further below. The depicted imaging system 205 may be considered to correspond to a basic configuration and includes a first light source 210 for illuminating an object 220 with a first light 211, typically laser light, as part of optical triangulation for 3D imaging of the object 220. The first light 211 is provided in the form of a light plane. The object may be referred to as a target or a measurement object. A camera 230 having an image sensor 231 is arranged to sense the first light reflected from the object 220 as part of the optical triangulation for 3D imaging, i.e., configured and positioned relative to each other for optical triangulation. This typically involves placing the first light source 210 and camera 230 in a predetermined fixed position and with a known relationship to each other for light triangulation. The camera 230 has a field of view that covers at least part of the light plane, particularly the portion of the object 220 and other objects that the system is configured to image that may intersect the light plane.

[0028] The imaging system 205 is set up and configured based on the Scheimpflug principle to have a focal plane 251 that is co-located and therefore aligned with the light plane corresponding to the first light 211 in the example, which, as noted above, is typically always the case in prior art optical triangulation based imaging systems.

[0029] As an alternative to a laser light or laser plane, another example of structured light that can be used as the first light in the embodiments herein and that is not traditionally referred to as a light plane, but which is equivalent to such in a similar effect, is a light edge, i.e., the edge of the illuminated area.

[0030] The object 220 is therefore illuminated and images may be acquired as in conventional optical triangulation, which may involve movement of the first light source 210 and / or measurement object 220 relative to one another, so that at different successive times, different successive portions of the object 220 are illuminated by the first light source 210 and the first light 211, and the reflected light is sensed by the camera 230 and the image sensor 231. The camera 230 may be a prior art camera, and the image sensor 231 may be a prior art image sensor 231.

[0031] Image frames and / or information derived from the image frames provided by camera 230 and image sensor 231 may be transferred, such as transmitted, to a computing device (not shown), e.g., a computer or the like, for further processing outside of camera 230. Such further processing may additionally or alternatively be performed by a computing unit or device (not shown) that is separate, i.e., separate from image processor 231, but still included in, such as integrated with, camera 230 or a unit that includes camera 230. The computing device (not shown) may be configured to control devices involved in optical triangulation and / or involved in other operations.

[0032] 1 at least with respect to optical triangulation, i.e., with respect to the light source, camera, image sensor, how they are positioned relative to each other and the measurement object, how the system is configured to move and image the measurement object, etc. Thus, the camera 230 and first light source 210 may be fixed relative to each other, and the system may be configured to move the object 220 relative to them. Through the sensing by the image sensor 231, each image frame is associated with a respective time point when the image frame was sensed, i.e., acquired, and with a respective portion of the measurement object 220 from which the image sensor 231 sensed the reflected first light 211 at each time point.

[0033] An object 220 is shown positioned within the field of view of a camera 230. A first light source 210 is configured to illuminate the measurement object 220 with a first light 211, which is reflected by the object 220 and acquired by the camera 230 and image sensor 231 as part of the optical triangulation.

[0034] 2B schematically illustrates image sensor 231 in cross section CC. Image sensor 231 may therefore be a prior art image sensor, and at least some embodiments herein are based on such prior art image sensors with some additions, as described below.

[0035] FIG. 2C illustrates schematically a top view of the image sensor 231 with cross section CC marked.

[0036] The image sensor 231 has an image sensing area 233, which is a portion of an image sensing portion 235 having light-sensitive pixel elements, e.g., in the form of an image sensor chip. The image sensing portion 235 is formed and / or disposed on or within a support structure 232, e.g., an image sensor substrate. The image sensing area 233 of the image sensing portion 235 is covered by a transparent protective layer 236, typically of glass, to prevent damage to the image sensing area 233 and the image sensing portion 235 while still allowing light and images to be sensed. Although not shown, there is typically a small space, or air gap, between the transparent protective layer 236 and the image sensing area 233. As can be seen in the top view of FIG. 2B , the protective layer 236 may extend somewhat outside the image sensing area 233 of the image sensing portion 235, such as by being supported by and / or over the support structure 232 to which it may be attached.

[0037] FIG. 3A schematically illustrates a first exemplary image sensor 331 according to some embodiments herein with a corresponding cross-section CC as compared to the prior art image sensor 231 in FIG. 2B to facilitate comparison and identification of differences.

[0038] FIG. 3B schematically illustrates a top view of the image sensor 331 in FIG. 3A with cross-section CC marked.

[0039] 3C schematically illustrates a simplified first example of an imaging system 305 according to some embodiments herein, comprising a camera 330 with an image sensor 331 and a first light source 310 for illuminating the object 320 with a first light 311, typically laser light, as part of optical triangulation for 3D imaging of the object 320. The first light 311 is provided in the form of a light plane. The camera 330 and image sensor 331 are positioned for sensing the first light reflected from the object 320 as part of said optical triangulation. The imaging system 305 is set up and configured based on the Scheimpflug principle such that the first focal plane 351-1 is co-located with, and therefore aligned with, the light plane corresponding to the first light 311.

[0040] In what follows, we will focus on the differences compared to image sensor 231 and imaging system 205 in Figures 2A-2C, i.e., what is new in Figures 3A-3C, and therefore, corresponding parts shown in Figures 2A-2C and 3A-3C may be the same.

[0041] As already mentioned, the embodiments herein may be formed based on existing image sensors according to the prior art, for example by adding or replacing layers above the image sensing area of ​​such existing image sensors.

[0042] The main structural difference between image sensor 331 and image sensor 231 is the additional separate optical plate layer 338 stacked on transparent protective layer 336. In addition to separate optical plate layer 338 and transparent protective layer 336, image sensor 331 also includes image sensing portion 335 having image sensing area 333, and support structure 332. Therefore, except for separate optical plate layer 338, the rest of the portions of image sensor 331 as shown in the figures may be as their counterparts in FIGS. 2B-2C , i.e., image sensor 331, and therefore image sensors according to embodiments herein, may be formed by modification of a prior art, e.g., conventional, image sensor by the addition of separate optical plate layer 338, which may at least partially cover the image sensing area and be disposed on an existing transparent protective layer covering the image sensing area. Further details regarding various embodiments herein will be discussed and disclosed below.

[0043] Separate optical plate layer 338 may simply be an additional layer of the same material as transparent protective layer 336, such as glass. It may be preferable to attach separate optical plate layer 338 to transparent protective layer 336 with an optical adhesive, which should have the same refractive index as separate optical plate layer 338 and / or transparent protective layer 336. The same refractive index means that the refraction that occurs is not significantly different and does not need to be considered. It may therefore be sufficient to design separate optical plate layer 338 with a focus on its thickness to achieve the desired result, i.e., to form an additional focal plane at a desired distance from the focal plane provided without separate optical plate layer 338. This will be explained further below. As used herein, optical adhesive refers to an adhesive that is transparent and has the optical properties of glass, at least after it is applied and cured.

[0044] Without optical adhesive, optical plate layer 338 and transparent protective layer 336 in physical contact, for example, by simply clamping them together, would typically result in undesirable and detrimental optical interference patterns for imaging. An alternative to optical adhesive is to place them with a small gap, i.e., an air gap or space between the layers to avoid physical contact, in other words, a gap similar to that typically between transparent protective layer 336 and image sensing area 333. Solutions based on such gaps and what may be suitable gaps will be further described and illustrated below.

[0045] Separate optical plate layer 338 has the consequence that image sensing area 333 is no longer covered solely by protective layer 336 with thickness t1 339-1; it is now covered instead by both separate optical plate layer 338 and protective layer 336, which together form optical plate 337. There are two regions or portions of different layer thicknesses covering image sensing area 333 of image sensing unit 335, corresponding to first optical plate portion 334-1 and second optical plate portion 334-2. First optical plate portion 334-1 now corresponds to the portion of image sensing area 333 that is not covered by separate optical plate layer 338, and therefore is covered only by transparent protective layer 336. Light incident on the first portion is refracted by transparent protective layer 336 and travels through its thickness, designated t1 339-1, to reach image sensing area 333 of image sensing unit 335. In the illustrated example, second optical plate portion 334-2 corresponds to the portion of image sensing area 333 that is covered by separate optical plate layer 338 and therefore covered by both this layer and transparent protective layer 336. Light incident on the second portion is refracted by and travels through a layer whose thickness is designated t2 339-2 and corresponds to the thickness of both separate optical plate layer 338 and transparent protective layer 336 before reaching image sensing area 333 of image sensing unit 335.

[0046] Note that in Figure 3B image sensing area 333 and optical plate portions 334-1, 334-2 are drawn slightly offset with respect to each other and at a slightly different scale, this is done solely to allow the areas and portions to be visually distinguished from each other in the figure.

[0047] Even if the separate optical plate layer 338 is made of the same material as the transparent protective layer 336, as described above, the thickness difference between t1 339-1 and t2 339-2 means that the two optical plate portions 334-1, 334-2 will have different refractive properties, i.e., the same light will be refracted differently toward the image sensing area 333 depending on which optical plate portion the light is refracted through. More specifically, because the frequency and wavelength of the incident light also affect how light is refracted, even light incident at the same frequency (or wavelength) and the same angle of incidence on each optical plate portion will be refracted slightly differently toward the image sensing area 333. However, it should be recognized that differences in refraction due to different frequencies of light can be considered negligible under most practical operating conditions for the embodiments herein. Because that is the practical situation, its effect will be mentioned further. The fact that light of different frequencies at different angles of incidence is refracted differently is, of course, not new, and its effect also exists in prior art imaging systems, such as imaging system 205.

[0048] As will be appreciated by those skilled in the art, light incident on image sensing area 333 comes from the lens of a camera, such as camera 230 or 330, although such lens is not shown in FIGS. 2 and 3. Differences in how light is refracted by the optical plate portions manifest as a focal "offset." In other words, depending on which optical plate portion the light is refracted through, it will travel a different distance to be focused on image sensing area 333. This is further explained below in connection with FIGS. 3D-3E. As a result, the image sensing area portions covered by the at least two optical plate portions each become associated with a different focal plane. That is, in the illustrated example, there are two focal planes: a first focal plane 351-1 associated with the first optical plate portion 334-1 and the portion of the light-sensing area 333 covered by the first optical plate portion 334-1; and A second focal plane 351-2 associated with second optical plate portion 334-2 and the portion of light-sensing area 333 covered by second optical plate portion 334-2.

[0049] First focal plane 351-1 would, of course, be the same without separate optical plate layer 338. If only separate optical plate layer 338 were different between imaging systems 205 and 305, first focal plane 351-1 would therefore be the same as focal plane 251. Second focal plane 351-2 would be further away than first focal plane 351-1, i.e., further away from camera 330, as also illustrated in the figure.

[0050] When the imaging system 305 is set up for optical triangulation and configured based on the Scheimpflug principle so that the first focal plane 351-1 is co-located with, and therefore aligned with, the light plane corresponding to the first light 311 used for optical triangulation and provided by the light source 311, the second focal plane 351-2 will be at least substantially parallel to the first focal plane 351-1.

[0051] In principle, any separation distance can be achieved based on the embodiments herein, making them suitable for a wide variety of applications and fields of view.

[0052] As used above and herein, an optical plate refers to an optical window or an optical filter. An optical filter has a filtering effect on certain light, for example, light of certain frequencies or wavelengths, that would otherwise pass through, i.e., if such frequencies or wavelengths are present in the light, they are removed when passing through the optical plate. An optical window refers to the case where there is no such filtering effect.

[0053] It will be appreciated that the embodiments described above in connection with FIGS. 3A-3C can be implemented quite easily. For an existing prior art imaging system, such as imaging system 205, using a prior art image sensor, it may be determined or selected where an additional, second focal plane is desired to be located a certain distance away from where the existing focal plane is located. A separate optical plate layer, such as separate optical plate layer 338, may then be added at a thickness that achieves the second focal plane. It should also be determined which portion or portions of image sensing area 333 will be used with the additional focal plane, and this portion is then covered by the separate optical plate layer. Finding the thickness that achieves the second focal plane at the desired distance can be achieved through routine testing and experimentation and / or based on knowledge of the camera, including its lens, and how and where it is positioned relative to the image sensor, which allows the thickness to be calculated.

[0054] The basic principles behind the embodiments herein and any additional focal plane offsets that the second focal plane may have are now described.

[0055] FIG. 3D schematically illustrates a simplified example of how two focal planes, first focal plane 351-1′ and second focal plane 351-2′, may relate to image sensor 331 and lens 340 of camera 330 in FIGS. 3A-3B. Note that what is shown in the figure is without application of the Scheimpflug principle, and therefore the illustrated example does not directly correspond to the situation shown in FIG. 3C. Focal planes 351-1′ and 351-2′ are therefore not the same as focal planes 351-1 and 351-2. The figure serves the purpose of illustrating the underlying principle of offset additional focal planes, which at the same time are easier to explain and understand without the Scheimpflug principle.

[0056] This figure illustrates that light from a point in first focal plane 351-1' will be focused to a point on image sensing area 333 that is not covered by separate optical plate layer 338, i.e., a point on image sensing area 333 that is reached by light refracted through first optical plate portion 334-1. This figure also illustrates that light from a point on second focal plane 351-2' will be focused to a point on image sensing area 333 that is covered by separate optical plate layer 338, i.e., a point on image sensing area 333 that is reached by light refracted through second optical plate portion 334-2.

[0057] Figure 3E is an example that provides assistance in understanding how and why this is the case, i.e., how and why a separate second focal plane is formed. This can be understood by considering how a single light beam is refracted with and without separate optical plate layer 338. In other words, Figure 3E schematically illustrates how one and the same light ray, in this example, a single incident light ray 353, is affected by thick versus thin optical plate portions. Shown in the figure are a first transparent layer 336', which may correspond to transparent protective layer 336, and a second transparent layer 338', which may correspond to separate optical plate layer 338.

[0058] As can be seen in this diagram, some basic assumptions have been made to simplify the example. For example, first transparent layer 336' and second transparent layer 338' are assumed to be made of a material that is optically denser than the medium through which light ray 353 travels before reaching the layer, as in the case of a layer of glass in air, which is typically the case for an image sensor in a camera. It is also assumed that the layers are made of the same material, or at least of a material with the same refractive index, such as glass. For example, if first transparent layer 336' is made of a material with refractive index n1 and second transparent layer 338' is made of a material with refractive index n2, then n2 = n1. When both layers are present and correspond to separate optical plate layers 338, for example, attached to transparent protective layer 336, they may be attached to each other using an optical adhesive, or more generally, a transparent attachment medium 341, whose refractive index is the same as the layers', to avoid undesired reflections and refractions at the contact interface, as described above.

[0059] Consider first how a single incident light ray 353 would be affected if second transparent layer 338' were not present, i.e., corresponding to the situation without a separate optical plate layer 338. Light beam 353 would then be refracted when it reaches first transparent layer 336' and, after passing through the thickness of that layer, would refract out on the other side of the layer. Light beam 353 would travel along first ray path 353a in this situation as shown in the figure.

[0060] Now consider the same single incident light ray 353 but with the presence of a second transparent layer 338', i.e., equivalent to a situation with a separate optical plate layer 338. Light beam 353 will then be refracted when it reaches second transparent layer 338' and will pass through the thickness of both second transparent layer 338' and first transparent layer 336' before refracting outward on the other side. Light beam 353 will in this situation follow second ray path 353b as shown in the figure.

[0061] It can be clearly seen that the result is an offset Δ between ray paths 353a-b. The offset Δ describes how the separation between focal planes can be achieved as described above, or in general, how different focal planes can be achieved, such as those resulting from an optical plate covering an image sensor having different optical plate portions with different refractive indices, e.g., different thicknesses and / or different optical densities, such as from materials with different refractive indices.

[0062] It should be noted that even though the offset Δ accounts for the additional focal plane and the separation between the focal planes, the separation between the focal planes is also affected by the lens used, such as lens 340′, its properties, and how and where it is located relative to the image sensor, e.g., image sensor 331.

[0063] 4A illustrates schematically an imaging system setup with a camera 430 according to some embodiments herein for imaging a flat dot-surface object 420. The camera 430 has an image sensor (not shown) equivalent to the image sensor 331, such that there are two different focal planes 451-1, 451-2 associated with two different image sensing area portions of the image sensor of the camera 430.

[0064] The object is positioned relative to the camera 430 so that the camera 430 can image the dot surface of the object 420 while it extends away from the camera 430 within its field of view, i.e., the camera 430 and the object 420 are positioned so that the cameras can image the dot surface at different depths.

[0065] FIG. 4B schematically illustrates a front view of the object 420 shown in FIG. 4A with the dot surface visible.

[0066] FIG. 4C shows a real image 460 captured by a corresponding real imaging system, camera, and object as illustrated in FIGS. 4A-4B, and thus resulting from a real implementation of an embodiment herein. This figure illustrates the effect of two focal planes in the image. Dotted lines are drawn in the figure to indicate the areas that are in focus. Each dotted line is therefore associated with a respective one of focal planes 451-1, 451-2. It can be clearly seen that there are two areas in the image that are in focus, each centered around its respective dotted line. This figure also shows the location of a distorted or blurred region 461 in image 460. The blurred region 461 is explained by an edge effect where two different image sensing area portions meet, located between focal planes 451-1 and 451-2, for example, around the edge of separate optical plate layer 338 above transparent protective layer 336.

[0067] FIG. 5A schematically illustrates a cross-section of a second exemplary image sensor 531 according to some embodiments herein with corresponding cross-section CC as compared to the prior art image sensor 231 in FIGS. 2B-2C and the image sensor 331 in FIGS. 3A-3B to facilitate comparison and identification of differences.

[0068] FIG. 5B schematically illustrates a top view of the image sensor 531 in FIG. 5A with cross-section CC marked.

[0069] In the following, we will focus on the differences, i.e., what is different, compared to image sensor 331. Therefore, corresponding parts that look the same will have corresponding reference numerals and will not be referred to as different parts, and may be the same in Figures 5A-5B as in Figures 3A-3B.

[0070] 5A-5B illustrate an embodiment in which a separate optical plate layer 538 is stacked on a transparent protective layer 536, but without the use of a transparent adhesive, such as an optical adhesive, as in the previous example. Instead, the layers are arranged with a small space 544, i.e., a gap, separating them, thereby avoiding stacking the layers in direct physical contact. The reason for this, as already indicated above, is to avoid undesirable interference phenomena at the layer contact interface. While the space 544 may be as small as practically achievable, the space 544 should typically be larger than the wavelength of the light to be sensed, and preferably substantially larger, i.e., several times larger, i.e., typically greater than 400 nm to 1000 nm. However, in any case, it is typically difficult to achieve a space between layers smaller than a tenth of a micrometer, such as less than 50 μm. In practice, a suitable space may be in the range of 50 to 100 μm. Since there is no clear benefit to a large space, it may be beneficial to keep it small. It is recognized that a larger gap will also involve a larger area between the focal planes, which may lead to distorted or blurred portions of the image, such as the blurred area 461 described with respect to Figure 4C.

[0071] In addition to the separate optical plate layer 538 and the transparent protective layer 536, the image sensor 531 includes an image sensing area 533 of the image sensing portion 535 and a support structure 532. In the illustrated example, there is a retainer 543 for the separate optical plate layer 538 so that the layer can be stacked while the space separates it from the transparent protective layer 536. The separate optical plate layer 538 is held and / or attached to the retainer 543, for example mechanically, such as by being clamped and / or glued. It should be noted that the one shown in this figure is only one example, and that there are of course several other types of retainers available and other ways in which the separate optical plate layers can be held and / or attached by the retainer or the like, such as a fixture or frame structure. Such other types of retainers may be attached, such as glued, to the image sensor outside and / or around the image sensing area, or at least outside the portion of it used for imaging. It will be appreciated that separate optical plate layer 538 can be attached directly to the surface of support structure 532, provided that image sensing area 533 and possibly also transparent protective layer 536 are not flush with the surface of support structure 532, or the like, but are recessed therein with a small offset corresponding to space 544, as is shown for simplicity in the examples herein. Another possibility is to include a separate optical plate layer in the camera lens unit, as will be described below.

[0072] Except for how separate optical plate layer 538 is positioned with respect to transparent protective layer 536, other features may be as in FIGS. 3A-3B and image sensor 331. Effects such as additional focal planes are also the same. Thus, in FIGS. 5A-5B and image sensor 531, separate optical plate layer 538 and protective layer 536 together form optical plate 537, with first optical plate portion 534-1 and second optical plate portion 534-2 covering image sensing area 533 of image sensing unit 535 of image sensor 531. First optical plate portion 534-1 corresponds to the portion of image sensing area 533 that is not covered by separate optical plate layer 538 and is therefore covered only by transparent protective layer 536. Light incident on the first portion is refracted by transparent protective layer 536 and travels through a thickness designated t1 539-1 thereof to reach image sensing area 533 of image sensing unit 535. In the illustrated example, second optical plate portion 534-2 corresponds to the portion of image sensing area 533 that is covered by separate optical plate layer 538 and therefore covered by both this layer and transparent protective layer 536. Light incident on the second portion is refracted by and travels through a layer whose thickness, designated t2 539-2, corresponds to the thickness of both separate optical plate layer 538 and transparent protective layer 536, including space 544, before reaching image sensing area 533 of image sensing unit 535.

[0073] 3A-3C, the embodiment relating to Figures 5A-5B can be easily implemented based on prior art image sensors. Furthermore, gaps such as space 544 allow for more flexible and variable types of implementation, for example, as described next.

[0074] 5C schematically illustrates a camera system including a camera housing portion 545' and a camera lens portion 546' that, when attached or mounted together, form a camera 530' having an image sensor 531' corresponding to image sensor 531 described above, but with a different holder. Camera lens portion 546' includes lens 540' and a separate optical plate layer 538' corresponding to separate optical plate layer 538. Camera housing portion 545' includes image sensor 531' excluding its separate optical plate layer 538', and thus includes, for example, image sensing portion 535' with image sensing area 533', its transparent protective layer 536', and support structure 532'. Thus, the portion of image sensor 531' included in camera housing portion 545' may correspond to a conventional image sensor.

[0075] Camera housing portion 545' and camera lens portion 546' are formed to fit together and to be attached together, preferably detachably and / or releasably, similar to, for example, a lens to a system camera housing. Once attached together, separate optical plate layer 538' is positioned in a desired relationship to transparent protective layer 536', i.e., such that there is a predetermined space 544' between the layers and such that separate optical plate layer 538' covers a desired predetermined area or portion of image sensing area 533', thereby corresponding to second optical plate portion 534-2. The portion of image sensing area 533' not covered by separate optical plate layer 538' therefore corresponds to first optical plate portion 534-1.

[0076] It should be appreciated that embodiments based on the camera system and camera 530′ of FIG. 5C enable one and the same camera housing, such as camera housing portion 545′, with a conventional image sensor, to be used with several different lens portions. The different lens portions may differ by their separate optical plate layers, for example by their refractive properties, for example by different thicknesses of the layers and / or refractive indices of the layer materials, and / or by which portions of the image sensing area 533′ the layers cover. Each such different lens portion, when mounted in the housing, may therefore lead to a predetermined additional second focal plane associated with the portion of the image sensing area 533′ that the separate optical plate layer covers. Another possibility is a lens portion, such as lens portion 546′, with a separate optical plate layer, such as separate optical plate layer 538′, that is adjustable and / or replaceable. For example, so that it can be adjusted to cover different portions and / or parts of the image sensing area 533′ and / or be replaced with another having different refractive properties, such as thickness. The embodiment of Figure 5C therefore facilitates flexible implementation of embodiments herein and providing a camera system that can be more easily adapted to different use cases and situations for which embodiments herein are implemented, without having to modify or replace the entire camera. It should be further appreciated that a flexible system such as that described in connection with Figure 5C also readily enables switching to the functionality of a conventional camera having only a single focal plane, for example, by switching to a lens section without a separate optical plate layer or simply removing the separate optical plate layer from the lens section.

[0077] For ease of explanation, the above-described embodiments comprise one additional separate optical plate layer, however, it should be appreciated that by applying the same principles on which the above embodiments are based, it is of course possible with other and / or further optical plate layers to form optical plates, which may, for example, be of different thicknesses and / or comprise three or more optical plate portions so that three or more different focal planes are achieved for one and the same camera and image sensor.

[0078] Moreover, although this may be preferred for easier implementation on existing image sensors, the underlying principles of the embodiments herein do not of course require that there must be a conventional transparent protective layer covering the image sensing area.

[0079] As should be appreciated, it is possible to form two or more optical plate portions that achieve two or more focal planes by using various combinations of optical plate layers and variations in the materials and thicknesses of such layers. Some further examples in this regard will be described below.

[0080] 6A-6D schematically illustrate different examples of alternatives to the above embodiments of how different refractive optical plate portions can be achieved for image sensors according to embodiments herein.

[0081] 6A shows an example in which first and second optical plate portions 634-1 a, 634-2 a are formed by two separate transparent layers 636 a, 638 a of different thicknesses that cover separate portions of an image sensing area 633 a of an image sensing portion 635 a. The separate layers 636 a, 638 a are made of the same material, but alternatively may be made of different materials, for example materials with different refractive indices.

[0082] 6B shows an example in which the first and second optical plate portions 634-1b, 634-2b are formed by a transparent single layer 638b that covers the image sensing area 633b of the image sensing portion 635b. Different thicknesses of the single layer 638b form the two optical plate portions 634-1b, 634-2b. The single layer is made from one and the same material.

[0083] FIG. 6C shows an example in which the first and second optical plate portions 634-1c, 634-2c are formed by two separate transparent layers 636c, 638c made from materials of the same thickness but different refractive index and covering separate portions of the image sensing area 633c of the image sensing portion 635c.

[0084] 6D shows an example in which first and second optical plate portions 634-1 d, 634-2 d are formed by a transparent single layer 638 d of equal thickness that covers image sensing area 633 d of image sensing portion 635 d. Second optical plate portion 634-2 d is now formed by providing a portion 638 d ′ of transparent single layer 638 d with a different refractive index, for example, by single layer 638 d being formed of a different material or by doping a material in that portion, or by any other known means for achieving a different refractive index in only a portion of a transparent single layer.

[0085] It should be noted that the examples herein of how optical plate portions with different refractive indices can be achieved for image sensors according to embodiments herein are merely examples, and several other methods and combinations are possible. Although not shown in any detailed examples and drawings, it should be noted that embodiments herein also comprise image sensors with not only one but two or more additional optical plate portions with different refractive indices covering different portions or parts of the image sensing area, thereby forming two or more additional focal planes. Those skilled in the art who are able to implement an image sensor with two optical plate portions based on what is disclosed herein can easily implement an image sensor with additional optical plate portions to form additional focal planes.

[0086] Embodiments relating to image sensors such as those described above will now be described more generally. These image sensor embodiments are for use with a lens, such as one of lenses 340, 540′, such as one of image sensors 331, 531, 531′. The lens is arranged to focus light onto an image sensing area, such as one of image sensing areas 333, 533 of the image sensor. The image sensor includes an optical plate, such as one of optical plates 337, 537, arranged to cover the image sensing area with at least two optical plate portions, such as optical plate portions 334-1 and 334-2 or 534-1 and 534-2. The optical plate may also be referred to as and / or correspond to an optical window or optical filter. The optical plate portions have different refractive properties such that light incident from the lens and refracted by the optical plate portions toward the image sensing area travels different distances to be focused on the image sensing area depending on which of the optical plate portions the light is refracted by. As a result, the image sensing area portions covered by the at least two optical plate portions become associated with different focal planes, for example focal planes 351-1 and 351-2 or focal planes 751-1 and 751-2.

[0087] Embodiments relating to a camera such as those described above, e.g., any one of cameras 330, 530′, 730, will now be described more generally. The camera includes an image sensor according to an embodiment, e.g., any one of image sensors 331, 531, 531′ as described above. The camera also includes a lens such as one of lenses 340, 540′. In some camera embodiments, a separate optical plate layer, such as separate optical plate layer 538′, is stacked on a transparent protective layer, such as transparent protective layer 536′, with a gap or space, e.g., space 544′, separating the layers to prevent the layers from coming into direct physical contact with one another. The separate optical plate layer, in these embodiments, may be part of a separate lens portion, such as lens portion 545′, of the camera, that includes a lens and is removably attached to and / or configured to be removably attached to a housing portion, such as housing portion 546′, of the camera, which housing portion includes an image sensing area, such as image sensing area 533′.

[0088] It should be appreciated that image sensors and cameras according to embodiments herein, such as those described above with respect to Figures 3, 5, and 6, may be used with imaging systems for optical triangulation, with or without application of the Scheimpflug principle, such as imaging system 305. Image sensors according to embodiments may also be used with other imaging systems and / or cameras where additional or more focal planes are desirable.

[0089] FIGS. 7A-7B schematically illustrate simplified second and third examples of optical triangulation-based imaging systems 705a-b. Each system includes a camera with an image sensor according to embodiments herein, and the examples correspond to two different use cases. Below, attention will be directed to differences compared to imaging system 305 in FIG. 3C , i.e., what is new and different in each one of FIGS. 7A-7B . Therefore, corresponding parts also shown in FIG. 3C and / or described with respect to that figure may be the same in FIGS. 7A-7B unless otherwise specified or excluded for some reason. In principle, imaging systems 705a-b may be considered two different enhanced versions of imaging system 305 for various applications.

[0090] 7A schematically illustrates the imaging system 705a as a second simplified example of an imaging system according to some embodiments herein, comprising a camera 730a with an image sensor 731a, which may be or correspond to any of the image sensors according to embodiments herein, such as those described above, to achieve two focal planes in the camera 730a, a first focal plane 751-1a and a second focal plane 751-2a.

[0091] Similar to the imaging system 305 of FIG. 3C, there is a first light source 710-1a for illuminating the object 720a with a first light 711-1a, typically a laser light, as part of optical triangulation for 3D imaging of the object 720a. The first light 711-1a is provided in the form of a light plane. A camera 730a and an image sensor 731a are positioned to sense the first light reflected from the object 720a as part of the optical triangulation for 3D imaging. The imaging system 705a is set up and configured based on the Scheimpflug principle such that the first focal plane 751-1a is co-located with, and therefore aligned with, the light plane corresponding to the first light 711-1a.

[0092] The primary difference between imaging system 705a and imaging system 305 is that imaging system 705a also includes an additional, second light source 710-2a for illuminating object 720a with second light 711-2a provided in the form of a light plane aligned with second focal plane 751-2a. Reflections from the object from both light planes can therefore be focused onto image sensor 731a. In some embodiments, second light 711-2a is the same type of light as first light 711-1a, but in other embodiments it may differ in some respects.

[0093] A second light source may illuminate the object 720a with a second light 711-2a as part of a further optical triangulation for 3D imaging of the object 720a. That is, two different optical triangulations for 3D imaging may be performed in parallel, one per optical plane, using the same camera and image sensor. This may result in faster overall 3D imaging of the object. For example, the object 720a may be scanned more quickly with a single camera and image sensor, and it may even be possible to scan two separate objects simultaneously. If the same portion of the object 720a is scanned twice, separated in time, by two optical planes provided by the first and second lights 711-1a and 711-2a, respectively, the scan data from the two occasions may be used to improve measurements. For example, the embodiments herein may improve measurements by making it possible to detect undesired movement of the object 720a, such as lateral movement in the scan direction, i.e., lateral movement that may occur between scans by the two optical planes.

[0094] An advantageous use for this purpose may be to detect undesired movement of an object being scanned, such as object movement within a light plane between images of the same object. The same surface location may be scanned by two light planes separated in time, and the images may be compared. If they are different, or too different according to some predetermined criteria, this indicates that there has been undesired movement of the object between the images. The movement can then be corrected, or in any case, it is useful to know that such movement has occurred. This type of undesired movement may be a particular problem and / or difficult to avoid in some applications. For example, when large, heavy, and / or irregular objects, such as logs and the like, are to be scanned. If a log moves laterally between images, such as tilting or rotating, it may not be possible with just a single light plane to know whether such movement has occurred, and the log may thereby appear bent according to 3D imaging even though it is not. Traditionally, when a 3D image is formed with a single optical and focal plane, the log may simply move laterally during the scan in which the optical triangulation image was acquired, and therefore may be erroneously deemed to be curved when it is not.

[0095] It should be noted that in the case of additional light sources, such as a second light source 710-2a, that provide light for further light triangulation, such imaging systems, e.g., imaging system 705a, may be calibrated for each light plane, e.g., separately for each light plane, and one light plane at a time may be used during calibration.

[0096] 7B schematically illustrates the imaging system 705b as a third simplified example of an imaging system according to some embodiments herein, which includes a camera 730b with an image sensor 731b, which may be or correspond to any of the image sensors according to embodiments herein, such as those described above, to achieve two focal planes in the camera 730a, a first focal plane 751-1b and a second focal plane 751-2b.

[0097] Similar to the imaging system 305 of FIG. 3C, there is a first light source 710-1b for illuminating the object 720b with a first light 711-1b, typically a laser light, as part of optical triangulation for 3D imaging of the object 720b. The first light 711-1b is provided in the form of a light plane. A camera 730b and an image sensor 731b are positioned to sense the first light reflected from the object 720b as part of the optical triangulation for 3D imaging. The imaging system 705b is set up and configured based on the Scheimpflug principle such that the first focal plane 751-1b is co-located with, and therefore aligned with, the light plane corresponding to the first light 711-1b.

[0098] The main difference between imaging system 705b and imaging system 305 is that imaging system 705b also includes a second light source 710-2b for illuminating object 720b with second light 711-2b. The second light source 710-2b and second light 711-2b are of a different type from the first light source 710-1b and first light 711-1b in these embodiments and are not related to optical triangulation per se. The second light 711-2b may be diffuse light, as shown in the figure. It may be provided by one or more light-emitting diodes (LEDs), which the second light source 710-2b may be based on. Diffused light allows for, for example, laser speckle-free reflectivity. Diffused light is closer to commonly occurring illumination and may therefore be better for acquiring 2D image data about object 720b than, for example, laser light or similar used as the first light 711-1b and for optical triangulation. The second light source 710-2b is positioned to direct the second light 711-2b so that it illuminates at least the second focal plane. Of course, the second light should not be directed and / or provided in such a way that it has a detrimental interference effect on the first light in the first focal plane and thereby a negative effect on optical triangulation. This is usually not an issue because the light, such as the first light 711-1b, used for optical triangulation is typically of a magnitude stronger than that needed to be used for the widely diverging second light due to additional diffusion. Some light shielding can, of course, be applied as needed. For embodiments herein, while acquiring images for optical triangulation, it is possible to acquire focal 2D image data about the object 720b, for example, to extract information about its surface in addition to the 3D data about the object 720b obtained through optical triangulation. Knowledge of the separation distance, i.e., the distance between the focal planes 751-1b and 751-2b, makes it possible to correlate the acquired 2D data with corresponding 3D data, i.e., for the same location on the object 720b.The second focal plane 751-2b may, in some of these embodiments, be located close enough to the first focal plane 751-1b that the acquired 2D data can be considered related to the 3D data acquired simultaneously, i.e., in the same image.

[0099] Embodiments relating to imaging systems such as those described above with respect to Figures 7A-7B will now be described more generally. The imaging system is for 3D imaging of an object, e.g., any of objects 320, 720, based on optical triangulation, such as any one of imaging systems 305, 705. The imaging system comprises a camera according to embodiments herein, e.g., any one of cameras 330, 530', 730 as generally described above. The camera comprises an image sensor according to embodiments herein, e.g., any one of image sensors 331, 531, 531' as generally described above.

[0100] The imaging system may further include a first light source, such as one of first light sources 310, 710-1, for providing a first light, e.g., one of first light sources 311, 711-1. The first light is in the form of a first light plane for illuminating the object as part of the optical triangulation. A camera with an image sensor may be disposed in the imaging system for acquiring reflected first light from the object as part of the optical triangulation. The imaging system may be configured such that at least a first focal plane, such as one of at least two focal planes 351-1, 751-1, such as provided by an image sensor according to embodiments herein, is co-located with the first light plane.

[0101] The imaging system may further include an additional second light source, such as second light source 710-2, for providing second light, e.g., second light 711-2, for illuminating the object or another object. A camera with an image sensor may be disposed in the imaging system to acquire reflected second light from the object or another object. The imaging system may further be configured such that another, second focal plane, such as one of second focal planes 351-2, 751-2 of the at least two focal planes, is located where the second light will be incident on the object or another object, reflected, and then acquired by the camera and image sensor. In some embodiments, the second light is in the form of a second light plane, and the second focal plane is co-located with the second light plane.

[0102] It should be noted that any enumerated terminology used herein, such as first device, second device, first surface, second surface, etc., should itself be considered open-ended, and the terminology itself does not imply any particular hierarchical relationship. Absent any explicit information to the contrary, designation by enumeration should simply be considered a means of providing different names.

[0103] As used herein, the expression "configured to" may mean that a processing circuit is configured or adapted, using software or hardware configurations, to perform one or more of the operations described herein.

[0104] As used herein, the term "number" or "value" may refer to any kind of number, such as a binary number, a real number, an imaginary number, or a rational number. Moreover, a "number" or "value" may be one or more symbols, such as a character or a string of characters. A "number" or "value" may also be represented by a string of bits.

[0105] As used herein, the phrases "may" and "in some embodiments" are typically used to indicate that the described features can be combined with any of the other embodiments disclosed herein.

[0106] In the drawings, features that may be present in only some embodiments are typically depicted using dotted or dashed lines.

[0107] When the word "comprises" or "comprising" is used, it shall be taken to mean open-ended, i.e. "consisting at least of."

[0108] The embodiments herein are not limited to the above embodiments. Various alternatives, modifications, and equivalents may be used. Therefore, the above embodiments should not be taken as limiting the scope of the present disclosure, which is defined by the appended claims. [Explanation of symbols]

[0109] 305 Imaging System 310 First Light Source 311 First Light 320 Object 330 Camera 331 Image Sensor 332 Support structure 333 Image Sensing Area 334-1 First optical plate part 334-2 Second optical plate part 335 Image sensing unit 336 Transparent protective layer 336' First transparent layer 337 Optical plate 338 Separate Optical Plate Layers 338' Second transparent layer 339-1 Thickness t1 339-2 Thickness t2 340 Lens 340' lens 341 Transparent adhesive medium 351-1 First focal plane 351-1' First focal plane 351-2 Second focal plane 351-2' Second Focal Plane 353 Incident ray 353a First ray path 353b Second ray path 420 Object 430 Camera 451-1 Focal plane 451-2 Focal plane 460 images 461 Blurred Area 531 Image Sensor 532 Support structure 533 Image Sensing Area 534-1 First optical plate part 534-2 Second optical plate part 535 Image sensing unit 536 Transparent protective layer 537 Optical plate 538 Separate Optical Plate Layers 539-1 Thickness t1 539-2 Thickness t2 543 Holder 544 Space 530' camera 531' image sensor 532' Support structure 533' Image sensing area 535' Image sensor 536' Transparent protective layer 538' Separate Optical Plate Layer 540' lens 544' space 545' Camera housing 546' Camera lens part 633a Image sensing area 633b Image sensing area 633c Image sensing area 633d image sensing area 634-1a First optical plate portion 634-1b First optical plate portion 634-1c First optical plate portion 634-1d First optical plate part 634-2a Second optical plate part 634-2b Second optical plate part 634-2c Second optical plate part 634-2d Second optical plate part 635a Image sensing unit 635b Image sensing unit 635c Image sensing unit 635d Image sensing unit 636a transparent layer 636c transparent layer 638a transparent layer 638b Transparent single layer 638c transparent layer 638d Transparent Single Layer 638d' part 705a Imaging System 705b Imaging System 710-1a First Light Source 710-1b First Light Source 710-2a Second Light Source 710-2b Secondary Light Source 711-1a First Light 711-1b First Light 711-2a Second Light 711-2b Second Light 720a Object 720b Object 730a camera 730b camera 731a Image Sensor 731b image sensor 751-1a First Focal Plane 751-1b First Focal Plane 751-2a Second Focal Plane 751-2b Second Focal Plane

Claims

1. An imaging system (305, 705) for three-dimensional imaging of an object (320, 720) based on optical triangulation, comprising a camera (330, 530', 730), The camera (330, 530', 730) 1. An image sensor (331, 531, 531') for use with a lens (340, 540') arranged to focus light onto an image sensing area (333, 533) of said image sensor (331, 531), wherein said image sensor (331, 531) is configured such that light incident from said lens (340, 540') and refracted by at least two optical plate portions (334-1, 334-2, 534-1, 534-2) towards said image sensing area (333, 533) is focused by at least one of said optical plate portions (334-1, 334-2, 534-1, 534-2). an image sensor comprising an optical plate (337, 537) which is an optical window or an optical filter arranged to cover the image sensing area (333, 533) with the optical plate portions (334-1, 334-2, 534-1, 534-2) having different refractive indices so that light travels different distances and is focused on the image sensing area depending on whether it is refracted by one of the optical plate portions (334-1, 334-2, 534-1, 534-2), and the image sensing area portions covered by the at least two optical plate portions (334-1, 334-2, 534-1, 534-2) are associated with at least two different focal planes (351-1, 351-2, 751-1, 751-2); the lens (340, 540'); Equipped with the imaging system (305, 705) further comprises a first light source (310, 710-1) for providing first light (311, 711-1) in the form of a first plane of light to illuminate the object (320, 720) as part of the optical triangulation, the camera (330, 530', 730) having the image sensor (331, 531, 531') is disposed in the imaging system for acquiring the first light reflected from the object (320, 720) as part of the optical triangulation, and the imaging system is configured such that at least one first focal plane (351-1, 751-1) of the at least two focal planes (351-1, 351-2, 751-1, 751-2) is co-located with the first plane of light; The imaging system (305, 705) further comprises an additional second light source (710-2) for providing a second light (711-2) to illuminate the object (320, 720) or another object, and the camera (330, 530', 730) having the image sensor (331, 531, 531') is disposed in the imaging system for acquiring the second light reflected from the object (320, 720) or the another object, and the image an imaging system (305, 705) configured such that a second focal plane (351-2, 751-2) of the at least two focal planes (351-1, 351-2, 751-1, 751-2) is located at a location where the second light is incident on the object (320, 720) or the other object, reflected, and then captured by the camera (330, 530', 730) and the image sensor (331, 531, 531').

2. 2. The imaging system (305, 705) of claim 1, wherein said different refractive properties are caused by different thicknesses (339-1, 339-2, 539-1, 539-2) and / or by different refractive indices of materials from which said optical plate portions (334-1, 334-2, 534-1, 534-2) are made.

3. 3. An imaging system (305, 705) as described in claim 1 or 2, wherein the image sensing area (333, 533) is covered by a transparent protective layer (336, 536), and at least one of the optical plate portions (334-1, 334-2, 534-1, 534-2) is formed by a separate optical plate layer (338, 538) stacked on the transparent protective layer (336, 536).

4. 4. The imaging system (305, 705) of claim 3, wherein the transparent protective layer (336, 536) and the separate optical plate layer (338, 538) are made from materials with the same refractive index.

5. 4. The imaging system (305, 705) of claim 3, wherein the separate optical plate layer (338, 538) is attached to the transparent protective layer (336, 536) by an optical adhesive having a refractive index corresponding to the refractive index of the separate optical plate layer (338, 538) or the transparent protective layer (336, 536).

6. 4. The imaging system (305, 705) of claim 3, wherein the separate optical plate layers (338, 538) are stacked in the transparent protective layer (336, 536) with spaces (544) separating each layer, thereby avoiding direct physical contact between each layer.

7. 4. The imaging system (305, 705) of claim 3, wherein the separate optical plate layers (538') are stacked on the transparent protective layer (536') with spaces (544') separating the layers to avoid direct physical contact with each other, and the separate optical plate layers (538') are part of a separate lens portion (546') of the camera (530') mounted in a housing portion (545') of the camera (530') comprising the image sensing area (533').

8. 2. The imaging system of claim 1, wherein the second light (711-2) is in the form of a second plane of light, and the second focal plane (351-2, 751-2) is co-located with the second plane of light.

Images