Rolling Spectral Filter

Abstract

A rolling spectral filter is disclosed. A device including a pixel array and a filter wheel can capture multiple images by exposing the pixel array. The device can rotate the filter wheel while capturing multiple images, and the filter wheel can include filter sections. In some implementations, the portion of the filter wheel in front of the pixel array includes two or more filter sections.

Description

[0001] background

[0002] An optical filter is a device that filters light incident on an optical filter based on wavelength. For example, a bandpass filter can transmit light within a specific wavelength range while rejecting (e.g., absorbing or reflecting) light falling outside that specific wavelength range (e.g., above or below a specific wavelength range). Bandpass filters are used in a variety of applications such as fluorescence microscopy, spectroscopic examination, clinical chemistry, imaging, and many others.

[0003] Overview

[0004] According to some implementations, a method may include capturing multiple images by a device including a pixel array and a filter wheel by exposing the pixel array, and rotating the filter wheel by the device while capturing the multiple images, wherein the filter wheel includes filter segments, and wherein a portion of the filter wheel in front of the pixel array includes two or more filter segments.

[0005] According to some implementations, a method may include capturing multiple images by a device comprising an array of pixel rows and a filter wheel by sequentially exposing each row of pixels of the array at a shutter speed and in a rolling shutter mode, and rotating the filter wheel by the device at a wheel speed while capturing the multiple images, wherein the filter wheel comprises filter sections, and wherein a portion of the filter wheel in front of the array comprises two or more filter sections.

[0006] According to some implementations, the device may include: an array of pixel rows; a filter wheel including filter sections, wherein a portion of the filter wheel located at the front of the array includes two or more filter sections; and one or more processors. The one or more processors may be configured to capture multiple images by exposing the array and to rotate the filter wheel while capturing multiple images.

[0007] 1. A method comprising:

[0008] Multiple images are captured by a device including a pixel array and filter wheels by exposing the pixel array; and

[0009] The filter wheel is rotated by the device while capturing the plurality of images.

[0010] The filter wheel includes filter sections, and

[0011] The filter wheel in the part in front of the pixel array includes two or more filter sections.

[0012] 2. The method according to Clause 1, wherein the pixel array comprises pixel rows, and

[0013] Capturing the plurality of images by exposing the pixel array includes sequentially exposing each row of pixels in the pixel array in a rolling shutter mode at a shutter speed.

[0014] 3. The method according to Clause 1, wherein capturing the plurality of images by exposing the pixel array includes simultaneously exposing pixels of the pixel array in a global shutter mode.

[0015] 4. The method according to Clause 1, wherein the filter wheel comprises a repeating sequence of filter sections, and

[0016] Each filter section in the repeating sequence of filter sections is a filter section of a different type from the other filter sections in the repeating sequence of filter sections.

[0017] 5. The method according to Clause 1, wherein:

[0018] The pixel array is the first pixel array.

[0019] The plurality of images are a first plurality of images.

[0020] The portion of the filter wheel in front of the first pixel array is the first part.

[0021] The device includes a second pixel array, and

[0022] The filter wheel includes two or more filter sections in the second portion in front of the second pixel array.

[0023] 6. The method described in Clause 5 further includes:

[0024] A second plurality of images are captured by exposing the second pixel array while capturing the first plurality of images and while rotating the filter wheel.

[0025] 7. The method described in Clause 6 further includes:

[0026] At least one of a 3D image or a 3D video is generated based on the first plurality of images and the second plurality of images.

[0027] 8. A method comprising:

[0028] A device comprising an array of pixel rows and filter wheels captures multiple images by sequentially exposing each row of pixels in a rolling shutter mode at a shutter speed; and

[0029] The filter wheel is rotated at a wheel speed by the device while capturing the plurality of images.

[0030] The filter wheel includes filter sections, and

[0031] The filter wheel in the front of the array comprises two or more filter sections.

[0032] 9. The method according to Clause 8, wherein the wheel speed corresponds to the shutter speed, such that for an image in the plurality of images, a single filter section in the filter section is positioned in front of the pixels of each exposed row.

[0033] 10. The method described in Clause 9 further includes:

[0034] Block light from filter sections other than the single filter section mentioned above.

[0035] 11. The method described in Clause 9 further includes:

[0036] Before exposing a row of pixels, reset the pixels in that row.

[0037] 12. The method described in Clause 9 further includes:

[0038] Capture multiple images using a single type of filter section located before the pixels of each exposed row; and

[0039] A composite image is generated by combining multiple images by mathematically transforming the pixel values ​​of the multiple images.

[0040] 13. The method according to Clause 8, wherein rotating the filter wheel includes rotating the filter wheel at the wheel speed such that a plurality of filter sections in the filter sections are positioned in front of each exposed row.

[0041] 14. An apparatus comprising:

[0042] An array of pixel rows;

[0043] Filter wheel, the filter wheel including filter sections,

[0044] The portion of the filter wheel located in front of the array comprises two or more filter sections; and

[0045] One or more processors, wherein the one or more processors are configured to:

[0046] Multiple images are captured by exposing the array; and

[0047] The filter wheel is rotated while capturing the plurality of images.

[0048] 15. The device according to Clause 14, wherein the pixel row has a row width, and

[0049] The portion of the filter section located at the front of the array has a section width greater than or equal to the row width.

[0050] 16. The device according to Clause 14, wherein the filter wheel comprises a repeating sequence of filter sections, and

[0051] At least one of the following conditions is true:

[0052] Each filter section in the repeating sequence of filter sections is a filter section of a different type from the other filter sections in the repeating sequence of filter sections.

[0053] Each filter section in the repeating sequence of filter sections allows a different wavelength range to pass through compared to other filter sections in the repeating sequence of filter sections, or

[0054] The repeating sequence of filter sections includes at least six different types of filter sections.

[0055] 17. The device according to Clause 14, wherein the filter wheel comprises at least 100 filter sections.

[0056] 18. The device according to Clause 14 further includes a curtain shutter located between the array and the filter wheel.

[0057] The curtain shutter blocks a subset of light from the filter section.

[0058] 19. The device according to Clause 14, wherein the filter section of the filter section includes subsections located in a column along the length of the filter section.

[0059] 20. The device according to Clause 19, wherein each subsection of the filter section allows a different wavelength range to pass through compared to the other subsections of the filter section. Brief description of the attached diagram

[0061] Figure 1-4 This is a diagram of an example implementation of an optical device including the imager and filter wheel described herein.

[0062] Figure 5 This is a diagram of an example implementation of an optical device that includes the multiple imagers and filter wheels described herein.

[0063] Figure 6 yes Figure 1-5 A diagram of example components of one or more optical devices.

[0064] Figure 7-8This is a flowchart of an example process related to capturing an image using an optical device that includes a filter wheel.

[0065] Detailed description

[0066] The following example implementation is described in detail with reference to the accompanying drawings. The same reference numerals in different drawings may identify the same or similar elements.

[0067] Static spectral imaging uses multiple color regions or filters (e.g., bandpass filters) arranged in contact with or just above an imager comprising an array of pixels. The color regions or filters are arranged in a repeating pattern on the pixel array. For example, a red-green-blue (RGB) Bayer filter comprises a pattern of two green filters, one blue filter, and one red filter placed over four pixels. This pattern repeats throughout the array. However, because the pattern covers four pixels, the spatial resolution of the imager is reduced to one-quarter. For example, a 1-megabit sensor (1,000 pixels x 1,000 pixels) with RGB filters has a spatial resolution of 250 kilobits. Furthermore, increasing the number of filter elements (e.g., to obtain more spectral information for each image) further reduces the spatial resolution of the imager.

[0068] Another method for capturing spectral and spatial information involves placing discrete filters side-by-side and moving them linearly throughout the imaging field as a single image is captured. However, this method requires the imager to reset the components of the discrete filters to their starting positions after capturing one image and before capturing another. Such resetting of the discrete filter components causes mechanical vibration. Furthermore, the number of spectral filters that can be included in the components is physically limited. Additionally, this method requires long image acquisition times (e.g., tens of seconds or minutes).

[0069] Some implementations described herein provide methods and / or devices that capture multiple images by exposing a pixel array and, while capturing the multiple images, rotate a filter wheel located in front of the pixel array. The filter wheel (e.g., a rolling spectral imaging filter) includes filter sections (e.g., filter elements), and a portion of the filter wheel in front of the pixel array includes two or more filter sections. The filter sections can be different types of filters to increase spectral resolution and can be located within the filter wheel in a repeating sequence. In some implementations, the filter sections can roll throughout the entire pixel array as the filter wheel rotates.

[0070] In some implementations, the method and / or device can capture an image in a rolling shutter mode by sequentially exposing each row of pixels in a pixel array at a shutter speed. Alternatively or additionally, the method and / or device can rotate a filter wheel at a wheel speed corresponding to the shutter speed, such that for one of multiple images, a single filter section in the filter section precedes the pixels of each exposed row. In some implementations, the method and / or device can capture an image in a global shutter mode by simultaneously exposing the pixels of the pixel array.

[0071] In this way, the method and / or device can increase spectral resolution without reducing spatial resolution. By rotating the filter wheel, the method and / or device can be free from mechanical vibration. Furthermore, or optionally, the number of filters that can be included in the filter wheel can be up to 100 or even more (e.g., 125 filter sections, 250 filter sections, 250 filter sections (each filter section comprising three or more sub-sections) and / or the like). In some implementations, by rotating the filter wheel, the method and / or device can have a shorter image acquisition time than that achievable using a linearly rotating component with filters.

[0072] Figure 1 This is a diagram of an example implementation 100 of an optical device including the imager 102 and filter wheel 104 described herein. Figure 1 As shown, the filter wheel 104 can be located in front of the imager 102, so that the light incident on the imager 102 passes through the filter wheel 104.

[0073] In some implementations, imager 102 may be a camera sensor, a charge-coupled device (CCD) sensor, and / or the like. Additionally or alternatively, imager 102 may include a two-dimensional (2D) pixel array comprising multiple rows of pixels, a scanned linear pixel array, and / or the like. In some implementations and as per this document regarding... Figure 3 Further described, an optical device (e.g., using imager 102) can capture one or more images in a rolling shutter mode by sequentially exposing each row of pixels of the array at shutter speeds. Alternatively or additionally, an optical device (e.g., using imager 102) can capture one or more images in a global shutter mode by simultaneously exposing the pixels of the array (e.g., all pixels of the array).

[0074] In some implementations, imager 102 may include one or more types of sensors (e.g., for capturing images at different wavelengths and / or in different spectral regions). For example, imager 102 may include silicon-based sensors, indium gallium arsenide (InGaAs)-based sensors, lead salt-based sensors (e.g., lead selenide (PbSe) sensors), and / or the like.

[0075] In some implementations and such as in Figure 1 As shown, filter wheel 104 may include multiple filter sections 106 (e.g., at least 100 filter sections and / or such). For example, as Figure 1 As shown, filter wheel 104 may include 125 filter sections 106. In some implementations, filter sections 106 may be placed (e.g., using binary multispectral (BMS) filter technology and / or the like) on a disk to form filter wheel 104. Alternatively or additionally, the filter sections 106 of filter wheel 104 may form a hyperspectral filter array.

[0076] In some implementations, and as discussed in this article... Figure 3 Further described, filter sections 106 can be filters of different types and can be located in filter wheels 104 in a repeating sequence. For example, each filter section in the repeating sequence of filter sections 106 can be a filter section of a different type from the other filter sections 106 in the repeating sequence of filter sections 106. Additionally or optionally, each filter section in the repeating sequence of filter sections 106 can allow a wavelength range to pass through that is different from the wavelength range passed through by the other filter sections 106 in the repeating sequence of filter sections 106. In some implementations, the repeating sequence of filter sections 106 may include at least six different types of filter sections 106, filter sections 106 that allow at least six different wavelength ranges to pass through, and / or the like.

[0077] In some implementations, and as discussed in this article... Figure 4 As further described, one or more filter sections 106 may include subsections located in a column along the length of one or more filter sections. For example, a filter section may include subsections, and each subsection may be a filter of a different type from other subsections in the filter section, may allow wavelength ranges to pass through that are different from those passed through other subsections in the filter section, and / or the like.

[0078] like Figure 1 As shown, the portion of the filter wheel 104 in front of the imager 102 may include two or more filter sections 106. For example, this portion of the filter wheel 104 in front of the imager 102 may include at least five filter sections, such as... Figure 1 As shown. In some implementations, filter section 106 may have a section width smaller than the size (e.g., length and / or width) of the pixel array in imager 102. Additionally or alternatively, and as discussed herein... Figure 3Further described, the pixel rows in the array of imager 102 may have a row width, and the filter section 106 may have a section width greater than or equal to the row width. For example, the portion of filter section 106 located in front of the array of imager 102 may have a section width greater than or equal to the row width.

[0079] In some implementations, the optical device may include one or more lens elements. For example, the optical device may include lenses to optimize the angle of light for performance. Additionally or alternatively, the optical device may include lenses, and the focusing speed of the lenses may be used to change the full width at half maximum (FWHM) of the filter section 106.

[0080] As also by Figure 1 As indicated by the curved arrow, the optical device can rotate the filter wheel 104 about an axis (e.g., at a wheel speed). For example, the optical device may include a motor to rotate the filter wheel 104. In some implementations, the optical device can rotate the filter wheel 104 while capturing (e.g., using imager 102) one or more images. For example, the optical device can capture multiple images by sequentially exposing each row of pixels of the array at a shutter speed in a rolling shutter mode, and can rotate the filter wheel 104 at a wheel speed while capturing multiple images. In some implementations, rotating the filter wheel 104 may not produce mechanical vibration. In this way, the filter wheel 104 may be a rolling hyperspectral synthesizer.

[0081] In some implementations, the optical device can control the wheel speed to correspond to the shutter speed, such that for one of multiple images, a single filter section in filter section 106 is positioned in front of the pixels of each exposed row. Additionally or alternatively, the optical device can control the wheel speed and / or shutter speed such that each pixel of the array is exposed to each type of filter section at least once, and an image at the highest spatial resolution of imager 102 can be generated for each type of filter section.

[0082] In some implementations, the optical device and / or imager 102 may include a curtain shutter located between the pixel array and the filter wheel 104 to block light from a subset of filter sections 106 of the filter wheel 104 (e.g., not captured by the pixels before readout). For example, in rolling shutter mode and when a single filter section in filter section 106 is in front of a pixel in each exposed row, the curtain shutter may prevent light from passing through filter sections 106 other than that single filter section to reach the pixel array. Additionally or alternatively, the optical device and / or imager 102 may reset the pixels (e.g., before exposing the pixels) to clear light captured by pixels from filter sections 106 other than a single filter section (e.g., the filter section of interest).

[0083] In some implementations, the optics can control the wheel speed and / or shutter speed in a way that the wheel speed and shutter speed are not correlated. For example, the optics can control the wheel speed such that the filter wheel 104 is timed to position many different types of filter sections 106 in front of exposed pixel rows (e.g., when exposed pixel rows merge light and are read out). In some implementations, controlling the wheel speed and / or shutter speed in this way allows the optics to select any combination of filter section types.

[0084] In some implementations, the optical device may use averaging techniques to minimize sensor noise by recording and averaging multiple scans (e.g., images) for each type of filter section, thereby generating a synthetic image for each type of filter section. For example, each pixel of imager 102 may be a spectrometer having a spectral resolution defined by multiple types of filter sections in filter wheel 104.

[0085] In some implementations, if filter section 106 includes RGB filters, the optical device can combine the RGB filter outputs from the pixels of imager 102 to generate a high-resolution image (e.g., a high spatial resolution image) of the field of view of imager 102. For example, filter section 106 may include eight types of filters with different spectral bandpass widths, and the optical device can produce a color-rich image with high spatial resolution (e.g., the maximum spatial resolution of imager 102 and / or the like). In some implementations, imager 102 may not require ultraviolet (UV) and / or infrared (IR) filtering (e.g., as required by conventional digital single-lens reflex (DSLR) cameras) because rotating filter section 106 can mitigate the effects of UV and / or IR light. Additionally or alternatively, rotating filter wheel 104 may not generate mechanical vibration or may generate less mechanical vibration (e.g., compared to components that roll linearly with the filter), which can allow the optical device to produce high-definition video with high spectral resolution.

[0086] In some implementations, the optical device may use the output from the pixels of imager 102 to generate a relative spectral scan (e.g., if bright and dark reference images are generated and / or similar). Additionally or alternatively, the optical device may provide (e.g., output, display, prompt to be displayed and / or similar) spectral content for regions of interest (ROIs) in a high spatial resolution image (e.g., identify ROIs in response to user input and / or similar).

[0087] In some implementations, the spectral resolution of the optical device can be adjusted based on the application of the optical device by varying the number of filter sections 106 on the filter wheel 104 and / or the number of different types of filter sections 106. For example, the optical device may be included in and / or attached to an unmanned aircraft (e.g., a drone), and the filter wheel 104 may include fewer filter sections 106 and / or fewer types of filter sections 106 (e.g., compared to other applications) to provide 4K, 8K, and / or 16K spatial resolution per pixel of the imager 102 acting as a spectrometer and / or with color-rich images and / or videos. In some implementations, the spectral resolution of the optical device can be adjusted for applications such as agriculture, pharmaceuticals, military, healthcare, and / or similar fields.

[0088] In some implementations, the optical device can be configured (e.g., based on user input and / or the like) to capture one or more images of an object in a narrow spectral bandwidth (e.g., using one type of filter section in filter wheel 104) and / or in several narrow spectral bandwidths (e.g., using several types of filter sections 106 in filter wheel 104). In other words, the optical device can be configured to change the spectral scheme used for capturing images and / or video (e.g., which type of spectral filter is used) based on the application, user input, and / or the like. For example, a user can select the ROI for spectral analysis via the optical device, and the optical device can control the imager 102 (e.g., by adjusting shutter speed, shutter timing, and / or the like) and / or the filter wheel 104 (e.g., by adjusting wheel speed, the position of filter wheel 104, and / or the like) based on that selection.

[0089] As indicated above, Figure 1 Provided as an example. Other examples may be provided with reference to [the example]. Figure 1 The examples described are different.

[0090] Figure 2 This is a diagram of an example implementation 200 of an optical device including the imager 202 and filter wheel 204 described herein. Figure 2 As shown, the filter wheel 204 can be located in front of the imager 202, allowing light incident on the imager 202 to pass through the filter wheel 204. (Also...) Figure 2 As shown, filter wheel 204 may include multiple filter sections 206.

[0091] In some implementations, the optical device, imager 202, filter wheel 204, and filter section 206 of example implementation 200 can be similar to those in... Figure 1 Shown in and about Figure 1Example implementation 100 describes the optical device, imager 102, filter wheel 104, and filter sections 106. However, unlike the 125 filter sections 106 of the filter wheel 104 in example implementation 100, the filter wheel 204 of example implementation 200 may include, for example... Figure 2 The 250 filter sections 206 are shown.

[0092] As indicated above, Figure 2 Provided as an example. Other examples may be provided with reference to [the example]. Figure 2 The examples described are different.

[0093] Figure 3 This is a diagram of an example implementation 300 of an optical device including the imager 302 and filter wheel 304 described herein. Figure 3 As shown, the filter wheel 304 can be located in front of the imager 302, allowing light incident on the imager 302 to pass through the filter wheel 304. (Also...) Figure 3 As shown, filter wheel 304 may include multiple filter sections 306.

[0094] In some implementations, the optical device, imager 302, filter wheel 304, and filter section 306 of example implementation 300 can be similar to those in... Figure 1 Shown in and about Figure 1 The example implementation 100 describes the optical device, imager 102, filter wheel 104, and filter section 106. However, Figure 3 A close-up view of a portion of the optical device is shown, including a portion of the filter wheel 304 located in front of the imager 302. (See attached image.) Figure 3 As shown, the imager 302 may include a 2D pixel array comprising multiple rows of pixels, including row 308.

[0095] As by Figure 3 As shown in the shaded area, filter wheel 304 may include a repeating sequence of filter sections 306, and each filter section in the repeating sequence may be a filter section of a different type from the other filter sections in the repeating sequence. For example, and as... Figure 3 As shown, the repeating sequence may include ten different types of filter sections 306. In some implementations, the different types of filter sections 306 may have different spectral characteristics and / or may have different narrow spectral passbands to generate spectral content for each pixel of the imager 302.

[0096] like Figure 3 As shown, filter section 306 can have a section width smaller than the size (e.g., length and / or width) of the pixel array in imager 302. In some implementations and as in Figure 3As shown with respect to row 308, the pixel rows in the array can have a row width, and filter section 306 can have a section width greater than or equal to the row width. Additionally or optionally, filter section 306 can have a section width and / or section shape such that rows of pixels (e.g., row 308) can be located below only a portion of a filter section (e.g., during image capture in rolling shutter mode). For example, the portion of filter section 306 located in front of the array of imager 302 can have a section width greater than or equal to the row width.

[0097] In some implementations, the optical device (e.g., using imager 302) can capture one or more images in rolling shutter mode by sequentially exposing each row of pixels of the array at shutter speeds. In this respect, row 308 can be the row of pixels exposed at a given moment during image capture in rolling shutter mode. Figure 3 As indicated by arrow 310 on imager 302, each row can be sequentially exposed from the first side of the array to the second side of the array opposite the first side at a shutter speed. Figure 3 As indicated by arrow 312, the optical device can rotate the filter wheel 304 clockwise at a wheel speed, so that the filter section 306 moves from the first side of the array over the imager 302 to the second side of the array.

[0098] In some implementations, the optical device can control the wheel speed and / or shutter speed such that, for an image, a single filter section in filter section 306 is positioned in front of the pixel of each exposed row. For example, as the exposed row changes from the first side of the array to the second side during a rolling shutter scan, filter wheel 304 can rotate at a wheel speed such that a single filter section moves from the first side of the array to the second side at the same speed as the rate at which the exposed row changes. In other words, a single filter section can move in sync with the rolling shutter scan. Furthermore or alternatively, the optical device can control the wheel speed and / or shutter speed such that each pixel of the array sees each type of filter section at least once, and an image at the highest spatial resolution of imager 302 can be generated for each type of filter section.

[0099] As indicated above, Figure 3 Provided as an example. Other examples may be provided with reference to [the example]. Figure 3 The examples described are different.

[0100] Figure 4 This is a diagram of an example implementation 400 of an optical device including the imager 402 and filter wheel 404 described herein. Figure 4 As shown, the filter wheel 404 can be located in front of the imager 402, allowing light incident on the imager 402 to pass through the filter wheel 404. (Also...) Figure 4 As shown, filter wheel 404 may include multiple filter sections 406. Although Figure 4 Only three filter sections are shown, but filter wheel 404 may include additional filter sections, as indicated by the dashed lines.

[0101] In some implementations, the optical device, imager 402, filter wheel 404, and filter section 406 of example implementation 400 can be similar to those in... Figure 1 Shown in and about Figure 1 The example implementation 100 describes the optical device, imager 102, filter wheel 104, and filter section 106. However, Figure 4 A close-up view of a portion of the optical device is shown, including a portion of the filter wheel 404 located in front of the imager 402. (See attached image.) Figure 4 As shown, the imager 402 may include a 2D pixel array comprising multiple rows of pixels, including row 408.

[0102] like Figure 4 As shown, filter section 406 may include subsections 410 located in a column along the length of one or more filter sections 406. For example, a filter section may include subsections 410, and each subsection may be a filter of a different type than other subsections 410 in that filter section, may allow different wavelength ranges to pass through compared to other subsections 410 in that filter section, and / or the like. In some implementations, filter section 406 may provide vertical filter lines on filter wheel 404 (e.g., lines extending generally outward from the center of filter wheel 404 to the outer periphery of filter wheel 404), and subsections 410 may provide lateral filter lines on filter wheel 404. In this way, filter wheel 404 may filter light incident on imager 402, wherein the filtering varies in two directions.

[0103] like Figure 4 As shown, filter section 406 may have a shape and / or size such that a row of pixels (e.g., row 408) may be located below only a portion of a filter section (e.g., during image capture in rolling shutter mode). Also, Figure 4 As shown, subsection 410 may have the shape and / or size in filter section 406 such that a row of pixels (e.g., row 408) can be subdivided into sub-rows by subsection 410. For example, subsection 410 may have a shape and / or size such that the boundary of subsection 410 is perpendicular to row 408. In some implementations, subsection 410 may subdivide a row into sub-rows with the same number of pixels.

[0104] As indicated above, Figure 4 Provided as an example. Other examples may be provided with reference to [the example]. Figure 4 The examples described are different.

[0105] Figure 5 This is a diagram of an example implementation 500 of an optical device including the multiple imagers 502 and 512 described herein and a filter wheel 504. (See diagram for reference.) Figure 5 As shown, the filter wheel 504 can be located in front of the imagers 502 and 512, allowing light incident on the imagers 502 and 512 to pass through the filter wheel 504. (Also...) Figure 5 As shown, filter wheel 504 may include multiple filter sections 506.

[0106] In some implementations, the optical devices, imagers 502 and 512, filter wheel 504, and filter section 506 of example implementation 500 can be similar to those in… Figure 1 Shown in and about Figure 1 The example implementation 100 describes the optical device, imager 102, filter wheel 104, and filter section 106. However, Figure 5 A close-up view of a portion of the optical device is shown, including a portion of filter wheel 504 located in front of imagers 502 and 512.

[0107] In some implementations, imagers 502 and 512 can be the same type of imager. For example, imagers 502 and 512 can both be CCD sensors. In some implementations, including two or more imagers (e.g., imagers 502 and 512) can reduce the need to be placed on a single imager. Additionally or alternatively, the optical device can use multiple imagers to generate three-dimensional images and / or videos with high spatial and spectral resolution.

[0108] In some implementations, imagers 502 and 512 can be different types of imagers. For example, imager 502 can be a silicon-based sensor for capturing images in visible wavelengths, and imager 512 can be a PbSe sensor for capturing images in infrared wavelengths.

[0109] In some implementations, the optical device may include more than two imagers. For example, the optical device may include a first pair of imagers and a second pair of imagers. In such an example, the first part of the filter wheel (e.g., as...) Figure 5The upper left portion of the filter wheel (as shown) can be located in front of the first pair of imagers, and the second portion of the filter wheel (e.g., the lower right portion of the filter wheel) can be located in front of the second pair of imagers. In some implementations, each of the first and second pairs of imagers may include silicon-based sensors and PbSe sensors, which enable optical devices (e.g., using image rotation software and / or the like) to generate three-dimensional images and / or videos with high spatial and spectral resolution.

[0110] As indicated above, Figure 5 Provided as an example. Other examples may be provided with reference to [the example]. Figure 5 The examples described are different.

[0111] Figure 6 This is a diagram of example components of device 600, which may correspond to optical devices of example implementations 100, 200, 300, 400, and / or 500. In some implementations, the optical devices of example implementations 100, 200, 300, 400, and / or 500 may include one or more devices 600 and / or one or more components of device 600. Figure 6 As shown, device 600 may include bus 610, processor 620, memory 630, storage unit 640, input unit 650, output unit 660 and communication unit 670.

[0112] Bus 610 includes components within the device 600 that enable wired and / or wireless communication. Processor 620 includes a central processing unit, graphics processing unit, microprocessor, controller, microcontroller, digital signal processor, field-programmable gate array, application-specific integrated circuit, and / or another type of processing component. Processor 620 is implemented in hardware, firmware, or a combination of hardware and software. In some implementations, processor 620 includes one or more processors that can be programmed to perform functions. Memory 630 includes random access memory, read-only memory, and / or another type of memory (e.g., flash memory, magnetic memory, and / or optical memory).

[0113] Storage component 640 stores information and / or software related to the operation of device 600. For example, storage component 640 may include a hard disk drive, disk drive, optical disk drive, solid-state drive, optical disk, digital multifunction disk, and / or another type of non-transitory computer-readable medium. Input component 650 enables device 600 to receive input, such as user input and / or sensed input. For example, input component 650 may include a touchscreen, keyboard, keypad, mouse, buttons, microphone, switch, sensor, GPS component, accelerometer, gyroscope, actuator, and / or the like. Output component 660 enables device 600 to provide output, for example, via a display, speaker, and / or one or more light-emitting diodes. Communication component 670 enables device 600 to communicate with other devices, for example, via a wired connection and / or wireless connection. For example, communication component 670 may include a receiver, transmitter, transceiver, modem, network interface card, antenna, and / or the like.

[0114] Device 600 can perform one or more processes described herein. For example, a non-transitory computer-readable medium (e.g., memory 630 and / or storage unit 640) can store a set of instructions (e.g., one or more instructions, code, software code, program code, and / or the like) for execution by processor 620. Processor 620 can execute this set of instructions to perform one or more processes described herein. In some implementations, execution of this set of instructions by one or more processors 620 causes one or more processors 620 and / or device 600 to perform one or more processes described herein. In some implementations, hard-wired circuitry may be used in place of or in combination with instructions to perform one or more processes described herein. Therefore, the implementations described herein are not limited to any particular combination of hardware circuitry and software.

[0115] Figure 6 The number and arrangement of components shown are provided as an example. Device 600 may include... Figure 6 The components shown may be additional, fewer, different, or differently arranged components compared to those components. Additionally or alternatively, a group of components of device 600 (e.g., one or more components) may perform one or more functions described as being performed by another group of components of device 600.

[0116] Figure 7 This is a flowchart related to an example process 700 of capturing an image using an optical device including a filter wheel. In some implementations, Figure 7 One or more process blocks can be executed by optical devices (e.g., optical devices implementing 100, 200, 300, 400, and / or 500 as examples). Additionally or optionally, Figure 7One or more process blocks may be executed by one or more components of device 600, such as processor 620, memory 630, storage component 640, input component 650, output component 660, communication component 670 and / or the like.

[0117] like Figure 7 As shown, process 700 may include capturing multiple images by exposing a pixel array (block 710). For example, as described above, an optical device may capture multiple images by exposing a pixel array. In some implementations, the optical device may include a pixel array and a filter wheel.

[0118] As in Figure 7 As further shown, process 700 may include rotating a filter wheel while capturing multiple images, wherein the filter wheel includes filter sections, and wherein a portion of the filter wheel in front of the pixel array includes two or more filter sections (block 720). For example, as described above, the optical device may rotate the filter wheel while capturing multiple images. In some implementations, the filter wheel includes filter sections. In some implementations, a portion of the filter wheel in front of the pixel array includes two or more filter sections.

[0119] Process 700 may include additional implementations, such as any single implementation or any combination of implementations described below and / or with respect to one or more other processes described elsewhere in this document.

[0120] In the first implementation, the pixel array comprises rows of pixels, and capturing multiple images involves exposing each row of pixels in the pixel array sequentially at a shutter speed and in a rolling shutter mode.

[0121] In the second implementation, either alone or in combination with the first implementation, multiple images are captured by exposing a pixel array, including pixels of the pixel array that are simultaneously exposed in a global shutter mode.

[0122] In the third implementation, either alone or in combination with one or more of the first and second implementations, the filter wheel comprises a repeating sequence of filter sections, and each filter section in the repeating sequence of filter sections is a filter section of a different type from the other filter sections in the repeating sequence of filter sections.

[0123] In the fourth implementation, either alone or in combination with one or more of the first to third implementations, the pixel array is a first pixel array, the plurality of images are a first plurality of images, the portion of the filter wheel in front of the first pixel array is a first portion, the device includes a second pixel array, and the second portion of the filter wheel in front of the second pixel array includes two or more filter sections.

[0124] In the fifth implementation, either alone or in combination with one or more of the first to fourth implementations, process 700 includes capturing a second plurality of images while capturing the first plurality of images and while rotating the filter wheel by exposing the second pixel array.

[0125] In the sixth implementation, either alone or in combination with one or more of the first to fifth implementations, process 700 includes generating at least one of a three-dimensional image or a three-dimensional video based on a first plurality of images and a second plurality of images.

[0126] although Figure 7 An example block of process 700 is shown, but in some implementations, process 700 may include... Figure 7 The blocks described herein may be additional blocks, fewer blocks, different blocks, or blocks arranged differently. Additionally or optionally, two or more blocks of process 700 may be executed in parallel.

[0127] Figure 8 This is a flowchart related to an example process 800 of capturing an image using an optical device including a filter wheel. In some implementations, Figure 8 One or more process blocks can be executed by optical devices (e.g., optical devices implementing 100, 200, 300, 400, and / or 500 as examples). Additionally or optionally, Figure 8 One or more process blocks may be executed by one or more components of device 600, such as processor 620, memory 630, storage component 640, input component 650, output component 660, communication component 670 and / or the like.

[0128] like Figure 8 As shown, process 800 may include capturing multiple images by sequentially exposing each row of pixels in an array of pixel rows at a shutter speed and in a rolling shutter mode (block 810). For example, as described above, an optical device may capture multiple images by sequentially exposing each row of pixels in an array of pixel rows at a shutter speed and in a rolling shutter mode. In some implementations, the optical device may include an array of pixel rows and a filter wheel.

[0129] As in Figure 8 As further shown, process 800 may include rotating a filter wheel at a wheel speed while capturing multiple images, wherein the filter wheel includes filter sections, and wherein a portion of the filter wheel in front of the array includes two or more filter sections (block 820). For example, as described above, the optical device may rotate the filter wheel at a wheel speed while capturing multiple images. In some implementations, the filter wheel includes filter sections. In some implementations, a portion of the filter wheel in front of the array includes two or more filter sections.

[0130] Process 800 may include additional implementations, such as any single implementation or any combination of implementations described below and / or with respect to one or more other processes described elsewhere in this document.

[0131] In the first implementation, the wheel speed corresponds to the shutter speed, such that for one of a plurality of images, a single filter section in the filter section is positioned in front of the pixels of each exposed row.

[0132] In the second implementation, either alone or in combination with the first implementation, process 800 includes blocking light from filter sections other than a single filter section.

[0133] In the third implementation, either alone or in combination with one or more of the first and second implementations, process 800 includes resetting the pixels of a row of pixels before exposing it.

[0134] In the fourth implementation, either alone or in combination with one or more of the first to third implementations, process 800 includes capturing multiple images with a single type of filter section located in front of the pixels of each exposed row, and combining the multiple images from the multiple images by mathematical transformations of the pixel values ​​of the multiple images (e.g., averaging and / or the like) to generate a synthetic image.

[0135] In the fifth implementation, either alone or in combination with one or more of the first to fourth implementations, rotating the filter wheel includes rotating the filter wheel at a wheel speed such that multiple filter sections in the filter section are positioned in front of each exposed row.

[0136] although Figure 8 An example block of process 800 is shown, but in some implementations, process 800 may include... Figure 8 The blocks described herein may be additional blocks, fewer blocks, different blocks, or blocks arranged differently. Additionally or optionally, two or more blocks of process 800 may be executed in parallel.

[0137] While the description of the above implementations includes optical devices comprising one or more imagers and filter wheels, some implementations may include attachment devices (e.g., sled devices and / or similar devices) for attaching to optical devices, imaging systems, and / or similar devices, wherein the attachment devices include filter wheels similar to the one or more filter wheels described with respect to the above implementations. For example, a sled device for attaching to and / or receiving a user device (e.g., a communication and / or computing device, mobile phone, smartphone, laptop computer, tablet computer, handheld computer, desktop computer, gaming device, wearable communication device, smartwatch, smart glasses, and / or similar devices) may include filter wheels such that the filter wheels are positioned in front of the user device's camera. In some implementations, the filter wheels may include synchronization filter sections (e.g., dark filter sections and / or similar devices), and the user device may be configured (e.g., via an application, via the sled device, and / or similar devices) to detect the synchronization filter sections and to synchronize the camera's shutter speed based on the synchronization filter sections (e.g., to control the shutter speed based on wheel speed and / or similar devices). Additionally or optionally, the skateboard device may communicatively connect to a user device (e.g., via a wired and / or wireless connection, such as Bluetooth, Wi-Fi, cellular, NFC, and / or the like), and may provide (and / or receive from) the user device information (e.g., camera shutter speed, wheel speed, and / or the like), commands (e.g., command signals to use the camera shutter speed, command signals to rotate the filter wheel at the wheel speed, command signals to stop rotating the filter wheel, and / or the like), and / or the like. In some implementations, one or more filters (e.g., RGB filters, Bayer filters, and / or the like) employed by the camera of the user device may support one or more filter sections of the filter wheel.

[0138] The foregoing disclosure provides illustrations and descriptions, but is not intended to be exhaustive or to limit implementations to the precise forms disclosed. Modifications and variations are possible based on the foregoing disclosure, or may be derived from the practice of implementation.

[0139] The aforementioned disclosure uses a spectrometer as an example; however, the calibration principles, procedures, and methods described herein can be used with any sensor (including, but not limited to, other optical and spectral sensors).

[0140] As used herein, the term “component” is defined to be broadly interpreted as hardware, firmware, and / or a combination of hardware and software.

[0141] It will be apparent that the systems and / or methods described herein can be implemented in various forms of hardware, firmware, or combinations of hardware and software. The actual dedicated control hardware or software code used to implement these systems and / or methods does not limit the implementation. Therefore, while the operation and behavior of the systems and / or methods are described herein without reference to specific software code, it should be understood that software and hardware can be used to implement the systems and / or methods based on those described herein.

[0142] Even if specific combinations of features are listed in the claims and / or disclosed in the specification, these combinations are not intended to limit the disclosure of various implementations. In fact, many of these features can be combined in ways not specifically listed in the claims and / or disclosed in the specification. Although each appended dependent claim may be directly subordinated to only one claim, the disclosure of various implementations includes each dependent claim in combination with each other claim in the group of claims.

[0143] No element, action, or instruction used herein should be construed as essential or necessary unless explicitly described herein. Furthermore, as used herein, the articles “a” and “an” are intended to include one or more items and may be used interchangeably with “one or more.” Furthermore, as used herein, the article “the” is intended to include one or more items associated with the article “the” and may be used interchangeably with “the one or more.” Furthermore, as used herein, the term “set” is intended to include one or more items (e.g., related items, unrelated items, a combination of related and unrelated items, etc.) and may be used interchangeably with “one or more.” Where only one item is intended to be described, the phrase “only one” or similar language is used. Furthermore, as used herein, the terms “has,” “have,” “having,” or the like are specified as open-ended terms. Furthermore, the phrase “based on” is intended to mean “at least partially based on,” unless explicitly stated otherwise. Furthermore, as used herein, the term “or” is intended to be inclusive when used in series and may be used interchangeably with “and / or” unless otherwise expressly stated (e.g., in combination with “any” or “only one of them”).

Claims

1. An image capture method, comprising: Multiple images are captured by a device including a pixel array and filter wheels by exposing the pixel array; as well as The filter wheel is rotated by the device while capturing the plurality of images. The filter wheel includes filter sections, which are positioned in the filter wheel in a repeating sequence. At any given time of rotation, the filter wheel comprises two or more filter sections in a portion of the front of the pixel array; Each filter section in the repeating sequence of the filter section is a filter section of a different type from the other filter sections in the repeating sequence of the filter section; The pixel array described therein comprises pixel rows; and Capturing the plurality of images by exposing the pixel array includes sequentially exposing each row of pixels in the pixel array in a rolling shutter mode at a shutter speed.

2. The method according to claim 1, wherein: The pixel array is the first pixel array. The plurality of images are a first plurality of images. The portion of the filter wheel in front of the first pixel array is the first part. The device includes a second pixel array, and The filter wheel includes two or more filter sections in the second portion in front of the second pixel array.

3. The method according to claim 2, further comprising: A second plurality of images are captured by exposing the second pixel array while capturing the first plurality of images and while rotating the filter wheel.

4. The method according to claim 3, further comprising: At least one of a 3D image or a 3D video is generated based on the first plurality of images and the second plurality of images.

5. The method of claim 1, wherein rotating the filter wheel comprises rotating the filter wheel at a wheel speed, wherein the wheel speed corresponds to the shutter speed, such that for an image in the plurality of images, a single filter section in the filter section is positioned in front of the pixels of each exposed row.

6. The method according to claim 5, further comprising: Block light from filter sections other than the single filter section mentioned above.

7. The method according to claim 5, further comprising: Before exposing a row of pixels, reset the pixels in that row.

8. The method according to claim 5, further comprising: Capture multiple images using a single type of filter section located before the pixels of each exposed row; as well as A composite image is generated by combining multiple images by mathematically transforming the pixel values ​​of the multiple images.

9. The method of claim 1, wherein rotating the filter wheel comprises rotating the filter wheel at a wheel speed such that a plurality of filter sections in the filter sections are positioned in front of each exposed row.

10. An optical device, comprising: An array of pixel rows; A filter wheel, the filter wheel comprising filter sections, the filter sections being positioned in the filter wheel in a repeating sequence of filter sections; At any time of rotation of the filter wheel, the portion of the filter wheel located in front of the array includes two or more filter sections; as well as One or more processors, wherein the one or more processors are configured to: Multiple images are captured by exposing the array; as well as Rotate the filter wheel while capturing the plurality of images; Each filter section in the repeating sequence of the filter section is a filter section of a different type from the other filter sections in the repeating sequence of the filter section; and Capturing the plurality of images by exposing the array includes sequentially exposing each row of pixels of the array in a rolling shutter mode at a shutter speed.

11. The device of claim 10, wherein the pixel row has a row width, and The portion of the filter section located at the front of the array has a section width greater than or equal to the row width.

12. The device according to claim 10, At least one of the following conditions is true: Each filter section in the repeating sequence of filter sections allows a different wavelength range to pass through compared to other filter sections in the repeating sequence of filter sections, or The repeating sequence of filter sections includes at least six different types of filter sections.

13. The device of claim 10, wherein the filter wheel comprises at least 100 filter sections.

14. The device of claim 10, further comprising a curtain shutter located between the array and the filter wheel. The curtain shutter blocks a subset of light from the filter section.

15. The device of claim 10, wherein the filter section comprises subsections located in a column along the length of the filter section.

16. The device of claim 15, wherein each subsection of the filter section allows a different wavelength range to pass through compared to the other subsections of the filter section.

Images