Image perception system

Abstract

According to one aspect, an image sensing system is provided that includes at least two rolling shutter image sensors, each rolling shutter image sensor comprising an array of pixels of a frame arranged in a plurality of image lines extending along a width direction and distributed in a scan direction perpendicular to the width direction, each rolling shutter image sensor scanning its frame by sequentially scanning each image line along the scan direction, scanning each image line including rendering pixels in each image line sensitive to light for a predetermined exposure time to determine an intensity readout for each pixel of the respective image line, and a first image line in the scan direction for each rolling shutter image sensor by staggering the scanning of the frames for each rolling shutter image sensor in time. the controller configured to initiate scanning of frames from the image sensor with a time delay relative to the other rolling shutter image sensors; a lens system that projects similar transmitted images of the object onto each rolling shutter image sensor, the transmitted images being projected onto each rolling shutter image sensor such that each pixel of the transmitted images corresponding to a location on the object matches a pixel or group of pixels of the other transmitted images corresponding to a respective location on the object, and such that the transmitted images are captured by each rolling shutter image sensor in different colors of light defined by different wavelengths; and a processor that identifies matching pixels of the different transmitted images corresponding to respective locations on the object and determines a color of the object based on an intensity readout of each matching pixel and the color of light to which each matching pixel was exposed while it was sensitive.

Description

Technical Field

[0001] The present invention relates to an image sensing system for determining the color of an object and a method for determining the color of an object.

Background Art

[0002] Multispectral imaging involves creating an image that has more spectral resolution (i.e., colors) than the three colors (red, green, and blue) that the human eye can distinguish. Generally, creating an image with color involves using an image sensor that can sense multiple colors. However, color sensors are more expensive than black-and-white image sensors, and sensors that can sense more than three colors are even more expensive and difficult to fabricate.

[0003] U.S. Patent Application Publication No. 2020 / 0322552 discloses a method that enables the use of information accessible by fluorescence imaging to be optimized. For this purpose, the method implements a combination of a protocol for calibration and synchronization of pulsed light that excites a fluorescent marker by operating a fluorescence camera in rolling shutter mode. With appropriate correction factors, it becomes possible to use the complete signal integrated by all of the camera's photodiodes so that no image is lost.

Summary of the Invention

[0004] According to one aspect, an image sensing system is provided, the image sensing system including at least two rolling shutter image sensors, each rolling shutter image sensor including an array of pixels of a frame disposed in a plurality of image lines extending along a width direction and distributed in a scanning direction perpendicular to the width direction, each rolling shutter image sensor scanning its frame by sequentially scanning each image line along the scanning direction, scanning each image line including determining an intensity reading for each pixel of each image line as sensitive to light for a predetermined exposure time; at least two rolling shutter image sensors; a controller configured to start scanning of a frame from a first image line in the scanning direction for each rolling shutter image sensor with a time delay with respect to other rolling shutter image sensors by temporally shifting the scanning of the frame for each rolling shutter image sensor; a lens system that projects similar transmitted images of an object onto each rolling shutter image sensor, the transmitted images being projected onto each rolling shutter image sensor such that each pixel of the transmitted image corresponding to a position on the object coincides with a pixel or a group of pixels of another transmitted image corresponding to the respective position on the object, and such that the transmitted images are captured by each rolling shutter image sensor in different colors of light defined by different wavelengths; and a processor that identifies matching pixels of different transmitted images corresponding to respective positions on the object and determines a color of the object by superimposing colors from each matching pixel based on the intensity reading of each matching pixel and the color of the light to which the respective matching pixel was exposed while the pixel was sensitive.

[0005] To project each transmitted image in a single color different from the colors of the other transmitted images, the lens system includes a prism that splits incident light into two or more different colors of light defined by different wavelengths, whereby each rolling shutter image sensor captures transmitted images having different colors of light.

[0006] This image sensing system includes a light generator that generates at least two different colors of light defined by different wavelengths to irradiate the target. This image sensing system includes a timing module that controls the light generator so that at least two different colors of light are sequentially generated during the scanning of each frame for each rolling shutter image sensor, so that each transmitted image is captured by each rolling shutter image sensor having two or more different colors.

[0007] Each image line of the transmitted image on each rolling shutter image sensor coincides with the image line of the transmitted image of another rolling shutter image sensor corresponding to the position on the target. The processor identifies the matching image lines and determines the color of each position of the target based on the intensity reading of each pixel in the matching image lines and the color of the light to which each pixel was exposed while each pixel was sensitive.

[0008] The matching pixels or the matching image lines are predetermined based on the configuration of the rolling shutter image sensor and the lens system.

[0009] In the simplest form, each rolling shutter image sensor has the same number of image lines to require a minimum amount of processing, and the image lines on each image sensor coincide with the corresponding image lines at the same positions in the scanning directions on different image sensors.

[0010] The switching frequency between the colors of light during the scanning of a single frame is equal to the number of image sensors or a multiple thereof. The switching frequency between the colors of light during the scanning of a single frame is equal to or greater than the number of image sensors.

[0011] The number of colors of light configured to be generated by the light generator is equal to the number of rolling shutter image sensors or a multiple thereof.

[0012] The time delay between the start of the frame scan for each rolling shutter image sensor is equal to the time required to scan a single frame divided by the number of rolling shutter image sensors.

[0013] The lens system includes an optical beam splitter that projects the same transmitted image onto at least two different rolling shutter image sensors.

[0014] According to a second aspect, there are provided at least two rolling shutter image sensors, each rolling shutter image sensor comprising an array of pixels of a frame arranged in a plurality of image lines extending along a width direction and distributed in a scanning direction perpendicular to the width direction, each rolling shutter image sensor scanning its frame by sequentially scanning each image line along the scanning direction, and scanning each image line includes determining an intensity reading for each pixel in each image line as sensitive to light for a predetermined exposure time, and a method for determining a color of an object using at least two rolling shutter image sensors, the method comprising: scanning the frames of each rolling shutter image sensor in a staggered manner such that the start of the frame scan for each rolling shutter image sensor is scanned with a time delay with respect to the other rolling shutter image sensors; projecting similar transmitted images onto each rolling shutter image sensor such that each pixel of the transmitted image corresponding to a position on the object coincides with a pixel or a group of pixels of another transmitted image corresponding to the respective position on the object, and the transmitted images are captured by each rolling shutter image sensor in different colors of light defined by different wavelengths; identifying matching pixels of different transmitted images corresponding to respective positions on the object; and determining the color of each respective position on the object by overlapping the colors from each matching pixel based on the intensity reading of each matching pixel in the transmitted image and the color of the light to which the respective matching pixel was exposed while the pixel was sensitive.

[0015] This method has the step of splitting incident light into two or more different colors of light defined by different wavelengths in order to project each transmitted image with a single color that is different in color from other transmitted images, so that each rolling shutter image sensor captures transmitted images with different colors of light.

[0016] This method has the step of sequentially generating at least two different colors of light defined by different wavelengths during the scanning of each frame of the rolling shutter image sensor so that each transmitted image is captured by each rolling shutter image sensor having two or more different colors. The time delay is shorter than the time required to scan a single frame.

[0017] Each image line of the transmitted image on each rolling shutter image sensor coincides with an image line of another transmitted image of another rolling shutter image sensor corresponding to a position on the object. This method has the step of identifying the coinciding image lines and determining the color of each position of the object based on the intensity reading of each pixel in the coinciding image lines and the color of the light to which each pixel was exposed while each pixel was sensitive.

[0018] The coinciding image lines are determined in advance based on the configuration of the rolling shutter image sensor and the projection of the transmitted image.

[0019] At least two different colors of light are sequentially generated at a frequency equal to the number of image sensors or a multiple thereof during the scanning of a single frame. The switching frequency between the colors of light during the scanning of a single frame is equal to or greater than the number of image sensors.

[0020] The number of colors of light generated is equal to the number of rolling shutter image sensors or a multiple thereof.

[0021] The time delay between the start of the frame scan for each rolling shutter image sensor and the start is equal to the time required to scan a single frame divided by the number of rolling shutter image sensors.

[0022] These and other aspects will become apparent with reference to the embodiments described hereinafter in this specification.

[0023] Next, exemplary embodiments will be described with reference only to the following drawings by way of example.

Brief Description of the Drawings

[0024]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Modes for Carrying Out the Invention

[0025] FIG. 1 shows an image sensing system 10 used to sense an image of object 12 and determine the color of object 12. The image sensing system 10 includes two rolling shutter image sensors 14 and a lens system 16 configured to project similar outgoing images of object 12 onto each image sensor 14. Thus, in this example, the lens system 16 is configured to project similar outgoing images of object 12. In some examples, there are more than two rolling shutter image sensors, and the lens system is configured to project more than two similar outgoing images of the object onto the image sensors such that at least one outgoing image is projected onto each image sensor.

[0026] The lens system 16 in this example includes two lenses 24 arranged adjacent to each other in front of each image sensor 14. The lens system 16 is thus configured to be disposed between the object 12 being imaged and each of the image sensors 14.

[0027] In other examples, the lens system may include a lens and one or more beam splitters, where the beam splitter is arranged between the lens and the image sensor or the lens is arranged between the beam splitter and each image sensor such that the beam splitter is configured to split the beam from the lens to project two or more identical images onto the image sensor.

[0028] The image sensing system 10, in this example, includes a light generator 18 configured to generate two different colors of light for irradiating the object 12. In this example, the light generator 18 includes two lights 20 that are green (G) and red (R), and these lights are configured to generate light flashes at different wavelengths (i.e., different colors of light). In other examples, the light generator can be configured to generate three different colors of light using three lights, or more than three colors using more than three lights. In a further example, the lights can be configured to continuously generate continuous light, in contrast to flashes. It will be understood that generating different colors of light can also be achieved by using just one light together with a plurality of different color filters. In some examples, the color of the light can be different from green and red.

[0029] The image sensing system 10 includes a timing module 26 configured to control the light generator 18 to generate different colors of light at appropriate intervals, that is, to control different lights 20 of the light generator 18 to flash at precise times, which will be described in more detail later with reference to FIG. 2.

[0030] The image sensing system 10 further includes a controller 29 configured to control the timing of the scanning of the image sensor 14, which will also be described in more detail later with reference to FIG. 2.

[0031] FIG. 2 shows a plurality of frames 30 having transmitted images 42 sequentially captured using two rolling shutter image sensors 14 (a first rolling shutter image sensor 14a and a second rolling shutter image sensor 14b) in a capture time map 48, where the two image sensors 14 have the same frame rate.

[0032] Each rolling shutter image sensor 14 comprises an array of pixels 32 of a frame 30 that are arranged along an image line 34 extending along a width direction 36 and are distributed in a scanning direction 40 perpendicular to the width direction 36. In this example, each image sensor 14 comprises only four image lines 34 for the sake of simplicity. It will be understood that the image sensors can have any suitable number of image lines.

[0033] Each rolling shutter image sensor 14 is configured to scan an entire frame 30 of pixels 32 by sequentially scanning each image line 34 along the scanning direction 40. In other words, each rolling shutter image sensor 14 scans an image line 34 and then the adjacent image line 34 in the scanning direction 40 follows.

[0034] The scanning of each image line 34 includes sequentially scanning each pixel 32 of the image line 34 along the width direction 36. In other words, the image sensor 14 scans a pixel 32 of the image line 34 and then the adjacent pixel of the image line 34 in the width direction 36 follows. Scanning each image line 34 includes making the pixels 32 in each respective image line 34 sensitive to light for a predetermined exposure time and then reading out an intensity reading for each pixel 32.

[0035] FIG. 2 shows that a plurality of frames 30 of a first image sensor 14a and a second image sensor 14b are sequentially captured, and the time delay compared between the scanning of the frame 30 from the first image sensor 14a and the scanning of the frame 30 from the second image sensor 14b is t d is. The time delay t d is controlled by a controller 29 of the image sensing system 10. In other words, the controller 29 delays the scanning of the frame 30 for each image sensor 14 from the first image line 34 in the scanning direction 40 by a time delay t dconfigured to temporally shift the scanning of the frame 30 with respect to each image sensor 14 so as to start with it (shown in FIG. 2). In this example, the time delay t d is the same when compared between all the frames 30 captured by the first image sensor 14a and the frames 30 captured by the second image sensor 14b, and is equal to half of the time required to scan the entire frame 30. In other examples, the time delay may be more than half of the time required to scan the entire frame or less than half of the time required to scan the entire frame with respect to the time required to scan the entire frame. The time delays at which the scanning starts may not be equal between different image sensors.

[0036] Next to the image sensor 14, a scanning time map 50 is shown. The scanning time map 50 shows the passage of time from left to right. For each image line 34, the scanning time map 50 shows an exposure time 52 and a subsequent intensity readout 54. In other words, while the image line 34 becomes sensitive to light during a predetermined exposure time 52, the image sensor 14 is configured to read out the pixels 32 of the image line 34. Reading out the pixels 32 includes determining the intensity for each pixel 32 of each image line 34 corresponding to the intensity of the light received at each pixel 32 during the exposure time 52.

[0037] The start of the exposure time 52 for each image line 34 in the scanning direction 40 is offset by a predetermined amount of time from the preceding image line 34, which corresponds to the amount of time 54 required to read out the image line 34. Accordingly, each image line 34 is sensitive to light for the same amount of time, but the light sensitivity for each image line 34 starts at different times.

[0038] The timing module 26 of the image sensing system 10 is configured to control the light generator 18 to sequentially generate two different colors of light during the scanning of each frame 30 of each image sensor 14. Therefore, each frame 30 of each image sensor 14 captures a transmitted image irradiated by two different colors of light while the pixels 32 of the respective image sensors 14 are sensitive to the light. In other words, each transmitted image 42 captured by each image sensor 14 has at least two different colors of light (best shown in FIG. 3).

[0039] In this example, the flash 56 of light from the light generator 18 is controlled to irradiate the object 12 at intervals, and the flash 56 of light alternates between green (G) and red (R), and the timing is determined such that at least one green flash 56 and at least one red flash 56 irradiate the object 12 during the scanning of each frame 30 for each image sensor 14. In this example, each flash 56 of light is generated to irradiate the object 12 when half of the image lines 34 of a single frame 30 are sensitive. Therefore, there are two flashes of light during the scanning of each frame 30. In the simplified example shown in FIG. 2, this means that when only the first two image lines 34 of the first frame 30 of the first image sensor 14a are sensitive, a red (R) flash 56 of light is generated, and then when only the first two image lines 34 of the first frame 30 of the second image sensor 14b are sensitive and the third and fourth image lines 34 of the first image sensor 14a are sensitive, a green (G) flash 56 of light is generated to irradiate the object. This is repeated for each frame 30.

[0040] An exemplary transmitted image 42 captured as a result of the alternating colors of light flashed at intervals is shown in FIG. 3.

[0041] As shown in Fig. 3, a plurality of transmitted images 42 are shown in the capture time map 48 together with the colors (G) and (R) of the light that irradiates the object 12 while the image line 34 is sensitive. It can be seen that the lower half of the transmitted image 42 in the first frame 30 of the first image sensor 14a and the upper half of the transmitted image 42 in the first frame 30 of the second image sensor 14b are both irradiated with green light simultaneously. Therefore, by combining the upper half of the transmitted image 42 from the second image sensor 14b and the lower half of the transmitted image 42 from the first image sensor 14a, it is possible to create a joined image 60 from the transmitted image 42 that shows the entire object 12 irradiated with green light in the time required to read out half of the frame 30. This can be repeated for the upper half of the second frame 30 from the first image sensor 14a and the lower half of the first frame 30 of the second image sensor 14b that are irradiated with red light (R) simultaneously, and thus, it is possible to create a joined image 60 of the entire object 12 irradiated with red light (R) in the time required to read out half of the frame 30. This can be repeated for each half-frame 30 read out from both image sensors 14 that capture the object 12 irradiated with a single color of light, resulting in a plurality of joined images 60 derived from the transmitted image 42.

[0042] Referring back to Fig. 1, the image sensing system 10 further includes a processor 28 configured to identify matching pixels 32 of different transmitted images 42 that each correspond to the same position on the object 12. For example, pixel A shown as the first pixel 32 in the scanning direction and the width direction for each frame 30 in Fig. 3 corresponds to the same position on the object 12 for each frame 30, so pixel A on each frame 30 matches pixel A on other frames 30.

[0043] In this example, since the transmitted image 42 in this example is projected onto each image sensor 14 with the same size, and each image sensor 14 has the same number of image lines 34, assuming that the object 12 being imaged does not move quickly, each pixel 32 of a certain transmitted image 42 corresponding to a certain position on the object 12 coincides with the pixel 32 at the same position of the image sensor 14 of another transmitted image 42 corresponding to the same position on the object 12. Further, the entire image line 34 of a certain transmitted image 42 corresponding to a certain position on the object 12 coincides with the entire image line 34 on another transmitted image 42 corresponding to the same position on the object 12.

[0044] It should be understood that the transmitted images do not have to be of the same size, and a pixel of a certain transmitted image corresponding to a certain position on the object can correspond to a group of pixels of another transmitted image corresponding to the same position on the object.

[0045] The matching image lines 34 and pixels 32 are predetermined by the configuration of the lens system 16 and the image sensor 14. Therefore, only a very small amount of processing power is required to identify the matching image lines 34 or the matching pixels 32. When the probability that the image sensing system captures a fast-moving object is high, more processing is required to identify similar, and thus highly probable, parts of each transmitted image corresponding to the same position on the object.

[0046] In other examples, when the projected transmitted images are not of the same size, the entire image lines do not coincide, but still, the matching pixels can be predetermined based on the configuration of the lens system and the image sensor.

[0047] The processor 28 is configured to determine the color of each matching pixel based on the intensity readings of the matching pixel A irradiated by green light and red light. For example, by overlapping the green-joined image 60 and the red-joined image 60, even when the image sensor 14 being used is a monochrome image sensor, it is possible to create the output image 70 with the color of the object 12 determined for each pixel. In other words, the processor 28 is configured to determine the color of the object 12 based on the intensity readings of each matching pixel 32 and the color of the light to which each matching pixel 32 was exposed while being sensitive. The determined color has a spectral resolution including the wavelengths of the two colors of light generated by the light generator 18 during the scanning of each frame 30.

[0048] Since the joined image 60 is directly derived from the transmitted image 42, it will be understood that the intermediate step of forming the joined image 60 does not require determining the color. Rather, it is simply possible to determine the color by identifying the matching pixels 32 from the transmitted image 42 and the color of the light to which the pixels 32 were exposed, and overlapping different colors and their intensities.

[0049] With a plurality of image sensors 14 and the staggered scanning of each image sensor, the effective frame rate of the output image 70 (i.e., the image with the determined color) increases. In this example, the effective frame rate doubles, but in other examples with more image sensors, the effective frame rate further increases by a multiple of the number of image sensors.

[0050] While it has been described heretofore that two light flashes 56 are generated during the scanning of one frame 30, in other examples, the timing module may control the light generator 18 to generate light flashes at different frequencies, such as more than two flashes during the scanning of each frame. Alternatively, generating the light flash 56 at a frequency equal to or a multiple of the number of image sensors during the scanning of a single frame will reduce the processing required to decompose colors, because the matching image lines are always the same between adjacent frames in time.

[0051] In some examples where there are more than two image sensors, there may be more lights configured to generate different colors of light. In an example where there are three image sensors, there may be three or more lights that generate three or more different colors of light. When the number of colors of light is equal to or a multiple of the number of image sensors, the processing required to determine the color is reduced.

[0052] In this example, the time delay between the start of scanning of different rolls has been described as equal to half of the time required to scan the entire frame, but in other examples, the time delay can be any appropriate time delay. When the time delay is equal to the time required to scan a single frame divided by the number of image sensors, as in this example, the processing required to determine the color is reduced. Further, when the number of image lines scanned during the time delay period is equal to the number of image lines exposed to a single flash of light, the processing required to determine the color is further reduced, because the same image lines always match between frames with different colors, and as a result, identifying the overlapping matching image lines is always the same between adjacent frames captured by the image sensors.

[0053] FIG. 4 shows a second exemplary image sensing system 100 used to sense an image of object 12 and determine the color of object 12. The second exemplary image sensing system 100 includes three rolling shutter image sensors 140 similar to the rolling shutter image sensor 14 in the first exemplary image sensing system 10.

[0054] The image sensing system 10 includes a lens system 160 configured to project similar transmitted images of object 12 onto each image sensor 140. Thus, in this example, the lens system 160 is configured to project three similar transmitted images of object 12. In some examples, there may be only two rolling shutter image sensors, or more than three image sensors, and the lens system may be configured to project two or more than three similar transmitted images of the object onto the image sensors.

[0055] The lens system 160 in this example includes a trichromatic prism 162 configured to split the incoming light from object 12 into three different colors of light defined by different wavelengths. In this example, the different colors are green (G), blue (B), and red (R). Each image sensor 140 is positioned to capture a transmitted image 142 (shown in FIG. 5) having different colors of light using a lens 24 disposed between the prism 162 and each image sensor 140. In other examples, there may be only one lens disposed between the object and the prism. Each image sensor 140 may thus be a black and white image sensor because it is known which color of light each image sensor 140 is receiving, and thus the intensity of the received light corresponds to the intensity of that color.

[0056] The second exemplary image sensing system 100 includes a processor 128 that is configured to identify matching pixels of different transmitted images 142 corresponding to respective positions on the object 12 and determine the color of the object based on the intensity readings of each matching pixel and the color of the light exposed while each matching pixel is sensitive.

[0057] The second exemplary image sensing system 100 includes a controller 129 that is configured to control the timing of the scanning of the image sensor 140 in a manner similar to the first exemplary image sensing system 10 shown in the capture time map 48 of FIG. 2. In other words, the controller 129 controls the scanning of each image sensor 140 such that there is a time delay during the scanning of the frame 130 for each image sensor 140.

[0058] Although the lens system has been described as including a trichromatic prism that divides incoming light into three transmitted images of different colors, in some examples, the prism may divide the light into any suitable number of different colors corresponding to the number of image sensors.

[0059] FIG. 5 shows a plurality of frames 130 captured by the first image sensor 140a, the second image sensor 140b, and the third image sensor 140c. In this example, the time delay t d between the start of the scanning of one image sensor 140 and the start of the scanning of another image sensor 140 is equal to one-third of the time required to scan the entire frame 130. In other examples, the time delay may be longer than one-third of the time required to scan the entire frame or shorter than one-third of the time required to scan the entire frame. The time delays at which the scanning starts may not be equal between different image sensors.

[0060] When the time delay in the first exemplary image sensing system 10 and the second exemplary image sensing system 100 is constant among a plurality of image sensors and is equal to the time required to scan a single frame divided by the number of image sensors, a constant effective increased frame rate is maximized.

[0061] Due to the configuration of the lens system 160, each image sensor 140 captures the transmitted image 42 in a single color, but the color is different for each image sensor 140.

[0062] The output image 170 can be derived from the transmitted image 142 after the scanning of any single transmitted image 142 (excluding the first two images of the sequence, or the first n - 1 images of the sequence where n is the number of image sensors) is completed. For example, after the first frame 130 from each image sensor 140, R1, G1, and B1 are captured, and the output image 170 can be created by the processor 128 determining the color of each matching pixel in the frames R1, G1, and B1. As in the first exemplary image sensing system 10, the pixels or the entire image lines match based on the configuration of the lens system and the image sensor. When the next frame R2 ends after the time required to scan one-third of the frame 130, the next output image 170 can be created based on R2, G1, and B1. Since each output image 170 can then be derived from the transmitted image 142 that ended at the closest point in time from each image sensor 140 (i.e., the transmitted image 142 that ended at the closest point in time for each color), new output images are created at a rate three times the frame rate of each image sensor 140. Thus, by the staggered scanning of the plurality of image sensors 140, the effective frame rate of the image sensing system 100 can be increased for the determined color image (i.e., the output image 170).

[0063] FIG. 6 is a flowchart showing steps of a method 200 for determining a color of an object using at least two rolling shutter image sensors 14, 140 as shown in the first exemplary image sensing system 10 or the second exemplary image sensing system 100.

[0064] Block 202 of method 200 has the step of scanning frames 30, 130 for each image sensor 14, 140 in a staggered manner where the start of scanning the frames 30, 130 for each image sensor 14, 140 is time-delayed by a time delay t with respect to another image sensor 14, 140. d

[0065] Block 204 of method 200 has the step of projecting similar transmitted images 42, 142 onto each image sensor 14, 140, where in this case each pixel 32 of the transmitted images 42, 142 corresponding to a certain position on the object 12 coincides with a pixel 32 or a group of pixels 32 of another transmitted image 42, 142 corresponding to the respective position on the object 12, so that the transmitted images 42, 142 are captured by each rolling shutter image sensor 14, 140 in different colors of light defined by different wavelengths. In other words, each individual transmitted image is captured in a different color of light, or each individual transmitted image is captured in a single color of light, but each transmitted image is captured in a different color of light.

[0066] Block 206 of method 200 has the step of sequentially generating different colors of light for irradiating the object during the scanning of each frame of the rolling shutter image sensor so that each transmitted image is captured by the rolling shutter image sensor having two or more colors at that moment. For example, the first exemplary image sensing system 10 is used to generate a transmitted image 42 in which each of different portions of the transmitted image 42 captures two or more colors of light.

[0067] For example, block 206 that uses a second exemplary image sensing system 100 having a prism 162 that splits incident light into a plurality of different colors is optional, and it will be understood that a light generator is not necessary. Each individual transmitted image 142 is captured with only a single color of light, but each of the image sensors 140 captures a different color of light. The transmitted images 142 project the same field of view of the object 12, but each projects a different color of the object 12.

[0068] Block 208 of method 200 includes identifying matching pixels of different transmitted images 42, 142 corresponding to respective positions on the object 12, reading the intensities of each matching pixel in the transmitted images 42, 142, and determining the color of each position of the object 12 based on the color to which the respective matching pixels were exposed while they were sensitive. The determined color has a color spectrum that includes the color of the light generated in block 206 or the color of the light projected onto the image sensor by the prism. As already explained, by being able to identify the entire matching image line 34, the processing required to determine the color of the entire image is reduced.

[0069] When the time delay for starting the frame scan between the image sensors is equal to 1 / (number of image sensors), it is possible to assume that the entire image line on one image sensor coincides with another image line on another image sensor. Therefore, the processing required to identify the matching pixels to determine the color of the object being imaged is reduced. It will be understood that the time delay is constant between different image sensors and over time, and the processing required for color determination is much less.

[0070] Variations to the disclosed embodiments can be understood and effected by those skilled in the art in light of the drawings, the disclosure, and the appended claims, by practicing the principles and techniques described herein. The terms “comprising,” “including,” and “having” in the claims do not exclude other elements or steps, and a singular element does not exclude a plurality. The functions of several items recited in the claims may be fulfilled by a single processor or other unit. For example, a processor and a controller may be integrated in the same unit. Merely because certain means are recited in mutually different dependent claims does not mean that these means cannot be advantageously combined. A computer program may be stored or distributed on a suitable medium, such as an optical storage medium or a solid-state medium, together with other hardware or as part of other hardware, but it may also be distributed in other forms, such as via the Internet or other wired or wireless communication systems. Any reference signs in the claims should not be construed as limiting the scope.

Claims

【Claim 1】 At least two rolling shutter image sensors, each rolling shutter image sensor comprising an array of pixels of a frame arranged in a plurality of image lines extending along a width direction and distributed in a scanning direction perpendicular to the width direction, each rolling shutter image sensor scanning its frame by sequentially scanning each image line along the scanning direction, and scanning each image line includes determining an intensity readout for each pixel of each image line as being sensitive to light for a predetermined exposure time, at least two rolling shutter image sensors; A controller configured to start the scanning of the frame from the first image line in the scanning direction for each rolling shutter image sensor with a time delay with respect to other rolling shutter image sensors by temporally shifting the scanning of the frame for each rolling shutter image sensor, wherein the time delay between the start and the start of the scanning of the frame for each rolling shutter image sensor is equal to the time required to scan a single frame divided by the number of rolling shutter image sensors, the controller; A lens system for projecting a similar transmitted image of an object onto each rolling shutter image sensor, wherein the transmitted image is projected onto each rolling shutter image sensor such that each pixel of the transmitted image corresponding to a position on the object coincides with a pixel or a group of pixels of another transmitted image corresponding to each of the positions on the object, an image sensing system comprising the lens system; The image sensing system is (i) a light generator that generates at least two different colors of light defined by different wavelengths for irradiating the object, and a timing module that controls the light generator to sequentially generate the at least two different colors of light during the scanning of each frame for each rolling shutter image sensor, such that each transmitted image is captured by each rolling shutter image sensor, or (ii)To project each transmitted image in a single color different from the colors of other transmitted images, the lens system includes a prism that splits incident light into two or more different colors of light defined by different wavelengths, so that each rolling shutter image sensor captures the transmitted images in different colors of light, The image sensing system is, identifying matching pixels of different transmitted images corresponding to respective positions on the object, and determining the color of the object by superimposing the colors from each matching pixel based on the intensity readings of each matching pixel and the color of the light to which the respective matching pixel was exposed while the pixel was sensitive. The image sensing system further comprises a processor. Image sensing system. **Claim 2** Each image line of the transmitted image on each rolling shutter image sensor coincides with an image line of another transmitted image of another rolling shutter image sensor corresponding to a position on the object, and the processor identifies the coinciding image lines and determines the color of the respective positions of the object based on the intensity readings of each pixel in the coinciding image lines and the color of the light to which the respective pixel was exposed while the pixel was sensitive. The image sensing system according to claim 1. **Claim 3** The matching pixels or the matching image lines are predetermined based on the configurations of the rolling shutter image sensor and the lens system. The image sensing system according to claim 1. **Claim 4** The frequency of switching between the colors of light during the scanning of a single frame is equal to the number of image sensors or a multiple thereof. The image sensing system according to any one of claims 1 to 3. **Claim 5** The number of colors of light generated by the light generator is equal to the number of rolling shutter image sensors or a multiple thereof. The image sensing system according to any one of claims 1 to 3. **Claim 6** At least two rolling shutter image sensors, each rolling shutter image sensor comprising an array of pixels of a frame arranged in a plurality of image lines extending along a width direction and distributed in a scanning direction perpendicular to the width direction, each rolling shutter image sensor scanning its frame by sequentially scanning each image line along the scanning direction, and scanning each image line comprising determining an intensity readout for each pixel of each image line as sensitive to light for a predetermined exposure time, a method for determining a color of an object using the at least two rolling shutter image sensors, The method comprises: Scanning the frames of each rolling shutter image sensor in a staggered manner such that the start of scanning of the frame for each rolling shutter image sensor is scanned with a time delay with respect to the other rolling shutter image sensors, the time delay between the starts of scanning of the frame for each rolling shutter image sensor being equal to the time required to scan a single frame divided by the number of rolling shutter image sensors, Projecting similar transmitted images onto each rolling shutter image sensor such that each pixel of the transmitted image corresponding to a position on the object coincides with a pixel or group of pixels of another transmitted image corresponding to the respective position on the object, The method further comprises: (i) Splitting incident light into two or more different colors of light defined by different wavelengths in order to project each transmitted image with a single color different from the other transmitted images, whereby each rolling shutter image sensor captures transmitted images having different colors of light, the transmitted images being captured by each rolling shutter image sensor with different colors of light defined by different wavelengths, or (ii) Generating sequentially at least two different colors of light defined by different wavelengths for irradiating the object during scanning of each frame of the rolling shutter image sensor such that each transmitted image is captured by each rolling shutter image sensor, The method further comprises: Identifying matching pixels of different transmitted images corresponding to respective positions on the object, and determining the color of each position of the object by superimposing colors from each matching pixel based on the intensity reading of each matching pixel in the transmitted image and the color of the light to which each of the matching pixels was exposed while the pixel was sensitive. Method. **Claim 7** Each image line of the transmitted image on each rolling shutter image sensor coincides with an image line of a transmitted image of another rolling shutter image sensor corresponding to a position on the object. The method according to claim 6, comprising identifying matching image lines and determining the color of each position of the object based on the intensity reading of each pixel in the matching image lines and the color of the light to which each of the pixels was exposed while the pixel was sensitive. **Claim 8** The method according to claim 6 or 7, wherein the at least two different colors of light are sequentially generated at a frequency equal to the number or a multiple thereof of the image sensors during the scanning of a single frame. **Claim 9** The method according to claim 6 or 7, wherein the number of colors of light generated is equal to the number or a multiple thereof of the rolling shutter image sensors.

Images