Image perception system

Abstract

According to one aspect, an image sensing system is provided that includes a rolling shutter image sensor including an array of pixels in a frame distributed in a scan direction orthogonal to a width direction and arranged into a plurality of image lines extending along the width direction, the rolling shutter image sensor configured to scan the frame by sequentially scanning each image line along the scan direction, wherein scanning each image line includes exposing pixels of each image line to light for a predetermined exposure time and determining an intensity readout of each pixel of the respective image line; and a lens system configured to project at least two similar output images of an object onto the image sensor, the output images being offset from one another in the scan direction. the output image is projected onto an image sensor such that each pixel of the output image corresponding to a location on the object coincides with a pixel or group of pixels of another output image corresponding to a distinct location on the object; a light generator configured to generate light of at least two different colors defined by different wavelengths to illuminate the object; a timing module configured to control the light generator to sequentially generate light of the at least two different colors during the scanning of one frame; and a processor that identifies matched pixels of the different output images corresponding to distinct locations on the object and resolves the color spectrum of the distinct locations on the object based on an intensity reading of each matched pixel and the color of light to which the distinct matched pixel was exposed while being viewed.

Description

Technical Field

[0001] This application relates to an image sensing system for determining the color of an object and a method for determining the color of an object.

Background Art

[0002] A multispectral image relates to creating an image having more spectral resolution (i.e., colors) than the three colors (red, green, blue) distinguishable by the human eye. Generally, creating an image with color involves the use of an image sensor capable of sensing multiple colors.

Summary of the Invention

Problems to be Solved by the Invention

[0003] However, color sensors are more expensive than black-and-white image sensors, and sensors capable of sensing three or more colors are even more expensive and difficult to manufacture.

[0004] US2008 / 0177185 discloses a skin area detection imaging device for detecting a skin area of a human body as an object, which includes two optical lenses that form two unit images on an imaging element by collecting light from the object illuminated with near-infrared light, a rolling shutter that sequentially reads out the unit images, and two LEDs that emit light of different wavelengths (850 mn and 940 nm) in the near-infrared range. When reading out the two unit images, a microprocessor switches the two LEDs respectively. Based on the difference in reflectance for different wavelengths of near-infrared light, the skin area of one read unit image is displayed with a different brightness from the other read unit image. The microprocessor compares the two unit images and determines an area where the luminance difference is equal to or greater than a predetermined value as the skin area. This enables the detection of the skin area in a short time.

Means for Solving the Problems

[0005] According to a first aspect, there is provided an image sensing system, which is a rolling shutter image sensor including an array of pixels in a frame arranged into a plurality of image lines distributed in a scanning direction orthogonal to a width direction and extending along the width direction, the rolling shutter image sensor being configured to scan the frame by sequentially scanning each image line along the scanning direction, and scanning each image line including exposing the pixels of each image line to light for a predetermined exposure time and determining an intensity readout value for each pixel of an individual image line; a lens system configured to project at least two similar outgoing images of an object onto the image sensor, the outgoing images being offset from each other in the scanning direction, and the outgoing images being projected onto the image sensor such that each pixel of an outgoing image corresponding to a position on the object coincides with a pixel or group of pixels of another outgoing image corresponding to an individual position on the object; a light generator configured to generate at least two different colors of light defined by different wavelengths for irradiating the object; a timing module configured to control the light generator to sequentially generate at least two different colors of light during the scanning of one frame; and a processor configured to identify coincident pixels of different outgoing images corresponding to individual positions on the object and resolve a color spectrum of an individual position on the object based on the intensity readout value of each coincident pixel and the color of the light exposed while the individual coincident pixel is being sensitized.

[0006] The at least two outgoing images can be projected onto the image sensor with the same size.

[0007] The coincident pixels may be predetermined based on the lens system or may be calculated in real time by processing the outgoing images and comparing pixels from one outgoing image with pixels of another outgoing image to determine the coincident pixels.

[0008] The lens system can be configured to project an array of output images having at least two different columns of output images onto the image sensor. The columns can be distributed along the width direction, and each column has at least two output images distributed in the scanning direction. Here, each output image can be offset in the scanning direction from all other output images.

[0009] The output images of a certain column can be split across the frame such that the upper part of a certain output image is projected onto the lower part of the image sensor, and the remaining lower part of another output image is projected onto the upper part of the image sensor in the same column. Here, the upper and lower parts together define one overall output image.

[0010] The exposure time of each image line can be made equal to the time taken to read out the plurality of image lines spanned by each output image.

[0011] The timing module can control the light generator to generate lights of different colors at intervals, so that the light generator can be controlled to generate at least two different colors of light during the intensity reading of the image lines of at least one output image within the frame.

[0012] The light generator can be configured to periodically and sequentially generate lights of different colors, and each period can be generated within the frame and have the ability to generate lights of different colors in a predetermined sequence. Each subsequent period can have the same sequence as the previous period, but starts with the second color of the light in the previous period and ends with the first color of the light in the previous period.

[0013] The timing module can control the light generator to generate light at intervals, so that during the scanning of each output image, there are the same number of intervals as the number of columns of the output images projected onto the image sensor.

[0014] The light generator may be configured to generate the same number of colors as the number of intervals of light generation within the frame. The intervals may be at regular intervals.

[0015] The light generator may be configured to generate a flash or to generate continuous light emission.

[0016] The processor may be configured to resolve the color spectrum of an individual position of interest based on the intensity readout values of an individual pixel and at least one preceding matching pixel for each readout of an individual pixel.

[0017] According to a second aspect, a method of resolving the color of an object is provided using a rolling shutter image sensor including an array of pixels in a frame that are distributed in a scanning direction orthogonal to a width direction and arranged into a plurality of image lines extending along the width direction, the rolling shutter image sensor being configured to scan the frame by sequentially scanning each image line along the scanning direction, scanning each image line including exposing the pixels of each image line to light for a predetermined exposure time and determining a readout value of the intensity of each pixel of an individual image line. The method includes projecting at least two similar projected images offset from each other along the scanning direction onto the image sensor, each pixel of the projected image corresponding to a position in the object being matched with a pixel or group of pixels of another projected image corresponding to an individual position in the object; sequentially generating at least two colors of light defined by different wavelengths to irradiate the object during scanning of the frame of the image sensor; identifying matching pixels of different projected images corresponding to an individual position of the object; and resolving the color spectrum of the individual position of the object based on the intensity readout of each matching pixel of the projected image and the color of the light exposed while the individual matching pixel is being sensitized.

[0018] This method can be a computer-implemented method.

[0019] The method can include projecting an array of emitted images onto an image sensor in at least two different columns of emitted images. The columns can be distributed along the width direction, and each column has at least two emitted images distributed in the scanning direction. Here, each of the emitted images can be offset from all other emitted images in the scanning direction.

[0020] The exposure time can be equal to the time taken to read out a plurality of image lines spanned by each emitted image.

[0021] Lights of different colors can be generated at intervals, so that at least two lights of different colors can be generated when reading the intensity of the image lines of at least one emitted image in the frame.

[0022] Lights of different colors can also be generated periodically and sequentially. Each period can be generated within a frame and can have generating lights of different colors in a predetermined sequence. Each subsequent period can have the same sequence as the preceding period, but starts with the second color of the light in the preceding period and ends with the first color of the light in the preceding period.

[0023] Lights of different colors can be generated at intervals, and during the scanning of each emitted image, there are the same number of intervals as the number of columns of the emitted images projected onto the image sensor.

[0024] Generating light can include generating a flash or generating continuous light.

[0025] The projection of at least two emitted images on a rolling shutter image sensor may have no overlapping parts (i.e., different emitted images may not occupy the same space on the image sensor).

[0026] These and other aspects will become apparent from the embodiments described below and will be described with reference to the embodiments.

Brief Description of the Drawings

[0027]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5a

Figure 5b

Figure 6

Best Mode for Carrying Out the Invention

[0028] Hereinafter, exemplary embodiments will be described with reference to the following drawings, which are merely exemplary.

[0029] FIG. 1 shows an image sensing system 10 used for sensing an image of an object 12.

[0030] The image sensing system 10 has a rolling shutter image sensor 14 and a lens system 16 configured to project three similar projected images of the object 12 onto the image sensor 14. In some examples, the lens system can be configured to project two or more similar projected images of the object onto the image sensor.

[0031] The lens system 16 of this embodiment has three lenses 24 arranged adjacent to each other in front of the image sensor 14. The lens system 16 is thus arranged between the imaging object 12 and the image sensor 14.

[0032] In other examples, the lens system can have a lens and one or more beam splitters. The beam splitter is disposed between the lens and the image sensor, such that the beam splitter is configured to split the beam from the lens and project two or more identical images onto the image sensor.

[0033] By projecting two or more output images onto a single image sensor 14, although the resolution of each image decreases, the frame rate of the image sensor 14 can be effectively increased. In this example, since three output images are projected onto the image sensor, the frame rate is substantially tripled compared to the normal use of the image sensor. When two images are projected onto a single image sensor, the frame rate of the image sensor is substantially doubled.

[0034] The image sensing system 10 includes, in this example, a light generator 18 configured to generate light of three different colors to illuminate the object 12. In this example, the light generator 18 has three lights 20 configured to generate flashes of different wavelengths (i.e., light of different colors). In other examples, the light generator may be configured to generate light of two different colors with two lights, or may generate light of three or more different colors. In a further example, the lights may be configured to generate continuous light back-to-back, as opposed to flashes. It should be understood that generating light of different colors can also be achieved with only one light using a plurality of different color filters.

[0035] FIG. 2 shows a rolling shutter image sensor 14 including an array of pixels 30 within a frame 32 that are distributed in a scanning direction 40 orthogonal to the width direction 36 and arranged into image lines 34 extending along the width direction 36.

[0036] In the example of FIG. 2, the lens system 16 is configured to project three exit images 42, namely a first exit image 42a, a second exit image 42b, and a third exit image 42c, onto the image sensor 14. The exit images 42 are each offset from one another in the scanning direction 40, such that as a result, they do not overlap in the scanning direction 40. In other words, the three exit images 42 are arranged in a row in the scanning direction 40, from the first exit image 42a to the third exit image 42c. The exit images 42 in this embodiment are projected onto the image sensor 14 with the same size. As a result, each pixel 30 of one exit image 42 corresponding to a position on the object 12 coincides with the pixel 30 of another exit image 42 corresponding to the same position on the object 12. It should be understood that the exit images do not necessarily have to be of the same size, and that the pixels 30 of one exit image corresponding to a position on the object may coincide with a group of pixels 30 of another exit image corresponding to the same position on the object.

[0037] For example, in FIG. 2, the first exit image 42a has a pixel A, which coincides with pixel B of the second exit image 42b, which coincides with pixel C of the third exit image 42c. Pixels A, B, and C correspond to the same position on the object 12 being imaged.

[0038] The rolling shutter image sensor 14 is configured to scan the entire frame 32 of pixels 30 by sequentially scanning each image line 34 along the scanning direction 40. In other words, in the scanning direction 40, it is configured to scan the adjacent image line 34 following the image line 34.

[0039] The scanning of each image line 34 includes sequentially scanning each pixel 30 of the image line 34 along the width direction 36. In other words, the image sensor 14 scans the adjacent pixel within the image line 34 following the pixel 30 within the image line 34 in the width direction 36.

[0040] Scanning each image line 34 includes, as will be described in more detail with reference to FIG. 3, exposing the pixels 30 of the individual image line 34 to light for a predetermined exposure time, and then reading out each pixel 30.

[0041] Figure 3 shows an image sensor 14 having a plurality of image lines 34, and a first emitted image 42a, a second emitted image 42b, and a third emitted image 42c projected onto the image sensor 14. A scanning time map 50 is shown adjacent to the image sensor 14. The scanning time map 50 indicates the passage of time from left to right. For each image line 34, the scanning time map 50 indicates an intensity readout value 54 following an exposure time 52. In other words, when the image line 34 is exposed for a predetermined exposure time 52, the image sensor 14 is configured to read out the pixels 30 of the image line 34. The readout of the pixels 30 involves determining the intensity of each pixel 30 of an individual image line 34 corresponding to the intensity of the light received by the individual pixels 30 during the exposure time 52.

[0042] The start of the exposure time 52 for each image line 34 in the scanning direction 40 is offset by a predetermined time amount from the preceding image line 34 corresponding to the time required for the readout 54 of the image line 34. Thus, each image line 34 is exposed for the same amount of time, but the exposure starts at different times for each image line 34.

[0043] While the flash 56 illuminates the object 12, only the image lines 34 spanned by one emitted image 42 are sensitive. This is possible because the sensitivity of each image line 34 is arranged in a staggered pattern in the scanning direction 40. As such, the exposure time 52 for each image line 34 is equal to the time taken to read out the plurality of image lines 34 spanned by each emitted image 42. This means that at any given time, the number of image lines 34 sensitive to light is equal to the number of image lines 34 spanned by each emitted image 42.

[0044] Thus, each flash of light 56 affects only the intensity readout 54 of the image lines 34 associated with a single emitted image 42. In this example, there are three flashes 56, and one flash 56 corresponds to each emitted image 42 projected onto the image sensor 14.

[0045] Returning to FIGS. 1 and 2 and referring to these, the image sensing system 10 has a timing module 26 configured to control the light generator 18 to generate lights of different colors at appropriate intervals, that is to say, to control different lights 20 of the light generator 18 to blink at the correct time.

[0046] More specifically, in this example, the timing module 26 is configured to control the light generator 18 to generate a flash of one color of light 56a-56c at a time when all of the image lines 34 of one outgoing image 42 have been exposed and the image lines 34 of another outgoing image 42 have not been exposed. The timing module 26 is configured to sequentially generate another flash of another color 56a-c at a time when all of the image lines 34 of another outgoing image 42 have been exposed and the image lines 34 of the other outgoing image 42 have not been exposed.

[0047] In this example, as shown in FIGS. 2 and 3, during the exposure time 52 of the first four image lines 34 of the scanning direction 40 spanned by the first outgoing image 42a, the timing module 26 controls the light generator 18 to generate a flash of red light 56a.

[0048] During the exposure time of the next four image lines 34 of the scanning direction 40 spanned by the second outgoing image 42b, the timing module 26 controls the light generator 18 to generate a flash of green light 56b.

[0049] During the exposure time of the last four image lines 34 of the scanning direction 40 spanned by the third outgoing image 42c, the timing module 26 controls the light generator 18 to generate a flash of blue light 56c. This is repeated for each scan of the frame 32.

[0050] In this example, red, green, and blue colors were used in this order, but in other examples, any three colors of light can be used, and the colors of the light can blink in any order during the scanning of the frame.

[0051] The image sensing system 10 further includes a processor 28 configured to identify matching pixels 30 of different emitted images 42, each corresponding to the same position on the object 12, such as pixels A, B, and C shown in FIG. 2.

[0052] The processor 28 is configured to resolve the color spectrum of individual positions on the object 12 based on the intensity readings of the matching pixels A, B, and C and based on the color of the light to which the object 12 was exposed when the individual pixels A, B, and C were sensitized, even if the image sensor 14 used is a monochrome image sensor.

[0053] For example, during the exposure (i.e., sensitization) of pixel A, the object 12 is illuminated by a flash of red light 56a. During the exposure of pixel B, the object 12 is illuminated by a flash of green light 56b. During the exposure of pixel C, the object 12 is illuminated by a flash of blue light 56c. Accordingly, the intensity reading value of pixel A corresponds to the amount of red light reflected at the individual position on the object 12, the reading value of pixel B corresponds to the amount of green light reflected at the individual position on the object 12, and the intensity reading value of pixel C corresponds to the amount of blue light reflected at the individual position on the object 12.

[0054] In this example, the color Xc of the position on the object 12 corresponding to pixels A, B, and C can be determined by superimposing the intensity reading values of the three matching pixels A, B, and C. TIFF0007690587000001.tif14122

[0055] Accordingly, the processor 28 can resolve the color Xc of each position on the object 12 corresponding to the matching pixels A, B, and C with Equation 1. The resolved color Xc will have a spectral resolution that includes the wavelengths of the three colors of light generated by the light generator 18 during the scanning of the frame 32.

[0056] It should be understood that this superposition of intensity readout values can be used with any two or more colors of light, and that the spectral resolution of the resolved color Xc will include the wavelengths of the colors of light used to illuminate the object 12 during the scan of frame 32.

[0057] The superposition of intensity readout values can be achieved with different colors of light that blink up to the same number as the number of output images 42 projected onto the image sensor 14, using any number of output images 42 projected onto the image sensor 14. The greater the number of colors (i.e., wavelengths) of light used to illuminate the object 12 for the coincident pixels 30 during the scan of frame 32, the higher the spectral resolution of the resolved color Xc.

[0058] Sometimes, using fewer colors to illuminate the object 12 for the coincident pixels 30 during the scan of frame 32 can result in a more accurate determination of the resolved color Xc. For example, in a group of coincident pixels 30 such as pixels A, B, and C, if there are two or more pixels 30 that sense light while illuminating the object 12 with light of the same color, the intensity readout values for these pixels 30 can be used to provide an error metric that the processor 28 can use to more accurately resolve the color Xc.

[0059] For example, if during the exposure (i.e., light sensing) of pixel A, the object 12 is illuminated with a flash of red light, during the exposure of pixel B, the object 12 is illuminated with a flash of green light, and during the exposure of pixel C, the object 12 is illuminated with a flash of another red light, the spectral resolution will only include the wavelengths of red and green light, but the intensity readout values of pixels A and C should be the same. If they are not the same, there is an error in the readout value, which can be mitigated, for example, by using the average intensity of pixels A and C as in Equation 2 below and superimposing it with the intensity readout value of pixel B. TIFF0007690587000002.tif15125

[0060] FIG. 4 shows a roller shutter image sensor 14 onto which an exit image 42 is projected with a configuration different from FIGS. 2 and 3, which more preferably utilizes the entire frame 32 of the image sensor 14.

[0061] In this example, the lens system 16 is configured to project an array of exit images 42 onto the image sensor 14, in this example nine images. The nine exit images 42 are arranged into three columns 60 that are distributed along the width direction 36. Each column 60 has a total of three exit images 42 that are distributed in the scanning direction 40. In other examples, there may be two or more columns of exit images, and two or more exit images may be present in each column. The projection of the exit images 42 onto the image sensor 14 has no overlapping portions (i.e., different exit images do not occupy the same space on the image sensor 14).

[0062] Each column 60 of the exit images 42 is offset in the scanning direction 40 from an adjacent column 60 of the exit images 42. As a result, in some columns, the exit images 42 are split across the frame 32 such that a portion of the exit image 42 is projected onto one end (e.g., the upper part) of the column 60. The remaining portion of the exit image 42 is projected onto the other end (e.g., the lower part) of the column 60. As a result, two portions of one column define one entire exit image.

[0063] In this example, in the first column 60a in the width direction 36, each exit image 42 is whole. In the second column 60b in the width direction 36, the exit image 42 is offset by one third of the exit image 42 in the scanning direction 40 compared to the first column 60a. In other words, one third of a portion of the exit image 42 is projected onto the upper part of the second column 60b, and the remaining two thirds of the individual exit image 42 is projected onto the lower part of the second column 60b. In the third column 60c in the width direction 36, the exit image is offset by one third of the exit image 42 in the scanning direction 40 compared to the second column 60b. Thus, the first column 60a is also offset by one third of one exit image 42 compared to the third column 60c.

[0064] This effectively increases the frame rate of the image sensor 14 by a factor of nine. The reading order of each overall output image 42 is the first output image in the scanning direction 40 of the first column 60a, the first overall output image 42 of the second column 60b, the first overall output image 42 of the third column 60c, and so on, followed by the second output image 42 of each column 60, and so on.

[0065] The timing module 26 controls the light generator 18 to generate light of different colors at regular intervals. In this example, the intervals are spaced such that three flashes 56 are generated during the exposure of each image line 34. In other words, during the time required to read out all of the image lines 34 of one output image 42, the timing module 26 controls the light generator 18 to generate three flashes 56. In this example, the number of intervals of the flashes 56 during the exposure of each image line 34 corresponds to the number of columns 60 of the output image 42.

[0066] In this example, the colors of the light generated by the light generator 18 are three colors: red, green, and blue (R, G, B). In this example, each flash 56 is a different color from the preceding flash 56. It should be understood that in other examples, there may be two consecutive flashes of light of the same color.

[0067] The light generator 18 of the present embodiment is configured to periodically generate light of different colors. Here, each period has three flashes 56 of light of different colors in a predetermined sequence. Each period has the same sequence as the preceding period, but starts with the second color of the light of the preceding period and ends with the first color of the light of the preceding period.

[0068] Therefore, in this example, the first period includes the generation of light in the sequence RGB, the second period generates light in the sequence GBR, and the third period generates light in the sequence BRG. Each period is generated within one frame, and in this example, a plurality of periods are generated within one frame. In other examples, any other suitable sequence can be used, and the change between subsequent periods can be any suitable change.

[0069] Processor 28 identifies the matching pixels corresponding to one position in object 12 in the nine emitted images 42, which are represented by pixels A2 - I2 in this example. Each of the pixels A2 - I2 is sensitive between three different flashes 56.

[0070] For each pixel, the intensity readout value corresponds to the amount of light reflected from the colored flash 56 by the object 12. This can be expressed by Equation 3 below. TIFF0007690587000003.tif69131

[0071] Equations A2, D2, and G2 are the same, which leaves seven equations to solve for the three unknowns R, G, and B. Processor 28 solves these equations to determine the unknown variables, and thereby can solve for the color Xc of the position in object 12 using Equation 1. Only three of the three equations of Equation 3 are needed to solve for the three unknowns, but if all the information is used, this can reduce noise. The least - squares error fit can be used to most accurately solve for R, G, and B.

[0072] This can be repeated for each pixel 30, and even for a monochrome image sensor, the color Xc of each position of the imaging object 12 can be solved. The spectral resolution of the solved color Xc will be limited by the wavelength of the color of the flashing light. Therefore, the more colors of the flashing light there are, the higher the spectral resolution of the solved color Xc will be. There can be as many colors of the flashing light as the number of emitted images 42. Therefore, if there are nine emitted images 42, there can be nine different colors of the flashing light, which gives nine different equations regarding the matching pixels to solve for the intensities of the nine different colors.

[0073] Although it has been described that the emitted image projected onto the image sensor has three columns, it should be understood that any suitable number n of columns of the image distributed along the width direction may exist, and the columns may be offset by 1 / n of the emitted image from the adjacent columns. This ensures that each emitted image 42 projected onto the image sensor is offset from each other emitted image 42, so that two emitted images 42 are not exposed to light at the same timing.

[0074] Therefore, the resolved color Xc of the object 12 can be determined after reading out each frame 32 using the matching pixels that are consistent throughout the frame 32. As a result, the color resolution has a frame rate equal to the frame rate of the image sensor 14. However, the resolved color Xc of the object 12 can also be determined after reading out each matching pixel 30 if the reading of the previous eight matching pixels is used on a rolling basis. For example, when the intensity of pixel E2 is read out, the color Xc of the object 12 at the position corresponding to pixel E2 can be resolved using the intensity readout value of pixel E2 together with the intensity readout values of pixels A2 to D2 in the same frame 32 and the intensity readout values of pixels F2 to I2 in the previous frame 32. This increases the effective frame rate of the color resolution by the number of emitted images 42 projected onto the image sensor 14, in this case nine times.

[0075] In an example where the light is configured to generate continuous light (i.e., continuous illumination) instead of a flash, the equation for the intensity of each pixel is slightly different. For example, in the case of pixel Z, the intensity of the light received while the pixel is sensitive is based on the ratio of the time the pixel is exposed to the light of each color while the pixel is sensitive, for example TIFF0007690587000004.tif23126.

[0076] Similar to the example described above, on the condition that each matching pixel has a corresponding equation and there are as many different equations for the matching pixels as the number of different light colors, the processor can resolve the color at the position of the object corresponding to the matching pixel with these equations.

[0077] Having a plurality of emitted images 42 projected onto the image frame and generating light at a plurality of intervals during the scanning of one frame 32 means that the frequency of light change is higher than, for example, when only color light generation at one interval is present during the scanning of the frame. Such a high-frequency change in light color means that the change in light becomes difficult for the user to see and ultimately is recognized by the user only as continuous light rather than rapidly flashing light.

[0078] Figures 5a and 5b respectively show a third example and a fourth example of the projection of the emitted image 42 onto the image sensor 14.

[0079] In Figure 5a, there are three emitted images 42 projected onto column 160 in the same manner as the first column 60a in Figure 4, and there is one large emitted image 142 projected onto the remaining space of the image sensor 14. Such an arrangement is useful when it is desirable to have a high frame rate with low spatial resolution for certain features and a high spatial resolution with a low frame rate for other features. Thus, the larger emitted image 142 provides a higher spatial resolution than the column 160 of the emitted images 42, but in order to determine the color of each pixel, the frame must be scanned several times. It must be ensured that each pixel of the larger emitted image 142 is exposed to light of different colors in each subsequent scan of the frame 32.

[0080] In Figure 5b, there are two emitted images 242 projected onto the image sensor 14 at column 260, and the projected images are stretched in the width direction 36 so as to cover the image sensor 14 as much as possible. This increases the horizontal resolution of the emitted images 242.

[0081] FIG. 6 is a flowchart showing a method 300 for resolving the color of an object 12 with a rolling shutter image sensor 14. Method 300 begins at block 302 and projects at least two emitted images 42, 142, 242 offset from each other along a scanning direction onto the image sensor 14, as shown in FIGS. 2-5. In the emitted images 42, 142, 242, each pixel 30 of the emitted images 42, 142, 242 coincides with another pixel 30 or pixel group of another emitted image 42, 142, 242 and corresponds to a position in the object 12.

[0082] In block 304, method 300 includes sequentially generating at least two different colors of light having different wavelengths to illuminate the object 12. The generation of the light can be controlled to be generated at intervals, for example, as described with reference to FIGS. 2, 3, and 4.

[0083] In block 306, method 300 includes scanning the pixels 30 of the image sensor 14 to obtain intensity readout values for each pixel 30. In this example, the exposure time of each image line 34 of the image sensor 14 is equal to the time required to read out a plurality of image lines 34 spanned by each emitted image 42.

[0084] In block 308, method 300 includes identifying the coincident pixels of different emitted images 42, 142, 242 corresponding to positions in the object 12, and block 310 includes resolving the color spectrum Xc of the positions on the object 12 based on the intensity readout values of each coincident pixel 30 and the color of the light exposed while the individual pixel 30 is photosensitive.

[0085] It should be understood that all of blocks 302-308 can be executed simultaneously.

[0086] Although the timing module has been described as controlling the light generator to generate different colors of light at regular intervals, the intervals do not have to be regular. This is because the processor can similarly resolve the colors at irregular intervals of light generation.

[0087] Furthermore, it is described that the processor identifies matching pixels of different emitted images. The matching pixels may be predetermined based on the lens system or may be calculated in real time by processing the emitted images and comparing pixels from one emitted image with pixels of another emitted image. It will be apparent that the accuracy of color resolution may be affected by the movement of the object during scanning. The less movement during scanning, the better the color resolution of each point of the object. Using a processor that compares pixels in real time and determines the matching pixels can enable some compensation for movement during scanning.

[0088] Modifications to the disclosed embodiments can be understood and implemented by those skilled in the art who practice the principles and techniques described herein upon review of the figures, the disclosure, and the appended claims. In the claims, the term "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. One processor or other unit can perform the functions of a plurality of items recited in the claims. The mere fact that certain means are recited in mutually different dependent claims does not indicate that a combination of these means cannot be used advantageously. A computer program can be stored or distributed in a suitable medium, such as an optical storage medium or a solid-state medium, supplied together with or as part of other hardware, but can also be distributed in other forms, such as via the Internet or other wired or wireless communication systems. Any reference signs in the claims should not be construed as limiting the scope of the invention.

Claims

Claim 1 An image sensing system, A rolling shutter image sensor including an array of pixels within a frame arranged into a plurality of image lines distributed in a scanning direction orthogonal to a width direction and extending along the width direction, wherein the rolling shutter image sensor scans the frame by sequentially scanning each image line along the scanning direction, and scanning each image line includes exposing the pixels of each image line to light for a predetermined exposure time and determining a read value of the intensity of each pixel of an individual image line, the rolling shutter image sensor; A lens system that projects an array of similar outgoing images of an object onto the image sensor, the array including at least two different columns of outgoing images, the columns being distributed along the width direction, each column including at least two outgoing images distributed in the scanning direction, each of the outgoing images being offset from all other outgoing images in the scanning direction, and the outgoing images being projected onto the image sensor such that each pixel of an outgoing image corresponding to a position in the object coincides with a pixel or group of pixels of another outgoing image corresponding to an individual position in the object, the lens system; A light generator that generates at least two different colors of light defined at different wavelengths for illuminating the object; A timing module that controls the light generator to sequentially generate at least two different colors of light during the scanning of one frame; An image sensing system having a processor that identifies matching pixels of different outgoing images corresponding to individual positions in the object and resolves the color of an individual position of the object based on the intensity read value of each matching pixel and the color of the light exposed while the individual matching pixel is being sensitized. Claim 2 The image sensing system according to claim 1, wherein the exposure time of each image line is equal to the time taken to read a plurality of image lines spanned by each outgoing image. Claim 3 The image sensing system according to claim 1 or 2, wherein the timing module controls the light generator to generate different colors of light at intervals, and the light generator is controlled to generate at least two different colors of light during the intensity reading of the image lines of at least one of the outgoing images within the frame. Claim 4 The image sensing system according to claim 3, wherein the light generator periodically and sequentially generates lights of different colors, each period being generated within a frame and including generating lights of different colors in a predetermined sequence, and each subsequent period having the same sequence as the preceding period, but starting with the second color of the light of the preceding period and ending with the first color of the light of the preceding period.

5. The image sensing system according to any one of claims 1 to 4, wherein the timing module controls the light generator to generate light at intervals, and during the scanning of each emitted image, there are the same number of intervals as the number of columns of the emitted image projected onto the image sensor.

6. The image sensing system according to any one of claims 1 to 5, wherein the light generator generates a flash or continuous light.

7. The image sensing system according to any one of claims 1 to 6, wherein the processor resolves the color spectrum of the individual position of the object based on the intensity readout value of an individual pixel and the intensity readout values of at least one preceding matching pixel for the readout of each pixel.

8. In a method of resolving the color of an object using a rolling shutter image sensor including an array of pixels within a frame that are distributed in a scanning direction orthogonal to the width direction and arranged into a plurality of image lines extending in the width direction, the rolling shutter image sensor scans the frame by sequentially scanning each image line along the scanning direction, and scanning each image line includes exposing the pixels of each image line to light for a predetermined exposure time and determining a readout value of the intensity of each pixel of an individual image line, the method comprising: projecting an array of similar emitted images onto the image sensor with at least two different columns of emitted images, the columns being distributed along the width direction, each column including at least two emitted images distributed in the scanning direction, the emitted images being offset from all other emitted images along the scanning direction, and each pixel of the emitted image corresponding to a position in the object matching a pixel or group of pixels of another emitted image corresponding to an individual position in the object; sequentially generating at least two different colors of light defined by different wavelengths to illuminate the object during the scanning of the frame of the image sensor. identifying matching pixels of different emission images corresponding to individual positions of the object, and resolving a color spectrum of the individual positions of the object based on intensity readout values of each of the matching pixels of the emission images and colors of light exposed while each of the matching pixels is being sensitized, a method.

9. The method according to claim 8, wherein the exposure time is equal to a time taken to read out a plurality of image lines across each emission image.

10. The method according to claim 8 or 9, wherein lights of different colors are generated at intervals, and at least two lights of different colors are generated during intensity readout of an image line of at least one emission image within a frame.

11. The method according to claim 10, wherein lights of different colors are sequentially generated periodically, each period being generated within a frame and including generating lights of different colors in a predetermined sequence, each subsequent period having the same sequence as the preceding period, but starting with a second color of the light of the preceding period and ending with a first color of the light of the preceding period.

12. The method according to claim 11, wherein lights of different colors are generated at intervals, and there are the same number of intervals as the number of columns of the emission image projected onto the image sensor during scanning of each emission image.

13. The method according to any one of claims 8 to 12, wherein the step of generating the light includes generating a flash or generating continuous light.

Images