Image sensor and method of operating the same

Abstract

Disclosed herein is a method of using an image sensor comprising N sensing areas for taking an image of a scene, the N sensing areas being physically separated from each other, the method comprising: for i = 1,..., P, and j = 1,..., Q(i), placing the image sensor at a position (i, j) and taking a partial image (i, j) of the scene using the image sensor while the image sensor is at the position (i, j), thereby taking a total of R partial images, where R is the sum of Q(i), i = 1,..., P, where P > 1, where Q(i), i = 1,..., P, are positive integers and not all of 1, where for i = 1,..., P, a position group (i) comprises the positions (i, j), j = 1,..., Q(i), and where the minimum distance between two positions in two different position groups is substantially larger than the maximum distance between two positions in the same position group; determining a combined image.

Description

TECHNICAL FIELD

[0001] The disclosure herein relates to an image sensor and a method of operating the same. BACKGROUND

[0002] A radiation detector is a device that measures a characteristic of radiation. Examples of the characteristic can include a spatial distribution of intensity, phase, and polarization of the radiation. The radiation can be radiation that has interacted with an object. For example, the radiation measured by the radiation detector can be radiation that has penetrated from or reflected from the object. The radiation can be electromagnetic radiation such as infrared light, visible light, ultraviolet light, X-rays, or gamma rays. The radiation can be other types such as alpha rays and beta rays. An image sensor can include a plurality of radiation detectors. Radiation can include radiation particles such as photons (electromagnetic waves) and subatomic particles. SUMMARY

[0003] Disclosed herein is a method of using an image sensor, the image sensor comprising N sensing areas for taking an image of a scene, N being a positive integer, the N sensing areas being physically separated from each other, the method comprising: for i = 1,..., P, and j = 1,..., Q(i), placing the image sensor at a position (i, j) relative to the scene, and taking a partial image (i, j) of the scene using the image sensor while the image sensor is at the position (i, j), thereby taking a total of R partial images, where R is the sum of Q(i), i = 1,..., P, where P is an integer greater than 1, where Q(i), i = 1,..., P, is a positive integer and not all of 1, where for i = 1,..., P, a position group (i) comprises the positions (i, j), j = 1,..., Q(i), and where a minimum distance between a position in one of the position groups (i), i = 1,..., P, and a position in another one of the position groups (i), i = 1,..., P, is substantially greater than a maximum distance between two positions in the same one of the position groups (i), i = 1,..., P; and determining a combined image of the scene based on the R partial images.

[0004] According to an embodiment, N is greater than 1.

[0005] According to an embodiment, the placing of the image sensor at the positions (i, j) for i = 1,..., P, and j = 1,..., Q(i) is performed one by one.

[0006] According to an embodiment, Q(i), i = 1,..., P, is the same and greater than 1.

[0007] According to an embodiment, the minimum distance is close to and smaller than a size of a sensing region of the N sensing regions.

[0008] According to an embodiment, the minimum distance is greater than 100 times the maximum distance.

[0009] According to an embodiment, the maximum distance is smaller than 10 times a size of a sensing element of the N sensing regions.

[0010] According to an embodiment, the placing the image sensor at the positions (i, j) for i = 1,..., P, and j = 1,..., Q(i) comprises moving the image sensor directly from one position of one of the groups of positions (i), i = 1,..., P, to another position of another of the groups of positions (i), i = 1,..., P, and does not comprise moving the image sensor directly from one position of one of the groups of positions (i), i = 1,..., P, to another position of the same group of positions.

[0011] According to an embodiment, the determining the combined image comprises stitching the partial images (i, 1), i = 1,..., P, to form a stitched image of the scene.

[0012] According to an embodiment, the determining the combined image further comprises determining, for i = 1,..., P, an enhanced partial image (i) based on the partial images (i, j), j = 1,..., Q(i).

[0013] According to an embodiment, the determining the combined image further comprises using, for i = 1,..., P, the enhanced partial image (i) instead of the partial image (i, 1) of the stitched image.

[0014] According to an embodiment, the determining the combined image further comprises equalizing resolutions of different regions of the stitched image after performing the using.

[0015] According to an embodiment, the determining the combined image further comprises using, for i = 1,..., P, the enhanced partial image (i) instead of the partial image (i, 1) of the stitched image if a resolution of the enhanced partial image (i) is higher than a resolution of the partial image (i, 1).

[0016] According to an embodiment, the determining the enhanced partial image (i) comprises determining positions of the positions (i, j), j = 1,..., Q(i), relative to each other.

[0017] According to an embodiment, said determining said positions (i,j), j = 1,..., Q(i), relative to each other comprises using markers that are stationary relative to said scene.

[0018] According to an embodiment, said determining said positions (i,j), j = 1,..., Q(i), relative to each other comprises: up-sampling said partial images (i,j), j = 1,..., Q(i), thereby up-sampling partial images (i,j), j = 1,..., Q(i), respectively; and correlating said up-sampled partial images (i,j), j = 1,..., Q(i), to determine said positions (i,j), j = 1,..., Q(i), relative to each other.

[0019] According to an embodiment, said determining said combined image comprises, for i = 1,..., P, determining an enhanced partial image (i) based on said partial images (i,j), j = 1,..., Q(i).

[0020] According to an embodiment, said determining said combined image further comprises stitching said enhanced partial images (i), i = 1,..., P, to form a stitched image of said scene.

[0021] According to an embodiment, said determining said combined image further comprises equalizing resolutions of different regions of said stitched image. BRIEF DESCRIPTION OF DRAWINGS

[0022] Figure 1 A radiation detector according to an embodiment is schematically illustrated.

[0023] Figure 2A A simplified cross-sectional view of the radiation detector according to an embodiment is schematically illustrated.

[0024] Figure 2B A detailed cross-sectional view of the radiation detector according to an embodiment is schematically illustrated.

[0025] Figure 2C An alternative detailed cross-sectional view of the radiation detector according to an embodiment is schematically illustrated.

[0026] Figure 3 A top view of a package comprising a radiation detector and a printed circuit board (PCB) according to an embodiment is schematically illustrated.

[0027] Figure 4 A cross-sectional view of an image sensor according to an embodiment is schematically illustrated, wherein Figure 3 a plurality of said packages in Figure 3 are mounted to a system printed circuit board.

[0028] Figures 5A-5DA top view of the image sensor in operation according to an embodiment is shown.

[0029] Figures 6A-6D A top view of the image sensor in operation according to an alternative embodiment is shown.

[0030] Figure 7 A flowchart illustrating the operation of the image sensor according to an embodiment is shown.

[0031] Figure 8 Another flowchart summarizing and generalizing the operation of the image sensor according to an embodiment is shown.

Detailed Implementation Methods

[0032] Figure 1 The diagram schematically illustrates a radiation detector 100 as an example. The radiation detector 100 may include an array of sensing elements 150 (also referred to as pixels 150). This array may be a rectangular array (e.g., Figure 1 (As shown), cellular array, hexagonal array, or any other suitable array. Figure 1 The radiation detector 100 in the example has 28 sensing elements 150 arranged in 4 rows and 7 columns. However, the radiation detector 100 can typically have any number of sensing elements 150 arranged in any manner.

[0033] Each sensing element 150 can be configured to detect radiation incident upon it from a radiation source (not shown) and can be configured to measure characteristics of the radiation (e.g., particle energy, wavelength, radiative flux, and frequency). The radiation can include particles such as photons (electromagnetic waves) and subatomic particles. Each sensing element 150 can be configured to count the number of radiating particles incident upon it over a period of time, whose energy falls into multiple energy boxes. All of the sensing elements 150 can be configured to count the number of radiating particles incident upon it in multiple energy boxes within the same time period. When the incident radiating particles have similar energies, the sensing element 150 can be simply configured to count the number of radiating particles incident upon it over a period of time without measuring the energy of each individual radiating particle.

[0034] Each sensing element 150 may have its own analog-to-digital converter (ADC) configured to digitize an analog signal representing the energy of an incident radiated particle into a digital signal, or to digitize an analog signal representing the total energy of multiple incident radiated particles into a digital signal. The sensing elements 150 may be configured to operate in parallel. For example, while one sensing element 150 measures an incident radiated particle, another sensing element 150 may be waiting for the radiated particle to arrive. The sensing elements 150 need not be individually addressable.

[0035] The radiation detector 100 described herein can be used for applications such as X-ray telescopes, mammography, industrial X-ray defect detection, X-ray microscopy or photomicrography, X-ray casting inspection, X-ray non-destructive testing, X-ray welding inspection, and X-ray digital subtraction angiography. The radiation detector 100 can also be used in place of photographic film, photographic film, photoexcited phosphorescent plate, X-ray image intensifier, scintillator, or X-ray detector.

[0036] Figure 2A Schematic illustration according to an embodiment Figure 1 A simplified cross-sectional view of a radiation detector 100 along line 2A-2A. More specifically, the detector 100 may include a radiation-absorbing layer 110 and an electronic layer 120 (e.g., an application-specific integrated circuit) for processing or analyzing electrical signals of incident radiation generated in the radiation-absorbing layer 110. The radiation detector 100 may or may not include a scintillator (not shown). The radiation-absorbing layer 110 may include a semiconductor material such as silicon, germanium, gallium arsenide, cadmium telluride, zinc cadmium telluride, or combinations thereof. The semiconductor material may have a high mass attenuation coefficient for the radiation of interest.

[0037] Figure 2B This is illustrated as an example. Figure 1 A detailed cross-sectional view of the radiation detector 100 along line 2A-2A. More specifically, the radiation absorption layer 110 may include one or more diodes (e.g., pin or pn) consisting of one or more discrete regions 114 of a first doped region 111 and a second doped region 113. The second doped region 113 may be separated from the first doped region 111 by an optional intrinsic region 112. The discrete regions 114 are separated from each other by the first doped region 111 or the intrinsic region 112. The first doped region 111 and the second doped region 113 have opposite types of doping (e.g., region 111 is p-type and region 113 is n-type, or region 111 is n-type and region 113 is p-type). Figure 2B In the example, each discrete region 114 of the second doped region 113, together with the first doped region 111 and the optional intrinsic region 112, forms a diode. That is, in Figure 2B In the example, the radiation-absorbing layer 110 includes a plurality of diodes (more specifically, seven diodes corresponding to...). Figure 1 The array contains seven sensing elements 150 in one row. The plurality of diodes have electrical contacts 119A as a shared electrode. The first doped region 111 may also have discrete portions.

[0038] The electronics layer 120 can include an electronic system 121 adapted to process or interpret signals generated by radiation incident on the radiation absorption layer 110. The electronic system 121 can include analog circuitry such as filter networks, amplifiers, integrators, comparators, or digital circuitry such as microprocessors and memory. The electronic system 121 can include one or more analog-to-digital converters. The electronic system 121 can include components that are shared by the sensing elements 150 or components that are dedicated to individual sensing elements 150. For example, the electronic system 121 can include an amplifier dedicated to each sensing element 150 and a microprocessor shared among all sensing elements 150. The electronic system 121 can be electrically connected to the sensing elements 150 through the vias 131. The space between the vias can be filled with a filler material 130, which can increase the mechanical stability of the connection of the electronics layer 120 to the radiation absorption layer 110. Other bonding techniques are possible to connect the electronic system 121 to the sensing elements 150 without using the vias 131.

[0039] When radiation from a radiation source (not shown) strikes the radiation absorption layer 110, which includes diodes, the radiation particles can be absorbed and generate one or more charge carriers (e.g., electrons, holes) through several mechanisms. The charge carriers can drift under an electric field to an electrode of one of the diodes. The electric field can be an external electric field. The electrical contacts 119B can include discrete portions, each of which is in electrical contact with a discrete region 114. The term "electrical contact" can be used interchangeably with the word "electrode." In embodiments, the charge carriers can drift in different directions such that the charge carriers generated by a single radiation particle are substantially not shared by two different discrete regions 114 ("substantially not shared" means here that less than 2%, less than 0.5%, less than 0.1%, or less than 0.01% of these charge carriers flow to a different one of the discrete regions 114 than the rest of the charge carriers). The charge carriers generated by a radiation particle incident around the footprint of one of the discrete regions 114 are substantially not shared by another one of the discrete regions 114. A sensing element 150 associated with one of the discrete regions 114 can be a region around the discrete region 114 to which substantially all (more than 98%, more than 99.5%, more than 99.9%, or more than 99.99%) of the charge carriers generated by a radiation particle incident therein flow. That is, less than 2%, less than 1%, less than 0.1%, or less than 0.01% of the charge carriers flow outside the sensing element 150.

[0040] Figure 2C schematically illustrates a radiation absorption layer 110 according to an embodiment Figure 1An alternative detailed cross-sectional view of the radiation detector 100 along line 2A-2A. More specifically, the radiation absorption layer 110 can include a resistor, but not a diode, of a semiconducting material such as silicon, germanium, gallium arsenide, cadmium telluride, cadmium zinc telluride, or a combination thereof. The semiconducting material can have a high mass attenuation coefficient for the radiation of interest. In embodiments, Figure 2C The electronic layer 120 in Figure 2B The electronic layer 120 in

[0041] When the radiation impinges on the radiation absorption layer 110 including the resistor, but not a diode, the radiation can be absorbed and generate one or more charge carriers through several mechanisms. One radiation particle can generate 10 to 100,000 charge carriers. The charge carriers can drift under an electric field to the electrical contacts 119A and 119B. The electric field can be an external electric field. The electrical contact 119B includes discrete portions. In embodiments, the charge carriers can drift in different directions such that the charge carriers generated by a single radiation particle are substantially not shared by two different discrete portions of the electrical contact 119B ("substantially not shared" means here that less than 2%, less than 0.5%, less than 0.1%, or less than 0.01% of these charge carriers flow to a different group of discrete portions than the rest of the charge carriers). The charge carriers generated by a radiation particle incident around a footprint of one of the discrete portions of the electrical contact 119B are substantially not shared by another of the discrete portions of the electrical contact 119B. One of the sensing elements 150 associated with one of the discrete portions of the electrical contact 119B can be a region around the discrete portion to which substantially all (more than 98%, more than 99.5%, more than 99.9%, or more than 99.99%) of the charge carriers generated by a radiation particle incident therein flow. That is, less than 2%, less than 0.5%, less than 0.1%, or less than 0.01% of the charge carriers flow outside the sensing element associated with one of the discrete portions of the electrical contact 119B.

[0042] Figure 3 A top view schematically showing a package 200 including the radiation detector 100 and a printed circuit board 400 according to embodiments. The term "printed circuit board" as used herein is not limited to a particular material. For example, a printed circuit board can include a semiconductor. The radiation detector 100 is mounted to the printed circuit board 400. For clarity, the connections between the radiation detector 100 and the printed circuit board 400 are not shown. The printed circuit board 400 can have one or more radiation detectors 100. The printed circuit board 400 can have a region 405 not covered by the radiation detector 100 (e.g., a region for accommodating a bond wire 410). The radiation detector 100 can have a sensing area 190 that is the sensing elements 150 (e.g.,Figure 1 The radiation detector 100 can have a peripheral region 195 near its edges. The peripheral region 195 has no sensing elements, and the radiation detector 100 does not detect radiation particles incident on the peripheral region 195.

[0043] Figure 4 A cross-sectional view of an image sensor 490 according to an embodiment is shown schematically. The image sensor 490 can include a plurality of packages 200 mounted to a system printed circuit board 450. Figure 3 The image sensor 490 can include two packages 200 as an example. Electrical connections between the printed circuit board 400 and the system printed circuit board 450 can be achieved by bonding wires 410. To accommodate the bonding wires 410 on the printed circuit board 400, the printed circuit board 400 has a region 405 that is not covered by the radiation detector 100. To accommodate the bonding wires 410 on the system printed circuit board 450, there is a gap between the packages 200. The gap can be about 1 mm or more. Radiation particles incident on the peripheral region 195, the region 405, or the gap cannot be detected by the packages 200 on the system printed circuit board 450. Figure 4

[0044] A blind zone of a radiation detector (e.g., the radiation detector 100) refers to an area of the radiation receiving surface of the radiation detector where incident radiation particles cannot be detected by the radiation detector. A blind zone of a package (e.g., the package 200) refers to an area of the radiation receiving surface of the package where incident radiation particles cannot be detected by the radiation detector or a radiation detector in the package. In an embodiment, a blind zone of an image sensor (e.g., the image sensor 490) has a set of packages (e.g., packages mounted on the same printed circuit board, packages arranged in the same layer) includes the combination of the blind zones of the packages in the set and the gaps between the packages. Figure 3 and Figure 4 In the example shown, the blind zones of the packages 200 include the peripheral region 195 and the region 405. A blind zone (e.g., 488) of an image sensor (e.g., the image sensor 490) has a set of packages (e.g., packages mounted on the same printed circuit board, packages arranged in the same layer) includes the combination of the blind zones of the packages in the set and the gaps between the packages.

[0045] In an embodiment, the image sensor 490 including the radiation detector 100 can have the blind zone 488 where incident radiation cannot be detected. However, in an embodiment, the image sensor 490 having the sensing region 190 can take partial images of an object or a scene (not shown) and then the taken partial images can be stitched to form a complete image of the entire object or scene.

[0046] Figures 5A-5D ​A top view of the image sensor in operation according to an embodiment is shown schematically. For simplicity, only two sensing regions 190a and 190b of the image sensor 490 and the blind region 488 (i.e. other parts of the image sensor 490 such as the peripheral region 195 (not shown)) are shown. In an embodiment, a carton 510 enclosing a metal sword 512 can be located between the image sensor 490 and a radiation source (not shown) before the page. The carton 510 is between the image sensor 490 and an observer's eye. In the following, for generality, the carton 510 enclosing the metal sword 512 can be referred to as an object / scene 510+512. Figure 4

[0047] In an embodiment, the operation of the image sensor 490 in capturing images of the object / scene 510+512 can be as follows. First, as shown in Figure 5A the object / scene 510+512 can be stationary and the image sensor 490 can be moved to a first image capturing position relative to the object / scene 510+512. Then, when the image sensor 490 is at the first image capturing position, the image sensor 490 can be used to capture a first partial image 520.1 of the object / scene 510+512.

[0048] Next, in an embodiment, as shown in Figure 5B the image sensor 490 can be moved to a second image capturing position relative to the object / scene 510+512. Then, when the image sensor 490 is at the second image capturing position, the image sensor 490 can be used to capture a second partial image 520.2 of the object / scene 510+512.

[0049] Next, in an embodiment, as shown in Figure 5C the image sensor 490 can be moved to a third image capturing position relative to the object / scene 510+512. Then, when the image sensor 490 is at the third image capturing position, the image sensor 490 can be used to capture a third partial image 520.3 of the object / scene 510+512.

[0050] ​In this embodiment, the size and shape of the sensing regions 190a and 190b, as well as the positions of the first, second, and third image capturing positions, are such that any partial image of the partial images 520.1, 520.2, and 520.3 overlaps with at least one other partial image of the partial images 520.1, 520.2, and 520.3. For example, the distance 492 between the first and second image capturing positions can be close to and less than the width 190w of the sensing region 190a; therefore, the first partial image 520.1 overlaps with the second partial image 520.2.

[0051] In an embodiment, if any partial image of partial images 520.1, 520.2, and 520.3 overlaps with at least one other partial image of partial images 520.1, 520.2, and 520.3, the partial images 520.1, 520.2, and 520.3 can be stitched together to form a more complete image 520 of the object / scene 510+512. Figure 5D In this embodiment, the size and shape of the sensing areas 190a and 190b, as well as the positions of the first image capture position, the second image capture position, and the third image capture position, allow the stitched image 520 to cover the entire object / scene 510+512, such as... Figure 5D As shown.

[0052] Figures 6A-6D The diagram schematically shows a top view of the image sensor 490 in operation according to an alternative embodiment. In this embodiment, with regard to the capture of the partial images 520.1, 520.2, and 520.3, Figures 6A-6D The operation of the image sensor 490 described herein can be similar to Figures 5A-5D The operation of the image sensor 490 described above. In this embodiment, in addition to the capture of partial images 520.1, 520.2, and 520.3 as described above, Figures 6A-6D The operation of the image sensor 490 may also include the following.

[0053] In this embodiment, after capturing the third portion of image 520.3, the image sensor 490 can be moved to the first image capturing position. Figure 6A The fourth image capture position is located at or near the solid rectangle 490 in the middle. Figure 6AThe image sensor 490 can then be used to capture a fourth partial image 520.4 of the object / scene 510+512 when the image sensor 490 is at the fourth image capture position (dashed rectangle 490 in FIG. 5B). The first image capture position and the fourth image capture position can be considered to belong to a first group of positions. The first partial image and the fourth partial image can be considered to belong to a first group of partial images.

[0054] In an embodiment, after capturing the fourth partial image 520.4, the image sensor 490 can be moved to a fifth image capture position (solid rectangle 490 in FIG. 5B) at or near the second image capture position (dashed rectangle 490 in FIG. 5B). The image sensor 490 can then be used to capture a fifth partial image 520.5 of the object / scene 510+512 when the image sensor 490 is at the fifth image capture position. The second image capture position and the fifth image capture position can be considered to belong to a second group of positions. The second partial image and the fifth partial image can be considered to belong to a second group of partial images. Figure 6B Figure 6B In an embodiment, after capturing the fifth partial image 520.5, the image sensor 490 can be moved to a sixth image capture position (dashed rectangle 490 in FIG. 5B) at or near the third image capture position (solid rectangle 490 in FIG. 5B). The image sensor 490 can then be used to capture a sixth partial image 520.6 of the object / scene 510+512 when the image sensor 490 is at the sixth image capture position. The third image capture position and the sixth image capture position can be considered to belong to a third group of positions. The third partial image and the sixth partial image can be considered to belong to a third group of partial images.

[0055] In an embodiment, the positions of the six image capture positions can be such that the maximum distance between an image capture position of one of the first, second, and third groups of positions and another image capture position of another of the first, second, and third groups of positions is substantially greater (e.g., greater than 10 times, greater than 20 times, greater than 50 times, or greater than 100 times) than the maximum distance between two image capture positions of the one of the first, second, and third groups of positions. In other words, the minimum distance between two image capture positions of two different groups of positions is substantially greater than the maximum distance between two image capture positions of the same group. Figure 6C Figure 6C In an embodiment, after capturing the sixth partial image 520.6, the image sensor 490 can be moved to a seventh image capture position (dashed rectangle 490 in FIG. 5B) at or near the fourth image capture position (solid rectangle 490 in FIG. 5B). The image sensor 490 can then be used to capture a seventh partial image 520.7 of the object / scene 510+512 when the image sensor 490 is at the seventh image capture position. The fourth image capture position and the seventh image capture position can be considered to belong to the first group of positions. The fourth partial image and the seventh partial image can be considered to belong to the first group of partial images.

[0056] In an embodiment, the positions of the six image capture positions can be such that the maximum distance between an image capture position of one of the first, second, and third groups of positions and another image capture position of another of the first, second, and third groups of positions is substantially greater (e.g., greater than 10 times, greater than 20 times, greater than 50 times, or greater than 100 times) than the maximum distance between two image capture positions of the one of the first, second, and third groups of positions. In other words, the minimum distance between two image capture positions of two different groups of positions is substantially greater than the maximum distance between two image capture positions of the same group.

[0057] ​​In an embodiment, the minimum distance can be close to and less than the width 190w of the sensing region 190a. Figure 5A For example, the minimum distance can be in the range of 80% to 99.99% of the width 190w. In an embodiment, the minimum distance can be greater than 100 times the maximum distance. In an embodiment, the maximum distance can be less than 10 times the size of the sensing element 150.

[0058] In an embodiment, the image sensor 490 can only move directly from an image taking position of a group of positions to another image taking position of another group of positions. This means that in this embodiment, the image sensor 490 can not move directly from an image taking position of a group of positions to another image taking position of the same group of positions. For example, in this embodiment, the image sensor 490 can move directly from the third image taking position to the fifth image taking position because the third image taking position and the fifth image taking position belong to two different groups of positions (i.e., the third group of positions and the second group of positions, respectively). However, in this embodiment, the image sensor 490 can not move directly from the third image taking position to the sixth image taking position because the third image taking position and the sixth image taking position belong to the same group of positions (i.e., the third group of positions).

[0059] Next, in an embodiment, after the six partial images 520.1-6 have been taken, a combined image 620 of the object / scene 510+512 can be determined based on the six partial images 520.1-6 as follows. Specifically, in an embodiment, the partial images 520.1, 520.2, and 520.3 can be stitched to form a stitched image of the object / scene 510+512. Figure 6D

[0060] Next, in an embodiment, a first enhanced partial image can be determined for the first group of partial images based on the partial image 520.1 of the first group of partial images and the partial image 520.4, and then used in place of the partial image 520.1 of the stitched image. In other words, the partial image 520.4 is used to enhance the partial image 520.1 of the stitched image. More specifically, in an embodiment, the first enhanced partial image can be determined as follows. First, the positions of the first image taking position and the fourth image taking position relative to each other can be determined by (A) measuring using a marker or (B) inter-image correlation estimation.

[0061] ​In method (A), in an embodiment, markers can be added at a fixed position relative to the object / scene 510+512, so that at least one of the markers is present in each of the partial images 520.1 and 520.4 of the first partial image set. For example, a marker 630 Figure 6A ) is shown for illustration (other markers are not shown for simplicity), and its image is in the partial images 520.1 and 520.4. In an embodiment, the marker can have a shape of a cross (e.g. marker 630). In an embodiment, the marker can comprise a metal such as aluminum. With the position of the marker relative to the object / scene 510+512 known, the position of the first image taking position and the fourth image taking position relative to each other can be measured.

[0062] In an embodiment, method (B) can involve associating two partial images 520.1 and 520.4 to determine the position of the first image taking position and the fourth image taking position relative to each other. Specifically, in an embodiment, two portions of two partial images 520.1 and 520.4 of the first partial image set can be compared to determine a correlation coefficient. In an embodiment, if the determined correlation coefficient exceeds a predetermined threshold, the two portions from the two partial images 520.1 and 520.4 can be considered to be the same, so that the position of the first image taking position and the fourth image taking position (520.4 and 520.4, respectively) relative to each other can be estimated. In an embodiment, if the determined correlation coefficient does not exceed a predetermined threshold, the two portions from the two partial images 520.1 and 520.4 can be considered to be different, and another two portions of the two partial images 520.1 and 520.4 of the first partial image set can be compared, and so on.

[0063] In an embodiment, before performing the above correlation process, the resolution of two partial images 520.1 and 520.4 of the first partial image set can be increased (up-sampling). In an embodiment, the up-sampling process can be performed using interpolation.

[0064] In an embodiment, after the position of the first image taking position and the fourth image taking position relative to each other is determined as described above, a resolution enhancement algorithm (also known as super-resolution algorithm) can be applied to the two partial images 520.1 and 520.4 of the first partial image set to form a first enhanced partial image. In an embodiment, the first enhanced partial image can be used to replace the first partial image 520.1 in the stitched image.

[0065] In an embodiment, similarly, a second enhanced partial image can be determined for the second partial image group based on the two partial images 520.2 and 520.5, and then used to replace the partial image 520.2 in the stitched image. Similarly, a third enhanced partial image can be determined for the third partial image group based on the two partial images 520.3 and 520.6, and then used to replace the partial image 520.3 in the stitched image. In an embodiment, after the three replacements described above, if different regions of the stitched image have different resolutions, an algorithm can be executed to make the entire stitched image have the same resolution, thereby producing the combined image 620. Figure 6D ).

[0066] Figure 7 Flowchart 700 is shown, which summarizes the process according to... Figures 6A-6D The operation of the image sensor 490. Specifically, in step 710, the image sensor 490 moves through different image capture positions and captures partial images while at these positions. In step 720, a portion of the captured partial images (e.g., one partial image from each group of partial images) is stitched together to form a stitched image of the scene. In step 730, for each group of partial images, an enhanced partial image is determined based on the partial images in the group, and the resulting enhanced partial image is used to replace the corresponding partial image of the stitched image (i.e., to enhance the stitched image). In step 740, if necessary (i.e., if different regions of the stitched image have different resolutions), resolution equalization is performed on the stitched image to obtain the object / scene 510+512 (…). Figure 6D The combined image 620.

[0067] Figure 8 A flowchart 800 summarizing and outlining the operation of the image sensor 490 according to an embodiment is shown. In step 810, the image sensor 490 can be arranged at different locations, and different partial images of the same scene can be captured by the image sensor 490 at these different locations, wherein the minimum distance between two locations in two different sets of locations is substantially greater than the maximum distance between two locations in the same set of locations. In an embodiment, the image sensor 490 can be arranged one after another at these different locations (i.e., one location after another) to capture those partial images. In step 820, a combined image of the scene can be determined based on the captured partial images.

[0068] In the above embodiments, reference is made to Figure 4 and Figures 5A-5DThe image sensor 490 includes two sensing regions 190a and 190b, which are physically isolated from each other by the blind region 488, and have a rectangular shape. In general, the image sensor 490 can include N sensing regions (N is a positive integer), which can be physically separated from each other by blind regions (e.g., the blind region 488), can have any size or shape, and can be arranged in any manner.

[0069] In the above embodiment, the cardboard box 510 surrounding the metal sword 512 is used as an example of the object or scene being inspected. In general, any object or scene can be inspected using the image sensor 490.

[0070] In the above embodiment, the image sensor 490 includes two sensing regions 190 and moves through three position sets of two image capture positions. In general, the image sensor 490 can include N sensing regions (N is a positive integer) and move through P position sets (P is an integer greater than 1), where each position set can have any number of image capture positions. Thus, the number of image capture positions of the position sets need not be the same.

[0071] In the above embodiment, the image sensor 490 moves through the six image capture positions in the order of first, second, third, fourth, fifth, and then sixth image capture positions. In general, the image sensor 490 can move through the six image capture positions (or any number of image capture positions) in any order. For example, the image sensor 490 can move through the six image capture positions in the order of first, second, third, fifth, sixth, and then fourth image capture positions.

[0072] In the above embodiment, the object / scene 510+512 remains stationary and the image sensor 490 moves relative to the object / scene 510+512. In general, any arrangement of movement is possible as long as the image sensor 490 moves relative to the object / scene 510+512. For example, the image sensor 490 can remain stationary and the object / scene 510+512 can move relative to the image sensor 490.

[0073] In the above embodiment, the partial images 520.1, 520.2, and 520.3 are stitched to form a stitched image of the object / scene 510+512. In general, a combination of partial images with one partial image of each partial image set can be stitched to form a stitched image of the object / scene 510+512. For example, the partial images 520.1, 520.5, and 520.6 can be stitched to form a stitched image of the object / scene 510+512.

[0074] In the above embodiments, the stitching is performed, and then the enhanced partial images are determined and used to enhance the stitched image. Alternatively, the enhanced partial images are determined before the stitching is performed. For example, the first, second, and third enhanced partial images can be determined as described above. Then, the first, second, and third enhanced partial images can be stitched to form a combined and complete image of the object / scene 510+512 (e.g., image 620). Many other possible ways of processing the six partial images 520.1-520.6 can be used to form a combined and complete image of the object / scene 510+512. Figure 6D

[0075] In the above embodiments, the first, second, and third enhanced partial images are used to replace the corresponding partial images in the stitched image. In alternative embodiments, if the resolution of an enhanced partial image is not higher than the resolution of the partial image in the stitched image that the enhanced partial image is supposed to replace, then the enhanced partial image of the partial image group is not used for such replacement.

[0076] For example, if the resolution of the first enhanced partial image is not higher than the resolution of the partial image 520.1, then the first enhanced partial image of the first partial image group is not used to replace the partial image 520.1 of the stitched image. This can happen when the distance (or offset) between the corresponding first image capture position and fourth image capture position is K times the size of the sensing element 150, where K is a non-negative integer.

[0077] While various aspects and embodiments have been disclosed herein, other aspects and embodiments will be apparent to those skilled in the art. The various aspects and embodiments disclosed herein are for purposes of illustration and are not intended to be limiting, with the true scope and spirit being delineated by the following claims.​

Claims

1. A method using an image sensor, the image sensor comprising N sensing regions for capturing images of a scene, where N is a positive integer, the N sensing regions being physically separated from each other, the method comprising: For i = 1, ..., P, and j = 1, ..., Q(i), the image sensor is placed at a position (i, j) relative to the scene, and a partial image (i, j) of the scene is captured using the image sensor when it is at the position (i, j), thereby capturing a total of R partial images. Where R is the sum of Q(i), i = 1, ..., P, Where P is an integer greater than 1. Where Q(i), i = 1, ..., P, are positive integers and not all of them are 1. Where for i = 1, ..., P, the position group (i) includes the position (i, j), j = 1, ..., Q(i), and Wherein the minimum distance between a position in one position group (i), i = 1, ..., P and another position in another position group (i), i = 1, ..., P is substantially greater than the maximum distance between two positions in the same position group (i), i = 1, ..., P; and A combined image of the scene is determined based on the R partial images. The determination of the combined image of the scene includes: stitching the partial images (i, 1), i = 1, ..., P, to form a stitched image of the scene; for i = 1, ..., P, determining an enhanced partial image (i) based on the partial images (i, j), j = 1, ..., Q(i); and for i = 1, ..., P, if the resolution of the enhanced partial image (i) is higher than the resolution of the partial image (i, 1), then using the enhanced partial image (i) to replace the partial image (i, 1) of the stitched image; or The determination of the combined image of the scene includes: for i = 1, ..., P, determining an enhanced partial image (i) based on the partial image (i, j), j = 1, ..., Q(i); stitching the enhanced partial image (i), i = 1, ..., P to form a stitched image of the scene; and balancing the resolution of different regions of the stitched image.

2. The method of claim 1, wherein N is greater than 1.

3. The method of claim 1, wherein placing the image sensor at positions (i, j) for i = 1, ..., P, and j = 1, ..., Q(i) is performed one at a time.

4. The method as described in claim 1, wherein Q(i), i = 1, ..., P, are the same and greater than 1.

5. The method of claim 1, wherein the minimum distance is close to and smaller than the size of the sensing area among the N sensing areas.

6. The method of claim 1, wherein the minimum distance is greater than 100 times the maximum distance.

7. The method of claim 1, wherein the maximum distance is less than 10 times the size of the sensing elements of the N sensing regions.

8. The method of claim 1, wherein placing the image sensor at the position (i, j) for i = 1, ..., P, and j = 1, ..., Q(i) comprises moving the image sensor directly from one position in one of the position groups (i), i = 1, ..., P, to another position in another position group (i), i = 1, ..., P, and does not comprise moving the image sensor directly from one position in one of the position groups (i), i = 1, ..., P, to another position in the same position group.

9. The method of claim 1, wherein determining the combined image of the scene further comprises, after performing the process of replacing the partial image (i, 1) of the stitched image with the enhanced partial image (i), equalizing the resolution of different regions of the stitched image.

10. The method of claim 1, wherein determining the enhanced portion image (i) includes determining the positions (i, j), j = 1, ..., Q(i), relative to each other.

11. The method of claim 10, wherein determining the positions (i, j), j = 1, ..., Q(i), relative to each other, includes using markers that are stationary relative to the scene.

12. The method of claim 10, wherein determining the positions (i, j), j = 1, ..., Q(i), relative to each other comprises: Upsampling is performed on the partial images (i, j), j = 1, ..., Q(i), thereby upsampling the partial images (i, j), j = 1, ..., Q(i) respectively; and The upsampled partial images (i, j), j = 1, ..., Q(i) are correlated to determine the positions (i, j), j = 1, ..., Q(i) relative to each other.

Images