System and method for stitching sequential images of object

Abstract

To provide systems and methods for stitching sequential images of an object.SOLUTION: A system comprises: a transport device for moving at least one object; at least one 2D digital optical sensor; and a controller operatively coupled to the 2D digital optical sensor. The controller performs the steps of: a) receiving a first digital image; b) receiving a second digital image; and c) stitching the first digital image and the second digital image using a stitching algorithm to generate a stitched image.SELECTED DRAWING: None

Description

[Technical field]

[0001] (Technology field) The present invention relates to the technical field of image stitching, in particular to the high speed stitching of two-dimensional images generated in a windowed conveyor system. [Background technology]

[0002] BACKGROUND OF THEINVENTION Vision systems are used in a wide variety of applications and industries to measure, inspect, align objects, and / or decode symbologies such as one-dimensional and two-dimensional bar codes. These systems are based on the use of image sensors to capture images of objects and process these captured images using an on-board or interconnected vision system processor. Images are typically captured as an array of pixels, with each pixel having a different color and / or intensity.

[0003] A common use for such imaging systems is to track and classify objects moving along conveyors in manufacturing and logistics operations. Typically, such imaging systems capture all sides of the object being tracked, which in many cases results in the capture of six sides of a cubic object. In such systems, it is necessary to capture the bottom of the object, which is the side of the object that is in contact with the conveyor.

[0004] In some instances, line scan cameras can be employed to address object movement and wide fields of view. However, such solutions are not applicable to certain object geometries and line configurations. In addition, while line scan image sensors tend to be less expensive than conventional format area scan sensors, the overall system cost using line scan image sensors can be significantly higher than conventional format area scan sensors due to increased computational and processing requirements. For example, line scan image sensors require the complete image to be reconstructed in software line by line. Furthermore, alignment of object motion and timing of image acquisition are critical.

[0005] Area-scan image sensors quickly image a defined area, allowing for easier setup and alignment and more flexibility than line-scan sensors. However, even with area-scan image sensors, only a partial view of the object may be captured, depending on factors such as the size of the object, the imager resolution, and the conveyor geometry. Thus, some processing may be required to reconstruct the complete image.

[0006] Another consideration is which side of the object should be imaged, as system configurations may vary depending on whether the bottom, top, or other side of the object should be imaged. For example, in some applications, some or all sides of the object are imaged, including the bottom. In other applications, imaging of only one side is required. For example, the bottom may be imaged instead of the top to avoid complications associated with imaging objects with different heights, which may have different distances from the camera.

[0007] In some applications, one or more cameras may be used to image each side of an object, and in other applications, two or more cameras may be used to image one side of an object. In applications where one side of an object is imaged, a single standard resolution camera may not provide adequate resolution, and an appropriate area of ​​interest and two or more cameras may be used to image that side of the object. Summary of the Invention [Means for solving the problem]

[0008] Embodiments of the present invention may provide high resolution images of the surfaces of objects of various shapes and sizes as they move through a conveyor system. Such embodiments may be relevant to a solution to the problem of scanning the bottom surface of a box or other object moving on a high speed conveyor belt by using one or more two-dimensional cameras. Current systems rely on line scan cameras to image objects such as boxes.

[0009] Scanning the bottom surface in logistics tunnel applications is particularly challenging because the physical constraints of the system limit visibility of the bottom surface of the box to only a small slice at a time. Small image slices can be acquired (through the gap between two sections of the conveyor belt) and then stitched together to generate an image for the bottom surface.

[0010] In an embodiment, the system may include a transport device for moving at least one object, where at least one substantially planar surface of the object is moved within a known plane locally around the periphery of the viewing zone, and where the substantially planar surface of the object is occluded except when the at least one substantially planar surface passes through the viewing zone; at least one two-dimensional digital optical sensor configured to capture at least two successive two-dimensional digital images of the at least one substantially planar surface of the at least one object being moved within the known plane around the periphery of the viewing zone; and a controller operably coupled to the two-dimensional digital optical sensor, the controller performing the steps of: a) receiving a first digital image; b) receiving a second digital image; and c) stitching the first digital image and the second digital image by using a stitching algorithm to generate a stitched image.

[0011] In an embodiment, the controller may repeat steps a)-c). The first digital image may comprise at least one of a captured digital image or a resultant image of a previously stitched image. The surface moved through the known plane may be one of a bottom surface of the object or a side surface of the object, and the transport device may comprise a viewing zone positioned on a corresponding surface of the transport device, and the two-dimensional digital image may be captured through the viewing zone. The transport device may comprise a conveyor, and the viewing zone may comprise one of an optical window in the conveyor, a gap in a surface of the conveyor, or a gap between two conveyor devices.

[0012] In an embodiment, the system may further include a light source configured to illuminate one or more objects in the viewing zone, and the controller may be coupled to the light source to control illumination from the light source, and the controller may cause the light source to blink intermittently. The sensor may be configured to detect the presence or absence of an object on the transport device, and may control the acquisition of an image based on the presence or absence of the object on the transport device. The system may further include a mirror, and the two-dimensional digital optical sensor may capture the digital image through the mirror. The controller may decode the mark based on the stitched image.

[0013] In an embodiment, the stitching algorithm may include determining a two-dimensional coordinate transformation to be used to align the first digital image and the second digital image, warping the second digital image by using the found two-dimensional coordinate transformation, and blending the warped second digital image with the first digital image. The transport device may operate according to a substantially linear motion, and the controller may comprise a model of the substantially linear motion of the transport device, and the stitching algorithm may use the model of the substantially linear motion of the transport device. The system may further comprise a motion encoder, and the control of the acquisition of the images based on the motion encoder, and the determining of the image transformation may further be based on an estimate of the object translation generated by the motion encoder. The system may further comprise a light source configured to illuminate one or more objects in the viewing zone, and the controller may be coupled to the light source to control illumination from the light source based on the estimate of the object translation generated by the motion encoder, and the controller may intermittently blink the light source based on the estimate of the object translation generated by the motion encoder.

[0014] In an embodiment, the two-dimensional optical sensor may be configured to capture a reduced field of view, the reduced field of view being determined manually or by analyzing an entire image acquired at a set time. The approximate two-dimensional coordinate transformation may be estimated through a training process using multiple digital image slices captured for a calibration moving object having a known pattern of features. The system may further include multiple two-dimensional digital optical sensors, each configured to capture multiple successive digital images associated with one or more objects through the viewing zone, the multiple two-dimensional digital optical sensors configured to capture the digital images substantially simultaneously.

[0015] In embodiments, at least one of each digital image from a camera may be stitched together with image slices captured substantially simultaneously by each other camera to form a combined image slice, and the combined image slices may be stitched together to form a complete image, or each digital image from each camera may be stitched together with digital images of successively captured digital images from that camera, and the stitched images from each camera of the multiple cameras may be stitched together to form a complete image, or each digital image from each camera may be directly stitched into the same resulting image of previously stitched images from all cameras.

[0016] In an embodiment, a computer-implemented method may include moving at least one object by a transport device, where at least one substantially planar surface of the object is moved within a locally known plane around a periphery of the viewing zone, the substantially planar surface of the object being occluded except when the at least one substantially planar surface passes through the viewing zone; capturing at least two successive two-dimensional digital images of the surface of the object being translated within the locally known plane around the periphery of the viewing zone by at least one two-dimensional digital optical sensor; and stitching the first digital image and the second digital image together using a stitching algorithm to generate a stitched image.

[0017] In an embodiment, the stitching may be performed iteratively and may include determining a two-dimensional coordinate transformation to be used to align the first digital image and the second digital image, warping the second digital image by using the found two-dimensional coordinate transformation, and blending the warped second digital image with the first digital image. The controller may decode the mark based on the stitched images. There may be a plurality of two-dimensional digital optical sensors capturing images, and at least one of each digital image from the two-dimensional digital optical sensors may be stitched together with image slices captured substantially simultaneously by each of the other two-dimensional digital optical sensors to form a combined image slice, and the combined image slices may be stitched together to form a complete image, or each digital image from each two-dimensional digital optical sensor may be stitched together with digital images of successively captured digital images from that two-dimensional digital optical sensor, and the stitched images from each two-dimensional digital optical sensor of the plurality of two-dimensional digital optical sensors may be stitched together to form a complete image, or each digital image from each two-dimensional digital optical sensor may be directly stitched into the same resultant image of previously stitched images from all two-dimensional digital optical sensors.

[0018] In an embodiment, the stitching may be performed using parallel processing, or the controller decodes the marks based on the stitched image, and both the stitching and the decoding are performed using parallel processing.

[0019] In an embodiment, the stitching may be performed iteratively and may include determining a two-dimensional coordinate transformation to be used to align the first digital image and the second digital image, warping the second digital image by using the found two-dimensional coordinate transformation, and blending the warped second digital image with the first digital image. The controller may decode the mark based on the stitched images. There may be a plurality of two-dimensional digital optical sensors capturing images, and each digital image from the two-dimensional digital optical sensor is stitched together with image slices captured substantially simultaneously by each of the other two-dimensional digital optical sensors to form a combined image slice, and the combined image slices are stitched together to form a complete image, or each digital image from each two-dimensional digital optical sensor is stitched together with digital images of successively captured digital images from that two-dimensional digital optical sensor, and the stitched images from each two-dimensional digital optical sensor of the plurality of two-dimensional digital optical sensors are stitched together to form a complete image, or each digital image from each two-dimensional digital optical sensor is directly stitched into the same resulting image of previously stitched images from all two-dimensional digital optical sensors. The stitching may be performed by using parallel processing, or the controller may decode the mark based on the stitched image, and the stitching and decoding may be performed by using parallel processing.

[0020] The method may further include controlling illumination from a light source configured to illuminate an object in the viewing zone based on an estimate of the object translation generated by the motion encoder. The method may further include intermittently blinking the light source based on an estimate of the object translation generated by the motion encoder. The method may further include controlling illumination from a light source configured to illuminate an object in the viewing zone. The method may further include intermittently blinking the light source. The method may further include detecting the presence or absence of an object on the conveyor device by using a sensor. The method may further include controlling acquisition of an image based on the presence or absence of an object on the conveyor device. The method may further include controlling illumination from the light source based on the presence or absence of an object on the conveyor device. The method may further include forming a single stitched image based on a fixed distance the two-dimensional digital camera is positioned from the viewing zone. The method may further include capturing a digital image by using the two-dimensional digital camera via a mirror.

[0021] The conveyor may operate according to a substantially linear motion. The method may further include modeling the substantially linear motion of the conveyor. The method may further include using a model of the substantially linear motion of the conveyor to perform the stitching. The method may further include capturing the plurality of digital images substantially simultaneously by using a plurality of two-dimensional digital cameras, each two-dimensional camera configured to capture a plurality of successive digital images associated with one or more objects through a viewing zone. The method may further include stitching each digital image from a camera with images captured substantially simultaneously by each other camera to form a combined image, and the combined image slices stitched together form a complete image. The method may further include stitching each digital image from each camera together with digital images of the successively captured digital images from that camera, and stitching together the stitched images from each camera of the plurality of cameras to form a complete image.

[0022] Many other embodiments are described throughout this specification. All of these embodiments are intended to be within the scope of the invention disclosed herein. Although various embodiments are described herein, it should be understood that not all objectives, advantages, features, or concepts need to be achieved in accordance with any particular embodiment. Thus, for example, a person skilled in the art will recognize that the invention can be realized or carried out in a manner that achieves or optimizes one advantage or group of advantages as taught or suggested herein, without necessarily achieving other objectives or advantages as may be taught or suggested herein.

[0023] The methods and systems disclosed herein may be implemented in any means for achieving various aspects, and may be implemented in the form of a machine-readable medium that embodies a set of instructions that, when executed by a machine, causes the machine to perform any of the operations disclosed herein. These and other features, aspects, and advantages of the present invention will become readily apparent to those skilled in the art and will be understood with reference to the following description, the appended claims, and the accompanying drawings, in which the invention is not limited to any particular disclosed embodiment or embodiments. The present invention provides, for example, the following items. (Item 1) 1. A system comprising: a transport device for moving at least one object, wherein at least one substantially planar surface of the object is moved in a locally known plane around a viewing zone, the substantially planar surface of the object being occluded except when the at least one substantially planar surface passes through the viewing zone; at least one two-dimensional digital optical sensor configured to capture at least two successive two-dimensional digital images of the at least one substantially planar surface of the at least one object being moved in the known plane around the periphery of the viewing zone; a controller operably coupled to the two-dimensional digital optical sensor, the controller comprising: a) receiving a first digital image; b) receiving a second digital image; c) stitching the first digital image and the second digital image together using a stitching algorithm to generate a stitched image; The controller and A system comprising: (Item 2) The system described in the above item, wherein the controller repeats steps a) to c). (Item 3) The system of any preceding claim, wherein the first digital image comprises at least one of a captured digital image or an image resulting from a previously stitched together image. (Item 4) the surface that is moved through the known plane is one of a bottom surface of the object or a side surface of the object; the transport device comprising a viewing zone positioned on a corresponding surface of the transport device; 2. The system of claim 1, wherein the two-dimensional digital image is captured through the viewing zone. (Item 5) The system of any of the preceding items, wherein the transport device comprises a conveyor and the viewing area comprises one of an optical window in the conveyor, a gap in a surface of the conveyor, or a gap between two conveyor devices. (Item 6) The system of any of the preceding items, further comprising a light source configured to illuminate the one or more objects in the viewing zone, the controller coupled to the light source to control illumination from the light source, the controller intermittently blinking the light source. (Item 7) 2. The system of claim 1, further comprising a sensor configured to detect the presence or absence of an object on the transport device and to control image acquisition based on the presence or absence of an object on the transport device. (Item 8) 2. The system of claim 1, further comprising a mirror, the two-dimensional digital optical sensor capturing the digital image through the mirror. (Item 9) The system of any preceding claim, wherein the controller decodes marks based on the stitched image. (Item 10) The stitching algorithm above is determining a two-dimensional coordinate transformation to be used to align the first digital image and the second digital image; warping the second digital image by using the found two-dimensional coordinate transformation; blending the warped second digital image with the first digital image; 3. The system according to any one of the preceding items, comprising: (Item 11) The system of any of the preceding claims, wherein the transport device operates according to a substantially linear motion, the controller comprises a model of the substantially linear motion of the transport device, and the stitching algorithm uses the model of the substantially linear motion of the transport device. (Item 12) 2. The system of claim 1, further comprising a motion encoder, the system controlling image acquisition based on the motion encoder, and determining the image transformation is further based on an estimate of object translation generated by the motion encoder. (Item 13) The system of any of the preceding items, further comprising a light source configured to illuminate the one or more objects in the viewing zone, the controller coupled to the light source to control illumination from the light source based on the estimate of object translation generated by the motion encoder, and the controller intermittently blinking the light source based on the estimate of object translation generated by the motion encoder. (Item 14) The system of any of the preceding items, wherein the two-dimensional optical sensor is configured to capture a reduced field of view that is determined by one of manually or by analyzing an entire image acquired at a set time. (Item 15) 2. The system of claim 1, wherein the approximate two-dimensional coordinate transformation is estimated through a training process using multiple digital image slices captured of a calibration moving object having a known pattern of features. (Item 16) The system of any of the preceding items, further comprising a plurality of two-dimensional digital optical sensors, each configured to capture a plurality of successive digital images through the field of view, the plurality of successive digital images being associated with the one or more objects, the plurality of two-dimensional digital optical sensors configured to capture the digital images substantially simultaneously. (Item 17) each digital image from the camera is stitched together with image slices captured substantially simultaneously by each other camera to form a combined image slice, and the combined image slices are stitched together to form a complete image; each digital image from each camera is stitched together with a digital image of the successively captured digital images from that camera, and the stitched images from each camera of the plurality of cameras are stitched together to form a complete image; or Each digital image from each camera is stitched directly into the same resulting image of the previously stitched images from all cameras. The system according to any of the above items, wherein the system is at least one of the following: (Item 18) 1. A computer-implemented method, comprising: moving at least one object with a transport device, wherein at least one substantially planar surface of the object is moved within a locally known plane around a periphery of a viewing zone, the substantially planar surface of the object being occluded except when the at least one substantially planar surface of the object passes through the viewing zone; capturing, by at least one two-dimensional digital optical sensor, at least two successive two-dimensional digital images of the surface of the object being translated in the known plane locally at the periphery of the viewing zone; stitching the first digital image and the second digital image together using an algorithm to generate a stitched image; A computer-implemented method comprising: (Item 19) The joining is performed repeatedly, and the joining is performed in the following manner: determining a two-dimensional coordinate transformation to be used to align the first digital image and the second digital image; warping the second digital image by using the found two-dimensional coordinate transformation; blending the warped second digital image with the first digital image; The method according to any of the above items, comprising: (Item 20) 2. The method of claim 1, wherein the controller decodes marks based on the stitched image. (Item 21) There are a plurality of two-dimensional digital optical sensors that capture images; Each digital image from the two-dimensional digital optical sensor is stitched together with image slices captured substantially simultaneously by each of the other two-dimensional digital optical sensors to form a combined image slice, and the combined image slices are stitched together to form a complete image, or Each digital image from each two-dimensional digital optical sensor is stitched together with a digital image of the successively captured digital images from that two-dimensional digital optical sensor, and the stitched images from each two-dimensional digital optical sensor of the plurality of two-dimensional digital optical sensors are stitched together to form a complete image, or 2. The method according to any of the preceding items, wherein each digital image from each two-dimensional digital optical sensor is directly stitched into the same resultant image of previously stitched images from all two-dimensional digital optical sensors. (Item 22) The stitching is performed using parallel processing, or 2. The method of claim 1, wherein the controller decodes marks based on the stitched image, and the stitching and decoding are performed by using parallel processing. Abstract (summary) The system may include a transport device for moving at least one object, where at least one substantially planar surface of the object is moved in a known plane locally around the periphery of the viewing zone, and where the substantially planar surface of the object is occluded except when the at least one substantially planar surface passes through the viewing zone; at least one two-dimensional digital optical sensor configured to capture at least two successive two-dimensional digital images of the at least one substantially planar surface of the at least one object being moved in the known plane around the periphery of the viewing zone; and a controller operably coupled to the two-dimensional digital optical sensor, the controller performing the steps of: a) receiving a first digital image; b) receiving a second digital image; and c) stitching the first digital image and the second digital image by using a stitching algorithm to generate a stitched image. [Brief description of the drawings]

[0024] So that the above-mentioned features of the invention may be understood in detail, a more particular description of the invention briefly summarized above may be had by reference to embodiments, some of which are illustrated in the accompanying drawings. It should be noted, however, that the accompanying drawings illustrate only typical embodiments of the invention, and that the invention may admit of other equally effective embodiments.

[0025] [Figure 1] FIG. 1 illustrates an example of a schematic top view of a system according to an embodiment.

[0026] [Diagram 2] FIG. 2 illustrates an example of a schematic bottom view of a system, according to an embodiment.

[0027] [Diagram 3]FIG. 3 is an illustration of an example of sequential images captured by a two-camera sequential stitching system, according to an embodiment.

[0028] [Figure 4] FIG. 4 is an illustration of an example stitched image generated by a system according to an embodiment.

[0029] [Diagram 5] FIG. 5 illustrates an example flow diagram of an image stitching process, according to an embodiment.

[0030] [Figure 6] FIG. 6 illustrates an example of a feature extraction process, according to an embodiment.

[0031] [Figure 7a] 7a and 7b illustrate examples of feature matching, according to an embodiment. [Figure 7b] 7a and 7b illustrate examples of feature matching, according to an embodiment.

[0032] [Figure 8] FIG. 8 illustrates an example of a calibration pattern, according to an embodiment.

[0033] [Figure 9] FIG. 9 illustrates an example of a calibration process, according to an embodiment.

[0034] [Figure 10] FIG. 10 illustrates a schematic top view of a two-camera system, according to an embodiment.

[0035] [Figure 11] FIG. 11 illustrates a schematic top view of a single camera side imaging system, in accordance with an embodiment.

[0036] [Figure 12] FIG. 12 illustrates a schematic diagram of components of a control system, according to an embodiment.

[0037] Other features of the present embodiments will become apparent from the following detailed description. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0038] Detailed Description of the Embodiments In the following detailed description of the preferred embodiment, reference is made to the accompanying drawings, which form a part of this specification, and which show by way of illustration specific embodiments in which the invention may be practiced. It is to be understood that other embodiments may be utilized and structural changes may be made without departing from the scope of the invention. Electrical, mechanical, logical, and structural changes may be made to the embodiments without departing from the spirit and scope of the present teachings. Thus, the following detailed description is not to be taken in a limiting sense, and the scope of the present disclosure is defined by the appended claims and their equivalents.

[0039] FIG. 1 illustrates a schematic top view of a system according to an embodiment. The system may include a transport device, such as a conveyor 120 or a robotic arm (not shown), at least one camera 132, and a controller / processor 1200. The conveyor 120 may translate an object 110, such as a parcel, across a belt, rollers, or other conveyor mechanism through the field of view of the camera / imager 132. Similarly, the robotic arm (not shown) may grasp or otherwise hold an object 110, such as a parcel, by, for example, a robotic hand or claw, or by a suction device, etc. The robotic arm may move the object through the field of view of the camera / imager 132. The transport device (regardless of type) may translate the object 110 in a known plane locally around a visibility zone, such as visibility zone 121. However, in embodiments, the translation of the object 110 when not in the visibility zone may not be a translation in a known plane. Rather, outside the field of view, the transport device may translate the object 110 in any direction or form of motion. In certain examples described herein, a conveyor or robotic arm may be referenced, but it should be understood that in embodiments, the transport device may include a conveyor, a robotic arm, or any other device capable of transporting an object.

[0040] The conveyor 120 may have a viewing zone 121 on its surface. For example, the viewing zone 121 may include a window, such as a glass window or a scratch-resistant plastic window, or the viewing zone 121 may include an opening. The viewing zone 121 may be any size, but typically the width of the viewing zone 121 (perpendicular to the direction of conveyor travel) is as large as possible to allow capture of the entire bottom of each object 110. For example, in embodiments, the width of the viewing zone 121 may be approximately equal to the width of the conveyor 120, or may be only slightly smaller than the width of the conveyor 120 to accommodate the structure of the conveyor 120. In embodiments in which the viewing zone 121 includes a window, the length of the window (in the direction of conveyor travel) may be relatively large, but not so large as to impede the progress of objects on the conveyor. For example, a typical window may be approximately one-quarter inch to two inches long. In embodiments in which the viewing zone 121 includes a gap, the gap cannot be so large as to impede the progress of objects on the conveyor. For example, a relatively large gap may be large enough that an object falls off the conveyor 120 through the gap. Similarly, the gap may be large enough that an object on the conveyor may have non-uniform motion, with the leading edge falling slightly and then returning. In some embodiments, the gap may be provided by an opening in a section of the conveyor 120, while in other embodiments, the gap may be provided by a separation between two sections of the conveyor 120. These are just examples of a viewing zone. In embodiments, the viewing zone may have some physical implementation, such as a window or a gap, while in other embodiments, the viewing zone may simply be some region in space where the camera / imager 132 may capture an image (of appropriate quality and resolution to be processed as described below) of the surface of the object 110.

[0041] In the example shown in FIG. 1, an object may be translated on conveyor 120 to image the bottom surface of the object. In this example, the bottom surface of the object is in the plane of conveyor 120. Thus, the plane of conveyor 120 may be considered to form a known plane. In an embodiment, the bottom surface of the object may be imaged in conjunction with gravity to ensure that the bottom surface of the object is in the known plane, conveyor 120. The known plane may then form a plane on which the surface of the object may be imaged directly or through a window or gap in any device in the known plane.

[0042] In other embodiments, any surface of the object may be imaged in relation to the instrument to ensure that the surface of the object is in a known plane or parallel to a known plane. This known plane may then form a plane in which the surface of the object may be imaged directly in any instrument in the known plane or through a window or gap. Thus, embodiments may image the bottom, top, or any side of the object. Furthermore, it should be noted that such surfaces may not themselves be perfectly planar or flat. For example, in some cases the surface of a box may be planar, but in some cases the surface of the box may be dented, curved, or folded. In some cases, the object may not be a box, but an envelope, pouch, or other irregularly shaped object. The important point is that the surface of the object to be captured is substantially planar, i.e., sufficiently planar for the camera to capture an image of the object's surface with sufficient sharpness for the image to be processed as described herein.

[0043] The camera 132 may capture images through the viewing zone 121. The camera 132 may capture images directly or the camera 132 may capture images through a series of mirrors (not shown). In this example, one camera is shown, but in embodiments, one or more cameras may be used. For example, if the width (perpendicular to the direction of travel of the conveyor 120) is too large to be captured by one camera or is too large to be captured with adequate image quality by one camera, two or more cameras may be used. The embodiments may apply to any number of cameras. Examples of such embodiments are described in more detail below.

[0044] The camera 132 may capture images sequentially as the object is translated through the camera's field of view. The sequential images may be captured in a particular order. It is not required that the sequential images be captured one after the other without intermediate images being captured. For example, in an embodiment, fewer than all possible images may be captured, but such images are still sequential. In an embodiment, some images in a sequence may be skipped, but the resulting images are still sequential. For example, in an embodiment, every nth image may be captured (or skipped) to form a sequence, or all images may be captured or skipped at irregular intervals. Similarly, in an embodiment, each captured image may be processed, or every nth captured image may be processed (or skipped). The term sequential encompasses any and all such sequences.

[0045] A light source 131 may accompany the camera to provide adequate illumination and illuminate dark surface objects. For example, in an embodiment, the light source 131 may include a strobe, such as a strobe light or strobe lamp, which may produce flashes of light when controlled by the controller / processor 1200. Such strobe lights may include intermittent light sources, such as xenon flash lamps or flash tubes, or may include more continuous light sources operated in an intermittent manner, such as light emitting diodes (LEDs), incandescent lamps, halogen lamps, etc. The purpose of the strobe light is to provide a sufficient amount of light at a very short exposure time when the object is moving, so that the images captured do not have motion blur. In comparison to strobe lights, line scan cameras may require a light that is constantly on, which may not be energy efficient. The intensity and duration of each flash may be controlled by the controller / processor 1200 or other exposure control circuitry based on factors that may include the reflectivity of the object being captured, reflections from any windows that may be present in the viewing zone 121, the speed of the conveyor, etc. In other embodiments, the light source 131 may include fixed or constant illumination. Similarly, the intensity of such illumination may be controlled by the controller / processor 1200 or other exposure control circuitry based on factors that may include the reflectivity of the object being captured, reflections from any windows that may be present in the viewing zone 121, the speed of the conveyor, etc. The light source 131 may be at a different angle relative to the window than the camera to prevent oversaturation of portions of the image due to reflections (such as may occur if a window present in the viewing zone 121 is made of glass or another transparent material). In other embodiments, polarization may be used to prevent reflection issues.

[0046] In some embodiments, the system may include a speed or translation sensor or encoder 122 attached to the conveyor 120. The sensor / encoder 122 may be connected to the controller / processor 1200 to provide information about the speed of the conveyor 120 and / or the translation of the object 110 on the conveyor 120. Such speed and / or translation may be measured periodically or otherwise repeatedly to detect changes in speed and / or translation. In embodiments, the sensor / encoder 122 may be a rotary encoder, a shaft encoder, or any other electromechanical device that may convert the angular position or motion of a wheel, shaft, or axle of the conveyor 120, or a wheel, shaft, or axle attached to the conveyor 120, into an analog signal or digital code that may be received by the controller. The controller / processor may convert such an analog signal into a digital representation by using an analog-to-digital converter. In embodiments, the encoder 122 may provide an absolute or relative speed or translation output.

[0047] In an embodiment, the encoder 122 may determine the translation of an object on the conveyor 120 and may output one or more pulses per unit distance of travel of the object 110 or the conveyor 120. For example, the encoder 122 may be positioned to provide one revolution of rotational motion per 12 inches of travel of the conveyor 120. The encoder 122 may be positioned to provide a number of pulses per revolution, such as 24 pulses per revolution. In this example, one pulse from the encoder 122 may occur for every half inch of travel of the conveyor 120. In an embodiment, the revolutions per travel of the conveyor 120 and the number of pulses per revolution may be fixed or these parameters may be adjustable or programmable.

[0048] The controller / processor 1200 may be arranged to cause the camera 132 to capture an image based on each pulse or many pulses from the encoder 122. Each such captured image may be used to form an image slice to be stitched together as described below. If the image resolution is known or can be determined, the image resolution of each motion unit for each pulse may be determined. For example, if the camera 132 captures an image that is one inch long (in the direction of conveyor motion) and it is arranged or determined that the camera 132 has a resolution of 200 dots per inch in that direction, it may be determined that a new image slice is captured every 100 pixels or half an inch since each pulse of the encoder 122 occurs every half an inch. In addition, it may be understood that if each image slice is 200 pixels or one inch long and a new image slice is captured every 100 pixels or half an inch, each new image slice overlaps the previous image slice by 100 pixels or half an inch. It should be noted that in many instances, the terms "dot" and "pixel" may be used interchangeably. It should further be noted that these numbers are merely examples. In embodiments, any value of rotational movement per conveyor 120 run, any value of pulses per revolution, any value of aggregate camera 132 resolution, and any value of camera 132 resolution per unit distance, as well as any combination of rotational movement per conveyor 120 run, pulses per revolution, aggregate camera 132 resolution, and camera 132 resolution per unit distance may be utilized.

[0049] The controller / processor 1200 shown in the embodiment in FIG. 1 may connect to and communicate with the camera 132. The controller / processor 1200 may receive image data from the camera 132 and perform image stitching to form a combined image. The received images may be stored in a buffer for later processing or may be processed immediately. In an embodiment, the controller / processor 1200 may receive a stream of images and use the captured images to determine if an object is present. For example, the controller / processor 1200 may perform background detection on the captured images to determine that an object is present in the viewing zone 121. If the presence of an object in the viewing zone 121 is detected, the controller / processor 1200 may perform image stitching and / or additional image capture. In other embodiments, the controller / processor 1200 may receive a trigger from a sensor 140, 141, such as an electro-optical sensor such as a photo eye, to initiate capturing an image and intermittently flashing a light source. The sensor may include an infrared or visible light emitter 140 and a photoeye including a photocell, photodiode, or phototransistor to detect when the light from the emitter 140 is blocked (indicating the presence of an object) or not blocked (indicating the absence of any object). The controller / processor 1200 may also be connected to the optical encoder 122 to receive object translation data.

[0050] 2 illustrates a schematic view from below of a system, according to an embodiment. In this example, a parcel 110 is partially visible in the viewing zone 121. A single image captured by the camera 132 may not be able to capture the entire bottom surface of the parcel 110. By stitching together several images, a composite image of the entire bottom surface of the parcel 110 can be created.

[0051] FIG. 3 illustrates an exemplary sequence of images 300 acquired by a two-camera sequential stitching system, according to an embodiment. Each strip represents a single image captured by a camera. The left column shows image capture from a first camera, and the right column shows images captured by a second camera. The viewing zone 121 in this case is a wide rectangle. The captured images may overlap from image to image for each camera. In addition, there may be overlap in coverage between the two cameras. In an embodiment, at least an image and an immediately following image may be stitched together to form an output image. In an embodiment, each image in the sequence of images may be successively stitched to a previously stitched image to form a composite image.

[0052] 4 is an illustration of an example stitched image generated by a system according to an embodiment. In this image, the entire surface of the parcel can be seen. Although multiple images were stitched together to create the image, the image appears to be a single capture.

[0053] FIG. 5 illustrates a flow diagram of an embodiment of an image stitching operation. The operations in the flow diagram may be performed, for example, by controller / processor 1200. In an embodiment, process 500 may successively stitch together image slices to form a complete result image of the destination object. The process begins with an input process 501, where image slices may be successively acquired from each camera as described above and stored in a memory buffer, typically located in controller / processor 1200. At 502, the system reads the image slices from the buffer. The system may then perform an alignment process 510 to find a two-dimensional coordinate transformation that aligns the current image slice to the result image. Process 522 may then be used to warp the current image slice and blend it into the result image by using the two-dimensional coordinate transformation. Processes 502-522 may then be repeated for all acquired destination image slices to produce a final result image 524.

[0054] With regard to two-dimensional coordinate transformation, a digital image is a two-dimensional array in which each pixel has a location (coordinates) and an intensity value (color or grayscale level). A two-dimensional coordinate transformation relates locations from one image to the other. Image warping and blending processes apply that two-dimensional coordinate transformation to determine the intensity value at each pixel in the resulting image.

[0055] At process 512, it may be determined whether the image slice read from the buffer is the first slice in a series of acquired slices for an object. If so, process 500 may continue at 514, where an identity two-dimensional coordinate transformation may be applied to the image slice. For example, the identity transformation may lead to copying the image slice as is to the result image at step 522. In other words, for each object, the first image slice may be acquired and copied as is to the result image, which may be an empty result image. When the next image slice is acquired, image warping may be applied to warp the next image slice to the result image in a manner that allows the stitching to be performed correctly.

[0056] If in process 512 it is determined that the image slice read from the buffer is not the first slice in the series of acquired slices, process 500 continues with process 520, where it may be determined that a two-dimensional coordinate transformation should be used to align the two digital images currently being processed, and in process 522, the newly read digital image may be warped by using the found two-dimensional coordinate transformation, and the warped digital image may be blended with the other digital image being processed.

[0057] For example, in some embodiments, the two-dimensional coordinate transformation may be found by extracting features in the form of keypoints from the current image slice. The keypoints may then be used to search for matching keypoints in a second image, which may be represented in this process by a result image including all the previously stitched image slices. In process 520, the corresponding set of matched keypoints may be used to fit a two-dimensional coordinate transformation that aligns the current image slice to the result image. For example, in some embodiments, the two-dimensional coordinate transformation may be determined by extracting a first set of features from the first digital image, extracting a second set of features from the second digital image, matching at least one extracted feature from the first set of features and the second set of features, and determining the image transformation based on the matched features.

[0058] The process of feature extraction may use, for example, histogram of oriented gradients (HOG) features to find keypoints with rich texture. The HOG feature map is an array with n-dimensional feature vectors as entries. Each feature vector describes a local image patch called a tile. The image region of interest (ROI), which may be a region of overlap between two slices, is first divided into non-overlapping tiles of fixed size (e.g., 8 pixels by 8 pixels). Then, for each tile, a one-dimensional histogram of gradient orientations is calculated over its pixels. Gradient magnitudes and orientations are calculated at each pixel, for example, by using a finite difference filter. For color images, the color channel with the largest gradient magnitude is typically used. Then, the gradient orientations at each pixel are quantized into one of “n” orientation bins with voting strengths according to the gradient magnitude to collectively build a histogram of gradient orientations in this tile as a vector of length “n”.

[0059] A brief look at FIG. 6 illustrates an example of extracting keypoints based on HOG features. An image ROI 602 can be divided into non-overlapping tiles to compute HOG features. These tiles can then be grouped into larger regions 603 (dotted lines). For each region 604, as illustrated, every 2 tiles by 2 tiles can be grouped into one patch with a candidate keypoint (the candidate keypoint is at the center of the patch). For example, 606 and 610 are candidate keypoints at the center of 2 tiles by 2 tiles patches 608 and 612, respectively. These patches are overlapping, and each patch is represented by its center point and a score that is a function of the gradient orientation and magnitude determined in the HOG features of that tile. For each region, the patch with the maximum score can be determined and selected as a keypoint if its score exceeds a pre-determined threshold, otherwise this region cannot be represented by any keypoint, which indicates that this region does not have a reliable keypoint that can be used for alignment. This results in a distribution of keypoints across the image ROI, which may be the output from the feature extraction process. The image ROI may be defined as the overlap between the two images to be stitched together. The number and size of the regions may be predetermined based on factors such as image resolution, total image size, number and size of tiles, number and size of patches, etc.

[0060] Returning to FIG. 5, process 520 may then use the extracted keypoints from the current image slice to find matching keypoints in the result image. Referring to FIG. 7a, each keypoint extracted from the image ROI of the new image slice 704 may be used to search for a corresponding keypoint in the result stitched image 702 that includes all the previously stitched image slices. For example, the region represented by 708 in the new slice 704 had patch 714 centered on keypoint 716 as the selected keypoint with the maximum score in this region. The goal of the process is to find the correct match point 724 at the center of the match patch 726 in region 706 in the result image 702. For example, template matching of patch 714 in region 722 in region 706 may be used to find the match patch 726.

[0061] In some embodiments, a new image slice is acquired at every specific predetermined physical distance, which maps through the image resolution to a specific known transformation (typically a conversion such as in a two-dimensional coordinate transformation) that relates every image slice to the previous image slice and thus to the result image.

[0062] In an embodiment, the object may be translated according to a substantially linear motion. Substantially linear motion is motion at a substantially constant speed and in substantially one direction. In an embodiment in which the object is translated at a substantially constant speed, a new image slice may be acquired at a certain predetermined physical distance by acquiring images at a substantially fixed time interval. The speed may be considered substantially constant and the time interval may be considered substantially fixed if the change in speed and interval is small enough that the resulting change in acquisition distance is small enough that stitching and other processes described herein can still be successfully performed. If the line along which the object moves with respect to the point or region to which the line is oriented is constant enough that stitching and other processes described herein can still be successfully performed, the motion may be considered substantially one direction. One example of linear motion may be an object that may be moved on a straight (linear) conveyor. However, this is only an example. The present technology is equally applicable to any embodiment in which motion occurs substantially in one direction. In embodiments in which an encoder is provided, new image slices may be acquired at certain predetermined physical distances based on distance information provided by the encoder.

[0063] A transformation (typically a conversion) that maps to a particular known transformation through image resolution may be referred to herein as an approximate two-dimensional coordinate transformation, since it is based on an ideal acquisition setting. In a practical setting, this two-dimensional coordinate transformation may map a key point 716 in the new image slice 704 to a point 718 in the result image 702 that may not be a perfect match. Such a mismatch may be due to, for example, vibration of the object or a slight delay in the camera acquisition. A relatively small search space 722 may then be centered on the point 718, and template matching may be performed only in this search space to find the correct match 724 for the key point 716. In an embodiment, the mismatch may be small enough, or the stitching accuracy may be high enough, to allow decoding of the mark on the object, or to allow further processing or analysis of the image. For example, marks on the object that may be decoded may include one-dimensional or two-dimensional barcodes, etc. Similarly, text on the object may be recognized, or other features of the object or other features on the object may be recognized or analyzed. In addition, the approximate 2D coordinate transformation may be used directly without refinement in the case of objects that do not have features to use for refinement, such as, for example, a plain cardboard box. In this case, images that do not have sharp edges do not require perfect alignment, so the approximate 2D coordinate transformation may be good enough to be used for alignment.

[0064] It should be noted that the stitching and decoding processes may be performed by using sequential processing, parallel processing, or some combination of the two. For example, in an embodiment, the stitching of images may be performed in parallel. For example, each of the multiple pairs of images may be stitched in parallel. Similarly, in an embodiment, the stitching of images may be performed in parallel with the decoding of marks on the object, or in parallel with other further processing of the images of the object. Such parallel stitching, decoding, and / or other processing may be performed by using any number of known parallel processing techniques.

[0065] FIG. 7b illustrates an important case related to the extraction of keypoints. Region 712 in image 704 illustrates an example of a bad keypoint 728. Patch 730 centered on point 728 has strong edges, but the edges all have the same orientation. If an approximate two-dimensional coordinate transformation is applied to keypoint 728, the transformation maps it to point 732, which is very close to the correct match, but a search in search space 736 can result in multiple keypoints (e.g., 738 and 742) that perfectly match 728. This explains why the selection of keypoints is based on a score using HOG features to ensure that the patch not only has strong edges, but also different orientations. This means that a candidate keypoint such as 728 will not be selected as a keypoint (because the keypoint with the maximum score does not exceed the selection threshold), and furthermore, regions such as 712 will not have a keypoint extracted from it. In an embodiment not shown in FIG. 7, features to be matched can be extracted from each region and then matched. For example, the matching method can be one of matching corresponding keypoints by using a least squares method, or finding the best corresponding positions of keypoints by using known alignment methods such as normalized correlation.

[0066] Returning to FIG. 5, process 520 may then use the calculated set of corresponding keypoints to fit a two-dimensional coordinate transformation that aligns the new image slice to the result image. For example, a random sample consensus (RANSAC) process may be used to fit an accurate transformation that removes any outliers resulting from the matching problem. In process 522, the image slices to be stitched together may be warped by using the determined two-dimensional coordinate transformation. The two-dimensional coordinate transformation changes the spatial configuration of the images. Herein, the two-dimensional coordinate transformation may be used to correct spatial inconsistencies in the images or image slices to be stitched together. For example, the two-dimensional coordinate transformation may be used to align the two images so that they are ready to be blended. Preferably, all overlapping pixels may be aligned to the exact same location in the two images. However, even if this is not possible, the two-dimensional coordinate transformation may provide sufficient alignment for the blending to be successful.

[0067] If at least a portion of the image slices to be stitched overlaps the image being stitched, the overlapping portions may be blended to form the resulting image. In some embodiments, the slices are roughly aligned with a translation in the direction of movement. This divides every new slice into an overlapping region and a new region as illustrated in FIG. 7. The blending of overlapping pixels may use a weighted average that allows for a seamless transition between the resulting image and the new image slices. For example, the upper portion of the overlapping region may give a higher weight to the resulting image (which is gradually reduced to the lower portion of the overlapping region), which may give a higher weight to the new slices at the edge of the overlapping region such that there are no seam lines between the overlapping region and the new region of each new slice. In embodiments, other types of blending may be utilized.

[0068] After completing process 520 for the current slice, at 524, process 500 loops back to 502 to obtain another slice to be processed. If there are no more slices to be processed, at 524, a stitched image may be output. It should be noted that in an embodiment, the output image may include only the two slices being stitched together, or may include all slices in the sequence of image captures being stitched together. It should be noted that in an embodiment where multiple cameras are used, such as the example shown in FIG. 10, the system creates image slices that overlap between the two cameras 133a and 133b, and also overlap in the translation of the parcel along the conveyor 120. In such an embodiment, images captured by more than one camera may be stitched together as well. For example, an image slice from each camera may be stitched together with other image slices from that camera, and the resulting stitched images from each camera of the multiple cameras may be stitched together to form a final complete image. In some embodiments, each image slice from a camera may be stitched together with image slice(s) captured substantially simultaneously by other camera(s) to form a combined image slice, and the combined image slices may be stitched together to form a complete image. In some embodiments, each image slice acquired from each camera may be stitched directly into the same resulting image of the previously stitched images from all cameras. The stitching process that may be used to stitch images or image slices from different cameras is similar to the process described above for stitching image slices from the same camera.

[0069] FIG. 8 illustrates an exemplary calibration plate 800 that can be used to automate calculations for different parameters of the system. The plate can include a checkerboard pattern that includes a plurality of alternating black and white squares or checkers 802. These alternating squares can have fixed dimensions, such as 10 mm by 10 mm. These fixed dimensions can provide the ability to determine the image resolution at a particular fixed working distance. In addition, the calibration plate or pattern 800 can include data matrix fiducials or patterns 804. These data matrix fiducials or patterns 804 can encode physical details about the plate, such as the exact physical dimensions of the checkers and the reference of the coordinates of each point with respect to a fixed physical coordinate system defined on the plate. In addition, it can also be used to determine the presence or absence of a mirror in the optical path between the camera and the object.

[0070] The setup process may be used to initialize the acquisition settings illustrated in FIG. 1. This process may be used to configure and calibrate the hardware and software used to perform the stitching process 500. The setup process begins by acquiring at least one image similar to the image illustrated in FIG. 9 for a stationary calibration plate placed over the viewing window. Region 121 in FIG. 9 corresponds to region 121 representing the viewing zone in FIG. 1. The checker corners are extracted from the image and the Data Matrix code is decoded, which provides an accurate set of pairs of points in the image domain and corresponding locations in the physical domain. This set may be used to automatically: Find the camera's field of view that defines the portion of the captured image 902 that corresponds to the viewing zone. The image sensor can be set to capture only that portion, which in many sensors is a significant factor in capture speed. - Determining whether a view in the effective field of view is a perspective view or a non-perspective view. In many embodiments, perspective can be considered a form of distortion of the image being captured. Therefore, for setup purposes, it can be advantageous to ensure that the image does not contain such perspective distortion, either by physically correcting the setup or by warping the image post-acquisition to correct for perspective effects. - Calculate the image resolution in dpi at this working distance. Then, after the setup process, the system can be locked in place and the orientation of all components can be fixed, including the orientation of the camera with respect to the viewing zone that defines the perspective.

[0071] In the stitching algorithm 500, and particularly in the matching process that is part of process 520, an approximate two-dimensional coordinate transformation is referenced, which can expedite the matching process by allowing a small search space for matching each keypoint. In some embodiments, this approximate two-dimensional coordinate transformation can be a simple translation in the direction of movement, which can be calculated by multiplying the physical distance between successive acquisitions by the image resolution calculated in the setup process. Such a transformation can be determined based on a translation that is a substantially linear motion. As explained above, a motion can be considered to be a substantially linear motion if the motion is at a substantially constant speed and in substantially one direction.

[0072] For example, if an encoder is used to trigger the acquisition of image slices and is set to produce 24 pulses per resolution (one resolution is 12 inches), this results in a new image slice being acquired every half inch the object moves. If the image resolution is calculated as 200 dpi in the setup process, then the approximate two-dimensional coordinate transformation is a simple 100 pixel translation in the direction of movement. In embodiments, any value of rotational movement per progression of the conveyor 120, any value of pulses per resolution, and any value of resolution, as well as any combination of rotational movement per progression of the conveyor 120, pulses per resolution, and resolution may be utilized.

[0073] In some embodiments, the relationship of individual image slices (from which an approximate two-dimensional coordinate transformation is estimated) may be established by using a training process that uses image slices acquired for a calibration moving object having a known pattern of features, such as the calibration plate or pattern 800 shown in Figure 8. In this training process, an approximate two-dimensional coordinate transformation relating successive image slices may be determined by analyzing data matrix fiducials 804 and checkers 802 acquired across successive image slices.

[0074] FIG. 10 illustrates a top-down schematic with a two-camera system 133, according to an exemplary embodiment. Although two cameras are shown in this example, the embodiment may include more than two cameras, and the techniques described in relation to two cameras may be adapted to three or more cameras. The system in this figure may include a conveyor 120, two cameras 133a, 133b, a light source 131, and a controller / processor 1200. The conveyor 120 translates an object 110, such as a parcel, across the conveyor 120, which may include a viewing zone 121, which may include an opening, window, gap, etc. The two cameras 133a, 133b may capture images through the viewing zone 121 via a mirror 135. For example, more than two cameras may be used if the width (perpendicular to the direction of travel of the conveyor 120) is too wide to be captured by one camera, or is too wide to be captured with adequate image quality by one camera. The embodiments may be applicable to any number of cameras. Examples of such embodiments are described in more detail below. In embodiments, the cameras 133a, 133b may be configured to capture images substantially simultaneously (e.g., within the timing precision of electronic circuitry that causes the cameras 133a, 133b to function simultaneously).

[0075] In this example, the system produces image slices that overlap between the two cameras 133a, 133b and also overlap in the translation of the parcel along the conveyor 120. In such an embodiment, images captured by more than one camera may be stitched together as well. In an embodiment, the use of a mirror 135 allows the cameras 133a, 133b to be positioned such that they do not have to directly face each other in the viewing zone 121. The controller / processor 1200 may be arranged to cause the cameras 133a, 133b to capture images simultaneously or at different times.

[0076] An example of an embodiment in which the side of an object may be imaged is shown in FIG. 11. In this example, an object 110 may be translated on a conveyor 120 to image the side of the object. In this example, the side of the object is parallel to a known plane of the side 1104. The known plane of the side 1104 may be implemented, for example, by using a sidewall attached to the conveyor 120. Such a sidewall may be made of a transparent material such as glass or clear plastic, as shown, or the sidewall may be made of an opaque material such as metal. In an embodiment, such a sidewall may be high enough that a viewing zone 121, such as a window, gap, or opening, must be included in the known plane of the side 1104. In an embodiment, the camera 132 may be positioned or oriented to image the object at or through the viewing zone 121. In an embodiment, such a sidewall may be low enough that the side of the object may be directly imaged without the use of a window, gap, or opening. In an embodiment, the alignment device 1102 may be used to ensure that the side of the object 110 is parallel or aligned with the side known plane 1104. In an embodiment, the alignment device 1102 may be implemented using a mechanical device, such as, for example, a spring-loaded flap as shown in this example, that applies pressure to the object 110 to align the side of the object 110 parallel to the side known plane 1104. In an embodiment, the alignment device 1102 may be implemented using, for example, an electromechanical device that applies pressure to the object 110 to align the side of the object 110 parallel to the side known plane 1104.

[0077] 12 illustrates a schematic diagram of components of a controller / processor 1200, according to an embodiment. The controller / processor 1200 includes an input / output interface 1204 for receiving images from a camera. The input / output interface 1204 may also be connected to the encoder 122 or the light source 131 for controlling the encoder 122 or the light source 131. Input / output devices (including but not limited to keyboards, displays, pointing devices) may be directly coupled to the system or may be coupled to the system through an intervening input / output controller.

[0078] The controller / processor 1200 may have one or more CPUs 1202A. The controller / processor 1200, in an embodiment, has network capabilities provided by a network adapter 1206 that connects to a communications network 1210. The network adapter may also be coupled to other data processing systems or storage devices through intervening private or public networks. The network adapter 1206 allows software and data to be transmitted between the controller / processor 1200 and external devices. Examples of network adapters may include modems, network interfaces (such as Ethernet cards), communications ports, or PCMCIA slots and cards, and the like. The software and data transferred through the network adapter are in the form of signals, which may be, for example, electrical, electromagnetic, optical, or other signals capable of being received by the network adapter. These signals are provided to the network adapter through a network. The network carries the signals and may be implemented using wire or cable, fiber optics, a phone line, a cellular phone link, an RF link, and / or other communications channels.

[0079] The controller / processor 1200 may include one or more computer memory devices 1208 or one or more storage devices. The memory device 1208 may include a camera data capture routine 1212 and an image stitching routine 1214. An image data buffer 1216 is also included in the memory device, as is an operating system 1218.

[0080] The routines provided for the invention take the form of a computer program product, in embodiments, accessible from a computer usable or computer readable medium (which provides program code for use by or in connection with a computer or any instruction execution system). For purposes of this description, a computer usable or computer readable medium can be any apparatus that can contain, store, communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.

[0081] The medium can be an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system (or apparatus or device), or a propagation medium. Examples of a computer-readable medium include a semiconductor or solid-state memory, a magnetic tape, a removable computer diskette, a random access memory (RAM), a read-only memory (ROM), a rigid magnetic disk, or an optical disk.

[0082] A data processing system suitable for storing and / or executing program code includes at least one processor coupled directly or indirectly to memory devices through a system bus. The memory devices can include local memory used during actual execution of the program code, mass storage, and cache memory that provides temporary storage of at least some of the program code to reduce the number of times the code must be read from mass storage during execution.

[0083] Various software embodiments are described in terms of this exemplary computer system. After reading this description, it will become apparent to one of ordinary skill in the relevant art how to implement the invention using other computer systems and / or computer architectures.

[0084] Controller / processor 1200 may include a display interface (not shown) that transfers graphical, textual, and other data (such as from a frame buffer (not shown)) for display on a display unit.

[0085] In this disclosure, the terms "computer program medium," "computer usable medium," and "computer readable medium" are used generally to refer to media such as main memory and storage memory, removable storage devices, and hard disks installed in hard disk drives.

[0086] Computer programs (also called computer control logic) are stored in main memory and / or storage memory. Such computer programs, when executed, enable the computer system to implement the features of the present invention as discussed herein. In particular, the computer programs, when executed, enable one or more processors to perform the operations described above.

[0087] From the above description, it can be understood that the present invention provides a system, computer program product, and method for efficient execution of image stitching. In the claims, the singular reference of an element is not intended to mean "only one" unless expressly stated, but rather "one or more." All structural or functional equivalents of the elements of the exemplary embodiments described above that are now known or that later become known to those skilled in the art are intended to be encompassed within the scope of the claims. No claim element herein should be construed under the provisions of 35 U.S.C. Section 112, paragraph 6, unless the element is expressly recited using the phrase "means for" or "step for."

[0088] While the above description of the invention enables one skilled in the art to make and use what is currently believed to be the best mode thereof, one skilled in the art will understand and recognize the existence of alternatives, modifications, variations, combinations, and equivalents of the specific embodiments, methods, and examples herein. One skilled in the art will recognize that what is disclosed is merely exemplary, and that various modifications may be made within the scope of the invention. In addition, while a particular feature of the teachings may be disclosed with respect to only one of several implementations, such feature may be combined with one or more other features of other implementations as may be desired and advantageous for any given or specific function. Furthermore, to the extent that the terms "comprise", "include", "have", "have", "have", or variations thereof are used in any of the detailed description and claims, such terms are intended to be inclusive in a manner similar to the term "comprise".

[0089] Other embodiments of the teachings will be apparent to those skilled in the art from consideration of the specification and practice of the teachings disclosed herein. Accordingly, the invention should not be limited by the described embodiments, methods, and examples, but by all embodiments and methods within the scope and spirit of the invention. Accordingly, the present invention is not limited to the specific embodiments as exemplified herein, but is limited only by the scope of the following claims.

Claims

[Claim 1] The invention described in this specification.

Images