Global camera path optimization

Abstract

A method for three-dimensional data reconstruction comprising: capturing multiple individual images of image data of an object surface, wherein each of the multiple individual images of image data is captured from a camera position along a camera path, and each of the multiple individual images of image data comprises a conventional image of the object surface as seen from the camera position and data for a three-dimensional reconstruction of the object surface as seen from the camera position;Selecting a subset of the multiple frames of image data to provide multiple key frames, wherein each of the multiple key frames relates to at least one other of the multiple key frames by a section of the camera path including a first rotation and a first translation determined based on one or more common points in the three-dimensional reconstruction of the object surface in each of the respective key frames, wherein the remaining of the multiple frames of image data are non-key frames; providing a three-dimensional model of the object surface;Determining a second rotation and translation from one of the multiple key frames to at least one of the non-key frames, positioned sequentially between the one of the multiple key frames and a sequentially adjacent one of the multiple key frames; obtaining three-dimensional reconstruction information of the object surface from the camera position of at least one of the non-key frames to provide oversampled three-dimensional data; and adding the oversampled data to the three-dimensional model based on the second rotation and translation, wherein providing the three-dimensional model further comprises generating the three-dimensional model of the object surface using the camera path and three-dimensional reconstruction for each of the multiple key frames.

Description

The present application claims priority from US Provisional Patent Application No. 61 / 019,159, filed January 4, 2008, which is incorporated herein by reference in its entirety. Field of Invention The present invention relates generally to three-dimensional imaging, and more particularly to optimizing the computation of a global camera path used in a three-dimensional reconstruction. background In a three-dimensional image reconstruction method, a number of images or image sets of an object are captured by a camera detected, traveling in a path across the surface of the object. Information from this catalog of information can then be used to a three-dimensional one based on the camera path and individual three-dimensional measurements acquired along the camera path to reconstruct the model of the object. The path of the camera can be very long and complex, requiring motion estimation from frame to frame conditional, which accumulates significant errors along its length. These errors can result in a variety of reconstruction image errors all in one resultant three-dimensional model, such as duplicate surfaces when the camera path enters the same area with an error of camera position between the two scans. An error can also be due to calibration problems, inaccuracies in camera distortion models used to determine three-dimensional data, and so on. While there are a variety of techniques for minimizing errors along an entire camera path, there remains a need for improved ones global path optimization techniques appropriate for use with the data-intensive nature of high-fidelity three-dimensional reconstruction path optimizations are suitable. summary

[0005] Various techniques are disclosed herein for improving global path optimization in a system that calculates the camera path for used the three-dimensional reconstruction. A subset of individual images or "frames" of data, also hereinafter referred to as "Frame" denotes, for the global path, the keyframes or key single images or keyframes can be used to reduce the computational complexity of the optimization while preserving all three-dimensional detail in the optimized model remains by relating other measurements to the optimized keyframe path. In one aspect, a method for three-dimensional reconstruction disclosed herein includes acquiring multiple frames of image data of a surface of an object captured from a camera position along a camera path, and incorporating a conventional image of the object from the camera position and data for a three-dimensional reconstruction of the object's surface; the Selecting a subset of frames of image data to provide multiple keyframes, each through a section of the camera path including a rotation and a translation based on one or more common points in the three-dimensional reconstruction is related to at least one other key frame, the remaining multiple frames of image data are non-key frames; providing a three-dimensional model of the object; the determining a second rotation and a second translation from one of the key frames to a non-key frame sequentially between positioned at one of the key frames and a sequentially adjacent key frame; the winning of three-dimensional Reconstruction information of the object surface from the camera position of at least one of the non-key frames to oversampled provide three-dimensional data; and adding the oversampled three-dimensional data to the three-dimensional model based on the second rotation and the second translation. A camera movement can be based on the rotation and the translation between two adjacent key frames are estimated. It can optimize the estimation of camera movement between two adjacent ones Giving keyframes by creating consistency between motion parameters using an overdetermined system of constraint equations of motion, where the motion parameters consist of rotational and translational information. It can also give an optimization of the camera movement between two adjacent non-key frames by using an overdetermined System of equations of motion constraint consistency between movement parameters is generated. The three-dimensional reconstruction can be based on the generated consistency between the movement parameters are generated. The data for a three-dimensional reconstruction of the object surface can consist of at least another channel image to provide disparity data. The three-dimensional model can also generate a three-dimensional model of the object using the camera path and the three-dimensional reconstruction for each of the key frames exhibit. Three-dimensional object surface reconstruction information can be obtained from the camera position for all of the non-key frames can be obtained between two adjacent key frames. A subset of the multiple frames can be based on a quality metric of the three-dimensional reconstruction can be selected. Selecting the subset of the multiple frames can be done using a graphical analysis to ensure that all of the key frames are used in the three-dimensional reconstruction. In one aspect, a computer program product disclosed herein performs the steps of: Acquiring multiple frames from Image data of an object surface, each of the plurality of frames of image data being acquired from a camera position along a camera path and each of the multiple frames of image data includes a conventional image of the object from the camera position and data for a three-dimensional reconstruction of the object surface seen from the camera position; selecting a subset of the multiple frames of image data in order to to provide a plurality of key frames, each of the plurality of key frames spanning a portion of the camera path including a rotation and a translation; which are based on one or more common points in the three-dimensional reconstruction of the object surface are determined in each of the respective key frames relates to at least one other of the plurality of key frames, wherein the remaining of the plurality of frames of image data are non-key frames; providing a three-dimensional model of the object; determining a second rotation and a second translation from one of the key frames to at least one of the non-key frames, positioned sequentially between the one of the key frames and a sequentially adjacent one of the key frames; win the three-dimensional reconstruction information of the object surface from the camera position of at least one of the non-key frames provide oversampled three-dimensional data; and adding the oversampled three-dimensional data to the three-dimensional Model based on the second rotation and the second translation. In one aspect, a method for interactively reducing accumulated error in a global path is disclosed herein, which comprises: acquiring a plurality of frames of image data of an object surface, each of the plurality of frames of image data from a camera position is captured along a camera path and each of the multiple frames of image data is a conventional image of the object from the camera position and data for a three-dimensional reconstruction of the object surface as seen from the camera position; Creating a three-dimensional modeling the object using the camera path and the data for the three-dimensional reconstruction; Identify two of the multiple frames of image data representing a candidate for an accumulated error in the camera path, relative to one another; and display the three-dimensional model along with a graphical annotation representing a recommended scanning path to account for the accumulated error reduce. The data for a three-dimensional reconstruction of the object surface can be obtained from at least one other channel image, to provide disparity data. Acquiring one or more frames of image data along the recommended scan path can be performed to reduce the accumulated error. Two of the multiple frames can be identified, further identifying frames of image data separated along the camera path by a much greater distance than along the object surface. In one aspect, there is provided a computer program product having computer-executable code embodied on a computer-readable medium is executed, disclosed, which performs the following steps: acquiring multiple frames of image data of an object surface, each of the a plurality of frames of image data is acquired from a camera position along a camera path, and each of the plurality of frames of image data a conventional image of the object from the camera position and data for a three-dimensional reconstruction of the object's surface from the Camera position seen includes; Generating a three-dimensional model of the object using the camera path and the data for the three-dimensional reconstruction; Identifying two of the multiple frames of image data that are a candidate for an accumulated error in represent the camera path, relative to each other; and displaying the three-dimensional model along with a graphical annotation representing a recommended scan path to reduce accumulated error. In one aspect, a system is disclosed herein that includes a camera, a monitor, a processor, and comprises a memory, the memory storing a computer program executable by the processor to perform the following steps perform: acquiring multiple frames of image data of an object surface, each of the multiple frames of image data consisting of a Camera position is detected along a camera path and each of the multiple frames of image data is a conventional image of the object from the camera position and data for a three-dimensional reconstruction of the object surface as seen from the camera position; Generating a three-dimensional model of the object using the camera path and the data for the three-dimensional reconstruction; Identify from two of the plurality of frames of image data representing a candidate for an accumulated error in the camera path relative to one another; and displaying the three-dimensional model along with a graphical annotation representing a recommended scan path around the reduce accumulated errors. In one aspect, there is disclosed herein a method for global path optimization, comprising: acquiring multiple frames of image data Object surface, wherein each of the plurality of frames of image data is captured from a camera position along a camera path and each of the several frames of image data, a conventional image of the object from the camera position and data for a three-dimensional reconstruction the object surface seen from the camera position; and minimizing an error function for multiple camera positions along the Camera path, the error function being a system with equations for translational components of an error and for rotational components of the error, where the error function contains the translational components and the rotational components using a weight matrix couples, providing an optimized camera path. The data for a three-dimensional reconstruction of the object surface can obtained from at least one other channel image to provide disparity data. The system of equations can be non-linear be a system of equations. The translational component of the error can form a system of non-linear equations. The rotational component of error can form a system of non-linear equations. A three-dimensional model can be created based on the camera path and the data for the three-dimensional reconstruction can be generated and the three-dimensional model can be refined based on the optimized camera path will. A subset of the multiple frames of image data may be selected to provide multiple key frames, each of the multiple key frames through a portion of the camera path including a rotation and a translation based on one or several common points in the three-dimensional reconstruction of the object surface in each of the respective key frames are related to at least one other key frame, with the remaining of the multiple frames of image data being non- are key frames. The weight matrix can be chosen to decouple the error function around a centroid of common surface data for two or more three-dimensional reconstructions. An error function can be minimized to a calibration condition based on the resulting error function minimization. In one aspect, there is disclosed herein a computer program product having computer executable code that performs the following steps performs: acquiring multiple frames of image data of an object surface, each of the multiple frames of image data from a camera position is captured along a camera path and each of the plurality of frames of image data is a conventional image of the object from the camera position and comprises data for a three-dimensional reconstruction of the object's surface as seen from the camera position; and minimizing an error function for multiple camera positions along the camera path, where the error function is a system of equations for translation components of a error and for rotational components of the error, where the error function contains the translational components and the rotational components couples using a weight matrix, thereby providing an optimized camera path. Brief description of the drawings The invention and the following detailed description of certain of its embodiments are to be understood with reference to the following figures to understand. Figure 1 shows a three-dimensional scanning system. 2 shows a schematic representation of an optical system for a three-dimensional camera. Figure 3 shows a processing pipeline for acquiring three-dimensional data from a video camera. Figures 4A and 4B depict camera paths for a three-dimensional camera. Figure 5 shows a user interface image where additional data is requested from a software tool. Figures 6A and 6B depict accumulated errors in camera paths. Figure 7 is a flowchart of a three-dimensional reconstruction method including global path optimization for improved Accuracy. Detailed description In the following text, references to singular elements should be understood to include plural elements and vice versa belong unless otherwise expressly stated or apparent from the text. Grammatical conjunctions should be any and all disjunctive and conjunctive combinations of linked clauses, sentences, words, etc., unless otherwise stated or from the context. In the systems and methods described herein, a number of global motion optimization techniques are used to Improve accuracy of three-dimensional reconstructions based on camera path. The following description explains in detail scanning technologies and focuses on three-dimensional dental applications imaging; however, it will be apparent that variations, adaptations, and combinations of the methods and systems below can be used by people become apparent with ordinary knowledge of the art. For example, while describing an image-based system, non-image based scanning techniques, such as time-of-flight or structured light techniques using patterned projections, in similarly, use camera path-based reconstruction, which can benefit from the advantages described here. while as a Another example of digital dentistry is a useful application of the improved accuracy resulting from the techniques described here results, global path optimization can also be usefully used to create three-dimensional animation models or three-dimensional Scans for computer vision applications or for imaging applications. All such variants, adaptations and Combinations are intended to fall within the scope of this disclosure. In the following description, the term "image" generally refers to a two-dimensional set of pixels that form a two-dimensional view of an object in an image plane. The term "image set" generally refers to one Set of related two-dimensional images that could be resolved into three-dimensional data. The term "point cloud" generally denotes a three-dimensional set of points that form a three-dimensional view of the object, consisting of a number two-dimensional images is reconstructed. A number of such point clouds can also be registered in a three-dimensional image acquisition system and combined into an aggregate point cloud built up from images captured by a moving camera. Thus will It should be understood that pixels generally refer to two-dimensional data and dots generally refer to three-dimensional data if none other meaning is specifically indicated or is clear from the context. The terms “three-dimensional model”, “three-dimensional surface representation”, “digital Surface representation”, “three-dimensional surface map” and the like, as used herein, mean any three-dimensional denote a surface map of an object, e.g. a point cloud of surface data, a set of two-dimensional polygons, or any other data representative of all or some surfaces of an object obtained via the acquisition and / or processing of three-dimensional scan data unless a different meaning is specified or otherwise clear from the context. A "three-dimensional Representation” can be any of the three-dimensional surface representations previously described, as well as volumetric and other representations unless a different meaning is specified or otherwise clear from the context. In general, the terms “rendering” or “rendering” or “playback” or "Representation" a two-dimensional visualization of a three-dimensional object, e.g. B. for display on a monitor. However it will be understood that various three-dimensional rendering technologies exist and with the systems and methods disclosed herein can be usefully used. For example the systems and methods described herein may include a holographic display, an autostereoscopic display, an anaglyph display, a head mounted stereo display or any other two dimensional and / or three dimensional display. So should Rendering in the description herein shall be interpreted broadly unless a different meaning is indicated or otherwise taken out of context emerges. As used herein, the term "dental object" is intended to refer generally to objects associated with dentistry related. This may include intraoral structures, e.g. B. the teeth and more typically the human teeth, z. B. single teeth, Quadrants, full arches, pairs of arches (which can be separate or in occlusal positions of different types), soft tissue and the like, as well as bone and all other supporting or surrounding structures. As used herein, the term "intraoral structures" refers to both natural Structures in a mouth as described above as well as artificial structures, e.g. B. any of those described below Dental objects that may be present in the mouth. Dental objects may include "restorations," which are commonly so understood that may include components that restore the structure or function of the existing dentition, e.g. B. crowns, bridges, facets, inlays, onlays, amalgams, composites and various substructures, e.g. e.g. stump caps and the like, as well as temporary restorations for use during the fabrication of a permanent restoration. Dental objects may also include a "prosthesis" that Dentures replaced by removable or permanent structures, e.g. B. dentures, partial dentures, implants, held prostheses, etc. Furthermore, can Dental objects also include “appliances” or “equipment” that are used for correcting, aligning or otherwise temporary or permanent adjustment of the bit, e.g. B. removable orthodontic appliances, surgical Stents, bruxism appliances, snoring splints, appliances for indirect bracket placement, etc. In addition, dental objects “Small parts” belong that are attached to the bit for a long time, e.g. B. Implant holders, implant abutments, orthodontic Brackets and other orthodontic components. "Transitional components" from dental technology can also be used for dental objects Manufacturing count, e.g. B. Tooth models (complete and / or partial), wax-ups, investment molds, etc., as well as shells, bases, molds and other components used in the manufacture of restorations, prostheses, etc. Dental objects can also be categorized as natural dental objects, e.g. B. the teeth, bones and other intraoral structures previously described, or as artificial dental objects, z. B. Restorations, prostheses, appliances, small parts and transition components from dental technology production as described above. Such terms as z. B. "digital dental model", "digital dental impression" etc. should be three-dimensional Denote representations of dental objects used in various aspects of acquisition, analysis, prescription, and manufacture unless a different meaning is indicated or clear from the context. Such terms as B. "Dental model" or "dental impression" is intended to denote a physical model, e.g. B. a cast, printed or otherwise manufactured physical expression of a dental object. Unless otherwise specified, the term "model" may be used alone Use to denote a physical model and / or a digital model. In addition, it will also be understood that such terms as e.g. B. ”tool” or ”control” in their Used to describe aspects of a user interface in general to denote a variety of techniques used on a graphical User interface or other user interface can be used to receive user input, the processing trigger or control, including e.g. B. Drop-down lists, radio buttons, cursor and / or mouse actions (select by point, select by area, Drag-and-drop operations, etc.), checkboxes, command lines, text entry fields, messages and alarms, progress indicators, etc. Into a Tool or control can also include any physical hardware related to user input, e.g. B. a mouse, a keyboard, a display, keypad, trackball, and / or any other device that receives physical input from a user and the physical Converts input to input for use in a computerized system. Thus, in the following description, the terms "Tool," "control," and the like are to be interpreted broadly unless a more specific meaning is otherwise provided is or is evident from the context. Figure 1 shows a three-dimensional scanning system used with the systems and methods described herein can. In general, the system 100 may include a camera 102 that captures images of a surface 106 of an object 104, e.g. B. one dental patient, and forwards the images to a computer 108, which has a display 110 and one or more user input devices 112, 114 may have, e.g. a mouse 112 or a keyboard 114 . The camera 102 can also have an integrated input or output device 116, e.g. B. a control input (e.g. button, touchpad, thumbwheel) or a display (e.g. LCD or LED display) to provide status information. Camera 102 may include any camera or camera system suitable for capturing images from which a three-dimensional point cloud or other three-dimensional data can be recovered. For example, camera 102 can Multiple aperture system as disclosed in US-A-7372642 (Rohály et al.), the entire contents of which are incorporated herein by reference is recorded. While Rohály discloses a multi-aperture system, it will be appreciated that any multi-aperture system can similarly be used, which is suitable for reconstructing a three-dimensional point cloud from a number of two-dimensional images. In a multiple aperture embodiment, the camera 102 may have multiple apertures, including a center aperture positioned along a central optical axis of a lens that provides a central channel for the camera 102 along with any associated imaging hardware forms. In such embodiments, the center channel may provide a conventional video image of the object being scanned while a number axially displaced channels results in image sets containing disparity information, which is used in the three-dimensional reconstruction of a surface for can be used. In other embodiments, a separate video camera and / or channel may be provided to to achieve the same result, i. H. a video of an object corresponding in time to a three-dimensional scan of the object, preferably from the same perspective or from a perspective with a fixed, known relationship to the camera 102 perspective. The camera 102 may also or instead have a stereoscopic, trioscopic or other multiple camera or other configuration in which a A number of cameras or optical paths are kept in fixed relation to one another in order to produce two-dimensional images of an object Get a number of different perspectives. The camera 102 may perform appropriate processing to derive a three-dimensional Point cloud from an image set or a number of image sets, or each two-dimensional image set can be sent to an external processor are sent, the z. B. contained in the computer 108 described later. In other embodiments, the camera 102 may be structured Use light, laser scanning, tachymetric distance measurement or any other technology used to collect three-dimensional data or two-dimensional data that can be resolved into three-dimensional data. Although the techniques described later be able to profitably use video data acquired by a video-based three-dimensional scanning system, it will be understood that any other three-dimensional scanning system can be supplemented with a video acquisition system containing appropriate video data or image data concurrently or otherwise synchronized with the acquisition of three-dimensional data.

[0032] In one embodiment, the camera 102 is a freely positionable handheld probe having at least one user input device 116, e.g. a button, lever, dial, thumbwheel, switch, or the like for user control of the imaging system 100, e.g. B. to start and stop samples. In one embodiment, the camera 102 can be shaped and sized for dental scanning. In particular, camera 102 may be shaped and sized for intraoral scanning and data collection, e.g. B. by insertion into a mouth of an imaging object and traversing an intraoral surface 106 at a suitable distance to obtain surface data of teeth, gums etc. to capture. Via such a continuous data acquisition method, the camera 102 can include a point cloud of surface data sufficient spatial resolution and accuracy to detect dental objects, e.g. e.g. prosthetics, small parts, apparatus etc. from it, either directly or via various intermediate processing steps. In other embodiments, surface data from a dental model, e.g. B. a dental prosthesis, to determine the correct adjustment with the help of a previous scan of the corresponding dentition, z. B. one for tooth surface prepared for the prosthesis. Although not shown in Figure 1, it will be appreciated that a number of supplemental illumination systems are used during image acquisition can be put to good use. For example, the ambient lighting can be enhanced with one or more spotlights, that illuminate the object 104 to speed up image acquisition and improve depth of field (or spatial depth of resolution). In addition or alternatively, the camera 102 may include a strobe, flash, or any other light source to provide illumination of the To supplement object 104 during image acquisition. Object 104 can be any object, collection of objects, portion of an object, or any other entity. In particular, with regard to the dental techniques discussed herein, object 104 may include human dentition consisting of a mouth of a dental patient is detected intraorally. A scan can partially or completely correspond to a three-dimensional representation of the dentition a specific purpose of scanning. This allows the scan to create a digital model of a tooth, a quadrant of teeth, or a full cluster of teeth with two opposing arches plus soft tissue or any other relevant intraoral structure capture. The scan can capture multiple representations, e.g. B. a tooth surface before and after preparation for a re restoration. As mentioned later, this data can be used for subsequent modelling, e.g. B. the design of a restoration or the determination of a margin line for it. During the scan, a center channel of the camera 102 or a separate Video system capture a video of the teeth from the viewpoint of the camera 102 from. In other embodiments where e.g. Legs completed fabrication is virtually test fitted to a surface preparation, the scan may comprise a dental prosthesis, e.g. B. a Inlay, a crown or any other dental prosthesis, small dental parts, a dental appliance, etc. The object 104 can also or instead of a Have dental model, z. B. a plaster cast, a wax-up, an impression or a negative impression of a tooth, teeth, soft tissue or some combination of these. The computer 108 can e.g. B. comprise a personal computer or other processing device. In one embodiment Computer 108 includes a personal computer with a dual 2.8 GHz Opteron CPU, 2 gigabytes of read / write memory, a TYAN Thunder K8WE motherboard and a 250 gigabyte 10,000 rpm hard drive. In a current embodiment, the system can be operated in this way be that it acquires more than five thousand points per image set in real time using the techniques described herein and an overall point cloud stores from several million points. Of course, this point cloud can be further processed for subsequent data handling to take into account, e.g. B. by decimating the point cloud data or creating a corresponding grid from surface data. In use herein, the term "real-time" generally means a situation with no observable latency between processing and display. In In a video-based sampling system, real-time refers specifically to processing in the time between frames of video data, depending on specific video technologies can vary between about fifteen frames per second and about thirty frames per second. more general Processing capabilities of the computer 108 may vary depending on the size of the object 104, speed of image acquisition, and desired spatiality Resolution of three-dimensional points vary. The computer 108 may also include peripheral devices, e.g. a keyboard 114, display 110 and mouse 112 for user interaction with camera system 100 . Display 110 may be a touch screen display that accepts user input can receive direct physical dialogue with the display 110. In another aspect, the display may be an autostereoscopic display or the like. have that can display stereo images. Communications flows between the computer 108 and the camera 102 may use any suitable communications link, e.g. a wired connection or a wireless connection, e.g. B. based on IEEE 802.11 (also known as wireless Ethernet), BlueTooth or any other suitable wireless standard, e.g. B. by means of a radio frequency, infrared or other wireless communication medium. In medical imaging or other sensitive applications, wireless image transmission from the camera 102 to the computer 108 be protected. The computer 108 can generate control signals for the camera 102, in addition to conventional image capture commands may have camera operations, e.g. e.g. focus or zoom. In an example of the general operation of a three-dimensional imaging system 100, the camera 102 can be two-dimensional Capture sets of images at a video rate while scanning the camera 102 over a surface of the object. The two-dimensional Image sets can be passed to computer 108 for the derivation of three-dimensional point clouds. The three-dimensional data for everyone The newly acquired two-dimensional image set can be derived and attached to existing three-dimensional ones using a number of different techniques Data are adjusted or "appended". Such a system can use camera motion estimation to determine the position of the Camera 102 does not have to be tracked independently. A useful example of such a technique is disclosed in commonly owned U.S. Application No. 11 / 270,135, filed November 9, 2005, the entire contents of which are incorporated herein by reference. However, it becomes clear Be assured that this example is not limiting and that the principles described herein apply to a wide range of three-dimensional Image acquisition systems are applicable.

[0038] The display 110 may include any display suitable for video rendering or rendering at other rates with a level of detail is that corresponds to the acquired data. Suitable displays include cathode ray displays, liquid crystal displays, light emitting diode displays and the like. In general, the display 110 may be operatively coupled to the computer 108 and capable of receiving display signals therefrom. This Display can be CRT or flat panel, three-dimensional display (e.g. anaglyph display), autostereoscopic three-dimensional display or any other suitable two-dimensional or three-dimensional rendering hardware. In some According to embodiments, the display can have a touch screen interface, e.g. B. capacitive, resistive or acoustic surface waves (also called dispersive signal) touch screen technologies or any other suitable technology for sensing physical interactions with the display 110 are used. The system 100 may include a computer-usable or computer-readable medium. Computer-usable medium 118 may include a or multiple memory chips (or other chips, e.g., a processor that includes memory), optical disks, magnetic disks, or others magnetic media, etc. In various embodiments, computer-usable medium 118 may include removable storage (e.g., a USB device, tape drive, external hard drive, etc.), remote storage (e.g. network attached storage), volatile or non-volatile computer memory, etc. The computer-usable medium 118 may contain computer-readable instructions for execution by the Computer 108 included to perform the various methods described herein. In addition or instead, the computer-usable Medium 118 store data received from camera 102, store a three-dimensional model of object 104, computer code save for rendering and display etc. FIG. 2 illustrates an optical system 200 for a three-dimensional camera that can be used with the systems and described herein Method can be used, e.g. B. for the camera 102 previously described with reference to FIG. The optical system 200 may include a primary optical device 202 used in any type of vision system can come. Generally, a primary optical device herein refers to an optical system having an optical channel. Normally used this optical channel shares at least one lens and has a shared image plane in the optical system, although in the following Description Variants thereof are expressly described or may otherwise be apparent from the context. The optical system 200 can be a single primary lens, a group of lenses, an objective lens, mirror systems (including traditional mirrors, digital mirror systems, digital light processors or the like), confocal mirrors and any other optical means suitable for use with those described herein systems are suitable. The optical system 200 can e.g. B. used in a stereoscopic or other multi-image camera system will. Other optical devices may include holographic optical elements or the like. In various configurations, the primary optical device 202 may include one or more lenses, e.g. B. an objective lens (or lens group) 202b , a field lens 202d , a relay lens 202f, etc. The objective lens 202b may be at or near the entrance pupil 202a of the optical system 200. The field lens 202d can lie at or near a first image plane 202c of the optical system 200. The relay lens 202f can light beams in the optical system 200 forward onto. Furthermore, the optical system 200 such components such. B. aperture elements 208 with one or more apertures 212, a refocusing device 210 with one or more refocusing elements 204, one or more sampling devices 218 and / or a number of sensors 214a, 214b, 214c. The optical system 200 may be designed for active wavefront sampling, which is understood to include any technique that is used to scan a sequence or collection of optical data from an object 220 or objects, including optical data related thereto contribute to detecting two-dimensional or three-dimensional characteristics of the object 220, with optical data for the detection of movement are used, optical data are used for speed measurement or object tracking or similar. More details of an optical A system that can be used as optical system 200 of FIG. 2 is provided by US-A-7372642, the entire contents of which are incorporated herein by reference is recorded. More generally, it will be understood that while FIG. 2 illustrates one embodiment of an optical system 200, there are numerous ones variants are possible. A prominent feature of the optical system, related to the later discussion, is the use of an optical Center channel that offsets conventional video or still images at one of the sensors 214b concurrently with one or more images locations (e.g., at 214a and 214c) that capture three-dimensional information. This center channel image can be used on a user interface presented for inspection, tagging, or otherwise by a user during a user session, as described below can be manipulated. 3 shows a three-dimensional reconstruction system 300 that includes a high-speed pipeline and a high-precision pipeline used. In general, the high-speed processing pipeline 330 aims to process three-dimensional data in real time, such as with a video frame rate used by an associated display - is to provide while the high-precision processing pipeline 350 aims to get the highest possible accuracy from camera measurements subject to any external computational or timing constraints that imposed by the system hardware or an intended use of the results. A data source 310, such as the Camera 102 described above provides image data or the like to system 300 . The data source 310 can for example, hardware such as LED ring lights, wall sensors, a frame grabber, a computer, an operating system, and any other suitable hardware and / or software for obtaining data used in a three-dimensional reconstruction, include. Images from data source 310, such as center channel images containing conventional video images and side channels containing disparity data used to retrieve depth information may be forwarded to the real-time processing controller 316 will. The real-time processing controller 316 may also provide camera control information or other feedback to the data source 310 provide to be used in subsequent data collection, or to specify data already in the data source 310 have been obtained, which are required by the real-time processing controller 316. Full resolution images and associated image data may be held in a full-resolution frame buffer 322. For example, the stored images can be sent to the High-precision processing controller 324 provided or for image verification by a human user during subsequent processing steps are saved. The real-time processing controller 316 can generate images or frames for the reconstruction of three-dimensional surfaces in real-time from the two-dimensional source data to the high speed (video rate) processing pipeline 330. In a exemplary embodiment, two-dimensional images from an image set, such as side-channel images, from a two-dimensional Image registration module 332 to be registered. Based on the results of the two-dimensional image registration, a three-dimensional Point cloud generation module 334 generate a three-dimensional point cloud or other three-dimensional representation. the Three-dimensional point clouds from individual image sets can be combined by a three-dimensional stitching module 336 . After all the appended measurements may be combined by a three-dimensional generation module 338 into a three-dimensional model. That The resulting model can be stored as a high-speed three-dimensional model 340 . The high-precision processing controller 324 may provide images or frames to the high-precision processing pipeline 350. FIG. Separate image sets may have two-dimensional image registration performed by a two-dimensional image registration module 352 becomes. Based on the results of the two-dimensional image registration, a three-dimensional point cloud generation module 354 a three-dimensional point cloud or another three-dimensional representation can be generated. The three-dimensional point clouds of individual Image sets can be joined using a three-dimensional stitching module 356 . The global motion optimization, also referred to herein as global path optimization or global camera path optimization, can be derived from a global Motion optimization module 357 can be performed to reduce errors in the resulting three-dimensional model 358. in the in general, the path of the camera as it acquires the frames can be calculated as part of the three-dimensional reconstruction process will. In a post-processing refinement procedure, the calculation of the camera path can be optimized, i.e. the accumulation of errors along the length of the camera path can be compensated for by supplemental frame-to-frame motion estimation with some or all global path information is minimized. Based on global information such as individual frames of data in the image store 322, the high-speed three-dimensional model 340, and intermediate results in the high-precision processing pipeline 350 the high-precision model 370 can be processed to account for errors in the camera path and resulting image errors in the reconstructed model reduce. As a further refinement, a grid projection module 360 ​​can project a grid onto the high speed model. The resulting images may be warped or warped by warping module 362 . Warped images can be used to to facilitate alignment and joining between images, for example by reducing the initial error in a motion estimation. The warped images can be provided to the two-dimensional image registration module 352 . The feedback of the three-dimensional High-precision model 370 into the pipeline may be iterated until a metric, such as an attachment precision or a minimum error threshold, is obtained. Various aspects of the system 300 of FIG. 3 are described in more detail below. In particular, a describes a model refinement method that can be used by the high-precision processing controller 324 to to refine three-dimensional high-precision model 370 using measurement data in image memory 322. It should be understood that various processing modules or steps implied by the modules shown in this figure are exemplary in nature and that the order of processing or the sequence of processing steps is modified, omitted, repeated, rearranged or supplemented may without departing from the scope of this disclosure. 4A shows an object 410 for imaging along with a path 415 that a camera can follow while creating a three-dimensional image scanning of a surface of the object 410 is obtained. The direction of path 415 is generally indicated by arrow 416 . The object 410 (as shown) may be an upper tooth impression or any other object for which three-dimensional surface data is sought. On Starting at a starting point 420, the camera may follow an arc 430 to a second point 422. The camera can then do a segment 432 to a third point 424 follow. The camera can then follow a second arc 434 to a fourth point 426 . The camera can then follow a second segment 436 to return approximately to the starting point 420 . It should be noted that path 415, dem the camera follows is irregular rather than uniform, and that while a particular path 415 is depicted, more generally any path from the camera, including paths that turn on themselves, cross identical areas two or more times, and / or omit various surfaces of object 410 altogether. It should also be noted that the camera path usefully extends to the starting point 420 can return, but this is not essential for the three-dimensional reconstruction as described here. The camera can do hundreds or capture thousands of images or more as the camera traverses the path around such a dental object. Figure 4B shows locations where additional scan data could usefully be acquired to improve the accuracy of a three-dimensional improve reconstruction. For example, arcs 440, 442, 444, and 446 may be scanned (e.g., traversed by the camera path be used) to provide cross-connections between different lengths of camera path. Data could be useful for example any area can be detected that can improve the computational accuracy of a three-dimensional reconstruction, such as areas in which the Length of a camera path between two measurements of the surface (e.g. image sets or image data) significantly greater than the distance between two corresponding surface locations in the world coordinate system for the camera path. As another example, this may include areas where separate three-dimensional measurements fail for a general area of ​​the reconstructed three-dimensional model be registered on each other or more generally, where sections of the model or individual measurements indicate accumulated errors in the global camera path included. Keyframes (as described below) can be used to subset this elevation of measurements providing coverage for all or a substantial portion of the scanned object. Figure 5 shows a user interface displaying a graphical query for additional scan data. After the camera dem Following Path 415 illustrated above, a software tool can be used to identify various locations where additional data could usefully be detected to reduce accumulated error in a global camera path as discussed above. a monitor 510 may display an image 520 as a three-dimensional reconstruction of the scanned object, for example, and an arrow 530 may displayed on monitor 510, which indicates where additional scanning is recommended. The user can then continue to use a camera, such as camera 102 of FIG. 1, to scan the area indicated by arrow 530. More generally, areas for additional scanning for a user can be identified in a graphical user interface that uses a reconstructed from the camera path three-dimensional model together with arrows or other designations or graphic annotations showing a recommended scanning path represent, indicates. After a user enriches a camera path with additional samples, the resulting data can be used to resolve differences (i.e., errors) in the global camera path as generally described throughout this disclosure. Figure 6A illustrates a camera path in a world coordinate system. The camera starts at a starting point 610 and follows as shown by an arrow 625, in a counter-clockwise direction along a path 620 and returns to an end point that is in a fixed Coordinate system, such as an arbitrarily selected world coordinate system, coincides with the starting point 610. Figure 6B shows a camera path in a camera coordinate system. If a camera's path 620 in the world coordinate system traverses, errors may accumulate in a computed camera path 635 such that a measured endpoint 640 differs from that measured starting point 630 appears located away in the coordinate system, even though these points are identical in the world coordinate system are. In one aspect, one or more cross-connections such as those described above with respect to FIG to mitigate accumulated errors in the calculated camera path 635. Figure 7 is a flowchart for a three-dimensional reconstruction method including global path optimization for enhanced Accuracy. As shown in step 710, the method 700 may begin with pre-processing. It is understood that the described here Preprocessing presupposes the availability of a number of frames of image data, making up a camera path and a three-dimensional model can be reconstructed. The information for the three-dimensional reconstruction can be generated in a variety of ways, including coming from structured light projection, shadowing-based three-dimensional reconstruction, or disparity data. Disparity data can be generated by a conventional image plus one or more other channels or side channels. The preprocessing can determine the number of frames available, the amount of overlap between adjacent frames, identification and elimination frames with blurred or badly distorted images and any other appropriate processing steps. An estimate of the number desired keyframe can be initially determined during the pre-processing step. As shown in step 712, key frames can be selected from all frames of data coming along from a camera of a camera path can be detected. In general, computational costs can be reduced by storing some data and some Calculations and processing steps are performed solely with reference to key frames. These key frames can be related in a way that requires a characterization of a complete camera path, typically by the Registration of overlapping three-dimensional data allowed in respective key frames. Various methods are known in the art to select a subset of frames from data as key frames, including techniques based on an image overlap, a camera travel distance, the number of intervening non-key frames, and so on. Key frames can also be held or its based on an image overlap amount from the previous key frame and / or a candidate for a following one Key frames (if available) can be selected. Too little overlap can affect frame-to-frame registration. Too much of Overlapping can create excess keyframes that require additional processing. Key frames can be based on a spatial displacement can be selected. Keyframes can also be selected based on sequential shifting. This type of sequential shifting could mean, for example, that every tenth frame is selected as a key frame. In one aspect, keyframes can be selected based on any number of appropriate criteria while data is being acquired will. In another aspect, keyframe pairs can be determined post hoc by using all possible candidate keyframes to be examined. All possible keyframe pairs can be examined and candidates can be removed, for example where it is there is not enough overlap to form an attachment. More generally, any technique used to select a Subset of frames in a data set that is usefully used to select key frames for processing in order to use the reduce computational complexity. Once all keyframes have been selected, additional processing can be performed. For example full image data (full resolution center and side channel images) for each keyframe along with image signature data, Point cloud centroid calculations and any other measured or calculated data are stored in order to use the To support key frames in a three-dimensional reconstruction method as described here. As shown in step 714, candidate attachments may be identified. In general, an attachment is a relationship between two separate three-dimensional measurements from two different camera positions. Once an attachment is established, a Rotation and a translation for a camera's path between the two positions can be determined. In a complementary way the three-dimensional measurements from the positions are combined into a portion of a three-dimensional model. candidate attachments can be around each key frame, such as from the key frame to some or all of the frames of data between the key frames and adjacent key frames are analyzed. In another aspect, a candidate can be attached to any other keyframes, or to reduce computational complexity, to each keyframe within a spatial or sequential neighborhood around the key frame. Attachments may be based on the frames originally shown. It can also be useful to warp or deform two-dimensional images during registration and other steps in an attachment process to warp to improve accuracy and / or speed. Attachments may also or instead be observed on others epipolar relationships in source data. As shown in step 716, attachments for the entire camera path can be selected from the pool of candidate attachments will. The selection of attachments can e.g. B. based on the smallest calculated errors in resulting sections of the three-dimensional model. In general, anyone can Keyframes can be appended to one or more other keyframes, and each non-keyframe can be appended to at least one sequentially adjacent key frames are appended. As shown in step 718, a graphical analysis may be performed using the key frames and associated append be used to calculate a global path for the camera, which is used to obtain a three-dimensional model. the graphical analysis can view each key frame as a node or vertex, and each appendage as an edge between a pair of nodes regard. A key frame is selected as a starting point. A breadth or depth first search can be performed through the graph, to identify attachments that can connect the current keyframe to another keyframe. Any key frame can be marked while the graphic is being processed. A check can be performed to see if all key frames are were reached within the graph. When traversing the attachments in the graphical analysis does not reach all the key frames been identified, the largest subgraph is identified. This sub-graph can be examined to see if the entire three-dimensional image can be modeled. Certain sub-graphics may not be necessary to complete the three-dimensional imaging. When the camera over dwelling on a specific area of ​​an object's surface, or when the camera looped on an area multiple times the associated partial graphic(s) are not required. If a separate sub-graphic is identified that is needed to complete the three-dimensional To complete imaging, an optional branch back to step 712 may be performed. For example, a set of Keyframe selected that did not have sufficient concatenation from one keyframe to the next keyframe. By choosing a different set of keyframes, sufficient splicing can be achieved to provide a complete graphic of all the ones needed aspects of three-dimensional imaging. A keyframe that is too sparse, meaning it does not have sufficient appendages to help build a graphic may indicate that a different set of keyframes should be selected. Based a global path can be selected on the graphic analysis and the graphic can then be analyzed to calculate the path optimize. As shown in step 720, a numerical optimization can be performed to eliminate errors in the calculated camera path based on available data for the complete camera path, such as cross-connects, the measurements that are spaced in time link together to reduce. In general, the goal of a numerical optimization is to get a calculated error based on an error function for the camera path and / or a reconstructed three-dimensional model. A useful formulation of Error minimization problem for a global camera path is presented below. In general, from three positions A, B and C, each related to one another and to a world coordinate system with an origin 0 related by rotational and translational motion parameters, a set of related measurements in one be detected way. The relationship between a point X expressed as XO in the world coordinate system and the same point XA, which is expressed in the A coordinate system can be expressed as: XA = ROAXO + TOA[Eq. 1] ROA is the rotation that brings points from the world to the A coordinate system. TOA is the translation of the world coordinate system to the A coordinate system. It should be understood that symbols X and T may represent a vector rather than a scalar, where X is e.g. e.g. x-, y- and z-coordinate values. Furthermore, it should be understood that the symbol R can represent a matrix. The following equations can be similarly represent the transformation between the world and the B and C coordinate systems, respectively: XB = ROBXO + TOB[Eq. 2] XC = ROCXO + TOC[eq. 3] By rearranging, Equations 1 and 2 can be represented as follows: XO = R-1OA(XA - TOA) = R-1OB(XB - TOB) [Eq. 4] The representation of a point in one camera coordinate system can be related to the same point in another coordinate system will. For example, as in Equations 1-3, coordinates of a point X can be converted from the A coordinate system to the B coordinate system as follows coordinate system: XB = RABXA + TAB[Eq. 5] The rotation RAB rotates points from the A to the B coordinate system, and TAB translates the origin of the A coordinate system to the B coordinate system. In the optimization, the position of each camera can be optimized based on measured transformations between positions. That that is, a number of camera-world rotations and camera-world translations ROn and TOn can be performed. In general, one of these can be defined as the identity rotation and the zero translation, with the remaining values ​​being optimized as described below. The rotations and translations can be measured for many pairs of cameras. For the ith frame-frame movement, let one be the cameras of the pair is camera A and the other is camera B. This can also be viewed as the ith attachment. Let RiAB be the measured one Rotation bringing points in the A-frame into the B-frame and let TiAB be the coordinates of the A-position as in Equation 5 in the B-frame expressed. The rotations and translations for all cameras ROn and TOn can be optimized. RiC,OA and RLiC,OB can be used as the candidate turns are defined; TiC,OA and TiC,OB can be defined as the candidate translations given to the A and B cameras of the i-th attachment. Furthermore, RiC,AB = RiC,OB(RiC,OA)-1 as the candidate rotation from A to B and TiC,AB = TiC,OB - RiC,ABTiC,OA the candidate translation from A to B can be defined. Note that the motion constraints with sufficient attachments provide an overdetermined system for can form equations of motion. Using these equations as a starting point, numerical optimization can be performed be performed for the rotation and translation components of each camera based on the measured attachments. In a decoupled optimization, rotational and translational components can be optimized independently. If a candidate sentence of camera rotations Rc, the corresponding camera-to-camera rotations R iC,AB can be calculated that each of the correspond to measured camera-camera rotations R. Hence the corresponding residual rotations are given by Riresiduum,AB - RiC,AB(RiAB)-1 . A scalar-valued rotation cost function er that depends on the candidate camera rotations can be calculated. In Equation 6, logSO(3)(R) gives the shear angle vector ? back, which corresponds to the rotation R. In other words, logSO(3)(R) gives the vector ? which has a cross product matrix [v]X which is the matrix logarithm of R. Next, a similar scalar valued cost function can be calculated for the translation given by the candidate rotations and translations depends. Equation 6 can be minimized as a non-linear optimization; Equation 7 can be minimized as a linear optimization will. In a conventional decoupled approach to solving these simultaneous systems of equations, the rotational error function can be divided into a quaternion expression to translate the numerical problem into a linear system of equations for solution. During this approach can increase computational efficiency, it offers an incomplete optimization solution. The decoupled approach described above does not provide a truly optimal in terms of maximum likelihood, since it cannot use information from the translational portion of the attachments in determining rotation. To do a coupled optimization To achieve this, a weight can be used to calculate the contributions of rotational and translational components to a combined cost function to match: Several approaches can be used to optimize this cost function, but in one embodiment the Weights expressed as matrices. Different attachments can be made based on a number of factors including the number of points in the attachment (e.g., shared content), the quality of a particular three-dimensional measurement, and / or any others Factors affecting the known reliability of an attachment are given different weights. In one approach, they can Weight matrices also reflect the anisotropy error in each collected point, such as due to the acquisition of depth information disparity measurements, resulting in measurement accuracy that varies with distance from the camera. In some cases, Equation 8 can be reformulated so that the rotational and translational weights are decoupled for each attachment (i.e. WiC is block diagonal). In particular, this can happen in the case where the motion attachments are made from three-dimensional Point correspondences with isotropy point error can be recovered. In this case, the optimal solution for a given attachment i can be between Camera A and Camera B match the point cloud seen by Camera A to that seen by Camera B. if XiA and XiB are the positions of the center of the point cloud in the A and B systems, respectively, then rit in Equation 8 can be calculated as follows based on the candidate camera position can be replaced by the residual displacement between the point cloud centers: rit,ctr = XiB - (RiC,ABXiA + RiC,AB) [Eq. 9] Equation 8 can then be rewritten as: This coupled optimization problem can still be considered non-linear. It should be understood that others Optimizations falling within the scope of this disclosure are also possible. In general, by minimizing Equation 8, both rotational and translational errors can be minimized simultaneously. The weight matrices can, for example, according to "First Order Error Propagation of the Procrustes Method for 3D Attitude Estimation" by Leo Dorst, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 27, No. 2, February 2005, pp. 221-9, which are incorporated herein in their entirety incorporated by reference. Once a consistent set of motion parameters has been generated, it can three-dimensional model to be updated. In one aspect, the residual error can be used as a calibration metric. If the total error or part of the error is minimized the residual error can be evaluated. If a minimized error falls beyond a certain threshold, then based on a Conclusion that the inability to yield better quality is due to a miscalibration or other malfunction of the camera system, the calibration for the camera and the associated hardware are recommended. The threshold can be empirical based on the specific camera hardware equipment be determined or can be learned experimentally for a given system over time. When a system is new or freshly aligned was obtained, minimized expected error values ​​can be obtained. If minimized error values ​​deviate from these expected values, a Legal for trade evaluation flags are set or some other warning or message is generated indicating that the tool is legal for trade should be. As shown in step 722, oversampling may be performed to generate a three-dimensional model with data from non- Enrich keyframes. For example, non-keyframes can be registered with nearby keyframes to create small local ones generate reconstruction attachments that contain the full image detail available from non-key frames. In this way path optimization can be performed on a keyframe based dataset, making the data requiring processing are reduced while additional data points from non-key frames are available for use in the final three-dimensional model being held. It will be appreciated that any of the foregoing system and / or methods, in hardware, software or any combination thereof, suitable for the data acquisition and modeling technologies described here can be realized. This includes the realization in one or more microprocessors, microcontrollers, embedded microcontrollers, programmable digital signal processors, or other programmable devices along with internal and / or external memory. This can also or instead of an or multiple application specific integrated circuits, programmable gate arrays, programmable logic array components, or include any other device or devices that can be configured to process electronic signals. It will further be understood that an implementation may include computer executable code written using a structured programming language, such as C, an object-oriented programming language such as C++, or any other high or low level programming language (including assembly languages, hardware description languages ​​and database programming languages ​​and technologies), which stored, compiled, or translated to run on any of the foregoing devices, as well as heterogeneous combinations of processors, processor architectures, or combinations of different hardware and software. Consequently, here in one aspect discloses a computer program product comprising computer-executable code which, when run on one or more computing devices is performed performs any and / or all of the steps described above. At the same time, processing via devices, such as a camera and / or a computer and / or a manufacturing facility and / or a dental laboratory and / or a server in a number be distributed by ways, or all functionality can be integrated into a dedicated standalone device. All such Permutations and combinations are intended to fall within the scope of the present disclosure. SummaryGlobal camera path optimization

[0083] Various techniques are disclosed herein for improving global path optimization in a system that calculates the camera path for used the three-dimensional reconstruction. A subset of frames of data for the global path can be used to define the Reduce computational complexity of optimization while preserving all three-dimensional detail in the optimized model by others Measurements are related to the optimized keyframe path. QUOTES INCLUDED IN DESCRIPTION This list of the documents listed by the applicant was generated automatically and is only for better information reader added. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Patent Literature Cited - US7372642A [0031, 0042] Non-patent Literature Cited - "First Order Error Propagation of the Procrustes Method for 3D Attitude Estimation" by Leo Dorst, IEEE Transactions Pattern Analysis and Machine Intelligence, Vol. 27, No. 2, February 2005, pp. 221-9

[0079]

Claims

[1] Method for three-dimensional data reconstruction, comprising: acquiring multiple frames of image data of an object surface, each of the multiple frames of image data from a camera position along of a camera path and each of the plurality of frames of image data is a conventional image of the object from the camera position and data for a three-dimensional reconstruction of the object surface seen from the camera position; selecting a subset of the multiple frames of image data to provide multiple key frames, each one of the multiple Key frames move through a portion of the camera path including a rotation and a translation based on a or several common points in the three-dimensional reconstruction of the object surface in each of the respective key frames are related to at least one other of the plurality of key frames, with the remaining of the plurality of frames of image data being non- key frames are; providing a three-dimensional model of the object; determining a second rotation and a second translation from one of the key frames to at least one of the non-key frames, the sequentially positioned between the one of the key frames and a sequentially adjacent one of the key frames; Obtaining the three-dimensional reconstruction information of the object surface from the camera position of at least one of the non- key frames to provide oversampled three-dimensional data; and Adding the oversampled data to the three-dimensional model based on the second rotation and the second translation. [2] The method of claim 1, further comprising estimating camera motion between two adjacent key frames, wherein the Estimate based on rotation and translation. [3] The method of claim 2, further comprising optimizing the estimate of camera motion between the two adjacent ones Keyframes by creating consistency between motion parameters using an overdetermined system of having equations of motion constraint, where the motion parameters consist of rotational and translational information. [4] The method of claim 3, further comprising optimizing camera motion between two adjacent non-key frames by creating consistency between motion parameters using an overdetermined system of having equations of motion. [5] The method of claim 4, further comprising updating the three-dimensional reconstruction based on the generated consistency between has the movement parameters. [6] The method of claim 1, wherein the data for a three-dimensional reconstruction of the object surface from at least one other Channel image obtained to provide disparity data. [7] The method of claim 1, wherein providing a three-dimensional model further includes creating a three-dimensional model of the object using the camera path and the three-dimensional reconstruction for each of the key frames. [8] The method of claim 1, further comprising obtaining three-dimensional reconstruction information of the object surface from the camera position for all of the non-key frames between two adjacent key frames. [9] The method of claim 1, wherein selecting the subset of the plurality of frames based on a quality metric of the three-dimensional reconstruction based. [10] The method of claim 1, wherein selecting the subset of the plurality of frames using a graphical analysis is determined to ensure that all of the key frames are used in the three-dimensional reconstruction. [11] A computer program product comprising computer-executable code embodied on a computer-readable medium, the does the following when running on one or more computing devices: Capturing multiple frames of image data of an object surface, each of the multiple frames of Image data is captured from a camera position along a camera path and each of the multiple frames of image data is a conventional image of object from the camera position and data for a three-dimensional reconstruction of the object's surface as seen from the camera position; Selecting a subset of the multiple frames of image data to provide multiple key frames, each one of the multiple key frames extending through a portion of the camera path including a rotation and a translation based on one or several common points in the three-dimensional reconstruction of the object surface in each of the respective key frames are related to at least one other of the plurality of key frames, with the remaining of the plurality of frames of image data being non- key frames are; providing a three-dimensional model of the object; determining a second rotation and a second translation from one of the key frames to at least one of the non-key frames, the sequentially positioned between the one of the key frames and a sequentially adjacent one of the key frames; Obtaining the three-dimensional reconstruction information of the object surface from the camera position of at least one of the non- key frames to provide oversampled three-dimensional data; and adding the oversampled three-dimensional data to the three-dimensional model based on the second rotation and the second translation. [12] The computer program product of claim 11, further comprising computer executable code that includes the step of estimating a Performs camera movement between two adjacent keyframes, estimating based on rotation and translation. [13] The computer program product of claim 12, further comprising computer executable code that performs the step of optimizing the estimate of camera motion between the two adjacent keyframes by creating consistency between motion parameters Using an overdetermined system of constraint equations of motion, the motion parameters consist of information about rotation and translation. [14] The computer program product of claim 13, further comprising computer executable code that performs the step of optimizing the Camera motion between two adjacent non-key frames by creating consistency between motion parameters under Having use of an overdetermined system of equations of motion constraint. [15] The computer program product of claim 14, further comprising computer executable code that performs the step of updating the three-dimensional reconstruction based on the generated consistency between the motion parameters. [16] Computer program product according to claim 11, wherein the data for a three-dimensional reconstruction of the object surface from at least another channel image to provide disparity data. [17] The computer program product of claim 11, wherein providing a three-dimensional model further generating a three-dimensional model of the object using the camera path and the three-dimensional reconstruction for each of the key frames having. [18] The computer program product of claim 11, further comprising computer executable code that performs the step of obtaining three-dimensional reconstruction information of the object surface from the camera position for each of the non-key frames between two has adjacent key frames. [19] The computer program product of claim 11, wherein selecting the subset of the plurality of frames based on a quality metric of three-dimensional reconstruction based. [20] The computer program product of claim 11, wherein selecting the subset of the plurality of frames using a graphical analysis to ensure that all of the key frames are used in the three-dimensional reconstruction. [21] A method for interactively reducing accumulated error in a global path, the method comprising: Capturing multiple frames of image data of an object surface, each of the multiple frames of Image data is captured from a camera position along a camera path and each of the multiple frames of image data is a conventional image of object from the camera position and data for a three-dimensional reconstruction of the object's surface as seen from the camera position; creating a three-dimensional model of the object using the camera path and the three-dimensional reconstruction data; relatively identifying two of the plurality of frames of image data that represent a candidate for an accumulated error in the camera path to each other and Displaying the three-dimensional model along with a graphical annotation representing a recommended scanning path around the reduce accumulated errors. [22] The method of claim 21, wherein the data for a three-dimensional reconstruction of the object surface from at least one other Channel image obtained to provide disparity data. [23] The method of claim 21, further comprising acquiring one or more frames of image data along the recommended scan path to reduce the accumulated error. [24] The method of claim 21, wherein identifying two of the plurality of frames further includes identifying frames of image data which are separated along the camera path by a much greater distance than along the object surface. [25] A computer program product comprising computer-executable code embodied on a computer-readable medium, the does the following when running on one or more computing devices: acquiring multiple frames of image data of an object surface, each of the multiple frames of image data from a camera position along of a camera path and each of the plurality of frames of image data is a conventional image of the object from the camera position and data for a three-dimensional reconstruction of the object surface seen from the camera position; creating a three-dimensional model of the object using the camera path and the three-dimensional reconstruction data; relatively identifying two of the plurality of frames of image data that represent a candidate for an accumulated error in the camera path to each other and Displaying the three-dimensional model along with a graphical annotation representing a recommended scanning path around the reduce accumulated error. [26] Computer program product according to claim 25, wherein the data for a three-dimensional reconstruction of the object surface from at least another channel image to provide disparity data. [27] The computer program product of claim 25, further comprising computer executable code that performs the step of detecting one or multiple frames of image data along the recommended scan path to reduce accumulated error. [28] The computer program product of claim 25, wherein identifying two of the plurality of frames further includes identifying frames of image data separated along the camera path by a much greater distance than along the object surface. [29] A system comprising a camera, a monitor, a processor and a memory, the memory storing a computer program, executable by the processor to perform the following steps: acquiring a plurality of frames of image data of an object surface, each of the plurality of frames of image data being acquired from a camera position along a camera path and each of the plurality of frames of image data being a conventional image of the object from the camera position and data for a three-dimensional reconstruction of the object surface seen from the camera position; creating a three-dimensional model of the object using the camera path and the three-dimensional reconstruction data; identifying two of the plurality of frames of image data that represent a candidate for an accumulated error in the camera path; relative to each other and Displaying the three-dimensional model along with a graphical annotation representing a recommended scanning path around the reduce accumulated error. [30] The system of claim 29, wherein the data for a three-dimensional reconstruction of the object surface from at least one other Channel image obtained to provide disparity data. [31] The system of claim 29, wherein the memory storing the computer program product of claim 29 is further computer-executable comprises code that performs the step of capturing one or more frames of image data along the recommended scan path to reduce the accumulated error. [32] The system of claim 29, wherein identifying two of the plurality of frames further includes identifying frames of image data which are separated along the camera path by a much greater distance than along the object surface. [33] A method for global path optimization, the method comprising: acquiring multiple frames of image data of an object surface, each of the multiple frames of image data from a camera position along a camera path is captured and each of the multiple frames of image data is a conventional image of the object from the camera position and data for a three-dimensional reconstruction of the object surface seen from the camera position; and Minimize an error function for multiple camera positions along the camera path, where the error function is a system of equations for contains translational components of an error and for rotational components of the error, where the error function contains the translational components and couples the rotation components using a weight matrix, thereby providing an optimized camera path. [34] The method of claim 33, wherein the data for a three-dimensional reconstruction of the object surface from at least one other Channel image obtained to provide disparity data. [35] The method according to claim 33, wherein the system of equations is a non-linear system of equations. [36] The method of claim 35, wherein the translational component of the error forms a system of non-linear equations. [37] The method of claim 35, wherein the rotational component of the error forms a system of non-linear equations. [38] The method of claim 33, further comprising generating a three-dimensional model based on the camera path and the data for the three-dimensional reconstruction and refining the three-dimensional model based on the optimized camera path. [39] The method of claim 33, further comprising selecting a subset of the multiple frames of image data by multiple provide key frames, each of the plurality of key frames through a portion of the camera path including a rotation and a translation based on one or more common points in the three-dimensional reconstruction of the object surface are determined in each of the respective key frames is related to at least one other key frame, wherein the remaining of the multiple frames of image data are non-key frames. [40] The method of claim 33, wherein the weight matrix is ​​selected to make the error function more common about a centroid Decouple surface data around for two or more three-dimensional reconstructions. [41] The method of claim 33, wherein minimizing an error function further evaluating a calibration state based on the itself having resulting error function minimization. [42] Computer program product comprising computer-executable code embodied on a computer-readable medium which, when embodied on running one or more computing devices that performs the following steps: acquiring multiple frames of image data of an object surface, each of the multiple frames of image data from a camera position along a camera path is captured and each of the multiple frames of image data is a conventional image of the object from the camera position and data for a three-dimensional reconstruction of the object surface seen from the camera position; and Minimize an error function for multiple camera positions along the camera path, where the error function is a system of equations for contains translational components of an error and for rotational components of the error, where the error function contains the translational components and couples the rotation components using a weight matrix, thereby providing an optimized camera path. [43] Computer program product according to claim 42, wherein the data for a three-dimensional reconstruction of the object surface from at least another channel image to provide disparity data. [44] Computer program product according to claim 42, wherein the system of equations is a non-linear system of equations. [45] The computer program product of claim 44, wherein the translational component of the error forms a system of non-linear equations. [46] The computer program product of claim 44, wherein the rotational component of the error forms a system of non-linear equations. [47] The computer program product of claim 42, further comprising computer executable code that includes the step of generating a three-dimensional model based on the camera path and the data for the three-dimensional reconstruction and refinement of the three-dimensional model based on the optimized camera path. [48] ​​The computer program product of claim 42, further comprising computer executable code that includes the step of selecting a subset of the multiple frames of image data to provide multiple key frames, each of the multiple key frames through a portion of the camera path including a rotation and a translation based on one or more common Points in the three-dimensional reconstruction of the object surface are determined in each of the respective key frames, with at least is related to another key frame, with the remaining of the multiple frames of image data being non-key frames. [49] The computer program product of claim 42, wherein the weight matrix is ​​selected to center the error function around a centroid decouple common surface data around for two or more three-dimensional reconstructions. [50] The computer program product of claim 42, wherein minimizing an error function is further based on evaluating a calibration state on the resulting error function minimization.

Images