Three-dimensional scanner having sensors with overlapping fields of view

Abstract

The system includes a projector configured to project a plurality of non-coded elements onto an object, the projector having a first optical axis. The system includes a first camera having a first lens and a first sensor. The first lens defines a second optical axis. The system includes a second camera having a second lens and a second sensor. The second lens defines a third optical axis. The projector, the first camera, and the second camera are arranged substantially on a line in a first direction. The first optical axis is substantially parallel to the second optical axis, and the second optical axis is substantially parallel to the third optical axis. A center of the first sensor is displaced along the first direction away from the second optical axis, and a center of the second sensor is displaced along the first direction away from the third optical axis.

Description

[Technical Field]

[0001] The present invention relates generally to three-dimensional scanners, and more particularly to three-dimensional scanners having sensors with overlapping fields of view. [Background technology]

[0002] A three-dimensional (3D) scanner is a device that constructs a 3D model of the surface of a physical object. 3D scanners have applications across many fields, including industrial design and manufacturing, computerized animation, science, education, medicine, art, design, and others. Summary of the Invention

[0003] The present disclosure relates to 3D scanning technology. One approach to 3D scanning is the use of so-called "structured light," in which a projector projects a known light pattern onto the surface of an object. For example, light from a projector can be directed through a slide with a pattern printed on it. The shape of the object's surface is inferred from the distortion of the light pattern captured by a camera. One or more cameras can be used to acquire images of the reflection of the pattern on the object. By measuring the position of elements of the pattern in the image (e.g., measuring the distortion of the pattern), a computer system can determine the object's location on the surface using simple geometric calculations, such as a triangulation algorithm. The structured light approach can be contrasted with other approaches, such as the time-of-flight approach, in which a laser range finder finds the distance to a surface by timing the round-trip time of a pulse of light that is raster-scanned across the surface.

[0004] To determine the location of an object on its surface, a computer system needs to know which elements in the image correspond to which elements on the slide. There are two general approaches to solving this problem: one method utilizes coded elements, and the other method relies on non-coded elements. With coded elements, the elements in the pattern have some unique identifying characteristic that allows the computer system to work out which imaged elements correspond to which elements on the object. With non-coded elements (e.g., lines), some other method is required to distinguish one imaged element from another.

[0005] In some embodiments, a method is provided for disambiguating imaged features (e.g., lines) in a non-coded structured light approach to 3D scanning. The method is performed using a scanner having a projector and at least two cameras. The projector projects a plurality of lines onto the surface of an object. The reflections are imaged by a first camera and a second camera. Features (e.g., distorted lines) are detected in a first image from the first camera, and a correspondence to the features in the projected pattern is hypothesized. Using this hypothesis, the location of the features is transformed to a second image in the second camera. If the features are also not present in the second image, the hypothesis is rejected. In some embodiments, if the hypothesis results in a location on the object that is outside the depth of focus of the first camera, the hypothesis is rejected.

[0006] In some embodiments, a 3D scanner is provided. In some embodiments, the 3D scanner uses a structured light approach. In some embodiments, the 3D scanner uses non-coded elements. Overlapping the fields of view of two or more cameras and projectors improves the performance of the 3D scanner. In some embodiments, to improve ease of manufacturing, among other advantages, the two or more cameras and projectors have optical axes that are substantially parallel. In this way, two or more cameras (including their sensors and optics) can be mounted on parallel planes. For example, sensors for two or more cameras may be mounted on a single printed circuit board. To maximize the overlap of the fields of view of two or more cameras, the sensor of at least one camera is displaced (e.g., shifted) relative to its optical axis.

[0007] To this end, the 3D scanner includes a projector configured to project a plurality of non-coded elements onto the object. The projector has a first optical axis. The 3D scanner further includes a first camera having a first lens and a first sensor. The first lens focuses a reflection of a first portion of the plurality of non-coded elements onto the first sensor. The first lens defines a second optical axis. The 3D scanner includes a second camera having a second lens and a second sensor. The second lens focuses a reflection of a second portion of the plurality of non-coded elements onto the second sensor. The second lens defines a third optical axis. The projector, the first camera, and the second camera are offset from one another in a first direction. The first optical axis is substantially parallel to the second optical axis, and the second optical axis is substantially parallel to the third optical axis. The center of the first sensor is displaced along the first direction away from the second optical axis. In some embodiments, the center of the second sensor is displaced along the first direction away from the third optical axis. [Brief explanation of the drawings]

[0008] For a better understanding of the various embodiments described, please refer to the following description of the embodiments in conjunction with the following drawings:

[0009] [Figure 1] FIG. 1 is a schematic illustration of a front view of a 3D scanner, according to some embodiments; [Figure 2] FIG. 1 is a schematic illustration of a side view of a 3D scanner, according to some embodiments; [Figure 3] 1 is a schematic diagram of cameras with overlapping fields of view, according to some embodiments; [Figure 4] 1 is a schematic diagram of a camera with a sensor displaced relative to the optical axis of the camera, according to some embodiments. [Figure 5] FIG. 1 is a schematic illustration of a side view of a 3D scanner, according to some embodiments; [Figure 6] FIG. 1 is a schematic illustration of a side view of a 3D scanner, according to some embodiments; [Figure 7] FIG. 1 is a schematic illustration of a side view of a 3D scanner, according to some embodiments; [Figure 8] 1 illustrates a flow diagram of a method for generating a 3D model of an object according to some embodiments; [Figure 9] FIG. 1 is a block diagram of a 3D scanner, according to some embodiments;

[0010] The various figures described above generally illustrate different embodiments of 3D scanners provided by the present disclosure. However, it will be understood that certain features of a scanner (e.g., camera, projector, etc.) shown in and described with reference to one figure may be similar to those described with reference to other scanners shown in other figures. For the sake of brevity, such details will not be repeated throughout this disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0011] Reference will now be made to the embodiments, examples of which are illustrated in the accompanying drawings. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the various embodiments being described. However, it will be apparent to those skilled in the art that the various embodiments being described may be practiced without these specific details. In other instances, well-known methods, procedures, components, circuits, and networks have not been described in detail as not to unnecessarily obscure the inventive aspects of the embodiments.

[0012] It should be noted that as used herein, the term offset is used to refer to the relative position of the camera and detector within the body (e.g., housing) of the scanner, while the term displacement is used to describe the position of the sensor relative to the optical axis of the camera, or the position of the slide relative to the optical axis of the projector.

[0013] FIG. 1 shows a front view of a 3D scanner 100, which includes a projector 114 that projects one or more non-coded elements onto an object to be measured. In some embodiments, the one or more non-coded elements are part of a non-coded light pattern (i.e., the non-coded light pattern comprises one or more non-coded elements). The 3D scanner 100 includes three cameras (a first camera 102, a second camera 106, and a third camera 110). In some embodiments, fewer than three cameras are used. For example, in some embodiments, two cameras are used. In other embodiments, only one camera is used. In some embodiments, more than three cameras are used (e.g., four cameras, five cameras). The first camera 102 includes a first sensor 104, the second camera 106 includes a second sensor 108, and the third camera 110 includes a third sensor 112.

[0014] In some embodiments, the three cameras are for capturing a non-coded pattern of light reflected from an object. In some embodiments, the non-coded pattern of light includes a structured light pattern, such as lines or other repeating elements. The term non-coded pattern means that such lines or repeating elements lack individual, unique characteristics that allow specific elements of the pattern to be identified within the captured image. In some embodiments, two or more cameras are used to identify specific elements (e.g., lines) recorded within each of the images of the two or more cameras. In some embodiments, the non-coded pattern includes a set of simple elements (e.g., lines, dots, small bars) having relatively small dimensions in at least a first direction (e.g., x-direction, y-direction). The individual characteristics of these simple elements are not salient enough to distinguish them from an image acquired from a single camera that is not close enough to the projector 114 .

[0015] Each of the first camera 102, the second camera 106, and the third camera 110 records and transmits image data of the light emitted by the projector 114 and reflected from the object to be measured to a computer system (e.g., the computer system of the 3D scanner 900 and / or the remote device 936 in FIG. 9).

[0016] A line 118 on the first sensor 104 indicates the displacement of the center of the first sensor 104 from the geometric center of the first camera 102 along both the x and y directions. In some embodiments, the geometric center of the first camera 102 is defined by the optical axis of its optical system. A line 120 on the second sensor 108 indicates the displacement of the center of the second sensor 108 from the geometric center of the second camera 106 along both the x and y directions. Further details regarding the displacement of the sensor centers are shown in FIG. 4 . The center of the third sensor 112 of the third camera 110 is not displaced. Therefore, no line indicating displacement is shown on the third sensor 112.

[0017] Each camera has a field of view, as described in more detail below. In some embodiments, the field of view of one of the cameras is different from the field of view of the other cameras. For example, in some embodiments, the field of view of the object from the third camera 110 is smaller than the field of view of the object from the first camera 102. In some embodiments, the field of view of the object from the third camera 110 is larger than the field of view of the object from the first camera 102. In various embodiments, each of the fields of view of the cameras may be the same or different.

[0018] Projector 114 includes slide 116, and line 122 indicates the displacement of the center of slide 116 along the x-direction from the geometric center of projector 114. In some embodiments, the geometric center of projector 114 is defined by the optical axis of projector 114, as described in subsequent figures. Note that in some embodiments, the displacement of the center of slide 116 is arbitrary (i.e., in some embodiments, the slide is centered relative to the projector).

[0019] In some embodiments, projector 114, first camera 102, second camera 106, and third camera 110 are all disposed on the same plane (e.g., in the xy plane at a particular value of z). In some embodiments, one or more of projector 114, first camera 102, second camera 106, and third camera 110 are located at different z values ​​(e.g., one or more cameras and / or projectors extend above or below the plane of FIG. 1).

[0020] 2 shows a 3D scanner 200 in which a projector 202 and three cameras (a first camera 204, a second camera 206, and a third camera 208) are arranged in a substantially linear fashion along the y direction. For example, the positions of the projector, the first camera 204, the second camera 206, and the third camera 208 do not change substantially along the x direction. In some embodiments, the three cameras are arranged on one side of the projector 202. For ease of representation, the displacement (if any) of the center of the slide from the optical axis of the projector 202 and the displacement (if any) of the centers of one or more of the sensors of the first camera 204, the second camera 206, and the third camera 208 from their respective optical axes are not shown in FIG. 2.

[0021] Figure 3 shows a view of a 3D scanner 300 along the zy plane. The 3D scanner 300 includes a projector 302, a first camera 304, and a second camera 306. The projector 302 has an optical axis 308, the first camera 304 has an optical axis 310, and the second camera 306 has an optical axis 312.

[0022] In some embodiments, the optical axis 308 of the projector 302 is defined by an optical system (e.g., a lens system) within the projector 302. In some embodiments, the optical axis 310 of the first camera 304 and the optical axis 312 of the second camera 306 are defined by imaging optics (e.g., imaging lenses or lens systems) within the first camera 304 and the second camera 306, respectively. In some embodiments, the optical axis passes through the center of curvature of each surface of the optical elements of the imaging optics and coincides with the axis of rotational symmetry of the imaging optics.

[0023] Imaging optics within first camera 304 and second camera 306 form sharp images (on the respective sensors of first camera 304 and second camera 306) of objects located within region 314. Region 314 is bounded by near plane 316 and far plane 318.

[0024] Images of objects located at distances along the z-direction from the projector 302 and the first and second cameras 304, 306 closer to the near plane 316 or farther than the far plane 318 are blurred by defocus. Stated another way, in some embodiments, the near plane 316 and the far plane 318 are defined by a threshold resolution. In some embodiments, the threshold resolution is the resolution required to detect and / or distinguish individual elements on the surface of the object. Outside the region 314 defined by the near plane 316 and the far plane 318, defocus results in an inability to resolve the object using the threshold resolution. The best focal plane 324 is located within the region 314. In some embodiments, surfaces positioned at a z-distance closely coinciding with the best focal plane 324 form the sharpest images on the sensors of the first and second cameras 304, 306.

[0025] The field of view of a particular camera is the area or region of object space that is captured on that camera's sensor. In some embodiments, region 314 also represents a portion of the field of view of each of first camera 304 and second camera 306. Projector 302 also projects a non-coded pattern of light into the field of view of at least one of first camera 304 and second camera 306 (note that in some embodiments, the fields of view of first camera 304 and second camera 306 overlap, and the projector projects the non-coded pattern of light where the fields of view overlap).

[0026] Mechanically positioning the projector 302, the first camera 304, and the second camera 306 so that their fields of view overlap improves the performance of the 3D scanner 300 because, if a projected element of the pattern is within the field of view of each of the cameras (e.g., the first camera 304, the second camera 306), the images acquired by the different cameras can be used to identify the projected element (e.g., identify its correspondence with the projected pattern), as described with reference to method 800 of FIG. 8 . In some embodiments, to improve ease of manufacture, among other advantages, the two or more cameras and projector have optical axes that are substantially parallel, as shown in FIG. 3 . In this manner, two or more cameras (including their sensors and optics) can be mounted on parallel planes (as shown in FIG. 7 ). For example, the sensors for two or more cameras may be mounted on a single printed circuit board. To maximize the overlap of the fields of view of the two or more cameras, the sensor of at least one camera is displaced with respect to its optical axis.

[0027] To that end, the optical axis 308 of the projector 302, the optical axis 310 of the first camera 304, and the optical axis 312 of the second camera 306 are substantially parallel to one another. In other words, the angle formed between any pair of the optical axes 308, 310, and 312 is close to zero (e.g., within design tolerances). To maximize the overlap between the field of view of the first camera 304, the field of view of the second camera 306, and the field of view of the projector 302, the geometric center of the sensor of each of the first camera 304 and the second camera 306 is displaced from the optical axis of the respective camera. In some embodiments, the geometric center of the sensor is the center of gravity of the sensor. In some embodiments, the geometric center of the sensor is the center pixel of the sensor.

[0028] In some embodiments, the geometric center of the slide within projector 302 is also displaced from the optical axis of the projector such that the field of view of projector 302 overlaps the fields of view of first camera 304 and second camera 306. As used herein, the term field of view of a projector (or, equivalently, field of view of a projection) is used to mean the area onto which a slide pattern is projected. In some embodiments, projector 302 includes a light source such as a lamp, LED, or laser, and the optical system includes a condensing lens. The condensing lens converts a diverging beam from the light source (e.g., a point source) into a substantially parallel beam to illuminate an object, such as a slide, within the projector. In some embodiments, the slide defines a plurality of geometric elements. In various embodiments, the geometric elements comprise dots and / or horizontal (or vertical) parallel lines or bands. In some embodiments, the projector's optical system includes additional optics (e.g., lenses) after the slide.

[0029] That is, each camera of 3D scanner 300 images each point within the measurement region. It is often desirable to increase the measurement region of an object (e.g., the region of the object from which usable data is acquired in each image for the purpose of generating a 3D reconstruction of the object). Increasing the overlap of the fields of view of the cameras (e.g., cameras 304 and 306) increases the measurement area of ​​the object. When the fields of view fully overlap (e.g., at best focal plane 324), each of the cameras in 3D scanner 300 receives reflected light from the same measurement region. In some embodiments, when three cameras are included in the 3D scanner, the object being measured will have corresponding image points on all three cameras simultaneously. In some embodiments, when four cameras are included in the 3D scanner, the object being measured will have corresponding image points on all four cameras simultaneously.

[0030] FIG. 4 illustrates how the center of a sensor is displaced (e.g., shifted) relative to the geometric center of a camera (e.g., any of the cameras shown in previous or subsequent figures, such as camera 102 or 106 in FIG. 1 ), according to some embodiments. The camera 400 includes a sensor 402 (depicted by a rectangle with a dash-dotted outline) having a center 404. In some embodiments, the center 404 of the sensor 402 is a pixel at the center of an array of pixels of the sensor 402. In some embodiments, the center 404 is the center of gravity of the sensor 402. The camera 400 also includes imaging optics 406 (e.g., one or more lenses) that image one or more objects within a field of view 410 onto the sensor 402. The imaging optics 406 defines an optical axis 408. In some embodiments, the optical axis 408 of the imaging optics 406 is an axis of symmetry (e.g., rotational symmetry) that passes through the center of curvature of a lens (or other optical element) in the imaging optics 406. Illustratively, imaging optics 406 is represented by a single lens. In some embodiments, imaging optics 406 is an imaging system including one or more lenses and / or mirrors.

[0031] In some embodiments, the imaging optics 406 capture a field of view 410 (having a width 418 along the y-direction) that is larger than the sensor 402 can detect. For example, the imaging optics 406 forms an image 412 of the field of view 410. In FIG. 4, two edge rays (shown as dotted lines) of the field of view 410 are shown schematically as passing through the center of the imaging optics 406. For example, beyond the edge rays, aberrations such as vignetting become too large to collect suitable data. The image 412 has a lateral dimension 416 (along the y-direction as shown) that is larger than the width of the sensor 402.

[0032] 4, displacing the center 404 of the sensor 402, for example along the y-direction, changes the detection field of view 414 of the camera 400 as measured by the sensor 402. The detection field of view 414 is also indicated schematically by two edge rays (shown as solid lines) that pass through the center of the imaging optics 406 and reach the boundary of the sensor 402. As a result of the displacement 432, the detection field of view 414 becomes asymmetric with respect to the optical axis 408 of the camera 400.

[0033] The field of view of the sensor 402 (e.g., the angle of coverage of the sensor 402) depends on the displacement 432 of the center 404 of the sensor 402 along the y-direction. The field of view also depends on the distance 420 along the z-direction between the imaging optics 406 and the sensor 402. The distance 420 is a device design parameter and is known. The dimensions of the sensor 402 (e.g., along the y-direction, along the x-direction) are also known.

[0034] 4 shows a camera 400 in the zy plane and its field of view 414. If the imaging optics 406 include spherical optics (e.g., spherical lenses, spherical mirrors), the field of view 410 of the imaging optics 406 extends in the xy plane and also has a width 418 along the x direction.

[0035] In some embodiments, in addition to displacing the center 404 of the sensor 402 along the y direction, the center 404 is also displaced along the x direction (e.g., for embodiments in which the camera 400 is offset from the projector in the x direction). For a sensor with a center displaced along both the x and y directions, the image field of view is asymmetric with respect to the optical axis 408 along both the y and x directions (as shown in FIG. 4). In some embodiments, the center 404 of the sensor 402 is displaced only along the x direction and not along the y direction. For a sensor with a center displaced only along the x direction and not along the y direction, the image field of view is symmetric with respect to the y direction but asymmetric along the x direction.

[0036] FIG. 5 shows a 3D scanner 500 including a projector 502, a first camera 504, a second camera 506, and a third camera 508. The projector 502 includes an optical system 510 configured to direct light from a light source 518 onto an object scene 526. In some embodiments, the light is projected through a patterned slide such that the light includes non-coded elements. In some embodiments, the non-coded pattern of light includes dashed lines, curves, or an array of dots with predetermined or calibrated positions on the projected pattern. In some embodiments, a common characteristic of non-coded patterns is that the pattern includes a set of simple elements (e.g., lines, dots, small bars) that have a relatively small dimension in at least a first direction. The individual features of these simple elements are not significant enough to identify the elements by an image acquired from a single camera that is not close enough to the projector. Instead, in some embodiments, each non-code element projected by the projector and reflected by the measurement object is identified by comparing the coordinates of the line perceived by each of the cameras (e.g., its correspondence with the projected slide pattern is identified). In some embodiments, a larger offset of the cameras from each other and from the projector may lead to better accuracy in identifying the correspondence between the imaged element and the projected pattern (e.g., using the method described with respect to FIG. 8).

[0037] Optical system 510 has an associated optical axis 528. The first camera includes optical system 512 and optical sensor 520. Optical system 512 has an associated optical axis 530. The second camera includes optical system 514 and optical sensor 522. Optical system 514 has an associated optical axis 532. The third camera includes optical system 516 and optical sensor 524. Optical system 516 has an associated optical axis 534. Optical axis 528 is substantially parallel to optical axis 530, optical axis 532, and optical axis 534.

[0038] FIG. 5 shows projector 502, first camera 504, second camera 506, and third camera 508 arranged in a substantially straight line along the y direction. The field of view 538 of first camera 504 is substantially symmetric about optical axis 530 because the center of optical sensor 520 is not displaced relative to the geometric center of first camera 504. The center of optical sensor 522 is displaced to the right in the y direction in FIG. 5, resulting in a field of view 540 of second camera 506 that is asymmetric in the y direction about optical axis 532. The center of optical sensor 524 is further displaced to the right in the y direction in FIG. 5, resulting in a field of view 542 of third camera 508 that is asymmetric in the y direction about optical axis 534. A slide (not shown) within light source 518 is displaced to the left in FIG. 5 along the y direction, producing a field of view 536 of projector 502 that is asymmetric with respect to optical axis 528. By appropriately displacing the sensor and projector 502, the field of view 536 of the projector 502, the field of view 538 of the first camera 504, the field of view 540 of the second camera 506, and the field of view 542 of the third camera 508 substantially overlap in the object scene 526.

[0039] In some embodiments, the 3D scanner 500 includes one or more processors (e.g., processor 902 in FIG. 9 ) configured to receive data recorded by the optical sensors. In some embodiments, the data generated by the optical sensors is very large. For example, some optical sensors provide over 100 frames per second (fps), and for accuracy reasons, it is often desirable to use the highest possible frame rate (e.g., processing more frames provides a clearer 3D model with better accuracy for the same scan time). As an example, three ultra-high-resolution (HD) cameras can generate a throughput of 30 gigabits per second. In some embodiments, to solve this problem, the one or more processors configured to receive the data recorded by the optical sensors are field-programmable gate arrays (FPGAs), capable of massive data processing parallelism. However, a disadvantage of FPGAs is that they typically include a small amount of on-board random access memory (RAM).

[0040] In some embodiments, to solve this problem, the optical sensors are configured to provide data continuously along a readout direction parallel to the direction in which the sensors are displaced (e.g., the y-direction). In some embodiments, the 3D scanner 500 includes an FPGA through which data is read out from each optical sensor (e.g., data from each sensor is read out onto a single common FPGA, avoiding the need for inter-chip communication). At any given time during readout, the FPGA stores, for each optical sensor, a portion of an image of the object scene that includes the same subset of elements projected onto the object's surface, but less than all of the elements (e.g., the same subset is stored simultaneously in the RAM of each FPGA). In some embodiments, the method 800 described below is performed for each respective element (e.g., by calculating the spatial point on the object's surface that corresponds to that element) while the image of that element is stored in the RAM of the FPGA from each optical sensor. In this way, the method 800 is performed as the readout occurs.

[0041] Figure 6 shows a 3D scanner 600 that includes a projector 602 and a single camera 604. The projector 602 includes an optical system 606 having an associated optical axis 608. The projector 602 also includes a slide 620. A center 622 of the slide 620 is displaced along the y direction, to the left of Figure 6, relative to the optical axis 608 by a distance 628. The displacement of the center 622 of the slide 620 results in a field of view 616 of the projector 602 that is asymmetric with respect to the optical axis 608.

[0042] The single camera 604 includes an optical system 610 having an associated optical axis 612. The single camera 604 also includes a sensor 624. A center 626 of the sensor 625 is displaced along the y direction, to the right in FIG. 6, by a distance 630 relative to the optical axis 612. The displacement of the center 626 of the sensor 624 results in a field of view 618 of the single camera 604 that is asymmetric with respect to the optical axis 612. The field of view 616 of the projector 602 and the field of view 618 of the single camera 604 overlap at a plane 614 in the object scene 611. The plane 614 is indicated by a line in the zy plane. The plane 614 extends in the xy plane. The optical axis 608 of the projector 602 is substantially parallel to the optical axis 612 of the single camera 604. That is, the angle between the optical axis 608 and the optical axis 612 is close to zero.

[0043] Figure 7 shows a 3D scanner 700 that includes a projector 702, a first camera 704, and a second camera 706. The projector 702 includes an optical system 708 having an associated optical axis 730. The projector 702 also includes a slide 714. The center of the slide 714 is displaced along the y direction, to the left of Figure 7, relative to the optical axis 730 by a distance 736. The displacement of the center of the slide 714 results in a field of view 738 of the projector 702 that is asymmetric relative to the optical axis 730.

[0044] The first camera 704 includes an optical system 710 having an associated optical axis 732. The first camera 704 also includes a sensor 716 having a center that is displaced along the y direction to the right in FIG. 7 relative to the optical axis 732. The displacement of the sensor 716 results in a field of view 740 of the first camera 704 that is asymmetric with respect to the optical axis 732.

[0045] The second camera 706 includes an optical system 712 having an associated optical axis 734. The second camera 706 also includes a sensor 718 having a center that is displaced to the right in FIG. 7 along the y direction relative to the optical axis 734. The displacement of the sensor 718 results in a field of view 742 of the second camera 706 that is asymmetric with respect to the optical axis 734.

[0046] In some embodiments, the projector 702, the first camera 704, and the second camera 706 are connected to an optical holder 720. In some embodiments, the projector 702, the first camera 704, and the second camera 706 are mounted on the optical holder 720. In some embodiments, the optical holder 720 is configured to mechanically couple the projector 702, the first camera 704, and the second camera 706. In some embodiments, the sensor 716 of the first camera 704 and the sensor 718 of the second camera 706 are supported on a common mechanical support 722. In some embodiments, the sensor 716 of the first camera 704 and the sensor 718 of the second camera 706 are fabricated (e.g., directly) on the mechanical support 722 (i.e., the mechanical support 722 is a monolithic structure). In some embodiments, the mechanical support 722 comprises a printed circuit board (PCB). In some embodiments, the mechanical support 722 is flat. In some embodiments, sensor 716 and sensor 718 are therefore positioned in a common plane. In some embodiments, connecting element 724 couples mechanical support 722 to optics holder 720. For example, a single mechanical support for the different camera optics mechanically couples projector 702, first camera 704, and second camera 706.

[0047] The field of view 738 of the projector 702, the field of view 740 of the first camera 704, and the field of view 742 of the second camera 706 overlap at a plane 728 in the object scene 726. The plane 728 is indicated by a line in the zy plane. The plane 718 extends in the xy plane. The optical axis 730 of the projector 702 is parallel to the optical axis 732 of the first camera 704 and the optical axis 734 of the second camera 706.

[0048] Mounting the sensors of different cameras on the same mechanical support (and / or machining the sensors directly on the mechanical support) simplifies sensor manufacturing and provides faster temperature stabilization between the first camera 704 and the second camera 706. For example, having a single mechanical support quickly stabilizes the temperature between sensors 716 and 718. When the sensors are mounted on the same mechanical support, misalignment errors associated with camera offsets along the z-direction are smaller. In some embodiments, optical holder 720 fastens and fixes optical system 708, optical system 710, and optical system 712 to the same substrate. Fixing optical system 708 to the same substrate reduces misalignment in the z- and x-directions between the different optical systems and allows their desired relative offset in the y-direction to be more accurately maintained.

[0049] Monochrome sensors are commonly used in 3D scanners to capture the light pattern generated by a projector light source and reflected by the object being measured. Monochrome sensors do not include color filters. 3D scanners often have a separate color sensor for capturing texture information of an object. In some embodiments, the texture information includes color information and, optionally, one or more non-color characteristics of the object's appearance (e.g., specularity). For example, a color light sensor includes one or more built-in filters that detect both the color and brightness of incident light. In some embodiments, these two functions (e.g., object texture sensing and 3D contour mapping) are combined in a single color camera (e.g., first camera 504 having a color sensor configured to receive different wavelengths of reflected light from the object). One frame from the color camera captures an image of the projected light pattern reflected from the measurement object, and subsequent frames capture the object's color or texture information. In some embodiments, the color light sensor has lower sensitivity due to the presence of a color filter. In some embodiments, the remaining cameras (e.g., cameras 506 and 508) are monochrome cameras.

[0050] 8 shows a flow diagram of a method 800 for generating a 3D model of an object, according to some embodiments. In some embodiments, method 800 is performed by a 3D scanner (e.g., any of the 3D scanners described with reference to the preceding figures). In some embodiments, some operations of method 800 are performed by a computer system different from the 3D scanner (e.g., a computer system that receives and processes data from the 3D scanner, such as remote device 936 in FIG. 9 ). Some operations of method 800 are optionally combined, and / or the order of some operations is optionally changed. For ease of explanation, method 800 is described as being performed by 3D scanner 700 ( FIG. 7 ).

[0051] The method 800 includes projecting 802 a plurality of elements onto a surface of an object (e.g., using the projector 702). In some embodiments, the elements are non-coded elements. In some embodiments, the elements are lines.

[0052] The method 800 includes capturing 804 a first image of a portion of a plurality of lines reflected from an object using a first camera (e.g., first camera 704 of FIG. 7 ), the portion of the plurality of lines captured in the first image being within a field of view of the first camera.

[0053] The method 800 includes capturing 806 a second image of a portion of the plurality of lines reflected from the object using a second camera (eg, second camera 706 of FIG. 7). A portion of the lines captured in the second image are within the field of view of the second camera.

[0054] In some embodiments, the first image and the second image are captured substantially simultaneously, hi some embodiments, the first image and the second image are captured while the plurality of elements are projected onto the surface of the object.

[0055] In some embodiments, one of the first camera and the second camera is a color (e.g., RGB) camera. In some embodiments, the other of the first camera and the second camera is a monochrome camera. In some embodiments, the elements are strobed (e.g., illuminated) onto the surface of the object. Images acquired while the elements are projected onto the surface of the object are used for 3D reconstruction of the surface of the object (as described below). In addition, in some embodiments, the color camera also acquires images while the elements are not projected onto the surface of the object. In some embodiments, the color camera acquires images between strobe projections of the projector. Thus, in some embodiments, the color images of the object do not include reflections of the non-coded elements. In some embodiments, images acquired while the elements are not projected onto the surface of the object are used to generate textures for 3D reconstruction of the surface of the object.

[0056] The method 800 includes calculating (808) a plurality of possible spatial points on the surface of the object using image points on the first image corresponding to each of the plurality of lines projected onto the surface of the object. In some embodiments, calculating the plurality of possible spatial points includes calculating z-components of the plurality of possible spatial points. In some embodiments, each possible spatial point on the surface of the object is calculated by assuming a correspondence between an imaged element in the first image and an element in the projected pattern on the slide. Once the correspondence is known, the spatial point on the surface of the object can be determined by utilizing a triangulation algorithm, taking into account the geometry of the 3D scanner. Triangulation, as used herein, refers to using the positions in the image of known elements projected onto the surface of the object, along with knowledge of the scanner geometry, to determine the location of a point on the surface of the object.

[0057] Some of the possible (e.g. hypothesized) spatial points may be located outside the depth of field (e.g. depth of focus) of the 3D scanner 700 (e.g. located outside the region 314 in Figure 3 ). Thus, the method 800 includes rejecting, from the plurality of possible spatial points, spatial points that are outside the depth of field of the first camera, which rejection results in a set of remaining spatial points.

[0058] The method 800 includes, for each spatial point of the set of remaining spatial points, determining whether the respective spatial point corresponds to an imaged line in the second image (e.g., by mapping points in the first image to points in the second image using the geometry of the 3D scanner 700 and assumed spatial points on the surface of the object). 812 ) is included.

[0059] Method 800 further includes storing (814) each spatial point as a corresponding location on the surface of the object in accordance with a determination that the respective spatial point corresponds to an imaged line in the second image. For example, the fact that the hypothesized spatial coordinates of a line in the first image map to a line in the second image confirms that the assumption was correct. In some embodiments, additional images may be required to eliminate ambiguity in the correspondence between elements in the first image and elements in the projected pattern. Thus, in some embodiments, additional cameras are used to acquire the additional images (e.g., FIG. 5 shows a scanner with three cameras).

[0060] In some embodiments, method 800 includes removing the respective spatial point from the memory as a possible corresponding location on the surface of the object in accordance with a determination that the respective spatial point does not correspond to an imaged non-coded element in the third image. In some embodiments, method 800 includes eliminating a hypothesized correspondence between the imaged element and an element of the projection pattern from the set of possibilities.

[0061] It should be understood that the particular order described for the operations in Figure 8 is merely an example and is not intended to indicate that the described order is the only order in which the operations can be performed. Those skilled in the art will recognize various ways to reorder the operations described herein.

[0062] 9 is a block diagram of a 3D scanner 900 according to some embodiments. It should be noted that the 3D scanners described with reference to any of the previous figures may include any of the features described with respect to the 3D scanner 900. For example, any of the 3D scanners described with reference to the previous figures may include a computer system as described with respect to the 3D scanner 900. The 3D scanner 900 or the computer system of the 3D scanner 900 typically includes a memory 904, one or more processors 902, a power supply 906, a user input / output (I / O) subsystem 908, one or more sensors 903 (e.g., cameras), one or more light sources 311 (e.g., projectors), and a communication bus 910 for interconnecting these components. The processor 902 executes modules, programs, and / or instructions stored in the memory 904, thereby performing processing operations.

[0063] In some embodiments, processor 902 includes at least one central processing unit. In some embodiments, processor 902 includes at least one graphical processing unit. In some embodiments, processor 902 includes at least one field programmable gate array.

[0064] In some embodiments, memory 904 stores one or more programs (e.g., sets of instructions) and / or data structures. In some embodiments, memory 904 or its non-transitory computer-readable storage medium stores the following programs, modules, and data structures, or a subset or superset thereof: an operating system 912, which includes procedures for handling various basic system services and performing hardware-dependent tasks; a network communications module 918 for connecting the 3D scanner to other computer systems (e.g., remote devices 936) via one or more communications networks 950; a user interface module 920 that receives commands and / or input from a user via the user input / output (I / O) subsystem 908 and provides output for presentation and / or display on the user input / output (I / O) subsystem 908; a data processing module 924 for processing or pre-processing data from the sensor 903, optionally including performing any or all of the operations described with respect to method 800 (FIG. 8); A data acquisition module 926 for controlling the readout of the cameras, projectors, and sensors; Storage 930, including buffers, RAM, ROM, and / or other memory for storing data used and generated by the 3D scanner 900.

[0065] The above-identified modules (e.g., data structures and / or programs including sets of instructions) need not be implemented as separate software programs, procedures, or modules; thus, various subsets of these modules may be combined or otherwise reconfigured in various embodiments. In some embodiments, memory 904 stores a subset of the above-identified modules. Additionally, memory 904 may store additional modules not described above. In some embodiments, modules stored in memory 904 or its non-transitory computer-readable storage medium provide instructions for implementing respective operations in the methods described below. In some embodiments, some or all of these modules may be implemented using dedicated hardware circuitry (e.g., FPGAs) that encompass some or all of the module functionality. One or more of the above-identified elements may be executed by one or more of processors 902.

[0066] In some embodiments, the user input / output (I / O) subsystem 908 communicatively couples the 3D scanner 900 to one or more devices, such as one or more remote devices 936 (e.g., external displays), via a communications network 950 and / or via wired and / or wireless connections. In some embodiments, the communications network 950 is the Internet. In some embodiments, the user input / output (I / O) subsystem 908 communicatively couples the 3D scanner 900 to one or more integrated or peripheral devices, such as a touch-sensitive display.

[0067] In some embodiments, sensor 903 includes a first optical sensor (e.g., a CCD) (e.g., sensor 716, FIG. 7) that collects texture (e.g., color data), a second optical sensor (e.g., a CCD) (e.g., sensor 718) that collects 3D data, and a motion sensor (e.g., a 9-degree-of-freedom (DOF) sensor that may be implemented using a microelectromechanical system (MEMS), a gyroscope, and one or more Hall sensors). In some embodiments, the first optical sensor collects 3D data in addition to collecting texture.

[0068] In some embodiments, the light source 911 (e.g., a component of a projector described herein) includes one or more lasers. In some embodiments, the one or more lasers include vertical cavity surface emitting lasers (VCSELs). In some embodiments, the light source 911 also includes an array of light emitting diodes (LEDs) that generate visible light.

[0069] Communication bus 910 optionally includes circuitry (sometimes referred to as a chipset) that interconnects and controls communications between system components.

[0070] The foregoing description has been described with reference to specific embodiments for purposes of explanation. However, the illustrative description above is not intended to be exhaustive or to limit the invention to the precise form disclosed. Many modifications and variations are possible in light of the above teachings. The embodiments have been chosen and described to best explain the principles of the invention and its practical application, and thereby enable others skilled in the art to best utilize the invention and various described embodiments, with various modifications as may be suitable for the particular use contemplated.

[0071] Terms such as "first," "second," and the like are sometimes used herein to describe various elements, but it should be understood that these elements should not be limited by these terms. These terms are used only to distinguish one element from another. For example, a first sensor can be referred to as a second sensor, and similarly, a second sensor can be referred to as a first sensor, without departing from the scope of the various embodiments described. Although a first sensor and a second sensor are both sensors, they are not the same sensor unless the context clearly indicates otherwise.

[0072] The terminology used in the description of various embodiments set forth herein is for the purpose of describing particular embodiments only and is not intended to be limiting. When used in the description of various described embodiments and in the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly dictates otherwise. It will also be understood that the term "and / or," as used herein, refers to and encompasses any and all possible combinations of one or more of the associated listed items. It will be further understood that the terms "includes," "including," "comprises," and / or "comprising," as used herein, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0073] As used herein, the term "if" is optionally interpreted to mean "when" or "upon" or "response to determining" or "response to detecting," depending on the context. Similarly, the phrases "when determined" or "when [the stated condition or event] is detected" are optionally interpreted to mean "when determined" or "in response to determining" or "when [the stated condition or event] is detected" or "in response to detecting [the stated condition or event]," depending on the context.

Claims

1. 1. A system comprising: a projector configured to project a plurality of non-coded elements onto an object, the projector having a first optical axis; a first camera comprising a first lens and a first sensor, the first lens focusing a reflection of a first portion of the plurality of non-coded elements onto the first sensor, the first lens defining a second optical axis; a second camera comprising a second lens and a second sensor, the second lens focusing reflections of a second portion of the plurality of non-coded elements onto the second sensor, the second lens defining a third optical axis; a mechanical support that mechanically couples the first camera to the second camera; Equipped with the mechanical support includes a printed circuit board on which the first sensor and the second sensor are disposed; the projector, the first camera, and the second camera are offset from one another in a first direction; the first optical axis is substantially parallel to the second optical axis, and the second optical axis is substantially parallel to the third optical axis; the center of the first sensor is displaced along the first direction away from the second optical axis; the center of the second sensor is displaced along the first direction away from the third optical axis; the projector includes a slide displaced along the first direction away from the first optical axis, and a field of view of the projector along which the plurality of non-coded elements are projected onto the object is asymmetric with respect to the first optical axis. system.

2. The system of claim 1 , wherein the projector, the first camera, and the second camera are arranged substantially in a straight line in the first direction.

3. 3. The system of claim 2, wherein the projector is positioned at a first end of the generally straight line, and the first camera and the second camera are positioned on one side of the projector, on the generally straight line, along the first direction.

4. The system of claim 1 , wherein the depth of field of the first camera is substantially the same as the depth of field of the second camera.

5. The system of claim 1 , wherein the center of the second sensor displaced along the first direction provides a view of the object from the second camera that is asymmetric with respect to the third optical axis.

6. 6. The system of claim 5, wherein the field of view of the projector from which the plurality of non-coded elements are projected onto the object, the field of view of the object from the first camera, and the field of view of the object from the second camera are configured to overlap.

7. The system of claim 6 , wherein the field of view of the object from the first camera is larger than the field of view of the projector.

8. a third camera having a third lens and a third sensor; the third lens focuses a reflection of a third portion of the plurality of non-coded elements onto the third sensor; the third lens defines a fourth optical axis; the projector is positioned at a first end of the substantially straight line, and the first camera, the second camera, and the third camera are positioned on one side of the projector on the substantially straight line along the first direction; the center of the second sensor displaced along the first direction provides a view of the object from the second camera that is asymmetric with respect to the third optical axis; 3. The system of claim 2, wherein the field of view of the projector from which the plurality of non-coded elements are projected onto the object, the field of view of the object from the first camera, and the field of view of the object from the second camera are configured to overlap.

9. The system of claim 8 , wherein the field of view of the object from the third camera is smaller than the field of view of the object from the second camera.

10. The system of claim 1 , wherein the mechanical support is configured to stabilize a temperature between the first sensor and the second sensor.

11. The system of claim 1 , further comprising an optics holder configured to mechanically couple the projector, the first camera, and the second camera.

12. The system of claim 1 , wherein the first sensor comprises a color sensor configured to receive reflected light of different wavelengths from the object.

13. The system of claim 12 , wherein the reflected light of different wavelengths from the object provides texture information of the object.

14. 13. The system of claim 12, further comprising one or more processors configured to receive a first image from the first sensor and a subsequent second image from the first sensor, the first image including the reflection of the first portion of the plurality of non-coded elements and the subsequent second image including a color image of the object that does not include the reflection of the plurality of non-coded elements.

15. 10. The system of claim 1, further comprising one or more processors configured to receive data recorded by the first sensor and data recorded by the second sensor, both the first sensor and the second sensor configured to provide data sequentially to the one or more processors along a readout direction.

16. The system of claim 15 , wherein the readout direction of the first camera is parallel to the first direction.

17. The system of claim 16 , wherein the one or more processors include a common processor that receives the data from both the first sensor and the second sensor.

18. 20. The system of claim 17, wherein the common processor is a field programmable gate array.

19. and one or more processors and a memory, the memory comprising: calculating a plurality of possible spatial points on the surface of the object using image points on a first image from the first camera corresponding to each non-coded element of the plurality of non-coded elements projected onto the surface of the object; rejecting, from the plurality of possible spatial points, spatial points that are outside a depth of field of the first camera, said rejecting resulting in a set of remaining spatial points; For each spatial point in the set of remaining spatial points, determining whether the each spatial point corresponds to an imaged non-coded element in a second image from the second camera; and storing the respective spatial points as corresponding locations on the surface of the object in accordance with a determination that the respective spatial points correspond to imaged non-coded elements in the second image.

20. The memory includes: determining whether the respective spatial points correspond to imaged non-coded elements in a third image from a third camera; 20. The system of claim 19, further storing instructions for: storing the respective spatial point as the corresponding location on the surface of the object in accordance with a determination that the respective spatial point corresponds to an imaged non-coded element in the third image.

21. 21. The system of claim 20, wherein the instructions further comprise instructions for removing the respective spatial points from the memory as possible corresponding locations on the surface of the object in accordance with a determination that the respective spatial points do not correspond to imaged non-coded elements in the third image.

22. 20. The system of claim 19, wherein determining whether the respective spatial points correspond to imaged non-coded elements in the second image comprises: calculating an image point in the second image that corresponds to the respective spatial point; and identifying whether a portion of a non-coded element is imaged at the image point.

23. The system of claim 1 , wherein the non-coded element is a line.

Images