Using a mobile scanner to generate textured models

Abstract

A method is performed on a movable scanner having one or more optical sensors. The method includes scanning an object having a surface using the movable scanner. The scanning generates color data from a plurality of orientations of the movable scanner relative to the object. The method also includes generating a pixel map of the surface of the object using at least the color data, the pixel map including, for each respective pixel of a plurality of pixels: a color value for a corresponding point on the surface of the object; and a value of a non-color property of the corresponding point on the surface of the object.

Description

Technical Field

[0001] The present invention relates generally to generating texture models, and more particularly to generating texture models using a movable scanner, such as a structured light 3D scanner and / or a photogrammetric scanner. Background Art

[0002] One goal in computer graphics is to accurately model the physical appearance of objects (including their colors) under different lighting conditions. To do this, it is necessary to have a texture model for the object. Generating a physically accurate texture model is more complex than simply taking pictures of objects to observe their colors. For example, when one takes a picture of an object, the picture only shows what the object looked like under the specific lighting conditions that existed when the picture was taken. In addition, certain lighting conditions, such as flash, can leave blurry highlights in photos (e.g., areas on an object that appear overly bright and shiny).

[0003] Thus, if a 3D model of an object is textured from such a photograph (e.g., if the texture model is generated using such a photograph) without any additional processing, the 3D model will not appear physically accurate. The challenge in generating a physically accurate texture model is to produce a model of the physical properties that affect the appearance of the object, rather than a model of the properties that appear, for example, in a single photograph. Such models are particularly useful for animation, so that the texture of an object can be accurately displayed from different angles and under various lighting conditions. Summary of the invention

[0004] According to some embodiments, a method for generating a texture model for an object is provided. The method includes scanning an object having a surface. The scanning generates color data from a plurality of orientations of the 3D scanner relative to the object. The method also includes generating a pixel map of the surface of the object using at least the color data, the pixel map including, for each respective pixel in a plurality of pixels: a color value of a corresponding point on the surface of the object; and a value of a non-color property of the corresponding point on the surface of the object.

[0005] According to some embodiments, an electronic system for generating a texture model of an object is provided. The electronic system includes a scanner (e.g., a mobile scanner, such as a mobile 3D scanner), the scanner including one or more optical sensors, one or more processors and a memory. The memory stores instructions that, when executed by one or more processors, cause the one or more processors to perform any of the methods described herein.

[0006] According to some embodiments, a non-transitory computer-readable storage medium is provided. The non-transitory computer-readable storage medium stores instructions that, when executed by an electronic system having one or more processors, a memory, and a movable scanner including one or more optical sensors, causes the one or more processors to perform any of the methods described herein. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] For a better understanding of the various embodiments described, reference should be made to the following description of the embodiments in conjunction with the following drawings, wherein like reference numerals refer to corresponding parts throughout the drawings.

[0008] Figure 1A-1B Various views of a 3D scanner are shown in accordance with some embodiments.

[0009] Figure 2 is a block diagram of a 3D scanner according to some embodiments.

[0010] Figure 3 is a schematic diagram of a data collection environment according to some embodiments.

[0011] Figure 4 According to some embodiments, Figure 3 An illustrative example of data collected by the data collection environment shown.

[0012] Figure 5A-Figure 5C A schematic example of an image of a surface of an object used to generate a textured 3D representation of the object according to some embodiments is shown.

[0013] Figure 5D shows a diagram generated according to some embodiments Figure 5A-Figure 5C Schematic example of a textured 3D representation of an object shown.

[0014] Figure 6 is a mechanical diagram of a 3D scanner according to some embodiments.

[0015] Figure 7A-7D A flow chart of a method of generating a texture model for an object according to some embodiments is shown. DETAILED DESCRIPTION

[0016] The disclosed embodiments use color data obtained by a movable scanner to generate a realistic texture map of an object. In some embodiments, the movable scanner is a three-dimensional (3D) scanner (e.g., a device that builds a 3D model of the surface of an object). These 3D scanners are used to generate a realistic texture 3D representation of a physical object. As described below, some 3D scanners are based on structured light methods, while other scanners are based on photogrammetry methods. 3D scanners have applications in many fields, including industrial design and manufacturing, computer animation, science, education, medicine, art, design, and other fields.

[0017] With conventional 3D scanners, the only type of per-pixel material information available to users is a color map assembled under inconsistent lighting. With such scanners, red, green, and blue (RGB) values ​​are generated for each pixel based on observations of a given area of ​​the object's surface. For example, the different color values ​​for each pixel are averaged or one of the color values ​​is selected as a representative (e.g., the brightest pixel). In other words, the textures generated by these scanners are assembled without taking into account the behavior of reflected light and the direction and intensity of each pixel that captures the color texture. The generated color map contains inconsistent lighting and shading information, and does not contain material information that helps to render the object realistically.

[0018] Alternatively, conventional systems designed specifically to generate realistic texture models of materials typically require complex systems with precisely positioned and carefully calibrated lights and cameras.

[0019] The present disclosure provides systems and methods for separating inconsistent lighting and shadows from the true surface color of an object using a movable scanner (e.g., a 3D scanner) without the need for a complex system of precisely positioned and carefully calibrated lights and cameras. In addition, the systems and methods provided herein allow material properties (reflectivity, specular color, metallicity, surface roughness) to be measured for each pixel of the texture map of the object. This is accomplished by using corresponding pixels from different images of the surface of the object (e.g., pixels captured at different orientations when the scanner moves relative to the object) and tracking data that provides information about the orientation at which different values ​​are captured. In other words, in some embodiments, the 3D scanner collects color values ​​(e.g., RGB values) of various pixels and records the angles at which these values ​​are captured. The color values ​​from the corresponding pixels are related to the material properties using equations that describe the behavior of light incident on the surface of the object at various angles (e.g., a Lambertian diffuse reflectance model or a more complex texture model). This process produces a single, corrected RGB value for each pixel, thereby eliminating stray highlights. In addition, properties such as the maximum specular value and the angle at which the value appears can be calculated.

[0020] These systems and methods allow for considerable improvements to texture models generated through 3D scanning. In addition, these systems and methods facilitate the detection of various material properties.

[0021] Reference will now be made to embodiments, examples of which are illustrated in the accompanying drawings. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the various described embodiments. However, it is apparent to one of ordinary skill in the art that the various described embodiments may be practiced without these specific details. In other cases, methods, processes, components, circuits, and networks known to those of ordinary skill in the art are not described in detail in order to avoid unnecessarily obscuring the inventive aspects of the embodiments.

[0022] Figure 1A-1B A front view and a rear view of a 3D scanner 100 according to some embodiments are shown. In some embodiments, the 3D scanner 100 is used to acquire data from which a true color map of an object can be generated. However, it should be understood that the 3D scanner 100 is only an example of such a device. According to embodiments of the present disclosure, other devices (such as a photogrammetric scanner) can also be used to generate a true color map of an object.

[0023] The scanner 100 includes a main housing 102, a handle 108, and a battery housing 110 (e.g., which contains a battery). In some embodiments, the 3D scanner 100 is a portable handheld scanner. To this end, in some embodiments, the 3D scanner 100 has a size of less than 30cm x 30cm x 30cm (e.g., placed in a box with a size of 30cm x 30cm x 30cm). In some embodiments, the 3D scanner 100 is light enough to be carried by a person in one hand (e.g., the 3D scanner 100 weighs about 2.5kg).

[0024] In some embodiments, the main body housing 102 can be separated from the handle 108. In some embodiments, the main body housing 102 can be mounted (e.g., without the handle 108 and the battery housing 110) to a separate device (e.g., a robotic motorized scanning arm) via mounting points 112 (e.g., mounting points 112a to 112c). Thus, the 3D scanner 100 can be converted from a handheld scanner to an industrial or laboratory scanner.

[0025] In some embodiments, the 3D scanner 100 generates a 3D model of an object by projecting a spatial pattern of light (referred to herein as "structured light") onto a surface of the object and using an optical sensor (e.g., a camera) to capture an image of the surface of the object when the spatial pattern of light is projected onto the surface of the object. To this end, the main housing 102 houses one or more internal light sources (e.g., Figure 6 The one or more internal light sources strobe (e.g., project pulsed light) at a specific frequency through a spatial pattern slider (e.g., Figure 6The structured light is projected onto the surface of the object through the light source optical device 104. The image of the structured light projected onto the surface of the object is collected by the camera optical device 106 housed by the main housing 102. One or more cameras / sensors (e.g., Figure 6 The CCD detector 602 in the imager records an image of the structured light projected onto the surface of the object. A 3D model of the shape of the surface of the object can be determined by the distortion in the pattern of light projected onto the surface of the object (i.e., where the distortion is caused by the shape of the surface of the object), as described in more detail in U.S. Patent No. 7,768,656, entitled "System and Method for Three-Dimensional Measurement of the Shape of Material Objects," which is incorporated herein by reference in its entirety.

[0026] In some embodiments, the internal light source is a laser. In some embodiments, the internal light source is a vertical cavity surface emitting laser (VCSEL). In some embodiments, the 3D scanner 100 operates as a Class 1 light source, meaning that the laser is considered Class 1 anywhere outside of the main housing 102 (i.e., as defined by 21 CFR Part 1040 as of the filing date of this disclosure).

[0027] Note that in some embodiments, the 3D scanner 100 provides sufficient illumination of the surface of the object so that images acquired under outdoor daylight conditions have a signal-to-noise ratio sufficient to reconstruct the 3D shape of at least a portion of the object (e.g., with an accuracy of at least 0.1 mm, 0.2 mm, 0.3 mm, 0.4 mm, or 0.5 mm; or alternatively, with a resolution of 0.25 mm, 0.5 mm, 0.75 mm, or 1 mm). A normal method of ensuring that a laser operates within the Class 1 range is to enclose the laser in a large housing so that when the light leaves the housing, it is attenuated enough to be considered Class 1. This approach does not work for handheld scanners, which must be relatively small in size (e.g., less than 30 cm x 30 cm x 30 cm). Some embodiments of the present disclosure operate the light source enclosed in the main housing 102 as a Class 1 light source by appropriately selecting pulse width, peak power, repetition rate and / or duty cycle, thereby allowing the scanner 100 to be handheld, safe and operable under normal daylight conditions (e.g., at 120,000 lux, 100,000 lux, 20,000 lux, 2,000 lux or 1,000 lux). This also allows the scanner 100 to have a relatively short minimum working distance (e.g., a minimum working distance of no more than 35 mm).

[0028] In addition, to increase the signal-to-noise ratio from the structured light data and to distinguish between the structured light data and the texture (e.g., color) data, in some embodiments, the VCSEL laser described above operates at a frequency outside the visible spectrum (e.g., an infrared (IR) frequency). In such embodiments, in addition to the camera lens (e.g., camera optics 106), the main housing 102 also encloses an IR / visible beam splitter that directs the IR light to the first optical sensor (e.g., Figure 6 and directs the visible light to a second optical sensor (e.g., Figure 6 In some embodiments, data from the first optical sensor is detected synchronously with the stroboscopic light from one or more internal light sources, so that data at the stroboscopic frequency is detected in priority to information at other frequencies. This can be accomplished, for example, using synchronous demodulation or by strobing the first optical sensor to collect data synchronously with the one or more light sources that generate light.

[0029] In some embodiments, a second optical sensor (eg, Figure 6 In some embodiments, the texture data (e.g., image) is used both for rendering the 3D reconstruction in color and for tracking the position of the scanner 100 and / or its rotation (e.g., through photogrammetric analysis), which in turn is used to register (e.g., align) the 3D scanner 100 at different positions relative to the object (e.g., as described below with reference to Figure 3 The structured light data (eg, 3D shape data) captured by the device.

[0030] In some embodiments, the main housing 102 also encloses one or more motion sensors that track the motion of the scanner 100. The one or more motion sensors optionally include a 3-axis accelerometer, a 3-axis gyroscope, and / or a 3-axis magnetometer to measure the position and / or rotation of the scanner 100. In some embodiments, the one or more motion sensors include all three of the 3-axis accelerometer, the 3-axis gyroscope, and the 3-axis magnetometer, thereby being referred to as a nine (9) degree-of-freedom (DOF) sensor, despite the fact that the scanner 100 only has six mechanical degrees of freedom (i.e., three positional DOF and three rotational DOF). The position and / or rotational data from the one or more motion sensors is used to track the position and / or rotation of the scanner 100 (e.g., by applying a Kalman filter), which in turn is used to register (e.g., align) the position and / or rotation data generated by the scanner 100 at different positions relative to the object (e.g., as described below with reference to Figure 3 The structured light data (e.g., 3D shape data) and color data (e.g., image) captured by the apparatus.

[0031] To facilitate onboard automatic processing (e.g., generating at least an initial 3D reconstruction of an object), and to provide a fully mobile scanning experience to the user, in some embodiments, the main body housing 102 houses a plurality of processors, including one or more field programmable gate arrays, one or more graphics processing units (GPUs), and / or one or more CPUs. In some embodiments, the main body housing 102 contains sufficient processing to generate at least an initial reconstruction of a 3D model of an object (e.g., a preview of the 3D model of the object).

[0032] In some embodiments, the main housing 102 also houses a display 114 that displays a graphical user interface of the scanner 100. In addition, while the scanner 100 is scanning an object, the graphical user interface of the scanner 100 displays an initial 3D reconstruction of the object (e.g., a preview of the 3D reconstruction of the object). In some embodiments, the display 114 is a touch-sensitive display (sometimes referred to as a touch screen), so that user input can also be received (e.g., to start scanning, pause scanning, end scanning, and otherwise control the scanner 100).

[0033] Figure 2 is a block diagram of a 3D scanner 100 and / or a 3D data processing computer system 310 that receives data from the 3D scanner 100 and performs post-acquisition processing (collectively, the scanner 100 and the system 310 are referred to as "system 100 / 310") according to some embodiments. Note that in some embodiments, Figure 2 The various modules described in are divided between the 3D scanner and the 3D data processing computer system 310.

[0034] The system 100 / 310 generally includes a memory 204, one or more processors 202, a power supply 206, a user input / output (I / O) subsystem 208, a sensor 209, a light source 211, and a communication bus 210 for interconnecting these components. The processor 202 executes the modules, programs and / or instructions stored in the memory 204 to perform processing operations.

[0035] In some embodiments, processor 202 includes at least one graphics processing unit. In some embodiments, processor 202 includes at least one field programmable gate array.

[0036] In some embodiments, the memory 204 stores one or more programs (e.g., instruction sets) and / or data structures. In some embodiments, the memory 204 or a non-transitory computer-readable storage medium of the memory 204 stores the following programs, modules, and data structures, or a subset or superset thereof:

[0037] Operating system 212;

[0038] Kalman filter module 214;

[0039] Motion tracking module 216;

[0040] Color tracking module 218;

[0041] 3D registration module 220;

[0042] Color unification module 222;

[0043] Fusion module 224;

[0044] Color calibration module 226;

[0045] Texture module 228; and

[0046] • Storage 230, including buffers, RAM, ROM, and / or other memory that stores data used and generated by system 100 / 310.

[0047] The above modules (e.g., data structures and / or programs including instruction sets) do not need to be implemented as separate software programs, steps or modules, so that various subsets of these modules can be combined or otherwise rearranged in various embodiments. In some embodiments, the memory 204 stores a subset of the above modules. In addition, the memory 204 can store additional modules not described above. In some embodiments, the modules stored in the memory 204 or the non-temporary computer-readable storage medium of the memory 204 provide instructions for implementing the corresponding operations in the following methods. In some embodiments, some or all of these modules can be implemented with a dedicated hardware circuit (e.g., FPGA) containing some or all of the module functions. One or more of the above elements can be executed by one or more of the processors 202.

[0048] In some embodiments, the I / O subsystem 208 communicatively couples the system 100 / 310 to one or more devices, such as one or more remote devices 236 (e.g., external displays), via a communication network 250 and / or via a wired and / or wireless connection. In some embodiments, the communication network 250 is the Internet. In some embodiments, the I / O subsystem 208 communicatively couples the system 100 / 310 to one or more integrated or peripheral devices, such as a display 114.

[0049] In some embodiments, the sensor 209 includes a first optical sensor (eg, a first CCD detector such as a Figure 6 A second optical sensor (e.g., a second CCD detector such as a CCD detector 602-a) that collects texture (e.g., color) data Figure 6The CCD detector 602-b in the figure and a motion sensor (e.g., a 9DOF sensor, which may be implemented using a microelectromechanical system (MEMS), a gyroscope, and one or more Hall sensors).

[0050] In some embodiments, light source 211 includes one or more lasers. In some embodiments, one or more lasers include vertical cavity surface emitting lasers (VCSELs). In some embodiments, light source 211 also includes an array of light emitting diodes (LEDs) that generate visible light.

[0051] Communications bus 210 optionally includes circuitry (sometimes called a chipset) that interconnects system components and controls communications between system components.

[0052] Figure 3 3D scanner 100 scans an object 302 having a surface. The scan generates color data (e.g., a color image of the surface of the object) from multiple orientations of 3D scanner 100 relative to object 302 (e.g., from position 304). For example, Figure 3 , the 3D scanner is repositioned from position 304a (where it acquires a first color image) to position 304b (where it acquires a second color image) to position 304c (where it acquires a third color image). Note that in some embodiments, the 3D scanner 100 acquires data corresponding to a three-dimensional (3D) shape simultaneously with acquiring color data (e.g., a color image) (e.g., by projecting a spatial pattern of light onto a surface of an object and acquiring an image of the surface of the object (e.g., different from the color image) when the spatial pattern of light is projected onto the surface of the object).

[0053] Note that each of the first color image (acquired using the 3D scanner 100 at position 304a), the second color image (acquired using the 3D scanner 100 at position 304b), and the third color image (acquired using the 3D scanner 100 at position 304c) each captures a point 306 on the surface of the object 302. That is, there is a corresponding pixel in each of the first, second, and third color images that corresponds to the point 306 on the surface of the object 302. In some embodiments, the corresponding pixel in each of the first, second, and third color images has an RGB value, from which the luminosity can be calculated as a linear combination of the red (R) value, the green (G) value, and the blue (B) value (e.g., a luminosity function). Such luminosity values ​​are sometimes referred to as "relative luminosity" and have values ​​normalized to 1 or 100.

[0054] In some embodiments, the data collected by the 3D scanner 100 is transmitted (e.g., via a wired or wireless connection) to the 3D data processing computer system 310, where it is received. In some embodiments, the 3D data processing computer system 310 generates a pixel map (e.g., a texture map) of the surface of the object. Alternatively, in some embodiments, the pixel map is generated by the 3D scanner 100.

[0055] Figure 4 According to some embodiments, Figure 3 The data collection environment 300 shown is an illustrative example of data collected. In particular, Figure 4 Data collected from points 306 on the surface of object 302 is shown. Figure 4 The graph in shows the luminosity of various data pixels of different color images acquired at different orientations relative to the surface of the object. To this end, in some embodiments, the 3D scanner 100 and / or the 3D data processing computer system 310 determines a pixel-level correspondence between pixels in different images of the surface of the object (e.g., in the example provided, registering the first, second, and third color images). In some embodiments, determining the pixel-level correspondence between pixels in different images of the surface of the object includes determining that a first pixel in the first color image and a second pixel in the second color image correspond to the same location on the surface of the object.

[0056] The luminosity values ​​are plotted relative to the angle of a nominal normal vector for a point 306 on the surface of the object. In some embodiments, the nominal normal vector is a normal vector from a 3D reconstruction of the object (e.g., reconstructed from data corresponding to a three-dimensional (3D) shape acquired during a scan).

[0057] The data includes a first pixel having luminance 404a from the first color image, a second pixel having luminance 404b from the second color image, and a third pixel having luminance 404c from the third color image. In some embodiments, a pixel is determined from the data (e.g., by selecting an angle at which the data pixel having the maximum luminance is collected, or by fitting the data to a predetermined function (e.g., a Gaussian function) and selecting an angle at which the fitted predetermined function has a maximum value, as described in reference to FIG. Figure 7A-7C Detailed description of method 700 in FIG. 7 ) The direction θ of the “true normal” N,T In some embodiments, a “true normal” direction is used, determining (e.g., by using data pixels acquired at angles exceeding a predefined angle relative to the true normal direction—the data pixels are Figure 4 The true normal direction can be used to determine other material properties such as specularity, metallic parameters, and surface roughness.

[0058] Figure 5A-Figure 5C Schematic examples (ie, diagrams) of images of a surface of an object used to generate a textured 3D representation of the object according to some embodiments are shown. For example, Figure 5A An image 502a of a nut taken from a first orientation is shown. In image 502a, a stray highlight 504a (e.g., a specular-like reflection from a light source that was present when image 502a was captured) can be observed on the nut. Stray highlight 504a is a specular (e.g., a specular-like) reflection from a light source that was present when image 502a was captured. Figure 5B A second image 502b is shown taken from a second orientation different from the first orientation, and Figure 5C A third image 502c is shown that is taken from a third orientation that is different from the first orientation and the second orientation. Figure 5D shows the generated Figure 5A-Figure 5C . The textured 3D representation 506 of the object (i.e., a nut) is shown in FIG. The textured 3D representation 506 of the object does not include stray highlights (e.g., does not include one or more stray highlights present in the image of the surface of the object). In addition, the textured 3D representation 506 is more Figure 5A-Figure 5C The image (raw data) provides a model that more accurately represents the texture (color) of both the object and the table, so that the texture of both the object and the table can be more accurately displayed under various lighting conditions.

[0059] Figure 6 is a diagram illustrating various components of a 3D scanner 100 according to some embodiments. In particular, Figure 6 A cross section of the main housing 102 (FIG. 1) is shown. Figure 6 As shown, the scanner 100 includes (among other components):

[0060] A first optical sensor 602a (e.g., a first charge coupled device (CCD) detector) that collects 3D data of the scanned object (e.g., collects images of structured light that is illuminated on and distorted by the surface of the object, from which a reconstruction of the 3D shape of the object can be determined). In some embodiments, the first optical sensor 602a is sensitive to infrared (IR) light (although the first optical sensor 602a may also be sensitive to visible light);

[0061] A second optical sensor 602b (e.g., a second CCD detector) that collects texture (e.g., color) data of the scanned object (e.g., collects images when the structured light is not displayed on the surface of the object, e.g., collects images between stroboscopic pulses of the structured light shining on the object). In some embodiments, the second optical sensor 602b is sensitive to visible light (although the second optical sensor 602b can also be sensitive to IR light);

[0062] A beam splitter 604 that separates visible light and IR light. The beam splitter 604 preferentially directs IR light received through the camera optics 106 to the first optical sensor 602a, and preferentially directs visible light received through the camera optics 106 to the second optical sensor 602b;

[0063] Light source 606. In some embodiments, light source 606 is a pulsed light source (e.g., a stroboscopic light source). In some embodiments, light source 606 is an infrared light source. In some embodiments, light source 606 is a laser. In some embodiments, light source 606 is a vertical cavity surface emitting laser. In some embodiments, light source 606 is configured to operate as a Class 1 laser anywhere outside of body housing 102 (e.g., by appropriate selection of pulse width, peak power, repetition rate, and / or duty cycle);

[0064] A slider 616 having a spatial pattern formed thereon (e.g., printed or etched) through which light is projected onto the surface of the object (e.g., the slider is opaque and transparent to form the spatial pattern). Thus, light generated by the light source 606 passes through the slider 616 and is projected toward (e.g., onto) the surface of the object as structured light by the light source optics 104; and

[0065] A cooling manifold 608 (labeled in several places in the figure) comprising a single piece of metal thermally coupled to the light source 606, the optical sensor 602, and a plurality of processors including a GPU at location 614 and an FPGA at location 612.

[0066] Due to the way 3D data is obtained (e.g., strobing light from a fairly powerful light source, and collecting and processing many images per second), heat generation and removal is one of the biggest challenges in designing a handheld 3D scanner that is capable of previewing a 3D reconstruction of an object in real time. While maintaining relatively low temperatures is important, maintaining consistent temperatures (both spatially and temporally) is at least as important. The cooling manifold 608 maintains the various components of the 3D scanner 100 at a consistent and stable temperature by providing thermal connections between the major heat-generating components of the 3D scanner. Thus, the cooling manifold 608 reduces the "warm-up" time required for the 3D scanner to reach a stable temperature, allowing for greater processing power, and increasing the amount of time the 3D scanner can collect data.

[0067] Figure 7A-7D A flowchart of a method 700 for generating a texture model for an object according to some embodiments is shown. In some embodiments, the method 700 is performed in a scanner (e.g., a single movable scanner) (e.g., Figure 1A-1BThe method 700 is performed at a 3D scanner 100 in the mobile scanner, or a photogrammetric scanner). The mobile scanner includes one or more optical sensors. In some embodiments, the mobile scanner includes a display. In some embodiments, the mobile scanner is a 3D scanner that includes a processor for generating a 3D reconstruction of at least a portion of the surface shape of the object. In some embodiments, the mobile scanner is a portable handheld scanner (e.g., having a size less than 30cm x 30cm x 30cm, so that the entire device can be placed in a box of 30cm x 30cm x 30cm). In some embodiments, the mobile scanner is not a 3D scanner (e.g., the mobile scanner is a photogrammetric scanner that does not generate a 3D reconstruction of the shape of the object, but instead generates a texture map for an existing textureless 3D model of the object). In some embodiments, the scanner is a photogrammetric scanner that generates a 3D reconstruction of the surface of the object. For ease of explanation, method 700 is described as being performed by a 3D scanner 100. However, method 700 may be performed using any mobile scanner constructed according to the present disclosure. In some embodiments, one or more of the operations of method 700 are performed on a computer system separate from the mobile scanner (e.g., Figure 3 The method 700 is performed on a system 310 in the embodiment of the present invention, which performs post-acquisition processing to generate a texture model. Conversely, in other embodiments, according to the teachings of the present disclosure, the 3D scanner 100 performs a method different from the method 700. Some operations in the method 700 are optionally combined and / or the order of some operations is optionally changed.

[0068] Method 700 can be used to separate inconsistent lighting and shadows from the true surface color of an object scanned using a movable scanner. In addition, method 700 can be used to determine the material properties (reflectivity, specular color, metalness, surface roughness) of each pixel of a texture map of an object. To this end, in some embodiments, method 700 uses corresponding pixels from different images of the surface of the object (e.g., images captured at different orientations of the scanner relative to the object) and tracking data that provides knowledge of the orientations that capture different values ​​to generate a single pixel of a texture map (hereinafter referred to as a pixel map). The color value from the corresponding data pixel is related to the material property using an equation that describes the behavior of light incident on the surface of the object at various angles (e.g., a Lambertian diffusion model or a more complex texture model, also referred to as an illumination or shadow model). In some embodiments, method 700 generates a single corrected RGB value for each pixel point, thereby removing stray highlights (i.e., specular reflections from a light source that exist when color data is collected, which do not necessarily exist when rendering a textured 3D reconstruction under different illumination). Method 700 also generates at least one per-pixel value for non-color material properties, such as the maximum specular value and the angle at which the value appears (e.g., a "true normal").

[0069] refer to Fig. 7A , the 3D scanner 100 scans (702) an object having a surface. The scanning generates color data from multiple orientations of the 3D scanner 100 relative to the object. In some embodiments, the 3D data processing computer system 310, which is separate from the 3D scanner 100, receives color data generated by the 3D scanner 100 from multiple orientations of the 3D scanner 100 relative to the object (e.g., as shown in reference Figure 3 In some embodiments, the 3D scanner 100 has no a priori knowledge about the orientation of the 3D scanner 100 relative to the object (eg, the orientation is not predefined or precisely calibrated).

[0070] In some embodiments, scanning includes (704) repositioning the 3D scanner 100 at multiple orientations relative to the object. Figure 3 As shown, the 3D scanner 100 is repositioned from position 304a to position 304b and then to position 304c. The 3D scanner 100 acquires an image of the object 302 at each of these positions.

[0071] In some embodiments, a single optical sensor obtains (706) color data from multiple orientations of the 3D scanner 100 relative to the object (e.g., the single optical sensor is repositioned during the scanning process). As described below, in some such embodiments, the 3D scanner 100 includes additional optical sensors that obtain data in addition to color data.

[0072] In some embodiments, the scan generates ( 708 ) data corresponding to a three-dimensional (3D) shape of at least a portion of a surface of the object (eg, data from which the 3D shape of at least a portion of a surface of the object may be reconstructed).

[0073] In some embodiments, the one or more optical sensors include (710) a first optical sensor (e.g., a first infrared CCD) (e.g., Figure 6 In some embodiments, the first optical sensor is used to generate data corresponding to a 3D shape of at least a portion of a surface of the object. Generating data corresponding to a three-dimensional (3D) shape of at least a portion of the surface of the object includes repeatedly performing the following operations: projecting a spatial pattern of light onto the surface of the object, and capturing an image of the surface of the object using the first optical sensor while the spatial pattern of light is projected onto the surface of the object.

[0074] In some embodiments, the one or more optical sensors include (712) a second optical sensor (e.g., a second optical camera) (e.g., Figure 6706). In some embodiments, the second optical sensor is used to generate color data (e.g., the second optical sensor is the single optical sensor described in operation 706). Generating color data from multiple orientations of the 3D scanner 100 relative to the object includes: using the second optical sensor to capture an image of the surface of the object with the 3D scanner 100 positioned at each of the multiple orientations relative to the object. Note that in such embodiments, multiple pixels from the color data are used to generate the value of a single pixel of the pixel map (e.g., where the multiple pixels include corresponding pixels from different images). To avoid confusion, pixels of the surface image of the object are sometimes referred to as "data pixels" herein.

[0075] In some embodiments, the image provides RGB values ​​for the data pixels. In some embodiments, the image obtained by the second optical sensor is different from the image obtained by the first optical sensor, which is used to generate data corresponding to a three-dimensional (3D) shape of at least a portion of the surface of the object. In some embodiments, the image used to generate the color data is obtained when the spatial pattern of light is not projected onto the surface of the object (e.g., between projections of the spatial pattern of light onto the surface of the object).

[0076] In some embodiments, the 3D scanner 100 determines (714) a pixel-level correspondence between pixels of data in different images of the surface of the object (e.g., registering a color image obtained by a second optical sensor). In some embodiments, determining a pixel-level correspondence between pixels in different images of the surface of the object includes determining that a first pixel in a first image and a second pixel in a second image (different from the first image) correspond to a same location on the surface of the object.

[0077] refer to Figure 7B In some embodiments, the 3D scanner 100 generates (716) a 3D reconstruction of at least a portion of the shape of the surface of the object using data corresponding to a three-dimensional (3D) shape of at least a portion of the surface of the object. In some embodiments, the data corresponding to the three-dimensional (3D) shape of at least a portion of the surface of the object is used to determine pixel-level correspondence (e.g., the data corresponding to the three-dimensional (3D) shape of at least a portion of the surface of the object is used to generate a 3D reconstruction, which is used to align color images). In some embodiments, the pixel-level correspondence is determined using image registration (e.g., feature detection and matching, or by defining correspondences between images). In some embodiments, the pixel-level correspondence is determined using photogrammetry (e.g., by using image analysis to determine that two pixels in different images correspond to the same location on the surface of the object). In some cases, the camera position and the 3D shape of the surface are known, and the pixel-level correspondence is calculated using projective geometry.

[0078] In some embodiments, the 3D reconstruction is a polygonal representation of the object (e.g., a polygonal mesh including a plurality of polygons, wherein each of the plurality of polygons represents a region of a surface of the object). Thus, each of the plurality of polygons has a nominal normal vector. Determining pixel-level correspondences between data pixels in different images of the surface of the object includes selecting pixels from the different images that correspond to regions represented by the same polygon as corresponding pixels (e.g., grouping together multiple pixels from the different images that represent the same polygon). Reference Figure 4 In some embodiments, the nominal normal vector is a nominal normal vector of the polygon. In some embodiments, selecting pixels from different images corresponding to the area represented by the same polygon as corresponding pixels comprises mapping the color data onto a coordinate system of a 3D reconstruction of the object.

[0079] In some embodiments, the 3D scanner 100 uses the 3D reconstruction of at least a portion of the surface shape of the object to determine (718) a plurality of orientations of the 3D scanner 100 relative to the object from which the color data was obtained. In some embodiments, motion data (e.g., from a 9DOF sensor) is also used to determine the plurality of orientations (e.g., to determine the angles of the pixels from which the color data was obtained relative to the surface of the object). For example, information from a plurality of sources (e.g., 3D data, color data, and motion data from a 9DOF sensor are combined to determine the orientation of the 3D scanner 100 relative to the object for different images).

[0080] The 3D scanner 100 generates (720) a pixel map of the surface of the object using at least the color data. For each corresponding pixel in a plurality of pixels, the pixel map includes: a color value for a corresponding point on the surface of the object; and a value of a non-color property for the corresponding point on the surface of the object. In some embodiments, for each corresponding pixel in the plurality of pixels, the pixel map includes a plurality of values ​​of non-color properties for the corresponding point on the surface of the object. In some embodiments, the pixel map is (722) a texture map of the surface of the object (e.g., including a set of parameter values ​​that are used together to realistically represent the 3D model). In some embodiments, the color value is a diffuse value of the pixel. In some embodiments, the diffuse value (e.g., RGB value) represents a tendency of a material to scatter light in all directions (e.g., equally in all directions). Note that, unlike a data pixel (i.e., a pixel of an image), a pixel of a pixel map represents an area of ​​the surface of the object. Thus, in some cases, a pixel of a pixel map is referred to as a surfel (i.e., a surface element), and the pixel map is referred to as a surfel map.

[0081] In some embodiments, the non-color property is a texture property (e.g., a property that affects the appearance of corresponding points on a surface of an object). In some embodiments, the one or more non-color properties are selected (724) from the group consisting of: a reflectivity parameter, a specular parameter, a metallic parameter, a surface roughness, or a normal direction. In some embodiments, the specular parameter represents a tendency that represents a material's tendency to scatter light like a mirror (e.g., the angle of incidence equals the angle of reflection).

[0082] In some embodiments, the color value of each pixel in the plurality of pixels of the pixel map is (726) a vector value (eg, an RGB value).

[0083] In some embodiments, the value of the non-color property of each of the plurality of pixels of the pixel map is (728) a scalar value (eg, a single value between 0 and 1).

[0084] In some embodiments, for each corresponding pixel in the plurality of pixels of the pixel map, the color value of the corresponding point on the surface of the object is (730) a corrected color value (e.g., a "true diffuse" value). Generating the corrected color value includes: determining a nominal color value for the corresponding pixel using the color data (e.g., using the corresponding pixel from a different image of the surface of the object); and adjusting the nominal color value for the corresponding pixel of the pixel map using a value of a non-color property of the corresponding point on the surface of the object (e.g., the 3D scanner 100 uses the color data to determine a "true normal" direction, and uses the "true normal" direction to determine a "true diffuse" value).

[0085] In some embodiments, adjustments to the nominal color values ​​of pixels of a pixel map are used to remove stray highlights (e.g., by reducing the intensity of pixels that appear overly bright due to reflections from flash and / or other lighting). In conventional systems, such stray highlights are removed manually (e.g., by an artist). Thus, some embodiments of the present disclosure avoid the need to manually remove hot spots from texture data.

[0086] refer to Figure 7C In some embodiments, the non-color property is (732) a normal direction. For each corresponding pixel in a plurality of pixels of the pixel map, a color value is determined using color data from a predefined range of angles relative to the normal direction of the corresponding pixel.

[0087] In some embodiments, the predefined angle range is an angle range greater than a diffusion threshold angle (734).For each corresponding pixel in the plurality of pixels of the pixel map, a specular parameter is determined using color data obtained at an angle relative to the normal direction less than the diffusion angle threshold.

[0088] In some embodiments, for each corresponding pixel in the plurality of pixels of the pixel map, a normal direction is determined (736) by selecting as the normal direction an angle at which a maximum luminosity occurs in the color data.

[0089] For example, in some embodiments, method 700 includes, for each corresponding pixel in a plurality of pixels of a pixel map, determining which of the corresponding data pixels has the highest luminosity. The angle at which the corresponding data pixel with the highest luminosity is obtained is determined as the normal direction (e.g., the "true normal"). Data pixels obtained at angles greater than the "true normal" (e.g., 55°) are used to determine an adjusted color value for the corresponding pixel using a Lambert diffuse reflectance model for diffuse luminosity:

[0090]

[0091] Where L d is the observation angle θ relative to the normal direction i Luminosity under o is the energy of light incident on the surface, and p is the diffuse color of the surface (sometimes also called the diffuse albedo of the pixel or surfel).

[0092] In some embodiments, the maximum value of the fitted Lambert diffuse reflectance model (e.g., θ ) is subtracted from the luminance of the data pixel with the highest luminance. i = 0 value), the specular parameter (e.g., glossiness, or conversely, roughness) is determined for the corresponding pixel according to the following microfacet theory rendering equation:

[0093] L o =L d +L s (2)

[0094] Where L s is the specular luminosity, and L o is the luminosity at normal incidence.

[0095] In some embodiments, for each corresponding pixel in the plurality of pixels of the pixel map, a normal direction is determined (738) by fitting color data (e.g., perceived luminosity) to a predetermined function of the angles at which the color data was obtained, and selecting as the normal direction the angle at which the fitted predetermined function has a maximum luminosity. As described above, data pixels obtained at angles greater than this "true normal" (e.g., 55°) are used to determine an adjusted color value for the corresponding pixel using a Lambert reflectance model for diffuse luminosity. According to equation (2), the maximum value of the fitted Lambert reflectance model (e.g., θ ) is subtracted from the highest value of the fitted perceived luminosity. i=0 value), a specular parameter (e.g., glossiness, or conversely, roughness) is determined for the corresponding pixel.

[0096] More generally, in some embodiments, method 700 includes, for each corresponding pixel of a plurality of pixels of a pixel map, determining a first parameter (e.g., a specular parameter or a true normal direction) using a first subset of color data corresponding to the pixel, and determining a second parameter (e.g., a diffuse parameter) using a second subset of color data corresponding to the pixel that is different from the first subset of color data. In some embodiments, as described above, the separation of the two subsets of color data (data for calculating diffuse color and data for calculating specular color) can be based on the angle at which the data is obtained. In some embodiments, the separation of the two subsets of color data can be done based on other principles such as absolute color or any variation thereof.

[0097] refer to Fig.7D In some embodiments, for each pixel in a plurality of pixels of a pixel map, a value of a non-color property at a corresponding point on a surface of an object is determined (740) by providing an image of the surface of the object and information indicating an orientation of the 3D scanner 100 relative to the object at which the image was obtained to a solver of a set of one or more texture equations (e.g., lighting and / or shading equations) (e.g., a texture model, which may include shading equations and / or a reflection model); and receiving as output from the solver of the set of one or more texture equations the value of the non-color property at the corresponding point on the surface of the object.

[0098] Note that in some embodiments, the texture model includes a set of equations conventionally used for physically based rendering. Thus, in some embodiments, operation 740 is equivalent to running a physically based rendering (PBR) model "in reverse." Typically, such PBR models are used by computer animators (e.g., for video games or movies), who input material properties they have selected and receive as output the perceived color that depends on the lighting selected by the animator. Operation 740 uses the perceived color at various angles as input and determines the material properties.

[0099] In some embodiments, the texture model is a bidirectional reflectance distribution function (BRDF) model. For example, the Lambertian diffuse reflectance model can be written as:

[0100]

[0101] Where l is the unit vector in the direction of the light source and v is the unit vector in the direction of the observer.

[0102] In some embodiments, the texture model is a microfacet specular BRDF model, such as the Cook-Torrance microfacet specular shading model or other microfacet models. The Cook-Torrance microfacet specular shading model assumes that the surface contains microfacets, each of which is specularly reflective. The Cook-Torrance microfacet specular shading model can be written as:

[0103]

[0104] Where h is the unit angle bisector of v and l, and n is the unit normal vector (e.g., a unit vector in the "true normal" direction). Functions D, F, and G are defined as follows. In some cases, the Cook-Terrance model is used to determine the values ​​of specular parameters (e.g., specular shading), while the Lambertian reflectance model is used to determine the diffuse values.

[0105] Only the facets with normal direction h contribute to the specular reflection component from l to v. Thus, the function D(h) is the facet slope distribution function, which represents the part of the facets oriented in the direction h. The function D can be written as:

[0106]

[0107] where α is the square of the roughness.

[0108] The function G is the geometric attenuation factor, which takes into account the situation where facets occlude each other and can be written as:

[0109]

[0110] G(l,v,h)=G1(v)×G1(l) (7)

[0111] Where k is a model-specific parameter. For example, in some embodiments, k is related to the roughness of the material.

[0112] The function F is a Fresnel term that describes how light reflects from each smooth microfacet and can be written as:

[0113] F(v,h)=F0+(1-F0)×2 (-5.55473(v·h)-6.98316)(v·h) (8)

[0114] where F o is the specular reflectivity at normal incidence.

[0115] Please note that the BRDF models and formulas described above, including the Cook-Torrance microfacet specular shading model, are only examples of texture models. Given the benefit of this disclosure, those skilled in the art will recognize that any number of texture models may be used in accordance with various embodiments of the present disclosure. Other texture models include, but are not limited to, the Phong reflection model, the Gaussian distribution model, the Beckmann distribution model, the Heidrich–Seidel anisotropy model, and / or the Ward anisotropy model.

[0116] Thus, some embodiments of the present disclosure assume that the texture of a surface on an object can be decomposed (e.g., additive) into multiple texture properties of a material (e.g., specular and diffuse luminosity), and that these texture properties can be related to color observations via a texture model. However, unless explicitly stated otherwise, the subsequent claims should not be interpreted as limited to a particular texture model.

[0117] In addition, as can be seen from the above equations, some texture models relate the position of one or more light sources to the observed color. In some embodiments, the 3D scanner includes a light source. In some embodiments, the light source on the 3D scanner is significantly brighter than any light source external to the 3D scanner. Thus, in some embodiments, the position of the light source is known, and the method includes using the known position of the light source in the texture equation. For example, the method includes assuming that specular reflections are caused only by the light source on the 3D scanner, and solving the texture equation does not take into account any light source external to the 3D scanner (e.g., solving the texture equation includes assuming that ambient lighting is negligible compared to light emitted by the light source on the 3D scanner).

[0118] In some embodiments, the 3D scanner 100 generates ( 742 ) a textured 3D representation of the object using the 3D reconstruction of at least a portion of the shape of the surface of the object and the pixel map.

[0119] In some embodiments, the 3D scanner 100 uses ( 744 ) the textured 3D representation of the object to animate the corresponding object in an animation (eg, by mapping a color map onto a 3D reconstructed polygonal mesh of the object).

[0120] In some embodiments, the non-color property is a normal direction, and the normal direction in the pixel map is used for metrology, inspection, and / or quality assurance purposes (e.g., surface inspection, scratch and crack detection, etc.). In some embodiments, the 3D scanner 100 detects one or more cracks (or scratches) and displays (e.g., on an integrated display or on a display separate from the 3D scanner 100) a 3D representation of the object with visual markings showing the locations of the cracks (or scratches). In some embodiments, based on a determination that the scratches (or cracks) of the object exceed a predefined threshold (e.g., a quality assurance threshold), the 3D scanner 100 generates a report indicating that the scratches (or cracks) of the object exceed the predefined threshold.

[0121] In some embodiments, the 3D scanner 100 displays (e.g., on an integrated display or a display separate from the 3D scanner 100) a pixel map (e.g., displaying a textured 3D representation of an object). In some embodiments, the 3D scanner 100 displays color values ​​of the pixel map without displaying values ​​of non-color properties of the pixel map (e.g., displaying a textured 3D representation of an object with only a diffuse color). In some embodiments, the 3D scanner 100 displays values ​​of non-color properties of the pixel map without displaying color values ​​of the pixel map (e.g., displaying a textured 3D representation of an object with only a diffuse color). In some embodiments, the 3D scanner 100 receives user input and, in response to the user input, switches between displaying a textured 3D representation of the object, displaying color values ​​of the pixel map, and displaying values ​​of the one or more non-color properties (e.g., in response to receiving the user input, the 3D scanner 100 switches from displaying the textured 3D representation of the object to displaying the color values ​​of the pixel map or the values ​​of the non-color properties; or, in response to receiving the user input, the 3D scanner 100 switches from displaying the color values ​​of the pixel map to displaying the textured 3D representation of the object or displaying the values ​​of the non-color properties; or, in response to receiving the user input, the 3D scanner 100 switches from displaying the values ​​of the non-color properties to displaying the textured 3D representation of the object or displaying the color values ​​of the pixel map).

[0122] For purposes of explanation, the foregoing description has been described with reference to specific embodiments and the accompanying drawings. However, the illustrative discussion above is not intended to be exhaustive or to limit the invention to the precise forms disclosed. In light of the above teachings, many modifications and variations are possible. The embodiments were chosen and described in order to best explain the principles of the invention and its practical application, thereby enabling others skilled in the art to best use the invention and the various described embodiments with various modifications to suit the particular purpose contemplated.

[0123] It should also be understood that although the terms first, second, etc. are used in some cases herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another element. For example, a first sensor can be referred to as a second sensor, and similarly, a second sensor can be referred to as a first sensor without departing from the scope of the various embodiments described. Both the first sensor and the second sensor are sensors, but they are not the same sensor unless the context clearly indicates otherwise.

[0124] The terms used in the description of the various embodiments described herein are only for describing specific embodiments, not for limiting. As used in the description of the various described embodiments and the appended claims, the singular forms "a", "an" and "the" are intended to also include plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any and all possible combinations of one or more of the related listed items. It will be further understood that when used in this specification, the terms "includes", "including", "comprises" and / or "comprising" specify the presence of stated features, wholes, steps, operations, elements and / or parts, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, parts and / or their groupings.

[0125] As used herein, the term "if" is optionally interpreted to mean "when" or "in" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [the condition or event] is detected" is optionally interpreted to mean "in determining" or "in response to determining" or "in detecting [the condition or event]" or "in response to detecting [the condition or event]," depending on the context.

Claims

1. A method for generating a texture model of an object, comprising: The object having a surface is scanned using a three-dimensional scanner, i.e., a 3D scanner, having one or more optical sensors, wherein the scanning: generating color data from a plurality of orientations of the 3D scanner relative to the object; as well as generating data corresponding to a three-dimensional shape (3D shape) of at least a portion of the surface of the object; generating a 3D reconstruction of the shape of the at least a portion of the surface of the object using the data corresponding to the three-dimensional (3D) shape of the at least a portion of the surface of the object, wherein the 3D reconstruction comprises a plurality of pixels; For each pixel in the plurality of pixels, determining a true normal direction; Generating a pixel map of the surface of the object using at least the color data, the pixel map comprising, for each respective pixel of a plurality of pixels: a color value of a corresponding point on the surface of the object, wherein for each corresponding pixel of the plurality of pixels of the pixel map, the color value is determined using color data from a predefined angular range relative to the true normal direction of the corresponding pixel and wherein the predefined angular range is an angular range greater than a diffuse threshold angle; and A value of a specular reflection property at the corresponding point on the surface of the object, wherein the value of the specular reflection property at the corresponding point on the surface of the object is determined using the true normal direction of the corresponding point on the surface of the object, wherein the value of the specular reflection property is determined using color data obtained at an angle relative to the true normal direction that is less than the diffuse angle threshold. The method of claim 1 , wherein the pixel map is a texture map of the surface of the object.

3. The method according to claim 1 also includes using at least the color data to generate a value of a non-color property of each corresponding pixel in the plurality of pixels, wherein the non-color property is selected from the group consisting of: a metalness parameter and a surface roughness.

4. The method of claim 1, wherein the scanning comprises repositioning the 3D scanner at the plurality of orientations relative to the object.

5. The method of claim 1, wherein a single optical sensor obtains the color data from the multiple orientations of the 3D scanner relative to the object.

6. The method according to claim 1, further comprising: Using the 3D reconstruction of the shape of the at least a portion of the surface of the object and a pixel map, a textured 3D representation of the object is generated.

7. The method according to claim 6, further comprising: The textured 3D representation of the object is used to animate the corresponding object in an animation.

8. The method according to claim 1, further comprising: Using the 3D reconstruction of the shape of the at least a portion of the surface of the object, the plurality of orientations of the 3D scanner relative to the object from which the color data was obtained are determined.

9. The method according to claim 1, wherein: The one or more optical sensors include a first optical sensor; and Generating the data corresponding to the three-dimensional shape (3D shape) of the at least a portion of the surface of the object comprises repeatedly performing the following operations: projecting a spatial pattern of light onto said surface of said object; as well as When the spatial pattern of light is projected onto the surface of the object, an image of the surface of the object is acquired using the first optical sensor.

10. The method of claim 1, wherein: The one or more optical sensors include a second optical sensor; as well as Generating the color data from the multiple orientations of the 3D scanner relative to the object includes acquiring an image of the surface of the object using the second optical sensor with the 3D scanner positioned at each of the multiple orientations relative to the object.

11. The method according to claim 10, further comprising: Pixel-level correspondences between pixels in different images of the surface of the object are determined.

12. The method of claim 1, wherein: for each respective pixel of the plurality of pixels of the pixel map, the color value of the corresponding point on the surface of the object is a corrected color value; and Generating the corrected color value comprises: determining a nominal color value of the corresponding pixel using the color data; as well as The nominal color value of the corresponding pixel of the pixel map is adjusted using the value of the specular property of the corresponding point on the surface of the object.

13. The method according to claim 1, wherein: For each corresponding pixel of the plurality of pixels of the pixel map, the true normal direction is determined by selecting an angle at which a maximum luminosity occurs in the color data as the normal direction.

14. The method according to claim 1, wherein: For each corresponding pixel in the plurality of pixels of the pixel map, the true normal direction is determined by: fitting the color data to a predetermined function of the angle at which the color data was obtained; and The angle at which the fitted predetermined function has the maximum luminosity is selected as the true normal direction.

15. An electronic system comprising: A 3D scanner comprising one or more optical sensors; one or more processors; and A memory storing instructions, which, when executed by the one or more processors, cause the one or more processors to perform the method of any one of claims 1-14.

16. A method for generating a texture model of an object, comprising: The object having a surface is scanned using a three-dimensional scanner, i.e., a 3D scanner, having one or more cameras, wherein the scanning: generating color data from a plurality of orientations of the 3D scanner relative to the object, wherein generating the color data from the plurality of orientations of the 3D scanner relative to the object comprises acquiring an image of the surface of the object using the one or more cameras positioned at each of the plurality of orientations relative to the object; as well as Generating data corresponding to a three-dimensional shape (3D shape) of at least a portion of the surface of the object, wherein generating the data corresponding to a three-dimensional shape (3D shape) of at least a portion of the surface of the object comprises repeatedly performing the following operations: projecting a spatial pattern of light onto said surface of said object; as well as capturing, using the one or more cameras, an image of the surface of the object while the spatial pattern of light is projected onto the surface of the object; generating a 3D reconstruction of the shape of the at least a portion of the surface of the object using the data corresponding to the three-dimensional (3D) shape of the at least a portion of the surface of the object; Generating a pixel map of the surface of the object, the pixel map comprising, for each respective pixel in a plurality of pixels: values ​​of non-color properties of corresponding points on the surface of the object; a color value of the corresponding point on the surface of the object, wherein the color value is generated using the color data and corrected according to the value of the non-color property of the corresponding point on the surface of the object; and Wherein, for each pixel of the plurality of pixels of the pixel map, the value of the non-color property of the corresponding point on the surface of the object is determined by the following steps: providing the image of the surface of the object and information indicating the orientation of the 3D scanner acquiring the image relative to the object to a solver of a set of one or more texture equations; and The values ​​of the non-color properties of the corresponding points on the surface of the object are received as output from the solver of the set of one or more texture equations. The method of claim 16 , wherein the pixel map is a texture map of the surface of the object.

18. The method of claim 16, further comprising generating, for each respective pixel in the plurality of pixels, a value of a non-color property of the respective pixel using at least the color data, wherein the non-color property is selected from the group consisting of: a metalness parameter and a surface roughness.

19. The method of claim 16, wherein the scanning comprises repositioning the 3D scanner at the plurality of orientations relative to the object.

20. The method of claim 16, wherein a single optical sensor obtains the color data from the multiple orientations of the 3D scanner relative to the object.

21. The method of claim 16, further comprising: Using the 3D reconstruction of the shape of the at least a portion of the surface of the object and a pixel map, a textured 3D representation of the object is generated.

22. The method according to claim 21, further comprising: The textured 3D representation of the object is used to animate the corresponding object in an animation.

23. The method of claim 16, further comprising: Using the 3D reconstruction of the shape of the at least a portion of the surface of the object, the plurality of orientations of the 3D scanner relative to the object from which the color data was obtained are determined.

24. The method of claim 16, wherein: The one or more optical sensors include a second optical sensor; as well as Generating the color data from the multiple orientations of the 3D scanner relative to the object includes acquiring an image of the surface of the object using the second optical sensor with the 3D scanner positioned at each of the multiple orientations relative to the object.

25. The method according to claim 24, further comprising: Pixel-level correspondences between pixels in different images of the surface of the object are determined.

26. An electronic system comprising: A 3D scanner comprising one or more optical sensors; one or more processors; and A memory storing instructions, which, when executed by the one or more processors, cause the one or more processors to perform the method of any one of claims 16-25.

Images