System and method

Abstract

To provide a system and method for augmenting 3D rendering corresponding to a physical object by sketching an outer surface of the physical object.SOLUTION: A 3D rendering system 106 includes: an input device 112 operable to physically trace over an outer surface of a physical object 101; a tracking device 113 configured to, as the input device physically traces over the outer surface of the physical object, output data representing a spatial position of the input device; and a 3D rendering device 108, which is a computer tablet or smartphone, configured to augment 3D rendering corresponding to the physical object based on the data representing the spatial position output from the tracking device, and output the augmented 3D rendering on a display.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] This application is directed to generating three-dimensional (3D) renderings of physical objects, annotating the 3D renderings, and refining the 3D renderings by physically tracing an input device over the physical objects. This application is also directed to distance measurement of curves traced by the input device. [Background technology]

[0002] In many industries, including the automotive industry, physical models, such as clay models, are used to model automobile designs and physically illustrate the design features of automobiles. Refining and extending physical models is a key task in designing cars and other industrial or consumer products. During the industrial design process, designers and 3D modelers mold the physical models with tools and tape-mark changes to the physical models. However, physically molding a physical model is time-consuming and often not easily reversible, as changes made to the model may require patchwork to be applied to the physical model in order to undo them.

[0003] Therefore, what is needed is a method and apparatus for rendering a 3D model of a physical object, augmenting the 3D model by sketching on the 3D model, and displaying the augmented 3D model digitally or virtually. Summary of the Invention [Means for solving the problem]

[0004] In one embodiment, a system includes a three-dimensional (3D) scanner configured to scan an exterior surface of a physical object and output data representing the exterior surface of the object. In one embodiment, the system includes a processor configured to receive the data representing the exterior surface of the object, generate a 3D model of the object based on the received data, and output a 3D rendering of the object based on the generated 3D model. In one embodiment, the system includes a display configured to receive the 3D rendering of the object and display the 3D rendering of the object. The system includes an input device operable to physically trace a portion of the exterior surface of the object, and a tracking device configured to track positioning of the input device as the input device physically traces at least a portion of the exterior surface of the object and output data representing at least one spatial position of the input device as the input device traces the object. The processor is configured to receive data representing the at least one spatial position of the input device, augment the 3D rendering of the object based at least in part on the data representing the at least one spatial position of the input device, and output the augmented 3D rendering of the object to the display in response to augmenting the 3D rendering of the object. In one embodiment, the display is configured to display the augmented 3D rendering of the object.

[0005] In one embodiment, the processor is configured to augment the 3D rendering of the object by at least identifying, based on data representing at least one spatial position of the input device, one or more curves having one or more respective spatial positions relative to an outer surface of the object, and superimposing the one or more curves on the 3D rendering of the object at one or more rendering positions corresponding to the one or more spatial positions relative to the outer surface of the object, respectively.

[0006] In one embodiment, the input device is pressure-sensitive and configured to sense pressure applied to the input device as the input device physically traces at least a portion of the exterior surface of the object and to output data representative of the pressure, and the processor is configured to determine one or more widths for each of the one or more curves based at least in part on the pressure applied to the input device as the input device physically traces at least a portion of the exterior surface of the object to form the one or more curves, and to superimpose the one or more curves, each having the one or more widths, on a 3D rendering of the object.

[0007] In one embodiment, the input device includes a pressure-sensitive tip operable to detect pressure applied to the input device as the input device physically traces at least a portion of the exterior surface of the object. In one embodiment, the input device includes a first control input operable to receive one or more respective width indications of the one or more curves. The input device is configured to output data representing the one or more respective width indications to the processor, the processor being configured to receive the data representing the one or more respective width indications, determine one or more widths for each of the one or more curves based on the data representing the one or more respective width indications, and superimpose the one or more curves, each having the one or more widths, on a 3D rendering of the object. In one embodiment, the display is a head-mounted display configured to display a 3D rendering of the object superimposed on a physical object otherwise visually viewable through the head-mounted display.

[0008] In one embodiment, a system includes a three-dimensional (3D) scanner configured to scan an exterior surface of a physical object and output data representing the exterior surface of the object. The system includes a processor configured to receive the data representing the exterior surface of the object, generate a 3D model of the object based on the received data, and output a 3D rendering of the object based on the generated 3D model. In one embodiment, the system includes a display configured to receive the 3D rendering of the object and display the 3D rendering of the object, and an input device operable to physically trace at least a portion of the exterior surface of the object. The system includes a tracking device configured to track positioning of the input device as the input device traces at least a portion of the exterior surface of the object and to output data representing at least one position of the input device in 3D space as the input device traces the exterior surface of the object. The processor is configured to receive the data representing the at least one position of the input device, modify the 3D model of the object based at least in part on the data representing the at least one position of the input device, generate an updated 3D rendering of the object based on the modified 3D model, and output the updated 3D rendering of the object to the display in response to generating the updated 3D rendering of the object. In one embodiment, the display is configured to display an updated 3D rendering of the object.

[0009] In one embodiment, the processor is configured to generate a 3D model of the object by generating a polygon mesh including a plurality of vertices and a plurality of edges. In one embodiment, the processor is configured to modify the 3D model of the object by changing at least one vertex of the plurality of vertices or one edge of the plurality of edges to correspond to at least one position of the input device in 3D space. In one embodiment, the processor is configured to modify the 3D model of the object by adding to the plurality of vertices at least a first vertex having a position in space corresponding to the at least one position of the input device in 3D space. In one embodiment, the processor is configured to modify the 3D model of the object by at least removing from the plurality of vertices a second vertex having a position in 3D space closest to the position of the first vertex. In one embodiment, the display is a head-mounted display configured to display a 3D rendering of the object superimposed on a physical object otherwise visually viewable through the head-mounted display, and further configured to display an updated 3D rendering of the object superimposed on the physical object otherwise visually viewable through the head-mounted display.

[0010] In one embodiment, the system includes a three-dimensional (3D) scanner configured to scan an exterior surface of a physical object and output data representing the exterior surface of the object. In one embodiment, the system includes a processor configured to receive data representing the exterior surface of the object, generate a 3D model of the object based on the received data, and output a 3D rendering of the object based on the generated 3D model. In one embodiment, the system includes a display configured to receive the 3D rendering of the object and display the 3D rendering of the object. The system includes an input device operable to physically trace at least a portion of the exterior surface of the object, and a tracking device configured to track positioning of the input device as the input device traces at least a portion of the exterior surface of the object and output data representing at least two positions of the input device as the input device traces the object. The processor is configured to receive the data representing the at least two positions, determine a distance between the at least two positions, and output data representing the distance.

[0011] The processor is configured to identify a curve and determine a distance between the at least two positions along the identified curve based on data representing positions of the input device between the at least two positions. The display is configured to receive data representing the distance and display the distance on the display. The input device includes a control input operable to receive a selection of a first operating mode among a plurality of operating modes of the input device and to output data indicative of the first operating mode.

[0012] In one embodiment, the processor is configured to receive data indicative of a first operational mode and, in response to receiving the data indicative of the first operational mode, determine a distance between at least two positions and output data representing the distance. In one embodiment, the input device receives a selection of a second operational mode from the input device's multiple operational modes via the control input and outputs data indicative of the second operational mode. The processor is configured to receive data indicative of the second operational mode and, in response to receiving the data indicative of the second operational mode, augment a 3D rendering of the object based on positioning information received from a tracking device tracking the input device as the input device traces at least a portion of an outer surface of the object. The processor is configured to receive data indicative of the second operational mode and, in response to receiving the data indicative of the second operational mode, modify a 3D model of the object based on positioning information received from a tracking device tracking the input device as the input device traces at least a portion of an outer surface of the object, and generate an updated 3D rendering of the object based on the modified 3D model. [Brief explanation of the drawings]

[0013] [Figure 1] FIG. 1 shows a three-dimensional (3D) scanner scanning a physical object. [Figure 2] Figure 2 shows a 3D rendering system. [Figure 3] FIG. 3 illustrates an input device according to one embodiment of the present disclosure. [Figure 4] FIG. 4 shows a flow diagram of a method for enhancing 3D rendering of an object. [Figure 5] FIG. 5 illustrates a flow diagram of a method for modifying a 3D rendering of an object based on the position of an input device. [Figure 6] FIG. 6 shows a flow diagram of a method for distance measurement based on the position of an input device. DETAILED DESCRIPTION OF THE INVENTION

[0014] 1 shows a three-dimensional (3D) scanner 102 scanning a physical object 101. The 3D scanner 102 may be any device configured to scan the physical object 101 or the exterior surface of the physical object 101 to generate a three-dimensional model of the physical object 101. The 3D scanner 102 may be a non-contact scanner or a contact scanner. Furthermore, the 3D scanner 102 may be an active scanner or an inactive scanner. The 3D scanner 102 may use any technique for scanning an object, such as time-of-flight (ToF) or triangulation.

[0015] The 3D scanner 102 may be a ToF 3D laser scanner. The 3D scanner 102 may be an active scanner that uses laser light to probe the physical object 101. The 3D scanner 102 may be a volumetric scanner. The 3D scanner 102 may include a ToF laser range finder. The laser range finder may determine the distance between the 3D scanner 102 and the surface of the physical object 101 based on the timing of the round-trip time of a pulse of light emitted by the 3D scanner 102. The 3D scanner 102 emits a laser pulse, detects the reflection of the laser pulse reflected by the surface of the physical object 101, and determines the time required between the time the laser pulse is emitted and the time the reflection of the laser pulse is detected (round-trip time). The 3D scanner 102 determines the distance between the 3D scanner 102 and the surface of the physical object 101 based on the determined time and the speed of light.

[0016] The 3D scanner 102 may emit laser pulses in a direction to scan the physical object 101. The 3D scanner 102 accordingly scans the physical object 101 from multiple viewpoints. The ToF laser range finder may scan the entire field of view one point at a time, or may change the viewing direction of the ToF laser range finder to scan different points on the outer surface of the object 101. The viewing direction may be changed either by rotating the range finder or by using a rotating mirror system, among others.

[0017] 2 shows a 3D rendering system 106. The system 106 includes a 3D scanner 102, a 3D rendering device 108 (shown in block diagram form), a display 110 (shown pictorially, for example as a head-mounted display), an input device 112, and a tracking device 113 for the input device 112. The 3D rendering device 108 includes a processor 114, a memory 116, and one or more communication devices 118. The memory 116 and the one or more communication devices 118 are communicatively coupled to the processor 114. The 3D rendering device 108 is communicatively coupled to the 3D scanner 102, the display 110, the input device 112, and the tracking device 113.

[0018] The processor 114 may be any type of computing device configured to perform the operations described herein. The processor 114 may be, among other things, a graphics processing unit (GPU) or a central processing unit (CPU). The processor 114 may also be, among other things, a controller, a microcontroller, or a microprocessor. The memory 116 may be any type of storage device configured to store data. The data may be graphics data (such as a 3D rendering of the surface of the physics object 101) or executable instructions that, when executed by the processor 114, cause the processor to perform the operations described herein.

[0019] The one or more communication devices 118 may be any type of communication device configured to exchange or communicate data with other communication devices. The communication device 118 may be a wireless communication device or a wired device, such as a modem or a transceiver, among others. The communication device 118 may receive data from or transmit data to another communication device. Although not shown in FIG. 2 , the other communication device may be part of the 3D scanner 102, the display 110, the input device 112, and / or the tracking device 113. The one or more communication devices, which may include one or more communication devices, may communicate using any type of protocol associated with each communication device. The protocol may be, among others, the Institute of Electrical and Electronics Engineers (IEEE) 802.11 protocol, the Bluetooth protocol, the Universal Serial Bus (USB) protocol, or a cellular communication protocol such as the 3rd Generation Partnership Project (3GPP®) Long Term Evolution (LTE) protocol.

[0020] It should be noted that the 3D rendering device 108 may be a computer, a tablet, or a smartphone, among others. The 3D rendering device 108 may be separate from the display 110 or the tracking device 102. However, in alternative embodiments, the 3D rendering device 108 may be part of the display 110 or the tracking device 102, and the operations performed by the 3D rendering device 108 may instead be performed by the display 110 and a processor, memory, or one or more communication devices of the 3D rendering device 108.

[0021] The 3D rendering device 108 receives, via one or more communication devices 118, signals carrying data representing the scanned physical object 101. The signals may be modulated and coded according to the respective modulation and coding of the communication protocols used by the one or more communication devices 118.

[0022] The one or more communication devices 118 demodulate and encode the signals and output data representing the scanned physical object 101 to the processor 114. The processor 114 evaluates the data representing the scanned physical object 101. The processor 114 generates a 3D model of the physical object 101 based on the data representing the physical object 101. The 3D model of the physical object 101 may include a polygon mesh including vertices and edges. The polygon mesh may also include surfaces. Each surface may be between three or more respective edges of the edges. One of the vertices has a spatial location that corresponds to a spatial location of a point on the exterior surface of the physical object 101. The vertices, edges, and surfaces virtually (and digitally) represent the scanned physical object 101. The processor 114 stores the 3D model of the physical object 101 in memory 116. The processor 114 outputs the 3D model of the physical object 101 to the display 110 via one or more communication devices 118.

[0023] Display 110 may be a head-mounted display (HMD). As a head-mounted display, display 110 may be a virtual reality display or an augmented reality display. As an augmented reality display, display 110 may be transparent or semi-transparent. Thus, a viewer viewing physical object 101 through display 110 sees physical object 101 due to the transmissive properties of display 110. Because physical object 101 is viewable through display 110, display 110 may superimpose a 3D rendering of physical object 101 onto physical object 101 using a 3D model of the object. Thus, in such an embodiment, a viewer sees a 3D rendering of physical object 101 overlaid on physical object 101.

[0024] A viewer or user may use the input device 112 to annotate, enhance, refine, or modify (collectively "enhance") the 3D rendering of a physical object. A user may use the input device 112 to enhance the 3D rendering of a physical object by drawing, typically, one or more curves, or any other shape, on the 3D rendering. In this regard, the user may trace the input device or the tip of the input device in three-dimensional space over at least a portion of the physical object 101. The tracking device 113 tracks the position of the input device 112 in three-dimensional space and outputs data representing the position to the 3D rendering device 108. The 3D rendering device 108 receives the data representing the position of the input device 112 and generates an augmented 3D rendering of the physical object based on the data representing the tracked position of the input device 112. As will be appreciated from the description herein, the augmented 3D rendering of a physical object may include designs and features that appear virtually on or relative to the surface of the physical object but that do not otherwise appear in the physical object's actual three-dimensional space.

[0025] FIG. 3 illustrates an example of an input device 112 according to one embodiment. The input device 112 includes a housing 119, a tip 120, a marker 122, and multiple control inputs 124a, 124b, and 124c. The tip 120 may be pressure-sensitive. The marker 122 may be located on the tip 120 of the input device 112. In other embodiments, the marker 122 may be located elsewhere on the input device 112. The marker 122 may be a passive marker or an active marker used to track and determine the position of the tip 120. For example, the marker 122 may have a reflective coating that reflects light. Alternatively, or in addition, the marker 122 may be a light-emitting diode (LED) that actively illuminates light to track the tip 120 of the input device 112. In various embodiments, the marker 122 may be a strobe light that emits light having a specified wavelength or signature. In various embodiments, the input device 112 may be markerless, such that the position of the tip 120 or other portion of the input device may be tracked based on the shape or other characteristics of the input device 112 .

[0026] 2 , tracking device 113 tracks the spatial position of input device 112 or markers 122 on input device 112 as input device 112 moves through three-dimensional space. Tracking device 113 determines the spatial position of marker 122 and outputs data representing the position to 3D rendering device 108. In at least one embodiment, tracking device 113 may include one or more cameras, such as motion capture cameras, that capture images of marker 122 and may determine the position of the marker, and thus tip 120 and input device 112, based on the captured images.

[0027] The tracking device 113 may include a communication device (not shown). The tracking device 113 sends a signal via the communication device that includes data representing the spatial position of the input device 112. The 3D rendering device 108 receives the signal via one or more communication devices 118 and outputs the data representing the spatial position to the processor 114. The processor 114 determines the position of the input device 112 or the marker 122 based on the received position data. The processor 114 then augments the 3D rendering of the physical object based on the received position data.

[0028] For example, a user may physically trace the exterior surface of the physical object 101 with the input device 114 or the tip 120 of the input device 114 to draw a line, or generally a curve. In this manner, the input device 114 may be used to sketch (or diagram) a 3D rendering of the physical object. As the user traces the exterior surface of the physical object 101, the tracking device 113 tracks the spatial position of the tip 122 and outputs data representing the position to the 3D rendering device 108. The 3D rendering device 108 augments the 3D rendering of the physical object by adding a corresponding curve to the 3D rendering of the physical object. The curve may be a set of points that are connected to each other and have a position in space that corresponds to the position of the tip detected by the tracking device 113. The 3D rendering device 108 superimposes the curve onto the 3D rendering of the physical object. The 3D rendering device 108 then generates an augmented 3D rendering of the physical object. The augmented 3D rendering includes the 3D rendering of the (previously generated) physical object with the curve superimposed.

[0029] The 3D rendering device 108 outputs data representing an augmented 3D rendering of the physical object to the display 110. The display 110 displays the augmented 3D rendering of the physical object. Note that detecting the spatial position of the input device 112, generating the augmented 3D rendering, and outputting the data representing the augmented 3D rendering to the display 110 may be performed in real time. Thus, a user viewing the display 110 sees a curve in the augmented 3D rendering in real time as the user "draws" using the input device 112 (or as the user uses the input device 112 to trace the outer surface of the physical object 101). Note that the term "curve" is used herein to refer to any general shape drawn by the user using the input device 112. The curve may be, for example, a straight line or any other shape.

[0030] In one embodiment, the tip 120 of the input device 112 may be pressure-sensitive. The input device 112 may detect pressure applied to the tip by the user as the user manipulates the input device 112. The pressure may be used to determine the thickness of a curve drawn by the user. The input device 112 may output data representing the pressure applied to the tip 120. The input device 112 may output the pressure data to the 3D rendering device 108. As described herein, the input device 112 may include a communication device (not shown) operable to communicate with one or more communication devices 118 of the 3D rendering device 108 and operable to output a signal including data representing the pressure applied to the tip 120. The one or more communication devices 118 of the 3D rendering device 108 may receive the signal and output data representing the pressure to the processor 114. The processor 114 determines the pressure based on the received pressure data. The processor 114 renders the curve with a line thickness corresponding to the determined pressure. The relationship between pressure and thickness may be proportional, such that a greater amount of pressure applied by the user results in rendering a thicker curve.

[0031] The processor 114 may evaluate the determined pressure along with the position of the tip 120. The processor 114 generates a curve based on both the pressure data and the position data to be superimposed on a 3D rendering of the physical object, with the thickness of the curve at a location in space corresponding to the determined pressure applied to the tip 120 at that location in space.

[0032] Multiple control inputs 124a-c of the input device 112 may be used to control the attributes of the curve. For example, a first control input 124a may be used to select between operational modes of the input device 112. A first operational mode may be an enhancement of the 3D rendering as described herein, in which one or more additional curves are superimposed on the 3D rendering. A second operational mode may be a modification of the 3D rendering, and a third operational mode may be a distance measurement as described herein. A user may operate the first control input 124a, which may be a multi-pole or multi-directional switch, to select from various available operational modes.

[0033] Similarly, the second and third control inputs 124b, 124c may be used to select attributes of the curve, such as the color, type, or thickness of the lines that make up the curve. In one embodiment, the second control input 124b may be used to select the color of the curve, such as red, green, or blue, among others, and / or the type of curve, such as solid or dashed, among others. In one embodiment, the third control input 124c may be used to select a static or constant thickness of the curve. The thickness selected using the third control input 124c may override or take precedence over the thickness determined based on the pressure applied to the tip 120. In one embodiment, the control inputs may be user-configurable. For example, a user may specify a control input function associated with each of the control inputs 124a-c that is different from the default control input function of the input device 112.

[0034] It should be noted that the input device 112 of FIG. 3 is exemplary and non-limiting. In various embodiments, any other type of input device 112 may be used. The input device 112 may have a different form factor than that illustrated in FIG. 3. In one embodiment, the input device may be a joystick, touchpad, pressure-sensitive pad, or wheel, among others. Furthermore, the input device 112 may have more or fewer control inputs than those illustrated in FIG. 3.

[0035] Input device 112 outputs data representing the selected motion mode and / or curve attributes to 3D rendering device 108. 3D rendering device 108 receives the data representing the selected motion mode and / or curve attributes and uses this data, along with data representing the position of tip 120, to generate an augmented 3D rendering of the physical object. For example, 3D rendering device 108 may color the curve or render the curve to have a thickness according to the received attributes.

[0036] In addition to or instead of enhancing the 3D rendering of the physics object 101, the 3D rendering device 108 may refine or modify the 3D rendering of the physics object 101 based on user input provided using the input device 112. To refine or modify the 3D rendering of the physics object (and improve the accuracy of the 3D rendering), a user may use the input device to trace the exterior surface of the physics object 101. For example, the user may trace the physics object 101 to provide a precise location of the tip 120 at or near the exterior surface of the physics object 101. The location of the tip 120 is then used to modify the 3D rendering of the physics object 101 and improve the accuracy of the 3D rendering of the physics object 101.

[0037] As the user utilizes the input device 112 to trace the exterior surface of the physical object 101, the tracking device 113 tracks the position of the tip. The tracking device 113 outputs data representing the spatial position of the tip 120 to the 3D rendering device 108. The position may be expressed in a Cartesian coordinate system of three-dimensional space as three coordinates (e.g., (x, y, z)) or a position in space expressed in a polar coordinate system as three coordinates relative to a reference point (or origin) (e.g., radial distance, polar angle, and azimuth angle). The position tracking of the input device 112 may have a more precise spatial resolution than the 3D scanner 102 separately used to generate a three-dimensional model of the physical object, as described above with respect to FIGS. 1 and 2 . The 3D rendering device 108 receives the data representing the tracked position of the tip 120 of the input device 112 and uses the tracked position data to adjust or modify the 3D model that provides a 3D rendering of the physical object.

[0038] As described herein, the 3D rendering of the physics object may include a plurality of vertices, with each pair of vertices connected by one of the plurality of edges. The 3D rendering device 108 may set the position of the tip 120 received from the tracking device 113 as one of the plurality of vertices. Thus, the 3D rendering of the physics object is adjusted based on the data position received from the tracking device 113. Furthermore, the 3D rendering device 108 may remove existing vertices of the 3D rendering and replace them with vertices of the received position of the input device 112. The removed vertices may be vertices having positions in Euclidean space closest to the received position of the input device 112. The 3D rendering device 108 may remove the vertices and replace them with new vertices having positions corresponding to (or identical to) the spatial position of the tip 120 received from the tracking device 113. In this manner, the 3D rendering device 108 uses the tracked position data of the input device 112 to iteratively refine the 3D rendering of the physics object as the input device 112 traces portions of the surface of the physics object. Based on the adjustments made to the 3D model of the physics object, the 3D rendering device 108 generates an updated 3D rendering of the physics object 101 and outputs data representing the updated 3D rendering to the display 110.

[0039] Thus, the 3D rendering device 108 first generates a 3D model of the physical object 101 based on data representing the scanned physical object 101 output by the 3D scanner 102. The 3D rendering device 108 then refines the 3D model based on data representing the position of the input device 112 or the tip 120 of the input device 112 as the input device 112 traces portions of the surface of the physical object. Thus, the 3D rendering device 108 progressively improves the 3D rendering of the physical object.

[0040] In one embodiment, the system 106 may be used to measure distances in space. This distance, which may be a Euclidean distance, may exist anywhere in space. The distance may be, for example, between two points on the exterior surface of the physical object 101. To measure the distance, a user may place the tip 120 of the input device 112 at a first location and move the tip 120 along the surface of the physical object to a second point different from the first point.

[0041] When the tip 120 is at the first point, the tracking device 113 determines a first spatial position of the tip and outputs the first position data to the 3D rendering device 108. The 3D rendering device 108 stores the first position data in memory 116. The user then moves the tip 120 of the input device 112 along the surface of the physical object to a second point. The tracking device 113 determines a second position associated with the second spatial position. The tracking device 113 outputs the second position to the 3D rendering device 108. Upon receiving the first and second positions, the 3D rendering device 108 determines the Euclidean distance between the first and second positions. The 3D rendering device 108 then outputs data indicative of the distance to the display 110 for display to the user or to any other device that outputs the distance to the user.

[0042] It should be noted that in various embodiments, the distance may be the linear distance between two points, such as a first and second point. Additionally or alternatively, the distance may be the length of an arc or curve traced by the tip 120 of the input device 112. As the user traces the curve, the tracking device 113 determines the spatial position of the tip 120 in real time and outputs data representing the position to the 3D rendering device 108. It is recognized that it may be advantageous for the user to slowly trace the curve or arc, allowing the tracking device 113 to determine various positions of the tip 120 in small distance increments relative to each other and with greater granularity. Determining the displacement of the tip 120 in smaller increments leads to improved accuracy in determining the length of the curve.

[0043] It should be noted that in various embodiments, the tracking device 113 may be part of the 3D scanner 102, or the tracking device 113 may be unnecessary and the 3D scanner 102 may perform the tracking functions performed by the tracking device 113. Thus, the 3D scanner 102 may track the spatial position of the input device 112 and output data representing the tracked position to the 3D rendering device 108. The tracking device 113 may be an outside-in tracking device, in which a camera or other sensor at a fixed location and pointed towards the input device 112 tracks the movement of the input device as it moves within the viewable range of the camera or other sensor. Furthermore, the tracking device 113 may be part of or included in the head-mounted display or the 3D rendering device 108. Alternatively, or in addition, the display 110 may include inside-out tracking, whereby the display 110 may include a camera that "watches" or observes the external surrounding environment or space to determine the position of the display 110 or input device 112 relative to that environment or space.

[0044] Figure 4 shows a flow diagram of a method 400 for enhancing 3D rendering of an object. In method 400, a 3D scanner, such as the 3D scanner 102 described with reference to Figure 1, scans an exterior surface of a physical object at 402. At 404, a 3D rendering device, such as the 3D rendering device 108 described with reference to Figure 2, generates a 3D model of the object based on data resulting from scanning the exterior surface at 402. The 3D model of the object may include vertices, edges, and surfaces determined from the 3D scan of the object's exterior surface.

[0045] At 406, a display, such as display 110 described with reference to FIG. 2, displays a 3D rendering of the physical object based on the generated 3D model. The display may be a virtual reality (VR) or augmented reality (AR) display. The physical object may be transparently viewable through the display. The display may superimpose a 3D rendering on the physical object, which is otherwise viewable through the display. At 408, a tracking device, such as tracking device 113 described with reference to FIG. 2, tracks positioning of the input device as the input device physically traces at least a portion of the object's exterior surface. At 410, the tracking device determines at least one spatial position of the input device as the input device traces the object's exterior surface.

[0046] At 412, a 3D rendering device augments a 3D rendering of the object based at least in part on one or more tracked positions of the input device. A user may physically trace the input device over a portion of the exterior surface of a physical object to draw a curve or arbitrary shape. The tracking device tracks the input device as the user physically traces the input device over the exterior surface of the physical object. Data representing the spatial position of the input device is provided to the 3D rendering device, which uses this data to determine the shape of the curve as well as its position relative to the 3D rendering of the object. The 3D rendering device augments the 3D rendering to include a rendering of the curve. At 414, a display displays the augmented 3D rendering of the object.

[0047] Figure 5 shows a flow diagram of a method 500 for modifying a 3D rendering of an object based on a tracked position of an input device. Steps 502, 504, 506, 508, and 510 of method 500 are similar to steps 402, 404, 406, 408, and 410 of method 400 described with reference to Figure 4. Method 500 includes scanning an exterior surface of a physical object at 502, generating a 3D model of the object based on the scan of the exterior surface at 504, and displaying a 3D rendering of the object based on the generated 3D model at 506. Method 500 also includes tracking positioning of the input device as the input device physically traces at least a portion of the exterior surface of the object at 508, and determining at least one spatial position of the input device as the input device traces the exterior surface of the object at 510.

[0048] To provide more accurate physical coordinates of the surface of a physical object, a user may physically trace an input device over the surface of the physical object. By tracing or positioning the input device over the surface of the physical object while tracking the input device, the user effectively provides the positioning (or coordinates) of the surface. The more accurate data reflecting the positioning of the surface can be used to correct and improve the 3D rendering of the physical object (e.g., if a 3D scan of the object is inaccurate).

[0049] Thus, as opposed to augmenting the 3D rendering, method 500 proceeds at 512 to modify, by the 3D rendering device, a 3D model of the object based at least in part on one or more tracked positions of the input device. The tracked spatial position of the input device is used to refine or improve the accuracy of the 3D model of the object rather than augmenting or appending to the 3D rendering. As described herein, the position of the input device is included as a vertex in the modified 3D model of the object. After modifying the 3D model of the object, the display displays an updated 3D rendering of the object at 514 based on the modified 3D model of the object.

[0050] 6 shows a flow diagram of a method 600 of distance measurement based on one or more tracked positions of an input device. In method 600, a tracking device tracks the spatial positioning of an input device as the input device traces at least a portion of an outer surface of a physical object at 602. A user may trace the outer surface of the physical object to measure the distance between two points or positions along the outer surface of the physical object. At 604, the tracking device determines at least two positions of the input device as the input device traces the object.

[0051] At 606, the 3D rendering device determines a distance between at least two positions. The distance may be a Euclidean distance between the at least two positions. The distance may be a linear distance along a straight line or a distance along a curve traversed by the input device. A curve traversed by the input device may be approximated by multiple short line segments extending between multiple sensed positions of the input device as the input device traverses the curve. The distance along the curve may be determined by summing the individual distances of the short line segments. At 608, the 3D rendering device outputs data representing the distance, which may be displayed on a display.

[0052] The various embodiments described above can be combined to provide further embodiments. These and other changes can be made to the embodiments in light of the above detailed description. In general, in the following claims, the terms used should not be construed to limit the claims to the specific embodiments disclosed in the specification and claims, but should be construed to include all possible embodiments, along with the full range of equivalents to which such claims are entitled. Accordingly, the claims are not limited by this disclosure.

Claims

1. an input device operable to physically trace an exterior surface of a physical object; a tracking device configured to output data representing a spatial position of the input device as the input device physically traces the exterior surface of the physical object; a computer configured to augment a 3D rendering corresponding to the physical object based on data representing the spatial position of the input device output from the tracking device, and output the augmented 3D rendering to a display; A system having:

2. the input device is configured to sense pressure applied to the input device as the input device physically traces an exterior surface of the physical object, and to output data representative of the pressure; the computer is configured to generate a curve having a width based on data indicative of the pressure output from the input device as the input device physically traces the exterior surface of the physical object, and to superimpose the curve having the width on the 3D rendering. The system of claim 1 .

3. the input device having a sensor operable to detect the pressure applied to the input device as the input device physically traces the exterior surface of the physical object; The system of claim 2 .

4. the input device includes a first control input operable to receive an indication of a width of the curve; the input device is configured to output data representing a width instruction to the computer; The computer receiving data representing the width indication; configured to superimpose on the 3D rendering of the physical object a curve having a width based on the data representing the width indication. The system of claim 2 .

5. the display is a head-mounted display, the head-mounted display is configured to display the 3D rendering superimposed on an entity of the physical object, while the entity of the physical object is visually visible through the head-mounted display when the 3D rendering is not displayed. The system of claim 1 .

6. augmenting a 3D rendering corresponding to a physical object based on data representing a spatial position of an input device output from a tracking device configured to output data representing a spatial position of the input device as the input device, operable to physically trace an exterior surface of the physical object, physically tracing the exterior surface of the physical object; outputting the augmented 3D rendering to a display; method.

7. the input device is configured to sense pressure applied to the input device as the input device physically traces an exterior surface of the physical object, and to output data representative of the pressure; forming a curve having a width based on data indicative of the pressure output from the input device as the input device physically traces the outer surface of the physical object; Superimposing a curve having said width on said 3D rendering. The method of claim 6.

8. the input device having a sensor operable to detect the pressure applied to the input device as the input device physically traces the exterior surface of the physical object; The method of claim 7.

9. receiving data representing a width instruction from the input device configured to output data representing the width instruction; superimposing a curve having a width based on data representing the width indication onto the 3D rendering of the physical object; The method of claim 7.

10. the display is a head-mounted display, the head-mounted display is configured to display the 3D rendering superimposed on an entity of the physical object, while the entity of the physical object is visually visible through the head-mounted display when the 3D rendering is not displayed. The method of claim 6.

Images