Tracking wand system and methods

Abstract

A tracking wand system is provided, including a tracking station, a camera assembly comprising a first camera and a second camera supported by the tracking station, and a tracking wand comprising a light source. A controller receives outputs from the first camera and the second camera to track a first location of the tracking wand relative to the tracking station, track a second location of the tracking wand relative to the tracking station, and map the first location and the second location in a three-dimensional coordinate system. The controller also digitally renders a workpiece model that spans the first location to the second location and outputs the workpiece model to a user.

Description

TRACKING WAND SYSTEM AND METHODSCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority under 35 U.S.C. § 119 to U.S. Provisional Patent Application No. 63 / 626,487 filed on January 29, 2024, U.S. Provisional Patent Application No. 63 / 641,206 filed on May 1, 2024, and U.S. Provisional Patent Application No. 63 / 671,368 filed on July 15, 2024, the entire contents of which are incorporated herein by reference.BACKGROUND

[0002] In construction applications, the fabrication of a workpiece (e.g., a piece of electrical metallic tubing (EMT) or conduit) may be a labor-intensive process that requires a user to manually take linear measurements, calculate bend angles, and determine the orientation of the workpiece (e.g., conduit, etc.) in space.SUMMARY

[0003] Some aspects of the disclosure provide a tracking wand system including a tracking station, a camera assembly comprising a first camera and a second camera supported by the tracking station, and a tracking wand comprising a light source. The tracking wand system further comprises a controller in communication with the camera assembly. The controller is to receive outputs from the first camera and the second camera to track a first location of the tracking wand relative to the tracking station, track a second location of the tracking wand relative to the tracking station, and map the first location and the second location in a three- dimensional coordinate system. The controller is further to digitally render a workpiece model that spans the first location to the second location and output the workpiece model to a user.

[0004] Some aspects of the disclosure provide a method including tracking a tracking wand in an environment with a dual-camera assembly of a tracking station, receiving a first input from the tracking wand to mark a first reference point in the environment, and determining a first location of the first reference point using information from the dual-camera assembly. The method also includes receiving a second input from the tracking wand to mark a second reference point in the environment, determining a second location of the second reference point using information from the dual-camera assembly, and mapping the first location and the second location in a three-dimensional coordinate system. The method further includes connecting the first location to the second location in the three-dimensional coordinate system to create a workpiece model, and outputting a digital rendering of the workpiece model via a display.

[0005] Some aspects of the disclosure provide a method including tracking a tracking wand in an environment with a dual-camera assembly of a tracking station, determining a physical location of the tracking wand in the environment using information from the dual-camera assembly, and receiving inputs from the tracking wand to mark a plurality of reference point locations in the environment. The method further includes generating a workpiece model based on the plurality of reference point locations, where the workpiece model is a digital rendering of a workpiece routed in the environment according to the reference point locations, and outputting an image of the workpiece model via a display.BRIEF DESCRIPTION OF THE DRAWINGS

[0006] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments of the invention and, together with the description, serve to explain the principles of embodiments of the invention:

[0007] FIG. 1 is a block diagram view of connected tool system according to aspects of the present disclosure.

[0008] FIG. 2 is an isometric view of an example tracking wand system according to some aspects.

[0009] FIG. 3 is another isometric view of the tracking wand system of FIG. 2.

[0010] FIG. 4 is a partial isometric view of a tracking wand of the tracking wand system of FIG. 2, within a casing of a tracking station.

[0011] FIG. 5 is an isometric view of the tracking wand of FIG. 4.

[0012] FIG. 6 is a partial isometric view of the tracking wand system of FIG. 2.

[0013] FIG. 7 is a partial exploded view of the tracking wand system of FIG. 2.

[0014] FIG. 8 is another partial exploded view of the tracking wand system of FIG. 2.

[0015] FIG. 9 is a block diagram view of a tracking wand system according to some aspects.

[0016] FIG. 10 is a flowchart of an example method, according to some aspects, for operating a tracking wand system.

[0017] FIG. 11 is a flowchart of an example method, according to some aspects, for calibrating a tracking wand system.

[0018] FIG. 12 is a flowchart of an example method, according to some aspects, for creating a digital workpiece model using a tracking wand system.

[0019] FIG. 13 is an example view illustrating relative measurement components in space for use with the methods of FIGS. 10-12.

[0020] FIGS. 14A, 14B, and 14C are digital views of a point cloud representation, a wireframe model, and a workpiece model, respectively, associated with the method of FIG. 12.

[0021] FIG. 15 is a digital view of a display of a computing device illustrating a digitally rendered workpiece model.DETAILED DESCRIPTION

[0022] The following discussion is presented to enable a person skilled in the art to make and use embodiments of the invention. Given the benefit of this disclosure, various modifications to the illustrated embodiments will be readily apparent to those skilled in the art, and the principles herein can be applied to other embodiments and applications without departing from embodiments of the invention. Thus, embodiments of the invention are not intended to be limited to embodiments shown, but are to be accorded the widest scope consistent with the principles and features disclosed herein.

[0023] As generally noted above, the fabrication of a workpiece may be a labor-intensive process that requires a user to manually take linear measurements, calculate bend angles, and determine the orientation of the workpiece in space. For example, currently, measurements and calculations are all handled through on-the-job training and reference books or applications. Further, users must plan the workpiece orientation mentally, without a visual representation of how the final product will look. As a result, the fabrication of the workpiece can be a timeconsuming and error prone process, which is undesirable in many construction applications.

[0024] To mitigate these issues, the user may utilize a visualization system (e.g., a mobile application), which may generate a model (e.g., a three-dimensional model) of the workpiece. In one example, the user may manually create a generalized workpiece that extends from a first location to a second location. The user may then use a tool, such as a tracking wand system, to take measurements as instructed by the visualization system. For example, these measurements may correspond to a work area where the workpiece is desired to be placed (e.g., between the first and second points). The measurement data can then be wirelessly communicated to a computing device (e.g., a mobile device, server, etc.) and associated with segments of the generalized workpiece in the visualization system. The visualization system may then generate a three-dimensional rendering of the workpiece within the simulated environment based on the inputted measurements, which automatically calculates the bend angles and positions on the workpiece. Following this, the user may use measurements associated with the application- generated rendering as instruction on where to cut or bend a workpiece or to provide instructions to a tool to automatically cut or bend a w orkpiece.

[0025] FIG. 1 shows an example of a connected tool system 100. The tool system 100 can include one or more tools (e.g., tools 102, 104, etc.) and a computing device 106. In some examples, the computing device 106 can be implemented as a mobile phone (e.g., a smart phone), a personal digital assistant (“PDA”), a laptop, a notebook, a netbook computer, a tablet computing device, etc. In some examples, the computing device 106 may be configured to communicate directly with the tools 102, 104 via a communication system 108 of the computing device 106. For example, the communication system 108 may permit the computing device 106 to exchange information with the tools 102, 104. In one particular example, the communication system 108 may permit the computing device 106 to receive information from a tool (e.g., tool 102, such as a tracking wand system) and output information to another tool (e.g., tool 104, such as an automated pipe bender). In some examples, the communication system 108 may be a wireless communication system (e.g., a wireless transceiver) or a wired communication system (e.g.. via a physical, wired network).

[0026] In some examples, the computing device 106 may include one or more controllers 110 each having a processor 112 and a memory 114. The processor 112 can be implemented as a programmable processor, an application specific integrated circuit (ASIC), one or more field programmable gate arrays (FPGAs), a group of processing components, or other suitable electronic processing components. The memory 114 (e.g., memory, memory unit, storage device, etc.) can include one or more devices (e.g., RAM, ROM, Flash memory, hard disk storage, etc.) for storing data or computer code for completing or facilitating the various processes, layers and modules described herein. The memory 114 can be or include volatile memory or non-volatile memory. The memory 114 can include database components, object code components, script components, or any other type of information structure for supporting the various activities and information structures described in the present application.

[0027] The computing device 106 may further include a display 116 and a corresponding user interface 118. In some examples, the display 116 and user interface 118 may permit one or more users (e.g., users 120, 122) to interact with the computing device 106. For example, the users 120, 122 may interact with the user interface 118 to input information into the computing device 106. In some examples, the information inputted into the computing device 106 may be depicted on the display 116 for review by the user(s) 120, 122.

[0028] In some examples, to facilitate the fabrication of a workpiece (e.g., EMT tubing, conduit, etc.) within a work area, the computing device 106 may include a visualization system 124. The visualization system 124 may be in the form of a mobile application (e.g., computerprogram) stored within the memory 1 14 of the computing device 106 and executed by the processor 112. The visualization system 124 may permit the users 120, 122 to build and review a three-dimensional (3D) model of a workpiece prior to cutting, bending, or otherwise performing work on the workpiece.

[0029] For example, the user may build the 3D model of the workpiece within the visualization system 124 (e.g., using the user interface 118). The user may then input measurements corresponding to one or more segments of the workpiece for analysis by the visualization system. In some examples, the user may take measurements as prompted by the visualization system (e.g., via a traditional tape measure or other tool) and manually input those measurements into the visualization system 124. However, in other examples, the user may utilize a connected tool (e.g., the tracking wand system 102), and may automatically transmit measurements from the tracking wand system 102 to the visualization system 124. Based on the inputted measurements, the visualization system 124 may generate instructions that may be followed by the user (e.g., user 122) to cut, bend, or otherwise perform work on the workpiece, without having to perform manual calculations on cut or bend locations for the workpiece.

[0030] In some examples, the visualization system 124 may transmit instructions (e.g., via the communication system 108) to one or more connected tools (e.g., tool 104) to perform automated cuts, bends, or other operations on the workpiece. In further examples, the tools 102, 104 may directly communicate with one another, such that the tool 104 alternatively or additionally receives instructions directly from the tracking wand system 102 to perform automated cuts, bends, or other operations on the w orkpiece. Thus, as should be appreciated, the number of manual inputs from the users 120. 122 may be reduced and overall efficiency may be increased.

[0031] FIGS. 2-8 illustrate an example tracking wand system 102 that may be used with the connected tool system 100 of FIG. 1. In some examples, the tracking wand system 102 can include a tracking station 130, shown in FIGS. 2. 3, and 6-8, and a tracking wand 132. shown in FIGS. 3-5. In some examples, the tracking station 130 can include abase 134 that can support a camera assembly 136 and include a display mount 138 for receiving a computing device 106, such as the computing device 106 of FIG. 1. Furthermore, FIG. 9 illustrates a schematic view of an example tracking wand system 102, such as the tracking wand system 102 of FIGS. 2-8.

[0032] Generally, according to some examples, the tracking station 130 can serve as a fixed point relative to the handheld tracking wand 132 to track movement of the tracking wand 132 in space. For example, the tracking station 130 can support the camera assembly 136 and canalso house electronics in communication with the camera assembly 136 that are configured to interpret data from the camera assembly 136. Such electronics can determine a location of the tracking wand 132 and, optionally, transmit the determined location to the computing device 106 for use with the visualization system 124. According to some examples, as shown in FIG. 9, such electronics of the tracking station 130 can include, but are not limited to, a controller 140 including a processor 142 and memory 144, a communication system 146, and a power source 148.

[0033] Referring to FIGS. 2. 3, and 6-8, the base 134 of the tracking station 130 can house the electronics of the tracking station 130, support the camera assembly 136. support a connected computing device 106, and, optionally, hold the tracking wand 132 when not in use (as shown in FIG. 3). In some examples, as shown, the base 134 can be substantially cuboid shaped. However, the base 134 may take on other shapes in other examples. Additionally, in some examples, the base 134 can include carrying handles 150 to assist a user with carrying the tracking station 130 to a desired location. The base 134 can also include a recess 152 configured to receive the power source 148, such as a battery. In some examples, the battery may be a rechargeable battery that can be removably coupled to the base 134. such as Milwaukee Tool M12™. Ml 8™, or MX FUEL™ rechargeable battery, or other battery types.

[0034] Referring back to FIG. 2, one side of the base 134 can include the display mount 138, allowing a user to removably mount a computing device 106. With further reference to FIGS. 6 and 7, in some examples, the display mount 138 can be a surface for receiving mounting hardware 154 for mounting the computing device 106 on the tracking station 130. For example, as shown in the exploded view of FIG. 7, the computing device 106 (e.g., atablet in this example), can be housed within a case 156 having a front case portion 1 8 and a back case portion 160. The case 156 can be coupled to a case adapter 162, and a quick mount connector 164 can be coupled to the mounting hardware 154. The case adapter 162 can further include a quick mount 166 configured to be releasably coupled to the quick mount connector 164, allowing a user to quickly and easily mount or dismount the computing device 106, stored in the case 156, to the base 134. In other examples, the case 156 can be directly coupled to the mounting hardware 154. Additionally, in some examples, the mounting hardware 154 can be configured to pivot relative to the display mount 138. As a result, a user can raise, lower, and / or pivot the computing device 106 relative to the base 134 so that the display 116 is easily viewable. However, in other examples, the mounting hardware 154 may be configured to couple the computing device 106 to the base 134 at a fixed position and angle.

[0035] While the present description and illustrated example references the mounting hardware 154, in some examples, other coupling methods may be used to couple the computing device 106 to the base 134. According to one example, the display mount 138 can include a track or shelf (not shown) that allows the computing device 106 to slide into or rest upon the display mount 138, though other coupling methods may be contemplated in some examples. As an additional alternative, in some examples, the tracking station 130 itself can incorporate one or more components of the computing device 106, such as the visualization system 124, the display 116, and / or the user interface 118. In such an example, the display mount 138 can be replaced with an integrated display 116, which may be a touch-screen display that also incorporates a user interface 118. Accordingly, FIG. 9 illustrates the tracking wand system 102 as including the display 116, though the display 116 may be part of the tracking station 130 or part of a connected computing device 106.

[0036] As noted above, the base 134 can further support the camera assembly 136. Additionally, in some examples, the camera assembly 136 can rotate relative to the base 134. For example, generally, as shown schematically in FIG. 9, the camera assembly 136 can be a dual-camera assembly including a first camera 168, a second camera 170, and a rotating platform 172. With further reference to FIGS. 2. 3, 6. and 8, the camera assembly 136 can include the rotating platform 172, supported by a locking cap 174, a camera body 176 that holds the first camera 168 and the second camera 170 a fixed distance from one another and is supported on the rotating platform 172, a camera shell 178 enclosing the camera body 176 on the rotating platform 172, and a first lens 180 and a second lens 182 aligned with the first camera 168 and the second camera 170. respectively.

[0037] More specifically, in some examples, the rotating platform 172 can be coupled to the locking cap 174 and can rotate with the locking cap 174. For example, the locking cap 174 can be rotated via a connected motor (not shown), as controlled by the controller 140 of the tracking station 130. The camera body 176 can be supported on the rotating platform 172 and, as a result, can rotate with the rotating platform 172. Furthermore, the camera body 176 can be covered by the camera shell 178, which include openings 183 for receiving the lenses 180, 182, allowing the cameras 168, 170 to capture image data outside the camera shell 178. The image data can be communicated to and processed by the controller 140. From this processing, as further described below, the controller 140 can detect a location of the tracking wand 132 in space and can control the motor to rotate the rotating platform 172 (and camera body 176) to track movement of the tracking wand 132. In other examples, however, the camera assembly136 may not include a motorized rotating platform 172. In such examples, a user (such as user 120) may manually rotate the rotating platform 172 relative to the base 134 to a desired orientation.

[0038] Referring now to the tracking wand 132. generally, the tracking wand 132 can be a handheld device that a user may hold to map location points in an environment, e.g., of a workpiece, as tracked by the tracking station 130. For example, the tracking station 130 and, more specifically, the controller 140 of the tracking station 130, can utilize input from cameras 168, 170 and a triangulation algorithm to track the tracking wand 132 in the environment. The controller 140 can further use a stereo measuring algorithm to determine a location of the tracking wand 132 relative to the tracking station 130. That is, with the tracking wand 132, the user can “mark” key points in the environment corresponding to desired locations of the workpiece, and the tracking wand 132 can communicate to the controller 140 to track the locations of these key points and / or relative measurements between marked locations. For example, these key points may be end points of the workpiece, bend points, obstacles, etc. The controller 140 can further communicate these locations to the visualization system 124 (either as part of the tracking station 130 or a separate computing device 106), and a digital rendering of the workpiece can be generated using the measurements, then displayed to the user via the display 1 16.

[0039] Referring to FIGS. 3-5, generally, the tracking wand 132 can include a housing 184, an electronics base 186, a button 188, a light ring 192, and a cap 194. In some examples, the housing 184 can house a power source 198 (shown in FIG. 9), a light source 200 (shown in FIG. 9), and electronics 202 such as, but not limited to, a controller 204 and a communication system 206 (shown in FIG. 9). The electronics base 186 can support the electronics 202 and can form part of the housing 184, or can be housed entirely within the housing 184 in some examples. Furthermore, in some examples, as shown in FIGS. 3-5, the electronics base 186 can include an electronics port 208, which may allow a wired connection, e.g., to the communication system 206 to enable data communication, and / or to the power source 198 to charge the power source 198.

[0040] In some examples, the tracking wand 132 can be stored in the tracking station 130. For example, as shown in FIGS. 3 and 4, the base 134 of the tracking station 130 can include a case 189 configured to store the tracking wand 132. With further reference to FIG. 4. internally, the case 189 can include an internal cavity 191 to receive the tracking wand 132, as well a lever 190 and a pusher 196. In some examples, the lever 190 can be a pivoting lever that"locks" onto the tracking wand 132 within the cavity 191 (e g., at locking point 193 shown in FIG. 4) when a user pushes the tracking wand 132 into the cavity 191, preventing the tracking wand 132 from falling out of the cavity 191 unless the lever 190 is pressed to release the tracking wand 132 from the lever 190. Furthermore, in some examples, the pusher 196 can be a spring-loaded mechanism at an end of the cavity 191 that can urge the tracking wand 132 out of the cavity 191 when the user presses the lever 190 to release the tracking wand 132 from the lever 190.

[0041] Referring now to FIG. 9, with reference to the electronics 202 of the tracking wand 132, in some examples, the controller 204 can include a processor and a memory (not shown) storing program instructions executable by the processor. In some examples, the communication system 206 may be a wireless communication system (e.g., a wireless transceiver) or a wired communication system (e.g., via a physical, wired network, such as through the electronics port 208). In response to a user pressing the button 188, the controller 204 can communicate the button press to the tracking station 130 via the communication system 206.

[0042] Upon receipt of the button press notification from the tracking wand 132, the tracking station 130 can determine a location of the tracking wand 132 in 3D space, for example, using the light source(s) 200. More specifically, in some examples, the light sources 200 can emit light from the tracking wand 132 through the light ring 192. In some examples, the light sources 200 may continuously emit light while, in other examples, the light sources 200 emit light only when the button 188 is pressed, or emit light at a specific flash rate when the button 188 is pressed. The tracking station 130 can analyze image data from the cameras 168, 170, using the light as an indication of the tracking wand location 132 within the image data, and determine the location of the tracking wand 132 in 3D space. The tracking station 130 can further use a predetermined, known distance from the light ring 192 to an end 210 of the tracking wand 132, e.g., at an end of the cap 194, to determine the user's desired marking location in 3D space (e.g., at the physical endpoint 210 of the tracking wand 132).

[0043] For example, the tracking station 130 may include tracking wand information stored in the memory 144, such as the predetermined distance from the light ring 192 to the end 210, a type, color, and / or pattern of light to be emitted through the light ring 192, and / or other data. In some examples, when the tracking station 130 is paired with the tracking wand 132 for wireless communication, the tracking wand 132 can provide its respective tracking wand information to the tracking station 130. In further examples, a user may input this trackingwand information to the tracking station 130, e.g., through the user interface 118 of the computing device 106.

[0044] In some examples, the cap 194 and / or the light ring 192 can be removable from the housing 184. For example, a user may desire to swap out one cap 194 for a larger, smaller, longer, or shorter cap 194. Upon swapping out the caps 194, the user may input a new cap length or size through the user interface 118 so that the tracking station 130 may store this new cap length or size to extrapolate the distance from the light ring 192 to the end 210. In some examples, the cap 194 can include a specific physical and / or digital indicator on it for the user and / or the tracking station 130 to identify the cap 194 and its properties, allowing the user to input the properties to the traction station 130, or allowing the tracking station 130 to automatically identify the properties via the digital indicator. Example digital indicators can include, but are not limited to, a Radio Frequency Identification (RFID) tag, a Near Field Communication (NFC) tag, or a barcode that can be read by the tracking station 130. allowing the traction station 130 to identify' the cap 194. Additionally, in some examples, the cap 194 may be cylindrical in shape, spherical, cuboid, or another shape, and the shape properties can also be input to or read by the tracking station 130. As one example, a respective cap 194 can be sized and / or shaped to include a diameter approximately equal to a diameter of the proposed workpiece. Thus, a user can select and use a cap 194 that substantially matches the proposed workpiece to help the user better visualize key points when using the tracking wand 132.

[0045] In some examples, the tracking wand 132 can include a single light ring 192, as illustrated, so that the tracking station 130 tracks a single light emission. In other examples, the tracking wand 132 may include additional light rings 192 spaced apart along a length of the tracking wand 132. In these examples, the tracking station 130 can use the known distance between the light rings 192 (e.g., two or more light rings 192), along with the calculated distance from the image data, to determine an orientation of the tracking wand 132. In some examples, the tracking wand 132 can further include a sensor (not shown), such as an inertial measurement unit (IMU). The tracking station 130 can use inputs from the two light rings 192 and the IMU to extrapolate location and orientation information of the end point 210 when the user presses the button 188. The tracking wand 132 can further communicate the orientation to the tracking station 130, to be used with the image data for location determination by the tracking station 130. As a further example, the tracking wand 132 can include a 3D cluster of light sources 200 (e.g., four, five, or more light sources 200) at the end point 210, allowing the tracking station 130 to determine full location and orientation information of the end point 210using only the light sources 200.

[0046] The above examples generally describe an active stereovision system, where the tracking wand 132 comprises the light source 200 and the tracking station 130 uses the image data from the camera assembly 136 to track the light source 200 in the environment. In some examples, the tracking wand system 102 may instead utilize a passive stereovision system. For example, the light ring 192 may include a retroreflective coating or prism(s), and the light source(s) 200 may instead be located at the camera assembly 136. As a result, the tracking station 130 can use the image data from the camera assembly 136 to track the reflected light from the tracking wand 132 in the environment, i.e., from the light sources 200 at the camera assembly 136, reflected by the light ring 192.

[0047] In view of the above, a user may use the tracking wand system 102 to mark locations within an environment corresponding to a desired run of a workpiece. For example, the user can initially register the tracking wand system 102 to the environment, e.g.. via a calibration procedure, and then can “touch” or “mark” locations within the environment corresponding to desired locations that the workpiece should be routed or should avoid.

[0048] Accordingly, FIG. 10 illustrates an example method 220 for operating a tracking wand system 102, according to some examples. Generally, one or more steps of this method 220 may be incorporated into software or firmware algorithms embedded within the controller 140 of the tracking station 130, and / or the controller 110 of the computing device 106, and / or the controller 204 of the tracking wand 132. For example the method steps may be stored in the form of instructions in a program storage area of the memory 144 of the controller 140, to be executed by the processor 142. Generally, the method 220 can include checking whether a tracking wand 132 is connected to the tracking station 130 (step 222), performing a calibration procedure (step 224), tracking the tracking wand via dual-camera stereo vision (step 226), checking for a button press of the tracking wand (step 228), obtaining image data from the time of the button press (step 230), filtering the image data to detect a target light in the image data from the light ring 192 of the tracking wand 132 (step 232), triangulating a location of the end point 210 of the tracking wand 132 in 3D space (step 234), communicating the location to the visualization system (step 236), and determining whether the tracking wand 132 is still connected to the tracking station 130 (step 238).

[0049] More specifically, at step 222, the tracking station 130 can determine whether a tracking wand 132 is connected. For example, a tracking wand 132 may be paired with the tracking station 130 via Bluetooth pairing (e.g., between the respective communication systems146, 206), or another wireless communication method. In another example, a tracking wand 132 can be connected to the tracking station 130 via a physical wired connection. Additionally, in some examples, the tracking station 130 may be connected to multiple tracking wands 132 at the same time. In some examples, when a tracking wand 132 is connected to the tracking station 130, the tracking station 130 can obtain tracking wand information specific to the particular tracking wand 132, such as a distance from the light ring 192 to the end point 210, a color or pattern of the light emitted from the light ring 192, etc.

[0050] When a tracking wand 132 is connected to the tracking station 130 (i. e. , YES at step 222). the tracking station 130 can perform a calibration procedure at step 224 to register the tracking wand system 102 to the environment, which is illustrated and described in more detail with respect to FIG. 11. Following the calibration procedure at step 224, the tracking station 130 can track the tracking wand 132 via stereo vision at step 226. More specifically, according to some examples, the tracking station 130 can continuously obtain video data from the two cameras 168, 170, analyze the video data to locate the tracking wand 132 within the video data, e.g., using stereo vision triangulation, and can rotate the rotating platform 172 so that the cameras 168, 170 continuously maintain the tracking wand 132 within their collective field of view. Alternatively, in some examples, a user may manually rotate the rotating platform 172 so that the desired tracking location of the tracking wand 132 will be in the field of view of the cameras 168, 170.

[0051] While the tracking station 130 tracks the tracking wand 132, the tracking station 130 can wait for communication from the tracking wand 132 indicating that the user has pressed the button 188 on the tracking wand 132 at step 228. When the user presses the button 188 (i.e., YES at step 228), the tracking station 130 can obtain image data from the cameras 168, 170 at the time of the button press (step 230) and filter the image data to detect a target light in the image data from the light ring 192 (step 232). For example, such filtering can include subtracting a background from the image data and. from the remaining data, performing image masking to isolate the target light in the image, e.g., via a threshold pixel value or pixel value. This threshold pixel value or pixel value range may be associated with the target light color, such as blue light, green light, ultraviolet light, infrared light, etc. Additionally, in some examples, such filtering can be performed to look for a particular flash rate within the image data to identify the target light in the image. Accordingly, this threshold value or range may be associated with the specific tracking wand 132, where different tracking wands 132 may include different respective light rings 192.

[0052] Upon filtering the target light in the image at step 232, the tracking station 130 can triangulate a location of the end point 210 of the tracking wand 132 in 3D space at step 234. For example, the tracking station 130 can implement a stereo measuring algorithm, using inputs from the calibration procedure, and the known distance from the light ring(s) 192 to the end point 210 to determine the location of the end point 210 in 3D space. In some examples, the tracking station 130 may further include orientation data in this determination. At step 236, the tracking station 130 can communicate this location to the visualization system 124.

[0053] Referring back to step 224 of the method 220, an example calibration procedure 224 is illustrated in FIG. 11. The calibration procedure 224 can be completed so that the tracking station 130 can set a proper coordinate frame associated with the environment to accurately track tracking wand locations and to accurately generate a workpiece model within the environment, as described further below with reference to FIG. 12. For example, after a user has placed the tracking station 130 in an area and connected the tracking wand 132, the user can initiate the calibration procedure through the visualization system 124 (e.g., through the user interface 118) via the tracking station 130 or the connected computing device 106, or the tracking station 130 (or connected computing device 106) can automatically initiate the calibration procedure when a tracking wand 132 is connected. Accordingly, while steps of the calibration procedure 224 may refer to the tracking station 130 performing certain actions, in some examples, such actions may be performed through the visualization system 124 on the tracking station 130 and / or on the computing device 106, or may be performed through another program executed by the tracking station 130.

[0054] Referring still to FIG. 11, when the calibration procedure is initiated (step 240). the tracking station 130 can prompt the user to record three points in space on a vertical surface, such as a ceiling (step 242), and record the resulting data (step 244), then prompt the user to record three points in space on a horizontal surface, such as a wall (step 246), and record the resulting data (step 248). For example, the tracking station 130 can prompt the user to record these points in space by displaying instructions via the display 116. Furthermore, the tracking station 130 can record the resulting data when the user presses the button 188 on the tracking wand 132 at each point. It should be noted that, while the procedure 224 is shown in FIG. 11 in a particular order, in some examples, steps 242 and 246 may be performed in a different order than what is shown.

[0055] Once the tracking station 130 receives the data from steps 244 and 248, the tracking station 130 can set up a 3D coordinate system using the vertical surface data as a referencevertical plane and the horizontal surface data as a reference horizontal plane (step 250), thus registering the tracking wand system 102 to the surrounding environment. This 3D coordinate system can then be used by the tracking station 130 in the stereo measuring algorithm to determine a distance between the end point 210 of the tracking wand 132 and the tracking station 130 each time the user presses the button 188, as described above with respect to the method 220 of FIG. 10.

[0056] While one example calibration method 224 is described with reference to FIG. 11, where three points on two different planes, e.g., vertical and horizontal, are provided to the tracking station 130, other calibration methods may be utilized in some examples. For example, a user can mark a first "origin" point in the environment, and then another point on each of the X, Y, and Z axes within the environment. As another example, a user can mark two points per X, Y. and Z axes (e.g., using the edge of a wall).

[0057] While the above methods 220, 224 are associated with dual-camera stereo vision tracking, in some examples, the tracking wand system 102 can include other sensors and tracking technologies in order to determine a location of the tracking wand 132 in the environment. According to one example, the tracking wand system 102 can include Lidarbased sensors and an IMU in the tracking wand 132 as inputs to a simultaneous localization and mapping (SLAM) algorithm, which can position the tracking wand 132 with respect to its environment. According to yet another example, ultrasonic tracking may be used, e.g., where the tracking station 130 sends and receive ultrasonic signals to determine tracking wand 132 distance and / or position. However, in other examples, other sensors and tracking methods may be used.

[0058] As noted above, with respect to the method 220 of FIG. 10, when the tracking station 130 determines a point location after the user presses the button 188 on the tracking wand 132, the tracking station 130 communicates the location to the visualization system 124 (at step 236). In some examples, the visualization system 124, either as part of the tracking station 130 or a separate computing device 106, can use the location to generate a three- dimensional rendering of a workpiece within a simulated environment, automatically calculating bend angles and positions on the workpiece.

[0059] For example, FIG. 12 illustrates an example method 260 of workpiece rendering using the tracking wand system 102 and the visualization system 124. As noted above, generally, the steps of the method 260 may be performed through the visualization system 124, which may be part of the tracking station 130 or of a connected computing device 106. Asshown in FIG. 12, the method 260 can include obtaining location information following a button press (step 262), graphing point locations in a coordinate system (step 264), connecting sequential points to create a wireframe model (step 266), applying a fdtering algorithm to the wireframe model to obtain a workpiece model (step 268). applying a bend radius to each bend in the workpiece model (step 270), and outputting a completed workpiece model (step 272).

[0060] More specifically, at step 262, the visualization system 124 can obtain location information of the tracking w and 132 after a user presses the button 188. With reference to a point location 274 shown in FIG. 13. such location information can include, for example, a point number (e.g., a sequential number assigned to the point location 274). a straight line distance (d) between the tracking station 130 and the point location 274, a polar angle (0) relative to a vertical axis 276, and an azimuthal or rotation angle (<|>). A user can mark these reference point locations 274, e.g., to designate a workpiece starting point, a workpiece ending point, a bend, an obstruction, or other points of interest for routing a workpiece.

[0061] It should be noted that, while the steps 262-272 of the method 260 are illustrated sequentially, in some examples, the visualization system 124 may first obtain all point locations for a workpiece before proceeding to step 264. In other examples, the visualization system 124 may cycle through all steps 262-272 each time a user presses the button 188.

[0062] Turning now to step 264, each point location 274 can be graphed in a coordinate system 278, such as a spherical coordinate system, as shown in FIG. 14A, creating a point cloud. At step 266, sequential point locations 274 (e.g., based on their point number) can be connected in the coordinate system 278 to create a wireframe model 280, as shown in FIG. 14B. For example, the wireframe model 280 can be a wireframe version of a desired workpiece. At step 268, a filtering algorithm is applied to the wireframe model 280 to obtain a workpiece model 282, as shown in FIG. 14C. For example, the filtering algorithm can adjust angles between sequential point locations 274 to a nearest common conduit bend angle, e.g. to provide a “best fit” based on a type of workpiece to be rendered. As an example, an 88-degree angle between point locations 274 in a wireframe model 280 may be adjusted to a 90-degree bend in the workpiece model 282. As another example, a 4-degree angle between point locations 274 in a wireframe model 280 may be adjusted to a 0-degree angle in the workpiece model 282. At step 270. a bend radius can be applied at each bend angle in the workpiece model 282, e.g., based on the type of workpiece to be rendered.

[0063] At step 272, the visualization system 124 can output a completed workpiece model 282, for example, via the display 116. For example, FIG. 15 illustrates an example display 116showing an image of the workpiece model 282. In some examples, the user may use the user interface 118 to confirm the model 282 as correct or may go back to adjust the model 282, e.g., by providing additional point locations 274 or other user input to adjust the model 282. Additionally, in addition to showing the workpiece model 282, in some examples, the visualization system 124 can output workpiece segment lengths between bends, bend angles, etc., providing the user with instructions on where to cut or bend a workpiece to match the model 282. Alternatively, the visualization system 124 can use the workpiece model 282 and associated information (e.g., segment lengths, bend angles, etc.) as instructions to a tool, such as the tool 104 illustrated in FIG. 1, to automatically cut or bend a workpiece to match the workpiece model 282. Additionally, while the workpiece model 282 is illustrated in FIG. 15 as including multiple segments and bends, in some examples, a workpiece model 282 may simply include two point locations 274 with a straight line (or workpiece) connected between the two point locations 274, allowing the user to obtain a physical length or distance between two points 274 using the tracking wand system 102 and the visualization system 124. As such, the tracking wand system 102 can be used as a distance measuring tool.

[0064] In addition or as an alternative to the above method 260, a user may use the tracking wand system 102 as a measurement tool following prompts from the visualization system 124. For example, the visualization system 124 may include a pre-rendered workpiece model 282 and may prompt the user to “mark” locations in the environment corresponding to locations of the pre-rendered workpiece model 282 using the tracking wand 132. In this manner, the visualization system 124 can use the location information from the tracking wand 132 to calculate segment lengths, bend angles, etc. of the pre-rendered workpiece model 282.

[0065] As another example, a user can initially mark workpiece start and end locations within the environment, corresponding to endpoints of the workpiece. The tracking station 130 can then generate an initial workpiece model 282 that spans the endpoints, and can prompt the user to mark other constraints within the workpiece model 282 by marking additional points, e.g., to define obstacles the workpiece should avoid, define corresponding objects or locations that the workpiece should run parallel to, define locations where bends are desired, etc. With these additional points, the tracking station 130 can update the workpiece model 282 with additional bends or runs between the endpoints.

[0066] In some implementations, devices or systems disclosed herein can be utilized, manufactured, or installed using methods embodying aspects of the invention. Correspondingly, any description herein of particular features, capabilities, or intendedpurposes of a device or system is generally intended to include disclosure of a method of using such devices for the intended purposes, a method of otherwise implementing such capabilities, a method of manufacturing relevant components of such a device or system (or the device or system as a whole), and a method of installing disclosed (or otherwise known) components to support such purposes or capabilities. Similarly, unless otherwise indicated or limited, discussion herein of any method of manufacturing or using for a particular device or system, including installing the device or system, is intended to inherently include disclosure, as embodiments of the invention, of the utilized features and implemented capabilities of such device or system.

[0067] Also as used herein, unless otherwise limited or defined, “or” indicates a nonexclusive list of components or operations that can be present in any variety of combinations, rather than an exclusive list of components that can be present only as alternatives to each other. For example, a list of “A. B, or C” indicates options of: A; B; C; A and B; A and C; B and C; and A, B, and C. Correspondingly, the term “or” as used herein is intended to indicate exclusive alternatives only when preceded by terms of exclusivity, such as “either,” “one of,” “only one of,” or “exactly one of.” For example, a list of “one of A. B, or C” indicates options of: A, but not B and C: B, but not A and C; and C. but not A and B. A list preceded by “one or more” (and variations thereon) and including “or” to separate listed elements indicates options of one or more of any or all of the listed elements. For example, the phrases “one or more of A, B, or C” and “at least one of A, B, or C” indicate options of: one or more A; one or more B; one or more C; one or more A and one or more B; one or more B and one or more C; one or more A and one or more C: and one or more of A. one or more of B, and one or more of C. Similarly, a list preceded by “a plurality of’ (and variations thereon) and including “or” to separate listed elements indicates options of multiple instances of any or all of the listed elements. For example, the phrases “a plurality of A, B, or C” and “tw o or more of A, B, or C” indicate options of: A and B; B and C A and C: and A, B, and C.

[0068] In some examples, aspects of the disclosed technology, including computerized implementations of methods according to the disclosed technology, can be implemented as a system, method, apparatus, or article of manufacture using standard programming or engineering techniques to produce software, firmware, hardware, or any combination thereof to control a processor device (e.g.. a serial or parallel general purpose or specialized processor chip, a single- or multi-core chip, a microprocessor, a field programmable gate array, any variety of combinations of a control unit, arithmetic logic unit, and processor register, and soon), a computer (e.g., a processor device operatively coupled to a memory), or another electronically operated controller to implement aspects detailed herein. Accordingly, for example, aspects of the disclosed technology' can be implemented as a set of instructions, tangibly embodied on a non-transitory computer-readable media, such that a processor device can implement the instructions based upon reading the instructions from the computer-readable media. Some examples of the disclosed technology can include (or utilize) a computing device such as an automation device, a special purpose or general purpose computer including various computer hardware, software, firmware, and so on. consistent with the discussion below. As specific examples, a computing device can include a processor, a microcontroller, a field- programmable gate array, a programmable logic controller, logic gates etc., and other typical components that are known in the art for implementation of appropriate functionality (e.g., memory, communication systems, power sources, user interfaces and other inputs, etc.). In some examples, a computing device can include a centralized hub controller that receives, processes and (re)transmits control signals and other data to and from other distributed computing devices (e.g., an engine controller, an implement controller, a drive controller, etc.), including as part of a hub-and-spoke architecture or otherwise.

[0069] The term "‘article of manufacture” as used herein is intended to encompass a computer program accessible from any computer-readable device, carrier (e.g., non-transitory signals), or media (e.g., non-transitory media). For example, computer-readable media can include but are not limited to magnetic storage devices (e.g., hard disk, floppy disk, magnetic strips, and so on), optical disks (e.g., compact disk (CD), digital versatile disk (DVD), and so on), smart cards, and flash memory’ devices (e.g., card, stick, and so on). Additionally, it should be appreciated that a carrier wave can be employed to carry computer-readable electronic data such as those used in transmitting and receiving electronic mail or in accessing a netyvork such as the Internet or a local area network (LAN). Those skilled in the art will recognize that many modifications may be made to these configurations without departing from the scope or spirit of the claimed subject matter.

[0070] Certain operations of methods according to the disclosed technology, or of systems executing those methods, may be represented schematically in the figures, or otherwise discussed herein. Unless otherwise specified or limited, representation in the figures of particular operations in particular spatial order may not necessarily require those operations to be executed in a particular sequence corresponding to the particular spatial order. Correspondingly, certain operations represented in the FIGS., or otherwise disclosed herein,can be executed in different orders than are expressly illustrated or described, as appropriate for particular examples of the disclosed technology. Further, in some examples, certain operations can be executed in parallel, including by dedicated parallel processing devices, or separate computing devices configured to interoperate as part of a large system.

[0071] As used herein in the context of computer implementation, unless otherwise specified or limited, the terms “component,” “system,” “module,” “block,” “device,” and the like are intended to encompass part or all of computer-related systems that include hardware, software, a combination of hardware and software, or software in execution. For example, a component may be, but is not limited to being, a processor device, a process being executed (or executable) by a processor device, an object, an executable, a thread of execution, a computer program, or a computer. By way of illustration, both an application running on a computer and the computer can be a component. One or more components (or system, module, and so on) may reside within a process or thread of execution, may be localized on one computer, may be distributed between two or more computers or other processor devices, or may be included within another component (or system, module, and so on).

[0072] As used herein, unless otherwise defined or limited, directional terms are used for convenience of reference for discussion of particular figures or examples. For example, references to downward (or other) directions or top (or other) positions may be used to discuss aspects of a particular example or figure, but do not necessarily require similar orientation or geometry' in all installations or configurations.

[0073] Also as used herein, unless otherwise limited or defined, “substantially parallel” indicates a direction that is within ± 12 degrees of a reference direction (e.g., within ± 6 degrees), inclusive.

[0074] Also as used herein, unless otherwise limited or defined, “substantially perpendicular” indicates a direction that is within ± 12 degrees of perpendicular a reference direction (e.g., within ± 6 degrees), inclusive.

[0075] Also as used herein, unless otherwise limited or defined, “integral” and derivatives thereof (e.g., “integrally”) describe elements that are manufactured as a single piece without fasteners, adhesive, or the like to secure separate components together. For example, an element stamped, cast, or otherwise molded as a single-piece component from a single piece of sheet metal or using a single mold, without rivets, screws, or adhesive to hold separately formed pieces together is an integral (and integrally formed) element. In contrast, an element formed from multiple pieces that are separately formed initially then later connected together,is not an integral (or integrally formed) element.

[0076] Additionally, unless otherwise specified or limited, the terms “about’’ and “approximately,’’ as used herein with respect to a reference value, refer to variations from the reference value of ± 15% or less, inclusive of the endpoints of the range. Similarly, the term “substantially equal” (and the like) as used herein with respect to a reference value refers to variations from the reference value of less than ± 10%, inclusive. Where specified, “substantially” can indicate in particular a variation in one numerical direction relative to a reference value. For example, “substantially less” than a reference value (and the like) indicates a value that is reduced from the reference value by 10% or more, and “substantially more” than a reference value (and the like) indicates a value that is increased from the reference value by 10% or more.

[0077] Also as used herein, unless otherwise limited or specified, “substantially identical” refers to two or more components or systems that are manufactured or used according to the same process and specification, with variation between the components or systems that are within the limitations of acceptable tolerances for the relevant process and specification. For example, two components can be considered to be substantially identical if the components are manufactured according to the same standardized manufacturing steps, with the same materials, and within the same acceptable dimensional tolerances (e.g., as specified for a particular process or product).

[0078] Unless otherwise specifically indicated, ordinal numbers are used herein for convenience of reference, based generally on the order in which particular components are presented in the relevant part of the disclosure. In this regard, for example, designations such as “first,” “second,” etc., generally indicate only the order in which a thus-labeled component is introduced for discussion and generally do not indicate or require a particular spatial, functional, temporal, or structural primacy or order.

[0079] The above detailed description is to be read with reference to the figures, in which like elements in different figures have like reference numerals. The figures, which are not necessarily to scale, depict selected embodiments and are not intended to limit the scope of embodiments of the invention. Skilled artisans will recognize the examples provided herein have many useful alternatives and fall within the scope of embodiments of the invention.

[0080] It is to be understood that the invention is not limited in its application to the details of construction and the arrangement of components set forth in the above description or illustrated in the drawings. The invention is capable of other embodiments and of beingpracticed or of being carried out in various ways. Also, it is to be understood that the phraseology and terminology7used herein is for the purpose of description and should not be regarded as limiting. The use of “including,"’ “comprising,’" or “having"’ and variations thereof herein is meant to encompass the items listed thereafter and equivalents thereof as well as additional items. Unless specified or limited otherwise, the terms “mounted,” "‘connected,” “supported,” and “coupled” and variations thereof are used broadly and encompass both direct and indirect mountings, connections, supports, and couplings. Further, “connected” and “coupled” are not restricted to physical or mechanical connections or couplings.

[0081] The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the invention. Given the benefit of this disclosure, various modifications to these embodiments will be readily apparent to those skilled in the art, and the principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

CLAIMS1. A tracking wand system comprising: a tracking station; a camera assembly comprising a first camera and a second camera supported by the tracking station; a tracking wand; and a controller in communication with the camera assembly, the controller to receive outputs from the first camera and the second camera to: track a first location of the tracking wand relative to the tracking station, track a second location of the tracking wand relative to the tracking station, map the first location and the second location in a three-dimensional coordinate system, digitally render a workpiece model that spans the first location to the second location, and output the workpiece model to a user.

2. The tracking wand system of claim 1, wherein the tracking wand includes a light source, and the controller is to track the first location and the second location using a stereo measuring algorithm with the outputs from the first camera and the second camera.

3. The tracking wand system of claim 2, wherein the controller is to track the first location and the second location using the stereo measuring algorithm with the outputs from the first camera and the second camera by filtering image data from the first camera and the second camerato detect an output of the light source of the tracking wand within the image data.

4. The tracking wand system of claim 3, wherein the controller is further to detect a first light source location of the output of the light source within the image data and extrapolate the first light source location to the first location based on a predetermined distance between the light source and a physical endpoint of the tracking wand.

5. The tracking wand system of claim 2, wherein the light source of the tracking wand comprises a plurality of light sources; and the tracking wand further comprises a first light ring that outputs light from a first light source of the plurality of light sources, and a second light ring that outputs light from a second light source of the plurality of light sources, thefirst light ring and the second light ring being located a fixed distance apart along the tracking wand.

6. The tracking wand system of claim 1, wherein the camera assembly comprises a rotating platform that is rotatable about a base of the tracking station, the first camera and the second camera being supported by and rotatable with the rotating platform.

7. The tracking wand system of claim 1, further comprising a display, wherein the controller is to output the workpiece model to the user by displaying an image of the workpiece model via the display.

8. The tracking wand system of claim 7, wherein the tracking station comprises a display mount to removably mount a computing device comprising the display.

9. The tracking wand system of claim 1, wherein the controller is further to output the workpiece model to an automated pipe bending tool with instructions to bend a physical workpiece to match the w orkpiece model.

10. The tracking wand system of claim 1, wherein the tracking wand comprises a communication system to wirelessly communicate with a communication system of the tracking station.

11. The tracking wand system of claim 1, wherein the tracking wand comprises a button, and wherein the controller is to track the first location of the tracking wand relative to the tracking station when the button is pressed a first time, and the controller is to track the second location of the tracking wand relative to the tracking station when the button is pressed a second time.

12. A method comprising: tracking a tracking wand in an environment with a dual-camera assembly of a tracking station; receiving a first input from the tracking wand to mark a first reference point in the environment; determining a first location of the first reference point using information from the dual-camera assembly: receiving a second input from the tracking wand to mark a second reference point in the environment;determining a second location of the second reference point using information from the dual-camera assembly; mapping the first location and the second location in a three-dimensional coordinate system; connecting the first location to the second location in the three-dimensional coordinate system to create a workpiece model; and outputting a digital rendering of the workpiece model via a display.

13. The method of claim 12, further comprising determining a physical distance between the first location and the second location and outputting the distance via the display.

14. The method of claim 12, further comprising performing a calibration procedure to generate the three-dimensional coordinate system, including receiving vertical surface inputs corresponding to a physical vertical surface of the environment and receiving horizontal surface inputs corresponding to a physical horizontal surface of the environment.

15. The method of claim 12, further comprising displaying a prompt to a user to provide the first input to mark the first reference point.

16. The method of claim 12, wherein the first input is a button press on the tracking wand.

17. The method of claim 12, further comprising: receiving a third input from the tracking wand to mark a third reference point in the environment; determining a third location of the third reference point using information from the dual-camera assembly: mapping the third location in the three-dimensional coordinate system; connecting the first location, the second location, and the third location sequentially in the three-dimensional coordinate system to create a wireframe model; and creating the workpiece model by applying a filtering algorithm to adjust bends in the wireframe model.

18. The method of claim 12. further comprising receiving user inputs to adjust the workpiece model after outputting the digital rendering of the workpiece model.

19. A method comprising: tracking a tracking wand in an environment with a dual-camera assembly of a tracking station;determining a physical location of the tracking wand in the environment using information from the dual -camera assembly; receiving inputs from the tracking wand to mark a plurality of reference point locations in the environment; generating a workpiece model based on the plurality of reference point locations, the workpiece model being a digital rendering of a workpiece routed in the environment according to the reference point locations; and outputting an image of the workpiece model via a display.

20. The method of claim 19, wherein the workpiece is one of tubing and conduit.

Images