Object fabrication and tracking with embedded fluorescent markers

Abstract

An object includes an object body formed from a build material and a marker embedded in a part of the object body. The marker is formed from a fluorescent marker material configured to emit radiation with a first wavelength when exposed to radiation with a second wavelength. The build material in the part of the object body is translucent to radiation with the first wavelength and the second wavelength.

Description

OBJECT FABRICATION AND TRACKING WITH EMBEDDED FLUORESCENT MARKERSCROSS-REFERENCES TO RELATED APPLICATIONS

[0001] This application claims the benefit of U.S. Provisional Application No. 63 / 530,295 filed August 2, 2023, the entire contents of which are incorporated herein by reference.BACKGROUND OF THE INVENTION

[0002] This invention relates to the fabrication and tracking of objects with embedded fluorescent markers.

[0003] Tags and markers play an important role in interactions with objects in human-computer interaction (HCI). The ability to identify and track objects in real time enables a wide range of applications, such as motion capture for animation and gaming, augmented and virtual reality (AR / VR), and object tracking for industrial and robotic applications.

[0004] Some conventional approaches to object tracking require the attachment of bulky and obtrusive external markers (e.g., reflective beads, infrared sensing modules, or fiducials) to the objects. In recent years, invisible object tagging has become increasingly important. The ability to track objects without the markers having a bulky form factor or being in the user’s direct view allows for a more natural and immersive experience for users and opens up new possibilities for tracking objects in real-life scenarios where visible markers would be impractical and obtrusive.

[0005] There is a need for object tracking markers that arc not the user’s direct view and arc suitable for real-world scenaios where users desire more natural and immersive experiences.SUMMARY OF THE INVENTION

[0006] Existing methods for invisible object tagging have several limitations that impede their widespread adoption. One significant limitation is their low signal-to-noise (SNR) ratio due to the poor resolution and clarity of the imaged marker. Because existinginvisible markers are embedded in the interior of objects, the markers are imaged from a weak signal, which needs to be amplified by optical and digital processes. These processes may result in long capture or decoding times, typically ranging from seconds to minutes per frame. This deteriorates further when objects are in motion, where the captured markers appeal- blurrier and are thus unidentified in most frames. In addition, existing invisible tags are often limited in terms of the variety of object colors they can be used with. For example, some existing techniques can only work if 3D-printed codes are embedded in black or white objects.

[0007] Aspects described herein address these limitations by fabricating objects with embedded passive invisible tags using fluorescent filaments that emit light in a specific near-infrared (NIR) wavelength, which NIR cameras can robustly detect. By isolating the markers from the rest of the scene using a matching filter, aspects are able to robustly track markers even when objects are in motion.

[0008] Some aspects described herein relate to object tracking markers (e.g., QR codes) that are embedded in an object and are formed from a fluorescent material. The markers are not visible to the naked eye due to their being embedded in the object. But when an object with an embedded fluorescent marker is exposed to an infrared light source with a first wavelength, the fluorescent material absorbs the light and emits light at a known different (e.g., longer) wavelength from the object that is detectable by a sensor. Because the emitted wavelength is known, optical filtering techniques can be used to isolate light emitted by the marker, increasing the SNR of the sensed light. The resulting high-contrast image of the marker reduces the amount of processing and time required to identify the marker, resulting in a marker that is usable in real-world applications where the object is in motion. The markers can be robustly tracked, even when embedded in objects in different surface colors are in motion.

[0009] In a general aspect, an object includes an object body formed from a build material and a marker embedded in a part of the object body. The marker is formed from a fluorescent marker material configured to emit radiation with a first wavelength when exposed to radiation with a second wavelength. The build material in the part of the object body is translucent to radiation with the first wavelength and the second wavelength.

[0010] Aspects may include one or more of the following features.

[0011] The build material in the part of the object body may be configured with a thickness that allows the radiation with the first wavelength and the second wavelength to pass. The object body and the marker may be formed using an additive manufacturing process. The marker may include an information-bearing code. The marker may include a two-dimensional code. The marker may be invisible to the naked eye. The first wavelength and the second wavelength may be in the near- infrared spectrum. The marker may include a plurality of different fluorescent marker materials.

[0012] In another general aspect, a method for detecting the marker in the object described above includes exposing the object to the first radiation including radiation with the second wavelength and capturing an image of the object when exposed to the first radiation including optically filtering radiation emitted from the object to isolate radiation emitted from the marker with the first wavelength.

[0013] In another general aspect, a method for fabricating the object described above includes depositing the build material to form the object body and depositing the fluorescent marker material to form the marker, wherein the deposited fluorescent marker material is embedded in the deposited build material.

[0014] Aspects may include one or more of the following features. At least one of the build material and the fluorescent marker material may be deposited using an additive fabrication process. The additive fabrication process may include a fused deposition modeling additive fabrication process.

[0015] Aspects may have one or more of the following advantages.

[0016] Among other advantages, aspects described herein relate to a fabrication method that uses fluorescent filaments to embed easily trackable markers in 3D-printed color objects. By using an infrared-fluorescent filament that "shifts" the wavelength of the incident light, an optical detection setup filters out all the noise to only have the markers present in the infrared camera image. The high contrast of the markers allows robust tracking of the markers regardless of objects’ surface color.

[0017] Aspects can advantageously be used in a variety of applications, such as custom-fabricated wearables for motion capture, tangible interfaces for AR / VR, rapid product tracking, and privacy-preserving night vision. Aspects advantageously exceed the detection rate of state-of-the-art invisible marking, and even small markers (l"xl") can be tracked at distances exceeding 2m.

[0018] Aspects include a software interface for automatically embedding the markers for the input object geometry and hardware modules that can be attached to existing mobile devices and AR / VR headsets. Aspects include an image processing pipeline that robustly localizes the markers in real time from the captured images.BRIEF DESCRIPTION OF THE DRAWINGS

[0019] FIG. 1 is a visible spectrum image of objects including fluorescent markers.

[0020] FIG. 2 is a near-infrared spectrum image of objects including fluorescent markers.

[0021] FIG. 3 is a near-infrared spectrum marker detection system.

[0022] FIG. 4a is an excitation and absorption spectrum of the marker material.

[0023] FIG. 4b is a light source and filtered emission spectrum.

[0024] FIG. 5 is an image processing pipeline.DETAILED DESCRIPTION1 OVERVIEW

[0025] Referring to FIGs. 1 and 2, a number of objects 100 (e.g., bottles) are formed using multi-material 3D-printing to include markers 102 that are invisible to the naked eye but are detectable in the near-infrared (NIR) spectrum. For example, in FIG. 1, the objects 100 are imaged by a visible light camera 104 as they are illuminated by a visible light source 106. The resulting image 108 shows four bottles with different shapes and colors but no markers are visible. In FIG. 2, the objects are recorded by an NIR camera 110 (through an optical filter 111, such as a longpass filter) as they are illuminated by a NIR light source 112. The image captured by the NIR camera 110 is processed in an image processing pipeline 113 to generate a detection image 114 that includes four markers 102 (one for each object) that are detected in the NIR spectrum. As is described in greater detail below, the markers 102 arc fabricated using a material that fluoresces in the NIR spectrum, resulting in the NIR camera 110 being able to obtain high-contrast images of the markers at a high frame-rate.2 MARKER DETECTION

[0026] Referring to FIG. 3, a marker 302 is shown embedded into a 3D printed object 300 (e.g., one of the bottles shown in FIG. 1). As was the case in FIG. 2, the object 300 and marker 302 are illuminated by the NIR light source 112 (e.g., a 760nm light source), which causes the marker 302 to fluoresce. The fluorescence of the marker 302 is recorded by the NIR camera 110 after it passes through the optical filter 111 (e.g., an 830nm longpass filter).2.1 Object and Marker Confi guration

[0027] The marker 302 is, for example, a 2-D code fabricated using a marker material that includes a substantially uniformly distributed dye that fluoresces in the NIR spectrum, as is described in greater detail below. In general, the marker 302 has a thickness, tmarker. The marker 302 is embedded in an object formed from an object material such as a polylactic acid (PLA) or acrylonitrile butadiene styreneacrylonitrile butadiene styrene (ABS). A thin layer (or “shell”) 316 of the object material overlays the marker 302 to obscure the marker from sight. The shell 316 has a thickness of / shell.

[0028] In general, the shell thickness, rSheii and the marker thickness, tsheii arc chosen to ensure that the NIR fluorescence emitted by the marker 302 is sufficiently intense to pass through the shell 316 and be captured by the NIR camera 110. For example, if the shell 316 is too thick and / or if the marker 302 is too thin, the intensity of the NIR light emitted by the marker 302 will not be sufficient for detection by the NIR camera 110. On the other hand, if the shell 316 is too thin, the marker 302 may be visible to the naked eye through the shell 316.

[0029] In some examples, the marker thickness, / marker is chosen to be the thickness of one layer of filament (e.g., / marker = 0.15mm). The thickness of the shell differs based on the object material used to fabricate the object. In particular, due to the high intensity of the fluoresced light from the marker, the emitted light can penetrate the shell above the fluorescent marker 302 and reach the camera’ s image sensor. Different colors of object material may be used, and each different color may affect the fluorescence from the marker 302 differently. As such the shell thickness, / shell is typically less than or equal to 3mm, depending on the color of the object material. If the shell 316 is too thick, the fluorescence is blocked but if the shell is too thin, the marker may be visible and a poor surface finish for the object may occur. One specific material choice is to use IR-PLA(which passes IR light) to produce black objects because carbon black in conventional filaments absorbs most wavelengths.2.1.1 Marker Material

[0030] As is noted above, the markers arc fabricated using a material that includes a dye that fluoresces in the NIR spectrum. In general, fluorescence is a physicai phenomenon where a material absorbs light at one wavelength and emits it at a longer wavelength. This occurs when a molecule absorbs the energy of light and temporarily enters an excited state, before releasing the energy as a lower-energy photon. The difference between these two wavelengths is called Stokes shift. By shifting the illumination from shorter wavelengths to longer ones, the fluorescent color can appear more saturated than it would by reflection alone, which enhances its detectability. In the example of FIG. 2, the NIR light source 112 emits light with a wavelength of 760nm and the fluorescent material emits light with a wavelength of 830nm.

[0031] In some examples, the marker material is a fluorescent filament such as an ABS (acrylonitrile butadiene styrene) filament that contains a NIR-fluorescent dye, which reacts to the NIR light source by fluorescing. Referring to FIG. 4a, the excitation spectrum (absorption) 418 and emission spectrum (unfiltered fluorescence) 420 of the marker material are shown. As can be seen in the unfiltered spectra of FIG. 4a, the emitted fluorescence has a longer wavelength than the absorbed light. While the material is most excited at wavelength 763nm, the peak of the emitted light is at 775nm. This Stokes shift of 12nm between the two spectra allows separation of the excitation and emission signals for the IR image capture. However, due to the spectral overlap highlighted in the graph, the excitation and emission signals are first filtered using optical an optical filter (e.g., optical filter 111 of FIG. 1).

[0032] When coupled with wavelength- specific optical components, the light emitted by the fluorescent filament results in high contrast between the marker and the rest of the IR image. This allows us to robustly detect and track the objects in which the markers are embedded.2.2 NIR Light Source

[0033] In some examples, the NIR light source 112 is chosen according to two considerations. First, the light emitted by the NIR light source 112 should be invisible to the user (i.e., there should be no emission below 700nm). Second, the light should excitethe fluorescent material as much as possible. To achieve this, the light source should have high power, and its peak wavelength should be as close to the material’s peak excitation wavelength (e.g., 763nm for certain materials described herein) as possible.

[0034] Referring to FIG. 4b, in some examples, to satisfy these considerations, the NIR light source 112 includes high-power LEDs with a peak intensity 421 at 760nm and has an intensity of zero at approximately 800nm.2.3 Optical Filter

[0035] Referring to FIGs. 4a and 4b, in some examples, the optical filter is a wavelength-specific filter that enhances recognition of fluorescent markers by minimizing interference from other wavelengths. As is shown in FIG. 4a, due to the natural characteristics of fluorescence, there is an overlap between the higher wavelength end of the excitation spectrum of the marker and the lower wavelength end of the emission spectrum of the marker. This overlap is mitigated or eliminated by the optical filter 111 to avoid overwhelming the weaker emitted fluorescence light with the brighter excitation light, which could significantly reduce marker contrast.

[0036] In some examples, the optical filter 111 is a longpass filter with a cut-on wavelength of 830nm, which blocks any wavelength under this value from entering the NIR camera 110. For example, the optical filter 111 blocks the excitation light emitted by the NIR light source 112 as well as visible environmental light from reaching the NIR camera 110. As a result, the only wavelength range that reaches the camera corresponds to the fluorescence from the marker itself (shown as curve 423 in FIG. 4b). As a result, there is no longer an overlap between the excitation and emission spectra because the NIR light source 112 cuts the emission at ~810nm, while the filtered fluorescence that enters the camera starts at ~820nm. This separation allows the high intensity of the markers in images captured by the camera 110.2.4 NIR camera

[0037] In some examples, the NIR camera 110 is a high-speed camera configured to observe the fluorescence emitted from the moving objects. In some examples, the NIR camera 110 is configured to capture images at a high frame rate and minimize motion blur by using a framerate such as 60-frames-per-second (fps) or higher. In one example, the camera 110 is a monochrome camera with a global shutter rather than a rolling shutter. Rolling shutters, which are commonly used in consumer-grade cameras, scan theimage sensor line by line from top to bottom, resulting in a time delay between the capture of each row. This can lead to motion blur in fast-moving scenes, which makes it difficult to detect the marker 302 accurately.

[0038] A global shutter captures the entire image simultaneously, resulting in no time delay between rows. This is important for real time tracking where every frame counts. Additionally, a monochrome camera achieves higher spatial resolution than color cameras, and eliminates the need for a color filter array, which reduces the amount of light reaching the image sensor and decreases the camera’ s sensitivity to NIR fluorescence. Together, these properties result in a camera that can record a real time video representation of the object with high temporal resolution.2.5 Image Processing

[0039] Referring to FIG. 5, an image processing pipeline 113 is configured to detect the markers in real time. In general, the software for detection and tracking of markers is as lightweight as possible for fast tracking. Only a small number of processing steps are applied to the main image frame when localizing the markers. For example, image data 532 is acquired by the NIR camera (e.g., a grayscale NIR spectrum image of a person wealing objects with embedded markers as shown in the visible light spectrum image 534). The grayscale image data 532 is processed in a binarization and polygon approximation module 536, which binarizes the image (e.g., using Otsu’s thresholding method). The binarization results in multiple identified contours. The binarization and polygon approximation module 536 then approximates each contour as a polygon. In image 538 output from the binarization and polygon approximation module 536, polygons with four sides are identified as the target markers. This approach keeps track of all markers irrespective of the data encoded in them (i.e., ID / string). This keeps the detection lightweight if the encoded data is not relevant to the application (e.g., when all markers are known to have the same ID).

[0040] The output of the binarization and polygon approximation module 536 is provided to a cropping and resizing module 540, which uses each marker’s bounding box to crop out a smaller patch from the original frame, as is shown in image 542. In some examples, the cropping and resizing module 540 adds a small padding around the marker, (e.g., 1 / 8 of the detected bounding box length). The patch is then resized to a certain size (e.g., 50px height). The image is then provided to an inversion module 544, which forms an inverted image 546 such that the markers are dark to follow the ArUco convention.The inverted image 546 is provided to a Gaussian blur module 548, which generates a blurred image 550 to remove noise from the image. The blurred image 550 is processed in a thresholding module 552, which applies an adaptive threshold with a block size of, for example, 15 to generate a thresholded image 554. The thresholded image 554 is provided to a decoding module 556 (e.g., the ArUco detection library), which decodes the marker ID from the thresholded image 554. The patch resizing in this part ensures that the applied block size is appropriate regardless of marker location and distance.

[0041] If no ArUco marker is found in the patch, the image processing pipeline 113 checks if there is a QR code or a Data Matrix. For this, substantially the same steps are followed with the exception that, because the codes have more bits than ArUcos, a rescaling height of lOOpx is used and an adaptive threshold block size of 9 is used. A standard library is then used to decode the codes. If an ArUco or a QR code is found in the patch, a detection result 560 is output, including the data and corners of the detected code in an array.

[0042] In some examples, to track decoded markers across frames a "caching" feature is used in the image processing pipeline 113. The aching feature helps track the marker data across frames, without having to run the decoding script in every new frame. When caching is enabled, the newly localized markers are matched to already decoded markers from the prior frame by comparing their comer coordinates. If a new marker is found to be similar to a previous marker, the marker data is retrieved from the previous frame. This is especially useful for QR codes, which traditionally take several frames to be decoded, whereas the fluorescent outline can already be tracked in all frames.

[0043] In one implementation, OpenCV is used to implement the image processing pipeline 113 and the Dynamsoft4 library is used for the detection of the 2D barcodes (i.e. , QR, Micro QR, and Data Matrix). For AR / VR applications, after running OpenCV’ s ArUco pose estimation, Unity’s coordinate transform features to convert the marker’s location to a common local coordinate frame.3 MARKER DESIGN AND FABRICATION

[0044] In general, CAD software with a special purpose marker embedding plugin is used to design objects with embedded markers. The designed objects are used in a fabrication process such as a 3D printing process to create a physical instance of the designed object.3.1 Marker Design

[0045] In some examples, to embed markers into an object, a user imports a 3D model into a CAD editor (e.g., Blender). Next, the user specifies how the fluorescent marker(s) should be implanted using a plugin designed to embed markers into objects. In some examples, the determining the marker location is important. For example, in tracking applications, it is important to pad the object surface with numerous markers, rather than a single one, to ensure the object can be tracked independently of the object orientation. The user can either manually pick exact target points on the object for the marker embedding, or use an automatic distribution mode. This mode uniformly distributes points on the object geometry based on a fixed inter-marker distance or a total number of desired markers. The user can specify whether the target points should be distributed on the whole object, the areas other than the base, or a specific area selected by the user.

[0046] In some examples, the plugin allows users to embed ArUco markers with the same ID ("Use fixed ArUco ID") or increasing IDs ("Use sequential ArUco IDs") based on the desired use case. Unique sequential IDs can help identify an object’s location and orientation. It is also possible to load a custom marker, such as a QR code or a Micro QR code, by selecting an image file.

[0047] In one example, the automatic uniform distribution of the markers is created by first determining the areas on the model that are large enough and close to being flat to accommodate a marker of the user’s specified size. This is done by comparing the angles between the normals of adjacent faces with the method faces_select_ linked_flat(), grouping the ones with an angle difference smaller than 0.1 rad, and ensuring that the contained surface area is greater than that of the marker. This threshold can be adjusted based on the user’ s preference and application if desired. This gives the subregions on the object geometry that are not too curvy or sharp. Next, to uniformly sample points, each subregion is temporarily duplicated, flattened, and its shape is approximated to a high degree of accuracy in an array of Is and 0s created with repeated calls of the function ray_cast(). The array is analyzed row by row, choosing rows that are a distance of half the marker side length apart. Points in each row are picked if they are sufficiently far from each other and the edges of the subregion. Finally, using the specifications set by the user, a knife projection is used to project the marker so it has the same curvature as the object at the sampled point. The projected marker is extruded to form a 3D model, which is exported separately.

[0048] In some examples, the user then specifies how deep the marker should be embedded under the object’s surface. For this, the user can enter a custom value or simply select the desired object color from a dropdown, after which the plugin automatically determines the appropriate Lheii. The user can then specify the dimensions of the marker. The marker thickness is set to the recommended / marker value by default, and the user can specify a desired side length of the marker. Finally, the user can specify whether the marker’s bottom edge should be parallel to a certain plane (e.g., the surface of the object). This helps users to align codes in the 3D printing direction, which can lower print time and avoid print failures that might occur due to frequent nozzle switching. Once the markers are embedded, users can export the resulting STL files for 3D printing.3.2 Marker Fabrication

[0049] To fabricate the object with the embedded marker, the exported STL files are imported into a slicer (e.g., the Cura slicer) that is compatible with a 3D printer’s software. In some examples, the “combing parameter” of the slicing software is disabled to ensure that the printer does not move over the already extruded pieces of fluorescent filament. This prevents the fluorescent filament from being mixed into the outer surface layers and thus becoming visible.4 APPLICATIONS

[0050] The marker fabrication and detection techniques described above can be used in various applications, including product tracking on conveyor belts, flexible wearables for motion capture, tangible and haptic interfaces for AR / VR, and privacy-preserving night vision.4.1 Motion Capture Systems

[0051] Motion capture, abbreviated as "mocap," describes the technologies used to capture, track, and digitize physical objects and humans in real time. Existing motion capture systems are typically based on external markers where a tracking marker is separately attached to the objects that are intended to be tracked. Such markers may be based on inertial or optical methods. Inertial systems use gyroscopes and accelerometers to measure the orientation and acceleration of objects. Optical methods use cameras totrack markers mounted on objects and software to translate the marker positions into motion data.

[0052] Widely used optical systems, such as OptiTrack and Vi-con, offer active and passive optical markers that can be tracked by setting up multiple cameras in the environment. Active markers commonly include LEDs, which are picked up by the cameras, or sensors that detect the IR radiation emitted by nearby light sources. Such markers often use an embedded power source and can be large compared to the object, (e.g., Vive Tracker [5] is >7cm in two dimensions). Passive markers, on the other hand, include beads coated with a retro-reflective material that reflects the light emitted by the detection hardware. Whether active or passive, most optical methods require multiple cameras to be set up and calibrated to accurately track objects, which can be both timeconsuming and costly (e.g., a 6-camera OptiTrack system costs >$9k). Furthermore, given that these markers are external, they need to be attached to objects, which limits their use and practicality in real-world use.

[0053] The techniques described herein address the limitations of conventional systems limitations by using embeddable fluorescent materials to create passive tracking markers that can be integrated into real-world objects and tracked using just one camera. This allows for more affordable, versatile, and unobtrusive tracking of objects in real- world applications without the need for manual and visible attachment of markers.4.2 Rapid Product Tracking

[0054] The techniques described herein can localize and track embedded objects and therefore can be used for industrial or commercial applications in which items need to be processed swiftly. This can be especially useful for product and packaging logging where, although the external labels may be intentionally or unintentionally removed, it is still crucial to keep track of the item origin and other supply chain or inventory-related data.

[0055] For example, in manufacturing and packaging industries, fluorescent markers could be integrated to enable product or part tracing as part of assembly lines. In other examples, on-the-fly inventory logging can be accomplished using a smartphone implementation of the tracking system described herein (e.g., a batch of products can be scanned by moving the phone to quickly capture all codes).4.3 Wearables for Tracking Human Motion

[0056] The techniques described herein can be used to fabricate custom wearables for tracking human motion. For example, both rigid and flexible wristbands can be 3D- printed with embedded markers. Embedding unique markers allows digitization of a user’s motion and enables the right and left hands to be distinguished (e.g., by having different markers assigned to each hand). Such tracking wearables can allow various use cases unobtrusively, such as creating digital twins and animations, increasing safety in human-machine collaborations, posture correction warnings, or device control.

[0057] Compared to existing tracking methods, techniques described herein preserve the user’s privacy since the camera only captures the marker, not the user’s face or environment. Furthermore, current methods, especially those based on machine learning, are usually tuned for able-bodied people’s hands. Wearables designed with the techniques described herein could support applications for people with limb differences or hand impairments. In one examples, a black wristband is fabricated using PLA and a red wristband is fabricated using using ABS. Both wristbands have a small thickness and are bendable. In the future, more custom wearables could be printed using more flexible materials such as TPU or by utilizing FDM-based textile fabrication methods.4.4 Tangible Interfaces for Mixed Reality Experiences

[0058] The techniques described herein can be used to embed markers into objects to turn them into tangibles with more precise tracking capabilities for mixed reality (MR). For example, opportunistic tangible interfaces could enhance augmented reality (AR) interactions or serve as haptic placeholders in virtual reality (VR).

[0059] For example, physical parts can be appropriated as precise AR input tools. A loudspeaker that has been unused in an office could be transformed into a passive tangible interface for controlling the volume and bass of a set of AR glasses by embedding markers into the dials of the loudspeaker. This allows a user to have a more natural and tangible input method while touching small buttons on the glasses can be cumbersome. Furthermore, the marker objects described herein can also include a part’s identifier. For example, the top and bottom knobs can be distinguished and assigned unique functionalities

[0060] In other examples, physical objects can be used as VR haptic props. The techniques described herein allows users to make use of the physical shape of existingreal-world objects (e.g., toys, gadgets, sports gear) as haptic proxies in VR. For example, a "lightsaber" toy could come with integrated markers, so it can be used as a different object in games. Another benefit is that the objects are fully passive, while typical VR game controllers contain infrared LEDs that need to be powered up.4.5 Privacy-Preserving Night Vision

[0061] In other examples, the techniques described herein can be used in a privacypreserving night vision system. Many traditional security cameras use infrared LEDs to monitor environments at night. However, these cameras might not be optimal for use in private environments, such as one’s bedroom, although users may still want an alternative method to ensure the security of their valuable belongings. The marker detection scheme used herein removes all details in the camera stream except for the marked objects. For example, valuables or containers with valuables inside them could be tagged with a marker and tracked by the detection system described above.5 ALTERNATIVES5.1 Filament Materials

[0062] In general, the fluorescent marker can be combined with both PLA and ABS materials. This makes it possible to have a variety of colors for the fabricated objects. In some examples, where the fluorescent filament is ABS, the main object geometry is also fabricated using ABS, as it ensures similar printing parameters (e.g., temperature) among the two parts. The only ABS color that does not pass the fluoresced light is black. This is because the carbon black used in the conventional filament absorbs most wavelengths. Thus, for producing black objects, an IR-PLA filament that passes IR light is used.Colors closer to infrared such as red, yellow, and orange, tend to be less opaque than other colors due to their dye. The specific filament used may also affect opacity as well as the filament vendor. V ariations might exist even among different batches from the same vendor. Users are recommended to determine the optimal values for their filaments by first printing test slabs.5.2 NIR Power

[0063] While the LED component used in AR / VR applications is rated 10W, it need not be used at full power. At 1.8W, a black IR PLA-covered sample can be tracked from up to 50cm away. At 3.5W, it can be tracked at 90cm away.5.3 Embedding Circular Markers

[0064] An alternative way to add tracking markers would be to use the fluorescent filament to insert small circular markers, similar to retro-reflective beads, underneath the object surface. Then robust triangulation-based tracking systems could be used. In the future, this could be used to improve detection for more intricate objects such as those with high-frequency surface details that would lead to local occlusion of the markers based on the viewing angle.5.4 Occlusion

[0065] Object occlusion is a typical problem for most tracking methods. The markers described herein could be occluded by a user’s hands and other objects. One way to overcome this is to couple the optical tracking with magnetic tracking methods (i.e., hybrid tracking). A magnetic material could be embedded in the printed object using magnetic filaments, which would enhance the tracking when the object is occluded by utilizing magnetic sensors. Another possible solution is to use CNN-based machine learning methods to recover or estimate occluded marker comers.5.5 Mass Production

[0066] The techniques and markers described herein may be adapted for mass-market manufacturing. For instance, instead of 3D printing, plastic over-molding or 2K-Shot Molding (Two-Shot Molding) could be utilized to include fluorescent materials in mass- manufactured products without any post-processing steps.

[0067] It is noted that the approaches described herein are not limited to any one type of marker (e.g., ArUco markers) and are applicable to other types of markers including QR codes, data matrices, barcodes, fiducial markers such as AprilTags and other suitable types of markers.6 IMPLEMENTATIONS

[0068] The approaches described above can be implemented, for example, using a programmable computing system executing suitable software instructions or it can be implemented in suitable hardware such as a field-programmable gate array (FPGA) or in some hybrid form. For example, in a programmed approach the software may include procedures in one or more computer programs that execute on one or more programmed or programmable computing system (which may be of various architectures such as distributed, client / server, or grid) each including at least one processor, at least one data storage system (including volatile and / or non-volatile memory and / or storage elements), at least one user interface (for receiving input using at least one input device or port, and for providing output using at least one output device or port). The software may include one or more modules of a larger program, for example, that provides services related to the design, configuration, and execution of a program. The modules of the program can be implemented as data structures or other organized data conforming to a data model stored in a data repository.

[0069] The software may be stored in non-transitory form, such as being embodied in a volatile or non-volatile storage medium, or any other non-transitory medium, using a physical property of the medium (e.g., surface pits and lands, magnetic domains, or electrical charge) for a period of time (e.g., the time between refresh periods of a dynamic memory device such as a dynamic RAM). In preparation for loading the instructions, the software may be provided on a tangible, non-transitory medium, such as a CD-ROM or other computer-readable medium (e.g., readable by a general or special purpose computing system or device), or may be delivered (e.g., encoded in a propagated signal) over a communication medium of a network to a tangible, non-transitory medium of a computing system where it is executed. Some or all of the processing may be performed on a special purpose computer, or using special-purpose hardware, such as coprocessors or field-programmable gate arrays (FPGAs) or dedicated, application- specific integrated circuits (ASICs). The processing may be implemented in a distributed manner in which different pails of the computation specified by the software are performed by different computing elements. Each such computer program is preferably stored on or downloaded to a computer-readable storage medium (e.g., solid state memory or media, or magnetic or optical media) of a storage device accessible by a general or special purpose programmable computer, for configuring and operating the computer when the storage device medium is read by the computer to perform the processing described herein. Theinventive system may also be considered to be implemented as a tangible, non-transitory medium, configured with a computer program, where the medium so configured causes a computer to operate in a specific and predefined manner to perform one or more of the processing steps described herein.

[0070] A number of embodiments of the invention have been described.Nevertheless, it is to be understood that the foregoing description is intended to illustrate and not to limit the scope of the invention, which is defined by the scope of the following claims. Accordingly, other embodiments are also within the scope of the following claims. For example, various modifications may be made without departing from the scope of the invention. Additionally, some of the steps described above may be order independent, and thus can be performed in an order different from that described.

Claims

WHAT IS CLAIMED IS:

1. An object comprising: an object body formed from a build material; and a marker embedded in a part of the object body, the marker formed from a fluorescent marker material configured to emit radiation with a first wavelength when exposed to radiation with a second wavelength; wherein build material in the part of the object body is translucent to radiation with the first wavelength and the second wavelength.

2. The object of claim 1, wherein the build material in the pail of the object body is configured with a thickness that allows the radiation with the first wavelength and the second wavelength to pass.

3. The object of claim 1 wherein the object body and the marker are formed using an additive manufacturing process.

4. The object of claim 1 wherein the marker includes an information-bearing code.

5. The object of claim 4 wherein the marker includes a two-dimensional code.

6. The object of claim 5 wherein the marker is one of an ArUco marker, a QR code, a data matrix, a barcode, and an AprilTag.

7. The object of claim 1 wherein the marker is invisible to the naked eye.

8. The object of claim 1 wherein the first wavelength and the second wavelength are in the near-infrared spectrum.

9. The object of claim 1 wherein the first wavelength and the second wavelength are in the ultraviolet spectrum.

10. The object of claim 1 wherein the marker includes a plurality of different fluorescent marker materials.

11. The object of claim 1 wherein the marker material includes acrylonitrile butadiene styrene filament that includes a NIR-fluorescent dye.

12. A method for detecting the marker in the object of any one of claims 1-10, the method comprising: exposing the object to first radiation including radiation with the second wavelength; and capturing an image of the object when exposed to the first radiation including optically filtering radiation emitted from the object to isolate radiation emitted from the marker with the first wavelength.

13. The method of claim 11 further comprising tracking a position of the object including repeatedly detecting the marker in the object over time.

14. A method for fabricating the object of any one of claims 1-10, the method comprising: depositing the build material to form the object body; and depositing the fluorescent marker material to form the marker; wherein the deposited fluorescent marker material is embedded in the deposited build material.

15. The method of claim 13 wherein at least one of the build material and the fluorescent marker material is deposited using an additive fabrication process.

16. The method of claim 14 wherein additive fabrication processing includes a fused deposition modeling additive fabrication process.

Images