Handheld object characterization system and method

Abstract

The handheld object includes a reference element having an angular identifier that is captured differently in an image depending on the camera's viewpoint. The angular identifier generates a first characteristic in a first captured image from a first viewpoint of the camera and a second characteristic in a second captured image from a second viewpoint of the camera. An orientation of the handheld object is determined based on the characteristic generated by the angular identifier in the captured images. A pointing target for the handheld object is also determined based on determining the orientation of the handheld object and the position of the reference element. The reference element also includes a device identifier. The camera captures a first portion of the device identifier in the first image and a second portion of the device identifier in the second image. These portions combine to form the complete device identifier.

Description

[Background technology]

[0001] This section is intended to introduce the reader to various aspects of art that may be related to various aspects of the present disclosure, which are described and / or claimed below. This discussion is believed to be helpful in providing the reader with background information to facilitate a better understanding of the various aspects of the present disclosure. As such, it should be understood that these statements are to be read in this light, and not as admissions of prior art.

[0002] In entertainment venues, for example, handheld objects can be used in coordination with other system components to facilitate interactive experiences. For example, a system may use a gyroscope in a handheld object to determine when the handheld object is pointed toward a particular target, and the system may select the target in response to this determination. In a theme park situation, a patron may use a handheld object to point at an animated figure in an attraction, and the system may cause the animated figure to output a user interactive experience in response to this detection (e.g., wagging the animated figure's tail). It has now been recognized that there is a need for improved (e.g., cost-effective) systems and methods for detecting characteristics of handheld objects to assist in orienting and positioning the handheld objects and / or providing an appropriate response to data associated with each handheld object. Summary of the Invention [Means for solving the problem]

[0003]

[0013] The following summarizes certain embodiments commensurate with the subject matter of the original claims. These embodiments are not intended to limit the scope of the disclosure, but rather merely to provide a brief summary of some disclosed embodiments. Indeed, the disclosure may include a variety of forms that may be similar to or different from the embodiments set forth below.

[0004] In one embodiment, an entertainment system includes a camera that captures an image of a handheld object including an angle-identifying element of the handheld object. The entertainment system also includes a controller having a processor and a memory. The memory stores machine-readable instructions that cause the processor to identify the angle-identifying element of the handheld object in the image and determine an orientation of the handheld object based on characteristics of the detected angle-identifying element in the image.

[0005] In one embodiment, a handheld object facilitates detection of the orientation of the handheld object by a monitoring system. The handheld object includes a handheld body and an angular identifier having a plurality of segments, each segment being visually distinguishable from the other segments. The handheld object also includes a lens disposed adjacent to the angular identifier such that one or more of the plurality of segments are visible through the lens to a camera positioned at a particular position relative to the angular identifier.

[0006] In an embodiment, one or more non-transitory computer-readable media store instructions that, when executed by at least one processor, cause the at least one processor to perform operations including receiving an image of a handheld object and identifying a reference element of the handheld object in the image. The instructions also cause the at least one processor to perform operations including determining a position of the handheld object based on the image and detecting an angular distinguishing feature of the reference element in the image. The instructions further cause the at least one processor to perform operations including determining an orientation of the handheld object based on characteristics of the angular distinguishing feature and determining a pointing target of the handheld object based on the position and orientation of the handheld object.

[0007] These and other features, aspects, and advantages of the present disclosure will be better understood from the following detailed description when read in conjunction with the accompanying drawings, in which like parts are designated by like numerals throughout. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a schematic diagram of a handheld object according to an embodiment of the present disclosure. [Figure 2] 2 is a schematic diagram of an angular discrimination element of the handheld object of FIG. 1 having nine segments, in accordance with an embodiment of the present disclosure. [Figure 3] 2 is a schematic diagram of a theme park attraction system with a user holding the handheld object of FIG. 1 at a first angle, in accordance with an embodiment of the present disclosure. [Figure 4] 2 is a schematic diagram of a theme park attraction system with a user holding the handheld object of FIG. 1 at a second angle, in accordance with an embodiment of the present disclosure. [Figure 5] 2 is a schematic diagram of a theme park attraction system with a user holding the handheld object of FIG. 1 at a third angle in accordance with an embodiment of the present disclosure. [Figure 6] 2 is a schematic diagram of a theme park attraction system with a user holding the handheld object of FIG. 1 at a fourth angle in accordance with an embodiment of the present disclosure. [Figure 7] FIG. 7 is a block diagram of the theme park attraction system of FIGS. 3-6 according to an embodiment of the present disclosure. [Figure 8] 2 is a flow diagram of a process for determining a pointing target for the handheld object of FIG. 1 according to an embodiment of the present disclosure. [Figure 9] 2 is a schematic diagram of a processor of the handheld object of FIG. 1 that determines the orientation of the handheld object based on an angle-discriminating feature according to an embodiment of the present disclosure. [Figure 10] 2 is a flow diagram of a process for determining a device identifier for the handheld object of FIG. 1 according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0009] One or more specific embodiments of the present disclosure will be described below. In the interest of brevity in describing these embodiments, not all features of the implementations are described herein. It will be understood that the development of any such implementation, as in any engineering or design project, requires numerous implementation-specific decisions to be made to achieve the developer's particular objectives, including compliance with system- and business-related constraints that may vary from implementation to implementation. Moreover, it will be understood that such a development effort may be complex and time-consuming, but would be a routine undertaking of design, fabrication, and manufacture for those of ordinary skill in the art having the benefit of this disclosure.

[0010] When introducing elements of various embodiments of the present disclosure, the articles "a," "an," and "the" are intended to mean the presence of one or more of the element. The terms "comprising," "including," and "having" are intended to be inclusive and mean that there may be additional elements other than the listed elements. Furthermore, references to "one embodiment" or "an embodiment" of the present disclosure are not intended to be interpreted as excluding the existence of additional embodiments that also contain the recited features.

[0011] The present disclosure relates generally to systems including handheld objects used for pointing to initiate interactions, and specifically to using the systems to determine the angle or identifier of the handheld object and facilitate designation of appropriate interactions based on such characteristics of the handheld object. Determining the angle at which the handheld object is held can facilitate determining a target at which the handheld object is pointed, thereby ensuring that the attraction outputs an appropriate user interactive experience. For example, based on the pointing direction of the handheld object, a particular target among multiple targets can be accurately designated as selected and an associated effect can be appropriately triggered. Furthermore, the attraction can output different user interactive experiences based on patrons. Thus, determining the identifier of the handheld object can ensure that the attraction outputs an appropriate user interactive experience relevant to the patron.

[0012] Specifically, handheld objects according to the present disclosure can include reference elements that enable detection of characteristics of the handheld object, including relative angles and / or data associated with the handheld object. In some embodiments, the reference elements include angular identifiers designed to be captured differently in images depending on the relative viewpoint of the camera. For example, the angular identifiers can generate a particular detection characteristic, such as a first color or pattern (e.g., blue, checkered, barcode) in a first captured image from a first viewpoint of the camera (e.g., when taken from the right side of the reference element), but a different detection characteristic, such as a second color or pattern (e.g., red, stripes, QR code) in a second captured image from a second viewpoint of the camera (e.g., when taken from the left side of the reference element). Thus, the orientation of the handheld object can be estimated based on the color or pattern generated by the angular identifiers in the captured images. In combination with determining the position of the reference elements, a pointing target for the handheld object can also be determined. Other aspects of the reference elements can enable detection of data associated with the handheld object (e.g., user identification information), which can also be used to generate desired effects. For example, images of a reference element captured from multiple angles can be combined to provide identification information (eg, a QR code) to the system.

[0013] The passive nature of the reference elements avoids the expense associated with including active elements (e.g., gyroscopes, communication devices) in the handheld object that transmit position information to a control system. Indeed, the reference elements can simply operate to appear based on self-illumination or external lighting. Thus, active monitoring of motion data within and communication of motion data from the handheld object can be avoided, beneficially avoiding the complexity and associated expense of each handheld object. In accordance with the present disclosure, passive operation includes operation of the reference elements with or without self-illumination, and excludes any internal measurement capabilities of the reference elements within the handheld object (e.g., gyroscope operation).

[0014] According to embodiments of the present disclosure, the system may include a camera, processor, and memory cooperatively programmed to provide a response to a particular detected characteristic associated with use of the handheld object. As one example, a user may point the handheld object at an animated object (e.g., a robot or other animated figure) at an attraction, and in response to determining that the pointing target is an animated object, the animated object may output a user interactive experience (e.g., wagging its tail). As another example, a user may point at a word on a poster, and in response to determining that the pointing target is the poster, a nearby speaker may output audio speaking the word in the poster. As yet another example, a user may point at an image of a person on an electronic display, and in response to determining that the pointing target is an image of a person, the display may play a video showing the person in the image moving.

[0015] In some embodiments, the determined orientation of the handheld object can facilitate determining whether the user is waving the handheld object in a particular pattern (e.g., a figure-eight pattern). For example, the orientation of the handheld object can be determined within multiple images captured by a camera. That is, the system can infer the orientation of the handheld object at each location based on the type of image captured at various locations using a table or algorithm stored in memory that correlates the orientation to the detected images. These orientations can then be compared to the particular pattern. If the orientation generally matches the particular pattern, an output device can output a user interactive experience.

[0016] As described above, systems according to the present disclosure can also detect characteristics of a handheld object, such as data associations (e.g., the user identity of the handheld object). Accordingly, the reference element can include a device identifier. For example, the device identifier can take the form of a barcode or a quick response (QR) code. While only a portion of the device identifier may be captured in an image, a camera (or multiple cameras) can capture multiple images of different portions of the device identifier, and combine or stitch these portions together to form the complete device identifier. In response to identifying the device identifier, an output device can output a user interactive experience associated with the device identifier.

[0017] 1 is a schematic diagram of a handheld object 10 according to an embodiment of the present disclosure. The handheld object 10 can include a body 12, which can be of any suitable shape. As shown, the body 12 can be shaped as a shaft or wand, although in other embodiments, the body 12 can include any shape that a user can use for pointing (e.g., a projecting device (such as a gun), a wearable item (such as a glove), or a wrist-mounted device (such as a watch)).

[0018] Body 12 may include a recess 13 in which reference element 14 is located. As shown, reference element 14 may be located on an end 15 of handheld object 10 and may provide an indication of where handheld object 10 is located (e.g., when using image recognition techniques). Specifically, reference element 14 may include a light source 16. A system or computing device attempting to determine the location of handheld object 10 may use image recognition techniques to detect light generated from light source 16 and emitted from reference element 14.

[0019] Light source 16 can be any suitable light-generating device that emits light in direction 18. For example, light source 16 can be located at the distal end of handheld object 10 and can be collimated to clarify the direction of light emitted from light source 16 and facilitate detection of where handheld object 10 is pointed based on light detection. As shown, light source 16 can be a light bulb, such as a liquid crystal display (LCD), light-emitting diode (LED), or organic LED (OLED). Light source 16 can be battery-powered and / or rechargeable (e.g., by plugging main body 12 into a charging station or connecting a charging cord to main body). To this end, a power source 19 (e.g., a battery, a capacitor, an energy-harvesting circuit) can be included. In some embodiments, light source 16 can be wirelessly powered (e.g., using ultra-high frequency (UHF) energy harvesting). While the illustrated light source 16 emits light in the visible spectrum, in some embodiments, light source 16 can also emit light in the non-visible spectrum (e.g., infrared or ultraviolet spectrum). In further or alternative embodiments, light source 16 may include a light reflecting device, such as a retroreflective material (e.g., retroreflective sheeting, retroreflective fabric, retroreflective glass beads, microprisms, encapsulated lenses sealed onto a fabric or plastic substrate, and / or metal tape), such that light incident on body 12 may be reflected by the light reflecting device to emit reflected light in direction 18.

[0020] The handheld object 10 can include an angular discrimination element 20 that allows light from the light source 16 to pass through. The angular discrimination element 20 can include a filter or screen (e.g., a pass-through filter) in the form of a printed image or the like. The angular discrimination element 20 can include multiple segments 22 (e.g., visible segments) that each expose light in a different way. For example, each segment 22 can include a different color filter that exposes light as a different respective color (e.g., so that the angular discrimination element 20 is a color wheel), a different pattern filter that exposes light as a different respective pattern, or any other suitable element that causes the light emitted from the reference element 14 to be distinguishably different from different viewpoints.

[0021] 1 shows sections 22A-D as different patterns, each representing a different color filter. For example, section 22A can be a red filter that makes light from light source 16 appear red, section 22B can be a blue filter that makes light from light source 16 appear blue, section 22C can be a yellow filter that makes light from light source 16 appear yellow, and section 22D can be a green filter that makes light from light source 16 appear green. Thus, reference element 14 can appear to emit red light or be red when viewed from a first perspective (e.g., closest to red filter 22A). Meanwhile, reference element 14 can appear to emit blue light or be blue when viewed from a second perspective (e.g., closest to blue filter 22B).

[0022] Although four segments 22A-D are shown in the angular discrimination element 20, it should be understood that any suitable number of segments 22 can be used in the angular discrimination element 20 to facilitate determining the angle at which the handheld object 10 is pointed. For example, FIG. 2 is a schematic diagram of the angular discrimination element 20 of the handheld object 10 of FIG. 1 having nine segments 22, in accordance with an embodiment of the present disclosure. As shown, segment 22A can be a red filter, segment 22B can be a blue filter, segment 22C can be a yellow filter, segment 22D can be a green filter, segment 22E can be an orange filter, segment 22F can be a purple filter, segment 22G can be a pink filter, segment 22H can be a brown filter, and segment 22I can be a gray filter, with any suitable number of filters having any suitable color, pattern, or other form that makes the light emitted from the reference element 14 distinguishable (visually distinguishable) from different viewpoints being contemplated. The segments 22 can be of any suitable form. For example, segments 22 may be highly reflective segments of material that reflect external light to facilitate detection by a monitoring system. As another example, each segment 22 may be a colored light bulb that may facilitate detection by a monitoring system through hue and / or behavior (e.g., flashing). However, in the exemplary embodiment of FIG. 1, segments 22 are pass-through (e.g., translucent or transparent) filters, and will be referred to as such with respect to other embodiments.

[0023] The handheld object 10 may also include a device identifier 24 that transmits light from the light source 16. Like the angular identification element 20, the device identifier 24 may include a filter or screen in the form of a printed image or the like. The device identifier 24 may include any suitable identifier, such as a barcode, a Quick Response (QR) code, a Universal Product Code (UPC), a serial number, a product number, or the like, operable to associate data (e.g., user information, profile) with the handheld object 10. FIG. 1 illustrates the device identifier 24 in the form of a printed QR code 26 that transmits light from the light source 16. However, like the segment 22, the device identifier 24 may take different forms (e.g., reflectors, bulb configurations). It should be noted that the angular identification element 20 and the device identifier 24 may also be a single feature. For example, the combination of the segment 22 and the device identifier 24 may include a multi-color QR code image that appears different colors when viewed from different angles through the lens 28 of the reference element 14, allowing a system processor to stitch together the combined views to identify the QR code.

[0024] The handheld object 10 may include a lens 28 (e.g., a refractive lens) that disperses light from the light source 16 to facilitate detection of the reference element 14 (e.g., to facilitate determining the location of the handheld object 10) and to facilitate determining the color of the light emitted from the reference element 14. Specifically, the lens 28 may be positioned adjacent the section 22 and the device identifier 24 (which may be combined) to refract the light so that different portions of the section 22 / device identifier 24 are visible from different angles. As will be understood in the art, the lens 28 may include various shapes (e.g., prismatic or spherical) for this purpose. As shown, the lens 28 is made of glass having a spherical shape, thereby magnifying the light emitted from the light source 16 and / or providing a magnified view of the section 22 of the angle identification element 20 that is visible to an observer (e.g., within the line of sight), although a lens 28 of any suitable shape or material that facilitates determining the color of the light emitted from the reference element 14 when viewing the handheld object 10 is contemplated. Lens 28 may represent a hemisphere, a triangular prism, or the like, and may be formed of plastic instead of glass. Additionally, some embodiments may use a series of baffles or other elements that direct light in a particular direction (e.g., indicative of the angle at which handheld object 10 is being held). Still further, in other embodiments, light source 16 is not present, and light is received into and dispersed from lens 28 to facilitate detection of the side of reference element 14 that can be correlated to positioning and / or related data.

[0025] In some embodiments, the handheld object 10 may include one or more holding features 30, such as one or more depressed finger indentations 30A and / or ridges 30B. The holding features 30 may guide the user to hold the handheld object 10 in a particular manner. For example, the holding features 30 may ensure that the user holds the handheld object 10 in a certain orientation so that the angular identification element 20 does not rotate while the user is holding or using the handheld object 10. The holding features 30 may also include any additional or other suitable features that prevent rotation of the handheld object 10 when held by the user, such as a finger guard relative to the user's gripping direction (e.g., making it difficult for the user to rotate the handheld object 10 by inserting an index finger). This may facilitate detection of the reference element 14 and evaluation of correct operation based on the associated image pattern.

[0026] In further or alternative embodiments, a user, via a controller or control system (such as controller 90 shown in FIG. 7 and described in further detail below), can perform a calibration process to determine how the user is holding the handheld object 10 (e.g., the initial orientation of the handheld object 10 when held by the user) so that the reference element 14 is emitting light corresponding to each section 22 present at a particular location (e.g., emitting light by reflecting external light or emitting light from an internal light source). For example, a camera communicatively coupled to the controller can observe the reference element 14 as having a blue color because the blue filter 22B is visible to the camera through the lens 28 and / or is within the camera's line of sight. Further detection of the color of the reference element 14 can indicate the angle of the handheld object 10 based at least on the calibration position. That is, if the camera subsequently captures an image of the reference element 14 as having a red color because the red filter 22A is visible to the camera through the lens 28, the controller can determine the angle by the position of the red filter 22A relative to the blue filter 22B on the angle identification element 20.

[0027] The handheld object 10 can be mass-produced relatively easily because the components of the handheld object 10 (e.g., the angular identification element 20, the device identifier 24, and the lens 28) are relatively simple (e.g., a color filter, a printed barcode, or a glass sphere). Furthermore, because these components are relatively small (e.g., they may have diameters of less than 3 centimeters (cm), less than 2 cm, less than 1 cm, less than 0.5 cm, or less than 0.3 cm, respectively), the visual impact of the reference element 14 can be small. That is, the reference element 14 may not impair the user experience of using the handheld object 10. Furthermore, if the light source 16 emits or reflects light that is invisible to the human eye (e.g., infrared light, ultraviolet light), the visual impact of the reference element 14 can be even smaller.

[0028] 3 is a schematic diagram of a theme park attraction or entertainment system 40 with a user 42 holding a handheld object 10 at a first angle, in accordance with an embodiment of the present disclosure. The theme park attraction system 40 may enable the user 42 to point the handheld object 10 at a particular target or move the handheld object 10 to perform a particular gesture or follow a particular pattern, and in response output a user interactive experience. For example, the theme park attraction system 40 may include a setting with a popular children's character, a television or movie-themed setting, a shooting gallery and a group of targets, etc.

[0029] The theme park attraction system 40 may include a camera 44 or other image capture device that captures images of the handheld object 10. The camera 44 may be communicatively coupled to a controller or control system (such as the controller 90 shown in FIG. 7 and described in further detail below) that determines the position or location of the reference element 14 on the handheld object 10 (e.g., based on the captured image of the handheld object 10). The controller may determine the position of the reference element 14 on a two-dimensional plane or in three dimensions. Additionally, the controller may determine the pointing angle of the handheld object 10 based on the color detected from the reference element 14. In some embodiments, the position of the camera 44 and the positions of the reference element 14 or components of the reference element 14 may be reversed. That is, the camera 44 may be attached or mounted to the handheld object 10, while the reference element 14, angle identifier 20, device identifier 24, and / or lens 28 may reside in fixed locations remote from the handheld object 10.

[0030] To determine the position of reference element 14 in three dimensions, theme park attraction system 40 may include a second camera (also represented by element 44) that captures an image of handheld object 10 that can provide a depth dimension (e.g., z-axis) relative to a two-dimensional plane (e.g., x- and y-axes). In further or alternative embodiments, reference element 14 may also include a second lens and a second angular discriminator (also represented by elements 28 and 20). In this manner, theme park attraction system 40, despite including only a single camera 44, can determine the position of reference element 14 in three dimensions by tracking a third axis of rotation based on two lenses and two angular discriminators.

[0031] As shown, based on the camera's perspective, the reference element 14 appears to emit green light 46. This appearance is due to the positioning of the green filter 22D relative to the line of sight of the camera 44, refraction through the lens 28 relative to the viewing angle of the camera 44, and the light source 16 of the handheld object 10 emitting light through the green filter 22D. This appearance is correlated to the orientation of the handheld object 10 relative to the camera 44, and this correlation can be provided by a processor (such as processor 92 shown in FIG. 7 and described in further detail below) of the controller of the park attraction system 40 employing an algorithm or table stored in a memory (such as memory 94 shown in FIG. 7 and described in further detail below) of the park attraction system 40. Thus, the controller can determine the angle at which the user 42 is holding the handheld object 10 based on the color of the reference element 14 and / or the determined position of the reference element 14. For example, detection of the green light 46 can be correlated to the user pointing the handheld object 10 upward.

[0032] The image captured by the camera 44 may also include a first portion 48 of the device identifier 24. However, the controller may be unable to read the device identifier 24 because only the first portion 48 is captured due to the refractive nature of the lens 28. That is, because the lens 28 can magnify the light emitted from the reference element 14, the device identifier 24 viewed through the lens 28 may also be magnified or zoomed in, causing the camera 44 to capture only the first portion 48 in the image. Thus, the controller may store the first portion 48 of the device identifier 24 in memory and combine it with other portions of the device identifier 24 until the complete device identifier 24 can be generated. For example, the memory and processor may be programmed to stitch together images of the device identifier 24 based on correlation of overlapping images to provide a consolidated image of the device identifier 24.

[0033] As shown, the theme park attraction system 40 may include one or more output devices 50, such as an animated figure, an electronic display 50A, or a speaker 50B. A controller (such as the controller 90 shown in FIG. 7 and described in further detail below) may instruct the output device 50 to output a user interactive experience (e.g., a particular movement or actuation, image, video, or audio data) based on the determined position of the handheld object 10, the determined angle of the handheld object 10, and / or the determined device identifier 24. For example, if the controller determines that the handheld object 10 is pointing at an animated object (e.g., a robot or other animated figure), the controller may instruct the animated object to output a user interactive experience (e.g., wagging its tail). As another example, if the controller determines that the handheld object 10 is pointing at a word on a poster, the controller may instruct the speaker 50B to output audio that speaks the word. As yet another example, if the controller determines that the handheld object 10 is pointing at an image of a person on the display 50A, it can instruct the display 50A to play a video showing the person in the image moving. If the controller determines that the handheld object 10 has moved in a particular pattern or sequence (e.g., based on determining the position of the reference element 14 and / or the angle at which the handheld object 10 is being held as captured in multiple images by the camera 44), it can instruct the output device 50 to output a user interactive experience (e.g., particular video and audio data) associated with these particular patterns or sequences (e.g., figure eights). Once the controller determines the (complete) device identifier 24, it can instruct the output device 50 to output a user interactive experience associated with the device identifier 24 (e.g., instructing the speaker 50B to state the user's name).

[0034] 4 is a schematic diagram of a theme park attraction system 40 with a user 42 holding a handheld object 10 at a second angle, in accordance with an embodiment of the present disclosure. As shown, based on the camera's perspective, the reference element 14 appears to emit red light 60 because the red filter 22A is visible to and / or within the line of sight of the camera 44 and the light source 16 of the handheld object 10 emits light through the red filter 22A. Thus, a controller (such as the controller 90 shown in FIG. 7 and described in further detail below) can determine the angle at which the user 42 is holding the handheld object 10 (e.g., pointing the handheld object 10 downward) based on the color of the reference element 14 and / or the determined position of the reference element 14.

[0035] The image captured by camera 44 may also include second portion 62 of device identifier 24. If the controller can combine the captured image of first portion 48 and second portion 62 of device identifier 24 from FIG. 3 to complete device identifier 24, it can read the complete device identifier 24 and use it to identify associated data. Otherwise, the controller can store second portion 62 of device identifier 24 in memory and combine it with other portions of device identifier 24 until the complete device identifier 24 is generated. Specifically, the controller can identify overlapping portions between portions of device identifier 24 to determine which portions of device identifier 24 match, combine these portions, and store them in memory. The controller can also identify ends of device identifier 24 in the portions of device identifier 24 and determine that the complete device identifier 24 has been combined if the combined portions of device identifier 24 include the ends of device identifier 24.

[0036] 5 is a schematic diagram of a theme park attraction system 40 with a user 42 holding a handheld object 10 at a third angle, in accordance with an embodiment of the present disclosure. As shown, based on the camera's perspective, the reference element 14 appears to emit blue light 70 because the blue filter 22B is visible to and / or within the line of sight of the camera 44 and the light source 16 of the handheld object 10 emits light through the blue filter 22B. Thus, the controller can determine the angle at which the user 42 is holding the handheld object 10 (e.g., pointing the handheld object 10 leftward from the user's perspective) based on the color of the reference element 14 and / or the determined position of the reference element 14.

[0037] The image captured by camera 44 may also include third portion 72 of device identifier 24. The controller can read the complete device identifier 24 if it can combine the image of first portion 48 of device identifier 24 from Figure 3 and the images of second portion 62 and third portion 72 from Figure 4 to provide a complete image of device identifier 24. Otherwise, the controller can store the image of third portion 72 of device identifier 24 in memory and combine it with other image portions of device identifier 24 until an image of the complete device identifier 24 can be generated.

[0038] 6 is a schematic diagram of a theme park attraction system 40 with a user 42 holding a handheld object 10 at a fourth angle, in accordance with an embodiment of the present disclosure. As shown, based on the camera's perspective, the reference element 14 appears to emit yellow light 80 because the yellow filter 22C is visible to and / or within the line of sight of the camera 44 and the light source 16 of the handheld object 10 emits light through the yellow filter 22C. Thus, a controller (such as the controller 90 shown in FIG. 7 and described in further detail below) can determine the angle at which the user 42 is holding the handheld object 10 (e.g., pointing the handheld object 10 to the right from the user's perspective) based on the color of the reference element 14 and / or the determined position of the reference element 14.

[0039] The image captured by camera 44 may also include fourth portion 82 of device identifier 24. The controller can read the complete device identifier 24 if it can combine images of first portion 48 of device identifier 24 from Figure 3, second portion 62 from Figure 4, third portion 72 from Figure 5, and fourth portion 82 to complete an image of the complete device identifier 24. Otherwise, the controller can store the image of fourth portion 82 of device identifier 24 in memory and combine it with images of other portions of device identifier 24 until the complete device identifier 24 can be generated.

[0040] 7 is a block diagram of the theme park attraction system 40 of FIGS. 3-6 in accordance with an embodiment of the present disclosure. As shown, a camera 44, which may be communicatively coupled to a controller 90, may capture an image of the handheld object 10. The image may include the color of the reference element 14 detected by the camera 44 via transmission through the lens 28. This color may correspond to the section 22 of the angular identification element 20 through which light emitted by the light source 16 passes or which reflects light and is directed toward the camera 44 via refraction by the lens 28. The image may also include a portion of the device identifier 24, which may be detected in a similar manner or as part of the color detection.

[0041] The controller 90 may include one or more processors (as shown and referred to in this disclosure as a single processor 92) and one or more memory or storage devices (as shown and referred to in this disclosure as a single memory device 94). The processor 92 may execute software programs and / or instructions stored in the memory device 94 that facilitate determining the position of the handheld object 10 and / or reference elements 14, determining the color of the reference elements 14, determining the angle of the handheld object 10, and / or determining the device identifier 24. Additionally, the processor 92 may include multiple microprocessors, one or more "general-purpose" microprocessors, one or more special-purpose microprocessors, and / or one or more application-specific integrated circuits (ASICs). For example, the processor 92 may include one or more reduced instruction set computer (RISC) processors. The memory device 94 may store information such as control software, look-up tables, and configuration data. The memory device 94 may include a tangible, non-transitory, machine-readable medium, such as volatile memory (e.g., random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM)), flash memory, one or more hard drives, and / or any other suitable optical, magnetic, or solid-state storage medium. The memory device 94 may store a variety of information and may be used for a variety of purposes, such as instructions that facilitate determining the position of the handheld object 10 and / or reference element 14, determining the color of the reference element 14, determining the angle of the handheld object 10 and / or the device identifier 24.

[0042] Specifically, the processor 92 may execute image recognition techniques stored in the memory device 94 to detect the reference elements 14, the color of the reference elements 14, and / or the device identifier 24 in images of the handheld object 10 captured by the camera 44. The processor 92 may then determine the angle of the handheld object 10 based, for example, on the position of the reference elements 14 and / or the color of the reference elements 14. This determination may include comparing the detected characteristics (e.g., color shade, pattern) to a table of information or executing an algorithm based on the detected characteristics to identify relative positioning and identity information of the associated handheld object 10.

[0043] The controller 90 may also be communicatively coupled to the output device 50 to instruct the output device 50 (e.g., an animated figure, an electronic display, a speaker) to output actions, images, video and audio data, etc. The controller 90 may be communicatively coupled to the camera 44 and / or the output device 50 by any suitable means, such as via wired communication or a communication network using a wireless communication protocol or technology (e.g., wireless, Bluetooth, WiFi, infrared, Ethernet, Thread, ZigBee, Z-Wave, KNX, mobile and / or microwave).

[0044] 8 is a flow diagram of a process 110 for determining a pointing target for a handheld object 10, according to an embodiment of the present disclosure. Process 110 may be performed by any suitable system capable of determining a pointing target for a handheld object 10, such as any component of theme park attraction system 40, including camera 44, controller 90, processor 92, and / or output device 50. While process 110 is described using a particular order of steps, it should be understood in discussion of this disclosure that the described steps may be performed in an order different from that shown, or that some described steps may be skipped or not performed entirely. In some embodiments, process 110 may be implemented by executing instructions stored on a tangible, non-transitory computer-readable medium, such as memory device 94, using a processor, such as processor 92.

[0045] As shown, at process block 112, processor 92 receives an image of handheld object 10. Specifically, camera 44 may capture an image of handheld object 10 (e.g., held by user 42) and transmit the image, specifically to controller 90 and processor 92. Accordingly, processor 92, which may reside within controller 90, may receive the image (e.g., data indicative of the captured image).

[0046] At process block 114, the processor 92 identifies the reference element 14 of the handheld object 10 in the image. For example, the processor 92 may use image recognition techniques (e.g., stored as instructions in the memory device 94) to detect characteristics (e.g., color, markings, lens shape) of the visible side of the reference element 14. These detected characteristics may be used to identify the reference element 14 and / or identify the relative positioning of the handheld object 10.

[0047] At process block 116, the processor 92 determines the location of the reference element 14 or handheld object 10. The processor 92 may determine the location of the reference element 14 or handheld object 10 in a two-dimensional plane (e.g., the image plane) or in three dimensions. This determination may be made by the processor 92 implementing image recognition techniques or algorithms, including machine learning, artificial intelligence, deep learning, convolutional neural networks, and the like. For example, the memory 94 may store an image recognition model that can be trained by inputting a sample image of the reference element 14 or handheld object 10 and an indication of where the reference element 14 or handheld object 10 is located within the sample image. The processor 92 may then use the trained image recognition model to identify the reference element 14 or handheld object 10 within the image and determine the location of the reference element 14 or handheld object 10 within the image (e.g., the pixel in the image that corresponds to the reference element 14 or handheld object 10).

[0048] At process block 118, the processor 92 detects visible characteristics of the angle-discriminating feature of the reference element 14 in the image. Specifically, the processor 92 may detect a color of light emitted from the reference element 14 that corresponds to the color of light filtered through the colored filter 22 of the angle-discriminating element 20. This detected feature may then be used at process block 120, where the processor 92 determines the orientation of the handheld object 10 based on the angle-discriminating feature. Specifically, the processor 92 may estimate the orientation of the handheld object 10 based on the color of light emitted by the reference element 14. For example, FIG. 9 is a schematic diagram illustrating the processor 92 determining the orientation of the handheld object 10 based on the angle-discriminating feature 130, according to an embodiment of the present disclosure. The processor 92 maps the position 132 of the reference element 14 determined at process block 116 onto a two-dimensional plane 134, although in some embodiments, the position 132 may be mapped in three dimensions. Coordinate axes 136 are shown for reference. As shown, the processor 92 detects that the angle discrimination feature 130 is blue (e.g., due to the color of light emitted from the reference element 14 through the blue filter 22B). The processor 92 can determine that the handheld object 10 is facing a direction 138 based on the position of the blue filter 22B on the angle discrimination element 20, for example, by using a table or algorithm that correlates such data. For example, the processor 92 can use an image recognition model stored in the memory 94 that can be trained by inputting sample images of the reference element 14 and an indication of the orientation of the reference element 14 in the sample images. The processor 92 can use this trained image recognition model to determine the orientation 138 of the reference element 14. In further or alternative embodiments, the processor 92 can use any other suitable image recognition technique or algorithm, including machine learning, artificial intelligence, deep learning, convolutional neural networks, and the like.

[0049] In some embodiments, the processor 92 can determine that the handheld object 10 is facing in a direction 138 based on how the user is holding the handheld object 10 (e.g., establishing a reference orientation using the holding feature 30 shown in FIG. 1 ), a calibration process, an estimated or known user height, and the user's position or location. For example, if the user's height is estimated or known, the orientation 138 can be inferred by estimating the edge position of the handheld object 10 (opposite the reference element 14) based on the user's height (e.g., the edge position can indicate the center of the user). Similarly, if the user's position or location is known, the orientation 138 can be inferred by estimating the edge position of the handheld object 10 based on the user's position. In some embodiments, the processor 92 can determine the angle 140 of the handheld object 10. For example, the angle 140 of the handheld object 10 is approximately 90 degrees with respect to the y-axis of the coordinate axes 136.

[0050] At process block 122, the processor 92 determines a pointing target for the handheld object 10 based on the orientation of the handheld object 10 and the position of the reference element 14. The pointing target may include an image, an object, a portion of an image, etc. at which the user is pointing the handheld object 10. Specifically, the processor 92 may determine the pointing target 144 by extending a direction 142 of the orientation of the handheld object 10 from the position of the reference element 14 and estimating the end point of the direction 142.

[0051] If, at decision block 124, the pointing target 144 is associated with a user interactive experience, then, at process block 126, the processor 92 instructs the output device 50 to execute the user interactive experience. As shown in FIG. 9, a graphical object 146 is provided (e.g., on a wall) or displayed (e.g., on an electronic display 50A). Because the pointing target 144 is on or coincides with the graphical object 146, the processor 92 may instruct the speaker 50B to output audio data (e.g., associated with the graphical object), instruct the display 50A to move the graphical object 146 (e.g., by playing a video), instruct an animated object to move, etc. If the pointing target 144 is not associated with a user interactive experience, the processor 92 repeats process 110. In this manner, the process 110 can enable the processor 92 to determine the pointing target 144 of the handheld object 10 without the expense or complexity associated with including active elements (e.g., gyroscopes, communication devices) on the handheld object 10 to transmit position information to the controller 90.

[0052] In some embodiments, the processor 92 can instruct the output device 50 to execute a user interactive experience based on one or more orientations of the handheld object. For example, the memory 94 can store a particular pattern (e.g., a zigzag pattern, a figure-eight pattern, a letter or number pattern), and the processor 92 can determine whether the user is moving the handheld object 10 in the particular pattern. This determination can be made by the processor 92 implementing image recognition techniques or algorithms, including machine learning, artificial intelligence, deep learning, convolutional neural networks, and the like. For example, the memory 94 can store an image recognition model that can be trained by inputting sample images in which the reference element 14 is moving or "drawn" in the particular pattern. The processor 92 can then use the trained image recognition model to determine whether the reference element 14 is moving in the particular pattern.

[0053] The processor 92 may also determine a device identifier 24 for the handheld object 10, which may uniquely identify the handheld object 10, associate the handheld object 10 with a user, and / or associate the handheld object 10 with a user profile. FIG. 10 is a flow diagram of a process 160 for determining a device identifier 24 for a handheld object 10 according to an embodiment of the present disclosure. The process 160 may be performed by any suitable device capable of determining a device identifier 24 for a handheld object 10, such as any component of the theme park attraction system 40, including the camera 44, the controller 90, the processor 92, and / or the output device 50. While the process 160 is described using a particular order of steps, it should be understood in discussion of the present disclosure that the described steps may be performed in an order different from that shown, or that some described steps may be skipped or not performed entirely. In some embodiments, the process 160 may be implemented by a processor, such as the processor 92, executing instructions stored on a tangible, non-transitory computer-readable medium, such as the memory device 94.

[0054] As shown, at process block 162, the processor 92 receives an image of the handheld object 10. Specifically, the processor 92, located within the controller 90, may receive the image from the camera 44 after the camera 44 captures the image of the handheld object 10 and transmits the image to the processor 92. At process block 164, the processor 92 uses the received image to determine at least a portion of the device identifier 24. Because the lens 28 of the reference element 14 can adjust the light (e.g., light emitted from the light source 16) used to detect the device identifier 24, only a limited portion of the device identifier 24 may be discernible from a given image. For example, the lens 28 may magnify or zoom in on the device identifier 24 visible through the lens 28, causing the camera 44 to capture only a portion of the device identifier 24 in the image. Specifically, for example, as shown in FIG. 3, the camera 44 may capture only a first portion 48 in the image.

[0055] At process block 166, the processor 92 combines the image of the portion of the device identifier 24 with any other images of the respective portion of the device identifier 24. For example, the processor 92 may store other images of the portion of the device identifier 24 determined from previously captured images in a memory or storage device, such as the memory device 94. Thus, the processor 92 may combine or splice the portion of the device identifier 24 (e.g., the first portion 48) with any stored portions. The processor 92 may combine these portions with each other using any suitable image combining or forming technique or algorithm. Specifically, the processor 92 may identify overlapping portions between portions of the device identifier 24 to determine which portions of the device identifier 24 match each other, and combine and store these portions in the memory 94.

[0056] At decision block 168, processor 92 determines whether the combined image of the portions of device identifier 24 forms an image of the complete device identifier 24 (e.g., an image sufficient to read and associate all relevant information with the device identifier 24). Specifically, processor 92 may identify the ends of the device identifier 24 in the portions of device identifier 24 and determine that the complete device identifier 24 has been combined if the combined portions of the device identifier 24 include the ends of the device identifier 24.

[0057] If the processor 92 determines that the combined portions of the device identifier 24 do not form a complete image of the device identifier 24, the image of the device identifier 24 is incomplete and therefore unreadable by the processor 92, and the processor 92 stores the portions of the device identifier 24 (e.g., in the memory device 94) at process block 170 and repeats the process 160 by receiving a subsequent image of the handheld object 10.

[0058] If the processor 92 determines that the combined portions of the device identifier 24 form a complete image of the device identifier 24, then the processor 92 reads the device identifier 24 at process block 172 and uses the device identifier 24 to identify the handheld object 10. In particular, the memory 94 may store a correlation between the device identifier 24 and identification data (e.g., identification number) of the handheld object 10, the user, and / or the user profile.

[0059] If, at decision block 174, the device identifier 24 is associated with a user interactive experience, then, at process block 176, the processor 92 instructs the output device 50 to perform the user interactive experience. For example, the device identifier 24 may be associated with a user profile. Thus, the processor 92 may query a database (e.g., stored in memory device 94) for a username, which may be a field in the user profile, and instruct the speaker 50B to state the user's name. If the device identifier 24 is not associated with a user interactive experience, the processor 92 repeats process 160.

[0060] In this manner, process 160 may enable processor 92 to determine device identifier 24 for handheld object 10 without the expense or complexity associated with including an active element (e.g., a communications device) in handheld object 10 to transmit identifying information to controller 90. In some embodiments, processes 110 and 160 may be combined. That is, processor 92 may perform processes 110 and 160 to determine device identifier 24 for handheld object 10 while determining the pointing target for handheld object 10.

[0061] While the embodiments described in this disclosure are susceptible to various modifications and alternative forms, specific embodiments have been shown by way of example in the drawings and have been described in detail herein. It should be understood, however, that the disclosure is not intended to be limited to the particular forms disclosed. The present disclosure covers all modifications, equivalents, and alternatives falling within the spirit and scope of the disclosure as defined by the following appended claims.

[0062] The technology shown and claimed herein refers to and applies to tangible objects and specific examples of a practical nature that will materially improve the art, and thus are not abstract, intangible, or purely theoretical. Furthermore, where any claim appended at the end of this specification contains one or more elements designated as "means for [performing] ... [function]" or "step for [performing] ... [function]," such elements are to be construed pursuant to 35 U.S.C. 112(f). Conversely, for any claim containing elements designated in any other manner, such elements are not to be construed pursuant to 35 U.S.C. 112(f). [Explanation of symbols]

[0063] 10 Handheld Objects 18 directions 40 Theme Park Attraction System 42 users 44 Camera 46 Green Light 48 First part of device identifier 50A

Claims

1. a camera configured to capture images of a handheld object including angular discriminators with different visual characteristics depending on the viewpoint; a controller having a processor and a memory storing machine-readable instructions; the machine-readable instructions comprising: combining the image including at least a portion of the angular discrimination element with a previously stored image of a portion of the angular discrimination element to form a combined image; identifying a visible characteristic of the angle identifier of the handheld object in the combined image; determining an orientation of the handheld object based on identified visual characteristics of the angle-identifying element in the combined image; determining a position of the handheld object within the combined image; determining a pointing target for the handheld object based on the position and orientation of the handheld object; configured to cause the processor to An entertainment system characterized by:

2. the handheld object, wherein the angular discrimination element comprises a refractive lens disposed over a plurality of visible segments, wherein light emitted through or reflected from each visible segment of the plurality of visible segments is distinguishable from light emitted through or reflected from other visible segments of the plurality of visible segments.

10. The entertainment system of claim 1.

3. the handheld object includes a light source, and each visibility section of the plurality of visibility sections includes a pass-through filter; 3. The entertainment system of claim 2.

4. the handheld object including a device identifier disposed adjacent to a lens configured to enable the camera to capture a second image of at least a portion of the device identifier.

10. The entertainment system of claim 1.

5. an output device configured to output a user interactive experience, wherein the memory stores machine-readable instructions configured to cause the processor to read the combined image and instruct the output device to output a user interactive experience based on the combined image.

10. The entertainment system of claim 1.

6. the memory stores a correlation between the device identifier and identification data of the handheld object, a user, a user profile, or any combination thereof; 5. The entertainment system of claim 4.

7. the device identifier comprises a barcode, a quick response (QR) code, a universal product code (UPC), a serial number, a product number, or any combination thereof; 5. The entertainment system of claim 4.

8. an output device configured to output a user interactive experience, the memory storing machine-readable instructions configured to cause the processor to instruct the output device to output the user interactive experience based on the orientation of the handheld object.

10. The entertainment system of claim 1.

9. the angular discrimination element includes a plurality of segments, each segment of the plurality of segments being visually distinguishable from the other segments of the plurality of segments; 10. The entertainment system of claim 1.

10. the plurality of segments in combination to provide a device identifier.

10. The entertainment system of claim 9.

11. a handheld object configured to facilitate detection of an orientation of the handheld object by a monitoring system; a main body configured to be handheld; a holding feature configured to induce a user to hold the handheld object in a particular manner; an angular identification element including a plurality of segments, each segment of the plurality of segments being visually distinguishable from the other segments of the plurality of segments, the plurality of segments combining to provide a device identifier indicative of the identity of the handheld object; Lenses and the lens is positioned adjacent to the angular discrimination element such that one or more segments of the plurality of segments are visible through the lens to a camera at a particular position relative to the angular discrimination element; The angle discrimination element has different visual characteristics depending on the viewpoint. A handheld object characterized by:

12. a light source configured to emit light through each of the plurality of segments and through the lens; 12. The handheld object of claim 11.

13. the light source comprises a liquid crystal display, a light emitting diode, or an organic light emitting diode; 13. The handheld object of claim 12.

14. the device identifier comprises a barcode, a quick response (QR) code, a universal product code (UPC), a serial number, a product number, or any combination thereof; 12. The handheld object of claim 11.

15. the plurality of sections are formed of a reflective material; 12. The handheld object of claim 11.

16. the lens is configured to magnify light emitted through or reflected by the plurality of segments.

12. The handheld object of claim 11.

17. One or more non-transitory computer-readable media having instructions stored thereon, the instructions, when executed by at least one processor, receiving an image of a handheld object; identifying a reference feature of the handheld object in the image, the reference feature including an angular identifier having different visual characteristics depending on the viewpoint; combining the image including at least a portion of the angular discrimination element with a previously stored image of a portion of the angular discrimination element to form a combined image; Detecting an angle identifier of the reference element in the combined image; determining an orientation of the handheld object based on a visual characteristic of the angle identification element; determining a position of the handheld object within the combined image; determining a pointing target for the handheld object based on the position and orientation of the handheld object; causing at least one processor to perform operations including:

1. One or more non-transitory computer-readable media comprising:

18. the instructions cause the at least one processor to perform operations including performing a calibration process to determine an initial orientation of the handheld object based on characteristics of the angle-identifying elements of the reference elements in the combined image.

18. One or more non-transitory computer-readable media according to claim 17.

Images