Handheld object property evaluation system and method

Abstract

A handheld object includes a reference element having an angle identification element, which is captured differently in an image depending on the viewpoint of a camera. The angle identification element generates a first characteristic in a first captured image from a first viewpoint of the camera and a second characteristic in a second captured image from a second viewpoint of the camera. The orientation of the handheld object is determined based on the characteristics generated by the angle identification element in the captured images. The target of the handheld object is also determined based on the orientation of the handheld object and the determined position of the reference element. The reference element also includes a device identifier. The camera captures a first portion of the device identifier in a first image and a second portion of the device identifier in a second image. The portions are combined to form a complete device identifier.

Description

Background Technology

[0001] This section aims to introduce the reader to various technical aspects that may relate to the various aspects of this disclosure, which are described and / or claimed below. This discussion is intended to help provide the reader with background information to facilitate a better understanding of the various aspects of this disclosure. Therefore, it is understood that these statements will be read from this perspective, rather than as an admission of prior art.

[0002] In entertainment venues, for example, handheld objects can be used in coordination with other system components to initiate interactive experiences. For instance, the system can use a gyroscope in the handheld object to determine if it is pointing towards a specific target, and the system can select the target in response to this determination. In a theme park setting, visitors can use a handheld object to point at an animated character at an attraction, and in response to this detection, the system can cause the animated character to output an interactive user experience (e.g., shake its tail). It is now recognized that there is a need for improved (e.g., more cost-effective) systems and methods for detecting the characteristics of handheld objects to facilitate the provision of appropriate responses to the orientation and positioning of handheld objects and / or data associated with the corresponding handheld object. Summary of the Invention

[0003] Certain embodiments corresponding in scope to the originally claimed subject matter are summarized below. These embodiments are not intended to limit the scope of this disclosure, but rather are intended only to provide a brief overview of some of the disclosed embodiments. In fact, this disclosure may include a wide variety of forms that may be similar to or different from the embodiments set forth below.

[0004] In one embodiment, an entertainment system includes a camera that captures an image of a handheld object, including angle identification elements of the handheld object. The entertainment system also includes a controller having a processor and a memory. The memory stores machine-readable instructions that instruct the processor to: identify angle identification elements of the handheld object in the image; and determine the orientation of the handheld object based on detected characteristics of the angle identification elements in the image.

[0005] In one embodiment, a handheld object facilitates the detection of the handheld object's orientation by a monitoring system. The handheld object includes: a body to be held; and an angle marker element having multiple segments. Each segment is visually distinguishable relative to the other segments. The handheld object also includes a lens positioned adjacent to the angle marker element such that one or more segments of the multiple segments are visible via the lens to a camera at a specific location relative to the angle marker element.

[0006] In an embodiment, one or more non-transitory computer-readable media store instructions that, when executed by at least one processor, cause the at least one processor to perform operations including: receiving an image of a handheld object; and identifying a reference element of the handheld object in the image. The instructions also cause the at least one processor to perform operations including: determining the position of the handheld object based on the image; and detecting angular identification features of the reference element in the image. The instructions further cause the at least one processor to perform operations including: determining the orientation of the handheld object based on characteristics of the angular identification features; and determining a target being pointed at by the handheld object based on its position and orientation. Attached Figure Description

[0007] These and other features, aspects, and advantages of this disclosure will become better understood when the following detailed description is read with reference to the accompanying drawings, in which the same characters denote the same parts throughout the drawings, wherein:

[0008] Figure 1 This is a schematic diagram of a handheld object according to an embodiment of the present disclosure;

[0009] Figure 2 It is a nine-segment structure according to an embodiment of this disclosure. Figure 1 A schematic diagram of the angle indicator element of the handheld object;

[0010] Figure 3 This is a schematic diagram of a theme park attraction system according to an embodiment of the present disclosure, wherein the user holds the device at a first angle. Figure 1 The object being held;

[0011] Figure 4 This is a schematic diagram of a theme park attraction system according to an embodiment of the present disclosure, wherein the user holds the device at a second angle. Figure 1 The object being held;

[0012] Figure 5 This is a schematic diagram of a theme park attraction system according to an embodiment of the present disclosure, wherein the user holds the device from a third angle. Figure 1 The object being held;

[0013] Figure 6 This is a schematic diagram of a theme park attraction system according to an embodiment of the present disclosure, wherein the user holds the device at a fourth angle. Figure 1 The object being held;

[0014] Figure 7 According to embodiments of this disclosure Figure 3-6 A diagram of a theme park attraction system;

[0015] Figure 8 It is used to determine according to embodiments of this disclosure. Figure 1A flowchart illustrating the process of a handheld object pointing towards a target;

[0016] Figure 9 According to embodiments of this disclosure Figure 1 A schematic diagram of a processor for a handheld object, which determines the orientation of the handheld object based on angular marker features; and

[0017] Figure 10 It is used to determine according to embodiments of this disclosure. Figure 1 A flowchart of the process of identifying the device identifier of a handheld object. Detailed Implementation

[0018] One or more specific embodiments will be described below. For the purpose of providing a concise description of these embodiments, not all features of the actual implementation are described in the specification. It should be noted that, as in any engineering or design project, numerous implementation-specific decisions must be made in the development of any such implementation to achieve specific goals that may vary from implementation to implementation, such as compliance with system-related constraints and business-related constraints. Furthermore, it should be appreciated that such development efforts may be complex and time-consuming, but for those skilled in the art who benefit from this disclosure, such development efforts will be nothing more than routine tasks of design, fabrication, and manufacturing.

[0019] When describing elements of various embodiments of this disclosure, the articles “a,” “an,” and “the” are intended to mean the presence of one or more of the elements. The terms “comprising,” “including,” and “having” are intended to be inclusive and mean that additional elements may be present in addition to those listed. Furthermore, it should be understood that references to “an embodiment” or “an embodiment” in this disclosure are not intended to be construed as excluding the existence of additional embodiments incorporating the described features.

[0020] This disclosure generally relates to a system including a handheld object for pointing to initiate interaction, and more particularly to using the system to determine the angle or identifier of the handheld object to facilitate the specification of appropriate interactions based on such characteristics of the handheld object. Determining the angle at which the handheld object is held facilitates the determination of the target to which the handheld object is pointed. This ensures that attractions output appropriate user interaction experiences. For example, a specific target among multiple targets can be accurately specified based on the pointing direction of the handheld object, and associated effects can be appropriately generated. Furthermore, attractions can output different user interaction experiences based on the visitor. Therefore, determining the identifier of the handheld object ensures that attractions output appropriate user interaction experiences associated with the visitor.

[0021] Specifically, the handheld object according to this disclosure may include a reference element that allows detection of characteristics of the handheld object, including relative angles and / or data associated with the handheld object. In embodiments, the reference element includes an angle marking element designed to be captured differently in an image depending on the relative viewpoint of the camera. For example, the angle marking element may generate a detected characteristic, such as a first color or pattern (e.g., blue, checked color, barcode), in a first captured image from a first viewpoint of the camera (e.g., when shot from the right side of the reference element), but generate a different detected characteristic, such as a second color or pattern (e.g., red, stripes, QR code), in a second captured image from a second viewpoint of the camera (e.g., when shot from the left side of the reference element). Therefore, based on the color or pattern generated by the angle marking element in the captured image, the orientation of the handheld object can be estimated. Combined with determining the position of the reference element, the target being pointed at by the handheld object can then be determined. Other aspects of the reference element may allow detection of data associated with the handheld object (e.g., user identification information), which can also be used to generate the desired effect. For example, images of a reference element captured from multiple angles can be combined to provide the system with identification information (e.g., a QR code).

[0022] The passive nature of the reference element avoids the overhead associated with including active elements (e.g., gyroscopes, communication devices) in the handheld object to transmit position information to the control system. In practice, the reference element can be simply operated to be observable based on self-illumination or external illumination. Therefore, active monitoring of motion data within the handheld object and the transmission of motion data from the handheld object can be avoided, which advantageously avoids the complexity and associated overhead for each handheld object. According to this disclosure, passive operation includes the operation of the reference element with or without self-illumination, and excludes any internal measurement functions of the reference element within the handheld object (e.g., operation of a gyroscope).

[0023] According to embodiments of this disclosure, a system may include a camera, a processor, and a memory coordinated and programmed to provide a response to certain detected characteristics associated with the use of a handheld object. As an example, a user may point the handheld object at an animated object at a location (e.g., a robot, or otherwise, an animated character), and, in response to determining that the pointed object is an animated object, the animated object may output a user interactive experience (e.g., wagging its tail). As another example, a user may point at words on a poster, and, in response to determining that the pointed object is a poster, a nearby speaker may output speech reciting the words from the poster. As yet another example, a user may point at an image of a person on an electronic display, and, in response to determining that the pointed object is an image of a person, the display may play a video showing the person in the image moving.

[0024] In some embodiments, the determined orientation of the handheld object can facilitate determining whether a user is waving the handheld object in a specific pattern (e.g., a number 8 pattern). For example, the orientation of the handheld object can be determined from multiple images captured by a camera. That is, based on the type of images captured at various locations, the orientation of the handheld object at each location can be derived by the system using a table or algorithm stored in memory that associates the orientation with the detected images. The orientation can be compared to a specific pattern. If the orientation roughly matches a specific pattern, the output device can output the user interaction experience.

[0025] As previously noted, the system according to this disclosure can also detect characteristics of the handheld object, such as data associations (e.g., the user identity of the handheld object). Therefore, the reference element may include a device identifier. For example, the device identifier may be in the form of a barcode or a quick response (QR) code. Although only a portion of the device identifier can be captured in an image, a camera (or multiple cameras) can capture multiple images of different portions of the device identifier and combine or stitch these portions together to form a complete device identifier. In response to identifying the device identifier, the output device can output a user interaction experience associated with the device identifier.

[0026] Through introduction, Figure 1 This is a schematic diagram of a handheld object 10 according to an embodiment of the present disclosure. The handheld object 10 may include a body 12, which may be of any suitable shape. As illustrated, the body 12 may be shaped as an axle or a staff; however, in other embodiments, the body 12 may include any shape that a user can point to using, for example, a projection device (such as a gun), a wearable item (such as a glove), or a wrist-worn device (such as a watch).

[0027] The main body 12 may include a recess 13 in which a reference element 14 is disposed. As illustrated, the reference element 14 may be disposed at the end 15 of the handheld object 10 and may provide an indication of where the handheld object 10 is positioned (e.g., when using image recognition technology). In particular, the reference element 14 may include a light source 16. A system or computing device attempting to position the handheld object 10 may use image recognition technology to detect light generated from the light source 16 and emitted from the reference element 14.

[0028] The light source 16 can be any suitable light-generating device that emits light along direction 18. For example, the light source 16 can be positioned at the distal end of the handheld object 10 and can even be collimated to facilitate specific guidance of the light emitted from the light source 16 and detection of where the handheld object 10 is pointing based on the detection of the light. As illustrated, the light source 16 can be a light bulb, such as a liquid crystal display (LCD), a light-emitting diode (LED), or an organic LED (OLED). The light source 16 can be battery-powered and / or rechargeable (e.g., by inserting the body 12 into a recharging station or by connecting a charging cable to the body). For this purpose, a power source 19 (e.g., a battery, a capacitor, a power harvesting circuitry system) can be included. In some embodiments, the light source 16 can be a wirelessly powered lamp (e.g., using ultra-high frequency (UHF) power harvesting). While the illustrated light source 16 emits light in the visible spectrum, in some embodiments, the light source 16 can emit light in the non-visible spectrum (e.g., the infrared or ultraviolet spectrum). In additional or alternative embodiments, the light source 16 may include a light reflecting device, such as a retroreflective material (e.g., a retroreflective sheet, a retroreflective fabric, a retroreflective glass bead, a microprism, an encapsulated lens sealed to a fabric or plastic substrate, and / or a metal strip). Accordingly, light entering the body 12 may be reflected by the light reflecting device, thereby causing reflected light to be emitted in direction 18.

[0029] The handheld object 10 may include an angle marker element 20 that allows light from the light source 16 to pass through it. The angle marker element 20 may include a filter or screen, such as a printed image (e.g., a pass-through filter). The angle marker element 20 may include multiple segments 22 (e.g., observable segments) that each cause different viewing light. For example, each segment 22 may include different color filters that cause light to be viewed as different corresponding colors (e.g., making the angle marker element 20 a color wheel), different pattern filters that cause light to be viewed as different corresponding patterns, or any other suitable means that causes light emitted from the reference element 14 to be distinguishably different from different viewpoints.

[0030] exist Figure 1In the diagram, sections 22A-D are illustrated as different patterns, each representing a different color filter. For example, section 22A could be a red filter that causes light from light source 16 to be perceived as red, section 22B could be a blue filter that causes light from light source 16 to be perceived as blue, section 22C could be a yellow filter that causes light from light source 16 to be perceived as yellow, and section 22D could be a green filter that causes light from light source 16 to be perceived as green. Accordingly, when viewed from a first viewpoint (e.g., closest to the red filter 22A), reference element 14 may appear to emit red light or be red in color. However, when viewed from a second viewpoint (e.g., closest to the blue filter 22B), reference element 14 may appear to emit blue light or be blue in color.

[0031] Although four segments 22A-D are shown in the angle marking element 20, it should be understood that any suitable number of segments 22 can be used in the angle marking element 20 to facilitate determining the angle at which the handheld object 10 is oriented. For example, Figure 2 It is according to an embodiment of the present disclosure having nine segments 22 Figure 1 A schematic diagram of the angle marking element 20 of the handheld object 10. As illustrated, segment 22A may be a red filter, segment 22B may be a blue filter, segment 22C may be a yellow filter, segment 22D may be a green filter, segment 22E may be an orange filter, segment 22F may be a purple filter, segment 22G may be a pink filter, segment 22H may be a brown filter, and segment 22I may be a gray filter. However, it is conceivable to have any suitable number of filters of any suitable color, pattern, or other form that cause light emitted from reference element 14 to be distinguishable from different viewpoints (e.g., visually distinguishable). Segments 22 may be in any suitable form. For example, segment 22 may be a highly reflective segment of material that reflects external light to facilitate detection by a monitoring system. As another example, segments 22 may each be colored light bulbs, which can facilitate detection by a monitoring system via coloring and / or operation (e.g., flashing). However, in Figure 1 In the illustrated embodiments, section 22 passes through a filter (e.g., semi-transparent or transparent), and the same applies to other embodiments.

[0032] The handheld object 10 may also include a device identifier 24 that allows light from the light source 16 to pass through. Like the angle marking element 20, the device identifier 24 may include a filter or screen, such as in the form of a printed image. The device identifier 24 may include any suitable identifier operable to associate data (e.g., user information, profile) with the handheld object 10, such as a barcode, quick response (QR) code, universal product code (UPC), serial number, product number, etc. Figure 1 The device identifier 24 is described as a printed QR code 26 that allows light from the light source 16 to pass through. However, as with section 22, the device identifier 24 can also take different forms (e.g., reflective material, bulb arrangement). It should be noted that the angular identification element 20 and the device identifier 24 can be a single feature. For example, the combination of section 22 and the device identifier 24 can include a multi-color QR code image that appears in different colors when viewed from different angles via the lens 28 of the reference element 14, and allows the combined views to be stitched together by a system processor to identify the QR code.

[0033] The handheld object 10 may include a lens 28 (e.g., a refractive lens) that disperses light from the light source 16 to facilitate the detection of the reference element 14 (e.g., to facilitate determining the position of the handheld object 10) and to facilitate the determination of the color of light emitted from the reference element 14. Specifically, the lens 28 may be positioned adjacent to the segment 22 and the device identifier 24 (which may be combined) such that light is refracted in a manner that allows different portions of the segment 22 / device identifier 24 to be viewed from different angles. As understood in the art, the lens 28 may include various shapes for this purpose (e.g., prismatic or spherical). As illustrated, the lens 28 has a spherical shape and is made of glass, thereby amplifying the light emitted from the light source 16 and / or showing a magnified view of the segment 22 of the angle identifier element 20, which is visible to the viewer (e.g., in their line of sight). However, any suitable shape or material of lens 28 is contemplated to facilitate the determination of the color of light emitted from the reference element 14 when viewing the handheld object 10. Lens 28 can represent a hemisphere, a triangular prism, etc., and can be made of plastic instead of glass. Furthermore, in some embodiments, a set of baffles or other devices may be used to aim light in certain directions (e.g., to indicate the angle at which the handheld object 10 is being held). It should be further noted that in other embodiments, there is no light source 16, and light is both received into and dispersed from lens 28 to facilitate the detection of aspects of reference element 14 that can be correlated with positioning and / or associated data.

[0034] In some embodiments, the handheld object 10 may include one or more gripping features 30, such as one or more recessed finger indentations 30A and / or ridges 30B. The gripping features 30 may guide the user to grip the handheld object 10 in a specific manner. For example, the gripping features 30 may ensure that the user grips the handheld object 10 in a constant orientation, preventing the angle indicator element 20 from rotating while the user is holding or using the handheld object 10. The gripping features 30 may include any additional features or other suitable features that prevent the handheld object 10 from rotating relative to the user's gripping orientation when held by the user, such as finger guards (e.g., to ensure that the user inserts their index finger into the guard, and thus is less likely to rotate the handheld object 10). This can facilitate proper assessment of motion based on the detection pattern and associated image of the reference element 14.

[0035] In additional or alternative embodiments, the user can access the controller or control system (e.g., in...) Figure 7 The controller 90 (described in the text and discussed in more detail below) performs a calibration process to determine how the user is holding the handheld object 10 (e.g., the initial orientation of the handheld object 10 when held by the user) such that the reference element 14 is emitting light corresponding to a corresponding segment 22 at a certain position (e.g., emitting light by reflecting external light or emitting light from an internal light source). For example, a camera communicatively coupled to the controller can perceive the reference element 14 as having a blue color because the blue filter 22B is visible to the camera via the lens 28 and / or in the camera's line of sight. Further detection of the color of the reference element 14 can indicate the angle of the handheld object 10, at least based on the calibration position. That is, if the camera then captures an image of the reference element 14 as having a red color due to the red filter 22A being visible to the camera via the lens 28, the controller can determine the angle resulting from the position of the red filter 22A relative to the blue filter 22B on the angle identification element 20.

[0036] Because the components of the handheld object 10 (e.g., angle marker element 20, device identifier 24, lens 28) are relatively simple (e.g., color filter, printed barcode, glass ball), the handheld object 10 can be relatively easy to mass-produce. Furthermore, because the components are relatively small (e.g., each can have a diameter of less than 3 cm, less than 2 cm, less than 1 cm, less than 0.5 cm, or less than 0.3 cm), the reference element 14 can have low visual impact. That is, the reference element 14 can not detract from the user experience of using the handheld object 10. Moreover, if the light source 16 emits or reflects light invisible to the human eye (e.g., infrared light, ultraviolet light), the reference element 14 can have even lower visual impact.

[0037] Considering this, Figure 3This is a schematic diagram of a theme park attraction or entertainment system 40 according to an embodiment of the present disclosure, wherein a user 42 holds a handheld object 10 at a first angle. The theme park attraction system 40 enables the user 42 to point the handheld object 10 at certain targets or move the handheld object 10 to perform certain gestures or follow certain patterns, and outputs a user interactive experience in response. For example, the theme park attraction system 40 may include settings with characters popular with children, settings themed around television or movies, photo galleries, target sets, etc.

[0038] The theme park attraction system 40 may include a camera 44 or other image capture device for capturing images of a handheld object 10. A controller or control system (such as...) Figure 7 The controller 90, described herein and discussed in more detail below, can be communicatively coupled to the camera 44 and determine the position or location of the reference element 14 of the handheld object 10 (e.g., based on captured images of the handheld object 10). The controller can determine the position of the reference element 14 in a two-dimensional plane or in three dimensions. Furthermore, the controller can determine the pointing angle of the handheld object 10 based on the color detected from the reference element 14. In some embodiments, the locations of the camera 44 and the reference element 14, or components of the reference element 14, can be reversed. That is, the camera 44 can be attached or mounted to the handheld object 10, while the reference element 14, the angle marking element 20, the device identifier 24, and / or the lens 28 can be positioned in a fixed location away from the handheld object 10.

[0039] To determine the position of reference element 14 in three dimensions, theme park attraction system 40 may include a second camera (also represented by element 44) for capturing images of the handheld object 10, which may provide a depth dimension (e.g., z-axis) to a two-dimensional plane (e.g., the xy-axis). In additional or alternative embodiments, reference element 14 may include a second lens and a second angle marker element (also represented by elements 28 and 20). In this way, theme park attraction system 40 may include a single camera 44, but is still able to determine the position of reference element 14 in three dimensions by tracking a third axis of rotation based on two lenses and two angle marker elements.

[0040] As explained, based on the camera's viewpoint, reference element 14 appears to emit green light 46. This is due to the positioning of the green filter 22D relative to the line of sight of camera 44, the refraction via the viewing angle of lens 28 relative to camera 44, and the light emitted by the light source 16 of the handheld object 10 through the green filter 22D. This is related to the orientation of the handheld object 10 relative to camera 44, and this correlation can be determined by the processor of the controller of the park attraction system 40 (e.g., in...). Figure 7The processor 92 described herein and discussed in more detail below is provided, which employs memory stored in the park attraction system 40 (such as, in...). Figure 7 The algorithm or table in memory 94 (described herein and discussed in more detail below) is used. Therefore, based on the color of reference element 14 and / or the determined position of reference element 14, the controller can determine the angle at which user 42 is holding the handheld object 10. For example, the detection of green light 46 can be correlated with the user pointing the handheld object 10 upwards.

[0041] The image captured by camera 44 may also include a first portion 48 of device identifier 24. However, the controller may not be able to read device identifier 24 because, due to the refractive properties of lens 28, only the first portion 48 is captured. That is, since lens 28 can amplify light emitted from reference element 14, when viewed through lens 28, lens 28 can also amplify or magnify device identifier 24, causing camera 44 to capture only the first portion 48 of the image. Accordingly, the controller may store the first portion 48 of device identifier 24 in memory to combine with other portions of device identifier 24 until a complete device identifier 24 can be generated. For example, the memory and processor may be programmed to stitch together images of device identifier 24 based on the correlation of overlapping images to provide a unified image of device identifier 24.

[0042] As described, the theme park attraction system 40 may include one or more output devices 50, such as animated characters, electronic displays 50A, or speakers 50B. A controller (such as...) Figure 7The controller 90, as described herein and discussed in more detail below, can instruct the output device 50 to output a user interactive experience (e.g., a certain movement or actuation, image, video, or audio data) based on the determined position of the handheld object 10, the determined angle of the handheld object 10, and / or the determined device identifier 24. For example, if the controller determines that the handheld object 10 is pointing at an animated object (e.g., a robot, or in other cases, an animated character), the controller can instruct the animated object to output a user interactive experience (e.g., wagging its tail). As another example, if the controller determines that the handheld object 10 is pointing at a word on a poster, the controller can instruct the speaker 50B to output the spoken word. As yet another example, if the controller determines that the handheld object 10 is pointing at an image of a person on a display 50A, the controller can instruct the display 50A to play a video showing the person in the image moving. If the controller determines that the handheld object 10 has moved in a certain pattern or sequence (e.g., based on determining the position of the reference element 14 and / or determining the angle of the handheld object 10 being held, as captured by the camera 44 in multiple images), the controller can instruct the output device 50 to output a user interaction experience (e.g., certain video and audio data) associated with a certain pattern or sequence (e.g., the number 8). Once the controller determines the (complete) device identifier 24, the controller can instruct the output device 50 to output a user interaction experience associated with the device identifier 24 (e.g., instructing the speaker 50B to say the user's name).

[0043] Figure 4 This is a schematic diagram of a theme park attraction system 40 according to an embodiment of the present disclosure, wherein a user 42 holds a handheld object 10 at a second angle. As illustrated, based on the camera's viewpoint, since the red filter 22A is visible to and / or within the line of sight of the camera 44, and the light source 16 of the handheld object 10 emits light through the red filter 22A, the reference element 14 appears to emit red light 60. Therefore, based on the color of the reference element 14 and / or the determined position of the reference element 14, the controller (e.g., in...) Figure 7 The controller 90, as described in the text and discussed in more detail below, can determine the angle at which the user 42 is holding the handheld object 10 (e.g., pointing the handheld object 10 downwards).

[0044] The image captured by camera 44 may also include the second part 62 of device identifier 24. If the controller is able to make the image from... Figure 3If the captured images of the first portion 48 and the second portion 62 of the device identifier 24 are combined to complete the device identifier 24, the controller may be able to read the complete device identifier 24 and use it to identify relevant data. If not, the controller may store the second portion 62 of the device identifier 24 in memory to combine with other portions of the device identifier 24 until a complete device identifier 24 can be generated. Specifically, the controller may identify overlapping sections between portions of the device identifier 24 to determine if portions of the device identifier 24 fit together and combine those portions to store in memory. The controller may also identify the ends of the device identifier 24 within the portions of the device identifier 24, and determine that the complete device identifier 24 has been combined when the combined portion of the device identifier 24 includes the ends of the device identifier 24.

[0045] Figure 5 This is a schematic diagram of a theme park attraction system 40 according to an embodiment of the present disclosure, wherein a user 42 holds a handheld object 10 at a third angle. As illustrated, based on the camera's viewpoint, since the blue filter 22B is visible to and / or within the line of sight of the camera 44, and the light source 16 of the handheld object 10 emits light through the blue filter 22B, the reference element 14 appears to emit blue light 70. Therefore, based on the color of the reference element 14 and / or the determined position of the reference element 14, the controller can determine the angle at which the user 42 is holding the handheld object 10 (e.g., pointing the handheld object 10 to the left from the user's viewpoint).

[0046] The image captured by camera 44 may also include the third part 72 of device identifier 24. If the controller is able to... Figure 3 The first part 48 of the device identifier 24, from Figure 4 If the images of the second part 62 and the third part 72 are combined to provide a complete image of the device identifier 24, the controller may be able to read the complete device identifier 24. If not, the controller may store the image of the third part 72 of the device identifier 24 in memory to combine with other image parts of the device identifier 24 until an image of the complete device identifier 24 can be generated.

[0047] Figure 6This is a schematic diagram of a theme park attraction system 40 according to an embodiment of the present disclosure, wherein a user 42 holds a handheld object 10 at a fourth angle. As illustrated, based on the camera's viewpoint, since the yellow filter 22C is visible to and / or within the line of sight of the camera 44, and the light source 16 of the handheld object 10 emits light through the yellow filter 22C, the reference element 14 appears to emit yellow light 80. Therefore, based on the color of the reference element 14 and / or the determined position of the reference element 14, the controller (e.g., in...) Figure 7 The controller 90, as described in the text and discussed in more detail below, can determine the angle at which the user 42 is holding the handheld object 10 (e.g., pointing the handheld object 10 to the right from the user's viewpoint).

[0048] The image captured by camera 44 may also include the fourth part 82 of device identifier 24. If the controller is able to... Figure 3 The first part 48 of the device identifier 24, from Figure 4 Part 2, 62, from Figure 5 If the images of the third part 72 and the fourth part 82 are combined to complete the image of the complete device identifier 24, the controller may be able to read the complete device identifier 24. If not, the controller may store the image of the fourth part 82 of the device identifier 24 in memory to combine with other images of the parts of the device identifier 24 until the complete device identifier 24 can be generated.

[0049] Figure 7 According to embodiments of this disclosure Figure 3-6 A block diagram of the theme park attraction system 40. As illustrated, a camera 44 communicatively coupled to a controller 90 can capture an image of a handheld object 10. The image may include the color of a reference element 14, as detected by the camera 44 via transmission through a lens 28. This color may correspond to a segment 22 of an angle-identifying element 20 through which light is emitted by a light source 16, or through which light is positively reflected, and which is directed toward the camera 44 by refraction via the lens 28. The image may also include a portion of a device identifier 24, which may be detected in a similar manner or as part of the color detection.

[0050] The controller 90 may include one or more processors (described herein and referred to as a single processor 92) and one or more memories or storage devices (described herein and referred to as a single memory device 94). The processor 92 may execute software programs and / or instructions stored in the memory device 94, the software programs and / or instructions facilitating the determination of the position of the handheld object 10 and / or reference element 14, the determination of the color of the reference element 14, and the determination of the angle of the handheld object 10 and / or device identifier 24. Furthermore, the processor 92 may include multiple microprocessors, one or more "general-purpose" microprocessors, one or more application-specific microprocessors, and / or one or more application-specific integrated circuits (ASICs). For example, the processor 92 may include one or more Reduced Instruction Set Computer (RISC) processors. The memory device 94 may store information such as control software, lookup tables, configuration data, etc. The memory device 94 may include tangible, non-transitory machine-readable media, such as volatile memory (e.g., random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM)), flash memory, one or more hard disk drives, and / or any other suitable optical, magnetic, or solid-state storage media. The memory device 94 can store a wide variety of information and can be used for a variety of purposes, such as facilitating the determination of the position of the handheld object 10 and / or the reference element 14, the determination of the color of the reference element 14, and the determination of the angle of the handheld object 10 and / or the device identifier 24.

[0051] Specifically, processor 92 can implement image recognition technology stored in memory device 94 to detect reference element 14, the color of reference element 14, and / or device identifier 24 in an image of the handheld object 10 captured by camera 44. Processor 92 can then determine the angle of the handheld object 10 based on, for example, the position and / or color of reference element 14. This may include: comparing the detected features (e.g., coloring, pattern) with an information table; or running an algorithm based on the detected features to identify relevant positioning and identification information of the handheld object 10.

[0052] The controller 90 can also be communicatively coupled to the output device 50 and instruct the output device 50 (e.g., an animated character, electronic display, speaker) to output motion, images, video, audio data, etc. The controller 90 can be communicatively coupled to the camera 44 and / or the output device 50 by any suitable means, such as via wired communication or via a communication network using wireless communication protocols or technologies (e.g., radio, Bluetooth, WiFi, infrared, Ethernet, Thread, ZigBee, Z-Wave, KNX, mobile and / or microwave).

[0053] Considering the preceding text, Figure 8This is a flowchart of a process 110 for determining a target of a handheld object 10 according to embodiments of the present disclosure. Process 110 can be implemented by any suitable system capable of determining a target of a handheld object 10 (such as any component of a theme park attraction system 40), including a camera 44, a controller 90, a processor 92, and / or an output device 50. While process 110 is described using steps performed in a specific sequence, it should be understood that the present disclosure contemplates that the described steps may be performed in a different sequence than described, and that some described steps may be skipped or not performed in full. In some embodiments, process 110 can be implemented by using a processor (such as processor 92) to execute instructions stored in a tangible, non-transitory computer-readable medium (such as memory device 94).

[0054] As illustrated, in process block 112, processor 92 receives an image of the handheld object 10. Specifically, camera 44 can capture an image of the handheld object 10 (e.g., as held by user 42) and specifically send the image to controller 90 and processor 92. Processor 92, which may reside in controller 90, can thus receive the image (e.g., data indicating the captured image).

[0055] In process block 114, processor 92 identifies reference element 14 of the handheld object 10 in the image. For example, processor 92 may use image recognition technology (e.g., stored as instructions in memory device 94) to detect observable characteristics of reference element 14 (e.g., color, markings, lens shape). The detected characteristics may be used to identify reference element 14 and / or determine the relative position of handheld object 10.

[0056] In process block 116, processor 92 determines the position of reference element 14 or handheld object 10. Processor 92 can determine the position of reference element 14 or handheld object 10 in a two-dimensional plane (e.g., the plane of an image) or in three dimensions. This can be done by processor 92 implementing image recognition techniques or algorithms (including machine learning, artificial intelligence, deep learning, convolutional neural networks, etc.). For example, memory 94 can store an image recognition model that can be trained by inputting sample images of reference element 14 or handheld object 10 and indications of where reference element 14 or handheld object 10 is located in the sample images. Processor 92 can then use the trained image recognition model to determine reference element 14 or handheld object 10 in an image and determine the position of reference element 14 or handheld object 10 in the image (e.g., the pixels of the image corresponding to reference element 14 or handheld object 10).

[0057] In process block 118, processor 92 detects the visibility characteristics of the angle marker features of reference element 14 in the image. Specifically, processor 92 can detect the color of light emitted from reference element 14, which corresponds to the color of light filtered, such as by the color filter 22 of angle marker element 20. Then, in process block 120, the detected characteristics can be used, wherein processor 92 determines the orientation of the held object 10 based on the angle marker features. Specifically, processor 92 can estimate the orientation of the held object 10 based on the color of light emitted by reference element 14. For example, Figure 9 This is a schematic diagram of a processor 92 according to an embodiment of the present disclosure, which determines the orientation of a handheld object 10 based on an angle marker feature 130. The processor 92 maps the position 132 of the reference element 14 on a two-dimensional plane 134, as determined in process block 116; however, in some embodiments, the position 132 may be mapped in three dimensions. Coordinate axis 136 is illustrated for reference. As illustrated, the processor 92 detects that the angle marker feature 130 is blue (e.g., due to the color of light passing through the blue filter 22B and emitted from the reference element 14). Based on the position of the blue filter 22B on the angle marker element 20, the processor 92 may, for example, determine the orientation 138 of the handheld object 10 by using a table or algorithm that correlates such data. For example, the processor 92 may use an image recognition model stored in memory 94, which can be trained by inputting sample images of the reference element 14 and indications of the orientation of the reference element 14 in the sample images. The processor 92 may use the trained image recognition model to determine the orientation 138 of the reference element 14. In additional or alternative embodiments, the processor 92 may use any other suitable image recognition technology or algorithm, including machine learning, artificial intelligence, deep learning, convolutional neural networks, etc.

[0058] In some embodiments, the processor 92 can be based on how the user holds the handheld object 10 (e.g., using...). Figure 1 The orientation 10 of the handheld object 10 is determined by the grip feature 30 shown (to establish a baseline orientation), calibration process, estimated or known user height, user position or location, etc. For example, if the user height is estimated or known, the end position of the handheld object 10 (relative to reference element 14) can be estimated based on the user height (e.g., the end position may point towards the center of the user), and orientation 138 can be inferred. Similarly, if the user position or location is known, the end position of the handheld object 10 can be estimated based on the user position, and orientation 138 can be inferred. In some embodiments, the processor 92 can determine the angle 140 of the handheld object 10. For example, the angle 140 of the handheld object 10 relative to the y-axis of coordinate axis 136 is approximately 90 degrees.

[0059] In process block 122, processor 92 determines the pointing target of handheld object 10 based on the orientation of handheld object 10 and the position of reference element 14. The pointing target may include an image, object, portion of an image, etc., where the user is pointing handheld object 10. Specifically, processor 92 may extend the direction 142 of the orientation of handheld object 10 from the position of reference element 14 and determine the pointing target 144 by estimating the endpoint of direction 142.

[0060] In decision box 124, if target 144 is associated with user interaction experience, then in process box 126, processor 92 instructs output device 50 to implement user interaction experience. For example... Figure 9 As described herein, a graphic object 146 is provided (e.g., on a wall) or displayed (e.g., on an electronic display 50A). Since a pointing target 144 is on or in line with the graphic object 146, the processor 92 can instruct the speaker 50B to output audio data (e.g., associated with the graphic object), instruct the display 50A to cause movement of the graphic object 146 (e.g., by playing video), instruct the animation object to move, and so on. If the pointing target 144 is not associated with the user interaction experience, the processor 92 repeats process 110. In this way, process 110 enables the processor 92 to determine the pointing target 144 of the handheld object 10 without the overhead or complexity associated with including active elements (e.g., a gyroscope, a communication device) in the handheld object 10 to send position information to the controller 90.

[0061] In some embodiments, processor 92 can guide output device 50 to perform user interaction experiences based on one or more orientations of the handheld object. For example, memory 94 can store certain patterns (e.g., a zigzag pattern, a number 8 pattern, a letter or number pattern), and processor 92 can determine whether the user is moving the handheld object 10 in a certain pattern. This can be done by processor 92 implementing image recognition techniques or algorithms (including machine learning, artificial intelligence, deep learning, convolutional neural networks, etc.). For example, memory 94 can store an image recognition model that can be trained by inputting sample images of a reference element 14 moving or "drawing" in a certain pattern. Processor 92 can then use the trained image recognition model to determine whether the reference element 14 is moving in a certain pattern.

[0062] The processor 92 can also determine the device identifier 24 of the handheld object 10, which can uniquely identify the handheld object 10, associate the handheld object 10 with a user, and / or associate the handheld object 10 with a user profile. Figure 10This is a flowchart of a process 160 for determining a device identifier 24 for a handheld object 10 according to embodiments of the present disclosure. Process 160 can be implemented by any suitable device capable of determining the device identifier 24 for the handheld object 10 (such as any component of a theme park attraction system 40), including a camera 44, a controller 90, a processor 92, and / or an output device 50. While process 160 is described using steps performed in a specific sequence, it should be understood that the present disclosure contemplates that the described steps may be performed in a different sequence than described, and that some described steps may be skipped or not performed in full. In some embodiments, process 160 can be implemented by using a processor (such as processor 92) to execute instructions stored in a tangible, non-transitory computer-readable medium (such as memory device 94).

[0063] As described, in process block 162, processor 92 receives an image of the handheld object 10. Specifically, after camera 44 has captured an image of the handheld object 10 and transmitted the image to processor 92, processor 92, located in controller 90, can receive the image from camera 44. Using the received image, in process block 164, processor 92 determines at least a portion of device identifier 24. Because the lens 28 of reference element 14 can adjust the light used to detect device identifier 24 (e.g., light emitted from light source 16), a limited portion of device identifier 24 can be distinguished from a given image. For example, when viewed through lens 28, lens 28 can magnify or enlarge device identifier 24, causing camera 44 to capture only a portion of device identifier 24 in the image. Specifically, for example, as Figure 3 As described herein, camera 44 may capture only the first portion 48 of the image.

[0064] In process block 166, processor 92 combines an image of a portion of device identifier 24 with any other image of a corresponding portion of device identifier 24. For example, processor 92 may store other images of portions of device identifier 24 determined from previously captured images in memory or a storage device (such as memory device 94). Accordingly, processor 92 may combine or stitch together portions of device identifier 24 (e.g., first portion 48) with any stored portions. Processor 92 may use any suitable image combining or forming technique or algorithm to combine the portions together. In particular, processor 92 may identify overlapping segments between portions of device identifier 24 to determine that portions of device identifier 24 fit together, and combine those portions for storage in memory 94.

[0065] In decision box 168, processor 92 determines whether the combined image of the portions of device identifier 24 forms an image of the complete device identifier 24 (e.g., an image sufficient to read device identifier 24 and associate device identifier 24 with all relevant information). In particular, processor 92 may identify the end of device identifier 24 in the portions of device identifier 24, and determines that the complete device identifier 24 has been combined when the combined portion of device identifier 24 includes the end of device identifier 24.

[0066] If the processor 92 does not determine that the combined portions of the device identifier 24 form a complete image of the device identifier 24, then the image of the device identifier 24 is incomplete and therefore cannot be read by the processor 92, and in process block 170, the processor 92 stores a portion of the device identifier 24 (e.g., in memory device 94) and repeats process 160 by receiving subsequent images of the handheld object 10.

[0067] If the processor 92 determines that the combined portions of the device identifier 24 form a complete image of the device identifier 24, then in process block 172, the processor 92 reads the device identifier 24 and uses the device identifier 24 to identify the handheld object 10. In particular, the memory 94 may store the correlation between the device identifier 24 and identification data (e.g., identification number) for the handheld object 10, the user, and / or the user profile.

[0068] In decision box 174, if device identifier 24 is associated with a user interaction experience, then in process box 176, processor 92 instructs output device 50 to implement the user interaction experience. For example, device identifier 24 may be associated with a user profile. Accordingly, processor 92 may query a database (e.g., stored in memory device 94) for the user's name (which may be a field in the user profile) and instruct speaker 50B to say the user's name. If device identifier 24 is not associated with a user interaction experience, processor 92 repeats process 160.

[0069] In this way, process 160 enables processor 92 to determine the device identifier 24 of the handheld object 10 without the overhead or complexity associated with including active elements (e.g., communication devices) in the handheld object 10 to send identification information to controller 90. In some embodiments, processes 110 and 160 can be combined. That is, by implementing processes 110 and 160, processor 92 can determine both the target of the handheld object 10 and the device identifier 24 of the handheld object 10.

[0070] While the embodiments set forth in this disclosure may be susceptible to various modifications and alternatives, specific embodiments have been shown by way of example in the accompanying drawings and have been described in detail herein. However, it should be understood that this disclosure is not intended to be limited to the particular forms disclosed. This disclosure will cover all modifications, equivalents, and alternatives falling within the spirit and scope of this disclosure as defined by the following appended claims.

[0071] The techniques proposed and claimed herein are referenced and applied to substantial objects and specific examples of practical nature that can arguably improve the technical field and are therefore not abstract, intangible, or purely theoretical. Furthermore, if any claim appended to this specification contains one or more elements designated as “component for [implementing]…[function]” or “step for [implementing]…[function]”, such elements are intended to be interpreted according to 35 USC 112(f). However, for any claim containing elements designated in any other way, such elements are not intended to be interpreted according to 35 USC 112(f).

Claims

1. An entertainment system, comprising: A camera configured to capture an image of a handheld object including a reference element of the handheld object, the reference element including an angle identification element and a device identifier; as well as The controller has a processor and a memory, the memory storing machine-readable instructions configured to cause the processor to: The image including at least a portion of the device identifier is combined with a previously stored image of a portion of the device identifier to form a combined image including the complete device identifier; Based on the complete device identifier in the combined image, determine the user interaction experience associated with the device identifier; The angle identification element of the handheld object is identified in the image including the reference element of the handheld object; as well as The orientation of the handheld object is determined based on the detected characteristics of the angle marking element in the image including the reference element of the handheld object.

2. The entertainment system according to claim 1, wherein, The processor is configured to determine the position of the handheld object in the combined image, and to determine the target to which the handheld object is pointing based on the position and orientation of the handheld object.

3. The entertainment system according to claim 1, comprising the handheld object, wherein, The angle marking element includes a plurality of observable segments, the handheld object includes a refractive lens positioned on the plurality of observable segments, and wherein light emitted through or reflected from each of the plurality of observable segments is distinguishable relative to light emitted through or reflected from other of the plurality of observable segments.

4. The entertainment system according to claim 3, wherein, The handheld object includes a light source, wherein each of the plurality of observable segments includes a filter.

5. The entertainment system according to claim 3, wherein, The device identifier is positioned adjacent to a lens, which is configured to enable the camera to capture a second image of at least a portion of the device identifier.

6. The entertainment system according to claim 1, comprising an output device configured to output a user interactive experience, wherein, The memory stores machine-readable instructions configured to cause the processor to read the combined image and, based on the combined image, instruct the output device to output the user interaction experience.

7. The entertainment system according to claim 1, wherein, The memory stores the correlation between the device identifier and identification data used for the handheld object, user, user profile, or any combination thereof.

8. The entertainment system according to claim 1, wherein, The device identifier includes a barcode, a quick response (QR) code, a universal product code (UPC), a serial number, a product number, or any combination thereof.

9. The entertainment system according to claim 1, comprising an output device configured to output a user interactive experience, wherein, The memory stores machine-readable instructions configured to cause the processor to instruct the output device to output the user interaction experience based on the orientation of the handheld object.

10. The entertainment system according to claim 1, wherein, The angle marking element includes multiple segments, and each of the multiple segments is visually distinguishable relative to the other segments.

11. The entertainment system according to claim 10, wherein, The device identifier is provided by a combination of the plurality of segments.

12. A handheld object configured to facilitate the detection of the orientation of the handheld object by a monitoring system, the handheld object comprising: The main body, configured to be held in hand; An angular identifying element comprising a plurality of segments, wherein each of the plurality of segments is visually distinguishable relative to the other segments, wherein the plurality of segments are combined to provide a device identifier; and A lens, positioned adjacent to the angle marker element, makes one or more of the plurality of segments visible to a camera at a specific location relative to the angle marker element via the lens, so that the monitoring system can determine the orientation of the handheld object through the one or more segments visible to the camera, and makes at least a portion of the device identifier visible to the camera, so that the monitoring system can determine the user interaction experience associated with the device identifier based on the complete device identifier by combining an image from the camera including at least a portion of the device identifier with a previously stored image of a portion of the device identifier to form a composite image including the complete device identifier.

13. The handheld object according to claim 12, comprising a light source, wherein, The light source is configured to emit light through each of the plurality of segments and through the lens.

14. The handheld object according to claim 13, wherein, The light source includes a liquid crystal display, a light-emitting diode, or an organic light-emitting diode.

15. The handheld object according to claim 12, wherein, The device identifier includes a barcode, a quick response (QR) code, a universal product code (UPC), a serial number, a product number, or any combination thereof.

16. The handheld object of claim 12, including a gripping feature configured to guide a user to hold the handheld object in a particular manner.

17. The handheld object according to claim 12, wherein, The multiple sections are made of reflective material.

18. The handheld object according to claim 12, wherein, The lens is configured to amplify light emitted through or reflected by the plurality of segments.

19. One or more non-transitory computer-readable media storing instructions, which, when executed by at least one processor, cause the at least one processor to perform operations including: Receive an image of the handheld object; Reference elements identifying the handheld object in the image, wherein, The reference element includes an angle identification element and a device identifier; The position of the handheld object is determined based on the image; The image including at least a portion of the device identifier is combined with a previously stored image of a portion of the device identifier to form a combined image including the complete device identifier; Detect the angle identification features of the reference element in the image of the handheld object; Based on the complete device identifier in the combined image, determine the user interaction experience associated with the device identifier; Based on the characteristics of the angle identification features, the orientation of the handheld object is determined; as well as Based on the position and orientation of the handheld object, the target to which the handheld object is pointing is determined.

20. One or more non-transitory computer-readable media according to claim 19, wherein, The instruction causes the at least one processor to perform an operation including: performing a calibration process to determine the initial orientation of the handheld object based on the characteristics of the angular identification features of the reference element in the combined image.

Images