Amusement park attraction system and method

Abstract

The projection target position of the handheld object is determined based on applying a translation coefficient, a scaling coefficient, and an offset to the position of a reference element of the handheld object detected by the camera on a two-dimensional plane. The translation coefficient is determined based on the difference between a calibration position on the plane and an initial position of the reference element corresponding to the calibration position, and serves to move the position of the reference element to generate the projection target position. The scaling coefficient is determined based on an estimated length of the user's arm holding the handheld object, and serves to scale the position of the reference element to generate the projection target position. The offset is determined based on a polynomial and serves to expand the distance between the projection target position and the calibration position.

Description

[Technical Field]

[0001] (Cross-reference with related applications) This application claims the benefit of U.S. Provisional Application No. 62 / 905,901, filed September 25, 2019, "Systems and Methods for Determining Projected Target Location of a Handheld Object," the entire disclosure of which is incorporated herein by reference.

[0002] (Technical field) This disclosure relates to a handheld object used for pointing in general, and more specifically, to determining the projection target position of a handheld object. [Background technology]

[0003] This section is intended to introduce to the reader various aspects that may relate to the various aspects of the disclosure described below. This discussion is intended to help the reader to better understand the various aspects of the disclosure. Therefore, please understand that this description should be read from this perspective and not as an admission of prior art.

[0004] A handheld object can be used to point at or select a target. For example, in an amusement park setting, a visitor can use a handheld object to point at an animated character in an attraction, and in response to detection, the system can cause the animated character to output a user interaction experience (e.g., wagging its tail). However, certain physical characteristics of the user's body may make it difficult to accurately determine whether the user is pointing at a target.

[0005] These and other features, aspects, and advantages of this disclosure will be better understood by reading the following detailed description with reference to the attached drawings, in which similar reference numerals represent similar elements. [Brief explanation of the drawing]

[0006] [Figure 1] This figure shows a user pointing a handheld object at a target according to an embodiment of the present disclosure. [Figure 2] This is a block diagram of an amusement park attraction system according to an embodiment of the present disclosure. [Figure 3] This figure shows a user orienting a handheld object to a calibration position according to an embodiment of the present disclosure. [Figure 4] This figure shows an example of applying one or more translation coefficients to the subsequent detection positions of the reference element of the handheld object in Figure 3, according to an embodiment of the present disclosure. [Figure 5] This figure shows an example of applying a scaling factor to a subsequent detection position of the reference element of the handheld object in Figure 3, according to an embodiment of the present disclosure. [Figure 6] This figure shows a user pointing a handheld object at different targets in the system according to an embodiment of the present disclosure. [Figure 7] This figure shows multiple reference element zones of different sizes and multiple projection target zones of a fixed size according to embodiments of the present disclosure. [Figure 8] This figure shows a plurality of reference element zones of a fixed size and a plurality of projection target zones of different sizes according to an embodiment of the present disclosure. [Figure 9] This is a flowchart of the process for determining the projection target position of the handheld object shown in Figure 3, according to an embodiment of the present disclosure. [Figure 10] This is a flowchart of a process for correcting distortion caused by the difference in shape between the arc-like nature of a user's arm movement and a two-dimensional plane, according to embodiments of the present disclosure. [Modes for carrying out the invention]

[0007] One or more specific embodiments of this disclosure are described below. Not all features of actual implementations can be described herein in order to provide a concise description of these embodiments. It should be recognized that, as with any industrial design or design project, the development of any such implementation will require numerous implementation-specific decisions to achieve the specific goals of the developers, including compliance with system-related and business-related constraints, which may vary between implementations. Furthermore, it should be recognized that while such development work can be complex and time-consuming, it will still be part of the routine design, fabrication, and manufacturing work for those skilled in the art who benefit from this disclosure.

[0008] When describing elements of the various embodiments of this disclosure, the articles “a,” “an,” and “the” are intended to indicate that there are one or more elements. The terms “equipped with,” “contains,” and “have” are intended to be comprehensive and to indicate that there may be further elements other than those described. In addition, it should be understood that any reference in this disclosure to “one embodiment” or “one embodiment” is not intended to be construed as excluding the existence of additional embodiments that also incorporate the described features.

[0009] This disclosure generally relates to handheld objects used for pointing, and more specifically to determining the projection target position of a handheld object. Specifically, a reference element can provide indication of where the handheld object is pointing. For example, in an amusement park setting, a user can use the handheld object to point at an animated object on an attraction (such as a robot or animated character), and in response to the detection of the reference element's position, the animated object can output a user interaction experience (e.g., wagging its tail). As another example, a user can point at a word on a poster, and in response to the detection of the reference element's position, a nearby speaker can output an audio of the word being spoken. As yet another example, a user can point at an image of a person on an electronic display, and in response to the detection of the reference element's position, a video can be played showing the person in the image moving.

[0010] The systems and methods disclosed herein include using a camera to determine the position of a reference element on a two-dimensional plane perpendicular to the direction of the camera. The camera can detect the reference element of a handheld object, which can be made of a material readily detectable by the camera (e.g., a back-reflective material). The position of the reference element can be used to determine the target position to which the user was pointing the handheld object. However, in some systems, the user's perception of where the handheld object is pointed may not coincide with the projected position of the location the user is pointing, which is determined based on the camera's field of view. This may be due to a variety of factors, including whether one eye is dominant (e.g., right or left eye), head tilt, weight shift, and tilt to one or the other. Any combination of these factors may cause the user's perception of where the handheld object is pointed to change, even if the user's hand is pointing the handheld object to the same location. The camera is just one example of the various photodetectors that can be used by this embodiment. Accordingly, references to the camera are representative of other photodetectors that may be used by embodiments of this disclosure.

[0011] The systems and methods disclosed herein include providing calibration points on a two-dimensional plane that a user can direct a handheld object towards. The position of a reference element relative to the two-dimensional plane can be determined as an initial position, and one or more translation factors can be determined based on the difference between the initial position and the calibration points. That is, the calibration points can correlate to where the user perceives the handheld object to be directed, while the initial position of the reference element can correlate to the position of the reference element on the two-dimensional plane from the perspective of the camera. The difference between these two can be used to translate the position of subsequent reference elements detected on the two-dimensional plane from the perspective of the camera to a projection target position (e.g., corresponding to where the user perceives to be directed or attempting to direct). That is, one or more translation factors can correct the difference between the user's perception of where the handheld object is directed and the camera's determination (judgment) of where the reference element is located on the two-dimensional plane.

[0012] Furthermore, in an interaction model, the user moves and directs the handheld object using an arm that can function as the radius of a sphere or a spherical cap, where the user's shoulder is treated as the center of the sphere. When the user moves the handheld object or directs it towards a different target, the respective positions of the reference elements of the handheld object may be different for each user, even though they are directed towards the same target. This may be due to differences in the length of the user's arm.

[0013] Therefore, the systems and methods disclosed herein determine the height of a reference element (e.g., from the ground) based on the initial position of the reference element and estimate the height of a user based on the height of the reference element. From the user's height, the length of the user's arm can be estimated, and this can be used to determine one or more scaling factors. The one or more scaling factors can scale or multiply the position of a subsequent reference element detected on a two-dimensional plane from the perspective of the camera to more accurately determine a projection target position (e.g., corresponding to where the user is perceived to be aiming or attempting to aim). In this way, the one or more scaling factors can correct for differences in the length of the user's arm.

[0014] When a camera detects a subsequent reference element position, one or more translation factors and one or more scaling factors can be applied to the subsequent reference element position to determine a projection target position relative to the two-dimensional plane. This embodiment can include a processor that operates to analyze data captured and transmitted by the camera to provide related data such as translation factors, scaling factors, and projection target positions relative to the two-dimensional plane.

[0015] Additionally, by functioning as the radius of a sphere or spherical segment with the user's arm centered on the shoulder, the user can move a handheld object in an arcuate or circular nature. However, a camera that determines the position of a reference element of a handheld object on a flat two-dimensional plane may distort the determined position of the reference element due to the difference in shape between the arcuate movement of the handheld object in space and the flat two-dimensional plane detectable by the camera.

[0016] Accordingly, the systems and methods disclosed herein can determine one or more offsets to be applied to the projection target position to correct this distortion. One or more offsets can be used to increase or increase the distance between the projection target position and the initial position by moving the projection target position to correct the difference in shape between the arcuate nature of the user's arm movement and a flat two-dimensional plane. For example, one or more offsets can be determined using polynomial regression, which fits the test data to one or more polynomials (e.g., cubic polynomials).

[0017] In some embodiments, multiple reference element zones (e.g., locations where a reference element is positioned along an arc based on the user's arm) can be determined to correspond to multiple projection target zones (e.g., projected onto a two-dimensional plane). Each projection target zone can correspond to a set of polynomials that can accurately correct the distortion applied to that projection target zone. Thus, the camera can detect a reference element within a reference element zone, determine that each projection target zone corresponds to a reference element zone, and use the set of polynomials corresponding to each projection target zone to determine one or more offsets to apply to the position of the reference element to correct this distortion. In such embodiments, multiple reference element zones can be of different sizes, as long as the multiple projection target zones are of the same size (e.g., the further the reference element zone is from the two-dimensional plane, the smaller the reference element zone), or multiple reference element zones can be of the same size, as long as the multiple projection target zones are of different sizes (e.g., the further the projection target zone is from the reference element, the larger the projection target zone).

[0018] As an introduction, Figure 1 shows a user 10 pointing a handheld object 12 at a target 14 according to an embodiment of the present disclosure. The target 14 can be a physical object, a figure, a photograph, a graphic, etc. In some cases, the target 14 can be an image output by a display. The target 14 can be printed, engraved, written on, projected, pasted, or otherwise displayed on a structure 15. The user's perception is shown by the first dashed line 16. That is, the user 10 perceives that he is pointing the handheld object 12 at the target 14, specifically at the target position 17. However, due to certain human factors such as one eye being dominant, head tilt, weight shift, leaning to one side or the other, despite the user's perception or intention, the user 10 actually points the handheld object 12 at the actual target position 18, as shown by the dashed line 19.

[0019] The handheld object 12 may represent or include any suitable object that the user 10 can use to point at or refer to the target 14, such as a stick, pencil, toy or model of a gun or weapon, or a cane. The handheld object 12 may include a reference element 20 that can facilitate the identification of the location that the user 10 is pointing to. Specifically, the camera 22 detects the position of the reference element 20, and the reference element 20 may be made of a material or means that makes it easier for the camera 22 to detect the reference element 20. For example, the reference element 20 may be made of a back-reflective material (e.g., back-reflective glass beads, microprisms, or encapsulated lenses sealed on a cloth or plastic substrate), metal tape, etc. In another example, the reference element 20 may include an identifier (e.g., a unique graphical design, a barcode, a quick-response (QR) code, etc.) that allows the camera 22 to identify the reference element 20. As shown in the figure, the reference element 20 may be located at the end 24 of the handheld object 12 opposite the end 26 where the user's hand 28 is holding the handheld object 12. This makes it easier to determine the direction in which the user is pointing the handheld object 12, but the reference element 20 can be placed on any part of the handheld object 12, or even on any part of the user 10.

[0020] The camera 22 can detect the position 30 of the reference element 20 relative to the two-dimensional plane 32. The position 30 can be used to determine a target position 17 that the user 10 perceives as being pointed at or about to be pointed at, by applying one or more translation coefficients. As shown in the figure, the two-dimensional plane 32 may share the same plane as the structure 15, but in some embodiments, the two-dimensional plane 32 and the structure 15 may not share the same plane. For example, the two-dimensional plane 32 and the structure 15 may be parallel to each other. Furthermore, the structure 15 may be semi-transparent, transparent, or have some other suitable property that allows the camera 22 to detect the position 30 of the reference element 20.

[0021] Specifically, a translation coefficient of 1 or more can be applied to the position 30 of the reference element 20 to correct the difference between the user's perception of where the handheld object 12 is pointed and the camera's determination of where the reference element 20 is located on the two-dimensional plane 32. The translation coefficient of 1 or more can be determined during a calibration process in which the user 10 points the handheld object 12 towards a calibration point and the camera 22 detects this initial position of the reference element 20 on the two-dimensional plane 32. The translation coefficient of 1 or more can represent a distance of 1 or more from the initial position to the calibration point (e.g., relative to the two-dimensional plane 32). In addition, the translation coefficient of 1 or more can mitigate or correct for factors such as eye dominance (e.g., right eye or left eye dominance), head tilt, weight shift, and tilt to one or the other side.

[0022] Furthermore, a scaling factor of 1 or more can be applied to the position 30 of the reference element 20 to account for or correct for differences in the user's arm length. That is, the user moves and points the handheld object 12 using an arm that can function as the radius of a sphere or sphere with the shoulder as the center of the sphere. When the user moves or points the handheld object 12 at different targets, the position of the reference element 20 of the handheld object 12 may differ between users because the users' arm lengths differ, even though they are pointing at the same target.

[0023] Therefore, the height of the reference element 20 (for example, from the ground) can be determined based on the initial position of the reference element 20, and the user's height can be estimated based on the height of the reference element 20. From the user's height, the user's arm length can be estimated, and this can be used to determine a scaling factor of 1 or more. A scaling factor of 1 or more can scale or enlarge the position 30 of the reference element 20 detected by the camera 22 on the two-dimensional plane 32.

[0024] Additionally, by applying an offset of 1 or more to the position 30 of the reference element 20, a projection target position for the handheld object 12 can be generated to compensate for distortion resulting from the arcuate or circular movement of the user's arm. That is, this distortion may arise from the difference in shape between the arcuate movement and the camera's detection position of the reference element 20's position 30 on a flat two-dimensional plane 32. The offset of 1 or more can be used to move the projection target position to increase or increase the distance between the projection target position and the initial position, thereby compensating for the difference in shape between the arcuate nature of the user's arm movement and the flat two-dimensional plane. For example, the offset of 1 or more can be determined using polynomial regression, which fits the test data to a polynomial such as a cubic polynomial.

[0025] In this way, a projection target position for the handheld object 12 can be generated, which may precisely coincide with the target position 17 that the user 10 perceives as being pointed at by the handheld object 12. Advantageously, unlike other specific systems, the translation coefficient, scaling coefficient, and offset can be determined using only one calibration point, thereby precisely determining the projection target position of the handheld object 12. On the other hand, in other applications (e.g., pointing devices used in presentations), calibration can be performed before actual execution (e.g., during the preparation phase) and is not visible to the audience or customers, so reducing calibration time may not be as important. However, in this example (e.g., an amusement park attraction), it may be important to conceal or not make the user 10 aware that calibration is taking place in order to provide an immersive user experience. Therefore, limiting the calibration process to a single point (e.g., pointing the handheld object 12 at a single calibration point) can play a role in enhancing or improving the user experience.

[0026] With this in mind, Figure 2 is a block diagram of a theme park attraction system 40 according to an embodiment of the present disclosure. The theme park attraction system 40 allows a user 10 to point a handheld object 12 at various targets 14, and can output a user interaction experience based on the determination that the user 10 has pointed the handheld object 12 at a target 14. For example, the theme park attraction system 40 may include an environment with popular children's characters, a television or movie-themed environment, a shooting range, a group of targets, and so on.

[0027] The amusement park attraction system 40 may include a handheld object 12 having a reference element 20, which is held and manipulated by the user 10. The amusement park attraction system 40 may also include a user interaction system 42 which includes a camera 22 that detects the position of the reference element on a two-dimensional plane 32. The amusement park attraction system 40 may further include a projected location determination system 44 which determines the projected target position of the handheld object 12. Specifically, the projected target position can represent a position on the two-dimensional plane 32 that the user 10 perceives as being pointed at or about to be pointed at. In fact, the closer the projected target position is to the target position 17, the more accurate the projected target position becomes.

[0028] The projection position determination system 44 may include a control device 46 having one or more processors (illustrated as a single processor 48) and one or more memory or storage devices (illustrated as a single memory device 50). The processor 48 can execute software programs and / or instructions stored in the memory device 50 to facilitate the determination of the projection target position of the handheld object 12. Furthermore, the processor 48 may include multiple microprocessors, one or more "general-purpose" microprocessors, one or more application-specific microprocessors, and / or one or more application-specific integrated circuits (ASICS). For example, the processor 48 may include one or more reduced instruction set (RISC) processors. The memory device 50 may store information such as control software, lookup tables, and configuration data. The memory device 50 may include volatile memory (e.g., random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM)), flash memory, one or more hard drives, and / or any other suitable tangible non-temporary machine-readable media such as optical, magnetic, or solid-state storage media. The memory device 50 can store various types of information and can be used for various purposes, such as providing instructions to facilitate the projection target position of the handheld object 12.

[0029] The projection position determination system 44 may also include a reference element position detection logic 52 that determines the position of the reference element 20 on the two-dimensional plane 32. Specifically, the projection position determination system 44 can be communicably coupled to the user interaction system 42 by some suitable means, such as via wired communication or over a communication network using wireless communication protocols or technologies (e.g., wireless, Bluetooth, WiFi, infrared, Ethernet, SLED, ZigBee, Z-Wave, KNX, mobile, and / or microwave). Thus, the reference element position detection logic 52 can receive captured images (e.g., imagery) showing the reference element 20 on the two-dimensional plane 32 from the camera 22. The reference element position detection logic 52 can determine the position of the reference element 20 on the two-dimensional plane 32, for example, represented by a two-dimensional coordinate system (e.g., x and y).

[0030] The projection position determination system 44 may further include a conversion logic 54 that converts the position of the reference element 20 determined by the reference element position detection logic 52 into a projection target position on the two-dimensional plane 32. The conversion logic 54 includes a translation logic 56 that determines one or more translation coefficients to correct the difference between the user's perception of where the handheld object 12 is pointed and the camera's determination of where the reference element 20 is located on the two-dimensional plane 32.

[0031] Specifically, the translation logic 56 can determine one or more translation coefficients by performing a single-point calibration process. This process includes receiving a calibration position on a two-dimensional plane 32, receiving the position of a reference element 20 on the two-dimensional plane 32 (for example, corresponding to when the user 10 points the handheld object 12 to the calibration position), and determining one or more translation coefficients based on the positional difference between the calibration position and the position of the reference element 20.

[0032] Figure 3 shows a user 10 orienting a handheld object 12 towards a calibration position 80 according to an embodiment of the present disclosure. The calibration position 80 can correspond to a physical object, a diagram, a photograph, a graphic, etc. In some cases, the calibration position 80 can correspond to an image output by a display. The user 10 may be prompted by instructions provided in some appropriate format (e.g., writing, engraving, printing, pasting, or displaying on a structure 15). The calibration position 80 can be configured so that the user's height can be detected in a controlled manner by positioning the arm in the same way, while the projection position determination system 44 of Figure 2 can determine the difference between the user's perception of where the handheld object 12 is pointed and where the user 10 is actually pointing the handheld object 12. For example, the calibration position 80 can be positioned so that the user 10 can extend their arm 82 as parallel as possible to the ground 84, for example, and at a certain angle to a plane parallel to the ground. In some embodiments, the calibration position 80 can be customized with respect to the user's height. In other words, in some embodiments, the calibration position 80 can be located lower on the structure 15 for users seated in vehicles such as wheelchairs, personal electric vehicles, or strollers. As another example, the calibration position 80 can be located higher on the structure 15 for adults than for children, and higher on the structure 15 for male users than for female users.

[0033] Therefore, the calibration position 80 can be predetermined and known by the projection position determination system 44. When prompted, the user 10 can extend their arm 82 and orient the handheld object 12 toward the calibration position 80. However, due to distortion effects caused by the human body, such as one eye being dominant, head tilt, weight shift, tilt to one or the other, the user's choice of hand to hold the handheld object 12 (e.g., right hand vs. left hand), physical constraints (e.g., affecting the range of motion), and whether the user's movement may be altered by an obstruction (e.g., a backpack or carrying a child), the user 10 may actually orient the handheld object 12 toward another location, such as the actual calibration position 86 indicated by the dashed line 88, despite the user's perception or intention to orient the handheld object 12 toward the calibration position 80, as indicated by the dashed line 85.

[0034] Camera 22 detects the position 90 of the reference element 20 on the two-dimensional plane 32 and sends the displayed value of the position 90 to the projection position determination system 44. Next, translation logic 56, which can be part of a human interaction model, can determine the position difference between the position 90 of the reference element 20 and a predetermined calibration position 80, which can be expressed in two-dimensional (e.g., x and y) coordinates. Using this difference, translation logic 56 generates one or more translation coefficients applicable to subsequent detection positions of the reference element 20, thereby moving the subsequent detection position of the reference element 20 and determining the subsequent projection target position of the handheld object 12 corresponding to the location where the user 10 was aiming the handheld object 12. The translation coefficients can be provided in the form of a transformation matrix, which can be applied to the subsequent detection position of the reference element 20 to generate the projection target position of the reference element 20 as shown below.

number

[0035] For example, Figure 4 is a diagram illustrating an example of applying one or more translation coefficients to the subsequent detection position 120 of the reference element 20 according to an embodiment of the present disclosure. As shown, during calibration, the position 90 of the reference element 20 is 2 units to the right (e.g., centimeters) and 1 unit above the calibration position 80 (e.g., centimeters). Therefore, the translation coefficients can include +2 horizontally and +1 vertically. Thus, in the transformation matrix, X can be set to +2 and Y can be set to +1. The translation logic 56 can apply the transformation matrix to the subsequent detection position 120 of the reference element 20 (e.g., [4, 2]) to move the subsequent detection position 120 2 units to the right and 1 unit above to generate a projected target position 122 located 6 units to the right and 3 units above the calibration position 80 (e.g., [6, 3]). Therefore, the translation logic 56 can compensate for the difference between the user's perception of where the handheld object 12 is pointed and the camera's determination of where the reference element 20 is located on the two-dimensional plane 32.

[0036] Returning to Figure 2, the conversion logic 54 may also include a scaling logic 58 that determines one or more scaling factors to compensate for differences in the user's arm length. That is, as shown in Figure 3, the user 10 moves and points the handheld object 12 using their arm 82, which can function as the radius of a sphere or sphere notch 92 with the shoulder as the center 94 of the sphere. When the user 10 moves the handheld object 12 to point it at different targets, the position of each reference element 20 of the handheld object 12 may differ for each user 10, even though they are pointing at the same target, due to the differences in the user 10's arm length.

[0037] Specifically, the scaling logic 58 can determine one or more scaling factors based on the position 90 of the reference element 20 detected by the camera 22 during the calibration process. The height 96 of the camera 22 from the ground 84 can be predetermined and known by the scaling logic 58. Therefore, the scaling logic 58 can determine the height 98 of the reference element 20 from the ground 84 based on the position 90 and the predetermined height 96 of the reference element 20. Based on the height 98 of the reference element 20, the user height estimation logic 60 of the scaling logic 58 can determine the user's height 100. Specifically, test or sample data of the position 90 of the reference element 20 and the user's height can be collected when the user 10 points the handheld object 12 towards the calibration position 80. The height 102 of the position 90 of the reference element 20 correlates with the user's height, and the scaling logic 58 can estimate the user's height 100 based on this predetermined correlation and the height 98 of the reference element 20. Models for identifying correlations can implement standard correlation tables between height and reach (e.g., ratios between height and arm length for various body types in a population).

[0038] Next, the user arm length estimation logic 62 of the scaling logic 58 can estimate the user's arm length 104 based on the user's height 100. The estimation can be performed based on a predetermined correlation (e.g., an algorithm or table based on empirical data) between the arm length 104 and the user's height 100. This predetermined correlation can be determined based on test or sample data, scientific data related to human body proportions, and / or some other suitable source.

[0039] The scaling logic 58 can determine a scaling factor of 1 or more based on the user's arm length 104. For example, when pointed away from the initial position (e.g., calibration position 80), the camera 22 can detect that the position of the reference element 20 is closer to the initial position for a user with a shorter arm length 104 compared to a user 10 with a longer arm length. Therefore, the scaling logic 58 can determine a larger scaling factor for a user with a longer arm length 104 compared to a user with a shorter arm length 104. The scaling logic 58 can apply a scaling factor of 1 or more to subsequent detected positions of the reference element 20 to scale the position (e.g., reduce or enlarge) to generate a projected target position for the reference element 20. The scaling factor includes horizontal and vertical components and can be provided in the form of a transformation matrix, which can be inserted into a transformation matrix containing translation coefficients from Equation 1 above, as shown below.

number

[0040] The values ​​of the scaling coefficients k1 and k2 can be determined based on correlation tests or sample data collected from users 10 pointing the handheld object 12 at various targets, and the arm lengths 104 of those users 10. For example, the scaling logic 58 can determine that the height 98 of the reference element 20 from the ground 84 is 1.25 meters, based on image data received from the camera 22 (e.g., the first or calibration image of the video). The user height estimation logic 60 can determine that the user's height 100 is approximately 1.8 meters, based on the height 98 of the reference element 20. The user arm length estimation logic 62 can determine that the user's arm length 104 is 0.6 meters, based on the user's height 100. Then, the scaling logic 58 can determine that the horizontal scaling coefficient k1 is 1.5 and the vertical scaling coefficient k2 is 1.75, based on the user's arm length 104. Therefore, the scaling logic 58 generates a transformation matrix of Equation 2 with k1=1.5 and k2=1.75, and the projection position determination system 44 can apply the transformation matrix to the subsequent detected position of the reference element 20 to generate a projection target position where the user 10 was trying to point the handheld object 12, which compensates for the difference in the user's arm length 104.

[0041] For example, Figure 5 is a diagram illustrating an example of applying a scaling factor to a subsequent detection position 120 of the reference element 20 according to an embodiment of the present disclosure. As shown, the subsequent detection position 120 of the reference element 20 is 4 units to the right (e.g., centimeters) and 4 units above (e.g., centimeters) of the calibration position 80 (e.g., [4,4]). When the transformation matrix of Equation 2, having a horizontal scaling factor k1=1.5 and a vertical scaling factor k2=1.75, is applied to the subsequent detection position 120, the subsequent detection position 120 is scaled horizontally by 1.5 to generate a projected target position 130 6 units to the right of the calibration position 80, and is scaled vertically by 1.75 to generate a projected target position 130 7 units above (e.g., centimeters) (e.g., [6,7]). Thus, the scaling logic 58 can compensate for differences in the user's arm length 104.

[0042] Returning to Figure 2, the projection position determination system 44 may include an arc distortion compensation logic 64 that compensates for the difference in shape between the arc-like nature 92 of the user's arm movement and the flat two-dimensional plane 32. For example, Figure 6 shows a user 10 pointing a handheld object 12 at different targets. As shown, the angle θ formed between the first position 140 of the user's arm 82 and the second position 142 of the user's arm 82 is the same as the angle between the third position 144 of the user's arm 82 and the fourth position 146 of the user's arm 82. However, when the camera 22 observes and captures data on a two-dimensional plane 32, the distance h0 between the first reference element position 148 corresponding to the first position 140 of the user's arm 82 and the second reference element position 150 corresponding to the second position 142 of the user's arm 82 is different from (for example, greater than) the distance h1 between the third reference element position 152 corresponding to the third position 144 of the user's arm 82 and the fourth reference element position 154 corresponding to the fourth position 146 of the user's arm 82.

[0043] Thus, the arc distortion correction logic 64 can determine one or more offsets to apply to the projection target position, which corrects this distortion. The one or more offsets can move the projection target position to increase or expand the distance between the projection target position and the initial position (e.g., the calibration position 80) to correct for the difference in shape between the arcuate nature 92 of the user's arm movement and the flat two-dimensional plane 32. For example, the one or more offsets can be determined using regression analysis to fit test or sample data from the user 10 aiming the handheld object 12 at various targets (e.g., along the arc 92 with the reference element 20). In some embodiments, the arc distortion correction logic 64 can fit the test data to a polynomial (e.g., a third-degree polynomial), although any suitable degree or type of equation can be used. For example, a first third-degree polynomial (Equations 3 and 4 below) is used to determine a horizontal offset for application to the projection target position to correct this distortion horizontally, and a second third-degree polynomial (Equations 5 and 6 below) is used to determine a vertical offset for application to the projection target position to correct this distortion vertically.

Number

Number

[0044] The horizontal component of the projection target position can be measured as the horizontal distance from the initial position (for example, when the calibration position 80 and / or user 10 are pointing the handheld object 12 directly at the camera 22), while the vertical component of the projection target position can be measured as the vertical distance from the initial position. As mentioned above, for any of polynomials 3 to 6, the constant a i , b i , c i a, b, c, d, e, f, g, h, k, and 1 can be determined by fitting test or sample data to a polynomial using polynomial regression analysis (these may differ depending on the formula). Thus, when user 10 moves and points the handheld object 12, an offset of 1 or more can be determined for each projection target position.

[0045] However, when the user 10 moves and points the handheld object 12, applying any of equations 3 to 6 to determine the horizontal and vertical offsets for each projection target position can be time-consuming and may use excessive computing resources (e.g., processing, memory, storage, or network resources). Therefore, in some embodiments, to more efficiently compensate for the difference in shape between the arc-like nature 92 of the user's arm movement and the flat two-dimensional plane 32, the arc distortion correction logic 64 can divide the arc 92 in which the reference element 20 may be located into a plurality of reference element zones, each of which can correspond to its respective projection target zone (e.g., projected onto the two-dimensional plane). Each projection target zone can correspond to a set of polynomials that can accurately correct the distortion that can be applied to that projection target zone. Thus, the camera 22 can detect the reference element 20 within a reference element zone, the arc distortion correction logic 64 can determine the respective projection target zone corresponding to the reference element zone, and the arc distortion correction logic 64 can apply the respective set of polynomials corresponding to each projection target zone to the position of the reference element to determine one or more offsets to be applied to the position of the reference element to correct this distortion. In such embodiments, multiple reference element zones can be of different sizes as long as multiple projection target zones are of the same size (for example, the further a reference element zone is from the 2D plane 32, the smaller the size of the reference element zone), or multiple reference element zones can be of the same size as long as multiple projection target zones are of different sizes (for example, the further a projection target zone is from the reference element 20, the larger the size of the projection target zone).

[0046] Figure 7 shows a plurality of reference element zones 170 of different sizes and a plurality of projection target zones 172 of constant size according to an embodiment of the present disclosure. As shown, the first reference element zone 174, which is closest to the two-dimensional plane 32, is the largest in size, the second reference element zone 176, which is the next closest to the two-dimensional plane 32, is the next largest in size (but smaller than the first reference element zone 174), the third reference element zone 178, which is the next closest to the two-dimensional plane 32, is the next largest in size (but smaller than the second reference element zone 176), and the fourth reference element zone 180, which is the next closest to the two-dimensional plane 32, is the next largest in size (but smaller than the third reference element zone 178). Although four reference element zones 170 are shown in Figure 7, it should be understood that some appropriate number of reference element zones 170 are assumed to be of some appropriate size, and the further the reference element zones 170 are from the two-dimensional plane 32, the smaller the size of the reference element zones 170 becomes. Furthermore, each projection target zone 172 is the same size as the other projection target zones 172 and corresponds to each set of polynomials that generate the respective offsets (e.g., horizontal and vertical offsets) that can be applied to the position of the reference element 20, corresponding to each reference element zone 170. Specifically, each set of polynomials corresponding to each projection target zone 172 is defined by a constant a, as specified in any of equations 3 to 6. i , b i , c i , a, b, c, d, e, f, g, h, k, and l can have different sets of values ​​(and can be different for each expression). By reducing the size of the reference element zone 170 as it is farther from the two-dimensional plane 32, the arc distortion correction logic 64 can correct the difference in shape between the arc nature 92 of the user's arm movement and the flat two-dimensional plane 32 in an efficient and resource-saving manner, while maintaining the same size of the projection target zone 172.

[0047] Figure 8 shows a plurality of reference element zones 190 of a constant size and a plurality of projection target zones 192 of different sizes according to an embodiment of the present disclosure. As shown in the figure, each reference element zone 190 is the same size. However, the first projection target zone 194, which is closest to the reference element 20, is the smallest in size, the second projection target zone 196, which is the next closest to the reference element 20, is the next smallest in size (but larger than the first projection target zone 194), the third projection target zone 198, which is the next closest to the reference element 20, is the next smallest in size (but larger than the second projection target zone 196), and the fourth projection target zone 200, which is the next closest to the reference element 20, is the next smallest in size (but larger than the third projection target zone 198). Although four projection target zones 192 are shown in Figure 8, it should be understood that some appropriate number of projection target zones 192 are assumed to be of some appropriate size, and the size of the projection target zones 192 increases as they are further away from the reference element 20. Each projection target zone 192 corresponds to a set of polynomials that can be applied to the position of the reference element 20 and generate their respective offsets (e.g., horizontal and vertical offsets). Specifically, each set of polynomials corresponding to each projection target zone 192 is defined by a constant a, as specified in any of equations 3 to 6. i , b i , c i , a, b, c, d, e, f, g, h, j, k, and l can have different sets of values ​​(and can be different for each expression). By increasing the size of the projection target zone 192 as the projection target zone 192 is farther from the reference element 20, the arc distortion correction logic 64 can efficiently and resource-savingly correct the difference in shape between the arc nature 92 of the user's arm movement and the flat two-dimensional plane 32 while maintaining the same size of the reference element zone 190.

[0048] For simplicity, please note that Figures 6-8 show the distortion resulting from the difference in shape between the arc-like nature 92 of the user's arm movement and the flat two-dimensional plane 32 only in the vertical (e.g., y) direction. However, the systems and methods disclosed herein are intended to correct distortion in any appropriate direction, including the horizontal (e.g., x) direction as revealed by equations 3 and 4, which provide a horizontal offset for correcting horizontal distortion, and the vertical (e.g., y) direction as revealed by equations 5 and 6, which provide a vertical offset for correcting vertical distortion.

[0049] Returning to Figure 2, if the projection position determination system 44 determines that the projection target position corresponds to a target 14 printed, engraved, written, attached, or otherwise displayed on the structure 15, the output device 66 of the user interaction system 42 can output a user interaction experience. The output device 66 can be any suitable device capable of outputting a desired user interaction experience, such as an electronic display, speaker, virtual reality device, augmented reality device, actuator, and / or animation device (e.g., a robot character). The target 14 may be part of the output device 66, fixed to it, attached to it, or included therein, or the target 14 may be detached from the output device 66. For example, in an amusement park environment, both the target 14 and the output device 66 may be animation objects of an attraction, and in response to the determination that the projection target position corresponds to an animation object, the animation object may output a user interaction experience (e.g., tail wagging). As another example, target 14 may be a word printed on a poster, and output device 66 may be a nearby speaker. In response to the determination that the projection target position corresponds to the word printed on the poster, the nearby speaker may output an audio of the word. In yet another example, target 14 may be an image of a person on an electronic display, and output device 66 may be an electronic display. In response to the determination that the projection target position corresponds to the image of that person, the electronic display may play a video showing the person in the image performing a characteristic action.

[0050] With this in mind, Figure 9 is a flowchart of a process 210 for determining the projection target position of a handheld object 12 according to an embodiment of the present disclosure. This process 210 can be performed by any suitable device capable of determining the projection target position of the handheld object 12, such as any component of a projection position determination system 44 including a control device 46, a processor 48, a reference element position detection logic 52, a transformation logic 54, a translation logic 56, a scaling logic 58, a user height estimation logic 60, and / or a user arm length estimation logic 62. Although the process 210 is described using a specific sequence of steps, it should be understood that the steps described may be performed in an order different from the illustrated order, and that certain steps described may be skipped or not performed at all. In some embodiments, the process 210 can be performed by executing instructions stored in a tangible, non-temporary computer-readable medium, such as a memory device 50, using a processor such as a processor 48.

[0051] As shown in the figure, in process block 212, the processor 48 receives an indication for calibrating the handheld object 12. This indication can be in the form of an image captured by the camera 22 (e.g., a first or calibration image of the video) including the presence of a reference element 20 of the handheld object 12. In some embodiments, a motion sensor or other suitable sensor that can indicate that a user 10 has entered the field of view of the camera 22 with the handheld object 12 having the reference element 20 can provide this indication.

[0052] In process block 214, the processor 48 receives the calibration position 80. Specifically, the calibration position 80 can be fixed on the structure 15 or displayed on the structure 15 by the processor 48, so the calibration position 80 can be predetermined and known to the processor 48.

[0053] In process block 216, the processor 48 receives the position of the reference element 20 of the handheld object 12. For example, the camera 22 can provide an image of the reference element 20 (e.g., the second or subsequent image of the video captured by the camera 22). The processor 48 can then instruct the reference element position detection logic 52 to determine the position of the reference element 20 on the two-dimensional plane 32.

[0054] In process block 218, processor 48 instructs translation logic 56 to determine one or more translation coefficients based on the position of the reference element 20 and the calibration position 80. One or more translation coefficients can compensate for the difference between the user's perception of where the handheld object 12 is pointed and the camera's determination of where the reference element 20 is located on the two-dimensional plane 32. Specifically, translation logic 56 can determine one or more translation coefficients by performing a single-point calibration process. This process includes receiving a calibration position on the two-dimensional plane 32, receiving the position of the reference element 20 on the two-dimensional plane 32 (for example, corresponding to when user 10 points the handheld object 12 at the calibration position), and determining one or more translation coefficients based on the positional difference between the calibration position and the position of the reference element 20.

[0055] The translation logic 56 can use this difference to generate one or more translation coefficients that can be applied to the subsequent detection position of the reference element 20, thereby moving the subsequent detection position of the reference element 20 and determining the subsequent projected target position of the handheld object 12 corresponding to the location where the user 10 was aiming the handheld object 12. The translation coefficients can be provided in the form of a transformation matrix, which can be applied to the subsequent detection position of the reference element 20 to generate the projected target position of the reference element 20 as shown in Equation 1.

[0056] In process block 220, processor 48 instructs user height estimation logic 60 to determine user 10's height 100 based on the position of reference element 20. In process block 222, processor 48 instructs user arm length estimation logic 62 to determine user 10's arm length 104 based on user 10's height 100.

[0057] In process block 224, processor 48 instructs scaling logic 58 to determine a scaling factor of 1 or more based on the user's arm length 104. Scaling logic 58 can provide the scaling factor using the transformation matrix of equation 2 above. The scaling factor can compensate for differences in the user's arm length 104 by scaling (e.g., multiplying) the position of the reference element 20 relative to the initial position (e.g., calibration position 80).

[0058] In process block 226, processor 48 instructs transformation logic 54 to determine the projection target position of the handheld object 12 based on the position of the reference element 20, a translation coefficient of 1 or more, and a scaling coefficient of 1 or more. Specifically, transformation logic 54 can generate the projection target position by applying the transformation matrix of equation 2, which includes a translation coefficient of 1 or more and a scaling coefficient of 1 or more, to the position of the reference element 20. That is, the projection target position can correspond to a location that user 10 is pointing or perceiving as being pointed at or about to point at.

[0059] In the decision block 228, the processor 48 determines whether the projection target position correlates with a user interaction element. The user interaction element can be any suitable target that acts as a trigger for executing a user interaction experience. For example, the user interaction element may include some feature of interest that the user 10 can expect to trigger a user interaction experience when pointing the handheld object 12 at it.

[0060] If the processor 48 determines that the projection target position correlates with a user interaction element, then in process block 230, the processor 48 instructs the user interaction system 42 to execute each user interaction experience using an appropriate output device 66. For example, the output device 66 can be an animated object in an attraction, and the user interaction system 42 can make the animated object bark, bark, talk, move, or flash. As another example, the output device 66 can be a speaker, and the user interaction system 42 can output sounds, voices, music, etc., to the speaker. As yet another example, the output device 66 can be an electronic display, and the user interaction system 42 can make the electronic display display images, play videos, etc.

[0061] If the processor 48 determines that the projection target position is not correlated with the user interaction element, in the determination block 232, the processor 48 determines whether or not it has received the next position of the reference element 20. If it has received it, the processor 48 repeats the process block 226 to determine the projection target position of the handheld object 12 based on the next position of the reference element 20, as well as the translation coefficient and scaling coefficient already determined from process blocks 218 and 224.

[0062] If the processor 48 determines that it has not received the next position of the reference element 20, the processor 48 repeats process block 212 to receive the next instruction to calibrate the handheld object 12 (e.g., from the next user 10). In this way, process 210 can determine the projection target position of the handheld object 12 using single-point calibration (e.g., without requiring the user 10 to point the handheld object 12 at two or more points in order to calibrate the projection position determination system 44), which compensates for both the difference between the user's perception of where the handheld object 12 is pointed and the camera's determination of where the reference element 20 is located on the two-dimensional plane 32, as well as the difference in the user's arm length 104.

[0063] Furthermore, the projection positioning system 44 can also compensate for distortion caused by the difference in shape between the arc-like nature 92 of the user's arm movement and the flat two-dimensional plane 32, as shown in Figure 6. Figure 10 is a flowchart of a process 240 for compensating for this distortion according to an embodiment of the present disclosure. Process 240 can be performed by any suitable device capable of compensating for this distortion, such as any component of the projection positioning system 44 including a control device 46, a processor 48, and / or arc distortion correction logic 64. Although process 240 is described as steps in a specific sequence, it should be understood that the steps described in this disclosure may be performed in an order different from the illustrated order, and that certain steps described may be skipped or not performed at all. In some embodiments, process 240 can be performed by executing instructions stored in a tangible, non-temporary computer-readable medium, such as a memory device 50, using a processor such as a processor 48.

[0064] As shown in the figure, in process block 242, the processor 48 receives the position of the reference element 20 of the handheld object 12. In some embodiments, the processor 48 can receive the projection target position of the handheld object 12.

[0065] In process block 244, the processor 48 determines the horizontal offset based on the position of the reference element 20 and the first polynomial. Specifically, the processor 48 can receive the projection target position of the handheld object 12 or determine the projection target position using process 210 in Figure 9. The processor 48 can then instruct the arc distortion correction logic 64 to apply polynomial 3 or 4 to the projection target position of the handheld object 12 to determine the horizontal offset.

[0066] In process block 246, the processor 48 determines the vertical offset based on the position of the reference element 20 and the second polynomial. Specifically, the processor 48 can instruct the arc distortion correction logic 64 to determine the vertical offset by applying polynomial 5 or 6 to the projection target position of the handheld object 12.

[0067] In process block 248, the processor 48 determines the projection target position of the handheld object 12 based on the position, horizontal offset, and vertical offset of the reference element 20. Specifically, the processor 48 can instruct the arc distortion correction logic 64 to apply (e.g., add) the horizontal offset to the horizontal component (e.g., x-coordinate) of the projection target position and to apply (e.g., add) the vertical offset to the vertical component (e.g., y-coordinate) of the projection target position in order to generate the projection target position.

[0068] In some embodiments, to more efficiently compensate for the difference in shape between the arc-like nature 92 of the user's arm movement and the flat two-dimensional plane 32, the arc distortion correction logic 64 can divide the arc 92 in which the reference element 20 may be located into a plurality of reference element zones, each of which can correspond to a respective projection target zone (e.g., projected onto a two-dimensional plane). Each projection target zone can correspond to a set of polynomials that can accurately correct the distortion applied to that projection target zone. Thus, the camera 22 can detect the reference element 20 within a reference element zone, the arc distortion correction logic 64 can determine the respective projection target zone corresponding to the reference element zone, and the arc distortion correction logic 64 can apply the respective set of polynomials corresponding to each projection target zone to the position of the reference element to determine one or more offsets to apply to the position of the reference element and correct this distortion. In such embodiments, as shown in Figure 7, multiple reference element zones can be of different sizes as long as multiple projection target zones are of the same size (for example, the further a reference element zone is from the 2D plane 32, the smaller the size of the reference element zone), or as shown in Figure 8, multiple reference element zones can be of the same size as long as multiple projection target zones are of different sizes (for example, the further a projection target zone is from the reference element 20, the larger the size of the projection target zone).

[0069] In this way, process 240 can compensate for the arc-like nature 92 of the user's arm movement and the flat two-dimensional plane 32. Furthermore, to compensate for the difference between the user's perception of where the handheld object 12 is pointed and the camera's determination of where the reference element 20 is located on the two-dimensional plane 32, the difference in the user's arm length 104, and the difference in shape between the arc-like nature 92 of the user's arm movement and the flat two-dimensional plane 32, process 240 in Figure 10 can be performed before, after, or as part of process 210 in Figure 9.

[0070] The embodiments described herein are susceptible to various modifications and alternative forms, although certain embodiments are illustrated in the drawings and described in detail herein. However, it should be understood that this disclosure is not limited to the specific embodiments disclosed herein. This disclosure includes all modifications, equivalents, and alternatives that fall within the spirit and scope of this disclosure as defined by the appended claims below.

[0071] The claimed technologies described herein refer to and apply to tangible objects and specific examples of a practical nature that are not abstract, intangible, or purely theoretical, but which certainly improve the art. Furthermore, if any of the claims appended to the end of this specification contain one or more elements designated as "...means for performing [function]" or "...steps for performing [function]," such elements should be interpreted in accordance with 112(f) of the United States Patent Act. On the other hand, any claim containing elements designated in any other form should not be interpreted in accordance with 112(f) of the United States Patent Act. [Explanation of symbols]

[0072] 10 users 12 Handheld objects 14 Targets 15 Structure 16. First dashed line 17 Target location 18 Actual target location 19 Dashed line 20 Criteria 22 cameras 24 End of a handheld object 26 End holding a handheld object 28 User's hands 30. Position of the reference element 32 2D plane

Claims

1. It is an amusement park attraction system, A camera configured to capture images of reference elements of a handheld object on a two-dimensional plane, and Output devices configured to output user interactive experiences, A user interaction system including, A projection position determination system is communicatively coupled to the user interaction system, Includes, The projection position determination system includes a control device having one or more processors and memory, The aforementioned memory is Determine one or more translation coefficients that represent the positional difference between the calibration position and the initial position of the reference element on the two-dimensional plane captured by the video, Based on the user's arm length estimated from the aforementioned video, one or more scaling factors unique to the user are determined to adjust the initial position of the reference element to correct for differences in the user's arm length. The current position of the reference element on the two-dimensional plane captured in the aforementioned video is determined, The projection target position of the handheld object is determined based on the current position of the reference element, the translation coefficient of one or more, and the scaling coefficient of one or more. In response to the determination that the projection target position corresponds to the target position, the output device is instructed to output the user interactive experience, It stores machine-readable instructions configured to cause one or more processors to perform the above-mentioned actions. Amusement park attraction system.

2. The amusement park attraction system according to claim 1, wherein the machine-readable instruction is configured to cause one or more processors to determine one or more translation coefficients and one or more scaling coefficients based on a first image of the video.

3. The amusement park attraction system according to claim 2, wherein the machine-readable instruction is configured to cause one or more processors to determine the current position of the reference element based on a second image of the video.

4. The amusement park attraction system according to claim 1, wherein the one or more translation coefficients correct the difference between the user's perception of where the handheld object is facing in three-dimensional space and the corresponding position of the reference element in the two-dimensional plane.

5. The amusement park attraction system according to claim 1, wherein the machine-readable instruction causes the one or more processors to determine the one or more scaling coefficients based on the position of the reference element on the two-dimensional plane.

6. The machine-readable instructions are provided to the one or more processors: The user's height is determined based on the position of the reference element on the two-dimensional plane. The length of the user's arm is determined based on the user's height. The amusement park attraction system according to claim 1.

7. The machine-readable instructions are provided to the one or more processors: Based on the projection target position and one or more polynomials, one or more offsets are determined. Apply the one or more offsets described above to the projection target position. After applying one or more of the aforementioned offsets to the projection target position, in response to the determination that the projection target position corresponds to the target position on the two-dimensional plane, the output device is instructed to output the user interactive experience. The amusement park attraction system according to claim 1, configured as described above.

8. The amusement park attraction system according to claim 7, wherein the one or more offsets correct the difference in shape between the arc-like nature of the user's arm movement and the two-dimensional plane.

9. The amusement park attraction system according to claim 7, wherein at least one of the one or more polynomials is a cubic polynomial.

10. It is a system, One or more processors, Memory for storing machine-readable instructions, The machine-readable instruction includes, Determine one or more translation coefficients that represent the difference in position between the calibration position on the two-dimensional plane and the initial position of the reference element of the handheld object in the first image on the two-dimensional plane, Based on the arm length of the user estimated from the first image, one or more scaling factors unique to the user are determined to scale the initial position of the reference element to correct for differences in the user's arm length. Determining the current position of the reference element in the second image on the two-dimensional plane, The projection target position of the handheld object is determined based on the current position of the reference element, the translation coefficient of one or more, and the scaling coefficient of one or more. In response to the determination that the projection target position corresponds to the target position, the system outputs a user interactive experience. The configuration is configured to have one or more processors perform the above-mentioned task. system.

11. The system according to claim 10, wherein the machine-readable instruction is configured to cause one or more processors to determine a transformation matrix including one or more translation coefficients and one or more scaling coefficients.

12. The system according to claim 11, wherein the machine-readable instruction is configured to cause one or more processors to determine the projection target position of the handheld object by applying the transformation matrix to the current position of the reference element.

13. The system according to claim 10, wherein the one or more translation coefficients include a horizontal component, and the machine-readable instruction is configured to cause the one or more processors to determine the horizontal component based on the horizontal difference between the calibration position and the initial position of the reference element.

14. The system according to claim 13, wherein the one or more translation coefficients include a vertical component, and the machine-readable instruction is configured to cause the one or more processors to determine the vertical component based on the vertical difference between the calibration position and the initial position of the reference element.

15. The system according to claim 10, wherein the one or more scaling coefficients include a horizontal component, and the machine-readable instruction is configured to cause the one or more processors to determine the horizontal component based on the length of the user's arm.

16. The system according to claim 15, wherein the one or more scaling coefficients include a vertical component, and the machine-readable instruction is configured to cause the one or more processors to determine the vertical component based on the length of the user's arm.

17. It is a method, A step of receiving a calibration position on a two-dimensional plane, The steps include receiving the initial position of the reference element of the handheld object on the two-dimensional plane captured by the image, A step of determining one or more translational coefficients based on the calibration position and the initial position of the reference element, A step of determining one or more scaling factors unique to the user for correcting differences in the user's arm length by scaling the initial position of the reference element based on the user's arm length estimated based on the initial position of the reference element, The steps include determining the current position of the reference element on the two-dimensional plane captured in the aforementioned video, A step of determining the projection target position of the handheld object based on the current position of the reference element, the one or more translation coefficients, and the one or more scaling coefficients, In response to the determination that the projection target position corresponds to the target position, the steps include outputting a user interactive experience, A method that includes this.

18. The method according to claim 17, wherein the step of determining one or more translation coefficients is based on the difference between the calibration position and the initial position of the reference element on the two-dimensional plane.

19. The method according to claim 17, further comprising the step of determining the user's height based on the initial position of the reference element.

20. The method according to claim 19, further comprising the step of determining the length of the user's arm based on the user's height.

21. The method according to claim 19, wherein the step of determining the one or more scaling factors is based on the user's height.

22. The method according to claim 20, comprising the step of determining one or more scaling factors based on the length of the user's arm.

Images