System and method for determining projected target position of handheld object

Abstract

A projected target position of the handheld object is determined based on applying a translation factor, a zoom factor, and an offset to a position of a reference element of the handheld object detected by the camera on a two-dimensional plane. The translation factor is determined based on a difference between a calibration position on the plane and an initial position of the reference element corresponding to the calibration position, and is used to shift the position of the reference element to generate a projection target position. A scaling factor is determined based on an estimated length of an arm of a user holding the handheld object and is used to scale a position of the reference element to generate a projected target position. The offset is determined based on a polynomial equation and is used to extend the distance between the projection target position and the calibration position.

Description

[0001] Cross-reference to related applications This application is a divisional application of Chinese Patent Application No. 202080067328.9, entitled "System and Method for Determining the Projected Target Position of a Handheld Object," which entered the Chinese national phase on March 25, 2022. This Chinese patent application claims the benefit of U.S. Provisional Application No. 62 / 905901, entitled "System and Method for Determining the Projected Target Position of a Handheld Object," filed on September 25, 2019. Both the Chinese patent application and the provisional application are hereby incorporated, in their entirety, by reference for all purposes. Background Technology

[0002] This disclosure generally relates to a handheld object for aiming, and more particularly to determining the projected target position of the handheld object.

[0003] This section aims to introduce the reader to various technical aspects that may relate to the various aspects of this disclosure, which are described and / or claimed below. This discussion is intended to help provide the reader with background information to facilitate a better understanding of the various aspects of this disclosure. Therefore, it is understood that these statements will be read from this perspective, rather than as an admission of prior art.

[0004] Handheld objects can be used to aim at or select a target. For example, in a theme park setting, customers can use a handheld object to aim at an animated character at an attraction, and in response to this detection, the system may cause the animated character to output an interactive user experience (e.g., wag its tail). However, it is now recognized that certain physical characteristics related to the user's body can make it difficult to accurately determine when a user is aiming at a target. Attached Figure Description

[0005] These and other features, aspects, and advantages of this disclosure will become better understood when the following detailed description is read with reference to the accompanying drawings, in which the same characters denote the same parts throughout the drawings, wherein: Figure 1 This is a diagram showing a user aiming a handheld object at a target according to an embodiment of the present disclosure; Figure 2 This is a block diagram of a theme park attraction system according to an embodiment of the present disclosure; Figure 3 This is a diagram showing a user aiming a handheld object at a calibration position according to an embodiment of the present disclosure; Figure 4 Applying one or more translation factors according to embodiments of this disclosure Figure 3 A diagram illustrating an example of the subsequently detected position of a reference element of a handheld object; Figure 5 Applying a scaling factor according to embodiments of this disclosure Figure 3 A diagram illustrating an example of the subsequently detected position of a reference element of a handheld object; Figure 6 This is a diagram showing a user aiming a handheld object at different targets on the system according to embodiments of the present disclosure; Figure 7 This is a diagram of multiple reference element regions of different sizes and multiple projection target regions of uniform size according to embodiments of the present disclosure; Figure 8 This is a diagram of a plurality of reference element regions of uniform size and a plurality of projection target regions of different sizes according to embodiments of the present disclosure; Figure 9 It is used to determine according to embodiments of this disclosure. Figure 3 A flowchart illustrating the process of projecting the target position of the handheld object; and Figure 10 This is a flowchart of a process for compensating for distortion caused by the difference in shape between the arcuate nature of the user's arm movement and a two-dimensional plane, according to embodiments of the present disclosure. Detailed Implementation

[0006] One or more specific embodiments will be described below. For the purpose of providing a concise description of these embodiments, not all features of the actual implementation are described in the specification. It should be appreciated that, as in any engineering or design project, numerous implementation-specific decisions must be made in the development of any such implementation to achieve the developer's specific goals, which may vary depending on the implementation, such as compliance with system-related constraints and business-related constraints. Furthermore, it should be appreciated that such development efforts may be complex and time-consuming, but for those skilled in the art who benefit from this disclosure, such development efforts will be nothing more than routine tasks of design, fabrication, and manufacturing.

[0007] When describing elements of various embodiments of this disclosure, the articles “a,” “an,” and “the” are intended to mean the presence of one or more of the elements. The terms “comprising,” “including,” and “having” are intended to be inclusive and mean that additional elements may exist in addition to those listed. Furthermore, it should be understood that references to “an embodiment” or “an embodiment” in this disclosure are not intended to be construed as excluding the existence of additional embodiments that also incorporate the described features.

[0008] This disclosure generally relates to a handheld object for aiming, and more particularly, to determining the projected target position of the handheld object. Specifically, a reference element can provide indication of where the handheld object is being aimed. For example, in a theme park setting, a user can use the handheld object to aim at an animated object at the attraction (e.g., a robot, or in other cases, an animated figure), and, in response to detecting the position of the reference element, the animated object can output an interactive experience for the user (e.g., wagging its tail). As another example, a user can aim at a word on a poster, and, in response to detecting the position of the reference element, a nearby speaker can output the speech of that word. As yet another example, a user can aim at an image of a person on an electronic display, and, in response to detecting the position of the reference element, the display can play a video showing the person moving in the image.

[0009] Currently disclosed systems and methods involve using a camera to determine the position of a reference element in a two-dimensional plane perpendicular to the camera's direction. The camera can detect the reference element of the handheld object, which may be made of a material that is more easily detected by the camera (e.g., a reflective material). The position of the reference element can be used to determine the target location where the user was aiming the handheld object. However, in some systems, the user's perception of where they are aiming the handheld object may not match the projected position determined based on the camera's view where the user is aiming. This can be due to a variety of factors, including eye dominance over another (e.g., right-eye dominance or left-eye dominance), head tilt, weight shift, tilting to one side or the other, etc. Any combination of these factors may cause the user's perception of where they are planning to move when their hand is aiming the handheld object in the same location. It should be noted that the camera is an example of various light detectors that can be used according to the present embodiments. Therefore, reference to the camera indicates other light detectors that can be used by embodiments of this disclosure.

[0010] Currently disclosed systems and methods involve providing a calibration point on a two-dimensional plane, at which a user can aim a handheld object. The position of a reference element relative to the two-dimensional plane can be determined as an initial position, and one or more translation factors can be determined based on the difference between the initial position and the calibration point. That is, the calibration point can be associated with the position where the user perceives they are aiming the handheld object, while the initial position of the reference element can be associated with the position of the reference element on the two-dimensional plane as seen from the camera's viewpoint. The difference between the two can be used to translate the subsequently detected position of the reference element on the two-dimensional plane as seen from the camera's viewpoint to the projected target position (e.g., corresponding to the position where the user perceives they are aiming or intend to aim). In other words, one or more translation factors can compensate for the difference between the user's perception of where they are aiming the handheld object and the camera's determination of where the reference element is located on the two-dimensional plane.

[0011] Furthermore, users use their arms to move and aim at the handheld object; the arm can act as the radius or spherical segment of a sphere in the interaction model, with their shoulder considered the center of the sphere. When a user moves the handheld object or aims at different targets, the corresponding position of the reference element of the handheld object may differ between users, even when aiming at the same target. This could be due to the different arm lengths of the users.

[0012] Therefore, currently disclosed systems and methods determine the height of a reference element (e.g., above the ground) based on its initial position, and estimate the user's height based on this height. The user's arm length can be estimated from this height, and this arm length can be used to determine one or more scaling factors. These scaling factors can scale or multiply the subsequently detected reference element position in a two-dimensional plane as seen from the camera's viewpoint to more accurately determine the projected target position (e.g., corresponding to the position where the user perceives they are aiming at or intend to aim at). In this way, the one or more scaling factors can compensate for differences in user arm lengths.

[0013] When the position of a subsequent reference element is detected by the camera, one or more translation factors and one or more scaling factors can be applied to the subsequent reference element position to determine the position of the projected target relative to a two-dimensional plane. Current embodiments may include a processor that operates to analyze data captured and transmitted by the camera to provide relevant data such as translation factors, scaling factors, the position of the projected target relative to the two-dimensional plane, and so on.

[0014] Furthermore, since the user's arm acts as the radius or cross section of a sphere, with their shoulder as the center, the user can move the held object in an arc or circle. However, the camera determining the position of the reference element of the held object on a flat two-dimensional plane may distort the determined position of the reference element due to the difference in shape between the arc-shaped movement of the held object in space and the flat two-dimensional plane, which can be detected by the camera.

[0015] Therefore, the currently disclosed systems and methods can determine one or more offsets to be applied to the projected target position to compensate for this distortion. These one or more offsets can shift the projected target position to increase or lengthen the distance between the projected target position and its initial position, thereby compensating for the shape difference between the arcuate nature of the user's arm movement and a flat two-dimensional plane. For example, these one or more offsets can be determined using polynomial regression, which fits test data to one or more polynomial equations (e.g., third-order polynomial equations).

[0016] In some embodiments, multiple reference element regions (e.g., where reference elements are positioned along an arc based on the user's arm) may be identified as corresponding to multiple projection target regions (e.g., projected onto a two-dimensional plane). Each projection target region may correspond to a set of polynomial equations that can accurately compensate for distortions applicable to that projection target region. Accordingly, a camera can detect reference elements within the reference element regions, determine that a corresponding projection target region corresponds to a reference element region, and the corresponding set of polynomial equations corresponding to the corresponding projection target region can be used to determine one or more offsets to be applied to the reference element to compensate for the distortion. In such embodiments, the multiple reference element regions may be of different sizes (e.g., the reference element regions decrease in size the farther they are from the two-dimensional plane), while the multiple projection target regions are of the same size, or the multiple reference element regions may be of the same size, while the multiple projection target regions are of different sizes (e.g., the projection target regions increase in size the farther they are from the reference elements).

[0017] Through introduction, Figure 1This is a diagram of a user 10 aiming a handheld object 12 at a target 14 according to an embodiment of this disclosure. The target 14 can be a physical object, a drawing, a photograph, a graphic, etc. In some cases, the target 14 can be an image output by a display. The target 14 can be printed, etched, written, projected, attached, or otherwise displayed on the structure 15. The user's perception is indicated by the first dashed line 16. That is, the user 10 perceives that they are aiming the handheld object 12 at the target 14, and specifically at the target location 17. However, due to certain human factors (such as one eye dominance over the other, head tilt, weight shift, tilting to one side or the other, etc.), regardless of the user's perception or intention, the user 10 actually aims the handheld object 12 at the actual target location 18 as indicated by the dashed line 19.

[0018] The handheld object 12 may represent or include any suitable object that the user 10 can use to aim or point at the target 14, such as a toy or model of a stick, pencil, gun, or weapon, a magic wand, etc. The handheld object 12 may include a reference element 20 that facilitates determining where the user 10 is aiming. Specifically, the camera 22 can detect the position of the reference element 20, and the reference element 20 may be made of a material or device that makes it easier for the camera 22 to detect it. For example, the reference element 20 may be made of a reflective material (e.g., a reflective glass bead, a microprism, or an encapsulated lens sealed to a fabric or plastic substrate), a metal strip, etc. In another example, the reference element 20 may include an identifier (e.g., a unique graphic design, a barcode, a quick-response (QR) code, etc.) that enables the camera 22 to identify the reference element 20. As illustrated, the reference element 20 may be located at an end 24 of the handheld object 12 opposite to end 26, where the user's hand 28 is holding the handheld object 12. This can help determine that the user is holding the object 12 in the direction he is aiming; however, the reference element 20 can be placed on any part of the object 12 or even the user 10.

[0019] Camera 22 can detect the position 30 of reference element 20 relative to two-dimensional plane 32. Position 30 can be used to determine the target position 17 that user 10 perceives as being aimed at or intending to aim at by applying one or more translation factors. As illustrated, two-dimensional plane 32 may share the same plane as structure 15; however, in some embodiments, two-dimensional plane 32 and structure 15 may not share the same plane. For example, two-dimensional plane 32 and structure 15 may be parallel to each other. Furthermore, in order for camera 22 to detect the position 30 of reference element 20, structure 15 may be made translucent, transparent, or include any other suitable properties that enable camera 22 to detect the position 30 of reference element 20.

[0020] Specifically, one or more translation factors may be applied to the position 30 of the reference element 20 to compensate for the discrepancy between the user's perception of where they are aiming the handheld object 12 and the camera's determination of where the reference element 20 is located on the two-dimensional plane 32. One or more translation factors may be determined during a calibration process in which the user 10 aims their handheld object 12 at a calibration point, and the camera 22 detects this initial position of the reference element 20 on the two-dimensional plane 32. One or more translation factors may represent one or more distances by which the initial position is shifted to result in the calibration point (e.g., relative to the two-dimensional plane 32). Additionally, one or more translation factors may mitigate or compensate for one-eye dominance over another (e.g., right-eye dominance or left-eye dominance), head tilt, weight shift, tilting to one side or the other, etc.

[0021] Furthermore, one or more scaling factors can be applied to the position 30 of the reference element 20 to account for or compensate for differences in user arm lengths. That is, users use their arms to move and aim the handheld object 12, with the arm acting as the radius or cross-section of a sphere, where their shoulder is the center of the sphere. When users move the handheld object 12 or aim at different targets, the corresponding position of the reference element 20 of the handheld object 12 may differ between users, even when aiming at the same target, due to the different arm lengths of the users.

[0022] Therefore, the height of reference element 20 (e.g., above the ground) can be determined based on the initial position of reference element 20, and the user height can be estimated based on the height of reference element 20. The user's arm length can be estimated from the user height, and the user's arm length can be used to determine one or more scaling factors. One or more scaling factors can be used to scale or multiply the position 30 of reference element 20 detected by camera 22 on two-dimensional plane 32.

[0023] Additionally, one or more offsets can be applied to the position 30 of the reference element 20 to generate a projected target position for the handheld object 12, compensating for distortion caused by the arcuate or circular movement of the user's arm. That is, this distortion may be caused by a shape difference between the arcuate movement and the camera's detection of the position 30 of the reference element 20 on the flat two-dimensional plane 32. One or more offsets can shift the projected target position to increase or lengthen the distance between the projected target position and the initial position, in order to compensate for the shape difference between the arcuate nature of the user's arm movement and the flat two-dimensional plane. For example, one or more offsets can be determined using polynomial regression that fits the test data to a polynomial equation (such as a third-order polynomial equation).

[0024] In this way, a projected target position for the handheld object 12 can be generated that highly matches the target position 17 that the user 10 perceives as where they are aiming the handheld object 12. Advantageously, unlike some other systems, only a single calibration point is used to determine the translation factor, scaling factor, and offset, and the projected target position of the handheld object 12 is accurately determined. However, in other applications (e.g., aiming devices used in performances), reducing calibration time may be less important because calibration can occur before the actual performance (e.g., during the preparation phase) and will not be observed by the audience or customers. However, in the current situation (e.g., at a theme park attraction), creating an immersive user experience by hiding the calibration process or preventing the user 10 from noticing it is in progress may be important. Accordingly, reducing the calibration process to a single point (e.g., aiming the handheld object 12 at a single calibration point) can be used to enhance or improve the user experience.

[0025] Considering this, Figure 2 This is a block diagram of a theme park attraction system 40 according to an embodiment of the present disclosure. The theme park attraction system 40 enables a user 10 to aim a handheld object 12 at various targets 14, and outputs a user interaction experience based on determining that the user 10 is aiming the handheld object 12 at the target 14. For example, the theme park attraction system 40 may include settings with characters popular with children, settings themed around television or movies, shooting ranges, sets of targets, etc.

[0026] The theme park attraction system 40 may include a handheld object 12 with a reference element 20, which is held and manipulated by the user 10. The theme park attraction system 40 may also include a user interaction system 42, which includes a camera 22 that detects the position of the reference element on a two-dimensional plane 32. The theme park attraction system 40 may also include a projection position determination system 44 that determines the projection target position of the handheld object 12. Specifically, the projection target position can represent the position on the two-dimensional plane 32 where the user 10 perceives that they are aiming at or intend to aim at. In fact, the closer the projection target position is to the target position 17, the more accurate the projection target position.

[0027] The projection position determination system 44 may include a controller 46 having one or more processors (described as a single processor 48) and one or more memories or storage devices (described as a single memory device 50). The processor 48 may execute software programs and / or instructions stored in the memory device 50, which facilitate the determination of the projection target position of the handheld object 12. Furthermore, the processor 48 may include multiple microprocessors, one or more "general-purpose" microprocessors, one or more application-specific microprocessors, and / or one or more application-specific integrated circuits (ASICs). For example, the processor 48 may include one or more Reduced Instruction Set Computing (RISC) processors. The memory device 50 may store information such as control software, lookup tables, configuration data, etc. The memory device 50 may include tangible, non-transitory machine-readable media, such as volatile memory (e.g., random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM)), flash memory, one or more hard disk drives, and / or any other suitable optical, magnetic, or solid-state storage media. The memory device 50 may store a wide variety of information and may be used for a variety of purposes, such as instructions facilitating the projection target position of the handheld object 12.

[0028] The projection position determination system 44 may further include reference element position detection logic 52 for determining the position of reference element 20 on the two-dimensional plane 32. Specifically, the projection position determination system 44 may be communicatively coupled to the user interaction system 42 via any suitable means, such as via wired communication or via a communication network using wireless communication protocols or technologies (e.g., radio, Bluetooth, WiFi, infrared, Ethernet, Thread, ZigBee, Z-Wave, KNX, mobile, and / or microwave). The reference element position detection logic 52 may thus receive an image (e.g., a video) captured by the camera 22 showing the reference element 20 on the two-dimensional plane 32. The reference element position detection logic 52 may determine the position of the reference element 20 on the two-dimensional plane 32 as expressed, for example, by a two-dimensional coordinate system (e.g., x and y).

[0029] The projection position determination system 44 may further include transformation logic 54, which transforms the position of the reference element 20, as determined by the reference element position detection logic 52, into a projection target position relative to the two-dimensional plane 32. The transformation logic 54 includes translation logic 56, which determines one or more translation factors that compensate for the discrepancy between the user's perception of where they are aiming the handheld object 12 and the camera's determination of where the reference element 20 is located on the two-dimensional plane 32.

[0030] Specifically, translation logic 56 can determine one or more translation factors by performing a single-point calibration process. This process includes: receiving a calibration position on the two-dimensional plane 32; receiving the position of the reference element 20 on the two-dimensional plane 32 (e.g., corresponding to when the user 10 aims the handheld object 12 at the calibration position); and determining one or more translation factors based on the positional difference between the calibration position and the position of the reference element 20.

[0031] Figure 3 This is a diagram illustrating, according to an embodiment of the present disclosure, that a user 10 aims a handheld object 12 at a calibration position 80. The calibration position 80 may correspond to a physical object, a drawing, a photograph, a graphic, etc. In some cases, the calibration position 80 may correspond to an image output from a display. The user 10 may be prompted by instructions provided in any suitable format (e.g., written, etched, printed, attached, or displayed on structure 15). The calibration position 80 may be provided to allow the user to similarly position their arm, enabling controlled detection of user height while also allowing… Figure 2 The projection position determination system 44 is able to determine the difference between the user's perception of where they are aiming the handheld object 12 and where the user 10 is actually aiming the handheld object 12. For example, the calibration position 80 may be positioned so that the user 10 can extend their arm 82 as close as possible to parallel to the ground 84, at an angle relative to a plane parallel to the ground, and so on. In some embodiments, the calibration position 80 may be tailored to the user's height. That is, in some embodiments, the calibration position 80 may be positioned lower on the structure 15 for users sitting in a vehicle (such as a wheelchair, personal electric vehicle, stroller, etc.). As another example, the calibration position 80 may be positioned higher on the structure 15 for adults than for children, and the calibration position 80 may be positioned higher on the structure 15 for male users than for female users, and so on.

[0032] Accordingly, calibration position 80 can be predetermined and known by projection position determination system 44. When prompted, user 10 can extend their arm 82 and aim the handheld object 12 at calibration position 80. However, due to distortion effects caused by the human body (such as one eye dominance over the other, head tilt, weight shift, tilting to one side or the other, the user's choice of hand to hold the handheld object 12 (e.g., right hand vs. left hand, physical limitations (e.g., physical limitations affecting range of motion), whether the user's movement may be altered due to obstruction (e.g., carrying a backpack or holding a child), etc.), although the user perceives or intends to aim the handheld object 12 at calibration position 80 as indicated by dashed line 85, user 10 may actually aim the handheld object 12 at another position (e.g., actual calibration position 86) as indicated by dashed line 88.

[0033] Camera 22 detects the position 90 of reference element 20 on two-dimensional plane 32 and sends an indication of position 90 to projection position determination system 44. Translation logic 56 (which may be part of a human interaction model) can then determine the positional difference between the position 90 of reference element 20 and a predetermined calibration position 80, which can be expressed in two-dimensional (e.g., x and y) coordinates. Translation logic 56 can use this difference to generate one or more translation factors, which can be applied to subsequently detected positions of reference element 20 to shift the subsequently detected positions of reference element 20 and determine the subsequent projection target position of handheld object 12 corresponding to the position where user 10 intends to aim handheld object 12. The translation factors can be provided in the form of a transformation matrix, which can be applied to subsequently detected positions of reference element 20 to generate the projection target position of reference element 20, as shown below: Equation 1 Where: x = the horizontal component of the position 90 of the reference element 20 on the two-dimensional plane 32; y = the vertical component of the position 90 of reference element 20 on two-dimensional plane 32; X = the horizontal difference between reference element 20 and calibration position 80 on two-dimensional plane 32; Y = the vertical difference between reference element 20 and calibration position 80 on two-dimensional plane 32; x' = the horizontal component of the projected target position of the handheld object 12 on the two-dimensional plane 32; and y' = the vertical component of the projection target position of the handheld object 12 on the two-dimensional plane 32.

[0034] For example, Figure 4This is a diagram illustrating an example of applying one or more translation factors to a subsequently detected position 120 of reference element 20 according to an embodiment of the present disclosure. As illustrated, during calibration, position 90 of reference element 20 is 2 units (e.g., cm) to the right of calibration position 80 and 1 unit (e.g., cm) upward from calibration position 80. Accordingly, the translation factor may include +2 in the horizontal direction and +1 in the vertical direction. Thus, in the transformation matrix, X may be set to +2, and Y may be set to +1. Translation logic 56 may apply the transformation matrix to the subsequently detected position 120 of reference element 20 (e.g., [4, 2]) to shift the subsequently detected position 120 2 units to the right and 1 unit upward to generate a projected target position 122 at 6 units to the right of calibration position 80 and 3 units upward (e.g., [6, 3]). Therefore, translation logic 56 can compensate for the difference between the user’s perception of where they are aiming the handheld object 12 and the camera’s determination of where the reference element 20 is located on the two-dimensional plane 32.

[0035] Return to Figure 2 The transformation logic 54 may also include scaling logic 58, which determines one or more scaling factors to compensate for differences in user arm lengths. That is, as... Figure 3 As shown, users 10 use their arms 82 to move and aim the handheld object 12. The arms 82 can act as the radius or cross section 92 of a sphere, with their shoulders as the center 94 of the sphere. When users 10 move the handheld object 12 to aim at different targets, the corresponding positions of the reference element 20 of the handheld object 12 may differ between users 10, even when aiming at the same target, due to the different arm lengths of users 10.

[0036] Specifically, scaling logic 58 can determine one or more scaling factors based on the position 90 of reference element 20 detected by camera 22 during the calibration process. The height 96 of camera 22 above ground 84 can be predetermined and known through scaling logic 58. Thus, scaling logic 58 can determine the height 98 of reference element 20 above ground 84 based on the position 90 of reference element 20 and the predetermined height 96. Based on the height 98 of reference element 20, user height estimation logic 60 of scaling logic 58 can determine the user's height 100. Specifically, test or sample data on the position 90 of reference element 20 and the height of user 10 when user 10 aims handheld object 12 at calibration position 80 can be collected. The height 102 of the position 90 of reference element 20 can be correlated with the height of user 10, and scaling logic 58 can estimate the user's height 100 based on this predetermined correlation and the height 98 of reference element 20. The model used to identify the correlation can be populated with a table of standard correlations between height and reach (e.g., the ratio between height and arm length for various body types in a population).

[0037] The scaling logic 58 and the user arm length estimation logic 62 can then estimate the user's arm length 104 based on the user's height 100. This estimation can be based on a predetermined correlation between the arm length 104 and the user height 100 (e.g., an algorithm or table based on empirical data). This predetermined correlation can be determined based on test or sample data, scientific data related to human proportions, and / or any other suitable source.

[0038] Scaling logic 58 can determine one or more scaling factors based on the user's arm length 104. For example, when pointing away from the initial position (e.g., calibration position 80), camera 22 can detect the position of reference element 20 closer to the initial position compared to user 10 with a shorter arm length 104, compared to user 10 with a longer arm length. Accordingly, scaling logic 58 can determine a larger scaling factor for user 10 with a longer arm length 104 compared to user 10 with a shorter arm length 104. Scaling logic 58 can apply one or more scaling factors to subsequently detected positions of reference element 20 to scale (e.g., shrink or enlarge) those positions to generate a projected target position for reference element 20. The scaling factor can include horizontal and vertical components, provided in the form of a transformation matrix, and interpolated into a transformation matrix that includes a translation factor from Equation 1 above, as shown below: Equation 2 Where: k1 = a horizontal scaling factor generated based on the user's arm length of 104; and k2 = Vertical scaling factor generated based on the user's arm length of 104.

[0039] scaling factor k 1 and k 2 The value can be determined based on relevant test or sample data collected from users 10 who aim the handheld object 12 at various targets and the arm length 104 of those users 10. For example, scaling logic 58 can determine the height 98 of the reference element 20 above the ground 84 to be 1.25 meters based on image data received from camera 22 (e.g., a first image or calibration image of the image). User height estimation logic 60 can determine the user's height 100 to be approximately 1.8 meters based on the height 98 of the reference element 20. User arm length estimation logic 62 can determine the user's arm length 104 to be 0.6 meters based on the user's height 100. Scaling logic 58 can then determine the horizontal scaling factor based on the user's arm length 104. k 1 The vertical scaling factor is 1.5. k 2 The value is 1.75. Therefore, scaling logic 58 can generate the transformation matrix in equation 2, where, k 1 =1.5, and, k 2 =1.75, and the projection position determination system 44 can apply the transformation matrix to the subsequently detected position of the reference element 20 to generate a projection target position in which the user 10 aims the handheld object 12, which compensates for the difference in the length of the user's arm 104.

[0040] For example, Figure 5 This is a diagram illustrating an example of applying a scaling factor to a subsequently detected position 120 of reference element 20 according to an embodiment of this disclosure. As illustrated, the subsequently detected position 120 of reference element 20 is 4 units (e.g., cm) to the right of calibration position 80 and 4 units (e.g., cm) upward from calibration position 80 (e.g., [4, 4]). A horizontal scaling factor will be applied. k 1 =1.5 and vertical scaling factor k 2The transformation matrix of Equation 2, with a value of 1.75, is applied to the subsequently detected position 120, causing the subsequently detected position 120 to be scaled horizontally by a factor of 1.5, thus generating a projected target position 130 6 units to the right of the calibration position 80, and vertically scaled by a factor of 1.7, thus generating a projected target position 130 7 units (e.g., [6, 7]) upwards. Therefore, the scaling logic 58 can compensate for differences in the user's arm length 104.

[0041] Return to Figure 2 The projection position determination system 44 may include arc distortion compensation logic 64, which compensates for the shape difference between the arcuate nature 92 of the user's arm movement and the flat two-dimensional plane 32. For example, Figure 6 This is a diagram showing user 10 aiming the handheld object 12 at different targets. As explained, the angle ϴ formed between the first position 140 and the second position 142 of the user's arm 82 is the same as that between the third position 144 and the fourth position 146 of the user's arm 82. However, when viewed and captured by camera 22 on the two-dimensional plane 32, the distance between the first reference element position 148 corresponding to the first position 140 of the user's arm 82 and the second reference element position 150 corresponding to the second position 142 of the user's arm 82 is... h0 Different from (e.g., greater than) the distance between the third reference element position 152 corresponding to the third position 144 of the user's arm 82 and the fourth reference element position 154 corresponding to the fourth position 146 of the user's arm 82. h1 .

[0042] Accordingly, the arc distortion compensation logic 64 can determine one or more offsets to be applied to the projected target position to compensate for the distortion. These offsets can shift the projected target position to increase or lengthen the distance between the projected target position and the initial position (e.g., calibration position 80) to compensate for the shape difference between the arcuate nature 92 of the user's arm movement and the flat two-dimensional plane 32. For example, these offsets can be determined using regression analysis, which fits test or sample data from when the user 10 aims the held object 12 at various targets (e.g., where the reference element 20 is along arc 92) to an equation. In some embodiments, the arc distortion compensation logic 64 can fit the test data to a polynomial equation (e.g., a third-order polynomial equation); however, any suitable order or type of equation can be used. For example, a first third-order polynomial equation (Equations 3 and 4 below) can be used to determine a horizontal offset to be applied to the projected target position to compensate for the distortion in the horizontal direction, and a second third-order polynomial equation (Equations 5 and 6 below) can be used to determine a vertical offset to be applied to the projected target position to compensate for the distortion in the vertical direction. Equation 3 (It may also be expressed as:) Equation 4 Equation 5 (It may also be expressed as:) Equation 6, Where: x 偏移 = The horizontal offset applied to the projected target position; y 偏移 =Vertical offset to be applied to the projected target position; x = the horizontal component of the projected target position; y = the vertical component of the projected target position; and a i b i c i a, b, c, d, e, f, g, h, k, and l are constants determined using regression analysis, where each constant may vary depending on the equation (e.g., the constants in equation 4). a The constants may differ from those in Equation 6. a ).

[0043] The horizontal component of the projected target position can be measured as the horizontal distance away from the initial position (e.g., corresponding to calibration position 80 and / or when user 10 aims the handheld object 12 directly at camera 22), while the vertical component of the projected target position can be measured as the vertical distance away from the initial position. As previously mentioned, for any of the polynomial equations 3-6, the constant a i b i c i a, b, c, d, e, f, g, h, k, and l can be determined by fitting the test or sample data to a polynomial equation using polynomial regression analysis (and possibly different between the equations). Accordingly, as user 10 moves and aims the handheld object 12, one or more offsets can be determined for each projected target position.

[0044] However, applying any of Equations 3-6 to determine horizontal and vertical offsets for each projected target position when the user 10 moves and aims the handheld object 12 can be time-consuming and consume excessive computational resources (e.g., processing, memory, storage devices, or networked resources). Accordingly, to more efficiently compensate for the shape difference between the arcuate nature 92 of the user's arm movement and the flat two-dimensional plane 32, in some embodiments, the arcuate distortion compensation logic 64 can divide the arc 92 where the reference element 20 can be located into multiple reference element regions, each of which can correspond to a corresponding projected target region (e.g., projected onto a two-dimensional plane). Each projected target region can correspond to a corresponding set of polynomial equations that can accurately compensate for distortions applicable to that projected target region. Accordingly, the camera 22 can detect the reference element 20 in the reference element region, the arcuate distortion compensation logic 64 can determine the corresponding projected target region corresponding to the reference element region, and the arcuate distortion compensation logic 64 can apply the corresponding set of polynomial equations corresponding to the corresponding projected target region to the position of the reference element to determine one or more offsets to be applied to the reference element to compensate for the distortion. In such embodiments, the multiple reference element regions may be of different sizes (e.g., the farther the reference element region is from the two-dimensional plane 32, the smaller the reference element region becomes), while the multiple projection target regions are of the same size, or the multiple reference element regions may be of the same size, while the multiple projection target regions are of different sizes (e.g., the farther the projection target region is from the reference element 20, the larger the projection target region becomes).

[0045] Figure 7This is a diagram of multiple reference element regions 170 of different sizes and multiple projection target regions 172 of uniform size according to embodiments of the present disclosure. As illustrated, the first reference element region 174, closest to the two-dimensional plane 32, is the largest in size, followed by the second reference element region 176, which is also closest to the two-dimensional plane 32 and is the largest in size (but smaller than the first reference element region 174), followed by the third reference element region 178, which is also closest to the two-dimensional plane 32 and is the largest in size (but smaller than the second reference element region 176), and finally, the fourth reference element region 180, which is also closest to the two-dimensional plane 32 and is the largest in size (but smaller than the third reference element region 178). Although in Figure 7 Four reference element regions 170 are described, but it should be understood that any suitable number of reference element regions 170 of any suitable size can be imagined, wherein the farther the reference element region 170 is from the two-dimensional plane 32, the smaller the reference element region 170 becomes in size. Furthermore, each projection target region 172 is the same size as the other projection target regions 172, corresponds to the corresponding reference element region 170, and corresponds to a corresponding system of polynomial equations that generate the corresponding offsets (e.g., horizontal and vertical offsets) that can be applied to the position of the reference element 20. In particular, each system of polynomial equations corresponding to the corresponding projection target region 172 can be used with respect to a constant a. i b i c i a, b, c, d, e, f, g, h, k, and l have different sets of values, as provided in any of equations 3-6 (and may differ between equations). The farther the reference element region 170 is from the two-dimensional plane 32, the smaller the size of the reference element region 170 becomes, while maintaining the same size of the projection target region 172. This allows the arc distortion compensation logic 64 to compensate for the shape difference between the arcuate nature 92 of the user's arm movement and the flat two-dimensional plane 32 in an efficient and resource-saving manner.

[0046] Figure 8 This is a diagram of a plurality of reference element regions 190 of uniform size and a plurality of projection target regions 192 of different sizes according to embodiments of the present disclosure. As illustrated, each reference element region 190 is of the same size. However, the first projection target region 194, closest to the reference element 20, is the smallest in size, followed by the second projection target region 196, which is also closest to the reference element 20 and is the smallest in size (but larger than the first projection target region 194), followed by the third projection target region 198, which is also closest to the reference element 20 and is the smallest in size (but larger than the second projection target region 196), and finally, the fourth projection target region 200, which is also closest to the reference element 20 and is the smallest in size (but larger than the third projection target region 198). Although in Figure 8Four projection target regions 192 are described, but it should be understood that any suitable number of projection target regions 192 of any suitable size can be imagined, wherein the farther the projection target region 192 is from the reference element 20, the larger the size of the projection target region 192 becomes. Each projection target region 192 corresponds to a corresponding reference element region 190, and also to a corresponding system of polynomial equations that generate a corresponding offset (e.g., horizontal and vertical offset) that can be applied to the position of the reference element 20. In particular, each system of polynomial equations corresponding to a corresponding projection target region 192 can be used with respect to a constant a. i b i c i a, b, c, d, e, f, g, h, j, k, and l have different sets of values, as provided in any of equations 3-6 (and may differ between equations). The farther the projection target area 192 is from the reference element 20, the larger the size of the projection target area 192 becomes, while maintaining the same size as the reference element area 190. This allows the arc distortion compensation logic 64 to compensate for the shape difference between the arcuate nature 92 of the user's arm movement and the flat two-dimensional plane 32 in an efficient and resource-saving manner.

[0047] It should be noted that, for the sake of simplicity, Figure 6-8 This describes distortion only along the vertical (e.g., y) direction caused by the difference in shape between the arcuate nature 92 of the user's arm movement and the flat two-dimensional plane 32. However, the currently disclosed systems and methods envision compensating for distortion along any suitable direction: including the horizontal (e.g., x) direction, as shown by Equations 3 and 4, which provide a horizontal offset to compensate for distortion along the horizontal direction; and the vertical (e.g., y) direction, as shown by Equations 5 and 6, which provide a vertical offset to compensate for distortion along the vertical direction.

[0048] Return to Figure 2If the projection position determination system 44 determines that the projection target position corresponds to a target 14 printed, etched, written, attached, or otherwise displayed on structure 15, then the output device 66 of the user interaction system 42 can output a user interaction experience. The output device 66 can be any suitable device capable of outputting the desired user interaction experience, such as an electronic display, speaker, virtual reality device, augmented reality device, actuator, and / or animation device (e.g., a robot figure). The target 14 can be part of, fixed to, attached to, or include the output device 66, or the target 14 can be detached from the output device 66. For example, in a theme park setting, both the target 14 and the output device 66 can be animated objects of the attraction, and, in response to determining that the projection target position corresponds to the animated object, the animated object can output a user interaction experience (e.g., wagging a tail). As another example, the target 14 can be a word printed on a poster, and the output device 66 can be a nearby speaker, and, in response to determining that the projection target position corresponds to a word printed on the poster, the nearby speaker can output the speech of saying that word. As another example, target 14 may be an image of a person on an electronic display, and output device 66 may be an electronic display, and in response to determining that the position of the projection target corresponds to the image of a person, the electronic display may play a video showing the person in the image performing a signature action.

[0049] Considering this, Figure 9 This is a flowchart of a process 210 for determining the projection target position of a handheld object 12 according to embodiments of the present disclosure. Process 210 can be performed by any suitable means capable of determining the projection target position of the handheld object 12 (such as any component of the projection position determination system 44, including controller 46, processor 48, reference element position detection logic 52, transformation logic 54, translation logic 56, scaling logic 58, user height estimation logic 60, and / or user arm length logic 62). Although process 210 is described using steps performed in a specific sequence, it should be understood that the steps described herein are contemplated to be performed in a different sequence than described, and that some described steps may be omitted or not all performed together. In some embodiments, process 210 can be implemented by using a processor (such as processor 48) to execute instructions stored in a tangible, non-transitory computer-readable medium (such as memory device 50).

[0050] As illustrated, in process block 212, processor 48 receives an instruction to calibrate handheld object 12. This instruction may be in the form of an image captured by camera 22 (e.g., a first image or calibration image) that includes the presence of reference element 20 of handheld object 12. In some embodiments, this instruction may be provided by a motion sensor or other suitable sensor capable of indicating that user 10 has entered the viewing area of ​​camera 22 with handheld object 12 having reference element 20.

[0051] In process block 214, processor 48 receives calibration position 80. In particular, calibration position 80 can be predetermined and known to processor 48, because calibration position 80 can be fixed to structure 15 or displayed by processor 48 on structure 15.

[0052] In process block 216, processor 48 receives the position of reference element 20 of handheld object 12. For example, camera 22 can provide an image of reference element 20 (e.g., a second or subsequent image of an image captured by camera 22). Processor 48 can then instruct reference element position detection logic 52 to determine the position of reference element 20 on two-dimensional plane 32.

[0053] In process block 218, processor 48 instructs translation logic 56 to determine one or more translation factors based on the position of reference element 20 and calibration position 80. One or more translation factors can compensate for the difference between the user's perception of where they are aiming the handheld object 12 and the camera's determination of where the reference element 20 is located on the two-dimensional plane 32. Specifically, translation logic 56 can determine one or more translation factors by performing a single-point calibration procedure. This procedure includes: receiving a calibration position on the two-dimensional plane 32; receiving the position of reference element 20 on the two-dimensional plane 32 (e.g., corresponding to when user 10 aims the handheld object 12 at the calibration position); and determining one or more translation factors based on the positional difference between the calibration position and the position of reference element 20.

[0054] Translation logic 56 can use this difference to generate one or more translation factors, which can be applied to subsequently detected positions of reference element 20 to shift those positions and determine the subsequent projected target position of the handheld object 12 corresponding to the position where user 10 intends to aim the handheld object 12. This translation factor can be provided in the form of a transformation matrix, which can be applied to subsequently detected positions of reference element 20 to generate the projected target position of reference element 20, as shown in Equation 1.

[0055] In process block 220, processor 48 instructs user height estimation logic 60 to determine the height 100 of user 10 based on the position of reference element 20. In process block 222, processor 48 instructs user arm length estimation logic 62 to determine the arm length 104 of user 10 based on the height 100 of user 10.

[0056] In process block 224, processor 48 instructs scaling logic 58 to determine one or more scaling factors based on the user 10's arm length 104. Scaling logic 58 may provide scaling factors as a transformation matrix of Equation 2 as shown above. The scaling factors may compensate for differences in the user's arm length 104 by scaling (e.g., multiplication) at the position of reference element 20 relative to an initial position (e.g., calibration position 80).

[0057] In process block 226, processor 48 instructs transformation logic 54 to determine the projection target position of the handheld object 12 based on the position of reference element 20, one or more translation factors, and one or more scaling factors. Specifically, transformation logic 54 can apply the transformation matrix of Equation 2, which includes one or more translation factors and one or more scaling factors, to the position of reference element 20 to generate the projection target position. That is, the projection target position can correspond to the position where user 10 perceives that they are aiming at or intend to aim at.

[0058] In decision box 228, processor 48 determines whether the location of the projected target is relevant to a user interaction element. The user interaction element can be any suitable target used as a trigger to implement a user interaction experience. For example, a user interaction element can include any feature of interest that user 10 might expect to trigger a user interaction experience when aiming with handheld object 12.

[0059] If the processor 48 determines that the projection target position is related to the user interaction element, then in process block 230, the processor 48 instructs the user interaction system 42 to use the appropriate output device 66 to implement the corresponding user interaction experience. For example, the output device 66 may be an animated object of a scenic spot, and the user interaction system 42 may cause the animated object to bark, meow, speak, move, blink, etc. As another example, the output device 66 may be a speaker, and the user interaction system 42 may cause the speaker to output sound, speech, music, etc. As yet another example, the output device 66 may be an electronic display, and the user interaction system 42 may cause the electronic display to display images, play videos, etc.

[0060] If the processor 48 determines that the projection target position is not related to the user interaction element, then in decision box 232, the processor 48 determines whether the next position of the reference element 20 has been received. If so, the processor 48 repeats process box 226 and determines the projection target position of the handheld object 12 based on the next position of the reference element 20 and the translation and scaling factors already determined from process boxes 218 and 224.

[0061] If the processor 48 determines that the next position of the reference element 20 has not yet been received, the processor 48 repeats process block 212 to receive the next instruction to calibrate the handheld object 12 (e.g., from the next user 10). In this way, process 210 can use single-point calibration to determine the projection target position of the handheld object 12 (e.g., without requiring the user 10 to aim the handheld object 12 at more than one point used to calibrate the projection position determination system 44), which compensates for two differences: the difference between the user's perception of where they are aiming the handheld object 12 and the camera's determination of where the reference element 20 is located on the two-dimensional plane 32, and the difference in terms of the user's arm length 104.

[0062] Furthermore, the projection position determination system 44 can also compensate for distortions caused by the shape difference between the arcuate nature 92 of the user's arm movement and the flat two-dimensional plane 32, such as... Figure 6 As explained in the document. Figure 10 This is a flowchart of process 240 for compensating for the distortion according to an embodiment of the present disclosure. Process 240 can be implemented by any suitable means capable of compensating for the distortion, such as any component of the projection position determination system 44, including controller 46, processor 48, and / or arc distortion compensation logic 64. Although process 240 is described using steps performed in a specific sequence, it should be understood that the steps described herein are contemplated to be performed in a different sequence than that described, and that some described steps may be omitted or not all performed together. In some embodiments, process 240 can be implemented by using a processor (such as processor 48) to execute instructions stored in a tangible, non-transitory computer-readable medium (such as memory device 50).

[0063] As illustrated, in process block 242, processor 48 receives the position of reference element 20 of handheld object 12. In some embodiments, processor 48 may receive the projection target position of handheld object 12.

[0064] In process block 244, processor 48 determines the horizontal offset based on the position of reference element 20 and the first polynomial equation. Specifically, processor 48 may receive the projected target position of handheld object 12 or use... Figure 9The process 210 determines the position of the projection target. The processor 48 can then instruct the arc distortion compensation logic 64 to apply polynomial equation 3 or 4 to the position of the projection target of the handheld object 12 to determine the horizontal offset.

[0065] In process block 246, processor 48 determines the vertical offset based on the position of reference element 20 and a second polynomial equation. Specifically, processor 48 can instruct arc distortion compensation logic 64 to apply polynomial equation 5 or 6 to the projected target position of handheld object 12 to determine the vertical offset.

[0066] In process block 248, processor 48 determines the projection target position of the handheld object 12 based on the position, horizontal offset, and vertical offset of reference element 20. Specifically, processor 48 can instruct arc distortion compensation logic 64 to apply (e.g., add) the horizontal offset to the horizontal component (e.g., x-coordinate) of the projection target position and apply (e.g., add) the vertical offset to the vertical component (e.g., y-coordinate) of the projection target position to generate the projection target position.

[0067] In some embodiments, to more efficiently compensate for the shape difference between the arcuate nature 92 of the user's arm movement and the flat two-dimensional plane 32, the arcuate distortion compensation logic 64 can divide the arc 92 where the reference element 20 can be located into a plurality of reference element regions, each of which can correspond to a corresponding projection target region (e.g., projected onto the two-dimensional plane). Each projection target region can correspond to a corresponding set of polynomial equations that can accurately compensate for distortions applicable to that projection target region. Accordingly, the camera 22 can detect the reference element 20 in the reference element regions, the arcuate distortion compensation logic 64 can determine the corresponding projection target region corresponding to the reference element region, and the arcuate distortion compensation logic 64 can apply the corresponding set of polynomial equations corresponding to the corresponding projection target region to the position of the reference element to determine the position to be applied to the reference element to compensate for one or more offsets of the distortion. In such embodiments, the plurality of reference element regions can be of different sizes (e.g., the reference element region decreases in size the farther it is from the two-dimensional plane 32), while the plurality of projection target regions are of the same size, such as Figure 7 As shown, multiple reference element areas may be of the same size, while multiple projection target areas may be of different sizes (e.g., the further the projection target area is from reference element 20, the larger the projection target area becomes), such as Figure 8 As shown in the diagram.

[0068] In this way, process 240 can compensate for the arcuate nature 92 of the user's arm movement and the flat two-dimensional plane 32. Furthermore, to compensate for the differences between the user's perception of where they are aiming the handheld object 12 and the camera's determination of where the reference element 20 is located on the two-dimensional plane 32, the differences in the length of the user's arm 104, and the differences in shape between the arcuate nature 92 of the user's arm movement and the flat two-dimensional plane 32, Figure 10 The process 240 can be performed Figure 9 The process was implemented before 210, in Figure 9 The process 210 is implemented or used as a subsequent step. Figure 9 This is part of the process 210.

[0069] While the embodiments set forth in this disclosure may be susceptible to various modifications and alternatives, specific embodiments have been shown by way of example in the accompanying drawings and have been described in detail herein. However, it should be understood that this disclosure is not intended to be limited to the particular forms disclosed. This disclosure will cover all modifications, equivalents, and alternatives falling within the spirit and scope of this disclosure as defined by the following appended claims.

[0070] The techniques proposed and claimed herein are referenced and applied to substantial objects and specific examples of practical nature, which arguably improve upon the art and are therefore not abstract, intangible, or purely theoretical. Furthermore, if any claim appended to this specification contains one or more elements designated as “component for [implementing]…[function]” or “step for [implementing]…[function]”, such elements are intended to be interpreted according to 35 USC 112(f). However, for any claim containing elements designated in any other manner, such elements are intended not to be interpreted according to 35 U.SC 112(f).

Claims

1. A method comprising: Receive the initial position of the reference element of the handheld object projected onto the two-dimensional plane in three-dimensional space; A first offset is determined along a first axis of the two-dimensional plane, the first offset compensating for distortion along the first axis associated with the arcuate movement of the handheld object in the three-dimensional space; A second offset is determined along a second axis of the two-dimensional plane, the second offset compensating for distortion along the second axis associated with the arc; The projection target position of the handheld object is determined based on the initial position of the reference element, the first offset, and the second offset; as well as In response to determining that the projected target position corresponds to the target position, the user interaction experience is output.

2. The method according to claim 1, wherein, Determining the projection target position includes shifting the initial position of the reference element along the first axis by the first offset.

3. The method according to claim 1, wherein, The first offset along the first axis includes a horizontal offset on the two-dimensional plane.

4. The method according to claim 1, wherein, Determining the projection target position includes shifting the initial position of the reference element along the second axis by the second offset.

5. The method according to claim 1, wherein, The second offset along the second axis includes a vertical offset in the two-dimensional plane.

6. The method according to claim 1, wherein, When the handheld object is moved across the three-dimensional space, the arc is at least partially defined by the length of the user's arm.

7. The method according to claim 1, wherein, Determining the first offset includes performing regression analysis to fit the test data to a first polynomial equation, and wherein determining the second offset includes performing regression analysis to fit the test data to a second polynomial equation.

8. The method according to claim 7, wherein, The first polynomial equation and the second polynomial equation are both of the third order.

9. A theme park attraction system, comprising: An output device configured to output the user's interactive experience; A controller having one or more processors and a memory storing machine-readable instructions configured to cause the one or more processors to: Determine a first offset along a first axis of a two-dimensional plane, the first offset compensating for distortion along the first axis associated with the arc of movement of the handheld object in three-dimensional space; A second offset is determined along a second axis of the two-dimensional plane, the second offset compensating for distortion along the second axis associated with the arc of the movement of the handheld object in the three-dimensional space; The projection target position of the handheld object is determined based on the initial position of the reference element of the handheld object on the two-dimensional plane, the first offset, and the second offset; as well as In response to determining that the projected target position corresponds to the target position, the output device outputs the user interaction experience.

10. The theme park attraction system according to claim 9, wherein, The first offset is determined based on the distance along the first axis from the initial position of the reference element of the projected target position of the handheld object.

11. The theme park attraction system according to claim 9, wherein, The second offset is determined based on the distance along the second axis from the initial position of the reference element of the projected target position of the handheld object.

12. The theme park attraction system of claim 9, comprising a camera configured to capture an image of the reference element of the handheld object on the two-dimensional plane.

13. One or more tangible, non-transitory computer-readable media, comprising instructions that, when executed by at least one processor, cause the at least one processor to: Receive the initial position of the reference element of the handheld object projected onto the two-dimensional plane; A first offset is determined along a first axis of the two-dimensional plane, the first offset compensating for distortion along the first axis associated with the arcuate movement of the handheld object in three-dimensional space; A second offset is determined along a second axis of the two-dimensional plane, the second offset compensating for distortion along the second axis associated with the arc of the movement of the handheld object in the three-dimensional space; The projection target position of the handheld object is determined based on the initial position of the reference element, the first offset, and the second offset; as well as In response to determining that the projected target position corresponds to the target position, the user interaction experience is output.

14. One or more tangible non-transitory computer-readable media according to claim 13, wherein, The instruction causes the at least one processor to divide the arc into a plurality of reference element regions, wherein each of the plurality of reference element regions is associated with a corresponding target region among a plurality of target regions on the two-dimensional plane.

15. One or more tangible non-transitory computer-readable media according to claim 14, wherein, Each of the plurality of reference element regions is associated with the following: A first equation that compensates for the distortion along the first axis associated with the corresponding target area and the arc in the two-dimensional plane; and The second equation compensates for the distortion along the second axis associated with the corresponding target area and the arc on the two-dimensional plane.

16. One or more tangible non-transitory computer-readable media according to claim 15, wherein, The first equation and the second equation are the third-order polynomial equations.

17. One or more tangible non-transitory computer-readable media according to claim 14, wherein, Each of the plurality of reference element regions is of the same size, and each of the plurality of target regions is of a different size.

18. One or more tangible non-transitory computer-readable media according to claim 14, wherein, The first target region of the plurality of target regions is farther away from the reference element than the second target region of the plurality of target regions, and wherein the first target region is larger in size than the second target region.

19. One or more tangible non-transitory computer-readable media according to claim 14, wherein, Each of the plurality of reference element regions is of a different size, and each of the plurality of target regions is of the same size.

20. One or more tangible non-transitory computer-readable media according to claim 14, wherein, The first reference element region of the plurality of reference element regions is farther away from the two-dimensional plane than the second reference element region of the plurality of reference element regions, and wherein the first reference element region is smaller in size than the second reference element region.

21. A method comprising: A first offset is determined along a first axis of a two-dimensional plane, the first offset compensating for distortion along the first axis associated with the arc by dividing the arc of movement of a reference element of a handheld object in three-dimensional space into a plurality of reference element regions, and wherein each of the plurality of reference element regions is associated with a corresponding target region among a plurality of target regions on the two-dimensional plane; The projection target position of the handheld object is determined based on the initial position of the reference element of the handheld object on the two-dimensional plane and the first offset; and In response to determining that the projected target position corresponds to the target position, the user interaction experience is output.

22. The method of claim 21, further comprising receiving the initial position of the reference element of the handheld object projected onto the two-dimensional plane.

23. The method according to claim 22, characterized in that, include: A second offset is determined along a second axis of the two-dimensional plane, the second offset compensating for distortion along the second axis associated with the arc by dividing the arc in the three-dimensional space into a plurality of additional reference element regions, wherein each of the plurality of additional reference element regions is associated with an additional corresponding target region among a plurality of additional target regions on the two-dimensional plane; as well as The projection target position of the handheld object is determined based on the initial position of the reference element of the handheld object, the first offset, and the second offset.

24. The method according to claim 23, wherein, Each of the plurality of additional reference element regions is of the same size, and each of the plurality of additional target regions is of a different size.

25. The method according to claim 23, wherein, The first additional target region of the plurality of additional target regions is farther away from the reference element than the second additional target region of the plurality of additional target regions, and wherein the first additional target region is larger in size than the second additional target region.

26. The method according to claim 23, wherein, Each of the plurality of additional reference element regions is of a different size, and each of the plurality of additional target regions is of the same size.

27. The method according to claim 23, wherein, The first additional reference element region of the plurality of additional reference element regions is farther away from the two-dimensional plane than the second additional reference element region of the plurality of additional reference element regions, and wherein the size of the first additional reference element region is smaller than that of the second additional reference element region.

28. The method according to claim 23, wherein, Determining the projection target position includes shifting the initial position of the reference element along the first axis by the first offset, shifting the initial position of the reference element along the second axis by the second offset, or both.

29. The method according to claim 23, wherein, The first offset along the first axis includes a horizontal offset in the two-dimensional plane, and the second offset along the second axis includes a vertical offset in the two-dimensional plane.

30. The method according to claim 23, wherein, Determining the first offset includes performing regression analysis to fit the test data to a first equation, and wherein determining the second offset includes performing additional regression analysis to fit the test data to a second equation.

31. The method according to claim 30, wherein, The first equation and the second equation are each a third-order polynomial equation.

32. The method according to claim 31, wherein, The first polynomial equation compensates for the distortion along the first axis associated with the corresponding target area and the arc shape on the two-dimensional plane, and wherein the second polynomial equation compensates for the distortion along the second axis associated with the corresponding additional target area and the arc shape on the two-dimensional plane.

33. The method according to claim 21, wherein, Each of the plurality of reference element regions is of the same size, and each of the plurality of target regions is of a different size.

34. The method according to claim 21, wherein, The first target region of the plurality of target regions is farther away from the reference element than the second target region of the plurality of target regions, and wherein the first target region is larger in size than the second target region.

35. The method according to claim 21, wherein, Each of the plurality of reference element regions is of a different size, and each of the plurality of target regions is of the same size.

36. The method according to claim 21, wherein, The first reference element region of the plurality of reference element regions is farther away from the two-dimensional plane than the second reference element region of the plurality of reference element regions, and wherein the first reference element region is smaller in size than the second reference element region.

37. A system comprising: The output device is configured to output the user interaction experience. A controller having one or more processors and memory, the memory storing machine-readable instructions configured to cause the one or more processors to: A first offset is determined along a first axis of a two-dimensional plane, the first offset compensating for distortion along the first axis associated with the arc by dividing the arc of movement of a reference element of a handheld object in three-dimensional space into a plurality of reference element regions, and wherein each of the plurality of reference element regions is associated with a corresponding target region among a plurality of target regions on the two-dimensional plane; The projection target position of the handheld object is determined based on the initial position of the reference element of the handheld object on the two-dimensional plane and the first offset; and In response to determining that the projected target position corresponds to the target position, the output device outputs a user interactive experience.

38. The system according to claim 37, wherein, The machine-readable instructions are configured to cause the one or more processors to receive the initial position of the reference element of the handheld object projected onto the two-dimensional plane.

39. The system according to claim 38, wherein, The machine-readable instructions are configured to cause the one or more processors to: A second offset is determined along a second axis of the two-dimensional plane, the second offset compensating for distortion along the second axis associated with the arc by dividing the arc in the three-dimensional space into a plurality of additional reference element regions, wherein each of the plurality of additional reference element regions is associated with an additional corresponding target region among a plurality of additional target regions on the two-dimensional plane; and The projection target position of the handheld object is determined based on the initial position of the reference element of the handheld object on the two-dimensional plane, the first offset, and the second offset.

40. One or more tangible, non-transitory computer-readable media, comprising instructions that, when executed by at least one processor, cause the at least one processor to: Receive the initial position of the reference element of the handheld object projected onto the two-dimensional plane; A first offset is determined along a first axis of the two-dimensional plane, the first offset compensating for distortion along the first axis associated with the arc by dividing the arc of the movement of the reference element of the handheld object in three-dimensional space into a plurality of reference element regions; A second offset is determined along a second axis of the two-dimensional plane, the second offset compensating for distortion along the second axis associated with the arc by dividing the arc in the three-dimensional space into a plurality of additional reference element regions, wherein each of the plurality of additional reference element regions is associated with an additional corresponding target region among a plurality of additional target regions on the two-dimensional plane; The projection target position of the handheld object is determined based on the initial position of the reference element of the handheld object on the two-dimensional plane, the first offset, and the second offset; and In response to determining that the projected target position corresponds to the target position, the user interaction experience is output.

Images