Image-based finger tracking and controller tracking
The image-based system addresses hand tracking challenges in VR games by using controller signals to generate accurate virtual hand representations, overcoming occlusion and complex hand movements through machine learning.
Patent Information
- Application Number
- JP2023531025
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2020-11-25
- Filing Date
- 2021-11-22
- Publication Date
- 2024-11-20
- Estimated Expiration
- 2041-11-22
AI Technical Summary
Existing hand tracking systems in virtual reality computer games face challenges in accurately representing the pose of a user's hand when it is partially occluded by a controller or has high degrees of freedom, such as the thumb, leading to blind spots and complex image recognition issues.
An image-based system that uses a camera attached to a head-mounted display to crop the image region containing the controller and hand, generates a virtual representation of the hand based on image analysis and touch signals from the controller, employing machine learning to fill in occluded parts using controller recognition without hand recognition.
The system effectively generates an accurate virtual representation of the hand by leveraging controller signals to overcome occlusion and complex hand movements, enhancing user interaction in VR games.
Smart Images

Figure 0007587698000001 
Figure 0007587698000002 
Figure 0007587698000003
Abstract
Description
[Technical field]
[0001] This application relates generally to image-based finger tracking and controller tracking in computer simulations such as computer games. [Background technology]
[0002] Hand tracking is desirable for applications such as virtual reality (VR) computer games to present a virtualized representation of a user's hands on a display such as a VR head-mounted display (HMD), for example, while playing a computer game in which the user picks up virtual objects.
[0003] Tracking the hands is possible, for example, by using a camera on the HMD, but as understood herein, both recognizing the hands and presenting an accurate virtual representation of their pose is complicated when the user's hands are partially occluded, for example, by the user's gripping of a computer game controller. Tracking the hands based on the sensors of the controller can result in "blind spots" for parts of the hand that are not close to the sensors or that can assume a high degree of freedom of movement, such as the thumb. Summary of the Invention
[0004] Thus, the device includes at least one processor programmed with instructions for identifying an image from at least one camera of a hand gripping a computer game controller, the instructions being executable to crop the image to a region including the controller and the hand, and presenting on a computer-controlled display a virtual representation of the hand generated based at least in part on image analysis of the region and at least one touch signal from the controller.
[0005] In some embodiments, the camera is attached to a head mounted display (HMD). The device may include an HMD. In an exemplary implementation, the touch signal may be generated from the operation of a control key element of the controller and / or from a sensor on the controller other than the control key element of the controller.
[0006] The instructions may be executable to use the touch signal to generate a virtual representation of the portion of the hand obstructed from the camera by the controller. In some embodiments, the instructions may be executable to generate the virtual representation using recognition of the controller without using recognition of the hand, in response to identifying a touch signal from the controller.
[0007] In an embodiment, the instructions may be executable to execute a machine learning (ML) module to generate the virtual representation by modifying key points of the template image based on the image in the region and the touch signal from the controller. The ML model may include at least one neural network (NN) and at least one heat map.
[0008] In another aspect, a method includes identifying an image of a hand gripping a computer simulated controller. The method also includes receiving at least one touch signal from the computer simulated controller and generating and displaying an image of a virtual hand based on both the image of the hand and the touch signal.
[0009] In another aspect, the device includes at least one computer storage, which is not a transitory signal, including instructions executable by at least one processor to receive at least one image of a person's hand gripping a computer game controller, receive at least one touch signal from the controller, generate a virtual hand representing the person's hand based on both the image and the touch signal, and display the virtual hand on at least one computer-controlled display.
[0010] The details of the present application, both as to its structure and operation, can best be understood in reference to the accompanying drawings, in which like reference numerals refer to like parts and in which: [Brief description of the drawings]
[0011] [Figure 1] FIG. 1 is a block diagram of an exemplary system illustrating computer components, some or all of which may be used in various embodiments. [Diagram 2] 1 illustrates an exemplary system showing components of a head mounted display, a computer simulation controller, and a virtual image processor. [Diagram 3] 1 illustrates example logic in an example flow chart form for generating an image of a virtual hand based on both a camera image and touch signals from a controller. [Diagram 5] 1 illustrates various exemplary poses of a person's hands holding an exemplary computer game controller. [Figure 6] 1 illustrates various exemplary poses of a person's hands holding an exemplary computer game controller. [Figure 7] 1 illustrates various exemplary poses of a person's hands holding an exemplary computer game controller. [Figure 8] 1 illustrates various exemplary poses of a person's hands holding an exemplary computer game controller. [Figure 9]1 illustrates various exemplary poses of a person's hands holding an exemplary computer game controller. [Figure 10] 1 shows an image of a hand holding a controller and an associated virtual image of the entire hand. [Figure 11] 1 illustrates an example machine learning (ML) module that can be used to generate a virtual hand image. [Figure 12] An example of a system flow is shown below. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0012] The present disclosure relates generally to computer ecosystems, including, but not limited to, aspects of consumer electronics (CE) device networks, such as computer gaming networks. The systems herein may include server and client components that may be connected over a network such that data may be exchanged between the client and server components. The client components may include one or more computing devices, including gaming consoles such as Sony PlayStation® or gaming consoles made by Microsoft® or Nintendo® or other manufacturers, virtual reality (VR) headsets, augmented reality (AR) headsets, portable televisions (e.g., smart televisions, Internet-enabled televisions), portable computers such as laptops and tablet computers, as well as smartphones and other mobile devices, including additional examples discussed below. These client devices may operate in a variety of operating environments. For example, some of the client computers may employ, by way of example, the Linux® operating system, a Microsoft® operating system, or a Unix® operating system, or operating systems produced by Apple, Inc.® or Google®. These operating environments may be used to run one or more browsing programs, such as browsers created by Microsoft® or Google® or Mozilla®, or other browser programs that can access web sites hosted by the Internet servers discussed below. Operating environments according to the present principles may also be used to run one or more computer game programs.
[0013] The server and / or gateway may include one or more processors that execute instructions that configure the server to send and receive data over a network such as the Internet. Alternatively, the clients and servers may be connected through a local intranet or a virtual private network. The server or controller may be instantiated by a gaming console such as a Sony PlayStation®, a personal computer, or the like.
[0014] Information may be exchanged between the clients and the servers over a network. For this purpose and for security, the servers and / or clients may include firewalls, load balancers, temporary storage, and proxies, as well as other network infrastructure for reliability and security. One or more servers may form an apparatus that implements a method for providing a secure community, such as an online social website, for network members.
[0015] The processor may be a single-chip processor or a multi-chip processor capable of implementing logic through various lines, such as address lines, data lines and control lines, as well as registers and shift registers.
[0016] Components included in one embodiment may be used in other embodiments in any suitable combination. For example, any of the various components described herein and / or illustrated in the figures may be combined, interchanged, or excluded from other embodiments.
[0017] "A system having at least one of A, B, and C" (and similarly "a system having at least one of A, B, or C" and "a system having at least one of A, B, C") includes systems having A alone, B alone, C alone, A and B together, A and C together, B and C together, and / or A, B and C together, etc.
[0018] Referring now specifically to FIG. 1, an exemplary system 10 is shown that may include one or more of the exemplary devices described above and further below in accordance with the present principles. A first of the exemplary devices included in the system 10 is a consumer electronics (CE) device such as an audio-video device (AVD) 12, such as an Internet-enabled television with a television tuner (equivalently, a set-top box that controls the television), without limitation. Alternatively, the AVD 12 may also be a computer-controlled Internet-enabled ("smart") phone, a tablet computer, a notebook computer, an HMD, a wearable computer-controlled device, a computer-controlled Internet-enabled music player, a computer-controlled Internet-enabled headphones, a computer-controlled Internet-enabled implantable device such as an implantable skin device, and the like. It should nevertheless be understood that the AVD 12 is configured to implement the present principles (e.g., to communicate with other CE devices to implement the present principles, to execute the logic described herein, and to perform any other functions and / or operations described herein).
[0019] Thus, to implement such principles, an AVD 12 may be established by some or all of the components shown in FIG. 1. For example, the AVD 12 may include one or more displays 14, which may be implemented by flat screens with high or ultra-high resolution "4K" or higher resolution, and may be touch-enabled to receive user input signals via touching the display. The AVD 12 may include at least one additional input device 18, such as one or more speakers 16 for outputting audio in accordance with the present principles, and an audio receiver / microphone for inputting audible commands to the AVD 12 to control the AVD 12. The exemplary AVD 12 may also include one or more network interfaces 20 for communicating over at least one network 22, such as the Internet, a WAN, a LAN, etc., under the control of one or more processors 24. A graphics processor 24A may also be included. The interface 20 may thus be, without limitation, a Wi-Fi transceiver, which is an example of a wireless computer network interface, such as, without limitation, a mesh network transceiver. It should be understood that processor 24 controls AVD 12, including the other elements of AVD 12 described herein, such as controlling display 14 to present images and receiving input therefrom, to implement the present principles. It should further be noted that network interface 20 may be a wired or wireless modem or router, or other suitable interface, such as, for example, a wireless telephony transceiver or the Wi-Fi transceiver discussed above.
[0020] In addition to the above, the AVD 12 may also include one or more input ports 26, such as, for example, a High Definition Multimedia Interface (HDMI) port or a USB port for physically connecting to another CE device, and / or a headphone port for connecting headphones to the AVD 12 to provide audio from the AVD 12 to a user through the headphones. For example, the input port 26 may be wired or wirelessly connected to a cable or satellite source 26a of audio-video content. Thus, the source 26a may be a separate or integrated set-top box, or a satellite receiver. Alternatively, the source 26a may be a game console or disc player that contains the content. When implemented as a game console, the source 26a may include some or all of the components described below in connection with the CE device 44.
[0021] AVD 12 may further include one or more computer memories 28, such as non-transitory, disk-based or solid-state storage, which may in some cases be embodied within the AVD's chassis as a standalone device, or as a personal video recording device (PVR) or video disk player, or as a removable memory medium, either internal or external to the AVD's chassis for playing AV programs. In some embodiments, AVD 12 may also include a location or position receiver, such as, but not limited to, a cellular receiver, a GPS receiver, and / or an altimeter 30, which is configured to receive geographic location information from a satellite or cellular tower and provide that information to processor 24 and / or determine the altitude at which AVD 12 is located in conjunction with processor 24. Component 30 may also be implemented by an inertial measurement unit (IMU), which typically includes a combination of accelerometers, gyroscopes, and magnetometers, to determine the position and orientation of AVD 12 in three dimensions.
[0022] Continuing with the description of the AVD 12, in some embodiments, the AVD 12 may include one or more cameras 32, which may be a thermal imaging camera, a digital camera such as a webcam, and / or a camera integrated into the AVD 12 and controllable by the processor 24 to collect pictures / images and / or videos in accordance with the present principles. Also included in the AVD 12 may be a Bluetooth transceiver 34 and other NFC elements 36 for communicating with other devices using Bluetooth and / or Near Field Communication (NFC) technologies, respectively. An exemplary NFC element may be a radio frequency identification (RFID) element.
[0023] Furthermore, the AVD 12 may include one or more auxiliary sensors 37 (e.g., motion sensors such as accelerometers, gyroscopes, cyclometers, or magnetic sensors, infrared (IR) sensors, optical sensors, speed and / or cadence sensors, gesture sensors (e.g., for sensing gesture commands)) that provide input to the processor 24. The AVD 12 may include an over-the-air TV broadcast port 38 for receiving OTA (over-the-air) TV broadcasts that provide input to the processor 24. In addition to the above, it is noted that the AVD 12 may also include an infrared (IR) transmitter and / or an IR receiver and / or an IR transceiver 42, such as an infrared data association (IRDA) device. A battery (not shown) may be provided to power the AVD 12, such as a kinetic energy harvester that can convert kinetic energy into electricity to charge the battery and / or power the AVD 12.
[0024] With further reference to FIG. 1, in addition to the AVD 12, the system 10 may include one or more other CE device types. In one embodiment, the first CE device 44 may be a computer game console that can be used to transmit computer game audio and video to the AVD 12 via commands sent directly to the AVD 12 and / or through a server, as described below, while the second CE device 46 may include similar components to the first CE device 44. In the illustrated embodiment, the second CE device 46 may be configured as a computer game controller operated by a player or as a head mounted display (HMD) worn by a player 47. In the illustrated embodiment, only two CE devices 44, 46 are shown, but it will be understood that fewer or more devices may be used. The devices herein may implement some or all of the components shown for the AVD 12. Some or all of the components shown for the AVD 12 may be incorporated into any of the components shown in the following figures.
[0025] Referring now to the at least one server 50 described above, the server includes at least one server processor 52, at least one tangible computer-readable storage medium 54, such as disk-based or solid-state storage, and at least one network interface 56 that, under the control of the server processor 52, enables communication with other devices of Figure 1 over the network 22, and may, in fact, facilitate communication between the server and client devices in accordance with the present principles. It should be noted that the network interface 56 may be, for example, a wired or wireless modem or router, a Wi-Fi transceiver, or other suitable interface, such as, for example, a wireless telephony transceiver.
[0026] Thus, in some embodiments, server 50 may be an entire Internet server or server "farm" and may include "cloud" functionality and perform such functionality such that devices of system 10 may access the "cloud" environment via server 50, for example, in the exemplary embodiment of a network gaming application. Alternatively, server 50 may be implemented by one or more game consoles, or other computers, in the same room or nearby as the other devices shown in FIG.
[0027] Figure 2 illustrates a display device 200, such as a virtual reality (VR) head mounted display (HMD), which may incorporate any or all of the components discussed above in connection with Figure 1, as well as other components illustrated in Figure 2. The exemplary HMD 200 illustrated in Figure 2 may include one or more video displays 202 controlled by one or more processors 204 that may wirelessly communicate with other components using one or more wireless network interfaces 206. The HMD 200 may also include one or more outward-facing cameras 208 for imaging objects, such as the hands of a wearer of the HMD 200.
[0028] The HMD 200 may be used to play video games executed by a source 210, such as a video game console and / or a remote server, under the control of one or more handheld controllers 212. The controllers 212 may include one or more operable control keys 214 for controlling the play of a game or simulation, each of which may be associated with one or more sensors 216 to generate a signal representative of the manipulation or contact of the associated control key 214. The controllers 212 may also include one or more non-control key sensors 218 not associated with a control key but located at known locations on the controller 212 to sense hand contact or proximity with the sensor 218 and provide a signal indicative thereof. The controllers 212 may include one or more processors 220 configured to transmit signals from the sensors 216, 218 and the control keys 214 to other components using one or more network interfaces 222. The controllers may also include one or more position sensors 223, such as inertial sensors, global positioning satellite sensors, accelerometers, magnetometers, gyroscopes, and combinations thereof.
[0029] One or more processors 224, such as any of the processors described herein, can receive signals from other components of FIG. 2 and can access instructions on one or more computer storage devices 226 to execute the logic described consistently herein consistent with the present principles.
[0030] An example of such logic is shown in Figure 3 and may be executed by any processor or combination of processors described herein. Beginning at block 300, an image of a controller 212, which may be held by a human hand, is received, for example, from the camera 208. Moving to decision diamond 302, it is determined whether a signal is received from any of the controller sensors 216, 218 indicating that the controller is actually being held. If not, the logic may move to block 304 to perform image recognition of the hand in free space and generate a virtual representation of the hand based solely on the image of the hand.
[0031] However, if it is determined that the controller is being held because a signal from the controller's sensor indicates that the controller is being held, the logic proceeds to block 306 to crop the image to an image of only the area of the controller and surrounding objects that may be inferred to be hands. In this manner, image recognition may be performed solely to recognize the controller, rather than more complex image recognition to recognize hands. The remaining image area after cropping may be processed through super-resolution as needed to enhance image details.
[0032] Proceeding to block 308, the cropped image may be analyzed to determine the pose of the hand based on both the image and the signals from the controller sensors 216, 218. Generally, portions of objects that surround the image of the controller and are inferred to be part of a human hand may be used to render a portion of a virtual image of the hand, and signals from the controller representing the contact points are used to "fill in" the invisible portions of the hand that are behind the image of the controller.
[0033] The frame of reference of the visible portion of the hand can be registered to the frame of reference of the controller in a variety of ways. For example, the position of the controller 212 may be derived from signals from the position sensor 223 and the visible portion of the hand registered using machine vision to the positions of the sensors 216, 218 indicating touch, and transformed to the frame of reference of the controller as represented by the position sensor 223. Alternatively, machine vision may be used to define a frame of reference based on the center of gravity of the image of the hand, and the position of the controller may be registered to the center of gravity based on the positions of the sensors 216, 218 indicating touch.
[0034] Furthermore, when a machine learning (ML) module is used as described further below, the model may be trained on a ground truth image of a hand holding a controller along with accompanying ground truth sensor signals and a resulting ground truth virtual full hand image that corresponds to the partial hand image holding the controller and the corresponding sensor signals.
[0035] 3 illustrates that a cropped region of the controller including the hand may be input to the ML module at block 308, and corresponding touch signals from the controller sensors 216, 218 generated at the same time the image was generated may be input to the ML module at block 310. The ML module uses both the sensor signals and the controller / hand image to output a virtual image of a complete hand in the same pose as if gripping the controller within the cropped region generated at block 306 at block 312. The virtual image is presented on a display, such as the HMD 200, at block 314.
[0036] 5-9 show cropped images 500-900 of hands 502-902 holding controllers 504-904, respectively, which may represent ground truth images for training and actual post-training cropped images generated in block 306 of FIG. 3.
[0037] FIG. 10 shows a cropped controller / hand image 1000 and a resulting full virtual hand image 1002 generated using image 1000 and sensor signals, which may represent ground truth input during ML module training or may be shown as an example of the virtual hand image output in block 312 of FIG. 3.
[0038] 11 illustrates an exemplary ML module or engine 1100 that can be used when it is not necessary to use an initial hand detection 1102. Instead, as previously described, left and right keypoint estimation stages 1104, 1106 (details of the right stage 1104 are shown for clarity only) can receive multiple images 1108 of hands holding a controller, cropped as needed and up-resolutioned as needed using super-resolution, according to principles described elsewhere herein. The images 1108 can be processed through a key neural network 1110, such as but not limited to a convolutional neural network (CNN).
[0039] The key NN 1110 generates both two-dimensional (2D) and 1D heatmaps 1112, 1114 from which keypoints 1116 are derived for modifying the pose of a template hand 1118 according to the keypoints 1116. The model parameters are learned by optimizing the minimum E(θ). This is just one of the heatmap techniques that can be used.
[0040] This generates a virtual image of the entire first hand 1120. A controller sensor signal 1122 from the controller in image 1108 is fed back to key NN 1110, as shown by line 1124, and / or the controller touch input signal 1122 may be fed directly to key NN 1108 along with image 1108.
[0041] Regarding the exemplary heatmap technique described herein, in one non-limiting implementation, K heatmaps of size W0×H0, {H1, H2, . . . , Hk} can be estimated, where each heatmap Hk indicates the location confidence of the kth keypoint of the virtual hand to be rendered (K keypoints in total). “Efficient Object Localization Using Convolutional Networks”, Tompson et al., arXiv:1411.4280v3 (June, 2015) describes an approach to generate heatmaps by processing images in parallel at multiple resolution banks to simultaneously capture features at different scales. The output is a discrete heatmap rather than a continuous regression. The heatmap predicts the probability of articulation at each pixel. A sliding window detector is implemented using a multi-resolution CNN architecture (a coarse heatmap model), thereby generating a coarse heatmap output. This is just one example of a heatmap technique that can be used.
[0042] 12, there are shown essentially identical system flows 1200, 1202 for multiple images from multiple respective cameras. Accordingly, the system flow 1200 is shown and disclosed in detail.
[0043] An image 1204 is received and a simulated controller 1206 is recognized within a rectangular sub-area 1208 of the image 1204, allowing a cropped image 1210 within the rectangle to be generated. The cropped image 1210 is input to one or more neural network(s) 1212, which also receive controller tracking information 1214. The tracking information 1214 may include the position of the controller 1206 in space, the rotation of the controller, the velocity and acceleration of the controller, and the rotational rate of the controller, as indicated by one or more sensors in the controller, such as an inertial motion unit (IMU), magnetometer, accelerometer, and gyroscope. As previously described, the neural network(s) 1212 may be trained using ground truth images of hands holding the controller and the accompanying controller tracking inputs. It should be noted that, as described elsewhere in this specification and shown in FIG. 12, the neural network(s) 1212 can also receive information from controller input element(s) 1215, such as face buttons and / or joysticks and / or finger sensors and / or grip buttons.
[0044] The neural network 1212 outputs a 2D heatmap 1216 and a 1D heatmap 1218 representing depth, which may result in hand joint positions 1219 in 3D. These parameters are used for skeletal fitting 1220, which is used to present in 3D an image 1222 of the hand pose in image 1204. Skeletal fitting is the result of a calibration process in which fixed bone lengths are obtained. Skeletal fitting tries to minimize the difference (energy) between the heatmap and the joint positions projected onto it, and the change in time with the last frame, resulting in optimal joint rotations for the calibrated skeletal structure.
[0045] If desired, a tracking controller 1214 can be used to project 1224 the image of the rectangle 1208 .
[0046] While the present principles have been described with reference to certain illustrative embodiments, it will be understood that these are not intended to be limiting and that a variety of alternative configurations may be used to implement the subject matter claimed herein.
Claims
1. At least one processor, identifying an image from at least one camera of a hand holding a computer game controller; cropping the image to an area including the computer game controller and the hand; presenting on a computer-controlled display a virtual representation of the hand generated based at least in part on the image analysis of the area and at least one touch signal from the computer game controller; the at least one processor programmed with instructions to perform Including, The instruction: The apparatus is executable to execute a machine learning (ML) module to generate the virtual representation by modifying key points of a template image based on the image within the region and the touch signal from the computer game controller.
2. The apparatus of claim 1 , wherein the at least one camera is mounted on a head mounted display (HMD).
3. The apparatus of claim 2 comprising the HMD.
4. The apparatus of claim 1 , wherein the touch signal is from a control key element of the computer game controller.
5. 2. The apparatus of claim 1, wherein the touch signal is from a sensor on the computer game controller other than a control key element of the computer game controller.
6. The instruction: The apparatus of claim 1 , wherein the apparatus is operable to use the touch signals to generate a virtual representation of the portion of the hand that is obstructed from the camera by the computer game controller.
7. The instruction:
13. The apparatus of claim 1, wherein the apparatus is executable to generate the virtual representation using recognition of the computer game controller without using hand recognition in response to identifying touch signals from the computer game controller.
8. The apparatus of claim 1 , wherein the ML module includes at least one neural network (NN) and at least one heatmap.
9. identifying an image of a hand gripping a computer-simulated controller; cropping the image of the hand gripping the computer simulated controller to an area including the controller and the hand; receiving at least one touch signal from the computer simulation controller; generating and displaying a virtual representation of a hand based on both the image of the hand and the touch signals, where the virtual representation is generated by executing a machine learning (ML) module to modify key points of a template image based on the image in the region and the touch signals from the computer simulation controller; A method comprising:
10. The method of claim 9 , comprising presenting the virtual representation on a computer-controlled display.
11. The method of claim 10 , wherein the computer-controlled display comprises a head-mounted display (HMD). generating said virtual representation using only said region and no other portion of said image of said hand gripping said computer-simulated controller; 10. The method of claim 9, comprising:
13. The method of claim 9 , wherein the touch signal is from a control key element of the computer simulation controller.
14. 10. The method of claim 9, wherein the touch signal is from a sensor on the computer simulation controller other than a control key element of the computer simulation controller.
15. 10. The method of claim 9, comprising using the touch signals to generate a virtual representation of the portion of the hand that is obstructed from a camera view by the computer simulation controller.
16. 10. The method of claim 9, comprising, in response to identifying a touch signal from the computer simulated controller, generating the virtual representation using recognition of the computer simulated controller without using hand recognition.
17. The method of claim 9 , wherein the ML module includes at least one key neural network (NN) and at least one heatmap.
18. At least one computer storage device, which is not a transitory signal, and which is accessed by at least one processor; receiving at least one image of a person's hand holding a computer game controller; cropping the image of the hand gripping the computer game controller to an area including the controller and the hand; receiving at least one touch signal from the computer game controller; generating a virtual representation of a hand based on both the image and the touch signals and displaying the virtual representation of the hand on at least one computer-controlled display, executing a machine learning (ML) module to generate the virtual representation by modifying key points of a template image based on the image within the region and the touch signals from the computer game controller; the at least one computer storage device comprising executable instructions to perform Including, the device.
Citation Information
Patent Citations
Information processing method, device, and program for causing computer to execute the information processing method
JP2018097517A
Rendering of virtual hand pose based on detected hand input
US20190102927A1
Methods of determining handedness for virtual controllers
US20190236344A1
Human pose analysis system and method
WO2020000096A1
