Dynamic Sensor Allocation

Abstract

To provide, although current handheld controllers provide a range of functions, further technical improvements to enhance user experiences.SOLUTION: A method includes receiving data corresponding to one or more objects in proximity to a controller, determining scores for configurations of the controller, ranking the scores of configurations of the controller, selecting a configuration of the controller among the configurations of the controller, and configuring a touch sensor of the controller according to the selected configuration of the controller.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This patent application claims priority to U.S. utility patent application Ser. No. 16 / 223,956, filed Dec. 18, 2018, the entire disclosure of which is incorporated herein by reference. [Background technology]

[0002] Handheld controllers are used, for example, in an array of architectures to provide input to remote computing devices. For example, handheld controllers are utilized in the gaming industry to allow players to interact with personal computing devices running game applications, game consoles, game servers, and / or the like. Handheld controllers can be used in virtual reality (VR) environments to mimic natural interactions such as grasping, throwing, squeezing, and the like. While current handheld controllers offer a range of functionality, further technological improvements can enhance the user experience. [Brief explanation of the drawings]

[0003] [Figure 1] FIG. 1 illustrates a controller with a hand holder in an open position according to an exemplary embodiment of the present disclosure. [Figure 2] FIG. 2 illustrates the controller of FIG. 1 in a user's open, palm-up hand, according to an exemplary embodiment of the present disclosure. [Figure 3] FIG. 3 illustrates the controller of FIG. 1 in a user's closed hands, according to an exemplary embodiment of the present disclosure. [Figure 4] FIG. 4 illustrates the controller of FIG. 1 in a user's palm-down closed hand, according to an exemplary embodiment of the present disclosure. [Figure 5] FIG. 5 illustrates a pair of controllers with hand holders in an open position according to an exemplary embodiment of the present disclosure. [Figure 6]FIG. 6 illustrates a touch sensor of the controller of FIG. 1 according to an exemplary embodiment of the present disclosure. [Figure 7A] FIG. 7A illustrates a configuration of a first controller of the touch sensor of FIG. 6 according to an exemplary embodiment of the present disclosure. [Figure 7B] FIG. 7B illustrates a configuration of a second controller of the touch sensor of FIG. 6 according to an exemplary embodiment of the present disclosure. [Figure 7C] FIG. 7C illustrates a third controller configuration of the touch sensor of FIG. 6 according to an exemplary embodiment of the present disclosure. [Figure 8] FIG. 8 illustrates an exemplary process for configuring touch sensors of a controller, according to an exemplary embodiment of the present disclosure. [Figure 9] FIG. 9 illustrates an exemplary process for configuring touch sensors of a controller, according to an exemplary embodiment of the present disclosure. [Figure 10] FIG. 10 illustrates an exemplary process for configuring touch sensors of a controller, according to an exemplary embodiment of the present disclosure. [Figure 11] FIG. 11 illustrates an exemplary process for configuring touch sensors of a controller, according to an exemplary embodiment of the present disclosure. [Figure 12] FIG. 12 illustrates exemplary components of the controller of FIG. 1, according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0004] Specifically described herein are handheld controllers having touch-sensitive controls, methods for using the output of the touch-sensitive controls, and methods for dynamically adjusting the touch-sensitive controls based on the size and / or grip of a user's hand operating the handheld controller. In some cases, the handheld controllers described herein can control remote devices (e.g., televisions, audio systems, personal computing devices, game consoles, etc.) and can participate in video game play and / or the like.

[0005] The handheld controller may include one or more controls, such as one or more joysticks, trackpads, trackballs, buttons, or other controls controllable by a user operating the handheld controller. Additionally or alternatively, the handheld controller may include one or more controls including a touch sensor configured to detect a user's presence, proximity, location, and / or gestures on each control of the handheld controller. The touch sensor may include a capacitive touch sensor, a force-resistant touch sensor, an infrared touch sensor, a touch sensor that uses acoustic waves to detect the presence or location of an object, the proximity of an object, and / or any other type of sensor configured to detect touch input at the handheld controller or the proximity of one or more objects to the handheld controller. Furthermore, in some cases, the touch sensor may include a capacitive pad.

[0006] The touch sensor is communicatively coupled to one or more processors of the handheld controller to transmit touch sensor data indicative of touch input at the handheld controller. The touch sensor data may also indicate the proximity or proximity of one or more fingers to the handheld controller. The touch sensor data may indicate the location of the touch input on the handheld controller and / or the location of the fingers relative to the handheld controller, potentially as it changes over time. For example, if a user's finger is hovering over or positioned away from the handheld controller, the touch sensor data may indicate how far the finger is extended or close to the handheld controller.

[0007] The handheld controller may include logic circuitry (e.g., software, hardware, firmware, etc.) configured to receive the touch sensor data and determine the presence and / or location (or "position") of a user's fingers on the handheld controller. For example, in cases where the touch sensor comprises capacitive pads, different areas or groups of the capacitive pads may represent or correspond to different fingers of the user, and the logic circuitry may determine the area and / or group of the capacitive pads from which to detect capacitance. The handheld controller may provide this information to a game or other application for performing one or more operations with the handheld controller, such as gestures performed with fingers touching or in proximity to the handheld controller. For example, the handheld controller may transmit the touch sensor data or other display values ​​to a game console, a remote system, another handheld controller, or another computing device. The computing device can use the touch sensor data and / or display values ​​to perform one or more operations, such as generating image data corresponding to the user's hand gestures.

[0008] Logic circuitry in the handheld controller (or a computing device communicatively coupled to the handheld controller) can use touch sensor data, such as capacitance values, to identify a controller configuration for a user. The handheld controller or computing device can store different controller configurations representing different assignments of capacitive pads to each of a user's fingers. That is, as described above, the touch sensor capacitive pads can be divided into groups, with each group corresponding to or associated with a respective finger on a hand (e.g., pinky, ring, middle, and index fingers). For each controller configuration, a touch sensor capacitive pad can be associated with a respective finger on the hand. Thus, when receiving data from the touch sensor, the logic circuitry can associate the touch sensor data with the user's corresponding finger, which can then be used to identify a hand gesture. In other words, by recognizing the capacitive pads corresponding to each finger on the hand, the logic circuitry can determine the user's corresponding hand gesture, such as fingers grasping the handheld controller and / or fingers not grasping the handheld controller. For example, the logic circuitry may determine that a user is grasping the handheld controller with their middle and ring fingers, but not their pinky. Thus, by recognizing a capacitive pad or group of capacitive pads corresponding to each finger on a hand, the logic circuitry may provide a representation of this gesture to an application configured to perform a predetermined action associated with the gesture or generate image data corresponding to the gesture (e.g., the middle and ring fingers are grasping an object, while the pinky finger is not grasping an object). Furthermore, by utilizing touch sensor data related to the proximity of the fingers to the handheld controller, such as detected capacitance values, the logic circuitry of the handheld controller can determine the amount of curl or extension associated with each finger (e.g., how far the finger is positioned from the handheld controller).

[0009] The handheld controller may dynamically adjust, detect, and accommodate various grips of a user or different users operating the handheld controller. For example, a user's grip may vary depending on how the user grips the handheld controller, the game the user is playing, and / or the physical characteristics of the user's hand (e.g., finger length, finger width, etc.). Thus, the touch sensor may adapt to different user grips. Furthermore, because different users grip the handheld controller in different ways, the touch sensor may adapt to the user's grip. In other words, even if different users have similar hands, or as the user progresses during gameplay, the user's grip may change (e.g., the user's fingers may grip different parts of the handheld controller). To accommodate various grips and enhance the gameplay experience, the logic circuit may rearrange or rebind the capacitive pads of the touch sensor according to different controller configurations. In doing so, the controller's logic circuit may associate touch sensor data with the user's specific fingers to accurately represent the user's hand gestures.

[0010] To briefly illustrate, a handheld controller or computing device communicatively coupled to a handheld controller (e.g., a game console) can generate a score using machine learning techniques and touch sensor data. The handheld controller or computing device can select the controller configuration (or a closely matching controller configuration) with the highest score and configure the handheld controller according to the selected controller configuration. Such a configuration can map specific capacitive pads of the touch sensors to the user's fingers (e.g., middle finger, ring finger, pinky finger, etc.). That is, to accurately represent the user's hand gestures in gameplay (e.g., in a VR environment), the handheld controller (or computing device) can configure the capacitive pads of the touch sensors to correspond to specific fingers based on the selection of the controller configuration. Subsequently, upon receiving the touch sensor data, the handheld controller can associate the capacitive pads with the corresponding fingers, thereby recognizing the position and / or proximity of the fingers relative to the handheld controller. However, the capacitive pads can also measure the proximity of the fingers relative to the handheld controller, for example, by measuring capacitance. By continuously recording the controller configuration, the handheld controller can dynamically adapt to the user's grip and associate capacitive pads with each of the user's fingers. The handheld controller can therefore reassign or relocate specific capacitive pads of the touch sensors to associate them with specific fingers of the user. The touch sensor data can then be used to accurately represent the user's hand (e.g., in a VR environment).

[0011] The handheld controller can also sense, detect, or measure the amount of force associated with a touch input on the handheld controller through touch sensors and / or pressure sensors. For example, when a user's finger presses the handheld controller, a portion of the controller, such as a cover disposed over the touch sensors and / or pressure sensors, may deflect and contact the touch sensors and / or pressure sensors. The pressure sensors may be coupled to one or more processors such that the finger touch input results in force data being provided to the one or more processors. The pressure sensors may provide force data to the one or more processors indicating the amount of force of the touch input. In some cases, the pressure sensors may include force-sensing resistor (FSR) sensors, piezoelectric sensors, load cells, strain gauges, capacitive pressure sensors that measure capacitive force, or any other type of pressure sensor. Furthermore, in some cases, the touch sensor data and / or force data can be interpreted together and associated with a predetermined command (e.g., squeeze).

[0012] While conventional handheld controllers may include sensors for sensing touch input, they statically map the touch sensors to associate them with specific fingers. However, such mapping does not reassign portions of the touch sensors, such as capacitive pads, to specific fingers or dynamically adapt the touch sensors to different fingers depending on the user's grip. This static mapping can result in a less-than-ideal user experience within a gameplay environment. For example, if the touch sensor data does not accurately map to each of the user's fingers, the generated hand image may not accurately depict the user's hand operating the handheld controller. The techniques and systems described herein improve upon existing techniques to dynamically assign or correlate capacitive pads of touch sensors to specific user fingers. In doing so, image data generated from the touch sensor data can accurately depict the user's fingers, enabling improved gameplay experiences and / or other applications controlled by the handheld controller.

[0013] 1 is a front view of an exemplary controller 100, which may include one or more touch-sensitive controls. As described herein, the touch-sensitive controls can generate touch sensor data that is utilized by the controller 100 and / or other computing devices to create user hand gestures. The touch sensor data can indicate the presence, position, proximity, and / or gestures of a user's fingers manipulating the controller 100. In some instances, the controller 100 can be utilized by electronic systems, such as a VR video game system, a robot, a weapon, or a medical device.

[0014] As shown, the controller 100 may include a controller body 110 having a handle 112 and a hand holder 120. The controller body 110 may include a head portion disposed between the handle 112 and a distal end 111 of the controller 100, which may include one or more thumb-operated controls 114, 115, 116. For example, the thumb-operated controls may include tilt buttons or any other button, knob, wheel, joystick, or trackball that is conveniently operated by a user's thumb during normal operation when the controller 100 is held in the user's hand.

[0015] The handle 112 may comprise a generally cylindrical tubular housing. In this regard, the generally cylindrical shape need not have a constant outer diameter or a perfectly circular cross section.

[0016] The handle 112 may include a proximity sensor and / or a touch sensor having a plurality of capacitive sensors spatially distributed partially or completely around the exterior surface of the handle 112. For example, the capacitive sensors can be spatially distributed beneath the exterior surface of the handle 112 and / or embedded beneath the exterior surface of the handle 112. The capacitive sensors can respond to a user touching, grasping, or gripping the handle 112 and identify the presence, position, and / or gesture of one or more of the user's fingers. Additionally, the capacitive sensors can respond to one or more fingers hovering or being placed over the handle 112. For example, one or more of the user's fingers may not grasp or wrap around the controller 100 but instead move over the exterior surface of the handle 112. To accommodate such and detect finger proximity and / or touch input, the exterior surface of the handle 112 may include an electrically insulating material.

[0017] The hand holder 120 can be coupled to the controller 100 and biases the user's palm against the outer surface of the handle 112. As shown in FIG. 1 , the hand holder 120 is in the open position. The hand holder 120 can optionally be biased in the open position by a curved elastic member 122 to facilitate inserting a user's hand between the hand holder 120 and the controller body 110 when gripping the controller 100. For example, the curved elastic member 122 can comprise a flexible metal piece that bends elastically, or can comprise an alternative plastic material such as nylon that is generally bendable elastically. A fabric material 124 (e.g., a neoprene sheath) can partially or completely cover the curved elastic member 122 to absorb shock or improve user comfort. Alternatively, the absorbent or fabric material 124 can be attached only to the side of the curved elastic member 122 that faces the user's hand.

[0018] The hand retainer 120 is adjustable in length, for example, by including a drawcord 126 that is tightened by a spring-loaded chock 128. The drawcord 126 may optionally have extra length for use as a strap. In some embodiments, an absorbent or fabric material 124 may be attached to the drawcord 126. Additionally, the curved elastic member 122 may be preloaded by the tension of the tightened drawcord 126; in such embodiments, the tension that the curved elastic member 122 applies to the hand retainer 120 (to bias the hand retainer 120 to the open position) causes the hand retainer to automatically open when the drawcord 126 is not tightened. However, alternative conventional methods for adjusting the length of the hand retainer 120 may be used, such as cleats, elastic bands (which temporarily stretch when a hand is inserted, thereby applying elastic tension to press against the back of the hand), or hook-and-loop strap attachments that allow for length adjustment.

[0019] The hand holder 120 can be disposed between the handle 112 and the tracking member 130 and can contact the back of the user's hand. The tracking member 130 can be secured to the controller body 110 and can optionally include two noses 132, 134, each protruding from a corresponding one of two opposing distal ends of the tracking member 130. In some cases, the tracking member 130 can include an arcuate portion having a generally arcuate shape. In some cases, the tracking member 130 can include tracking transducers disposed therein, with at least one tracking transducer disposed on each of the protruding noses 132, 134. The controller body 110 can include additional tracking transducers, such as a tracking transducer disposed adjacent the distal end 111.

[0020] The controller 100 may include a rechargeable battery disposed within the controller body 110, and the hand holder 120 may include a conductive charging wire electrically coupled to the rechargeable battery. The controller 100 may also include a radio frequency (RF) transmitter for communication with the rest of the electronic system (e.g., a game console). The rechargeable battery may power the RF transmitter, and the transmitted RF may be responsive to the thumb-operated controls 114, 115, 116, touch sensors (e.g., capacitive sensors) in the handle 112, and / or tracking sensors in the tracking member 130.

[0021] In some cases, the controller body 110 may comprise a single piece of injection-molded plastic or any other material that is rigid enough to transfer force from the user's finger to the touch sensor and thin enough to allow capacitive coupling between the user's finger and the touch sensor. Alternatively, the controller body 110 and tracking member 130 can be fabricated separately and later assembled together.

[0022] FIG. 2 is a front view of the controller 100, showing the controller 100 in operation with a user's left hand inserted therein but not gripping the controller body 110. In FIG. 2, the hand retainer 120 clamps onto the user's hand, physically biasing the user's palm against the outer surface of the handle 112. Herein, the hand retainer 120, when closed, can hold the controller 100 in the user's hand even when the hand is not gripping the controller body 110. As shown, when the hand retainer 120 is tightly closed around the user's hand, the hand retainer 120 can prevent the controller 100 from falling out of the user's hand. Thus, in some embodiments, the hand retainer 120 may allow the user to let go of the controller 100 without the controller 100 actually being released, thrown, and / or falling to the floor, enabling additional functionality. For example, when a user's release or return of their grip on the handle 112 of the controller body 110 is sensed, the release or grip can be incorporated into a game to display an object to be thrown or grasped (e.g., in a VR environment). The hand holder 120 may allow such functions to be accomplished repeatedly and safely.

[0023] The hand holder 120 may also prevent the user's fingers from moving excessively against the touch sensors, providing a more reliable sense of finger movement and / or placement on the handle 112 .

[0024] 3 and 4 show controller 100 during an operation in which a user's hand grasps controller body 110 and tightens hand holder 120 while holding controller 100 in the user's hand. As shown in FIGS. 3 and 4, the user's thumb can operate one or more of thumb operated controls 114, 115, and 116.

[0025] 5 illustrates that in certain embodiments, controller 100 may be the left controller in a pair of controllers with a similar right controller 500. In certain embodiments, controllers 100 and 500 may simultaneously track the movement and grip of a user's hands, for example, to enhance the VR experience.

[0026] 6 illustrates a proximity sensor or touch sensor 600 having a plurality of capacitive pads 602 configured to detect touch input on a controller (e.g., controller 100) as well as the proximity of one or more objects (e.g., fingers) to the controller 100. In some embodiments, the touch sensor 600 may additionally or alternatively include different types of sensors configured to detect touch input on the controller 100 or the proximity of a finger to the controller 100, such as infrared or acoustic sensors. As illustrated in FIG. 6, the capacitive pads 602 of the touch sensor 600 are not necessarily the same size and do not necessarily have approximately the same spacing between them. However, in some embodiments, the capacitive pads 602 may include a grid with approximately the same size and approximately the same spacing between them.

[0027] The touch sensor 600 may include a flexible printed circuit assembly (FPCA) 604 on which capacitive pads 602 are disposed. The FPCA 604 may include a connector 606 for connecting to a printed circuit board (PCB) of the controller 100, which may include one or more processors. The capacitive pads 602 may be communicatively connected to the connector 606 by traces 608 disposed on the FPCA 604. The capacitive pads 602 may provide touch sensor data (e.g., capacitance values) to one or more processors of the controller 100 via the traces 608 and the connector 606. As described in detail herein, the touch sensor data may indicate the proximity of a finger to the controller 100. That is, the touch sensor 600 may measure the capacitance of each capacitive pad 602, and the capacitance may be related to the proximity of a finger to the controller 100 (e.g., touching or disposed on the handle 112 of the controller 100).

[0028] The touch sensor 600 can be coupled to an inner surface within the controller body 110, such as a structure mounted within the handle 112 of the controller body 110 or a structure mounted below the handle 112 of the controller body 110. In doing so, the touch sensor 600 can be disposed below the outer surface of the handle 112 to detect the proximity of a finger to the handle 112. When coupled to the controller 110, the touch sensor 600 can extend angularly around the circumference or portion of the handle 112. For example, the FPCA 604 can be coupled (e.g., adhered) to the inner surface of the controller body 110 at the handle 112 to detect the proximity of a finger to the handle 112. In some embodiments, the touch sensor 600 can extend at least 100 degrees but not more than 170 degrees around the circumference of the handle 112. Additionally or alternatively, the touch sensor 600 can be coupled to an outer surface of the controller 110, such as the outer surface of the handle 112.

[0029] The capacitive pads 602 can be spaced apart from one another to detect the proximity of different fingers or different portions of a user's fingers (e.g., fingertips) to the controller 100. For example, as shown in FIG. 6 , the capacitive pads 602 are arranged in columns, rows, grids, sets, subsets, or groups 610. In some cases, an individual group 610 of capacitive pads 602 can correspond to a specific finger of a user (e.g., index finger, middle finger, ring finger, pinky finger). Additionally or alternatively, multiple groups 610 of capacitive pads 602, or capacitive pads 602 from multiple groups 610, can correspond to a single finger of a user. For example, two or more groups 610 can correspond to a finger of a user (e.g., middle finger).

[0030] 6, touch sensor 600 may include six groups 610 of capacitive pads 602, with groups 610 extending horizontally across the face of FPCA 604. However, in some embodiments, touch sensor 600 can include more than six groups 610 or fewer than six groups 610.

[0031] By arranging the capacitive pads 602 into groups 610 or assigning particular capacitive pads 602 to particular groups 610, the controller 100 (or another communicatively coupled computing device) can utilize touch sensor data (e.g., capacitance values) from the capacitive pads 602 to generate a user's hand gesture. That is, the touch sensor 600 can generate touch sensor data for use in detecting the presence, position, and / or gesture of a user's finger gripping the controller 100. In these cases, a user grips the controller 100 with a particular finger, hovers the particular finger over the controller 100, and applies a voltage to the capacitive pad 602 such that an electrostatic field is generated. Thus, when a conductor such as a user's finger touches or approaches the capacitive pad 602, a change in capacitance occurs. The capacitance can be sensed by connecting an RC oscillator circuit to the touch sensor 600 and noting that the time constant (and therefore the period and frequency of oscillation) changes with the capacitance. In this manner, when a user releases a finger from the controller 100, grips the controller 100 with a particular finger, or moves closer to the controller 100, the controller 100 can detect a change in capacitance.

[0032] The capacitance values ​​of the capacitive pads 602, or individual capacitive sensors within the grid on each capacitive pad 602, are used to determine the position of the conductor in addition to the proximity of the conductor relative to the capacitive pad 602. That is, when a user grips the controller 100, a particular finger and / or portion of a finger can contact the handle 112 of the controller 100. Because the finger acts as a conductor, such capacitive pads 602 below the handle 112 where the user touches the handle 112 can measure capacitance values. These capacitance values ​​are measured over time for use in identifying the user's gesture. However, in instances where a user hovers a finger or a particular portion of a finger away from the controller 100, the capacitance value can represent or relate to how far the finger is positioned from the controller 100. The touch sensor data can therefore be used to determine the proximity and / or position of the finger relative to the controller 100. Associating fingers with different capacitive pads 602 of the touch sensor 600 can be advantageous when a user's grip may vary throughout a gameplay experience or between different users. For example, in a first case, a user may have a wide grip, and all of the capacitive pads 602 of the touch sensor 600 can detect capacitance values ​​for use in creating image data. In a second case, a user may have a narrow grip, and fewer than all of the capacitive pads 602 of the touch sensor 600 can detect capacitance values ​​for use in creating image data. That is, to generate accurate image data depicting a hand gesture, capacitive pads 602 can be dynamically correlated or associated with specific fingers of the hand. In other words, touch sensor data (e.g., capacitance values) can be used by the controller 100 or a communicatively coupled computing device to generate a corresponding hand gesture of the user. By recognizing the capacitive pads 602 of the touch sensor 600 associated with each finger of the hand, the capacitance values ​​detected by the touch sensor 600 can be used to generate a corresponding hand gesture.Thus, by changing the user's grip, the capacitive pad 602 can rearrange or associate with different fingers such that capacitance values ​​generate accurate image data depicting the hand gesture.

[0033] The one or more processors can include algorithms and / or machine learning techniques that illustrate anatomically possible finger movements and advantageously use the touch sensor data to detect the user's hand opening, finger pointing, or other movement of the fingers relative to the controller 100 or relative to each other. In this manner, the controller 100 and / or the user's finger movements can aid in controlling a VR gaming system, a defense system, a medical system, an industrial robot or machine, or another device. In a VR application (e.g., for gaming, training, etc.), the touch sensor data can be used to release an object based on the detected release of the user's fingers from the exterior surface of the handle 112. Additionally or alternatively, one or more processors of a communicatively coupled computing device (e.g., a host computing device, a game console, etc.) with which the controller 100 interacts can use the touch data to detect gestures.

[0034] In some cases, the capacitive pad 602 can also detect a capacitance value corresponding to the amount of force applied to an associated portion of the controller 100 (e.g., force applied to the exterior surface of the handle 112 relative to at least one thumb-operated control 114, 115, 116, etc.). Additionally or alternatively, the touch sensor 600 or another portion of the controller 100 (e.g., the handle 112) can include a force-sensing resistor (FSR) that uses a variable resistance to measure the amount of force applied to the FSR. Because the controller 100 can be configured to be held by a user's hand, the FSR can be mounted on a structure within the controller body 110, such as a structure mounted within the handle 112 of the controller body 110 or a structure mounted below the controller body 110. In certain embodiments, the FSR, together with the capacitive pad 602, can facilitate sensing both the initiation of a user's grip and the relative strength of such a user's grip, which can facilitate certain gameplay features. In either embodiment, the FSR can generate force data for use in detecting the presence, position, and / or gesture of a user's fingers gripping the controller 100. When implemented in the controller 100, the FSR and / or the capacitive pads 602 can measure a resistance or capacitance value, respectively, that corresponds to the amount of force applied to an associated portion of the controller 100.

[0035] In some embodiments, one or more processors of the controller 100 can utilize touch sensor data and / or force data to detect the size of the hand gripping the handle 112 and / or adjust the threshold force required to register a touch input on the capacitive pad 602 and / or FSR according to the hand size. This can be useful so that force-based input is easier for users with smaller hands (and less easy, but not difficult, for users with larger hands).

[0036] 7A-7C illustrate various controller configurations. As discussed above, depending on the user's grip, the capacitive pads 602 of the touch sensor 600 can correspond to or be associated with specific fingers of the user. Various controller configurations can map the capacitive pads 602 to respective fingers and corresponding portions of the fingers (e.g., base, tip, middle, etc.). In doing so, touch sensor data generated from the touch sensor 600, or capacitance values ​​from the individual capacitive pads 602, can be associated with each of the user's fingers for use in generating image data indicative of hand gestures.

[0037] FIG. 7A illustrates a first controller configuration 700 in which capacitive pads 602 of a touch sensor 600 have capacitance values ​​that are utilized to generate image data corresponding to a hand gesture. That is, in FIG. 7A , the “blacked-out” capacitive pads 602 represent capacitive pads 602 whose touch sensor data is utilized to generate image data corresponding to a hand gesture. In comparison, the non-blacked capacitive pads 602 represent capacitive pads 602 whose capacitance values ​​are not utilized to generate image data corresponding to a hand gesture. However, the non-blacked capacitive pads 602 still have capacitance values ​​that can be measured for purposes of changing the controller configuration. For example, in the first controller configuration 700, if a user touches one of the non-blacked capacitive pads 602, the controller 100 or a communicatively coupled computing device can use this information to determine whether to change the controller configuration.

[0038] In detailing the first controller configuration 700, the first column 702, the second column 704, and the third column 706 of the touch sensor 600 may correspond to a first finger (e.g., a middle finger) of a user. The fourth column 708 and the fifth column 710 of the touch sensor 600 may correspond to a second finger (e.g., a ring finger) of a user. The sixth column 712 of the touch sensor 600 may correspond to a third finger (e.g., a pinky finger) of a user. By matching a column of the touch sensor 600 to a particular finger of a user and a particular capacitive pad 602 of a column to a particular finger, capacitance values ​​generated by the capacitive pad 602 of the touch sensor 600 can be used to generate image data depicting a hand gesture (i.e., how a user is holding the controller 100). For example, if a user grasps controller 100 while controller 100 is configured according to first controller configuration 700, and the user does not grasp controller 100 with a first finger, the capacitance values ​​received in columns 702, 704, and 706 may indicate that the first finger is not touching controller 100.

[0039] In other words, the capacitance values ​​for the first column 702, the second column 704, and the third column 706 associated with a first finger may indicate that the first finger is not touching the controller 100. Instead, the capacitance values ​​from the capacitance pads 602 in the first column 702, the second column 704, and / or the third column 706 may indicate a contact distance or proximity where the first finger hovers over the controller 100. Additionally, the capacitive pads 602 associated with the fourth column 708, the fifth column 710, and the sixth column 712 can detect capacitance values ​​indicative of second and third fingers touching the controller 100. Such capacitance values ​​received by each capacitive pad 602 can be used to generate hand image data indicative of, for example, an index finger being extended (e.g., pointing) compared to fingers curled around a target. Thus, capacitance values ​​from individual capacitive pads 602 can be used to determine how far a user's finger is placed from controller 100. However, as described above, such capacitive pads 602, whose capacitance values ​​are used to generate image data, can be used by controller 100 to determine whether to configure controller 100 according to another controller configuration.

[0040] 7B illustrates a second controller configuration 714, where the capacitance values ​​of the capacitive pads 602 of the touch sensor 600 are utilized to generate image data. That is, in FIG. 7B, the "blacked-out" capacitive pads 602 represent capacitive pads 602 whose capacitance values ​​are utilized to generate image data corresponding to hand gestures. In comparison, the non-blacked-out capacitive pads 602 represent capacitive pads 602 whose capacitance values ​​are not utilized to generate image data corresponding to hand gestures. However, the non-blacked-out capacitive pads 602 still have measurable capacitance values ​​for purposes of determining whether to switch between controller configurations.

[0041] In the second controller configuration 714, the first column 716, the second column 718, and the third column 720 of the touch sensor 600 may correspond to a first finger of a user. The fourth column 722 may correspond to a second finger of a user, and the fifth column 724 of the touch sensor 600 may correspond to a third finger of a user. Compared to the first controller configuration 700, the second controller configuration 714 may represent a smaller grip (or hand size) of a user operating the controller 100. In other words, for the second controller configuration 714, the user's grip may not contact the sixth column 726 of the touch sensor 600. By matching the columns of the touch sensor 600 to specific fingers, capacitance values ​​generated by the touch sensor 600 are utilized to generate image data depicting a hand gesture (i.e., how the user is holding the controller). By way of example, and by comparing the second controller configuration 714 to the first controller configuration 700, because the second controller configuration 714 may correspond to a smaller grip on the handle 112, the hands generated according to the second controller configuration 714 may be smaller than the hands generated according to the first controller configuration 700.

[0042] By dynamically remapping the capacitive pads 6062 of the touch sensor 600, the capacitive pads 602 can be associated with different fingers of the user or not associated with a finger of the user. Thus, the second controller configuration 700 may be more suitable for users with smaller hand sizes and / or smaller finger sizes than the first controller configuration 700. For example, using the first controller configuration 700 for a user with small hands may not accurately depict the user's hand gestures because the user touches the sixth column 712, and different columns of the first controller configuration 700 correspond to different fingers of the user. Thus, the second controller configuration 714 may more accurately associate the capacitive pads 602 with specific fingers of the user.

[0043] 7C illustrates a third controller configuration 728 in which capacitive pads 602 of touch sensor 600 have capacitance values ​​that are utilized to generate image data. That is, in FIG. 7C , the “blacked-out” capacitive pads 602 represent capacitive pads 602 whose capacitance values ​​are utilized to generate image data corresponding to hand gestures. In comparison, the non-blacked-out capacitive pads 602 represent capacitive pads 602 whose capacitance values ​​are not utilized to generate image data corresponding to hand gestures. However, the non-blacked-out capacitive pads 602 can still measure capacitance values ​​and generate touch sensor data for purposes of determining whether to switch between controller configurations.

[0044] In the third controller configuration 728, all of the capacitive pads 602 of the touch sensor 600 are shown solid black. In doing so, image data depicting, for example, a hand representation can be generated from capacitance values ​​from all of the capacitive pads 602. The third controller configuration 728 can indicate a larger hand size or a larger grip on the handle 112 of the controller 100 compared to the second controller configuration 714. Furthermore, compared to the first controller configuration 700, the third controller configuration 728 can accommodate a hand with a larger finger length, as indicated by all of the capacitive pads 602 in the solid black columns (compared to the first controller configuration 700). In the third controller configuration 728, the first column 730, the second column 732, and the third column 734 of the touch sensor 600 can correspond to a first finger of a user. The fourth column 736 and the fifth column 738 of the touch sensor 600 can correspond to a second finger of a user. The sixth column 740 of the touch sensor 600 may correspond to the user's third finger.

[0045] As discussed above, the capacitive pads 602 of the touch sensor 600 can be remapped to correspond to different fingers of a user. Depending on the user's grip or the size of the user's hand, for example, a particular single-row or double-row touch sensor 600 can correspond to a particular finger of the user (e.g., a middle finger), while in other cases, it can correspond to a different finger of the user (e.g., a ring finger). By receiving touch sensor data (e.g., capacitance values) indicative of the user's grip or finger position, the capacitive pads 602 of the touch sensor 600 can be remapped to correspond to different fingers of the user. In other words, the capacitive pads 602 can be designated in some cases by the handle 112 of the controller 100 to correspond to different fingers of the user, depending on the particular user's grip. In this sense, the controller 100 can include different controller configurations (i.e., a first controller configuration 700, a second controller configuration 714, and a third controller configuration 728) in which different fingers correspond to different capacitive pads 602. Compared to conventional approaches, dynamically adapting the capacitive pad 602 to a particular finger in this manner may enable the user's precise gestures to be generated in a VR environment.

[0046] 7A-7C depict a particular controller configuration or a particular amount of capacitive pads, the touch sensor 600 may represent additional controller configurations. For example, any combination of capacitive pads 602, grouping 610, or row of capacitive pads 602 may correspond to a particular user's finger or grip. Furthermore, the touch sensor 600 may include mappings for four or more fingers or capacitive pads 602. The touch sensor 600 may thus be associated with a user's grip, and the capacitive pads 602 may be mapped to a particular user's finger.

[0047] 8-11 illustrate various processes as collections of blocks in logical flow diagrams, which represent sequences of operations that may be implemented in hardware, software, or a combination thereof. In the context of software, the blocks represent computer-executable instructions that, when executed by one or more processors, perform the recited operations. Generally, computer-executable instructions include routines, programs, objects, components, data structures, etc. that perform particular functions or implement particular abstract data types. The order in which the operations are described is not intended to be construed as a limitation, and any number of the described blocks can be combined in any order and / or in parallel to perform a process.

[0048] FIG. 8 is a flow diagram of an exemplary process 800 for calibrating and configuring the touch sensor 600 for different controller configurations. At 802, logic circuitry of the controller 100 can receive touch sensor data from the touch sensor 600. For example, an object (e.g., a finger, thumb, etc.) can touch the controller 100 or come within proximity of the controller 100 (e.g., on the handle 112). The touch sensor data can indicate a capacitance value detected or measured by the capacitive pad 602 of the touch sensor 600. For example, if a finger touches the controller 100, the capacitance value may be larger compared to a finger hovering over the controller 100 without touching the controller 100. In this sense, the capacitance value can indicate the proximity of the finger to the controller 100. The logic circuitry of the controller 100 can convert the capacitance value into a digitized value.

[0049] In some cases, the touch sensor data received at 802 may represent raw data that has not been calibrated and / or normalized with other touch sensor data provided by other capacitive pads 602. That is, the touch data received at 802 may represent raw data in the sense that for a particular capacitive pad 602, the capacitive pad 602 may be capable of detecting a capacitance value or range of capacitance values ​​depending on the size of the capacitive pad 602 and the size of the user's finger and / or hand touching the controller 100.

[0050] At 804, the logic circuitry of the controller 100 can normalize the touch sensor data. For example, by repeatedly receiving touch sensor data from the touch sensor 600 (e.g., as a user interacts with the controller 100), the touch sensor data can indicate a capacitance value measured by the capacitive pad 602. Over time, the capacitance values ​​can indicate a range of capacitance values ​​detected or measured by each capacitive pad 602 of the touch sensor 600. For example, the capacitive pad 602 can detect a high capacitance value when the user grips a portion of the controller 100 over the capacitive pad 602 and a low capacitance value when the user does not grip a portion of the controller 100 over the capacitive pad 602. Thus, at 804, for each capacitive pad 602 of the touch sensor 600, the logic circuitry of the controller 100 can analyze the touch sensor data and determine a range of received capacitance values, a maximum received capacitance value, a minimum received capacitance value, an average capacitance value, and / or a median capacitance value. In some cases, the capacitance values ​​can be normalized within the range [0, 1].

[0051] At 806, the logic circuitry of the controller 100 can calibrate the touch sensor 600. As indicated by the sub-blocks of FIG. 8 , the process 800 can involve more detailed operations for calibrating the touch sensor 600. For example, calibrating the touch sensor 600 can include sub-blocks 808 and 810. As indicated by sub-block 808, calibrating the touch sensor 600 can include individual gesture recognition. The individual gesture recognition can correspond to individual gestures performed by a user on the controller 100. For example, if the capacitance values ​​of all or a majority of the capacitive pads 602 suddenly drop, the logic circuitry can associate this drop with the user releasing their hand or finger from the controller 100. The capacitance value received when the user suddenly releases their finger from the controller 100 can correspond to a low-level value in the range of capacitance values ​​detected for the particular capacitive pad 602 (e.g., the capacitance value represents when the finger is not touching the controller 100). The capacitance value received before the drop can correspond to the high end of a range of capacitance values ​​detected for the particular capacitive pad 602 (e.g., the capacitance value represents when a finger touches the controller 100). The range of capacitance values ​​allows the logic of the controller 100 to calculate a bias and scaling factor for the capacitance values ​​received by the controller 100. That is, knowing the scaling factor for the capacitive pad 602 allows for normalization of the received capacitance values.

[0052] As shown by sub-block 810, calibrating the touch sensor 600 can also include adjusting the successive low and / or high levels. Because the logic of the controller 100 can continuously receive touch sensor data from the touch sensor 600, the logic can continuously monitor the touch sensor data and recalibrate the low and / or high level capacitance values ​​for a range of capacitance values ​​for a given capacitive pad 602. For example, by continuously receiving touch sensor data from an individual capacitive pad 602, the logic of the controller 100 can determine whether the received capacitance value is lower or higher, respectively, than the previously determined low and / or high level capacitance value. Based on this determination, the logic of the controller 100 can update the low or high level capacitance value, thereby adjusting the range of capacitance values ​​for the particular capacitive pad 602. In doing so, the bias and / or scaling factor can be updated for use in normalizing the capacitance values.

[0053] Calibrating the touch sensor 600 can therefore aid in calculating biases and scaling factors for a particular capacitive pad 602 and a particular user operating the controller 100 .

[0054] At 812, logic circuitry in the controller 100 can configure the touch sensor 600 so that capacitive pads 602 are assigned to specific fingers of a user based on the specific controller configuration. For example, by recognizing that the middle finger is positioned above the ring finger, which is positioned above the pinky finger, the controller 100 can map specific capacitive pads 602 and their capacitance values ​​to specific fingers of the user. This mapping can occur for each controller configuration, so that each capacitive pad 602 is mapped to a corresponding finger and / or corresponding portion of a finger. However, as noted above, less than all capacitive pads 602 can be assigned to a finger. As the sub-blocks of FIG. 8 indicate, the process 800 can involve more detailed operations for configuring the touch sensor 600. For example, configuring the touch sensor 600 can include filtering noise in the low-level range of capacitance values ​​in sub-block 814 and capacitive pad and finger rejection in sub-block 816, each of which is described below.

[0055] In sub-block 814, the process 800 can filter out noise within a low-level range of capacitance values. For example, when a finger is not touching the controller 100 (e.g., hovering over or in close proximity to the handle 112), such capacitive pad 602 associated with the finger may be susceptible to noise. In other words, within the low-level range of capacitance values ​​for the capacitive pad 602, a small amount of measured capacitance may change the location of the user's finger. Generating image data corresponding to these changes may result in an uncomfortable drag on the finger within the VR environment. Instead, in instances where the capacitance value from the touch sensor 600 for an individual capacitive pad 602 drops below a certain threshold, below a threshold change from the previous capacitance value, or if the capacitance value is within a certain threshold of a low-level capacitance value for the capacitive pad 602, the logic circuitry of the controller 100 can suppress the capacitance value. In doing so, the controller 100 can ignore small spurious touch inputs on the touch sensor 600 that could otherwise cause uncomfortable drag on the fingers within the VR environment.

[0056] Capacitive pad and finger rejection in sub-block 816 may include identifying capacitive pads 602 whose low-level capacitance values ​​or high-level capacitance values ​​are within a threshold range of each other. For example, if the low-level and high-level capacitance values ​​are separated by a small range, the capacitive pad may not be able to accurately sense and detect the position of the user's finger with sufficient detail. Here, because the capacitive pad 602 can detect capacitance values ​​within the threshold range, the measured capacitance value cannot accurately correspond to the finger position.

[0057] Additionally or alternatively, a particular finger may be associated with multiple capacitive pads 602, which reduces the reliability of the detected capacitance value. In these situations, the capacitance values ​​received by the capacitive pads 602 may introduce noise. Ignoring a particular capacitive pad 602, or a group of capacitive pads 602, may increase the reliability that the capacitance value corresponds to a user's hand gesture.

[0058] From 812, the process can continue to "A" and process 900, described below with respect to FIG. 9. Additionally, while process 800 is described as being performed by controller 100, in some cases, one or more communicatively coupled computing devices can perform all or portions of the blocks of process 800. For example, controller 100 can transmit touch data received from touch sensor 600 to a computing device for calibrating and normalizing capacitance values, since one or more computing devices may include increased processing.

[0059] FIG. 9 is a flow diagram of an exemplary process 900 for calculating a score for a controller configuration and configuring the controller 100 according to a selected controller configuration. In some cases, process 900 can continue from “A” of process 800. At 902, logic circuitry of the controller 100 can record multiple controller configurations using capacitance values ​​received from the touch sensor 600 (after normalizing the capacitance values, as described above). For example, the logic circuitry can access data associated with individual controller configurations of the multiple controller configurations from a data store in the controller 100 or another communicatively coupled computing device. The logic circuitry can then provide the touch sensor data as input to the controller configuration. For example, the individual controller configurations may have previously been operated and / or calibrated using process 800 described above. As described above, the individual controller configurations can designate capacitive pads 602 of the touch sensor 600 that correspond to a user's finger. For example, a first controller configuration may associate a first capacitive pad 602 with a user's middle finger, while a second controller configuration may associate a second capacitive pad 602 with the middle finger, the first capacitive pad 602 and the second capacitive pad 602 representing different or different combinations of capacitive pads 602.

[0060] As shown in process 900, determining a score for a controller configuration may include sub-blocks such as calculating the variability within each finger group of the controller configuration at 904 and / or calculating the variability between finger groups of the controller configuration at 906. In some instances, the score for a controller configuration may represent the ratio of the variability within each finger group to the variability between each finger group, and may include an "f-statistic" or "f-test." The score may indicate the probability of the controller configuration matching the hand size, finger position, and / or user's grip. In other words, the controller configuration may represent how closely the user's fingers align or otherwise associate with the controller configuration. A high score may indicate a closely matched or compatible controller configuration, while a low score may indicate a controller configuration that is unlikely to match the user's grip. For example, for any suitable number of controller configurations, a first score may be attributed to the first controller configuration, a second score may be attributed to the second controller configuration, and so on.

[0061] In some cases, the score for a controller configuration may be initialized (i.e., the memory is set to zero) when the controller 100 is powered on. When the controller 100 receives touch sensor data, the logic circuitry can calculate a score for the controller configuration. In doing so, the logic circuitry may receive touch sensor data and update the score for the controller configuration; over time, as scores for controller configurations are calculated, the logic circuitry can select a controller configuration. A controller configuration that best fits the user's grip will have a higher score than other controller configurations that are not the best fit for the user's grip.

[0062] To illustrate determining a score for a controller configuration, as a user grasps the controller 100, the capacitive pads 602 can detect changes in capacitance based on the area of ​​the controller 100 grasped by the user or the proximity of the user's fingers to the controller 100. After the touch data is normalized and the touch sensors 600 are calibrated and / or configured (e.g., process 800), the touch sensor data is used to calculate a score for each controller configuration. Generally, recognizing that a user grasps the controller 100 with their middle finger positioned above their ring finger and their ring finger positioned above their pinky finger, capacitance values ​​received at the controller 100 can be associated with the middle finger, ring finger, and pinky finger, respectively. Furthermore, each controller configuration can include a predetermined layout in which particular capacitive pads 602 are associated with particular fingers of the user. That is, groups of capacitive pads 602 can correspond to particular fingers. For example, a large hand configuration may include a double row of capacitive pads 602 associated with the middle fingers, while a small hand configuration may include a single row of capacitive pads 602 associated with the middle fingers.

[0063] As the capacitive pads 602 are grouped together and a user touches a finger from the controller body 110, such capacitive pads 602 associated with a particular finger in a controller configuration may or may not detect a capacitance value. For example, if a user lifts their middle finger from the controller body 110 for a correct controller configuration, such capacitive pads 602 assigned to the middle finger will not measure a capacitance value or a low capacitance value. In some cases, the capacitance values ​​of the capacitive pads 602 in the middle finger group may experience approximately the same amount of capacitance change at approximately the same time. For example, for a correct controller configuration (e.g., a controller configuration with a high score), capacitive pads assigned to a particular finger will experience approximately the same capacitance or approximately the same change in capacitance at approximately the same time. The capacitive pads 602 for a correct controller configuration associated with the middle finger will therefore be statistically correlated with each other and will exhibit a high correlation with each other or low variation between each other. Calculating the variance within each finger group for an individual controller configuration may represent the variance between such capacitive pads 602 assigned to particular fingers (e.g., an "f-statistic" or "f-test"). In comparison, a controller configuration with high variance within finger groups may indicate an incorrect controller configuration or a controller configuration that is not well suited to the user's grasp. That is, a controller configuration with high variance within finger groups of capacitive pads 602 may indicate an inappropriate assignment of capacitive pads 602 to the user's respective fingers.

[0064] Calculating the variability between finger groups can indicate a displacement between one finger group of the capacitive pad 602 and another finger group of the capacitive pad 602 (e.g., an “f-statistic” or “f-test”). For example, referring back to the above example, when a user lifts their middle finger from the controller body 110 but keeps their little finger in contact with the controller body 110, there can be high variability between the capacitance values ​​of the capacitive pad 602 assigned to the middle finger and the capacitance values ​​of the capacitive pad 602 assigned to the little finger. A variability score can be generated for each finger and different controller configuration. Thus, for the middle finger, the variability between the middle finger group and the little finger group is calculated, and the variability between the middle finger group and the ring finger group is calculated. Similarly, the variability between the ring finger group and the little finger group can be calculated. This process can continue for each finger and each controller configuration. However, although the above example is described with respect to three fingers (e.g., a middle finger, a ring finger, and a little finger), the touch sensor 600 can be configured for any number of fingers.

[0065] When each controller configuration includes multiple determination scores, a combination (e.g., sum) of the scores may indicate how closely a particular controller configuration fits the user's grip. Additionally, in some cases, process 900 may determine a score for a controller configuration in cases where capacitance values ​​show a high degree of discrepancy compared to historical capacitance values. For example, the score for a controller configuration may include a moving average, and determining a new score for the controller configuration may update the moving average. Herein, the average for a controller configuration may be updated if a large discrepancy, or a difference above a threshold, exists between the determination score for the controller configuration and a previous or average score. In this sense, a single score for a controller configuration may not be biased or inappropriately weighed against previously calculated scores. Additionally, if a large discrepancy exists, the score for the controller configuration may be weighted.

[0066] At 908, the logic of the controller 100 can rank the controller configurations based at least in part on the decision score. As described above, the score associated with each controller configuration relates to the probability that the capacitive pad 602 assignments for that controller configuration map to or represent a user's grasp, i.e., that the capacitive pad 602 assignments for that controller configuration match the respective fingers of a hand.

[0067] At 910, the logic circuitry of the controller 100 can select a controller configuration. The controller configuration can indicate a capacitive pad 602 associated with each of the user's fingers. In doing so, the controller 100 continues to receive touch sensor data from the touch sensor 600, the touch sensor data associated with a particular finger of the user and a particular portion of the finger (e.g., base, tip, etc.). Recognizing the capacitive pad 602 of the touch sensor 600 that corresponds to each of the user's fingers can help accurately generate image data (using touch sensor data and / or force data) that depict the user's hand gestures.

[0068] At 912, the logic of the controller 100 can determine whether the selected controller configuration is different from the current controller configuration of the controller 100. For example, the controller 100 may include a default controller configuration, and at 912, the process 900 can determine whether the selected controller configuration is different from the default controller configuration. If the controller configurations are the same, meaning that the selected controller configuration (e.g., the configuration of the highest controller) is already configured in the controller 100, there may be no need to update or assign a different controller configuration. If so, the process 900 can take the "No" route and proceed to 916.

[0069] Alternatively, if the controller configurations are not identical, meaning that the selected controller configuration (e.g., the highest-ranking controller configuration) is not currently implemented on the controller 100, a different controller configuration needs to be updated or assigned. In other words, as a user's grip changes or the controller 100 can be switched between users, the controller configuration can be updated to assign capacitive pads 602 to each of the user's fingers. If so, the process 900 can take the "Yes" route and proceed to 914.

[0070] At 914, the logic of the controller 100 can configure the controller 100 according to the selected controller configuration. At 914, depending on the controller configuration selected (at 910), the capacitive pads 602 of the touch sensor 600 can be correlated to different fingers or different portions of fingers. Thus, in future cases, using the touch sensor data, the logic of the controller 100 can determine the fingers and / or individual finger portions touching the controller 100, as well as the proximity of the particular finger to the controller 100. Configuring the controller 100 according to the controller configuration can include assigning the capacitive pads 602 of the touch sensor 600 to correspond to particular fingers of the user. Assigning the capacitive pads 602 can include mapping the capacitive pads 602 to associate the capacitive pads 602 with respective fingers of the user. In some cases, less than all of the capacitive pads 602 of the touch sensor 600 can be mapped to respective fingers of the user. For example, if a user has small hands or a small grip, the user's grip may extend over a subset of the capacitive pads 602 of the touch sensor 600. In such a case, capacitance values ​​received from the remaining capacitive pads 602 are unusable when generating image data depicting the hand gesture. In contrast, a large hand may extend over the entire touch sensor 600, in which case all (or at least a threshold number or more) of the capacitive pads 602 can provide touch sensor data for use in generating image data.

[0071] At 916, the logic circuitry of the controller 100 can normalize the capacitance values ​​of the capacitive pads 602 using a set of weights assigned to each individual capacitive pad 602 in a group of capacitive pads 602 associated with a finger. For example, the weights assigned to each individual capacitive pad 602 can be associated with a configuration of the controller. For example, if four capacitive pads 602 of the touch sensor 600 are associated with a user's middle finger, all four capacitive pads 602 can be assigned the same weight. Thus, the capacitance values ​​received from these four capacitive pads 602 can include a weight of one-fourth. In doing so, the capacitance values ​​from these capacitive pads 602 can include the same weight used in determining the location of the finger. Furthermore, in some cases, the weight of a particular capacitive pad 602 can be set to zero based on capacitive pad and finger rejection, indicating that the capacitive pad 602 has reliability below a certain threshold.

[0072] Normalizing the touch sensor data at 916 may also include summing the capacitance values ​​from the capacitive pads 602 for each group of capacitive pads 602. As an example, if a middle finger is represented by four capacitive pads 602 for a particular controller configuration, then each capacitance value of the middle finger capacitive pads 602 may have a weight of 1 / 4. The logic circuitry can weight the capacitance values ​​for each capacitive pad 602 for a finger to indicate the contribution that the capacitance value for a given capacitive pad 602 makes to the total capacitance value for the finger.

[0073] Additionally or alternatively, normalizing the touch sensor data may include normalizing the touch sensor data according to weights previously applied to the capacitive pads 602. Capacitive pad and finger rejection (sub-block 816) allows less than all capacitive pads 602 in a group of capacitive pads 602 to have the same weight. As an example, a user may not touch a particular capacitive pad 602 in a group. The logic circuitry may then reject or be unable to resolve the capacitance values ​​of a particular capacitive pad 602 that the user does not touch, for example, if the user's hand is small or if the user places their hand separately on the controller 100. For example, as described above, if the low-level capacitance value and the high-level capacitance value have a small range (i.e., a small range), the capacitive pad 602 may be susceptible to a large amount of noise. Herein, the logic circuitry of the controller 100 may ignore the capacitance values ​​of a particular capacitive pad 602 in the weighted sum. In cases where capacitance values ​​are not used, the capacitance values ​​to be used may be summed and divided by the sum of the weights to be used. For example, if four capacitive pads 602 are assigned to a particular finger, each capacitive pad 602 may have a weight of one-fourth. However, if the capacitance value for one of the capacitive pads 602 is unreliable (e.g., contains a lot of noise), the weight of that one capacitive pad 602 may be negligible, such that the remaining three capacitive pads have a weight of one-third. In this case, the capacitance values ​​are summed and divided by the sum of the weights of the capacitive pads 602 to be used.

[0074] At 918, logic circuitry of controller 100 can assign a finger value to each of the user's fingers based at least in part on the touch sensor data. For example, after normalizing the touch sensor data to associate the touch sensor data with a particular finger, controller 100 can determine a finger value on a [0, 1] scale. The finger value assigned to a user's finger may indicate the relative position or curl of the finger with respect to controller 100. In this sense, controller 100 can determine the position of the user's finger with respect to controller 100 using capacitance values ​​detected from individual capacitive pads. In some instances, controller 100 can determine a finger value for each of the user's fingers or such fingers for which touch sensor 600 is configured.

[0075] At 920, the logic of the controller 100 can transmit the tactile values ​​(e.g., display values) to one or more computing devices. In some cases, the one or more computing devices can utilize the tactile values ​​and create image data on the controller 100 that depict the user's hand gesture. In some embodiments, the one or more computing devices can perform additional analysis of the tactile values. For example, the one or more computing devices can utilize curl logic to generate a finger curl in the image data. Additionally, in some cases, the controller 100 can transmit additional data captured and / or received by the controller, such as force data from a pressure sensor in the controller 100, when at least one finger presses against the surface.

[0076] From 920, process 900 may loop to step 802 of process 800. Thus, controller 100 can continuously receive capacitance values ​​according to the user's grip for use in creating image data and for use in determining whether to update the controller configuration of controller 100.

[0077] While some or all of process 900 is described as being performed by controller 100, in some cases, one or more communicatively coupled computing devices can perform all or a portion of the blocks of process 900. For example, because the computing device may include increased processing power, the controller can transmit touch data received from touch sensor 600 to the computing device for determining a score for the controller's configuration. The computing device can then transmit the controller's configuration to controller 100 or transmit a representation of the controller's configuration to controller 100.

[0078] 10 shows an exemplary process 1000 for configuring controller 100 according to one or more controller configurations. At 1002, process 1000 can receive touch sensor data from touch sensors 600 of controller 100. In some embodiments, the touch sensor data can indicate capacitance values ​​detected by each capacitive pad 602 of touch sensor 600.

[0079] At 1004, process 1000 can analyze the touch sensor data. In some cases, analyzing the touch sensor data can include processes 800 and / or 900 described herein above, whereby the touch sensor data is normalized and the touch sensor 600 is calibrated. For example, logic circuitry in controller 100 can analyze the touch sensor data and determine a score for each controller configuration. Selecting a controller configuration can include determining a ratio of (1) the variation in capacitance values ​​within a group of capacitive pads 602 assigned to a particular finger and (2) the variation in capacitance values ​​between groups of capacitive pads. For example, determining the variation within a finger group can include determining the variation in capacitance between a first capacitive pad and a second capacitive pad assigned to a first finger (e.g., an “f-statistic” or “f-test”). Determining the variation between finger groups may include determining the variation in capacitance between the total capacitance values ​​of the first capacitive pad and the second capacitive pad, and then comparing the total capacitance value to the total capacitance value of another group of capacitive pads 602 assigned to the second finger. A ratio is calculated for each controller configuration to determine the configuration that is most likely to fit the user's grip. In some cases, the controller 100 can transmit the touch sensor data to a computing device (e.g., a game console) for analysis, and the computing device can determine a score for each controller configuration.

[0080] At 1006, the process 1000 can select a first controller configuration 700. For example, after analyzing the touch sensor data (block 1004), the process 1000 can select a controller configuration (e.g., the first controller configuration 700) that best matches the hand size, finger size, and / or grip of the user. In instances where the computing device selects the first controller configuration 700, the computing device can send an indication of the selection to the controller 100.

[0081] At 1008, the process 1000 can configure the controller 100 according to the first controller configuration 700. As described herein and shown in FIG. 10 , configuring the controller 100 according to the first controller configuration 700 can include associating particular capacitive pads 602 of the touch sensor 600 with particular fingers and / or particular portions of fingers. Associating capacitive pads 602 with particular fingers can also include grouping one or more capacitive pads 602 together and associating a group of capacitive pads 602 with a finger. Additionally, as described above, depending on the hand size and / or user grip, less than all of the capacitive pads 602 of the touch sensor 600 can be mapped or otherwise assigned to fingers.

[0082] At 1010, the process 1000 can receive touch sensor data from the touch sensor 600 of the controller 100, and at 1012, the process 1000 can analyze the touch sensor data. The touch sensor data can also be used to generate image data using a configuration of the first controller.

[0083] At 1014, process 1000 can select a second controller configuration 714. For example, after analyzing the touch sensor data (block 1012), process 1000 can select a controller configuration (e.g., second controller configuration 714) that best matches the user's hand size and / or grip. For example, a user's grip can vary, or different users with different grips can operate the controller 100. In these cases, the controller configuration can be updated to accurately sense, detect, or otherwise associate different capacitive pads 602 of the touch sensor 600 with the user's fingers.

[0084] At 1016, the process 1000 can configure the controller 100 according to the second controller configuration 714. Configuring the controller 100 according to the second controller configuration 714 can include remapping or rebinding particular capacitive pads 602 of the touch sensor to particular fingers of the user compared to the first controller configuration 700.

[0085] 11 shows a process 1100 for calibrating a touch sensor of a controller, such as touch sensor 600 of controller 100, reducing noise in the touch sensor data, and using the touch sensor data to generate image data representative of a hand gesture performed by a user. In some cases, process 1100 can output a display value representing a finger value or finger position, such as how much the fingers are curled or extended.

[0086] At 1102, process 1100 can receive touch sensor data from touch sensor 600, where the touch sensor data indicates or indicates a raw capacitance value detected by a capacitive pad (e.g., capacitive pad 602) of touch sensor 600. As shown in FIG. 11, in some instances, process 1100 can receive a i , and a N As shown by , capacitance values ​​can be received from individual capacitive pads 602. In some cases, process 1100 can receive raw capacitance values ​​from touch sensor 600 for each frame displayed in the VR environment.

[0087] At 1104, the process 1100 can perform factory normalization to normalize the raw capacitance values. For example, the capacitive pads 602 may have different biases, scaling factors, and residual deviations depending on manufacturing conditions, the size of the capacitive pads 602, etc. In some cases, the factory normalization may include a primary calibration to remove bias in the capacitance values ​​and normalize the capacitance values.

[0088] At 1106, the process 1100 can perform a grasp calibration. As shown, the grasp calibration can include sub-blocks 1108, 1110, and 1112, which are described in more detail below.

[0089] In sub-block 1108, the process 1100 can perform statistical analysis to monitor, for each capacitive pad 602, the range of capacitance values, the maximum capacitance value received, the minimum capacitance value received, the average capacitance value, and / or the median capacitance value.

[0090] At 1110, process 1100 can perform detection of an individual gesture. Herein, process 1100 can analyze touch sensor data (i.e., capacitance values) after normalization according to factory normalization to detect individual gestures on controller 100. For example, if touch sensor data indicates a sudden drop in the capacitance value of capacitive pad 602, i.e., a portion thereof, process 1100 can associate this drop in capacitance value with the user releasing their hand or a particular finger from controller 100. The capacitance value received when the user suddenly releases their finger from controller 100 can correspond to a low-level value for the range of capacitance values ​​detected by the particular capacitive pad 602 (e.g., the capacitance value represents when the finger is not touching controller 100). The capacitance value received before the drop can correspond to a high-level value for the range of capacitance values ​​detected by the particular capacitive pad 602 (e.g., the capacitance value represents when the finger is touching controller 100). Depending on the range of capacitance values, the process 1100 can determine a bias and scaling factor for the capacitance values ​​of the capacitive pads 602 to normalize the capacitance values ​​received at each capacitive pad 602 .

[0091] At 1112, process 1100 can perform continuous calibration updates and decay. Because process 1100 can continuously receive touch sensor data from touch sensors 600, process 1100 can continuously monitor the touch sensor data and recalibrate or reset the low-level capacitance value and / or high-level capacitance value for a range of capacitance values ​​for a given capacitive pad 602. In other words, by continuously receiving touch sensor data from an individual capacitive pad 602, process 1100 can determine whether a capacitance value is lower or higher, respectively, than a previously determined low-level capacitance value and / or high-level capacitance value for the range. For example, as capacitance changes due to the gameplay experience (e.g., hands that are sweaty or dry, humidity, temperature, etc.), process 1100 can determine or set a new low-level capacitance value or a new high-level capacitance value, thereby adjusting the range of capacitance values ​​detected by the capacitive pad 602. Additionally, in some cases, continuous calibration can reduce reliance on process 1100 to determine individual gesture detections at 1110.

[0092] In some cases, process 1100 can assign a weight or percentage to a newly detected low-level capacitance value or a newly detected high-level capacitance value and update the low-level capacitance value or high-level capacitance value, respectively. For example, if process 1100 detects a capacitance value that is less than or equal to a previously detected low-level capacitance value during a particular time period, process 1100 can weight the capacitance value and update the low-level capacitance value.

[0093] Additionally, the low-level or high-level capacitance values ​​can decay over time depending on how the user grips the controller 100, environmental conditions (e.g., humidity), or other characteristics (e.g., skin wetness). The amount by which the low-level and high-level capacitance values ​​can decay can be limited so that the low-level and high-level capacitance values ​​are separated by a range threshold amount to reduce sensor noise. In some cases, the decay can depend on time and / or the rate of change of the capacitance values. For example, when a user taps a finger on the controller 100 or the controller 100 switches users, thereby potentially causing a change in the received capacitance values, the decay rate can increase, reducing the time required to update the low-level and / or high-level capacitance values.

[0094] As a result of the grasp calibration in 1106 and sub-blocks 1108-1112, the capacitance values ​​sensed from each capacitive pad 602 can be normalized on a [0, 1] scale, which can represent high and low levels for the capacitance values ​​sensed from the touch sensor 600 for a particular user grasp and individual capacitive pad 602.

[0095] At 1114, process 1100 can perform a weighted sum of the capacitance values. Because the capacitance values ​​are normalized on a [0, 1] scale, process 1110 can assign weights to the capacitance values ​​from the capacitive pads 602, depending on the controller configuration. That is, the capacitance values ​​are normalized between [0, 1], and a weight is assigned to each capacitance value received at each capacitive pad 602. For example, if a particular controller configuration includes five capacitive pads 602 assigned to a particular finger, the capacitance values ​​may include the same weight (e.g., 1 / 5). In other words, if a capacitive pad 602 detects the maximum capacitance value, the output of the weighted sum may equal 1.

[0096] As shown, determining the weighted sum may include sub-blocks 1116, 1118, 1120, and 1122. Sub-block 1116 may include a controller configuration model for controller 100. As described above, the controller configuration specifies a mapping of capacitive pads 602 of touch sensor 600 to associate a particular capacitive pad 602 with a particular finger of a user.

[0097] In sub-block 1118, process 1100 can perform dynamic controller configuration selection, where process 1100 inputs the capacitance values ​​into a controller configuration model and determines the best or most compatible controller configuration according to the user's grip. Selecting a controller configuration can include determining (1) the variation in capacitance values ​​for a group of capacitive pads 602 assigned to a particular finger and (2) the ratio of the variation in capacitance values ​​between groups of capacitive pads. The ratio is calculated for each controller configuration to determine the configuration that is most compatible with the user's grip.

[0098] In sub-block 1120, process 1100 can remove noise contained in the capacitance values. For example, when a finger is not touching controller 100, such as when the finger is fully extended, those capacitive pads 602 associated with the finger not touching controller 100 may be susceptible to noise. Herein, detecting a small amount of capacitance can result in a large amount of noise in the received capacitance value. In instances where the capacitance value from touch sensor 600 for an individual capacitive pad 602 drops below a certain threshold, or if the capacitance value is within a certain limit of a low-level capacitance value for the capacitive pad 602, process 1100 can suppress the detected capacitance. In other instances, in such situations, process 1100 can assign a low weight to the capacitance value.

[0099] In subblock 1122, process 1100 can reject particular capacitance values ​​from capacitive pads 602 of touch sensor 600 and / or fingers associated with each capacitive pad 602. For example, in 1120, process 1100 can identify capacitive pads 602 that have a small range between low-level capacitance values ​​or high-level capacitance values. In these situations, the capacitance values ​​received by the capacitive pads 602 can introduce noise, and ignoring particular capacitive pads 602 or groups of capacitive pads 602 can improve reliability that the touch sensor data corresponds to a user's hand gesture. That is, if the range of capacitance values ​​detected by a capacitive pad 602 is small, the capacitive pad 602 may be susceptible to a large amount of noise.

[0100] Additionally or alternatively, a particular finger may be associated with multiple capacitive pads 602 with low reliability. Rejecting a particular finger or group of capacitive pads 602 introduces contingency into the configuration of the controller for small hands. In these situations, each finger may be associated with an adjacent finger (e.g., the pinky finger associated with the ring finger).

[0101] At 1124, process 1100 can perform final normalization. For example, in some cases, a capacitive pad 602 assigned to a particular finger may not be able to detect a capacitance value, or the capacitance value may be unreliable. Herein, a user may not be able to touch a particular capacitive pad 602 of the touch sensor 600 due to hand size or in cases where the user has readjusted their grip. Furthermore, in some cases where low-level and high-level capacitance values ​​are separated by a small width or range, the capacitance value may be unreliable, and noise may significantly affect finger movement. To remove or reduce noise from these capacitive pads 602, final normalization 1124 can determine the confidence level of the capacitance value, and if the confidence level is low, the weight of the capacitance value from the capacitive pad 602 is removed from the weighted sum. In this case, the capacitance values ​​are summed and divided by the sum of the weights of the capacitive pads 602 used.

[0102] At 1126, process 1100 can filter and curve fit the touch sensor data to represent the user's hand gesture. The filtering and curve fitting can include linearizing the final normalization of the touch data on a [0, 1] scale to achieve a linear relationship between the touch sensor data and the finger position (e.g., curled, extended, partially extended, etc.). For example, the final normalization value determined at 1124 can be exponential, such that the final normalization value exponentially increases as the user's hand approaches and grasps the controller 100. In other words, the total capacitance value may be exponentially related to proximity with fingers positioned on / around the controller 100. Linearizing the capacitance value on a [0, 1] scale so that it correlates to finger position can reduce sensitivity and the effect that noise can have when a finger is extended from the controller 100 in addition to when the finger is touching or in proximity to the controller 100.

[0103] As shown, filtering and curve fitting can include various sub-blocks to achieve the final value used to generate the hand gesture. In filtering and curve fitting stage 1126, process 1100 may apply filtering either before or after curve fitting. For example, a sub-block may include filtering the capacitance values ​​to within a low-level range of capacitance values ​​if the capacitive pad 602 is susceptible to noise. In other words, within a high-level capacitance range, such as when a finger is gripping or in close proximity to the controller 100, the capacitive pad 602 is less susceptible to noise.

[0104] The process 1100 can apply adaptive filtering at 1128 to adjust the amount of filtering performed on the capacitance values. The adaptive filtering can be to more aggressively filter capacitance values ​​in a low range of capacitance values ​​compared to when the capacitance values ​​are in a high range of capacitance values. As shown, the adaptive filtering can include sub-blocks 1130, 1132, and 1134. Generally, the adaptive filtering at 1126 can use the results of sub-blocks 1130, 1132, and 1134 to determine how much noise is present in the normalized values ​​and to determine the amount of filtering to apply to the normalized capacitance values. Determining the amount of noise present in the capacitance values ​​can include determining the capacitive pads 602 used to generate the capacitance values, as well as high and low level capacitance values ​​for each capacitive pad 602. For example, the capacitive pad 602 may have a baseline noise, and if the range between high and low capacitance values ​​for the capacitive pad 602 is low, the baseline noise of the capacitive pad 602 may correspond to a large amount of finger movement (i.e., the baseline noise is a large portion of the range of capacitance values ​​that the capacitive pad 602 can sense). Here, the signal-to-noise ratio may be high. In comparison, if the range between high and low capacitance values ​​for the capacitive pad 602 is large, the baseline noise of the capacitive pad 602 may not introduce a large amount of finger movement. In these situations, to reduce the noise in the capacitance values, if the range of capacitance values ​​is small, the process 1100 may filter the capacitance values ​​more heavily than if the range of capacitance values ​​is large. The filtering and curve fitting 1126 may be repeated for each capacitive pad 602, as each capacitive pad 602 may have high and low capacitance values. Additionally, the amount of filtering applied at 1126 can depend on the capacitive pads 602 and / or the rejected capacitive pads 602 (eg, pad and finger rejection 1122).

[0105] The total noise prediction at 1130 may filter capacitance values ​​based on the capacitive pad 602 used, the weight assigned to the capacitive pad 602, as well as the respective reference noise for the capacitive pad 602. For example, the process 1100 may include a default capacitive pad noise at 1128, which may represent an estimated reference noise for each capacitive pad 602. The total noise prediction step at 1130 can therefore determine, for such capacitive pads 602 used, their respective reference noise values. The total noise prediction step can also determine an expected noise for the capacitive pad 602. For example, if the capacitive pad 602 used senses capacitance values ​​over a large range (i.e., between low and high capacitance values), the capacitance values ​​may contain a large amount of noise and less filtering may be applied. However, if the range of capacitance values ​​for the capacitive pad 602 is narrow (i.e., between low and high capacitance values), the capacitance values ​​may contain a large amount of noise and the process 1100 can apply a greater amount of filtering.

[0106] The dNorm / dt at 1132 can take into account changes in capacitance values ​​over time. For example, if the capacitance values ​​received from the capacitive pads 602 change significantly over a short period of time (e.g., one frame), the potential noise introduced into the capacitance values ​​can be ignored and weighted accordingly. That is, instead of filtering the capacitance values ​​to introduce latency, less filtering can be applied to the capacitance values ​​if they change by more than a threshold amount for a threshold amount of time. In this sense, if large finger movements are detected, less filtering can be applied, and if small finger movements are detected, more filtering can be applied.

[0107] The dCurl / dNorm in 1134 can filter the normalized capacitance value based on the amount of capacitance detected. For example, in the high range of capacitance values, where the finger is gripping the controller, noise may not have much effect on the finger position, so less filtering can be applied. However, in the low range of capacitance values, where the finger is moved away from or close to the controller, small changes in capacitance value can have a significant effect on the finger position, so more filtering can be applied. Here, small changes in capacitance value can result in large changes in finger gesture.

[0108] In sub-block 1138, the low pass filter may represent an adjustable low pass averaging filter that adjusts the amount of filtering on the detected capacitance values. In some cases, the amount of filtering may be on a [0, 1] scale and may be based on the result of the amount of filtering determined in adaptive filter 1128. That is, the low pass filter may filter the capacitance values ​​as determined from the adaptive filtering.

[0109] In sub-block 1140, process 1100 can curve fit capacitance values ​​on a [0, 1] scale and associate capacitance values ​​with finger positions or hand animations. For each finger, the output of the curve fit can include a number for each finger, the number indicating the finger position for each finger on the hand.

[0110] In sub-block 1142, process 1100 can apply a repulsion filter after curve fitting to filter changes in capacitance values ​​that fall below a threshold. For example, if the capacitance value does not change by a threshold amount on a [0, 1] scale, the capacitance value can be filtered. Such filtering can reduce finger pulls and movements perceived by the user.

[0111] In sub-block 1144, the joint model can correspond to a hand animation (e.g., a hand skeleton). For example, the joint model can create a hand animation that corresponds to the numbers assigned to the individual fingers of the hand from the curve fitting in 1140.

[0112] FIG. 12 illustrates typical components of a controller 1200, such as the controller 100. As illustrated, the controller 100 includes one or more input / output (I / O) devices 1202, such as the controls described above (e.g., joystick, trackpad, trigger, etc.), and / or potentially any other type of input or output device. For example, the I / O devices 1202 can include one or more microphones for receiving audio input, such as user voice input. In some implementations, one or more cameras or other types of sensors (e.g., inertial measurement units (IMUs)) can function as input devices for receiving gestural input, such as movements of the controller 100. In some embodiments, additional input devices can be provided in the form of a keyboard, keypad, mouse, touchscreen, joystick, control buttons, etc. The input devices can include further controls, such as basic volume control buttons for increasing and decreasing the volume, in addition to power and reset buttons. Meanwhile, the output devices can include a display, optical elements (e.g., LEDs), vibrators for creating tactile effects, speakers (e.g., headphones), and / or the like. For example, there may also be a simple optical element (e.g., an LED) that indicates a state of the controller 100, such as power on. While several examples have been provided, the controller 100 may additionally or alternatively include any other type of output device.

[0113] In some cases, output by one or more output devices may be based on input received by one or more of the input devices. For example, selecting a control touch input on the controller 100 may output a haptic response via a vibrator located near (e.g., below) the control and / or at any other location. In some cases, the output may vary based at least in part on the characteristics of the touch input on a touch sensor 600, such as a capacitive pad 602, disposed on or in the handle 112 of the controller 100. For example, a touch input at a first location on the handle 112 may result in a first haptic output, while a touch input at a second location on the handle 112 may result in a second haptic output. Furthermore, a particular gesture on the handle 112 may result in a particular haptic output (or other type of output). For example, a tap and hold gesture on the handle 112 (detected by the touch sensor 600) may result in a first type of haptic output, while a tap and release gesture on the handle 112 may result in a second type of haptic output, and a hard tap on the handle 112 may result in a third type of haptic output.

[0114] Additionally, controller 100 may include one or more communication interfaces 1204 to facilitate wireless connection to a network and / or one or more remote systems (e.g., a host computing device running an application, a game console, another controller, etc.). Communication interface 1204 may implement one or more of a variety of wireless technologies, such as Wi-Fi, Bluetooth, radio frequency (RF), etc. Additionally or alternatively, controller 100 may include a physical port to facilitate a wired connection to a network, connected peripherals, or pluggable network devices that communicate with other wireless networks.

[0115] In the illustrated implementation, the controller 100 further includes one or more processors 1206 and computer-readable media 1208. In some implementations, the processor 1206 may comprise a central processing unit (CPU), a graphics processing unit (GPU), both a CPU and a GPU, a microprocessor, a digital signal processor, or other known processing devices or components. Additionally or alternatively, the functionality described herein may be implemented, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems-on-chips (SOCs), complex programmable logic devices (CPLDs), etc. Additionally, each of the processors 1206 may have its own local memory, which may also store program components, program data, and / or one or more operating systems.

[0116] The computer-readable medium 1208 may include volatile and nonvolatile memory, as well as removable and non-removable media implemented in any method or technology for storing information, such as computer-readable instructions, data structures, program components, or other data. Such memory includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVDs) or other optical storage devices, magnetic cassettes, magnetic tape, magnetic disk storage devices or other magnetic storage devices, RAID storage systems, or any other medium usable to store the desired information and accessible by a computing device. The computer-readable medium 1208 may be implemented as a computer-readable storage medium (“CRSM”), which may be any available physical medium accessible by the processor 1206 to execute instructions stored on the computer-readable medium 1208. In one implementation, the CRSM may include random access memory (“RAM”) and flash memory. In other implementations, the CRSM may include, but is not limited to, read-only memory (“ROM”), electrically erasable programmable read-only memory (“EEPROM”), or any other tangible medium usable to store desired information and accessible by the processor 1206.

[0117] Some modules, such as instructions, data storage, etc., may be stored in the computer-readable medium 1208 and configured to execute on the processor 1206. Some exemplary functional modules are illustrated as being stored in the computer-readable medium 1208 and executed on the processor 1206, although the same functions may alternatively be implemented in hardware, firmware, or as a system on a chip (SOC). An operating system module 1210 may be configured to manage the hardware within and coupled to the controller 100, among other modules. Additionally, the computer-readable medium 1208 may store a network communication module 1212 that enables the controller 100 to communicate, via one or more of the communications interfaces 1204, with one or more other devices, such as personal computing devices running applications (e.g., game applications), game consoles, remote servers, other controllers, computing devices, etc. The computer-readable medium 1208 may further include a game session database 1214 for storing data related to games (or other applications) executing on the controller 100 or a computing device coupled to the controller 100.

[0118] The computer-readable medium 1208 may also include a device records database 1216 that stores data related to devices coupled to the controller 100, such as personal computing devices, game consoles, remote servers, etc. The computer-readable medium 1208 may further store game control instructions 1218 that configure the controller 100 to function as a game controller and universal control instructions 1220 that configure the controller 100 to function as a controller for other non-gaming devices. The computer-readable medium 1208 may additionally store a controller configuration 1222. The controller configuration 1222 may display or include data related to the assignment of the capacitive pads 602 of the touch sensor 600, associating a particular capacitive pad 602 with each finger of a user operating the controller 100.

[0119] Although the inventive subject matter has been described in language specific to structural features, it is to be understood that the inventive subject matter defined in the appended claims is not necessarily limited to the specific features described. Rather, the specific features are disclosed as exemplary forms of implementing the claims.

[0120] Terms Embodiments of the present disclosure can be described in view of the following clauses. Article 1. a touch sensor having a capacitive pad; one or more processors; One or more non-transitory computer-readable media storing computer-executable instructions that, when executed by one or more processors, cause the one or more processors to: receiving, from the touch sensor, capacitance values ​​detected by one or more of the capacitive pads, the capacitance values ​​corresponding to one or more objects touching or proximate to the controller; using the capacitance values ​​to determine, for a first controller configuration, a first variation in capacitance values ​​within individual finger groups of the first controller configuration, the individual finger groups of the first controller configuration including a middle finger, a ring finger, and a pinky finger; using the capacitance values ​​to determine, for a second controller configuration, a second variation in capacitance values ​​within individual finger groups of the second controller configuration, the individual finger groups of the second controller configuration including a middle finger, a ring finger, and a pinky finger; using the capacitance values ​​to determine, for a first controller configuration, a first variation in capacitance values ​​between individual finger groups of the first controller configuration; using the capacitance values ​​to determine, for a second controller configuration, a second variation in capacitance values ​​between individual finger groups of the second controller configuration; determining a first score for the first controller configuration based on a first variation in capacitance values ​​within individual finger groups of the first controller configuration and a first variation in capacitance values ​​between individual finger groups of the first controller configuration; determining a second score for the second controller configuration based on second variations in capacitance values ​​within individual finger groups of the second controller configuration and second variations in capacitance values ​​between individual finger groups of the second controller configuration; selecting one of the first controller configuration or the second controller configuration; and and one or more non-transitory computer-readable media configured to perform operations including configuring the touch sensor according to a configuration of the first controller or a configuration of the second controller. Article 2. Configuring the touch sensor includes: associating a first capacitive pad of the touch sensor with a middle finger; associating a second capacitive pad of the touch sensor with a ring finger; and 10. The controller of claim 1, further comprising associating a third capacitive pad of the touch sensor with the pinky finger. Article 3. 10. The controller of claim 1, wherein the capacitive pads are arranged in at least one of groups, rows, or columns. Article 4. The one or more non-transitory computer-readable media store computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform operations further including ranking the first score and the second score; 10. The controller of claim 1, wherein selecting the first controller configuration or the second configuration includes selecting the highest score from the first score and the second score. Article 5. The capacitance values ​​include a first capacitance value, and the one or more non-transitory computer-readable media store computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to: receiving, from the touch sensor, second capacitance values ​​detected by one or more of the capacitive pads, the second capacitance values ​​corresponding to one or more objects touching or proximate to the controller; using the second capacitance values ​​to determine, for the first controller configuration, a third variation in capacitance values ​​within individual finger groups of the first controller configuration; using the second capacitance values ​​to determine, for a second controller configuration, a fourth variation in capacitance values ​​within individual finger groups of the second controller configuration; using the second capacitance values ​​to determine, for the first controller configuration, a third variation in capacitance values ​​between individual finger groups of the first controller configuration; using the second capacitance values ​​to determine, for a second controller configuration, a fourth variation in capacitance values ​​between individual finger groups of the second controller configuration; determining a third score for the first controller configuration based at least in part on a third variation in capacitance values ​​within individual finger groups of the first controller configuration and a third variation in capacitance values ​​between individual finger groups of the first controller configuration; determining a fourth score for the second controller configuration based at least in part on a fourth variation in capacitance values ​​within individual finger groups of the second controller configuration and a fourth variation in capacitance values ​​between individual finger groups of the second controller configuration; selecting one of the first controller configuration or the second controller configuration; and The controller of clause 1, causing an operation to be performed, including configuring the touch sensor according to the configuration of the first controller or the configuration of the second controller. Article 6. The one or more non-transitory computer-readable media store computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to: determining a performance score for the first controller configuration based at least in part on the first score and the third score; and 6. The controller of clause 5, causing the operation to be performed, further comprising: determining a performance score for the second controller configuration based at least in part on the second score and the fourth score. Article 7. receiving data from the touch sensor representing a capacitance value detected by one or more capacitive pads of the touch sensor; determining, based at least in part on the data, a first score for a first controller configuration of the controller and a second score for a second controller configuration of the controller; The first controller is configured as follows: a first capacitive pad and a second capacitive pad assigned to a first finger; a third capacitive pad and a fourth capacitive pad assigned to a second finger; Determining the first score determining a variation in capacitance between the first capacitive pad and the second capacitive pad; determining a variation in capacitance between the third capacitive pad and the fourth capacitive pad; and determining a variation in capacitance between a total capacitance value of the first capacitive pad and the second capacitive pad and a total capacitance value of the third capacitive pad and the fourth capacitive pad; The second controller is configured as follows: a fifth capacitive pad and a sixth capacitive pad assigned to a first finger; a seventh capacitive pad and an eighth capacitive pad assigned to a second finger; Determining the second score is determining a variation in capacitance between the fifth capacitive pad and the sixth capacitive pad; determining a variation in capacitance between the seventh capacitive pad and the eighth capacitive pad; and determining a variation in capacitance between a total capacitance value of the fifth capacitive pad and the sixth capacitive pad and a total capacitance value of the seventh capacitive pad and the eighth capacitive pad; determining a selected controller configuration from among a first controller configuration and a second controller configuration; and configuring a touch sensor of the controller according to the selected controller configuration. Article 8. The data includes first data, and the method includes: receiving second data from the touch sensor representing a capacitance value detected by one or more capacitive pads of the touch sensor; associating the second data with one or more of the first finger or the second finger based at least in part on the selected controller configuration; 8. The method of clause 7, further comprising determining a bid price for at least one of the first finger or the second finger based at least in part on associating the second data with one or more of the first finger or the second finger. Article 9. 9. The method of clause 8, further comprising transmitting the bid price to one or more computing devices. Article 10. further comprising ranking the first score and the second score; 8. The method of clause 7, wherein determining the selected controller configuration includes determining a highest ranking controller configuration from among the first controller configuration and the second controller configuration. Article 11. The method of clause 7, wherein configuring the touch sensor according to the configuration of the selected controller includes associating the capacitive pads of the touch sensor according to the configuration of the first controller or associating the capacitive pads of the touch sensor according to the configuration of the second controller. Article 12. The data includes first data, and the selected controller configuration comprises a first selected controller configuration, and the method includes: receiving second data from the touch sensor representing a capacitance value detected by one or more capacitive pads of the touch sensor; determining a third score for the first controller configuration and a fourth score for the second controller configuration based at least in part on the second data; Determining the third score is determining a variation in capacitance between the first capacitive pad and the second capacitive pad; determining a variation in capacitance between the third capacitive pad and the fourth capacitive pad; and determining a variation in capacitance between a total capacitance value of the first capacitive pad and the second capacitive pad and a total capacitance value of the third capacitive pad and the fourth capacitive pad; Determining the second score is determining a variation in capacitance between the fifth capacitive pad and the sixth capacitive pad; determining a variation in capacitance between the seventh capacitive pad and the eighth capacitive pad; and determining a variation in capacitance between a total capacitance value of the fifth capacitive pad and the sixth capacitive pad and a total capacitance value of the seventh capacitive pad and the eighth capacitive pad; determining a second selected controller configuration from among the first controller configuration and the second controller configuration; and 8. The method of clause 7, comprising configuring a touch sensor of the controller according to the configuration of the second controller. Article 13. further comprising determining that the configuration of the second selected controller is different from the configuration of the first selected controller; Clause 14. The method of clause 12, wherein configuring the touch sensor of the controller in accordance with the configuration of the second selected controller is based, at least in part, on determining that the configuration of the second selected controller is different from the configuration of the first selected controller. a touch sensor having a capacitive pad; one or more processors; One or more non-transitory computer-readable media storing computer-executable instructions that, when executed by one or more processors, cause the one or more processors to: receiving data corresponding to one or more objects proximate to the controller; determining a score for the first controller configuration based at least in part on the data; determining a score for the second controller configuration based at least in part on the data; selecting a first controller configuration or a second controller configuration; and and one or more non-transitory computer-readable media configured to perform operations including configuring the touch sensor according to a configuration of the first controller or a configuration of the second controller. Article 15. The first controller is configured as follows: a first group of capacitive pads assigned to the middle finger; a second group of capacitive pads assigned to the ring finger; a third group of capacitive pads assigned to the little finger; The second controller is configured as follows: a fourth group of capacitive pads assigned to the middle finger; a fifth group of capacitive pads assigned to the ring finger; and and a sixth group of capacitive pads assigned to the little finger. Article 16. a first group of capacitive pads assigned to the middle finger for the first controller configuration is different from a fourth group of capacitive pads assigned to the middle finger for the second controller configuration; a second group of capacitive pads assigned to the ring finger for the first controller configuration is different from a fifth group of capacitive pads assigned to the ring finger for the second controller configuration; or The controller of clause 15, wherein at least one of the capacitive pads of the third group assigned to the little finger for the first controller configuration is different from the capacitive pads of the sixth group assigned to the little finger for the second controller configuration. Article 17. The data indicates a capacitance value detected by a capacitive pad of the touch sensor; The score for the first controller configuration is a variation in capacitance values ​​of the first group of capacitive pads; a variation in capacitance values ​​of the second group of capacitive pads; and a variation in capacitance value of the capacitive pads of the third group; and and a variation between the capacitance values ​​of the first group, the second group, and the third group; The score for the second controller configuration is a variation in capacitance value of the fourth group of capacitive pads; and a variation in capacitance value of the fifth group of capacitive pads; and a variation in capacitance value of the sixth group of capacitive pads; and and variation between the capacitance values ​​of the fourth group, the fifth group, and the sixth group. Article 18. 15. The controller of clause 14, wherein selecting the first controller configuration or the second controller configuration comprises selecting a highest ranking controller configuration from among the first controller configuration or the second controller configuration. Article 19. Configuring the touch sensor according to the configuration of the first controller or the configuration of the second controller includes: associating a first capacitive pad of the capacitive pads with a middle finger; associating a second capacitive pad of the capacitive pads with a ring finger; and 15. The controller of clause 14, comprising associating a third capacitive pad of the capacitive pads with a pinky finger. Article 20. One or more non-transitory computer-readable media store computer-executable instructions that, when executed by one or more processors, cause the one or more processors to perform operations including configuring a touch sensor of a controller according to another of the first controller configuration or the second controller configuration, wherein configuring the touch sensor according to another of the first controller configuration or the second controller configuration includes: associating a fourth capacitive pad of the capacitive pads with a middle finger; associating a fifth capacitive pad of the capacitive pads with a ring finger; and 20. The controller of clause 19, including associating a sixth capacitive pad of the capacitive pads with a pinky finger. The inventions described in the original claims of this application are set forth below. [1] receiving data from a touch sensor representing a capacitance value detected by one or more capacitive pads of the touch sensor; determining, based at least in part on the data, a first score for a first controller configuration of the controller and a second score for a second controller configuration of the controller; The first controller is configured as follows: a first capacitive pad and a second capacitive pad assigned to a first finger; a third capacitive pad and a fourth capacitive pad assigned to a second finger; Determining the first score includes: determining a variation in capacitance between the first capacitive pad and the second capacitive pad; determining a variation in capacitance between the third capacitive pad and the fourth capacitive pad; and determining a variation in capacitance between a total capacitance value of the first capacitive pad and the second capacitive pad and a total capacitance value of the third capacitive pad and the fourth capacitive pad; The second controller is configured as follows: a fifth capacitive pad and a sixth capacitive pad assigned to the first finger; a seventh capacitive pad and an eighth capacitive pad assigned to the second finger; Determining the second score includes: determining a variation in capacitance between the fifth capacitive pad and the sixth capacitive pad; determining a variation in capacitance between the seventh capacitive pad and the eighth capacitive pad; and determining a variation in capacitance between a total capacitance value of the fifth capacitive pad and the sixth capacitive pad and a total capacitance value of the seventh capacitive pad and the eighth capacitive pad; determining a selected controller configuration from among the first controller configuration and the second controller configuration; and configuring the touch sensor of the controller according to a selected controller configuration. [2] the data includes first data, and the method further comprises: receiving second data from the touch sensor representing a capacitance value detected by one or more capacitive pads of the touch sensor; associating the second data with one or more of the first finger or the second finger based at least in part on the selected controller configuration; The method of claim 1, further comprising determining a bid price for at least one of the first finger or the second finger based, at least in part, on associating the second data with the one or more of the first finger or the second finger. [3] The method of any of [1 or 2], further comprising transmitting the bid price to one or more computing devices. [4] further comprising ranking the first score and the second score; The method according to any one of [1 to 3], wherein determining the configuration of the selected controller includes determining the configuration of a highest-level controller from the configuration of the first controller and the configuration of the second controller. [5] A method according to any one of [1 to 4], wherein configuring the touch sensor according to the configuration of the selected controller includes associating a capacitive pad of the touch sensor according to the configuration of the first controller or associating a capacitive pad of the touch sensor according to the configuration of the second controller. [6] The data includes first data, and the selected controller configuration comprises a first selected controller configuration, and the method further comprises: receiving second data from the touch sensor representing a capacitance value detected by one or more capacitive pads of the touch sensor; determining a third score for the first controller configuration and a fourth score for the second controller configuration based at least in part on the second data; Determining the third score comprises: determining a variation in capacitance between the first capacitive pad and the second capacitive pad; determining a variation in capacitance between the third capacitive pad and the fourth capacitive pad; and determining a variation in capacitance between a total capacitance value of the first capacitive pad and the second capacitive pad and a total capacitance value of the third capacitive pad and the fourth capacitive pad; Determining the second score includes: determining a variation in capacitance between the fifth capacitive pad and the sixth capacitive pad; determining a variation in capacitance between the seventh capacitive pad and the eighth capacitive pad; and determining a variation in capacitance between a total capacitance value of the fifth capacitive pad and the sixth capacitive pad and a total capacitance value of the seventh capacitive pad and the eighth capacitive pad; determining a second selected controller configuration from among the first controller configuration and the second controller configuration; and The method according to any one of [1 to 5], comprising configuring the touch sensor of the controller according to the configuration of the second controller. [7] The method of any one of [1 to 6], further comprising determining that the configuration of the second selected controller is different from the configuration of the first selected controller, and configuring the touch sensor of the controller according to the configuration of the second selected controller is based, at least in part, on determining that the configuration of the second selected controller is different from the configuration of the first selected controller. [8] A touch sensor having a capacitive pad; one or more processors; One or more non-transitory computer-readable media storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to: receiving data corresponding to one or more objects proximate to the controller; determining a score for the first controller configuration based at least in part on the data; determining a score for the second controller configuration based at least in part on the data; selecting the first controller configuration or the second controller configuration; and and one or more non-transitory computer-readable media configured to perform operations including configuring the touch sensor according to a configuration of the first controller or a configuration of the second controller. [9] The first controller is configured as follows: a first group of capacitive pads assigned to the middle finger; a second group of capacitive pads assigned to the ring finger; a third group of capacitive pads assigned to the little finger; The second controller is configured as follows: a fourth group of capacitive pads assigned to said middle finger; a fifth group of capacitive pads assigned to said ring finger; [8] The controller of [8], further comprising: a sixth group of capacitive pads assigned to the little finger.

[10] the first group of capacitive pads assigned to the middle finger for the first controller configuration are different from the fourth group of capacitive pads assigned to the middle finger for the second controller configuration; the second group of capacitive pads assigned to the ring finger for the first controller configuration are different from the fifth group of capacitive pads assigned to the ring finger for the second controller configuration; or The controller of [8 or 9], wherein at least one of the capacitive pads of the third group assigned to the pinky finger for the first controller configuration is different from the capacitive pads of the sixth group assigned to the pinky finger for the second controller configuration.

[11] the data indicates a capacitance value detected by the capacitive pad of the touch sensor; The score for the first controller configuration is: a variation in capacitance values ​​of the first group of capacitive pads; and a variation in capacitance values ​​of the second group of capacitive pads; and a variation in capacitance values ​​of the third group of capacitive pads; and a variation between the capacitance values ​​of the first group, the second group, and the third group; The score for the second controller configuration is: a variation in capacitance values ​​of the fourth group of capacitive pads; and a variation in capacitance values ​​of the fifth group of capacitive pads; and a variation in capacitance values ​​of the sixth group of capacitive pads; and The controller according to any one of [8 to 10], comprising: a variation between the capacitance values ​​of the fourth group, the fifth group, and the sixth group.

[12] A controller described in any one of [8 to 11], wherein selecting the first controller configuration or the second controller configuration includes selecting a highest-level controller configuration from the first controller configuration or the second controller configuration.

[13] Configuring the touch sensor according to the configuration of the first controller or the configuration of the second controller includes: associating a first capacitive pad of the capacitive pads with a middle finger; associating a second capacitive pad of the capacitive pads with a ring finger; and The controller according to any one of [8 to 12], further comprising associating a third capacitive pad of the capacitive pads with a little finger.

[14] The one or more non-transitory computer-readable media store computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform operations including configuring the touch sensor of the controller according to another of the first controller configuration or the second controller configuration, wherein configuring the touch sensor according to another of the first controller configuration or the second controller configuration: associating a fourth capacitive pad of the capacitive pads with the middle finger; associating a fifth capacitive pad of the capacitive pads with the ring finger; and The controller according to any one of [8 to 13], further comprising associating a sixth capacitive pad of the capacitive pads with the little finger.

[15] The controller according to any one of [8 to 14], wherein the capacitive pads are arranged in at least one of groups, rows, or columns.

Claims

1. receiving, at a first time, first data corresponding to one or more objects proximate to a controller of a user manipulating the controller, wherein the controller comprises a left-hand controller configured to be held in a left hand or a right-hand controller configured to be held in a right hand, and the one or more objects include one or more fingers; selecting a first controller configuration of the controller associating one or more sensors of the controller with the one or more objects based at least in part on the first data; configuring the one or more sensors according to a configuration of the first controller; receiving second data corresponding to the one or more objects or one or more different objects proximate to the controller at a second time that is later than the first time, wherein a grip of the controller at the second time differs from a grip of the controller by the user at the first time by a length or width of the one or more fingers; selecting a second controller configuration of the controller that associates the one or more sensors of the controller with the one or more objects proximate to the controller or the one or more different objects based at least in part on the second data, wherein the second controller configuration differs from the first controller configuration to fit the one or more finger lengths or finger widths at the second time; and configuring the one or more sensors according to a configuration of the second controller; A method comprising:

2. transmitting the first data to one or more devices; and receiving instructions related to a configuration of the controller; Furthermore, Selecting a configuration for the first controller is based at least in part on the instruction. The method of claim 1.

3. The method of claim 1 , further comprising sending instructions to the one or more sensors to configure them according to a configuration of the controller.

4. Configuring one or more sensors according to a configuration of the first controller includes: associating a first sensor of the one or more sensors with a first object of the one or more objects; associating a second one of the one or more sensors with a second one of the one or more objects; 10. The method of claim 1, comprising:

5. The first controller is configured as follows: a first group of the one or more sensors assigned to a first object of the one or more objects; a second group of the one or more sensors assigned to a second one of the one or more objects; a third group of the one or more sensors assigned to the first object of the one or more objects; a fourth group of the one or more sensors assigned to the second one of the one or more objects; Equipped with the first group is different from the third group; or The second group is different from the fourth group.

2. The method of claim 1, wherein the at least one of

6. wherein the one or more sensors include a capacitive sensor, and the method further comprises: receiving a capacitance value detected by the capacitive sensor between the first time and the second time; associating the capacitance values ​​with the one or more objects based at least in part on a configuration of the first controller; The method of claim 1 further comprising:

7. The method of claim 1 , wherein the one or more sensors are located in or on the handle.

8. the first controller configuration is selected from a plurality of controller configurations; the second controller configuration is selected from among the plurality of controller configurations; The method of claim 1 , wherein each of the plurality of controller configurations represents a different assignment of the one or more sensors to one or more objects.

Images