Ar / VR navigation with authentication using integrated scroll wheel and fingerprint sensor user input apparatus

Abstract

To provide a VR or AR system and a method.SOLUTION: In a processing system, an integrated scroll wheel and FPS user input apparatus comprises a display, and an integrated scroll wheel and FPS user input apparatus 300 including a scroll wheel 310 for detecting a rotational navigation input from a user and a fingerprint sensor 320 (FPS) for detecting a biometric input from the user. The processing system receives the rotational navigation input via the scroll wheel 310, updates a user interface displayed on the display on the basis of the received rotational navigation input, including updating a user selection on a displayed menu, receives an activation input for the updated user selection on the displayed menu via FPS 320 of the integrated scroll wheel and FPS user input apparatus 300, and performs operation corresponding to the updated user selection on the displayed menu.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] Input devices such as touch sensors and fingerprint sensors are widely used in various electronic systems. Touch sensors and fingerprint sensors may identify the presence, position, movement, and / or characteristics of one or more input objects and often have a detection area defined by a surface. Touch sensors and fingerprint sensors may be used to provide an interface to an electronic system. For example, touch sensors and fingerprint sensors may be used as input devices for larger computing systems (e.g., an opaque touchpad and fingerprint sensor incorporated into a laptop computer or desktop computer or used as a peripheral device). Touch sensors and fingerprint sensors are also often used in smaller computing systems (e.g., a touch screen and fingerprint sensor incorporated into a smartphone).

[0002] In recent years, augmented reality (AR) and virtual reality (VR) systems have become increasingly common, and there may be use cases for touch sensors and fingerprint sensors in AR and VR systems. However, conventional touch sensor and fingerprint sensor devices are often not suitable for AR and VR environments, especially when the user cannot easily look at or view the user input device being operated.

Summary of the Invention

[0003] In exemplary embodiments, the Disclosure provides a virtual reality (VR) or augmented reality (AR) system. The system comprises a display configured to show a user interface to a user of the VR or AR system; a scroll wheel-FPS integrated user input device comprising a scroll wheel configured to detect rotational navigation input from the user and a fingerprint sensor (FPS) configured to detect biometric input from the user; and a processing system configured to update the displayed user interface by receiving the rotational navigation input via the scroll wheel of the scroll wheel-FPS integrated user input device and updating the user selection on the display menu based on the received rotational navigation input; and to receive activation input for the updated user selection on the display menu via the FPS of the scroll wheel-FPS integrated user input device and perform an action corresponding to the updated user selection on the display menu.

[0004] In a further exemplary embodiment, the processing system is configured to receive the biometric input from the FPS of the scroll wheel fingerprint sensor-integrated user input device, perform authentication based on the received biometric input, and, upon successful authentication, execute an action corresponding to the updated user selection on the display menu.

[0005] In a further exemplary embodiment, authentication based on the received biometric input includes determining whether the received biometric input matches a pre-registered saved template of an authenticated user.

[0006] In a further exemplary embodiment, the processing system is configured to receive additional activation inputs for other user selections on the display menu via the FPS of the scroll wheel FPS-integrated user input device, and to perform the other actions in response to the additional activation inputs without performing authentication for the other actions corresponding to the other user selections on the display menu.

[0007] In a further exemplary embodiment, both the scroll wheel FPS integrated user input device and the display are physically integrated into the same housing.

[0008] In a further exemplary embodiment, both the scroll wheel FPS integrated user input device and the display are physically integrated into the housing of a VR or AR headset.

[0009] In a further exemplary embodiment, the scroll wheel FPS integrated user input device and the display are physically integrated into separate housings of separate devices.

[0010] In other exemplary embodiments, the Disclosure provides a method for navigation. The method includes displaying a user interface to a user on a display; a processing system receiving rotational navigation input via a scroll wheel of a scroll wheel fingerprint sensor (FPS) integrated user input device; updating the displayed user interface on the display to update a user selection on a display menu based on the received rotational navigation input; the processing system receiving activation input for the updated user selection on the display menu; and the processing system performing an action corresponding to the updated user selection on the display menu.

[0011] In a further exemplary embodiment, the activation input is received via the FPS of the scroll wheel fingerprint sensor (FPS) integrated user input device.

[0012] In a further exemplary embodiment, the method further includes receiving biometric input via the FPS of the scroll wheel FPS-integrated user input device, and the processing system performing authentication based on the received biometric input. Performing an action corresponding to the updated user selection on the display menu is done in response to successful authentication.

[0013] In a further exemplary embodiment, authentication based on the received biometric input includes determining whether the received biometric input matches a pre-registered saved template of an authenticated user.

[0014] In a further exemplary embodiment, the method further includes the processing system receiving other activation inputs for other user selections on the display menu via the FPS of the scroll wheel FPS-integrated user input device, and the processing system executing the other actions in response to the other activation inputs without authentication for the other actions corresponding to the other user selections on the display menu.

[0015] In a further exemplary embodiment, both the scroll wheel FPS integrated user input device and the display are physically integrated into the same housing.

[0016] In a further exemplary embodiment, both the scroll wheel FPS integrated user input device and the display are physically integrated into the housing of a VR or AR headset.

[0017] In a further exemplary embodiment, the scroll wheel FPS integrated user input device and the display are physically integrated into separate housings of separate devices.

[0018] In yet another embodiment, the Disclosure provides a scroll wheel fingerprint sensor (FPS) integrated user input device. The device comprises a scroll wheel and an FPS configured to detect rotational navigation input from a user. The rotational navigation input from the user corresponds to clockwise or counterclockwise movement of the user's finger on the scroll wheel. The scroll wheel has an outer boundary and an inner boundary, the inner boundary defining the region enclosed by the scroll wheel. The FPS is located adjacent to the scroll wheel and provided in the region enclosed by the scroll wheel, and is configured to detect biometric input from the user.

[0019] In a further exemplary embodiment, the device further includes a tactile ridge provided between the scroll wheel and the FPS.

[0020] In a further exemplary embodiment, the FPS is further configured to detect activation input from the user.

[0021] In further exemplary embodiments, detecting activation input from the user is based on detecting the presence of the user's finger on the FPS, or detecting the force applied to the FPS by the user's finger.

[0022] In a further exemplary embodiment, the FPS includes an integrated button. Detecting activation input from the user is based on pressing the integrated button. [Brief explanation of the drawing]

[0023] [Figure 1A]FIG. 1A is a schematic block diagram of an exemplary input device. [Figure 1B] FIG. 1B is a schematic block diagram of an exemplary input device.

[0024] [Figure 2A] FIG. 2A is a block diagram illustrating an exemplary AR or VR environment. [Figure 2B] FIG. 2B is a block diagram illustrating an exemplary AR or VR environment. [Figure 2C] FIG. 2C is a block diagram illustrating an exemplary AR or VR environment.

[0025] [Figure 3] FIG. 3 is a schematic diagram illustrating a scroll wheel FPS integrated user input device in an exemplary embodiment.

[0026] [Figure 4A] FIG. 4A shows an exemplary manner of using the scroll wheel FPS integrated user input device shown in FIG. 3. [Figure 4B] FIG. 4B shows an exemplary manner of using the scroll wheel FPS integrated user input device shown in FIG. 3.

[0027] [Figure 5] FIG. 5 is a flowchart showing an exemplary process of using a scroll wheel FPS integrated user input device according to an exemplary embodiment.

DETAILED DESCRIPTION OF THE INVENTION

[0028] The following detailed description of the invention is essentially exemplary and is not intended to limit the present disclosure or the application and use of the present disclosure. Further, there is no intention to be bound by the theories implied or expressly stated in the above background art, brief description of the drawings, or the following detailed description of the invention.

[0029] Exemplary embodiments of this disclosure provide a device and method for navigation that performs seamless authentication in an augmented reality (AR) or virtual reality (VR) environment using a scroll wheel fingerprint sensor (FPS) integrated user input device. In various embodiments, positioning a scroll wheel around the fingerprint sensor enables a user-friendly and intuitive method for receiving various forms of user input to perform authentication-based navigation, even in situations where the user may not be able to look at or see the input device they are operating (e.g., AR and VR).

[0030] Figures 1A-1B illustrate an exemplary input device that illustrates how touch sensors and fingerprint sensors may be configured according to exemplary embodiments of the present disclosure. It will be understood that Figures 1A-1B are provided as examples, and the types of touch sensors and fingerprint sensors available in the exemplary embodiments of the present disclosure are not limited to the examples discussed herein in relation to Figures 1A-1B.

[0031] Figure 1A is a block diagram showing an exemplary input device 100 in which this embodiment may be implemented. The input device 100 may be configured to provide input to an electronic system (not shown for simplicity). In this specification, the term “electronic system” (or “electronic device”) means any system capable of processing information electronically. Examples of electronic systems include personal computing devices (e.g., desktop computers, laptop computers, netbooks, tablets, web browsers, e-book readers and personal digital assistants (PDAs)), wearable computers (e.g., smartwatches and activity tracking devices), combined input devices (e.g., physical keyboards, joysticks and key switches), data input devices (e.g., remote controls and mice), data output devices (e.g., display screens and printers), remote terminals, kiosks, game consoles (e.g., game consoles, portable game consoles, etc.), communication devices (e.g., mobile phones such as smartphones), and media devices (e.g., recorders, editors, players (e.g., televisions, set-top boxes, music players, digital photo frames) and digital cameras). In addition, the electronic system may be either a host or a slave to the input device 100.

[0032] The input device 100 may be implemented as part of the electronic system or may be physically separated from the electronic system. The input device 100 may be coupled to (and communicate with) components of the electronic system using wired or wireless interconnection and communication technologies, such as buses or networks. Illustrative technologies include Inter-Integrated Circuit (I2C), Serial Peripheral Interface (SPI), Personal System / 2 (PS / 2), Universal Serial Bus (USB), Bluetooth®, Infrared Data Association (IRDA), and various radio frequency (RF) communication protocols as defined in IEEE 802.11 or other standards.

[0033] In the example shown in Figure 1A, the input device 100 includes a sensor 105. The sensor 105 comprises one or more sensing elements configured to detect input provided by one or more input objects within the sensing area of ​​the input device 100. Examples of input objects include fingers, styluses, and hands. The sensing area may encompass any space above, around, inside, and / or near the sensor 105 on which the input device 100 can detect user input (e.g., user input provided by one or more input objects). The specific size, shape, and / or location of the sensing area (e.g., relative to an electronic system) may vary depending on the actual implementation. In some embodiments, the sensing area may extend in one or more directions from the surface of the input device 100 into space, for example, until the signal-to-noise ratio (SNR) of the sensor falls below a threshold suitable for accurate object detection. For example, the distance over which this sensing area extends in a particular direction may be on the order of less than one millimeter, on the order of millimeters, on the order of centimeters, or longer, and may vary significantly with the type of sensing technique used and / or the desired accuracy. In some embodiments, the sensor 105 may detect inputs that are not in physical contact with any surface of the input device 100, contact with an input surface of the input device 100 (e.g., a touch surface and / or a screen), contact with an input surface of the input device 100 coupled with some applied force or pressure, and / or a combination thereof. In various embodiments, the input surface may be provided by the surface of the sensor substrate on which the sensor element is located, or by a face sheet or other cover layer located above the sensor element.

[0034] The input device 100 includes one or more sensing elements to detect user input. Some implementations use an array of sensing elements or other regular or irregular patterns to detect input objects. The input device 100 may use different combinations of sensor components and sensing techniques to detect user input in the sensing area.

[0035] The input device 100 may use various sensing techniques to detect user input. Exemplary sensing techniques include capacitive, elastic, resistive, inductive, magnetic, acoustic, ultrasonic, and optical sensing techniques. In some embodiments, the input device 100 may use a capacitive sensing technique to detect user input. For example, the sensing area comprises one or more capacitive sensing elements (e.g., sensor electrodes) to generate an electric field. The input device 100 may detect the input based on a change in the capacitance of the sensor electrodes. For example, an object in contact with (or near) the electric field may cause a change in voltage and / or current at the sensor electrodes. Such a change in voltage and / or current may be detected as a “signal” indicating user input.

[0036] Sensor elements may be arranged in an array (regular or irregular pattern) or other configuration to detect input. In some implementations, separate sensing elements may be ohmic short-circuited and combined to form a larger sensor electrode. In some capacitive sensing implementations, a resistive sheet may be used to provide uniform resistance.

[0037] Exemplary capacitive sensing techniques may be based on “self-capacitance” (also known as “absolute capacitance”) and / or “mutual capacitance” (also known as “transformer capacitance”). Transformer capacitance sensing methods detect changes in the capacitive coupling between sensor electrodes. For example, an input object near the sensor electrodes may change the electric field between the sensor electrodes, thereby changing the measured value of the capacitive coupling of the sensor electrodes. In some embodiments, the input device 100 may perform transformer capacitance sensing by detecting the capacitive coupling between one or more transmitter sensor electrodes (also known as “transmitter electrodes” or “drive electrodes”) and one or more receiver sensor electrodes (also known as “receiver electrodes” or “pickup electrodes”). For example, the transmitter sensor electrodes may be modulated with respect to a reference voltage in order to transmit a transmitter signal, while the receiver sensor electrodes may be held at a relatively constant voltage in order to receive the transmitter signal. The reference voltage may be, for example, a substantially constant voltage or system ground. In some embodiments, both the transmitter sensor electrodes and the receiver sensor electrodes may be modulated. The signal received by the receiver sensor electrode may be affected by environmental interference (e.g., from other electromagnetic signals and / or from objects in contact with or near the sensor electrode). The sensor electrode may be a dedicated transmitter or receiver and may be configured to both transmit and receive.

[0038] In some implementations, the input device 100 is configured to provide images spanning one-dimensional, two-dimensional, three-dimensional, or higher-dimensional space. The input device 100 may have different sensor resolutions from embodiment to embodiment, depending on factors such as the specific detection technique and / or the scale of the information being detected. In some embodiments, the sensor resolution is determined by the physical arrangement of the array of sensing elements, although smaller sensing elements and / or smaller pitches can be used to specify a higher sensor resolution.

[0039] The input device 100 may be implemented as a fingerprint sensor having a sufficiently high sensor resolution to capture features for fingerprint identification. In some implementations, the fingerprint sensor has sufficient resolution to capture feature points (including ridge endpoints and branching points), directional fields (sometimes called "ridge flow"), and / or ridge skeletons. These may be called Level 1 and Level 2 features, and in exemplary embodiments, a resolution of at least 250 pixels per inch (ppi) is sufficient to reliably capture these features. In some implementations, the fingerprint sensor has sufficient resolution to capture higher-level features such as sweat gland pores and edge contours (i.e., the shape of the edges of individual ridges). These may be called Level 3 features, and in exemplary embodiments, a resolution of at least 750 pixels per inch (ppi) is sufficient to reliably capture these higher-level features. In silicon fingerprint sensors, a resolution of around 500 ppi is sometimes used to balance cost and performance, but it will be understood that resolutions of 5000 ppi (corresponding to a 5.08 μm pitch) or higher are achievable in silicon fingerprint sensors.

[0040] In some embodiments, the fingerprint sensor is implemented as a placement sensor (also known as an “area” sensor or “stationary” sensor) or a swipe sensor (also known as a “slide” sensor or “sweep” sensor). In a placement sensor implementation, the sensor is configured to capture fingerprint input when the user’s finger is stationary and held above the sensing area. Typically, a placement sensor comprises a two-dimensional array of sensing elements capable of capturing a desired area of ​​the fingerprint in a single frame. In a swipe sensor implementation, the sensor is configured to capture fingerprint input based on the relative movement between the user’s finger and the sensing area. In some embodiments, a swipe sensor may comprise a linear or thin two-dimensional array of sensing elements configured to capture multiple frames as the user’s finger is swiped or moves across the sensing area. The multiple frames may then be reconstructed to constitute a fingerprint image corresponding to the fingerprint input. In some implementations, the sensor is configured to capture both placement and swipe input.

[0041] In some embodiments, the fingerprint sensor is configured to capture less than the entire area of ​​the user's fingerprint in a single user input (referred to herein as a “partial” fingerprint sensor). Typically, the partial area of ​​the fingerprint captured and resulting from a partial fingerprint sensor is sufficient for the system to perform fingerprint matching from a single user input of the fingerprint (e.g., placing a finger once or swiping a finger once). Some exemplary imaging areas of a partial placement sensor are 100 mm². 2 or less than the imaging area. In other exemplary embodiments, the partially positioned sensor is 20-50 mm 2 It has an imaging area within that range. In some implementations, the partial fingerprint sensor has an input surface that is the same size as or substantially the same size as the imaging area.

[0042] In Figure 1A, the processing system 110 is included in the input device 100. The processing system 110 may comprise one or more integrated circuits (ICs) and / or some or all of other circuit components. The processing system 110 is coupled to the sensor 105 and is configured to operate the hardware of the input device 100 (e.g., the sensing hardware of the sensor 105) to detect input in the sensing region.

[0043] The processing system 110 may include a drive circuit section that applies a detection signal to the detection hardware of the input device 100 and / or a receiver circuit section configured to receive a result signal in the detection hardware. For example, the processing system 100 may be configured to apply a transmitter signal to the transmitter sensor electrode of the sensor 105 and / or to receive a detected result signal via the receiver sensor electrode of the sensor 105.

[0044] The processing system 110 may include a non-temporary computer-readable medium on which instructions executable by the processor (e.g., firmware code, software code, and / or similar) are stored. The processing system 110 can be implemented as a physical part of the sensor 105, or it may be physically separate from the sensor 105. The components of the processing system 110 may be arranged together, or they may be physically separated from each other. For example, the input device 100 may be a peripheral device coupled to a computing device, and the processing system 110 may include software configured to run on the central processing unit of the computing device, and one or more ICs (with associated firmware) separated from the central processing unit. In another example, the input device 100 may be physically integrated into a portable device, and the processing system 110 may include circuitry and firmware that are part of the main processor of the portable device. The processing system 110 may be dedicated to implementing the input device 100 and may perform other functions, such as operating a display screen or driving a tactile actuator.

[0045] The processing system 110 may operate the sensing element of the sensor 105 of the input device 100 to generate an electrical signal indicating input (or absence of input) in the detection area. The processing system 110 may perform any appropriate amount of processing on the electrical signal to translate or generate information to be provided to the electronic system. For example, the processing system 110 may digitize the analog electrical signal received via the sensor electrodes and / or filter or adjust the received signal. In some embodiments, the processing system 110 may subtract and otherwise compensate for a baseline associated with the sensor electrodes. For example, the baseline may represent the state of the sensor electrodes when no user input is detected. Thus, the information provided to the electronic system by the processing system 110 may reflect the difference between the signal received from the sensor electrodes and the baseline associated with each sensor electrode. Further examples include the processing system 110 identifying location information, recognizing input as a command, recognizing handwriting, matching biometric specimens, and so on.

[0046] In some embodiments, the input device 100 may include a touchscreen interface (e.g., a display screen) along with a fingerprint sensor, where the detection area of ​​the fingerprint sensor at least partially overlaps with the detection area of ​​the touchscreen interface. The display device may be any suitable type of dynamic display capable of displaying a visual interface to the user, such as a non-organic light-emitting diode (LED), organic LED (OLED), cathode ray tube (CRT), liquid crystal display (LCD), plasma display, organic luminescence (EL) display, or other display technology. The display may be flexible or inflexible, flat or curved, or of other shapes. The display may include a glass or plastic substrate for a thin-film transistor (TFT) circuit. The TFT circuit may be used to handle display pixels to provide visual information and / or other functions. The display device may include a cover lens (sometimes called a "cover glass") located above the display circuit and above the internal layers of the display module, the cover lens may also provide an input surface for the input device 100. Examples of cover lens materials include optically transparent amorphous solids such as chemically strengthened glass, and optically transparent crystalline structures such as sapphire. The input device 100 and the display device may share physical elements. For example, some of the same electrical components may be used for both displaying visual information and for input detection using the input device 100, such as using one or more display electrodes for both display updating and input detection. In another example, the display screen may be partially or fully operated by a processing system 110 that communicates with the input device 100.

[0047] Figure 1B is a block diagram showing an input device 100 equipped with a fingerprint sensor 105b. The fingerprint sensor 105b is configured to capture an image of a fingerprint from a finger 140. The fingerprint sensor 105b is located beneath a cover layer 112. The cover layer 112 provides an input surface on which a fingerprint is placed or swiped over the fingerprint sensor 105b. The sensing area 120 may have an input surface having an area larger, smaller, or similar in size to the entire fingerprint. The fingerprint sensor 105b has an array of sensing elements with a resolution set to detect changes in the surface of the finger 140. In some embodiments, the fingerprint sensor 105b may be located in the active area of ​​a display.

[0048] Figure 2A is a block diagram showing an exemplary AR or VR environment 200a in which exemplary embodiments of the present disclosure may be used. The environment 200a comprises, for example, an AR or VR headset 230 (e.g., in the form of goggles worn on the user's head to display VR or AR information to the user). In this embodiment, the AR or VR headset 230 may be self-contained and may have built-in processing capabilities to generate displays and receive user input (e.g., via user interface elements built into the AR or VR headset 230, such as manual control that can be operated by the user or via detection of the user's gestures by sensors).

[0049] In exemplary embodiments, the AR or VR headset 230 has a touch sensor and / or a fingerprint sensor, each having the respective processing systems implemented in the AR or VR headset 230 as described above with respect to Figure 1A-1B. A combined processing system that controls both the touch sensor and the fingerprint sensor may also be used.

[0050] Figure 2B is a block diagram showing another exemplary AR or VR environment 200b in which exemplary embodiments of the present disclosure may be used. Environment 200b comprises a host device 210 in addition to an AR or VR headset 230 (for example, in the form of goggles worn on the user's head to display VR or AR information to the user). In this embodiment, the AR or VR headset 230 communicates with the host device and, with the assistance of the host device 210, performs various processing operations—for example, related to generating displays for the user and processing received user input (for example, through user interface elements incorporated in the AR or VR headset 230, such as manual control that can be operated by the user or via sensor detection of the user's gestures). The host device 210 may be a computing device such as a personal computer, laptop, tablet, smartphone, game console, server, or any type of device capable of communicating with the AR or VR headset 230 and performing the processing tasks associated therewith.

[0051] In exemplary embodiments, the AR or VR headset 230 may include a touch sensor and / or a fingerprint sensor, each having the processing systems described above with respect to Figure 1A-1B. A combined processing system that controls both the touch sensor and the fingerprint sensor may also be used. The processing system may be implemented inside the AR or VR headset 230 and may communicate with the host device 210. Alternatively, the processing system may be implemented inside the host device 210.

[0052] Figure 2C is a block diagram showing yet another exemplary AR or VR environment 200c in which exemplary embodiments of the present disclosure may be used. The environment 200c comprises, for example, a host device 210, one or more portable user input devices 220, and an AR or VR headset 230 (for example, in the form of goggles worn on the user's head to display VR or AR information to the user). The portable user input devices 220 may comprise a single user input device to be held in one hand by the user, or two user input devices to be held in both hands by the user. The portable user input devices 220 may be, for example, a remote control device, a gaming controller, or other form suitable for being held in the user's hand, and may comprise buttons or other inputs (such as touch-sensitive surfaces) located at various positions thereon. The host device 210 may be a computing device such as, for example, a personal computer, a laptop, a tablet, a smartphone, a game console, a server, or any type of device capable of communicating with the display device 230 and the portable user input devices 220 and performing associated processing tasks.

[0053] In exemplary embodiments, one or both of the portable user input devices 220 may include a touch sensor and / or a fingerprint sensor, each having the processing systems described above with respect to Figure 1A-1B. A combined processing system controlling both the touch sensor and the fingerprint sensor may also be used. The processing system may be implemented inside the portable user input device 220 and may communicate with the host device 210. Alternatively, the processing system may be implemented inside the host device 210 or inside the AR or VR headset 230.

[0054] The environments 200a-c shown in Figures 2A-2C are merely examples, and it will be understood that exemplary embodiments of this disclosure may be implemented in other environments as well. For example, exemplary embodiments of this disclosure may be implemented in which a portable user input device is used in connection with a non-touch display screen, and the non-touch display screen is used in connection with a non-VR and non-AR system which may be implemented on the portable user input device and / or another display device. To give yet another example, the portable user input device may be replaced with a user input device that is not necessarily portable.

[0055] Figure 3 is a schematic diagram illustrating a scroll wheel FPS integrated user input device 300 in an exemplary embodiment. The scroll wheel FPS integrated user input device 300 comprises a scroll wheel 310, a fingerprint sensor (FPS) 320, and optionally a tactile ridge 330, and may be incorporated into an AR or VR headset 230 shown in Figures 2A-2C and / or a portable user input device shown in Figure 2C (based on being embedded, for example, on the side of the headset). It will also be understood that the AR or VR headset 230 and / or portable user input device 220 may have other buttons or input elements in addition to those shown in Figure 3.

[0056] The scroll wheel 310 may be, for example, a capacitive touch sensor, an optical touch sensor, or a mechanical sensor. The scroll wheel 310 is configured to detect when a user's finger (e.g., the user's thumb) is close to or in contact with the scroll wheel 310, and based on the input from the scroll wheel 310, the corresponding processing system may be able to resolve one-dimensional (1D) motion, relating to whether the user's finger is moving clockwise or counterclockwise around the scroll wheel (for example, to provide navigation that scrolls up or down or left or right in relation to the display), as well as the speed at which the user's finger is moving (for example, to provide navigation that scrolls relatively slowly or quickly in relation to the display). In one exemplary embodiment, the scroll wheel 310 may also be configured to detect two-dimensional (2D) motion, based on the detected input from the scroll wheel 310, so that the corresponding processing system can distinguish between clockwise or counterclockwise motion and other types of motion.

[0057] In the exemplary embodiment shown in Figure 3, the scroll wheel 310 has the shape of a wheel (i.e., two concentric circles defining an outer boundary and an inner boundary). However, in other embodiments, the scroll wheel 310 may have other shapes. For example, in some alternative embodiments, the scroll wheel 310 may have a shape other than circular and may be replaced by a rectangular scroll bar.

[0058] The FPS320 may be, for example, a capacitive FPS, an optical FPS, or an ultrasonic FPS. The FPS320 is configured to detect the user's fingerprint characteristics so that a corresponding processing system can authenticate the user based on the input detected from the FPS320. The FPS320 may also be force-sensitive so that it is configured to detect the magnitude of the force the user's finger is applying to the FPS320, and the corresponding processing system may register a button press action based on whether the magnitude of the force exceeds a certain threshold. Alternatively, the FPS320 may be integrated with a button, and the processing system may register a button press action based on whether the button has been pressed. In yet another alternative embodiment, the processing system may register a button press action based on whether it has detected that the user's finger is on the FPS320.

[0059] A tactile ridge 330 is optionally provided between the FPS 320 and the scroll wheel 310 so that the user can feel the inner boundary of the scroll wheel with their finger. The tactile ridge 330 can guide the user during scrolling movements and help prevent unintentional contact with the FPS 320 while attempting to scroll. The tactile ridge 330 can also help the user locate the FPS 320 when they need to perform authentication actions and / or button presses using the FPS 320. The tactile ridge 330 may be, for example, a relatively thin raised portion on the surface of the portable user input device, provided between the sensing area of ​​the FPS 320 and the sensing area of ​​the scroll wheel. The tactile ridge 330 may also have, for example, a pattern of bumps and ridges.

[0060] In one exemplary embodiment, the FPS320 and scroll wheel 310 may have separate individual controllers, or they may have an integrated controller that controls both the FPS320 and scroll wheel 310. In an exemplary embodiment, the scroll wheel 310 may be implemented as a single sensor having an embedded processor or application-specific integrated circuit (ASIC), and the FPS320 may be implemented as another sensor having another embedded processor or ASIC, so that both processors / ASICs communicate separately with the AR or VR headset and / or host device. In another exemplary embodiment, the scroll wheel 310 and FPS320, each having separate individual processors / ASICs, may be logically combined into a single input device that communicates through only one of the two processors / ASICs.

[0061] Regardless of the specific configuration of the processor / ASIC structures of the scroll wheel 310 and FPS 320, as described above, the scroll wheel 310 and FPS 320 may both be considered as part of the scroll wheel FPS integrated user input device 300, which may be integrated into an AR or VR headset. In exemplary embodiments, the scroll wheel FPS integrated user input device 300 may report to the processor of the AR or VR headset, which may then report to a host device, or it may be self-contained.

[0062] In an exemplary embodiment, the scroll wheel FPS integrated user input device 300 shown in Figure 3 may be implemented as part of a larger touchpad device, such that both the scroll wheel 310 and the FPS 320 are part of a larger touchpad device.

[0063] Figures 4A-4B illustrate exemplary embodiments of using the scroll wheel FPS integrated user input device shown in Figure 3. Left 400a of Figure 4A shows an exemplary clockwise movement made on the scroll wheel by the user's thumb 401, and right 400b of Figure 4A shows an exemplary corresponding menu displayed to the user via the display device. For example, based on the detection of the clockwise movement of the scroll wheel as illustrated in left 400a, the host device recognizes the clockwise movement as a command to navigate the menu downwards, and the host device adjusts the user interface displayed to the user accordingly by moving the current menu selection downwards by an amount based on the distance and / or speed of the clockwise movement, as shown in right 400b of Figure 4A. In this example, the user scrolled down from "Menu Option 1" to "Menu Option 7" based on a clockwise movement of the user's thumb on the scroll wheel. The user could similarly navigate back to "Menu Option 1" and then to "Menu Option 7" by moving their thumb counterclockwise on the scroll wheel.

[0064] With "Menu Option 7" being the current menu selection, the user may then provide activation or confirmation input for "Menu Option 7" by, for example, placing their finger above the FPS and pressing the FPS, pressing a button integrated with the FPS, or pressing any other button on the user input device or headset. If "Menu Option 7" supports an action using biometric authentication (for example, if the execution of a function associated with "Menu Option 7" corresponds to a financial transaction, an action requesting the user to log in, or any other action where enhanced security may be beneficial), the user may then hold their thumb above the FPS, as illustrated on the left side 400c of Figure 4B. When the user is authenticated via the FPS, the menu interface illustrated on the right side 400d may include a pop-up notification indicating that authentication is taking place. Based on the successful authentication, an action associated with "Menu Option 7" may then be performed (for example, an in-app or in-game purchase is completed, or the user is logged in to access a protected function).

[0065] As can be understood from Figures 4A-4B, exemplary embodiments of the present disclosure provide a scroll wheel FPS-integrated user input device that enhances the user experience, particularly in AR or VR systems, by providing seamless authentication integrated with navigation. Based on the proximity of the scroll wheel and the FPS sensing area (for example, the scroll wheel surrounds the FPS as shown in Figure 3), the user can quickly and intuitively transition from menu navigation to authentication and back to menu navigation without having to look at the user input device. Tactile ridges on the inner boundary of the scroll wheel further assist in guiding the user and helping the user easily locate the FPS.

[0066] It will be understood that the menu interfaces illustrated in sections 400b and 400d of Figures 4A-4B are merely illustrative, and various other types of interfaces may also use the scroll wheel FPS integrated user input device illustrated in Figure 3. For example, the scroll wheel FPS integrated user input device may be used to navigate between multiple icons (corresponding to each application) arranged in a row or grid, and may also be used for user authentication related to logging into each application selected by the user.

[0067] Figure 5 is a flowchart illustrating an exemplary process for using a scroll wheel FPS integrated user input device according to an exemplary embodiment.

[0068] In Stage 501, the user interface is displayed to the user. For example, in the environment of Figure 2, the host device may generate display information to be displayed on a VR or AR display device, and in other exemplary environments, the host device may generate display information to be displayed on other types of displays, such as conventional non-touch display screens. The user interface displayed to the user may include selectable items.

[0069] In Stage 503, navigation input is received from the user via a scroll wheel integrated with the scroll wheel FPS user input device, for example, via the scroll wheel shown in Figure 3. The scroll wheel may detect the movement of the user's finger (e.g., the user's thumb) in a certain manner (e.g., rotating clockwise or counterclockwise). Based on the movement detected via the scroll wheel, the host device may update the display in a manner corresponding to the detected movement, for example, by adjusting the current selection in the displayed user interface (e.g., adjusting the current menu selection as illustrated in Figure 4A). Other adjustments based on the movement detected via the scroll wheel include, for example, volume control or adjustment of other system settings, adjustment of the current display, or other adjustments that can intuitively correspond to the clockwise or counterclockwise rotation of the user's finger.

[0070] As described above, it will be understood that the host device may be separate from the display device and user input device (for example, as illustrated in Figure 2C), the host device may be integrated with the display device, and it may also include a user input device (for example, as illustrated in Figure 2A). For example, in one exemplary implementation, a device with a display may also include a processor for generating display information, thereby fulfilling the roles of both the host device and the display device. In another exemplary implementation, as illustrated in Figure 2A, a single device may fulfill the roles of the host device, the display device, and the user input device.

[0071] In stage 505, an activation input is received for the currently selected item in the displayed user interface. The activation input may be a user pressing a button on a user input device, such as a button integrated into the FPS of a scroll wheel FPS-integrated user input device, a button separately provided on another part of the user input device, or a button separately provided on another user input device. In another example, the activation input may be a user pressing the FPS of a scroll wheel FPS-integrated user input device, where the FPS is configured to detect the magnitude of the force applied, and the corresponding processing system is configured to detect the activation input based on whether the magnitude of the force applied is greater than a threshold. In yet another example, the activation input may be the detection of the presence of the user's finger on the FPS of a scroll wheel FPS-integrated user input device.

[0072] In one embodiment, for example, if the activation input is based on the detection of the user's finger, the system may distinguish between a slide-over motion from the scroll wheel onto the FPS (which may correspond to accidental contact with the FPS) and a lift-up motion from the scroll wheel followed by a placement motion on the FPS (which is more likely to correspond to intentional contact with the FPS). If the placement motion on the FPS occurs after the lift-up motion from the scroll wheel, the processing system may register the activation input as soon as it detects that a finger has been placed on the FPS. If a slide-over motion from the scroll wheel onto the FPS is detected, additional measures may be taken before registering the activation input. For example, the processing system may wait for at least a predetermined time until the user's finger remains (or remains stationary) on the FPS before registering the activation input. Furthermore, the processing system may request additional lift-and-put movements (i.e., lifting from the FPS and placing it back into the FPS) on the FPS before registering the activation input (in this case, a corresponding prompt may be displayed to the user to instruct them to perform the lift-and-put movements on the FPS in order to provide the activation input).

[0073] In exemplary embodiments, to detect sliding and overlapping motion, the scroll wheel may be configured as a 2D sensor capable of detecting not only clockwise and counterclockwise motion on the scroll wheel, but also diagonal sliding motion toward the center of the scroll wheel. In yet another exemplary embodiment, to detect sliding and overlapping motion, the scroll wheel and the FPS may share a processing system configured to control both the scroll wheel and the FPS, and transitions from the scroll wheel and the FPS may be detectable by the shared processing system. In yet another exemplary embodiment, to detect sliding and overlapping motion, the tactile protrusion of the scroll wheel-FPS integrated user input device may be further configured to include one or more sensing electrodes so that sliding and overlapping motion can be detected based on the detection of a transition, for example, from a state where the finger is on the scroll wheel to a state where the finger is on the FPS, passing through a state where the finger is simultaneously in contact with the scroll wheel, the FPS, and the tactile protrusion.

[0074] In one exemplary embodiment, the reception of the activation input in stage 505 corresponds to the selection of a menu item. In certain situations, such as navigating from a menu to a submenu and opening the submenu based on the selection of a menu item in the original menu, an action that does not require biometric authentication may be performed, and as a result, stages 507-511 may be unnecessary. In other situations, such as when the activation of the selected menu item results in a financial transaction, a login action, or some other action that would benefit from authentication, stages 507-511 are performed.

[0075] In Stage 507, biometric input is received via the FPS of a scroll wheel-integrated user input device. In certain exemplary implementations, scanning the FPS's sensing area in Stage 507 is performed in conjunction with receiving activation input. For example, once the user presses a button built into the FPS or places a finger on the FPS for activation, the FPS scan is automatically triggered, and the FPS scan begins so that Stages 505 and 507 can be considered to be performed simultaneously. In other exemplary implementations, scanning the sensing area is performed as a separate step, thereby prompting the user to first touch or press the FPS to provide activation input, and then to raise and place their finger again for authentication.

[0076] In Stage 509, the processing system corresponding to the FPS of the scroll wheel FPS integrated user input device performs authentication using the received biometric input (i.e., the user's fingerprint) by, for example, comparing the received biometric input with a stored template corresponding to one or more pre-registered authenticated users. It will be understood that a part of the authentication processing system may be implemented in the scroll wheel FPS integrated user input device, in a host device that communicates with the scroll wheel FPS integrated user input device, in a display device that communicates with the scroll wheel FPS integrated user input device, or in an AR or VR headset.

[0077] In Stage 511, the processing system (which may include a host device separate from the scroll wheel FPS integrated user input device) performs actions depending on the success of authentication. For example, after the user has been authenticated, the processing system may perform a financial transaction, perform a login action, or perform some other action. If authentication fails, the processing system may block the action requested by the user, and may further request the user to try authentication again.

[0078] In certain situations, there may be merit in performing continuous or periodic authentication of users logged into a system or application. For example, if a user is playing a game, the system or game application may want to continuously or periodically verify that the logged-in user is actually playing the game (and not a user who has logged in and then had another user play on their behalf) in order to prevent cheating. Similarly, if a user is using a confidential network application or system, the network application or system may want to continuously or periodically verify that the logged-in user is still using the network application or system in order to prevent fraud and / or security breaches. In these continuous or periodic authentication scenarios, a user of a scroll wheel FPS integrated user input device according to an exemplary embodiment of the Disclosure may be instructed (e.g., via the display or instruction manual) to keep the user's finger on the FPS of the scroll wheel FPS integrated user input device when not using the scroll wheel. The FPS of the scroll wheel FPS integrated user input device may then continuously or periodically scan the user's fingerprint in order to perform continuous or periodic authentication.

[0079] All references cited herein, including publications, patent applications, and patents, are incorporated herein by reference to the same extent as each reference is indicated to be incorporated by reference individually and specifically and presented herein.

[0080] In the context describing the invention (particularly in the context of the claims below), “a,” “an,” “the,” and “at least one” and similar reference subjects should be interpreted as encompassing both single and plural areas, unless otherwise indicated herein or clearly contrary to the context. The use of the term “at least one” after a list of one or more items (e.g., “at least one of A and B”) should be interpreted as meaning one item (A or B) selected from the listed items, or two or more combinations of the listed items (A and B), unless otherwise indicated herein or clearly contrary to the context. The terms “comprising,” “having,” “including,” and “containing” should be interpreted as open-ended terms (i.e., “including, but not limited to”) unless otherwise noted herein. Unless otherwise indicated herein, the ranges of values ​​described herein are intended merely as abbreviations for individually referring to each of the separate values ​​within that range, and each of the separate values ​​is incorporated into the specification as if it were described separately. All methods described herein can be performed in any appropriate order, unless otherwise indicated herein or clearly contrary to the context. Any examples or illustrative terms provided herein (e.g., “for example”) are intended merely to further illustrate the invention and do not impose limitations on the technical scope of the invention unless otherwise claimed. Nothing in the specification should be construed as indicating that any unclaimed component is essential for carrying out the invention.

[0081] Preferred embodiments of the present invention, including the best modes known to the inventors for carrying out the invention, are described herein. Variations of these preferred embodiments may be obvious to those skilled in the art by reading the above description. The inventors anticipate that those skilled in the art will use such variations as needed, and intend that the invention will be carried out in ways different from those specifically described in the specification. Accordingly, the invention includes all variations and equivalents of the subject matter described in the claims appended herein, to the extent permitted by applicable law. Furthermore, all combinations of the components described above in all possible variations are incorporated into the invention unless otherwise indicated herein or are clearly contrary to the context.

Claims

1. A virtual reality (VR) or augmented reality (AR) system, comprising: A display configured to display a user interface to a user of the VR or AR system; A scroll wheel FPS integrated user input device comprising a scroll wheel configured to detect a rotational navigation input from the user and a fingerprint sensor (FPS) configured to detect a biometric input from the user; A processing system, Receiving the rotational navigation input via the scroll wheel of the scroll wheel FPS integrated user input device; Updating the user interface displayed on the display so as to update a user selection on a display menu based on the received rotational navigation input; Receiving an activation input for the updated user selection on the display menu via the FPS of the scroll wheel FPS integrated user input device, the activation input being a detected biometric input; Performing authentication based on the detected biometric input; A processing system configured to perform an operation corresponding to the updated user selection on the display menu in response to successful authentication; Comprising, The processing system further: Receiving another activation input for another user selection on the display menu via the FPS of the scroll wheel FPS integrated user input device; Configured to execute the other operation in response to the other activation input without performing authentication for the other operation corresponding to the other user selection on the display menu; VR or AR system.

2. Performing authentication based on the detected biometric input includes determining whether the detected biometric input matches a stored template of a pre-registered authenticated user The VR or AR system according to claim 1.

3. Both the scroll wheel FPS integrated user input device and the display are physically integrated in the same housing The VR or AR system according to claim 1.

4. Both the scroll wheel FPS integrated user input device and the display are physically integrated in the housing of the VR or AR headset The VR or AR system according to claim 1.

5. The scroll wheel FPS integrated user input device and the display are physically integrated into separate enclosures of separate devices. The VR or AR system according to claim 1.

6. A method for navigation, comprising: displaying a user interface to a user by a display; receiving, by a processing system, a rotational navigation input via a scroll wheel of a scroll wheel fingerprint sensor (FPS) integrated user input device; updating the displayed user interface to update a user selection on a display menu on the display based on the received rotational navigation input; receiving, by the processing system, an activation input for the updated user selection on the display menu, the activation input being a biometric input detected via the FPS of the scroll wheel FPS integrated user input device; performing authentication by the processing system based on the detected biometric input; performing, by the processing system, an operation corresponding to the updated user selection on the display menu in response to successful authentication; receiving, by the processing system, another activation input for another user selection on the display menu via the FPS of the scroll wheel FPS integrated user input device; performing, by the processing system, the other operation in response to the other activation input without performing authentication for the other operation corresponding to the other user selection on the display menu; including method.

7. Performing authentication based on the detected biometric input includes determining whether the detected biometric input matches a stored template of a pre-registered authenticated user. The method according to claim 6.

8. Both the scroll wheel FPS integrated user input device and the display are physically integrated into the same enclosure. The method according to claim 6.

9. Both the scroll wheel FPS integrated user input device and the display are physically integrated into the enclosure of a VR or AR headset. The method according to claim 6.

10. The scroll wheel FPS integrated user input device and the display are physically integrated into separate enclosures of separate devices The method according to claim 6 [

11. ] One or more non-transitory computer-readable media having processor-executable instructions for navigation, wherein the processor-executable instructions, when executed, displaying a user interface to a user by a display receiving, by a processing system, rotational navigation input via a scroll wheel of a scroll wheel fingerprint sensor (FPS) integrated user input device updating the displayed user interface to update a user selection on a display menu on the display based on the received rotational navigation input receiving, by the processing system, an activation input for the updated user selection on the display menu, wherein the activation input is a biometric input detected via the FPS of the scroll wheel FPS integrated user input device performing authentication by the processing system based on the detected biometric input performing, by the processing system, an operation corresponding to the updated user selection on the display menu in response to successful authentication facilitating The processor-executable instructions, when executed, further receiving, by the processing system, another activation input for another user selection on the display menu via the FPS of the scroll wheel FPS integrated user input device executing, by the processing system, the other operation in response to the other activation input without performing authentication for the other operation corresponding to the other user selection on the display menu facilitating One or more non-transitory computer-readable media [

12. ] Performing authentication based on the detected biometric input includes determining whether the detected biometric input matches a stored template of a pre-registered authenticated user The one or more non-transitory computer-readable media according to claim 11 [

13. ] Both the scroll wheel FPS integrated user input device and the display are physically integrated into the same enclosure One or more non-transitory computer-readable media according to claim 11. **Claim 14** Both the scroll wheel FPS integrated user input device and the display are physically integrated into the housing of a VR or AR headset. One or more non-transitory computer-readable media according to claim 11. **Claim 15** The scroll wheel FPS integrated user input device and the display are physically integrated into separate housings of separate devices. One or more non-transitory computer-readable media according to claim 11.

Images