Auto pairing rotation vector

Abstract

An innovative technique is proposed that utilizes the Rotation Vector (RV) and Game Rotation Vector (GRV) for authentication. The proposed technique enables automatic pairing of devices when their RV / GRV are aligned with each other. The proposed technique also enables authentication of a user to a device using the RV / GRV.

Description

[Technical field]

[0001]

[0001] The present disclosure relates generally to device pairing, and, for example, to automatic pairing via rotation vectors, among other things. [Background technology]

[0002]

[0002] A rotation vector (RV) can be described as a quaternion parameterization of a device's orientation under the Earth's reference frame. For example, a fixed orientation reference frame may be defined by the directions east (E), north (N), and up (U). The RV may be specified by one or more values ​​indicating how many degrees the device has rotated about which axis (E, N, U). Electronic devices (e.g., smartphones, mobile terminals, smart glasses, wearables, etc.) may be equipped with inertial measurement unit (IMU) sensors (e.g., gyroscopes, accelerometers, magnetometers) to enable the device to calculate its own RV.

[0003]

[0003] RV has traditionally been used for positioning purposes, for example to determine the location of a device or to determine changes in the location of a device. However, the uses of RV can be extended beyond positioning applications. Summary of the Invention

[0004]

[0004] The following presents a simplified summary of one or more aspects and / or examples related to the apparatus and methods disclosed herein. As such, the following summary should not be considered an extensive overview of all contemplated aspects and / or examples, nor should the following summary be considered to identify key or critical elements of all contemplated aspects and / or examples or to delineate the scope related to any particular aspect and / or example. As such, the following summary is intended solely to present certain concepts related to one or more aspects and / or examples related to the apparatus and methods disclosed herein in a simplified form prior to the detailed description presented below.

[0005]

[0005] An exemplary first device is disclosed. The first device may include a memory, a communicator, and a processor communicatively connected to the memory and the communicator. The processor may be configured to determine a first rotation vector (RV) of a first camera of the first device utilizing an internal measurement unit (IMU). The processor may also be configured to receive one or more RVs from one or more devices, including a second RV from the second device. The second RV may be an RV of a second camera of the second device. The processor may be further configured to determine whether the second RV is aligned with the first RV. The processor may be further configured to auto-pair with the second device (120, 420) when the second RV is aligned with the first RV.

[0006]

[0006] An exemplary method of a first device is disclosed. The method may include utilizing an internal measurement unit (IMU) to determine a first rotation vector (RV) of a first camera of the first device. The method may also include receiving one or more RVs from one or more devices, including a second RV from the second device. The second RV may be an RV of a second camera of the second device. The method may further include determining whether the second RV is aligned with the first RV. The method may further include auto-pairing with the second device when the second RV is aligned with the first RV.

[0007] Another exemplary first device is disclosed. The first device may comprise means for determining a first rotation vector (RV) of a first camera of the first device utilizing an internal measurement unit (IMU). The first device may also comprise means for receiving one or more RVs from one or more devices, including a second RV from the second device. The second RV may be an RV of a second camera of the second device. The first device may further comprise means for determining whether the second RV is aligned with the first RV. The first device may further comprise means for auto-pairing with the second device when the second RV is aligned with the first RV.

[0008]

[0008] A non-transitory computer-readable medium is disclosed that stores computer-executable instructions for a configured first device. The computer-executable instructions may include one or more instructions that instruct the first device to determine a first rotation vector (RV) of a first camera of the first device using an internal measurement unit (IMU). The computer-executable instructions may also include one or more instructions that instruct the first device to receive one or more RVs from the one or more devices, including a second RV from the second device. The second RV may be an RV of a second camera of the second device. The computer-executable instructions may further include one or more instructions that instruct the first device to determine whether the second RV is aligned with the first RV. The computer-executable instructions may further include one or more instructions instructing the first device to auto-pair with the second device when the second RV is aligned with the first RV.

[0009]

[0009] An exemplary device is disclosed. The device may include a memory, a communicator, and a processor communicatively connected to the memory and the communicator. The processor may be configured to render a virtual scene based on a password of a user. The password may include a sequence of one or more symbols. The one or more symbols may include one or more visual symbols, one or more audio symbols, or both. The processor may also be configured to determine a selected vector sequence selected by the user within the virtual scene. The selected vector sequence may include a sequence of one or more vectors. Each vector may be a rotation vector (RV) or a game rotation vector (GRV). The processor may be further configured to determine whether the selected vector sequence matches the password. The processor may be further configured to authenticate the user when the selected vector sequence matches the password.

[0010]

[0010] An exemplary method of the device is disclosed. The method may include rendering a virtual scene based on a password of a user. The password may include a sequence of one or more symbols. The one or more symbols may include one or more visual symbols, one or more audio symbols, or both. The method may also include determining a selected vector sequence selected by the user in the virtual scene. The selected vector sequence may include a sequence of one or more vectors. Each vector may be a rotation vector (RV) or a game rotation vector (GRV). The method may further include determining whether the selected vector sequence matches the password. The method may further include authenticating the user when the selected vector sequence matches the password.

[0011]

[0011] Another exemplary device is disclosed. The device may comprise means for rendering a virtual scene based on a password of a user. The password may include a sequence of one or more symbols. The one or more symbols may include one or more visual symbols, one or more audio symbols, or both. The device may also comprise means for determining a selected vector sequence selected by the user in the virtual scene. The selected vector sequence may include a sequence of one or more vectors. Each vector may be a rotation vector (RV) or a game rotation vector (GRV). The device may further comprise means for determining whether the selected vector sequence matches the password. The device may further comprise means for authenticating the user when the selected vector sequence matches the password.

[0012]

[0012] A non-transitory computer-readable medium storing computer-executable instructions for a configured device is disclosed. The computer-executable instructions may include one or more instructions to instruct the device to render a virtual scene based on a user's password. The password may include a sequence of one or more symbols. The one or more symbols may include one or more visual symbols, one or more audio symbols, or both. The computer-executable instructions may also include one or more instructions to instruct the device to determine a selected vector sequence selected by the user in the virtual scene. The selected vector sequence may include a sequence of one or more vectors. Each vector may be a rotation vector (RV) or a game rotation vector (GRV). The computer-executable instructions may further include one or more instructions to instruct the device to determine whether the selected vector sequence matches the password. The computer-executable instructions may further include one or more instructions that instruct the device to authenticate the user when the selected vector sequence matches the password.

[0013]

[0013] Other features and advantages associated with the apparatus and methods disclosed herein will become apparent to one of ordinary skill in the art based on the accompanying drawings and detailed description. [Brief description of the drawings]

[0014]

[0014] A more complete understanding of the aspects and many of the attendant advantages of the present disclosure will become more readily apparent when considered with reference to the following detailed description, taken in conjunction with the accompanying drawings, which are presented merely to illustrate and not to limit the disclosure, and in which: [Figure 1]

[0015] 1A-1C depict simplified block diagrams of several sample aspects of components that may be employed in a device and configured to support authentication using rotation vectors, in accordance with one or more aspects of the present disclosure. [Diagram 2]

[0016] 1 illustrates an environment in which rotation vectors may be used for automatic pairing of devices, in accordance with one or more aspects of the present disclosure. [Diagram 3]

[0017] 1 illustrates an example scenario of user-to-user authentication using rotation vectors for automatic pairing of devices, in accordance with one or more aspects of the present disclosure. [Figure 4]

[0018] FIG. 1 illustrates a diagram of device functions and modules for user-to-user authentication using rotation vectors for automatic pairing, in accordance with one or more aspects of the present disclosure. [Diagram 5]

[0019] 1 illustrates a flowchart of an example method and process for using rotation vectors for automatic pairing, in accordance with one or more aspects of the present disclosure. [Figure 6A] 1 illustrates a flowchart of an example method and process for using rotation vectors for automatic pairing, in accordance with one or more aspects of the present disclosure. [Figure 6B] 1 illustrates a flowchart of an example method and process for using rotation vectors for automatic pairing, in accordance with one or more aspects of the present disclosure. [Figure 7] 1 illustrates a flowchart of an example method and process for using rotation vectors for automatic pairing, in accordance with one or more aspects of the present disclosure. [Figure 8]

[0020] 1 illustrates an example scenario for authenticating a user to a device using a rotation vector, in accordance with one or more aspects of the present disclosure. [Figure 9] 1 illustrates an example scenario for authenticating a user to a device using a rotation vector, in accordance with one or more aspects of the present disclosure. [Figure 10]

[0021] 1 illustrates a flowchart of an example method and process for using a rotation vector to authenticate a user, according to one or more aspects of the present disclosure. [Figure 11A] 1 illustrates a flowchart of an example method and process for using a rotation vector to authenticate a user, according to one or more aspects of the present disclosure. [Figure 11B] 1 illustrates a flowchart of an example method and process for using a rotation vector to authenticate a user, according to one or more aspects of the present disclosure. [Figure 12] 1 illustrates a flowchart of an example method and process for using a rotation vector to authenticate a user, according to one or more aspects of the present disclosure. [Figure 13] 1 illustrates a flowchart of an example method and process for using a rotation vector to authenticate a user, according to one or more aspects of the present disclosure. [Figure 14] 1 illustrates a flowchart of an example method and process for using a rotation vector to authenticate a user, according to one or more aspects of the present disclosure. [Figure 15]

[0022] 1A-1C depict simplified block diagrams of several example aspects of devices configured to utilize rotation vectors for user-to-user and user-to-device authentication in accordance with one or more aspects of the present disclosure. [Figure 16] 1A-1C depict simplified block diagrams of several example aspects of devices configured to utilize rotation vectors for user-to-user and user-to-device authentication in accordance with one or more aspects of the present disclosure. [Figure 17]

[0023] 1 illustrates various electronic devices in which one or more aspects of the present disclosure may be utilized.

[0015]

[0024] Other objects and advantages associated with the aspects disclosed herein will become apparent to those skilled in the art based on the accompanying drawings and detailed description. In accordance with common practice, features depicted by the drawings may not be drawn to scale. Thus, dimensions of depicted features may be arbitrarily increased or decreased for clarity. In accordance with common practice, some of the drawings have been simplified for clarity. Thus, the drawings may not depict all components of a particular apparatus or method. Moreover, like reference numerals refer to like features throughout the specification and figures. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016]

[0025] Aspects of the present disclosure are illustrated in the following description and related drawings directed to specific embodiments. Alternative aspects or embodiments may be devised without departing from the scope of the teachings herein. In addition, well-known elements of the exemplary embodiments herein may not be described in detail or may be omitted so as not to obscure the relevant details of the teachings in the present disclosure.

[0017]

[0026] In some described exemplary implementations, instances are identified where portions of the structure and operation of various components may be derived from known conventional techniques and then configured in accordance with one or more exemplary embodiments. In such instances, some internal details of the structure and / or operation of the known conventional components may be omitted to help avoid potentially obscuring the concepts illustrated in the exemplary embodiments disclosed herein.

[0018]

[0027] The terms used herein are for the purpose of describing particular embodiments only and are not intended to be limiting. As used herein, the singular forms "a", "an" and "the" are intended to include the plural unless the context clearly indicates otherwise. It is further understood that the terms "comprises", "comprising", "includes" and / or "including" as used herein specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0019]

[0028] As indicated above, many devices (e.g., smartphones, mobile terminals, smart glasses, etc.) may be capable of calculating their own RV based on measurements from IMU sensors (e.g., gyroscopes, accelerometers, magnetometers, etc.). Gyroscopes may provide instantaneous rotation (e.g., angle, velocity) measurements. That is, the gyroscopes may measure how fast the device is rotating.

[0020]

[0029] The accelerometer typically provides the direction of gravity in a measurement frame, i.e., the accelerometer can provide the direction of gravity relative to the device's current orientation, and can therefore be used to verify and / or correct orientation changes reported by the gyroscope against the gravity information.

[0021]

[0030] The magnetometer typically provides the device's orientation relative to magnetic north. Alternatively or in addition, if the magnetometer (or the device itself) is calibrated, an orientation relative to true north may be provided. Thus, the magnetometer may be used to verify and / or correct changes in orientation reported by the gyroscope relative to the Earth's north direction. Note that if only changes relative to the north direction are of interest, the difference between true north and magnetic north may not be significant. Note that the device may also calculate a Game RV (GRV) instead of or in addition to its own RV. In GRV, the Y-axis need not point to north, but may point to some other reference.

[0022]

[0031] 1 illustrates some example components (represented by corresponding blocks) that may be incorporated in devices 110 and 120 to support operations disclosed herein. As an example, one or both of devices 110, 120 may correspond to an end-user device such as a smartphone (also referred to as user equipment (UE)), a wearable unit such as smart glasses (e.g., for augmented reality (AR), extended reality (XR), etc.), a mobile device, etc. In another example, one or both devices may correspond to a terminal or a server that provides services to an end user.

[0023]

[0032] It will be understood that these components may be implemented in different implementations (e.g., in an ASIC, in a System-on-Chip (SoC), etc.) and in different types of devices. The components shown may also be incorporated into other devices in a communication system. For example, other devices in the system may include components similar to the described components to provide similar functionality. Also, a given device may include one or more of the components. For example, a device may include multiple transceiver components that enable the device to operate on multiple carriers and / or communicate via different technologies.

[0024]

[0033] The apparatuses 110, 120 may each include at least one communicator (represented by communicator 111, 121) for communicating with other devices. The communicators 111, 121 may be capable of communicating through wired and / or wireless protocols (e.g., wi-fi, Bluetooth, LTE, New Radio (NR), etc.). The communicator 111 may include at least one transmitter (represented by transmitter 112) for transmitting and encoding signals (e.g., messages, indicators, information, pilots, etc.) and at least one receiver (represented by receiver 113) for receiving and decoding signals (e.g., messages, indicators, information, pilots, etc.). The communicator 111 may also be referred to as a transceiver. The communicator 121 may include at least one transmitter (represented by transmitter 122) for transmitting signals (e.g., messages, indicators, information, pilots, etc.) and at least one receiver (represented by receiver 123) for receiving signals (e.g., messages, indicators, information, pilots, etc.). The communicator 111 may also be referred to as a transceiver.

[0025]

[0034] The transmitter and receiver may in some implementations comprise integrated devices (e.g., embodied as transmitter and receiver circuits in a single transceiver), in some implementations comprise separate transmitter devices and separate receiver devices, or in other implementations may be embodied in other ways. In one aspect, the transmitter may include multiple antennas, such as an antenna array, enabling each device to perform transmit "beamforming," as described further herein. Similarly, the receiver may include multiple antennas, such as an antenna array, enabling each device to perform receive beamforming, as described further herein. In one aspect, the transmitter and receiver may share the same multiple antennas, such that each device can only receive or transmit at a given time, but not both at the same time. The wireless transceiver of the device 120 (e.g., one of the wireless transceivers) may also include a Network Listen Module (NLM) for performing various measurements, and the like.

[0026]

[0035] The devices 110, 120 may also include other components used in conjunction with the operations disclosed herein. The device 110 may include a processing system 114 for providing functionality related to, for example, communication with other devices, authentication, rotation vector functionality, AR / XR functionality, object detection, etc. The device 120 may include a processing system 124 for providing functionality related to, for example, communication with other devices, authentication, rotation vector functionality, AR / XR functionality, object detection, etc. In an aspect, the processing systems 114, 124 may each include, for example, one or more general purpose processors, multi-core processors, ASICs, digital signal processors (DSPs), field programmable gate arrays (FPGAs), other programmable logic devices, processing circuits, or any combination thereof.

[0027]

[0036] The devices 110, 120 may each include a measurement component 116 and 126 for obtaining RV measurements. The measurement component 116 may measure a rotation vector associated with the device 110. The measurement component 116 may comprise a gyroscope, an accelerometer, a magnetometer, or any combination thereof. Similarly, the measurement component 126 may measure a rotation vector associated with the device 120. The measurement component 126 may comprise a gyroscope, an accelerometer, a magnetometer, or any combination thereof. The measurement components 116, 126 may also be referred to as inertial measurement units (IMUs) of the devices 110, 120.

[0028]

[0037] Apparatus 110, 120 may each include memory components 115 and 125 (e.g., each including a memory device) for maintaining information (e.g., information indicative of reserved resources, thresholds, parameters, etc.). In various implementations, memory 115 may include a computer-readable medium that stores one or more computer-executable instructions that instruct apparatus 110 (e.g., processing system 114 in combination with other aspects of apparatus 110) to perform any of the methods of FIGS. 5-7 and 10-14. Also, in various implementations, memory 125 may include a computer-readable medium that stores one or more computer-executable instructions that instruct apparatus 120 (e.g., processing system 124 in combination with other aspects of apparatus 120) to perform any of the methods of FIGS. 5-7 and 10-14.

[0029]

[0038] Additionally, devices 110, 120 may include user interfaces 117 and 127, respectively, for providing indications to a user (e.g., audible, visual, or tactile indications) and / or receiving user input (e.g., upon user actuation of a sensing device, such as a keypad, touch screen, microphone, tactile actuator, etc.).

[0030]

[0039] Devices 110, 120 may include camera components 118 and 128, respectively, for providing a view, for example, for taking still images and / or recording video. In one aspect, camera component 118 may be housed within device 110 (e.g., a cell phone camera). Alternatively or in addition, camera component 118 may be housed in a separate unit and a communication link (wired or wireless) may be established between camera component 118 and device 110. Similarly, camera 128 may be housed within device 120 (e.g., a cell phone camera). Alternatively or in addition, camera 128 may be housed in a separate unit and a communication link (wired or wireless) may be established between camera 128 and device 120.

[0031]

[0040] For convenience, devices 110, 120 are illustrated in FIG. 1 as including various components that may be configured according to various examples described herein. However, it will be understood that the illustrated blocks may have different functions in different designs. The components of FIG. 1 may be implemented in various ways. In some implementations, the components of FIG. 1 may be implemented in one or more circuits, such as, for example, one or more processors and / or one or more ASICs (which may include one or more processors), where each circuit may use and / or incorporate at least one memory component for storing information or executable code used by the circuit to provide this functionality. For example, some or all of the functionality represented by blocks 111, 114, 115, 116, 117, and 118 may be performed by the processor and memory components of device 110 (e.g., by execution of appropriate code and / or by appropriate configuration of the processor components). Similarly, some or all of the functionality represented by blocks 121, 120, 124, 125, 126, 127, and 128 may be performed by the processor and memory components of device 120 (e.g., by execution of suitable code and / or by suitable configuration of the processor components).

[0032]

[0041] Device 110 may send and receive messages to device 120 via link 160, which may be wireless, and messages include information regarding various types of communications (e.g., voice, data, multimedia services, associated control signaling, etc.). Wireless link 160 may operate over a target communications medium, exemplarily shown in FIG. 1 as medium 162, which may be shared with other communications as well as other radio access technologies (RATs). This type of medium may be comprised of one or more frequency, time, and / or spatial communications resources (e.g., encompassing one or more channels across one or more carriers) associated with communications between one or more pairs of transmitter / receivers, such as device 120 and device 110 over medium 162.

[0033]

[0042] In one aspect, it is proposed to use RV as a protocol for device pairing, e.g., automatic pairing of devices. Figure 2 shows an environment in which RV can be used for automatic pairing of devices. One or more devices in a neighborhood may each continuously update a list of nearby devices present, e.g., by using software over the air (SOTA) connection techniques. Each device may calculate its own RV, e.g., on its own processor. The RV may be proactively broadcast in network packets. A device may pair with another device by verifying that its RV is aligned with the RV of the other device.

[0034]

[0043] For the purpose of explanation, in Fig. 2, RVs associated with various devices are shown. The RVs may be broadcast by the respective devices. Here, RV1 may be assumed to represent an RV associated with a first device, and RV2 may be assumed to represent an RV associated with a second device, and an operation may be performed to pair the first device and the second device with each other. Either the first or second device may initiate the automatic pairing operation, but in the following description, it is assumed that the first device initiates it.

[0035]

[0044] The first device may recognize that RV2 is aligned with RV1. For example, the first device may verify that RV2 is in the opposite direction (plus or minus a threshold angle) to RV1. In other words, the first device and the second device (or at least their cameras) may be facing each other. Alternatively or in addition, the first device may verify that RV2 is in the same direction as RV1 (again, plus or minus a threshold angle). Note that the pairing direction may be a designer's choice. The threshold angle may be based on the accuracy of the device's measurement components, security requirements, etc.

[0036]

[0045] Upon determining that RV1 and RV2 are aligned, the first device may automatically pair with the second device. Once paired, the first device and the second device may exchange information with each other. For example, the first device may send a view to the second device. The view may be a first camera view (a view of the camera of the first device) and / or a rendered view, for example, by processing the first camera view with AR and / or XR rendering. Alternatively or in addition, the first device may receive a view from the second device. The view may be a second camera view (a view of the camera of the second device) and / or a rendered second camera view. In one aspect, the first device may further process the view received from the second device.

[0037]

[0046] One area where the proposed automatic pairing can be used is, for example, user-to-user (U2U) authentication in AR / XR situations. It can be of great value to enable AR / XR use cases (e.g., gaming, navigation, business collaboration). Various form factors (e.g., smartphones, wearables, etc.) can be supported. Figure 3 shows an example illustrating an exemplary scenario of U2U authentication using automatic pairing in RV. Here, we assume that two users are wearing smart glasses, such as AR / XR-enabled glasses. The content of the first user's glasses can be shared with the second user's glasses and vice versa. When both users see each other (e.g., for a threshold time, such as a few seconds), the devices can be paired with each other. By having the users see each other, two conditions can be guaranteed: 1) the respective RVs can be in opposite directions for alignment and / or alignment, and 2) computer vision algorithms from the AR / XR glasses can be used to verify that the second user is in the visible area. The AR / XR rendering can then respectively calculate a common virtual scene from the view angle of each device.

[0038]

[0047] Although the scenario shown in Figure 3 is described as U2U automatic pairing, it may be considered as device-to-device (D2D) automatic pairing since the first device and the second device are actually automatically paired. However, automatic pairing is also appropriate in U2U since it may be triggered by the actions of the users of the first and second devices.

[0039]

[0048] 4 shows a diagram of the systems and modules of a device for U2U authentication using rotation vectors for automatic pairing. Device 1 (or first device 410) may include a connection system 411, an RV / GRV system 413, an AR / XR system 414, an object detection 415, an IMU 416, a judgment module 417, a camera 418, and a selector 419. Each system or module 411, 413, 414, 415, 416, 417, 418, and 419 may be implemented in hardware or a combination of hardware and software. For example, each system or module 411, 413, 414, 415, 416, 417, 418, and 419 may be implemented through a hardware circuit or through one or more components of the apparatus 110 of FIG. 1.

[0040]

[0049] Similarly, device 2 (or second device 420) may include a connectivity system 421, an RV / GRV system 423, an AR / XR system 424, an object detection 425, an IMU 426, a determination module 427, a camera 428, and a selector 429. Each system or module 421, 423, 424, 425, 426, 427, 428, and 429 may be implemented in hardware or a combination of hardware and software. For example, each system or module 421, 423, 424, 425, 426, 427, 428, and 429 may be implemented through a hardware circuit or through one or more components of the apparatus 120 of FIG.

[0041]

[0050] 4, the processes performed by the first device 410 may be similar to those of the second device 420. Thus, the systems and modules of the first device 410 are described with the understanding that the description may apply to the systems and modules of the second device 420. In the first device 410, the RV / GRV system 413 may determine how the first device 410 is rotating based on measurements from the IMU 416. In one aspect, the RV / GRV system 413 may be implemented through a processor (e.g., processing system 114) and / or memory (e.g., memory component 115). The IMU 416 may be implemented through a measurement component (e.g., measurement component 116).

[0042]

[0051] Object detection 415 may perform computer vision processing of the view from camera 418. For example, object detection 415 may analyze what is in front of camera 418 (e.g., what is in front of AR / XR smart glasses) to detect one or more objects of interest. The objects of interest may include a wearable unit such as smart glasses (e.g., AR / XR glasses of the second user), a human face (e.g., the face of the second user), a mobile device (e.g., the second device 420 held by the second user), etc. In one aspect, object detection 415 may be implemented through a processor (e.g., processing system 114) and / or memory (e.g., memory component 115). Camera 418 may be implemented through a camera component (e.g., camera component 118).

[0043]

[0052] In the determination module 417, it may be determined whether an object of interest is detected. If so (i.e., there is an object within the view of the camera 418), the first RV (i.e., the RV of the first device 410) may be provided to the connection system 411 (e.g., through the mixer or selector 419). The connection system 411 may broadcast the first RV to other devices, including the second device 420, as part of a network protocol. The connection system 411 may also receive a second RV from the second device 420. The connection system 411 may then auto-pair the first device 410 with the second device 420 if the first RV and the second RV are aligned with each other. In general, auto-pairing may be considered as pairing of the first device and the second device that occurs automatically when it is determined that the first device and the second device are aligned with each other. In one aspect, the determination module 417 may be implemented through a processor (e.g., processing system 114) and / or memory (e.g., memory component 115). The connection system 411 may be implemented through a processor (e.g., processing system 114), memory (e.g., memory component 115), and / or communicator (e.g., communicator 111). The selector 419 may be implemented through a processor (e.g., processing system 114) and / or memory (e.g., memory component 115).

[0044]

[0053] 5 shows a flowchart of an example method 500 implemented by a device using RV for automatic pairing. Method 500 may also be considered an example method of implementing U2U authentication via RV. Method 500 may be implemented by a device such as any of devices 110, 120, 410, 420. For ease of reference, details of method 500 are described from the perspective of a first device (e.g., device 110, 410) implementing method 500. Thus, memory component 115 may be an example of a non-transitory computer-readable medium that stores executable instructions for the first device to implement method 500.

[0045]

[0054] In block 510, a first device (e.g., RV / GRV system 413, IMU 416) may determine a first rotation vector (RV) of a first camera (e.g., camera 418) of the first device. Means for performing block 510 may include measurement component 116, processing system 114, and / or memory component 115 of apparatus 110. Camera component 118 of apparatus 110 may be an example of a first camera.

[0046]

[0055] In block 520, a first device (e.g., connection system 411) may receive one or more RVs from one or more devices. Means for performing block 520 may include communicator 111, processing system 114, and / or memory component 115 of apparatus 110. Among the one or more RVs may be a second RV from a second device (e.g., apparatus 120, 420). The second RV may be an RV of a second camera (e.g., camera 428) of the second device.

[0047]

[0056] In block 530, the first device (e.g., RV / GRV system 413) may determine whether the second RV is aligned with the first RV. Means for performing block 530 may include the processing system 114 and / or the memory component 115 of the apparatus 110.

[0048]

[0057] 6A shows a flowchart of an example process that may be performed by a first device to implement block 530. In block 610, the first device (e.g., RV / GRV system 413) may determine whether the first RV and the second RV have comparable orientations. Means for performing block 610 may include the processing system 114 and / or the memory component 115 of the apparatus 110.

[0049]

[0058] In one embodiment, the orientations of the first and second RVs may be considered equivalent if they are in opposite orientations. For example, the cameras of the first and second devices may face each other. This is the situation shown in FIG. 3, where the first and second users are looking at each other. To allow for measurement errors, the first and second RVs may be considered aligned when they are in opposite orientations to each other within a threshold angle. For example, if the first RV is in an orientation of 0 degrees (although such orientations may be defined) and the threshold angle is 3 degrees, the second RV may be considered aligned if it is in an orientation of 177-183 degrees. Note that the threshold angle may be set to require some degree of accuracy from the user.

[0050]

[0059] Alternatively, the orientations of the first RV and the second RV may be considered equivalent if they are in the same orientation. For example, the camera of the first device and the camera of the second device may be looking in the same direction. In this case, measurement errors may also be taken into account. That is, the first device may determine that the first RV and the second RV are aligned when they are in the same orientation relative to each other within a threshold angle tolerance.

[0051]

[0060] If the first device determines that the first RV and the second RV do not have comparable orientations (the "N" branch from block 610), then the first device may determine that the first RV and the second RV are not aligned in block 650. The means for performing block 650 may include the processing system 114 and / or the memory component 115 of the apparatus 110.

[0052]

[0061] On the other hand, if the first device determines that the first RV and the second RV have comparable orientations (the "Y" branch from block 610), then the first device may determine that the first RV and the second RV are aligned in block 640. The means for performing block 640 may include the processing system 114 and / or the memory component 115 of the apparatus 110.

[0053]

[0062] However, in one embodiment, it may be desirable to verify that the user intends to align the RVs. One way for a user to indicate intent is to maintain alignment of the RVs for a threshold time, such as 2 seconds. For example, in FIG. 3, alignment of a first RV and a second RV may be deemed to be intentional if the users continue to look at each other for the threshold time. Block 620 is provided in a dashed box to indicate that it may be optional.

[0054]

[0063] In this aspect, if the first device determines that the first RV and the second RV have equivalent orientations (the "Y" branch from block 610), then the first device may determine whether a threshold time has elapsed in block 620. Means for performing block 620 may include the processing system 114 and / or the memory component 115 of the apparatus 110.

[0055]

[0064] If the first device determines that the threshold time has not yet elapsed (the "N" branch from block 620), the first device may return to block 610 to determine whether the orientations of the first and second RVs remain comparable. This means that the first and / or second devices may continuously monitor and broadcast their respective RVs. That is, blocks 510 and 520 may be performed continuously. If the first device determines that the threshold time has elapsed (the "Y" branch from block 620), the first device may return to block 640 to determine that the first and second RVs are aligned.

[0056]

[0065] Alternatively, a further check may be performed to determine whether the first and second RVs are aligned. In this alternative embodiment, in addition to the first and second RVs having comparable orientations, it may also be required to determine that the first and second users are actually facing each other. In other words, it may also be required that objects associated with the devices are visible to each other.

[0057]

[0066] This is shown in Figure 6B. In block 610, the first device (e.g., RV / GRV system 413) may determine whether the first RV and the second RV have comparable orientations. Means for performing block 610 may include the processing system 114 and / or memory component 115 of the apparatus 110. Details of an exemplary method for determining whether the first RV and the second RV have comparable orientations are discussed above and will not be repeated here for the sake of brevity.

[0058]

[0067] In block 630, the first device (e.g., object detection 415, camera 418) may detect whether an object associated with the second device is within the view of the first camera. In other words, the first device may determine whether an object of interest is within the view of the first camera. For ease of reference, this view may also be referred to as the first camera view. Such objects may include a face (e.g., a user's face), a wearable unit (e.g., smart glasses), a mobile device (e.g., the second device itself), etc. Means for performing block 630 may include the camera component 118, the processing system 114, and / or the memory component 115 of the apparatus 110.

[0059]

[0068] If the first device determines that the first RV and the second RV do not have comparable orientations (the "N" branch from block 610) or if the first device determines that the object associated with the second device is not within the first camera view (the "N" branch from block 630), then the first device may determine that the first RV and the second RV are not aligned in block 650. Means for performing block 650 may include the processing system 114 and / or the memory component 115 of the apparatus 110.

[0060]

[0069] On the other hand, if the first device determines that the first RV and the second RV have comparable orientations (the "Y" branch from block 610) and determines that an object associated with the second device is within the first camera view (the "Y" branch from block 630), then the first device may determine that the first RV and the second RV are aligned in block 640. Means for performing block 640 may include the processing system 114 and / or the memory component 115 of the apparatus 110.

[0061]

[0070] 6B, if blocks 610 and 630 both evaluate to true, then the first RV and the second RV may be determined to be aligned in block 640. If blocks 610 and / or 630 evaluate to false, then the first RV and the second RV may be determined to be unaligned in block 650.

[0062]

[0071] In one aspect, the first device may perform blocks 610 and 630 in parallel, or may perform block 610 followed by block 630, or may perform block 630 followed by block 610. When blocks 610 and 630 are performed in parallel, this may result in faster execution compared to performing the blocks sequentially. For example, if blocks 610 and 630 both evaluate to true, then performing these blocks in parallel should be faster than performing them sequentially. Of course, if block 610 (630) evaluates to false, then the performance of block (630 (610)) may be stopped.

[0063]

[0072] In one aspect, if block 610 is performed first, block 630 may be performed only when block 610 determines that there is a second RV aligned with the first RV. That is, block 630 may act as a check for block 610. In another aspect, if block 630 is performed first, block 610 may be performed only when block 630 detects that the object of interest is within the first camera view. That is, block 610 may act as a check for block 630. In this case, a low resolution camera may be sufficient. If blocks 610 and 630 are performed sequentially, there may be some power savings compared to performing blocks 610 and 630 in parallel. For example, if block 610 (630) is performed first and evaluates to false, the other block 630 (610) does not need to be performed at all.

[0064]

[0073] In one aspect, it may again be desirable to verify that the user intended to align the RVs, for example, by maintaining alignment of the RVs in each other's view for a threshold time. In this aspect, if the first device determines that the first and second RVs have comparable orientations ("Y" branch from block 610) and an object associated with the second device is within the first camera view ("Y" branch from block 630), then in block 620 the first device may determine whether the threshold time has elapsed. Means for performing block 620 may include the processing system 114 and / or memory component 115 of the apparatus 110.

[0065]

[0074] If the first device determines that the threshold time has not yet elapsed (the "N" branch from block 620), the first device may return to block 610 (to determine whether the orientations of the first and second RVs remain comparable) and block 630 (to determine whether the object associated with the second device remains within the first camera view). Again, this means that the first and / or second devices may continue to monitor and broadcast their respective RVs. If the first device determines that the threshold time has elapsed (the "Y" branch from block 620), the first device may return to block 640 to determine that the first and second RVs are aligned.

[0066]

[0075] 5, when it is determined that the first RV and the second RV are aligned, the first device (e.g., the connection system 411) may auto-pair with the second device in block 540. That is, upon determining that the first device and the second device are aligned in block 530, the first device (e.g., the RV / GRV system 413) may immediately proceed to pair with the second device. For example, the first device may pair with the second device without further input from a user of the first device. The means for performing block 540 may include the communicator 111, the processing system 114, and / or the memory component 115 of the apparatus 110.

[0067]

[0076] Alternatively, although not shown, the first device may request user permission to pair with the second device upon determining that the first and second devices are aligned in block 530. In this alternative aspect, pairing may occur when input from the user indicates that permission to pair has been granted.

[0068]

[0077] 7 shows a flowchart of an exemplary method 700 implemented by a device using RV for automatic pairing. Method 700 may be a more detailed version of method 500 and thus may also be considered as an exemplary method of implementing U2U authentication via RV. Again, the description of method 700 is provided from the perspective of a first device (e.g., device 110, 410) recognizing that the method may also be applied to other devices, such as a second device (e.g., device 120, 420). Memory component 115 may be an example of a non-transitory computer-readable medium that stores executable instructions for the first device to implement method 700.

[0069]

[0078] In block 710, the first device may determine a first rotation vector (RV) of a first camera of the first device. Block 710 may be assumed to be similar to block 510. Therefore, a detailed description thereof will be omitted for the sake of brevity.

[0070]

[0079] In block 715, the first device (e.g., connection system 411) may broadcast the first RV, for example, to other devices within the neighborhood of the first device. Means for performing block 715 may include communicator 111, processing system 114, and / or memory component 115 of apparatus 110.

[0071]

[0080] In block 720, the first device may receive one or more RVs from one or more devices, including a second RV from the second device. Block 720 may be assumed to be similar to block 520. Thus, a detailed description thereof will be omitted for brevity. It should be noted that blocks 710, 715, and 720 may be performed consecutively.

[0072]

[0081] In block 730, the first device may determine whether the second RV is aligned with the first RV. Block 730 may be assumed to be similar to block 530, including blocks in Figures 6A and 6B, and therefore a detailed description thereof will be omitted for the sake of brevity.

[0073]

[0082] When it is determined that the first RV and the second RV are aligned, the first device may auto-pair with the second device in block 740. Block 740 may be assumed to be similar to block 540, and therefore a detailed description thereof will be omitted for the sake of brevity.

[0074]

[0083] After automatic pairing with the second device, the first device (e.g., AR / XR system 414, camera 418, connectivity system 411) may share information with the second device in block 750. Means for performing block 750 may include communicator 111, camera component 118, processing system 114, and / or memory component 115 of apparatus 110.

[0075]

[0084] The shared information may include a first shared view. In one aspect, the first shared view may simply be a view of the first camera, e.g., a view captured by the first camera without any augmentation or extension. Such a view may also be referred to as a first camera view. Alternatively or in addition, the first shared view may be a rendered version of the first camera view, which may also be referred to as a first rendered view. For example, the first rendered view may be an augmented reality view of the first camera view, an extended reality view of the first camera view, or both.

[0076]

[0085] Instead of or in addition to sharing the first shared view, the first device (e.g., AR / XR system 414, connectivity system 411) may display a second shared view received from the second device in block 760. Block 760 may be performed after auto-pairing with the second device in block 740. The means for performing block 750 may include communicator 111, user interface 117, processing system 114, and / or memory component 115 of apparatus 110.

[0077]

[0086] The second shared view may simply be a view of the second camera of the second device, e.g., a view captured by the second camera without any augmentation or extension, which may also be referred to as the second camera view. Alternatively or in addition, the second shared view may be a rendered version of the second camera view, which may also be referred to as the second rendered view. For example, the second rendered view may be an augmented reality view of the second camera view, an extended reality view of the second camera view, or both. It should be noted that the first device may render the second camera view and / or may further render the second rendered view.

[0078]

[0087] As explained with respect to Figs. 2-7, RV can be used in U2U authentication (or more equivalently D2D authentication) implemented through automatic pairing of devices. However, it is also proposed to use RV for user-to-device (U2D) authentication, i.e. to use RV as a protocol to authenticate a user to a device. This can be considered as another way of implementing a login procedure to authenticate a user to a device. A user can pre-register a personal identification number (PIN) code. The PIN code can also be referred to as a password.

[0079]

[0088] In the AR / XR scene, the device may render a virtual scene that includes the user's PIN and another set of different randomly generated letters. In the virtual scene, the letters may be randomly placed in space. The device may calculate the RV / GRV (game RV). The user may select a password (i.e., PIN) sequence by facing each letter in turn and dwelling briefly on each letter. The device may compare the on-device RV / GRV log with a ground truth used during rendering to provide a pass / fail. In one aspect, the virtual scene may be larger than the viewable scene. For example, the virtual scene may be larger than the field of view (FOV) of the AR / XR glasses. The user may pan to different parts of the virtual scene when the FOV of the device is smaller than the virtual scene.

[0080]

[0089] FIG. 8 illustrates an exemplary scenario of authenticating a user to a device using rotation vectors. In particular, FIG. 8 illustrates an example virtual scene rendered by the device. A set of characters may be distributed throughout the rendered scene. The characters may vary in a number of ways, including color, font, size, etc. This means that PINs may be differentiated based on characteristics other than simply a sequence of characters. For example, a blue "Q" in blue may be differentiated from the letter "Q". As another example, a "Q" in Times Roman font may be differentiated from a "Q" in Bookman font. And even further, a 12-point "Q" may be differentiated from a 10-point "Q". Also, different security levels may be implemented. For example, RV / GRV detection may require different accuracy levels for different security level requirements.

[0081]

[0090] 8 illustrates characters as being displayed within the virtual scene, this should not be construed as limiting. Other types of visual components (e.g., icons, emojis, etc.) may be displayed as well. Thus, it may be said that the device may render a virtual scene including a set of symbols. The symbols may be any combination of visual elements such as characters, icons, emojis, etc. Again, the symbols may be differentiated based on any characteristic (e.g., color, size, font, etc.).

[0082]

[0091] Alternatively or additionally, if spatial audio is available in the device, the password may include one or more predefined sounds. This is illustrated in FIG. 9. When a certain sound event occurs (e.g., at 45 degrees right), the user may respond by turning the device so that the sound event is at the center. For example, assume that the password predefined sound includes a waterfall sound. In the virtual audio scene, the device may render a password sound (e.g., a waterfall sound) with a random direction associated with the RV. Along with the password sound, the device may also render another sound (e.g., a glass breaking sound) associated with a different RV. Then, to unlock, the user should track the sound associated with the password sound from among the rendered sounds, for example, by orienting the device to the center of the sound. Because the RV of the password sound may change at different authentication times, the user should track the sound every time for authentication.

[0083]

[0092] 10 shows a flowchart of an example method 1000 implemented by a device that uses an RV to authenticate a user. The method 1000 may also be considered an example method of implementing U2D authentication via an RV. The method 1000 may be implemented by a device such as any of the devices 110, 120, 410, 420. For ease of reference, details of the method 1000 are described in terms of the device 110 or 410 as implementing the method 1000. Thus, the memory component 115 may be an example of a non-transitory computer-readable medium that stores executable instructions for a first device to implement the method 1000.

[0084]

[0093] In block 1010, the device (e.g., AR / XR system 414) may render a virtual scene based on a password of the user. The password may be pre-registered on the device and may include a sequence of one or more symbols. The password symbols may include one or more visual symbols, one or more audio symbols, or both. Means for performing block 1010 may include the user interface 117, the processing system 114, and / or the memory component 115 of the apparatus 110. With respect to visual symbols, it should be noted that a visual symbol may be distinguished from another visual symbol based on one or more characteristics such as color, font (if the symbol is a character), size, etc.

[0085]

[0094] 11A shows a flowchart of an example process that may be performed by a device to implement block 1010 when the password includes one or more visual symbols. In block 1110, the device (e.g., RG / GRV system 413, AR / XR system 414) may distribute visual symbols of the password throughout the virtual scene. Means for implementing block 1110 may include the user interface 117, the processing system 114, and / or the memory component 115 of the apparatus 110. In one aspect, the visual symbols may be distributed randomly. For example, the distribution of visual symbols of the password in one authentication attempt may be different from the distribution of visual symbols of the password in another authentication attempt.

[0086]

[0095] Optionally, in block 1120, the device (e.g., RG / GRV system 413, AR / XR system 414) may distribute one or more visual symbols that are not included in the password throughout the virtual scene. Means for implementing block 1120 may include the user interface 117, the processing system 114, and / or the memory component 115 of the apparatus 110. In an aspect, these non-password visual symbols may be randomly generated. For example, the non-password visual symbols generated in one authentication attempt may not be the same as the non-password visual symbols generated in another authentication attempt. Alternatively or in addition, these non-password visual symbols may be randomly distributed. For example, the distribution of non-password visual symbols in one authentication attempt may be different from the distribution of non-password visual symbols in another authentication attempt.

[0087]

[0096] 11B shows a flowchart of an example process that may be performed by the device to implement block 1010 when the password includes one or more phonetic symbols. In block 1115, for each phonetic symbol of the password, the device (e.g., RG / GRV system 413, AR / XR system 414) may render the phonetic symbol in an RV or GRV determined for the phonetic symbol. Means for performing block 1115 may include the user interface 117, the processing system 114, and / or the memory component 115 of the apparatus 110. In one aspect, the RV or GRV for the phonetic symbol may be determined randomly. For example, the RV or GRV determined for the phonetic symbol in one authentication attempt may be different from the RV or GRV determined for the phonetic symbol in another authentication attempt.

[0088]

[0097] In block 1125, for at least one phonetic symbol of the password, the device (e.g., RG / GRV system 413, AR / XR system 414) may render another phonetic symbol in another RV or another GRV determined for the other phonetic symbol. Means for performing block 1125 may include the user interface 117, the processing system 114, and / or the memory component 115 of the apparatus 110.

[0089]

[0098] At least one sound symbol may be different from another sound symbol. For example, at least one sound symbol may be a symbol of a waterfall and another sound symbol may be a sound of glass breaking. Also, an RV or GRV may be different from another RV or another GRV. For example, if at least one sound symbol is rendered as emanating from the left, another sound symbol may be rendered as emanating from the right.

[0090]

[0099] Furthermore, the at least one phonetic symbol and another phonetic symbol may be rendered simultaneously. For example, the at least one phonetic symbol and another phonetic symbol may be rendered simultaneously or where the rendering of the at least one phonetic symbol and another phonetic symbol at least partially overlap each other. More generally, a time window may be defined during which both the at least one phonetic symbol and another phonetic symbol will be rendered, and the user may select between the phonetic symbols (e.g., by pointing at the rendered sound) during or immediately after the time window has elapsed.

[0091]

[0100] 10, in block 1020, a device (e.g., RG / GRV system 413, AR / XR system 414, IMU 416) may determine a sequence of one or more vectors selected by a user, also referred to as a selected vector sequence for clarity. Each vector of the selected vector sequence may be an RV or a GRV. Means for performing block 1020 may include the user interface 117, the processing system 114, and / or the memory component 115 of the apparatus 110.

[0092]

[0101] At block 1020, the user may reorient the device to enter a password in the virtual space. Note that block 1020 may apply when the password includes one or more visual symbols or when the password includes one or more phonetic symbols.

[0093]

[0102] 12 shows a flowchart of an example process that may be performed by a device to implement block 1020. In block 1210, the device (e.g., RV / GRV system 413, IMU 416) may determine a vector of the device that is an RV or a GRV. Means for performing block 1210 may include measurement component 116, processing system 114, and / or memory component 115 of apparatus 110.

[0094]

[0103] The device may log the vectors in the selected vector sequence in block 1230. Means for performing block 1230 may include the processing system 114 and / or the memory component 115 of the apparatus 110.

[0095]

[0104] In block 1240, the device may determine whether the vector sequence selection is finished. If not ("N" branch from block 1240), the device may return to block 1210. If not ("Y" branch from block 1240), the device may exit the process of implementing block 1020. Means for performing block 1240 may include the processing system 114 and / or the memory component 115 of the apparatus 110.

[0096]

[0105] In one aspect, it may be desirable to verify that a user intends to select a symbol in a virtual scene. One way for a user to indicate intent is for the user to explicitly indicate the selection of a symbol, for example, through a user interface (not shown). For example, if the virtual scene is displayed on a display of a device, such as a touch screen of a mobile device, the user may indicate by tapping the selected symbol on the screen. As another example, if the virtual scene is displayed on smart glasses, such as AR / XR glasses, the user may orient the glasses to center the selected symbol in the view and tap a button input.

[0097]

[0106] Another method is for the user to aim the device at the selected symbol and maintain that orientation for a threshold time, such as 2 seconds. Thus, after block 1210, the device (e.g., RV / GRV system 413, IMU 416) may determine whether the vector is held for the threshold time in block 1220. Means for performing block 1220 may include measurement component 116, processing system 114, and / or memory component 115 of apparatus 110.

[0098]

[0107] If the vector is held for the threshold time (the "Y" branch from block 1220), the device may proceed to block 1230 to log the vector. If not (the "N" branch from block 1220), the device may proceed to block 1240 to determine whether the vector sequence selection process is complete.

[0099]

[0108] 10, in block 1030, the device (e.g., RV / GRV system 413, AR / XR system 414) may determine whether the selected vector sequence matches the password. Means for performing block 1030 may include the processing system 114 and / or memory component 115 of the apparatus 110. Note that block 1030 may also apply when the password includes one or more visual symbols or when the password includes one or more phonetic symbols.

[0100]

[0109] 13 shows a flowchart of an example process that may be performed by a device to implement block 1030. In this process, a vector sequence corresponding to a password (password vector sequence) may be compared to a selected vector sequence. In block 1310, the device (e.g., RV / GRV system 413, AR / XR system 414) may generate a password vector sequence including one or more vectors based on the password and the virtual scene. Each vector in the password vector sequence may be an RV or a GRV. The means for performing block 1310 may include the processing system 114 and / or the memory component 115 of the apparatus 110.

[0101]

[0110] Recall that the symbols distributed in the virtual scene include the symbols of a password. The device may then determine, for each symbol (visual or audio) of the password, a corresponding vector in the virtual scene. In one aspect, the password vector sequence may be a sequence of RVs or GRVs randomly generated in block 1110 and / or block 1115. Thus, the password vector sequence may be generated in block 1310.

[0102]

[0111] In block 1320, the device may determine whether the number of vectors in the password vector sequence is equal to the number of vectors in the selected vector sequence. Means for performing block 1320 may include the processing system 114 and / or the memory component 115 of the apparatus 110.

[0103]

[0112] If the number of vectors in the password vector sequence and the number of vectors in the selected vector sequence are not equal (the "N" branch from block 1320), the device may determine that the selected vector sequence does not match the password in block 1340. Means for performing block 1340 may include the processing system 114 and / or the memory component 115 of the apparatus 110.

[0104]

[0113] If the number of vectors in the password vector sequence and the number of vectors in the selected vector sequence are equal ("Y" branch from block 1320), in block 1330, the device may determine whether all vectors in the password vector sequence match the corresponding vectors in the selected vector sequence within a threshold angle. For example, in order for a vector in the selected vector sequence to match the corresponding vector in the password vector sequence, the vector in the selected vector sequence should be within a threshold angle of the corresponding vector in the password vector sequence. The threshold angle may be set according to a desired security level. For example, if the security requirement is high, the threshold angle may be set low, i.e., narrow. This means that a higher accuracy is required from the user when the selected vector sequence is generated. The means for performing block 1330 may include the processing system 114 and / or the memory component 115 of the apparatus 110.

[0105]

[0114] If all vectors in the password vector sequence do not match the corresponding vectors in the selected vector sequence (the "N" branch from block 1330), the device may proceed to block 1340 to determine that the selected vector sequence does not match the password. Means for performing block 1340 may include the processing system 114 and / or the memory component 115 of the apparatus 110.

[0106]

[0115] If all vectors of the password vector sequence match corresponding vectors of the selected vector sequence (the "Y" branch from block 1330), then the device may determine that the selected vector sequence matches the password in block 1350. Means for performing block 1350 may include the processing system 114 and / or the memory component 115 of the apparatus 110.

[0107]

[0116] 14 shows a flowchart of another exemplary process that may be performed by a device to implement block 1030. In this process, a sequence of symbols corresponding to a selected vector sequence (selected symbol sequence) may be compared to a password. In block 1410, the device (e.g., RV / GRV system 413, AR / XR system 414) may generate a selected symbol sequence including one or more symbols based on the selected vector sequence. Means for performing block 1410 may include the processing system 114 and / or memory component 115 of the apparatus 110.

[0108]

[0117] In the case of visual symbols, each symbol of the selected symbol sequence may be a visual symbol located within a threshold angle of a location in the virtual scene indicated by a corresponding vector of the selected vector sequence. In the case of audio symbols, each symbol of the selected symbol sequence may be an audio symbol rendered within a threshold angle in the virtual scene. The device may then determine, for each vector of the selected vector sequence, a symbol (visual or audio) located at a location in the virtual scene within the threshold angle. Thus, the selected symbol sequence may be generated in block 1410. Again, the threshold angle may be set based on a desired security level.

[0109]

[0118] In block 1420, the device may determine whether the number of symbols in the password and the number of symbols in the selected symbol sequence are equal. Means for performing block 1420 may include the processing system 114 and / or the memory component 115 of the apparatus 110.

[0110]

[0119] If the number of symbols in the password and the number of symbols in the selected symbol sequence are not equal (the "N" branch from block 1420), then the device may determine that the selected vector sequence does not match the password in block 1440. Means for performing block 1440 may include the processing system 114 and / or the memory component 115 of the apparatus 110.

[0111]

[0120] If the number of symbols in the password and the number of symbols in the selected symbol sequence are equal (the "Y" branch from block 1420), then in block 1430 the device may determine whether all symbols in the password match corresponding symbols in the selected symbol sequence.

[0112]

[0121] If all symbols of the password do not match the corresponding symbols of the selected symbol sequence (the "N" branch from block 1430), the device may proceed to block 1440 and determine that the selected vector sequence does not match the password.

[0113]

[0122] If all symbols of the password match corresponding symbols of the selected symbol sequence (the "Y" branch from block 1430), then the device may determine that the selected vector sequence matches the password in block 1450. Means for performing block 1450 may include the processing system 114 and / or the memory component 115 of the apparatus 110.

[0114]

[0123] 10, in block 1040, the device may authenticate the user when the selected vector sequence matches the password. Means for performing block 1040 may include the processing system 114 and / or the memory component 115 of the apparatus 110.

[0115]

[0124] FIG. 15 illustrates an exemplary device 1500 represented as a series of interrelated functional modules connected by a common bus. Each of the modules may be implemented in hardware or as a combination of hardware and software. For example, the modules may perform the methods and processes of FIGS. 5-7 and may be implemented as any combination of modules of the systems / devices 110, 120, 410, 420 of FIGS. 1 and 4. The module 1510 for determining the first RV may correspond, at least in some aspects, to a measurement component (e.g., measurement component 116), a processor (e.g., processing system 114), and / or a memory (e.g., memory component 115). The module 1515 for broadcasting the first RV may correspond, at least in some aspects, to a communicator (e.g., communicator 111), a processor (e.g., processing system 114), and / or a memory (e.g., memory component 115). The module 1520 for receiving one or more RVs may correspond, at least in some aspects, to a communicator (e.g., the communicator 111), a processor (e.g., the processing system 114), and / or a memory (e.g., the memory component 115). The module 1530 for determining if the second RV is aligned with the first RV may correspond, at least in some aspects, to a processor (e.g., the processing system 114), and / or a memory (e.g., the memory component 115). The module 1540 for auto-pairing with the second device may correspond, at least in some aspects, to a communicator (e.g., the communicator 111), a processor (e.g., the processing system 114), and / or a memory (e.g., the memory component 115). The module 1550 for sharing the first shared view with the second device may correspond, at least in some aspects, to a communicator (e.g., the communicator 111), a processor (e.g., the processing system 114), and / or a memory (e.g., the memory component 115).The module for displaying the shared second view received from the second device 1560 may correspond to at least a communicator (e.g., communicator 111), a user interface (e.g., user interface 117), a processor (e.g., processing system 114), and / or a memory (e.g., memory component 115).

[0116]

[0125] FIG. 16 illustrates an exemplary device 1600 represented as a series of interrelated functional modules connected by a common bus. Each of the modules may be implemented in hardware or as a combination of hardware and software. For example, the modules may perform the methods and processes of FIGS. 10-14 and may be implemented as any combination of modules of the systems / devices 110, 120, 410, 420 of FIGS. 1 and 4. The module 1610 for rendering a virtual scene may correspond, at least in some aspects, to a user interface (e.g., user interface 117), a processor (e.g., processing system 114), and / or memory (e.g., memory component 115). The module 1620 for determining a selected vector sequence may correspond, at least in some aspects, to a processor (e.g., processing system 114) and / or memory (e.g., memory component 115). The module for determining whether the selected vector sequence matches the password 1630 may correspond, at least in some aspects, to a processor (e.g., processing system 114) and / or a memory (e.g., memory component 115). The module for authenticating the user 1640 may correspond, at least in some aspects, to a user interface (e.g., user interface 117), a processor (e.g., processing system 114), and / or a memory (e.g., memory component 115).

[0117]

[0126] 17 illustrates various electronic devices that may be integrated with any of the aforementioned systems / devices according to various aspects of the disclosure. For example, a mobile telephone device 1702, a laptop computer device 1704, and a terminal device 1706 may include an RV / GRV authentication device 1700. The devices 1702, 1704, 1706 illustrated in FIG. 17 are merely exemplary. Other electronic devices may also include a group of devices (e.g., electronic devices) including, but not limited to, mobile devices, handheld personal communication systems (PCS) units, portable data units such as personal digital assistants, global positioning system (GPS) enabled devices, navigation devices, set-top boxes, music players, video players, entertainment units, stationary data units such as meter reading equipment, communication devices, smartphones, tablet computers, computers, wearable devices, servers, routers, electronic devices implemented within automotive vehicles (e.g., autonomous vehicles), Internet of things (IoT) devices, or any other device that stores or retrieves data or computer instructions, or any combination thereof.

[0118]

[0127] The following numbered clauses describe example implementations.

[0119]

[0128] Clause 1: A method of a first device, comprising: utilizing an internal measurement unit (IMU) to determine a first rotation vector (RV) of a first camera of the first device; receiving one or more RVs from one or more devices including a second RV from the second device, which is an RV of a second camera of the second device; determining whether the second RV is aligned with the first RV; and automatically pairing with the second device when the second RV is aligned with the first RV.

[0120]

[0129] Clause 2: The method of clause 1, wherein determining whether the second RV is aligned with the first RV includes determining that the second RV is aligned with the first RV when the first RV and the second RV have equivalent orientations, wherein the first RV and the second RV have equivalent orientations if the orientation of the first RV is opposite to the orientation of the second RV within a threshold angle or if the orientation of the first RV is the same as the orientation of the second RV within a threshold angle.

[0121]

[0130] Clause 3: The method of clause 2, wherein determining whether the second RV is aligned with the first RV further includes determining that the second RV is aligned with the first RV when the orientations of the first RV and the second RV remain equivalent for a threshold time.

[0122]

[0131] Clause 4: The method of clause 1, wherein determining whether the second RV is aligned with the first RV includes determining whether the first RV and the second RV have equivalent orientations, where the first RV and the second RV have equivalent orientations when an orientation of the first RV is opposite to an orientation of the second RV within a threshold angle or when an orientation of the first RV is the same as an orientation of the second RV within a threshold angle, and determining whether an object associated with the second device is detected in a first camera view, the first camera view, wherein the second RV is determined to be aligned with the first RV when the first RVS and the second RVS have equivalent orientations and an object associated with the second device is detected in the first camera view.

[0123]

[0132] Clause 5: The method of clause 4, wherein the object associated with the second device is any one or more of a face, a wearable unit, and a mobile device.

[0124]

[0133] Clause 6: The method of clause 5, wherein the wearable unit is a smart glass.

[0125]

[0134] Clause 7: Any of the methods of clauses 1 to 6, further comprising broadcasting the first RV.

[0126]

[0135] Clause 8: Any of the methods of clauses 1 to 7, further comprising sharing the first shared view with the second device following automatic pairing with the second device, wherein the first shared view is the first camera view or the first rendered view, wherein the first camera view is a view of the first camera, and the first rendered view is a view after rendering the first camera view.

[0127]

[0136] Clause 9: The method of clause 8, wherein the first rendered view is an augmented reality (AR) view of the first camera view, an extended reality (XR) view of the first camera view, or both.

[0128]

[0137] Clause 10: Any of the methods of clauses 1 to 9, further comprising displaying a second shared view received from the second device following automatic pairing with the second device, the second shared view being a second camera view or a second rendered view, the second camera view being a view of the second camera, and the second rendered view being a view after rendering the second camera view.

[0129]

[0138] Clause 11: A method of a device comprising: rendering a virtual scene based on a user's password comprising one or more sequences of symbols, the sequences of symbols comprising one or more visual symbols, one or more audio symbols, or both; determining a selected vector sequence selected by the user within the virtual scene, the selected vector sequence comprising a sequence of one or more vectors, each vector being a rotation vector (RV) or a game rotation vector (GRV); determining whether the selected vector sequence matches the password; and authenticating the user when the selected vector sequence matches the password.

[0130]

[0139] Clause 12: The method of clause 11, wherein the password includes one or more visual symbols, and wherein when rendering the virtual scene, the method includes dispersing the one or more visual symbols of the password throughout the virtual scene.

[0131]

[0140] Clause 13: The method of clause 12, wherein rendering the virtual scene further comprises dispersing one or more visual symbols not included in the password throughout the virtual scene.

[0132]

[0141] Clause 14: The method of any of clauses 11 to 13, wherein the password includes one or more phonetic symbols, and wherein rendering the virtual scene includes, for each phonetic symbol of the password, rendering an phonetic symbol in an RV or GRV determined for the phonetic symbol, and for at least one phonetic symbol of the password, rendering another phonetic symbol in another RV or another GRV determined for another phonetic symbol, wherein the at least one phonetic symbol and the another phonetic symbol are rendered simultaneously, and the at least one phonetic symbol is different from the another phonetic symbol, and the RV or GRV is different from the another RV or another GRV.

[0133]

[0142] Clause 15: Any of the methods of clauses 11 to 14, wherein determining the selected vector sequence includes determining a vector of the device that is an RV or a GRV, and logging the vectors in the selected vector sequence, and the determining and logging of the vectors are repeated until the vector sequence selection process is completed.

[0134]

[0143] Clause 16: The method of clause 15, wherein determining the selected vector sequence further comprises logging a vector in the selected vector sequence when the vector is held for a threshold time.

[0135]

[0144] Clause 17: The method of any of clauses 11-16, wherein determining whether the selected vector sequence matches the password includes generating a password vector sequence based on the password and the virtual scene, the password vector sequence including one or more vectors, each vector being an RV or a GRV; determining whether a number of vectors in the password vector sequence and a number of vectors in the selected vector sequence are equal; determining whether all vectors in the password vector sequence match corresponding vectors in the selected vector sequence within a threshold angle; determining that the selected vector sequence does not match the password when it is determined that the number of vectors in the password vector sequence and the number of vectors in the selected vector sequence are not equal, or all vectors in the password vector sequence do not match corresponding vectors in the selected vector sequence within the threshold angle, or both; and determining that the selected vector sequence matches the password when it is determined that the number of vectors in the password vector sequence and the number of vectors in the selected vector sequence are equal and all vectors in the password vector sequence match corresponding vectors in the selected vector sequence within the threshold angle.

[0136]

[0145] Clause 18: The method of clause 17, wherein the threshold angle is set based on a security level.

[0137]

[0146] Clause 19: The method of any of clauses 11-16, wherein determining whether the selected vector sequence matches the password includes: generating a selected symbol sequence based on the selected vector sequence, the selected symbol sequence including one or more symbols, each symbol of the selected symbol sequence being a symbol located within a threshold angle of a position in the virtual scene indicated by a corresponding vector of the selected vector sequence; determining whether a number of symbols in the password and a number of symbols in the selected symbol sequence are equal; determining whether all symbols of the password match corresponding symbols of the selected symbol sequence; determining that the selected vector sequence does not match the password when it is determined that the number of symbols in the password and the number of symbols in the selected symbol sequence are not equal, or all symbols of the password do not match corresponding symbols of the selected symbol sequence, or both; and determining that the selected vector sequence matches the password when it is determined that the number of symbols in the password and the number of symbols in the selected symbol sequence are equal and all symbols of the password match corresponding symbols of the selected symbol sequence.

[0138]

[0147] Clause 20: The method of clause 19, wherein the threshold angle is set based on a security level.

[0139]

[0148] Clause 21: A first device comprising at least one means for carrying out the methods of any of clauses 1 to 10.

[0140]

[0149] Clause 22: A first device comprising a memory and a processor communicatively connected to the memory, the processor being configured to implement any of the methods of clauses 1 to 10.

[0141]

[0150] Clause 23: A non-transitory computer-readable medium storing code for a first device, the first device having a memory and a processor communicatively connected to the memory, and instructions stored in the memory and executable by the processor to cause the first device to perform any of the methods of clauses 1 to 10.

[0142]

[0151] Clause 24: A first device comprising at least one means for carrying out the methods of any of clauses 11 to 20.

[0143]

[0152] Clause 25: A first device comprising a memory and a processor communicatively connected to the memory, the processor being configured to implement any of the methods of clauses 11 to 20.

[0144]

[0153] Clause 26: A non-transitory computer-readable medium storing code for a first device, the first device having a memory and a processor communicatively connected to the memory, and instructions stored in the memory and executable by the processor to cause the first device to perform any of the methods of clauses 11 to 20.

[0145]

[0154] As used herein, terms such as "user equipment" (or "UE"), "user device", "user terminal", "client device", "communication device", "wireless device", "wireless communication device", "handheld device", "mobile device", "mobile terminal", "mobile station", "handset", "access terminal", "subscriber device", "subscriber terminal", "subscriber station", "terminal", and variations thereof may interchangeably refer to any suitable mobile or fixed device capable of receiving wireless communication and / or navigation signals. These terms include, but are not limited to, music players, video players, entertainment units, navigation devices, communication devices, smartphones, personal digital assistants, stationary terminals, tablet computers, computers, wearable devices, laptop computers, servers, automotive devices in automobiles, and / or other types of portable electronic devices that are typically carried by a person and / or have communication capabilities (e.g., wireless, cellular, infrared, short range radio, etc.). These terms are also intended to include a device that communicates with another device capable of receiving wireless communication and / or navigation signals, such as by a short-range wireless connection, an infrared connection, a wired connection, or other connection, regardless of whether the satellite signal reception, assistance data reception, and / or location-related processing is performed on that device or on another device. Furthermore, these terms are intended to include all devices, including wireless and wired communication devices, that can communicate with a core network via a radio access network (RAN), through which the UE can connect to external networks, such as the Internet, and to other UEs. Of course, other mechanisms for connecting to a core network and / or the Internet are also possible for a UE, such as via a wired access network, a wireless local area network (WLAN) (e.g., based on IEEE 802.11, etc.), etc.A UE may be embodied by any of several types of devices, including, but not limited to, a printed circuit (PC) card, a compact flash device, an external or internal modem, a wireless or wired phone, a smart phone, a tablet, a tracking device, an asset tag, etc. A communication link through which a UE can transmit signals to a RAN is called an uplink channel (e.g., a reverse traffic channel, a reverse control channel, an access channel, etc.). A communication link through which a RAN can transmit signals to a UE is called a downlink channel or a forward link channel (e.g., a paging channel, a control channel, a broadcast channel, a forward traffic channel, etc.). As used herein, the term traffic channel (TCH) can refer to either an uplink / reverse traffic channel or a downlink / forward traffic channel.

[0146]

[0155] The word "exemplary" is used herein to mean "serving as an example, instance, or illustration." Any detail described herein as "exemplary" should not be construed as advantageous over other examples. Likewise, the term "example" does not imply that all examples include the discussed features, advantages, or modes of operation. Furthermore, particular features and / or structures may be combined with one or more other features and / or structures. Moreover, at least a portion of the apparatus described herein may be configured to perform at least a portion of the methods described herein.

[0147]

[0156] It should be noted that the terms "connected" and "coupled," or any variation thereof, mean any direct or indirect connection or coupling between elements, unless the connection is expressly disclosed as being directly connected, and may encompass the presence of intermediate elements between two elements that are "connected" or "coupled" together through intermediary elements.

[0148]

[0157] Any reference herein to an element using a designation such as "first," "second," etc. is not intended to limit the quantity and / or order of those elements. Rather, these designations are used as a convenient method of distinguishing between two or more elements and / or instances of an element. Also, unless otherwise stated, a set of elements can comprise one or more elements.

[0149]

[0158] Those skilled in the art will appreciate that information and signals may be represented using any of a variety of different technologies and techniques. For example, the data, instructions, commands, information, signals, bits, symbols, and chips that may be referred to throughout the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination thereof.

[0150]

[0159] Nothing described or illustrated in this application is intended to publicly disclose any element, act, feature, benefit, advantage, or equivalent, whether or not that element, act, feature, benefit, advantage, or equivalent is recited in a claim.

[0151]

[0160] In the above detailed description, it can be seen that in each example, various features are grouped together. This method of disclosure should not be understood as the claimed examples having more features than are expressly recited in each claim. Rather, the disclosure may include fewer features than all of the individual examples disclosed. Thus, the following claims are hereby considered incorporated into this description, and each claim can stand alone as a separate example. Although each claim can stand alone as a separate example, it should be noted that a dependent claim can refer to a specific combination with one or more claims within the scope of the claim, while other examples can include or include a combination of the dependent claim with the subject matter of any other dependent claim, or a combination of any feature with other dependent claims and independent claims. Such combinations are suggested herein unless it is expressly stated that a specific combination is not intended. It is further intended that a feature of a claim can be included in any other independent claim, even if the claim is not directly dependent on the independent claim.

[0152]

[0161] It is further noted that the methods, systems and apparatus disclosed in the present description or claims may be implemented by a device comprising means for performing the respective acts and / or functions of the disclosed methods.

[0153]

[0162] Further, in some examples, an individual act may be subdivided into or include one or more sub-acts, and such sub-acts may be included in and be part of the disclosure of the individual act.

[0154]

[0163] Although the above disclosure illustrates exemplary examples of the present disclosure, it should be noted that various modifications and changes can be made herein without departing from the scope of the present disclosure as defined by the appended claims. The functions and / or acts of the method claims according to the examples of the present disclosure described herein need not be performed in any particular order. In addition, well-known elements may not be described in detail or may be omitted so as not to obscure the relevant details of the aspects and examples disclosed herein. Furthermore, although elements of the present disclosure may be described or claimed in the singular, the plural is contemplated unless limitation to the singular is explicitly stated.

Claims

1. Memory and A communication device and a processor communicatively connected to the memory and the communicator; 1. A first device comprising: utilizing an inertial measurement unit (IMU) to determine a first rotation vector (RV) of a first camera of the first device; receiving one or more RVs from the one or more devices, including a second RV from the second device, the second RV being an RV of a second camera of the second device; determining whether the second RV is aligned with the first RV; auto-pairing with the second device when the second RV is aligned with the first RV; The first device is configured to:

2. In determining whether the second RV is aligned with the first RV, the processor: The second RV is determined to be aligned with the first RV when the first RV and the second RV have the same orientation, and the first RV and the second RV are if the orientation of the first RV is opposite to the orientation of the second RV within a threshold angle; or if the orientation of the first RV is the same as the orientation of the second RV within the threshold angle, The first device of claim 1 having an equivalent orientation.

3. In determining whether the second RV is aligned with the first RV, the processor:

3. The first device of claim 2, further configured to determine that the second RV is aligned with the first RV when the orientations of the first RV and the second RV remain equivalent for a threshold time.

4. In determining whether the second RV is aligned with the first RV, the processor: the first RV and the second RV, if the orientation of the first RV is opposite to the orientation of the second RV within a threshold angle; or if the orientation of the first RV is the same as the orientation of the second RV within the threshold angle, determining whether the first RV and the second RV, which have equivalent orientations, have equivalent orientations; determining whether an object associated with the second device is detected within a first camera view, the first camera view; It is further structured as follows:

2. The first device of claim 1, wherein the second RV is determined to be aligned with the first RV when the first and second RVS have equivalent orientations and the object associated with the second device is detected within the first camera view.

5. The first device of claim 4 , wherein the object associated with the second device is any one or more of a face, a wearable unit, and a mobile device.

6. The first device of claim 5 , wherein the wearable unit is a smart glass.

7. the processor: The first device of claim 1 , further configured to broadcast the first RV.

8. the processor:

10. The first device of claim 1, further configured to share a first shared view with the second device following automatic pairing with the second device, the first shared view being a first camera view or a first rendered view, the first camera view being a view of the first camera, and the first rendered view being a view after rendering the first camera view.

9. 9. The first device of claim 8, wherein the first rendered view is an augmented reality (AR) view of the first camera view, an extended reality (XR) view of the first camera view, or both.

10. the processor:

9. The first device of claim 8, further configured to display a second shared view received from the second device following automatic pairing with the second device, the second shared view being a second camera view or a second rendered view, the second camera view being a view of the second camera, and the second rendered view being a view after rendering the second camera view.

11. 1. A method of a first device, comprising: Determining a first rotation vector (RV) of a first camera of the first device utilizing an inertial measurement unit (IMU); receiving one or more RVs from one or more devices, including a second RV from a second device, the second RV being an RV of a second camera of the second device; determining whether the second RV is aligned with the first RV; automatically pairing with the second device when the second RV is aligned with the first RV; A method comprising:

12. Determining whether the second RV is aligned with the first RV includes: determining that the second RV is aligned with the first RV when the first RV and the second RV have the same orientation, if the orientation of the first RV is opposite to the orientation of the second RV within a threshold angle; or if the orientation of the first RV is the same as the orientation of the second RV within the threshold angle, The method of claim 11 having equivalent orientations.

13. Determining whether the second RV is aligned with the first RV includes: the first RV and the second RV, if the orientation of the first RV is opposite to the orientation of the second RV within a threshold angle; or if the orientation of the first RV is the same as the orientation of the second RV within the threshold angle, determining whether the first RV and the second RV, which have equivalent orientations, have equivalent orientations; determining whether an object associated with the second device is detected within a first camera view, the first camera view; Including, 12. The method of claim 11, wherein the second RV is determined to be aligned with the first RV when the first and second RVS have equivalent orientations and the object associated with the second device is detected within the first camera view.

14. 12. The method of claim 11, further comprising, following automatic pairing with the second device, sharing a first shared view with the second device, wherein the first shared view is a first camera view or a first rendered view, the first camera view is a view of the first camera, and the first rendered view is a view after rendering the first camera view.

15. 15. The method of claim 14, wherein the first rendered view is an augmented reality (AR) view of the first camera view, an extended reality (XR) view of the first camera view, or both.

Images