Method and system for navigation in an immersive environment

Abstract

A method and system for controlling spatial state (e.g. locomotion) in an immersive environment (e.g. virtual reality) using a hardware controller 100 having an inertia sensor and an input feature (e.

Description

TECHNICAL FIELD The present disclosure relates to methods and systems for an immersive environment, and, in particular, for controlling locomotion in an immersive environment. BACKGROUND In recent decades, immersive technologies have become prevalent in both consumer technology markets and commercial settings. Immersive technologies combine hardware and software elements to place users in immersive environments. In virtual reality systems, a user is placed in a fully simulated three-dimensional digital environment. In mixed-reality systems a computer-generated virtual world is super-imposed on a real world, with both virtual and physical objects co-existing in the mixed reality environment, and virtual objects responding to changes in the physical world. In augmented reality systems, virtual objects are overlayed on the real-world. In augmented reality systems there is less interaction between the virtual and the real worlds than mixed reality systems. While immersive technology has been utilized across various fields for many years, recent advancements have significantly broadened its industrial application. For example, immersive systems are now being used to support collaborative computer-aided design and manufacturing, advanced simulation and testing, production line management, factory planning, and many other similar applications. The design specification for immersive systems in these applications differ from those in other areas. For example, industrial applications may demand higher performance, greater accuracy, and a deeper level of integration with native software applications than traditional use cases such as virtual reality gaming. The methods and technologies from traditional areas may not be appropriate or as well suited to industrial applications. This has provoked designers and manufacturers of immersive systems to reconceptualize and rethink immersive technology, resulting in new and innovative approaches to immersive hardware design. As immersive system design evolves, certain objectives and expectations nevertheless persist. This includes designing systems which minimize motion sickness and provide a comfortable and engaging user experience. In order to fulfil these objectives, one aspect for system designers to consider is the method of navigation within the immersive environment. Navigation is controlled through a navigation system - a software component which translates user input from the physical environment into movement in the immersive environment. An intuitive navigation system helps a user move through an immersive environment in an effortless and natural manner. In contrast, an unintuitive system may impact the user experience and prevent the user from moving through the environment in a desired manner. In the worst case, a poorly designed navigation system may render the immersive system unusable. Some immersive systems include hardware controllers with joysticks or D-pads. In these systems, navigation is achieved naturally by converting a directional input from the physical hardware controller to a corresponding direction in the immersive environment. In other systems the hardware controllers may lack traditional joystick or D-pad inputs, or similar. There is therefore a need for providing well-designed intuitive navigation methods and systems which are suited to different types of immersive hardware controller. SUMMARY It is an object of the invention to provide a method and system for controlling locomotion in an immersive environment. The foregoing and other objects are achieved by the features of the independent claims. Further implementation forms are apparent from the dependent claims, the description and the figures. According to a first aspect of the invention, a computer-implemented method for controlling locomotion in an immersive environment using a hardware controller is provided. The hardware controller comprises an input feature operable to transition between an inactive condition and a selected one of a first active condition and a second active condition, in response to an associated user interaction with the input feature. The selected active condition is maintained for as long as the associated user interaction is continued. The hardware controller comprises an inertial sensor for measuring inertial data of the hardware controller in a physical environment. The method comprises: receiving inertial data, I, from the hardware controller, detecting an active condition of the input feature at an initial time, t0, and, while the active condition is maintained: determining a change in the inertial data, A / , from the initial time, t0, to a current time, t, applying a mapping that maps the change in inertial data, A / , to a change in a spatial state in the immersive environment, based on the detected active condition, and updating the spatial state in the immersive environment in real-time based on an output of the mapping. The method according to the first aspect provides a natural and intuitive system for controlling locomotion in an immersive environment using data received from a hardware controller with an inertial sensor, and an input feature with two detectable active conditions. Preferably, the method comprises: displaying a user-interface element in the immersive environment, wherein the user-interface element comprises a real-time visual representation of a direction of movement in the immersive environment, based on the change in the spatial state. Preferably, the spatial state comprises a position and an orientation in the immersive environment. Preferably, the input feature comprises a button with an integrated touch-sensitive surface. Preferably, the first active condition is engaged in response to a user touching the touch sensitive surface. Preferably, the second active condition is engaged in response to a user actuating the button. Preferably, the inertial data comprises rotational data with respect to a fixed axis of orientation associated with the hardware controller. Preferably, the mapping comprises a mapping of the rotational data to a direction of movement in a horizontal plane in the immersive environment. Preferably, the mapping comprises a mapping of the rotational data to vertical or rotational direction of movement in the immersive environment. Preferably, the method comprises: displaying a user-interface element in the immersive environment, wherein the user-interface element comprises a real-time visual representation of a speed of movement in the immersive environment, based on the change in the spatial state. These and other aspects of the invention will be apparent from the embodiment(s) described below. BRIEF DESCRIPTION OF THE DRAWINGS For a more complete understanding of the present disclosure, and the advantages thereof, reference is now made to the following descriptions taken in conjunction with the accompanying drawings, in which: Figures IA and IB show a schematic diagram of a hardware controller for an immersive system, according to an example. Figure 2 shows a flow diagram of a method for controlling locomotion in an immersive environment, according to an example. Figure 3 is a perspective view of an immersive environment, according to an example; Figure 4 shows views of a user interface element for an immersive system, according to an example. Figure 5 shows views of a user interface element for an immersive system, according to an example. Figure 6, shows views of a user interface element for an immersive system, according to an example. Figure 7 illustrates an example of a data processing system in which embodiments of the present disclosure may be implemented. DETAILED DESCRIPTION Example embodiments are described below in sufficient detail to enable those of ordinary skill in the art to embody and implement the systems and processes herein described. It is important to understand that embodiments can be provided in many alternate forms and should not be construed as limited to the examples set forth herein. Accordingly, while embodiments can be modified in various ways and take on various alternative forms, specific embodiments thereof are shown in the drawings and described in detail below as examples. There is no intent to limit to the particular forms disclosed. On the contrary, all modifications, equivalents, and alternatives falling within the scope of the appended claims should be included. Elements of the example embodiments are consistently denoted by the same reference numerals throughout the drawings and detailed description where appropriate. The terminology used herein to describe embodiments is not intended to limit the scope. The articles “a,” “an,” and “the” are singular in that they have a single referent, however the use of the singular form in the present document should not preclude the presence of more than one referent In other words, elements referred to in the singular can number one or more, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises,” “comprising,” “includes,” and / or “including,” when used herein, specify the presence of stated features, items, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, items, steps, operations, elements, components, and / or groups thereof. Unless otherwise defined, all terms including technical and scientific terms used herein are to be interpreted as is customary in the art. It will be further understood that terms in common usage should also be interpreted as is customary in the relevant art and not in an idealized or overly formal sense unless expressly so defined herein. The methods and systems described herein may be used to control locomotion in an immersive environment. Herein an immersive environment may refer to a computergenerated virtual reality environment, a mixed-reality environment, an augmented reality environment, or similar. An immersive environment may be provided through an immersive system. An immersive system may comprise a headset with a display that uses stereoscopic technology to give a sense of depth in an immersive environment, and sensors, such as accelerometers and gyroscopes, which track the user’s head movements in real time. Many immersive systems also include hardware controllers which allow a user to provide user input and perform functions such as navigation and manipulation of objects in the immersive environment Hardware controllers may be handheld or wearable devices and may provide various features which enable a user to provide user input including, but not limited to: buttons, control pads, touch sensitive surfaces and the like. In addition, hardware controllers include sensors of various kinds including, but not, limited to: IR sensors, motion detection sensors, inertial sensors and the like. Figure IA is a schematic diagram of a hardware controller 100 for an immersive system, according to an example. The hardware controller 100 may be used with the method and systems described herein. The hardware controller 100 is a wearable device having a ring-like configuration, which a user may place around a finger, such as the index finger, on either hand. Advantageously, the ring-like configuration of the hardware controller 100 allow a user to continue wearing the controller 100 while performing other tasks, such as typing on a keyboard. This is particularly useful in applications where a user switches frequently between immersive and non-immersive environments, as is the case in many industrial applications. The hardware controller 100 comprises a first input feature I 10, and second input feature 120. In the example shown in Figure I A, the first input feature I 10, is a hybrid touch-sensitive pad and physical button and the second input feature 120 is a touch-sensitive pad. Figure I B is a diagram showing a user interaction with the hardware controller 100. In Figure I B a user is interacting with the first input feature I 10 by touching the touch sensitive surface with a thumb 130. The user may also interact by applying pressure with the thumb 130 to actuate the first input feature 110. When the user’s thumb contacts the surface of the first input feature I 10, a first active condition is detected by the hardware controller 100. When the user’s thumb 130 applies pressure to the button, a second active condition is detected by the hardware controller 100. The detected active condition is maintained for as long as the user interaction continues. In other words, if a user continues to maintain contact with the touch-sensitive surface of the first input feature I 10, then the controller will continue to detect the first active condition. If the user continues to hold the button down, then the controller will continue to detect the second active condition. If the user either stops interacting with the input feature or changes their interaction, then the detection of the corresponding active condition will cease and change to either the other active condition or an inactive condition. The hardware controller 100 further comprises an inertial sensor. The inertial sensor is arranged to measure inertial data of the hardware controller 100, in the physical environment. The inertial data may include rotational data of the hardware controller with respect to a fixed axis of orientation. In other examples the inertial data may also include additional data such as linear acceleration data or any other kinds of motion data which may be obtained by an inertial sensor. The hardware controller 100 may further comprise a communication module for communicating data wirelessly from the controller to a second device, such as an immersive headset or desktop computer. In one example the communication module may be a Bluetooth communication module. Bluetooth is an advantageous form of communication for lightweight devices such as the hardware controller 100, as it consumes less power than other forms of wireless communication such as Wi-Fi, while having better range than methods such as Near-Field Communication. The communication module may be used to communicate data such as data indicating when an active condition of either input feature is detected, and inertial data measured by the inertial sensor. The data may be communicated to a device which is communicatively coupled to the hardware controller for further processing, such as an immersive headset. The hardware controller 100 is one example of a hardware controller for an immersive system, which may be used with the methods and systems described herein. The skilled person would understand that hardware controllers which differ from the hardware controller 100 in various aspects also fall within the scope of the present invention. For example, controllers which differ from hardware controller 100 in terms of the arrangement of input features, a number of input features, the type of user input feature, the form and configuration of the controller and the method for affixing the controller to the user, may also fall within the scope of the present invention. Figure 2 is a flow diagram of a method 200, for controlling locomotion in an immersive environment. The method 200 may be implemented in conjunction with a hardware controller, such as the hardware controller 100, that include an inertial sensor and an input feature operable to transition between an inactive condition and a first active condition or a second active condition, in response to an associated user interaction with the input feature. The method 200 may be implemented by a data processor in a headset of an immersive system. The method 200 updates a spatial state in an immersive environment, based on data received from the hardware controller. In particular, as the position of the hardware controller varies in physical space, the spatial state of the user in the immersive environment varies. Herein, a spatial state refers to at least one of a position and orientation of a user in the immersive environment. At step 210, inertial data is received from the hardware controller. In one example, the inertial data comprises rotational data with respect to a fixed axis of orientation associated with the hardware controller. The data may be received over a wireless communication channel, such as Bluetooth communication channel, between the hardware controller and a target device, such as an immersive headset. The inertial data may be communicated continuously in real-time from the hardware controller. At step 220, an active condition of the input feature of the hardware controller is detected at an initial time, t0. In one implementation, in response to the hardware controller detecting either the first or second active condition, a signal may be communicated from the hardware controller to signal that an active condition is being detected. For example, if the hardware controller comprises a button, and an active condition corresponds to the button being pressed, then the hardware controller may communicate a signal to indicate that the button is being pressed. This signal may be communicated until the user interaction is ceased, for example, by a user releasing the pressed button. At step 230, a change in the inertial data, △ / , from the initial time, t0, to a current time, t is determined. When the method 200 is implemented in conjunction with the hardware controller 100, a change in the in inertial data from the initial time, t0, to a current time, t, comprises a change in rotation, which may be determined from rotational data received from the hardware controller 100. At step 240, a mapping is applied to the change in inertial data obtained at step 230, based on the detected active condition. The mapping maps the change in inertial data to a change in spatial state in the immersive environment. In examples of the method 200, the range of the mapping is continuous. This helps to ensure smooth movement within the immersive environment When the method 200 is implemented in conjunction with the hardware controller 100, in response to detecting the first active condition, the change in rotational data may be mapped to a change in direction in a horizontal plane in the immersive environment. A directional vector in a horizontal plane may be obtained by applying an axis-angle rotation of a vector in a vertical axis, using a pitch and roll determined from the rotational data. This results in a vector v = (X, Y) in a square [—1,1] X [—1,1]. A transformation from cartesian to polar co-ordinates may be applied to v to map it to a circle. Mapping vectors to a circle in this manner ensures that the movement in the immersive environment is at a consistent speed. In response to detecting the second active condition from the hardware controller 100, the change in rotational data may be mapped to a change in vertical displacement or orientation in the immersive environment, using a similar method. In examples of the method 200, the range of the mapping is continuous. At step 250, the spatial state in the immersive environment is updated in real-time based on an output of the mapping. For example, a movement direction obtained at step 240 may be used to change the position of the user in the immersive environment. As the output from step 240 varies, the direction of movement of the user in the immersive environment changes accordingly. The mapping is such that the user’s variation in hand position translates naturally, smoothly and intuitively into movement in the immersive environment. At step 260, the method checks whether the detected active condition is maintained. If the detected active condition is maintained, the method 200 returns to step 230 and steps 230 to 260 are repeated. The method 200 is repeated continuously while the detective active condition is maintained, so that the spatial state is updated continuously. When the detected active condition ceases, the method 200 ends. In a further embodiment, the method 200 may also be implemented in conjunction with a hardware controller which comprises two input features, where each input feature has an associated active condition. The method 200 may be applied to inertial data received from the controller, when the active condition of either one of the input features is detected. For example, in the hardware controller 100, rather than the two active conditions being associated with two different types of user interaction with the first input feature 110, a first active condition may be associated with pressing or touching the first input feature I 10 and a second active condition may be associated with touching the second input feature 120. According to an example, a user-interface element may be displayed in the immersive environment. In one example, the user interface element comprises a real-time visual representation of a direction of motion in the immersive environment. In some cases the user interface element comprises a real-time visual representation of a user’s speed in the immersive environment. In one embodiment the user interface element may appear in the user’s field of view in the immersive environment when an active condition is detected and disappear from the field of view, when an inactive condition is detected. In another embodiment, the user interface element may be displayed in the user’s field of view, irrespective of whether an active condition is detected. Figure 3 shows a field of view of a user in an immersive environment 300. In Figure 3, a user interface element 3 10 is displayed in the centre of the user’s field of view. In other embodiments, the user interface element 310 may be positioned elsewhere in the user’s field of view, such as the bottom left or bottom right side of the field of view. In some examples, the user interface element 310 may move, for example, depending on a user’s head position. The user interface element 310 comprises a dynamic ball 320 and directional arrows 330. In other embodiments, the directional arrows 330 may be absent from the user interface element 310. In use, the user interface element 3 10 dynamically changes, depending on the motion of the user through the immersive environment 300. The user interface element 310 provides the user with real-time visual feedback and assists the user with controlling motion in the immersive environment 300. Figure 4 shows an example of the user interface element 310 in use, with the hardware controller 100. In the example shown in Figure 4, the direction of motion represented by the user interface element is derived from the mapping of inertial data from the hardware controller 100, when a first active condition of the hardware controller is detected. In the hardware controller 100 detection of the first active condition corresponds to a user placing their thumb on the touch sensitive surface of the first input feature 110. As the user’s hand position changes, the direction indicated by the user interface element 310 changes. The views illustrated in Figure 4 correspond to directions of motion in a flat plane in the immersive environment. In use, the dynamic ball 320 moves in the direction of motion and an animated arrow 410 is displayed with the user interface element 310. Additionally, the directional arrows 330 may change tone depending on the direction of the dynamic ball 320, so that the directional arrows 330 in the direction of the dynamic ball are a lighter tone, and the directional arrows 330 away from the dynamic ball 320 are a darker tone, or vice-versa. The first view 420 shows the user interface element 310 when the user is moving forward in the immersive environment and corresponds to a user tilting the hardware controller 100 in a forward direction, while their thumb is engaged with the touch-sensitive surface of the first input feature I 10. Similarly, the second view 430 shows the element 3 10 when moving to the right, corresponding to the user tilting the hardware controller 100 to the right The third view 440 shows the element 310 when moving backward, corresponding to the user tilting the hardware controller 100 in a backward direction. The fourth view 450 shows the element 310 when moving to the left, corresponding to the user tilting the hardware controller 100 to the left. Similarly the fifth view 460 shows the element 310 when movement is in a forward and rightward direction, the sixth view 470 shows the element 310 when movement is in a backward rightward direction, the seventh view 480 shows the element 3 10 when movement is in a backward leftward direction and the eighth view 490 shows the element 310 when movement is in a forward, leftward direction. Figure 5 shows views of the user interface element 3 10 in which the direction of motion is derived from the mapping of inertial data from the hardware controller 100, when the second active condition is detected. In the hardware controller 100 detection of the second active condition corresponds to the user actuating the first input feature 110. The first view 510 shows the user interface element 3 10 when the user is moving upward in a vertical axis in the immersive environment, corresponding to the user tilting the hardware controller 100 in an upward direction, while pressing the first input feature I 10. Similarly, the second view 520 shows the element 310 when in a level position, the third view 530 shows the element 310 when moving downward in the vertical axis. The fourth view 540 shows the element 310 during a rotational movement about the vertical axis in an anticlockwise direction, corresponding to the user tilting the hardware controller in a leftward direction and holding down the first input feature I 10. The fifth view 550 shows the element 310 during a rotational movement about the vertical axis in an clockwise direction, corresponding to the user tilting the hardware controller in a rightward direction and holding down the first input feature I 10. Figure 6 shows an example in which the user interface element 310 provides a real-time visual representation of the speed in the immersive environment. In a first view 610, the dynamic ball 320 is in a first position 620, close to the centre 630 of the user interface element 310, indicating a low speed. In the second view 640, the dynamic ball 320 is at a second position 650, further away from the centre 630, indicating a higher speed. When the user changes their hand position to tilt the hardware controller 100 to a greater or lesser extent in a given direction, the movement speed in the corresponding direction in the immersive environment increases or decreases and the dynamic ball 320 moves either further from, or closer to the centre 630. The user interface element 3 10 represents one implementation of a user interface element which falls within the scope of the present disclosure. The skilled person would understand that user interface elements which differ in various aspects from the user interface element 310 also fall within the scope of the present disclosure including, but not limited to, aspects such as the response of the user interface element to data received from the hardware controller, the visual aspects of the user interface element, and the dynamic aspects of the user interface element. The present disclosure is described with reference to flow charts and / or block diagrams of the method, devices and systems according to examples of the present disclosure. Although the flow diagrams described above show a specific order of execution, the order of execution may differ from that which is depicted. In some examples, some blocks of the flow diagram may not be necessary and / or additional blocks may be added. It shall be understood that each flow and / or block in the flow charts and / or block diagrams, as well as combinations of the flows and / or diagrams in the flow charts and / or block diagrams can be realised by machine readable instructions. The machine-readable instructions may, for example, be executed by a general-purpose computer, a special purpose computer, an embedded processor or processors of other programmable data processing devices to realize the functions described in the description and diagrams. In particular, a processor or processing apparatus may execute the machine-readable instructions. Thus, modules of apparatus may be implemented by a processor executing machine-readable instructions stored in a memory, ora processor operating in accordance with instructions embedded in logic circuitry. The term 'processor' is to be interpreted broadly to include a CPU, processing unit, ASIC, logic unit, or programmable gate set etc. The methods and modules may all be performed by a single processor or divided amongst several processors. Such machine-readable instructions may also be stored in a computer readable storage that can guide the computer or other programmable data processing devices to operate in a specific mode. Figure 7 shows an example 700 of a processor 710 associated with a memory 720. The memory 720 comprises computer readable instructions 730 which are executable by the processor 710. The instructions 730 cause the processor to receive inertial data, I, from a hardware controller such as hardware controller 100, detect an active condition of the input feature at an initial time, t0, and, while the active condition is maintained: determine a change in the inertial data, △ / , from the initial time, t0, to a current time, t; apply a mapping that maps the change in inertial data, △ / , to a change in a spatial state in the immersive environment, based on the detected active condition; and update the spatial state in an immersive environment in real-time based on an output of the mapping. The present inventions can be embodied in other specific apparatus and / or methods. The described embodiments are to be considered in all respects as illustrative and not restrictive. In particular, the scope of the invention is indicated by the appended claims rather than by the description and figures herein. All changes that come within the meaning and range of equivalency of the claims are to be embraced within their scope.

Claims

I. A computer-implemented method for controlling locomotion in an immersive environment using a hardware controller, wherein the hardware controller comprises:an input feature operable to transition between an inactive condition and a selected one of a first active condition and a second active condition, in response to an associated user interaction with the input feature, wherein the selected active condition is maintained for as long as the associated user interaction is continued; andan inertial sensor for measuring inertial data of the hardware controller in a physical environment;the method comprising:receiving inertial data, I, from the hardware controller;detecting an active condition of the input feature at an initial time, t0; and,while the detected active condition is maintained:determining a change in the inertial data, △ / , from the initial time, t0, to a current time, t;applying a mapping that maps the change in inertial data, △ / , to a change in a spatial state in the immersive environment, based on the detected active condition; andupdating the spatial state in the immersive environment in realtime based on an output of the mapping.

2. The method of claim I, comprising:displaying a user-interface element in the immersive environment, wherein the user-interface element comprises a real-time visual representation of a direction of movement in the immersive environment, based on the change in the spatial state.

3. The method of claim I, wherein the spatial state comprises a position and an orientation in the immersive environment.

4. The method of claim I, wherein the input feature comprises a button with an integrated touch-sensitive surface.

5. The method of claim 4, wherein the first active condition is engaged in response to a user touching the touch sensitive surface.

6. The method of claim 4, wherein the second active condition is engaged in response to a user actuating the button.

7. The method of claim I, wherein the inertial data comprises rotational data with respect to a fixed axis of orientation associated with the hardware controller.

8. The method of claim 7 wherein, the mapping comprises a mapping of the rotational data to a direction of movement in a horizontal plane in the immersive environment.

9. The method of claim 7 wherein, the mapping comprises a mapping of the rotational data to vertical or rotational direction of movement in the immersive environment.

10. The method of claim I, wherein a range of values obtained from the mapping is continuous.II. A computer-implemented method for controlling locomotion in an immersive environment using a hardware controller, wherein the hardware controller comprises:a first input feature and a second input feature, wherein each of the first and second input features are operable to transition between an inactive condition and anassociated active condition, in response to a user interaction with a selected one of the first and second input features, wherein the associated active condition is maintained for as long as the user interaction is continued; andan inertial sensor for measuring inertial data of the hardware controller in a physical environment;the method comprising:receiving inertial data, I, from the hardware controller;detecting an active condition of the first input feature or the second input feature at an initial time, t0; and,while the detected active condition is maintained:determining a change in the inertial data, A / , from the initial time, to, to a current time, t;applying a mapping that maps the change in inertial data, A / , to a change in a spatial state in the immersive environment, based on the detected active condition; andupdating the spatial state in the immersive environment in realtime based on an output of the mapping.

12. A computer-readable medium comprising instructions that, when executed by a computer, cause the computer to carry out the method according to any one of claims I to I I.

13. An immersive system, comprising:an immersive headset comprising a display to display an immersive environment to a user;a hardware controller, communicatively coupled with the immersive headset, wherein the hardware controller comprises:an input feature operable to transition between an inactive condition and a selected one of a first active condition and a second active condition, in response to an associated user interaction with the input feature, wherein the selected active condition is maintained for as long as the associated user interaction is continued; andan inertial sensor for measuring inertial data of the hardware controller in a physical environment;a data processor; anda memory to store instructions that, when executed by the processor, cause the processor to execute the method according to any one of claims I to 10.

14. The system of claim 13, wherein the hardware controller is a ring-controller.

15. The system of claim 13, wherein the immersive headset and the hardware controller communicate via a Bluetooth communication channel.

16. The method of claim I, comprising displaying a user-interface element in the immersive environment, wherein the user-interface element comprises a real-time visual representation of a speed of movement in the immersive environment, based on the change in the spatial state.20

Images