A general-purpose spatial user interface

Abstract

Virtual-reality systems are challenged by safety, body-tracking and weight issues in their current incarnations. The present invention solves all these hurdles with a lightweight mechanical interface operated in the user's supine position. All four limbs move in space with an extended range of motion and yet the operator being so decoupled remains clear from any objects nearby. The four-piece keyboard concept interprets a plurality of desired actions in virtual space and the weight of various hardware components is distributed off the headgear onto the resting surface. This straightforward solution affords a level of immersive experience never before reached and can be expected to disrupt how we work, how we play how long we live and most-importantly, how we steward the health of our planet. This experience is delivered in a compact form factor ideal for applications where space and weight are highly constrained.

Description

A GENERAL-PURPOSE SPATIAL USER INTERFACEField of the Invention

[0001] There is a gradual migration towards spatial computing that is being driven by new display technologies, however the specialized application of virtual reality (VR) has so far been dominated by gaming. Thus, the default input systems are simply carry-overs from gaming controllers, or updated eye-hand tracking. However, such controllers struggle to offer heightened levels of intuitive immersion. As a result, interactive experience anchors virtual locomotion to the use of directional buttons and to the brain disbelief is hardly ever suspended.

[0002] To bridge the gulf towards an integrated immersive experience the translation of the familiar one-to-one mouse-action (graphical) interface has proven to be gated so far by physics. What might demonstrate the ‘teleportation’ proof-of-concept is a half-step like the role that QWERTY-keyboards once played. That is, writing with a quill pen is reduced to pressing inked stamp. The benefits simply in speed terms may be too good to not try going for. The modern keyboard succeeded in becoming the standard ironically by throttling the potential speed gains to avoid jamming mechanical typewriter keys.

[0003] Prior art has shown that it is not obvious how a keyboard analogue can even be extended to being useful in a third dimension. There seems to be little interest in developing a clearly inferior solution with low action-fidelity, to boot. Neither does an analogous mechanical “typewriter” exist to train users in its unintuitive use. That does not mean that a spatial input system a la the keyboard cannot work nor cannot get us over the perceived ‘immersion’ barrier just enough to enjoy believable VR experience.

[0004] The overwhelming barrier to feeling immersed boils down to the basic issue of user safety. If “fail” memes of awkwardly walking into walls or inadvertently destroying displays (with controllers being hurled at them) have taught us anything it is that serious injury and lawsuits await purveyors of VR equipment. Even after investing several tens of billions in research-&-development dollars, major tech players still warn of serious physical limitations for the safe deployment of their VR products, to the point that is appears impossible to dissociate VR with danger.

[0005] Another effective constraint faced in delivering a viable VR system to market is its appearance. Solutions that are bulky or uncool will face ‘fashion’ detractors. Currently, there are omni-directional treadmills, slide-mills with special footgear and body-tracking solutions on the market. Their users typically feel gravity and / or the tug of restraining body -worn gear. It can look exhausting watching users perform unnatural motions almost in mid-air. In addition to the overall weight of the systems there is the room real-estate eaten up by them, that factor into their perception. The weight of display hardware, computer-processing power and their accompanying unwieldy power-supply packs are presently severe limiting factors capping usetime and adoption.

[0006] In essence the first to market with a safe VR system might be crowned quite possibly with legitimate market dominance. Surprisingly, said system might end up being the lowest-cost solution emerging out of any the variants in development. The solution proposed in the present invention borrows from the basic four-limbed chassis concept (WO2023140881 Al) introduced by the author and extended by his virtual-reality operating-system (USPTO Application #63521063) concept. It includes a learned input system that accommodates a full range of virtual mobility as well as built-in or inherent safety, with a fast and limitless functionality reduced into a compact, ergonomic package.

[0007] The present invention follows the two other applications in the most general way, so that the first two are merely more special cases of this broad third application. Each aforementioned invention builds on the same mechanical body / design plan for successively less specialized markets. This generalization is necessary because voice-control is not the ultimate nor optimal input method. The best input is the fastest and this present general-purpose solution might always be the next best solution to thought-driven (BCHI) input. Even though that is on many orders of difficulty beyond what exists today it might only meagerly improve on the latency of the present invention by milliseconds at best.

[0008] Furthermore, the general -case solution is superior because it invites the user to spend more time using safe muscle-resistance exercise. The explicit ancillary benefits of the present invention include stronger engagement with real-life interactions, closer family relations, greater physical resilience and, most importantly, increased appreciation for the natural environment in general. In many respects, the opposite of what might be expected from marathon periods of gaming or drone-piloting (the implied direction of a “Wall-E” movie future)now becomes possible. Ideally, longer life expectancy and better quality of life can easily be expected.Brief Description of the Drawings

[0009] The invention will now be described, by way of example, with reference to the accompanying drawings, in which:

[0010] FIG. 1 illustrates from a top view an exemplary first-order framework for supporting the second-order mobility-accommodating elements, in accordance with an embodiment of the present invention;

[0011] FIG. 2 shows a prospective user employing the operational construct in FIG. 1,

[0012] FIG. 3 shows the 24 third-order input elements for one hand (right) which would be mirrored for the other hand. There is a key-less neutral thumb position on the stem which when applied with the palm in a comfortable gripping loop serves to manipulate the arm-drive cylinder(s).

[0013] FIG. 4 shows the foot positions that an operator would assume to activate 8 additional third-order input contacts / sensors for one foot which would be mirrored for the other foot. In the neutral level-center position the operator can manipulate the leg-drive cylinder(s).

[0014] FIG. 5 shows the optimal headgear form factor using visor in a costume from a popular music video as illustration.Detailed Description of the Invention

[0015] The following detailed description is merely exemplary in nature and is not intended to limit the described embodiments or the application and uses of the described embodiments. As used herein, the word “exemplary” or “illustrative” means “serving as an example, instance, or illustration.” Any implementation described herein as “exemplary” or “illustrative” is not necessarily to be construed as preferred or advantageous over otherimplementations. All of the implementations described below are exemplary implementations provided to enable persons skilled in the art to make or use the embodiments of the disclosure and are not intended to limit the scope of the disclosure, which is defined by the claims. For purposes of description herein, the terms “first,” “second,” “left,” “rear,” “right,” “front,” “vertical,” “horizontal,” and derivatives thereof shall relate to the invention as oriented in FIG. 1. Furthermore, there is no intention to be bound by any expressed or implied theory presented in the preceding technical field, background, brief summary or the following detailed description. It is also to be understood that the specific devices and processes illustrated in the attached drawings, and described in the following specification, are simply exemplary embodiments of the inventive concepts defined in the appended claims. Hence, specific dimensions and other physical characteristics relating to the embodiments disclosed herein are not to be considered as limiting, unless the claims expressly state otherwise.

[0016] In one embodiment of the present invention presented in FIG. 1 , a device 100 for navigating virtual reality provides a bed-based operator 118. The device includes connected, airpiston components 112 and supported by a central axial-chassis 102 (resembling a bilaterally- symmetric vertebrate-body plan). There are separate cylinder / pistons for each side of the body with separate electronically-controlled extraction (199) and retraction (198) valves to isolate the muscle conditioning for a specific limb. The upper- and lower-body chassis segments are separately adjustable in length to be custom-fitted for the patient’s height and may be linked by a double hinge (not shown) or elasti-bound slotting design that permits the device to be folded over on itself or to twist (like a blind person’s cane). There are gyroscopic sensors fixed on the chassis to allow the system to recognize the user’s desired spatial orientation. There is an optional assembly 114-116 that allows the chassis / spine to be elevated on the bed surface. The foot gear is replaceable by unlatching its strap. The device is designed to balance opposing forces between the upper and lower body. No mounting to a fixed location is required. The overall design simulates climbing up / down a ladder, in addition to walking, swimming or skiing. The choice of a flexible chassis material accommodates natural flexibility in operator movement. When not in use the device is designed to fold in half or just leaned up against the side of the bed or the wall until it is needed again.

[0017] This invention is a generalization of the earlier similar inventions for elderly fitness and explor-tainment by virtue of productivity features which together can easily serve as a ‘60% keyboard.’ With the operator safety-centric and ergonomic design users can exercise, play and now work while immersed digitally for extended periods of time without interruption. The first-order locomotion-navigation aspect of the device (arm- and leg-drive cylinders) ensures that users can move around freely in and out of every comer of a ‘metasphere’ or digital landscape. When the need arises to take a phone call or resume work on a project, the second-order features are available literally at one’s fingertips to generate computer-screen image(s), to contact voice assistants and / or to produce content.

[0018] One application -flow example is literal telecommuting. Rather than get in one’s car or take public transportation and travel / relocate to a common physical office, one would access the digital equivalent of one’s employer’s office building through an API on the company’s VPN Similarly, all of one’s coworkers in one’s organization would be doing the same. Everyone enters and shares the same digital space going about one’s usual tasks through the present spatial-user interface (SUI) invention. In reality, everyone remains physically at their respective homes. There is an immediate savings in time and transportation energy.

[0019] In one aspect, the operation of the productivity device 100 is not unlike playing a musical instrument. We work for an organization-enterprise, or analogously belong to an orchestra. We play a role in the company or generate well-defined tonal ranges using a particular instrument within a coordinated performance. At the end of the workday we would disband and scatter to live our own lives. To even join the enterprise in the first place we have to train ourselves to use the device productively. We practise to sharpen our efficiency to contribute as expected to the company’s mission. With the present SUI we can do the same . . .only digitally. The device 100 comprises: isolating the locomotion simulation of the lower-body by use of leg-driven struts 112; mirroring the spine of our body plan is the central chassis having two or more longitudinal segments 102;suspending the chassis in mid-air with the user’s knees is possible through (inflatable?) pads 106 / 206 which disengages our hands to arrange the other components of the device such as the power-supply unit 124, the headset 122, headphones 128 or networking component (smartphone) 130; chassis segments secured in place at connector(s) by ‘pegs’ or some other lock mechanism 196; mounting pneumatic cylinders 112 on either side of the chassis segments using pairs of brackets 194; enclosed piston rods 192 within each pneumatic cylinder; electronically-adjustable air valves for rod extraction 199 within each pneumatic cylinder; electronically-adjustable air valves for rod retraction 198 within each pneumatic cylinder; fitting extended-assemblies for footing 196, including straps (not shown) to mount rubber flip flops (or lightweight slippers) in the patient-user’s size; pendular-swinging heel stops 110 pivoting off of the center of each footing assembly; terminus key-board-like controllers 104 with loops for each palm, for input and haptic- response engines; power-supply unit combined with off-loaded logic hardware 124 with connector cord to headset 126; headset 122 with gyroscopic sensors to detect head position to be worn on operator’s face while in use; can be combined with networking capability and rudimentary display of asmartphone 130, 132 using a Google-Cardboard type of head gear (option not shown) if not employing specialized high-resolution pancake lens-displays for each eye in stereoscopic coordination; cloud servers 120 integrated through an API from smartphone / compute component to provide designed metasphere-environment experience or an enterprise-office simulation; separate audio components 128 for added spatial-sensory input; connecting hinge 114 for optional attachment assembly with manually-adjustable diameter 116 connected to the end of the lower second-end segment of device, that rests on bed surface under operator 118 to provide support if necessary (Fig. 2).

[0020] Instead of a waist harness in the earlier referenced virtual-mobility invention, a padded cushion in the shape of an outline made from a mould of the user’s body laying down with limbs in the air, might be more comfortable. Otherwise, the user could lay in a power- assisted cradle that pivots in all manner of directions much in the same way that the gun-deck seats are depicted on the ‘Millennium Falcon’ during a battle sequence midway through the film “Episode 4: A New Hope.” There, the gun operator’s seat mimics the orientation of the space guns mounted outside the space vessel.

[0021] In another aspect, inserting a palm into a loop / strap 308 keeps the user’s hands in place for locomotive navigation and at the same time gives access to a plurality and variety of hand-operated keys much like playing a wind instrument. The user’s thumb would otherwise rest between two buttons in a neutral position to manipulate the struts clear from any buttons. This key -utility provides a shortcut to voice-assistant responses. With or without a virtual keyboard in view of the display, the operator-user can invoke a myriad of menus from which to choose a vast array of actions within the metasphere experience. It serves as an outlet for productivity like being able to record notes and generate metadata at any instance. Rather than reaching with a hand to adjust the size of a display window or its position, a sequence of key presses would accomplish the same thing.

[0022] In a third aspect, there are pedal controls triggered by the foot positions depicted in Fig. 4. There are no positions matching strictly Shift-Control-Alt functions, but there are four assigned Function or soft keys at the user’s disposal to add to the eight (tilting the strut stem forwards, backwards, left or right) assigned for the hands. This is merely a suggested basic design and is open to customization for specific purposes. When the foot is straight no triggers are available which may be convenient to not overlap locomotion and productivity controls. The two angled triggers for each foot might not be limited to 30 and 60 degrees. For example, there might be three up-down pairs of triggers at 25-, 45- and 65-degrees.

[0023] Although the suggested set of 72 keys, more or less the equivalent of the traditional 60%-full keyboard, there is more extensive use of 2- and 3-key combinations. It is like learning to type on an invisible keyboard in order to type on a virtual one. An analogy might be like learning to back up a 53-foot tractor-trailer with precision. Once trained however presumably by specialized software over time it becomes an indispensable skill in VR space that will serve the user well for the future. It is possible that certifiable grades will be awarded in the future to individuals for their proficiency in various more challenging “keyboard” layouts.

[0024] The overall benefit of the hidden-keyboard utility is to elevate the feeling of control over manipulating one’s movements or the settings within a desired metasphereenvironment. At the same time, it serves as a general solution for everyday productivity in the real world. Gamers and non-gamers alike, as well as everyone in between, have the choice of using the SUI for a personal mix of play or work. This unbounded flexibility makes this platform accessible to essentially everyone. This is critical for mass-adoption and interoperability.

[0025] Examples of mobility use cases include a virtual situation where the user appears to be lifting a car. The operator extracts all strut-rods in unison and tries to retract them all at the same time. The virtual application constricts the four pairs of air valves to generate maximum resistance and in so doing requires considerable effort on the user’s part to move the hands and feet in opposite directions. A simpler use case would just be climbing a ladder or stairs, in which a user alternately extends a leg and contra-lateral arm while pulling the other leg and oppositearm. More complicated use cases include swimming virtually on the surface (or underneath the surface) of a body of water to simulating the flapping of wings like a bird.

[0026] As a whole this SUI offers immediate advantages in terms of freedom and comfort for the driver-user. The all-important sense of immersion can produce the unmatched sensation of “transportation” in space and time. Many of the headgear elements contributing to weight can be off-loaded on to the substrate (bed surface). In time the head gear can resemble a thin visor like that depicted in Fig. 5 credited to a costume in the ‘Shake It Off music video. There is no hint of ‘thermals’ at work at any time leaving almost the entire skin area of the face to feel figuratively and literally breathable. (More room for video to capture facial expressions using light-fields. ..No more dual-strap for the head ever.. .No unnecessary sensors nor outwardfacing displays. ..) All the hardware and their limitations will seem swept “under the rug” or tucked “behind a curtain” providing a seamless user experience, when in fact they are laying right next to the operator unseen, power is drawn directly from a wall outlet and there are no messy wires / cables to get tangled up in.

[0027] In summary, this present SUI does not reduce the human condition but greatly enhances it in VR. When properly tuned the operator’s health is optimized by the motivation towards being physically active doing all manner of virtual action in reality. The community of users are unlikely to feel isolated but rather more connected. This promotes positive cognitive health, especially if the user chooses to be learning to move and do new things with successive sessions sing the present SUI. It can strengthen the family unit, maximize safety control and eliminate unnecessary waste of real energy consumption. The concept of ‘co-working’ can be deprecated when one’s home is no longer a source of distraction, and ride-sharing might die out too if we curtail commuting. The goal towards reversing climate change is now very real and achievable.

[0028] Extreme use cases include space travel and military duty where volumes and weight are stringently confined. Researchers on Antarctica can escape their surroundings at any time of day. Virtually all parts of the world now become habitable. Economic productivity gets a significant lift compared to any time in historical record because oil-rig workers no longer need to be shuttled off deck to vacation at home. There will be much, much more automation infactories. There will additionally be more accountability because once at work virtually the monitoring by supervisors and coworkers will be as natural as working together physically. Global mega-population centers can stop encroaching on wildlife and risk attack from wildlife. In fact, biodiversity can be expected to flourish again. If we become less dependent on the fastfood industry, we will use less packaging material and cut down on overall pollution.

[0029] There may be less conflict among people because we won’t feel so cramped and in competition for scarce resources. Israel and Palestine can coexist in their separate versions of the same real estate; hence, a one-state solution is achieved. More peace can resolve the world’s drug-abuse problems. Citizens can save more of their discretionary spending and be able to afford larger families. Life in general can feel less hectic and pressured. All of these benefits can be gained without the need for more mining which is the overlooked necessity of the EV transition. The proliferated SUI can be installed using sustainable materials and not rely on compute-heavy infrastructure like blockchain or generative-AI technologies. The opportunity is there to avoid developing artificial-general-intelligence, and in doing so offers the human race a release valve from self-destruction. We might learn to live in harmony with the end of Moore’s Law.

[0030] Can the poor afford to join this platform? It is a distinct possibility that this SUI facilitates cross-training. There is an initial hurdle of cost but that can be paid over time, in the same way that smartphones were financed. Public policy can help make this as affordable as possible through the more efficient deployment of funding. This could lead to less cross-border migration. The safety measures implicit in this SUI invention can meaningfully reduce human trafficking and scams by instituting software gates (safeguarding personal-identification information against profiling) that can protect the most vulnerable among us from undesirable interaction. All avatar characteristics can be kept securely native to your local communications device. There is no need for an open-source environment or standardization, since all operation choices are made at the edge of computing and only the results and actions are “published.” In the end, it’s your SUI and your decisions on how you use it.

[0031] Those skilled in the art of designing and building user interfaces will immediately concede that no new technology is needed to make this SUI succeed. There is no investment intoa technical foundation layer that needs to be built. This platform instantly invites application layers to be built on top of its hardware. There are no physics problems to research and work around to waste valuable R&D dollars. We can all benefit from adopting this SUI right away. Technically, this invention is itself just a new application that happens to spawn SDK’s and new programming languages. Simply put, it replaces the ubiquitous game controller forever. It replaces the QWERTY. It replaces the mouse. Eye-, hand-, body-tracking is not compulsory in our shared futures. It simplifies technology, invites ecosystem-building and accelerates the conception of the next ‘killer-app.’

[0032] Those skilled in the art of mixed-reality devices will recognize that investment in developing improved ‘pass-through’ technology is no longer an imperative. There need not be a monetization strategy going hand-in-hand with this technology either. We can limit ads to in- app spend. We can build on a business model untainted by nefarious motives. We probably won’t get rid of adult content. We can democratize access to the most prestigious and global sporting events and live performances like PPV (pay-per-view) but much less expensively. We can transform the art of making movies without the need for expensive tent-pole productions and sequels.

[0033] Storytellers will capture live action the way that James-Cameron’s Avatar franchise does but now viewers can themselves move around in the same space that the story moves in. The possibilities are endless. Memories of loved ones can live on forever and feel as close as yesterday when shot through the Vision-Pro product from Apple that has recently hit the market. In fact (reality?), the technology to “film” 3D already exists today using small flying drones and ‘BulletTime’-like post-production software. We can reinforce our individual capacity for tolerance for seeing ourselves as reflections of other demographic groups; racial friction can become a relic. A high tide will truly lift all boats.

[0034] Since many modifications, variations, and changes in detail can be made to the described preferred embodiments of the invention, it is intended that all matters in the foregoing description and shown in the accompanying drawings be interpreted as illustrative and not in a limiting sense. Thus, the scope of the invention should be determined by the appended claims and their legal equivalence.

Claims

What claim is:

1. I claim a 3D user-interface system-invention characterized by or comprised of a digital version of a real or imagined spatial map stored on a web server accessed through a smartphone app paired with a goggle-headset, headphones and set of five gyroscopic sensors (two for hands, two for the feet and one for the head) capturing input from four spatial quadrants tracking one of the four operator’s extremities, while the operator is fixed in a supine position on a flat, well- cushioned surface, whereby general up-down motions of his / her limbs are interpreted and transformed via software into virtual locomotion;2. I claim the system of Claim 1 wherein a physical, electronically-variable resistance, four-strut chassis mobility controller is used having handles and footing for hands and feet, respectively, that electronically track the limb motions separately as well as the position of the chassis as a whole, while the operator lays in a supine position on a comfortable; the pneumatic valves determining the motion resistance can be electronically adjusted to suit the virtual environment being explored; there are small pillows attached to the center of the chassis to allow the operator to release the grip from one or both hands; there is a pendular loop of adjustable diameter on the second end of the chassis for added support and the footing design including swiveling heel-stops released by twisting the ankle slightly;3. The system of Claim 2 wherein there is a harness secured to the bed or a padded cushion moulded to the shape of the user that holds the user’s waist in place such that the user can be held at the center of the bed to maximize safety, but alternatively, a sophisticated power-assisted cradle supports the user off the floor sensing the leaning of the user in 3-dimensional space;4. The system of claim 1 wherein the operator can don VR headgear, with matching headphones and faced with an operating-system home menu navigate through a simulated 3D world all the while having access to communication with coworkers in a work setting or family in a home setting;5. The system of claim 1 wherein the operator uses a four-piece, 68-key interface including 24 separate buttons on either hand and 10 controls for each foot;6. The system of claim 1 or 2 wherein the use of a power bank can make the entire assembly portable for outdoor use if so desired; with efficient power management and lightweight construction materials and operator could enjoy many hours of continuous use and physical activity which may escape notice due to being absorbed in the immersive experiences;

Images