General purpose spatial user interface

Abstract

Virtual reality systems face challenges in the current form with respect to safety, body tracking and weight. The invention solves all these obstacles with a lightweight mechanical interface that operates in the supine position of the user. All limbs are in motion in space, the range of motion is extended, and the operator is thus decoupled, always maintaining a distance from any object in the vicinity. A four-piece keyboard concept explains multiple desired actions in virtual space, the weight of the hardware components is offloaded from the headset and distributed on the resting surface. This direct solution provides an unprecedented level of immersive experience and can be expected to revolutionize the way we work, entertain, live, and most importantly, take care of the health of our planet. This experience is delivered in a compact form factor that is well suited for applications where space and weight are highly restricted.

Description

Technical Field

[0001] Currently, driven by new display technologies, there is a gradual shift towards spatial computing; however, virtual reality (VR) as a dedicated application remains primarily gaming. Therefore, default input systems are merely extensions of game controllers, or upgraded versions of eye-hand tracking. However, such controllers struggle to provide a higher level of intuitive immersion. As a result, the interactive experience confines virtual movement to the use of directional buttons, and disbelief in the mind is almost never truly addressed.

[0002] To bridge the gap to a truly immersive experience, porting the familiar one-to-one mouse-based (graphical) interface has proven physically limited. Perhaps what truly demonstrates a proof-of-concept of "teleportation" is only a half-step, as the QWERTY keyboard once did. That is, writing with a quill pen is simplified to pressing a stamp dipped in ink. In terms of speed alone, the benefits are perhaps too good to pass up. Ironically, the modern keyboard became the standard by limiting potential speed gains to avoid the lag of mechanical typewriter keys.

[0003] Existing technology has shown that it's not immediately obvious how keyboard analogies can be extended to be useful in the third dimension. Moreover, it seems unlikely that many would be willing to develop a solution with low operational fidelity and significantly inferior performance. Similarly, there's no similar mechanical "typewriter" to train users to master its unintuitive operation. This doesn't mean that spatial input systems like keyboards can't work, nor does it mean they can't allow us to just overcome the perceived barriers to "immersion" and enjoy believable VR experiences.

[0004] The biggest obstacle to experiencing immersion ultimately boils down to the fundamental issue of user safety. If those embarrassing "fail" memes about bumping into walls or accidentally smashing displays (controllers being thrown at the screen) have taught us anything, it's that serious injuries and lawsuits await VR device vendors. Even with billions of dollars invested in R&D, large tech companies continue to warn of the severe physical (spatial) limitations to safely deploying their VR products, to the point that it seems impossible to decouple VR from danger.

[0005] Another practical limitation to bringing viable VR systems to market is their appearance. Bulky or uncool solutions will face backlash from those seeking a "fashionable" experience. Currently available are omnidirectional treadmills, gliders with special footwear, and body-tracking solutions. Users typically experience gravity and / or the dragging force of the restrictive equipment. Watching users make unnatural movements almost in mid-air can be tiring. Besides the overall weight of the system, the room space they occupy is also a factor affecting user experience. The weight of the display hardware, the processing power, and the bulky power supply are currently significant constraints on usage time and widespread adoption.

[0006] Essentially, the first to enter the market with a secure VR system is highly likely to be granted legitimate market dominance. Surprisingly, this system may ultimately become the lowest-cost solution among all the variants under development. The proposed solution draws upon the authors' basic four-limb chassis concept (WO2023140881A1) and extends it with their virtual reality operating system concept (USPTO Application #63521063). It includes a learning input system that accommodates omnidirectional virtual movement and built-in or inherent security, compressing rapid, unrestricted functionality into a compact, ergonomic monolithic structure.

[0007] This invention follows the other two applications in the most general way, thus the first two are merely special cases of this broad third application. Each of the above inventions is based on the same mechanical body / design, targeting an increasingly non-specialized market. This generalization is necessary because voice control is neither the ultimate nor the optimal input method. The best input is the fastest, and this generalized solution may always remain a suboptimal solution outside of thought-driven (BCHI) input. Although this is orders of magnitude more difficult than anything currently in use, it may only improve the latency of this invention by a mere few milliseconds.

[0008] Furthermore, this generalized solution is superior because it encourages users to spend more time on safe muscle resistance training. The explicit incidental benefits of this invention include: more engaged participation in real-life interactions, closer family relationships, greater physical resilience, and, most importantly, a deeper appreciation for the natural environment overall. In many ways, the opposite of what might be expected from long marathon gaming sessions or drone piloting (the future implied by the "Wall-E" movie) is now possible. Ideally, longer life expectancy and a higher quality of life can be fully anticipated. Attached Figure Description

[0009] The invention will now be described by way of example with reference to the accompanying drawings, wherein: Figure 1 An exemplary first-order frame for supporting a second-order motion adaptation element according to an embodiment of the present invention is shown from a top view. Figure 2 This shows the expected user usage. Figure 1 The scenario depicted in the operation is constructed as shown. Figure 3 The diagram shows 24 third-order input elements for one hand (right hand), with the other hand mirroring the settings. The lever body has a keyless thumb neutral position, used to operate the arm drive cylinder when the palm is comfortably positioned in the grip loop.

[0010] Figure 4 The diagram shows the foot position adopted by the operator to activate the eight additional third-order input contacts / sensors on one foot, with the other foot mirrored in the position. In the neutral, horizontal, centered position, the operator can manipulate the leg-driven cylinders.

[0011] Figure 5 Using a face mask used in an outfit from a popular music video as an example, we can illustrate the optimal headgear specifications. Detailed Implementation

[0012] The following detailed description is exemplary in nature only and is not intended to limit the described embodiments or their application and use. As used herein, the terms "exemplary" or "illustrative" mean "used as an example, instance, or illustration." Any implementation described herein as "exemplary" or "illustrative" is not necessarily to be construed as being more preferred or advantageous than other implementations. All implementations described below are exemplary implementations, provided to enable those skilled in the art to make or use embodiments of this disclosure, and are not intended to limit the scope of this disclosure as defined by the claims. For the purposes of this description, the terms "first," "second," "left," "rear," "right," "front," "vertical," "horizontal," and their derivatives should be referred to as... Figure 1 The present invention is to be understood in accordance with the orientation shown in the figures. Furthermore, this disclosure is not intended to be limited by any express or implied theory presented in the foregoing technical field, background art, brief summary of the invention, or the following detailed description. It should also be understood that the specific apparatus and processes shown in the drawings and described in the following specification are merely exemplary embodiments of the inventive concept as defined in the appended claims. Therefore, unless expressly stated otherwise in the claims, specific dimensions and other physical characteristics relating to the embodiments disclosed herein should not be considered limiting.

[0013] like Figure 1 As shown, in one embodiment of the invention, a device 100 for navigation in virtual reality supports a supine operator 118. The device includes a connected pneumatic piston assembly 112, supported by a central axial chassis 102 (resembling a bilaterally symmetrical vertebrate body structure). Independent cylinders / pistons are located on each side of the body, equipped with independent electronically controlled pneumatic valves (199) for piston rod extension and (198) for piston rod retraction, to target specific limb isolation muscle training. The upper and lower chassis segments are separately adjustable in length to accommodate the patient's height and can be connected via double hinges (not shown) or a flexible constraint groove design that allows the device to fold or twist (like a blind person's cane). A gyroscope sensor is fixed to the chassis to enable the system to identify the user's desired spatial orientation. An optional assembly 114-116 allows the chassis / spine to be raised on the bed surface. Foot equipment can be replaced by untying its straps. This device is designed to balance the opposing forces between the upper and lower body. It does not require fixed installation in a fixed position. The overall design simulates walking, swimming, skiing, and climbing / descending a ladder. The flexible chassis material accommodates the operator's natural flexibility during movement. When not in use, the device is designed to fold in half or simply lean against a bedside or wall until needed again.

[0014] This invention, with its several productivity features, is a generalization of similar inventions previously used for fitness and recreational exploration in the elderly, which together easily function as a "60% keyboard." With an operator-safe and ergonomic design, users can immerse themselves in a digital environment for extended periods, exercising, playing, and now even working. The device's first-order mobility (arm and leg actuators) ensures free access to every corner of the "metaspace" or digital landscape. When it's time to answer a call or continue working on a project, the second-order features are truly at your fingertips, used to generate computer screen images, contact voice assistants, and / or create content.

[0015] An example of this application process is true remote working. Instead of commuting or taking public transportation to / from a shared physical office, people access a digital peer of their employer's office building via an application programming interface (API) on the company's virtual private network (VPN). Similarly, all colleagues within the organization do the same. Everyone enters and shares the same digital space, using their own space's user interface (SUI) to perform their usual tasks. In effect, everyone remains physically in their own homes. This immediately results in savings in time and transportation energy consumption.

[0016] In one respect, operating the Productivity Device 100 is no different from playing a musical instrument. We work for an organization—a company—or, similarly, belong to an orchestra. We play a role in the company, or, in a coordinated performance, play a clearly defined range on a particular instrument. At the end of the workday, we disperse and go our separate ways. To join a company, we must first train ourselves to use the device efficiently. We practice to hone our efficiency, thereby contributing to the company's mission as intended. With this SUI, we can do the same thing… only digitally. Device 100 includes: Simulation of lower body movement using leg-driven support 112 to isolate it; Corresponding to the spinal mirror image of our body structure is the central chassis, which has two or more longitudinal segments 102; With the help of the (inflatable) pad 106 / 206, it is possible to suspend the chassis in mid-air with the user's knees, which frees up our hands to arrange other components of the device, such as the power unit 124, headband 122, audio component 128 or network communication component (smartphone) 130. The chassis sections are secured in place at the connector by a "pin" or some other locking mechanism 196; The pneumatic cylinder 112 is mounted on both sides of the chassis section using paired brackets 194; The enclosed piston rod 192 is located inside each pneumatic cylinder; The piston rod extension is equipped with an electronically adjustable pneumatic valve 199 located in each pneumatic cylinder; An electronically adjustable pneumatic valve 198 for piston rod retraction is located in each pneumatic cylinder; Install foot support extension component 196, including straps (not shown), to install a rubber flip-flop (or lightweight slipper) adapted to the patient-user size. The swing-type heel stop 110 pivots and swings from the center of each foot support component; End keyboard controller 104 with loops for each palm for input and haptic response engine; A power supply unit (combined with unloaded logic hardware) 124, with a connection cable 126 leading to the head-mounted device; The head-mounted device 122 is equipped with a gyroscope sensor for detecting head position and is worn on the operator's face during use; if a dedicated high-resolution pancake lens display for stereoscopic coordination of each eye is not used, the Google Cardboard-like head-mounted device (option not shown) can be used in conjunction with the networking capabilities and basic display of smartphones 130 and 132. The cloud server 120 integrates via application programming interfaces (APIs) from smartphones / computing components to provide a well-designed metaspace environment experience or enterprise office simulation; A separate audio component 128 is used to increase spatial sensory input; A connecting hinge 114, for an optional accessory assembly, with a manually adjustable diameter 116, connects to the end of the second end segment below the device, which rests on the bed surface below the operator 118, providing support when necessary. Figure 2 ).

[0017] Unlike the lumbar restraints in previously cited virtual mobility inventions, a soft cushion shaped like a model of the user's supine, limbs-free body might be more comfortable. Alternatively, the user could lie in a powered support frame that pivots in all directions, much like the gunner's seat on the Millennium Falcon depicted in the battle scene midway through the film *Star Wars: Episode IV – A New Hope*. There, the gunner's seat mimics the orientation of a space cannon mounted on the outside of the spaceship.

[0018] On the other hand, inserting the palm into the loop / strap 308 keeps the user's hands in place for navigation while providing access to multiple and varied hand operation keys, much like playing a wind instrument. Otherwise, the user's thumb would rest in a neutral position between two buttons, manipulating the support while avoiding any buttons. This keying functionality provides a shortcut to voice assistant responses. Regardless of whether a virtual keyboard is visible on the display, the operator-user can invoke numerous menus, selecting a wide variety of actions within the metaspace experience. It serves as an outlet for productivity, such as the ability to take notes and generate metadata at any time. A series of key presses accomplishes the same thing without reaching out to adjust the size or position of the display window.

[0019] In the third aspect, there exists a... Figure 4 The pedal control is foot-position triggered, as depicted in the diagram. There is no strict correspondence between the Shift-Control-Alt function and the pedals, but four function keys or soft keys are assigned to the user to complement the eight assigned to the hands (tilting the lever forward, backward, left, or right). This is merely a suggested basic design and can be customized for specific uses. No trigger keys are available when the foot is extended, which may facilitate preventing overlap between movement and productivity controls. The two angled trigger keys for each foot may not be limited to 30 and 60 degrees. For example, there could be three pairs of vertically arranged trigger keys at 25, 45, and 65 degrees respectively.

[0020] While a 72-key layout is recommended, roughly equivalent to 60% of a traditional full-size keyboard, 2-key and 3-key combinations will still be used more extensively. This is akin to learning to type on an invisible keyboard before learning to type on a virtual one. An analogy might be learning to precisely reverse a 53-foot tow truck. However, once trained over time with specialized software, it will become an indispensable skill in VR spaces, greatly benefiting users in the future. It may be possible to award individuals certified levels in the future, recognizing their proficiency in various, more challenging "keyboard" layouts.

[0021] The overall benefit of the hidden keyboard is enhanced control over one's own movement and desired settings within the metaspace environment. Simultaneously, it serves as a universal solution for everyday productivity in the real world. Gamers, non-gamers, and everyone in between can choose to use the SUI for a personalized play or work combination. This unrestricted flexibility makes the platform virtually accessible to everyone. This is crucial for widespread adoption and interoperability.

[0022] Examples of mobility use cases include a virtual scenario where the user appears to be lifting a car. The operator simultaneously extends all the support bars and attempts to retract them all at the same time. The virtual application retracts four pairs of pneumatic valves to generate maximum resistance, requiring the user to exert considerable force to move their hands and feet in opposite directions. A simpler use case is climbing a ladder or stairs, where the user alternately extends one leg and the opposite arm while pulling on the other leg and the opposite arm. More complex use cases include virtually swimming on (or beneath) the surface of water, culminating in simulating flapping wings like a bird.

[0023] Overall, this SUI offers immediate advantages to the driver-user in terms of freedom and comfort. The crucial sense of immersion creates an unparalleled feeling of "transportation" through space and time. Many of the weight-bearing head-mounted components can be unloaded onto the base (bed). Over time, the head-mounted equipment can become like... Figure 5Like the thin mask depicted in the image, this one comes from an outfit featured in the "Shake It Off" music video. There's no sign of "thermal effects" at any point, allowing almost the entire skin area of ​​the face to breathe, metaphorically and literally. (More space for video to use light fields to capture facial expressions…no more double headbands…no extra sensors, no outward-facing displays…) All the hardware and its limitations seem to be "swept under the rug" or "hidden behind a curtain," providing a seamless user experience, even though they lie unseen right next to the operator; power is drawn directly from a wall socket, and there are no tangled wires / cables.

[0024] In conclusion, this SUI does not diminish the human condition; rather, it greatly enhances it within VR. When properly calibrated, the operator's health is optimized by being motivated to remain physically active through various virtual movements in reality. User groups are less likely to feel isolated and more connected. This promotes positive mental health, especially when users choose to learn mobility and try new things through repeated use of the SUI's lessons. It can solidify family units, maximize security control, and eliminate unnecessary waste of real energy expenditure. When one's home is no longer a source of distraction, the concept of "co-working" can be abolished; carpooling may also disappear if we reduce commutes. The goal of reversing climate change becomes very real and achievable today.

[0025] Extreme use cases include space travel and military duty, where size and weight are strictly limited. Researchers in Antarctica can escape their environment at any time of day. Almost every part of the world is now habitable. Economic productivity is significantly higher than at any time in history because oil rig workers no longer need to be ferried off deck to go home for vacations. Automation in factories will increase dramatically. Furthermore, there will be more accountability, as supervision by supervisors and colleagues will feel as natural as working face-to-face once work is done virtually. Global mega-population centers can stop encroaching on wildlife habitats and avoid the risk of attacks from wildlife. In fact, biodiversity can be expected to flourish again. If we reduce our reliance on the fast food industry, we will use less packaging material and reduce overall pollution.

[0026] Interpersonal conflict may decrease because we will no longer feel so crowded, nor will we compete for scarce resources. Country A and Country B can coexist on the same land, each with its own version; thus achieving a one-country solution. More peace can solve the world's drug abuse problem. Citizens can save more on discretionary spending and are able to support larger families. Overall, life can be less hectic and stressful. All these benefits can be achieved without the need for more mining, a neglected necessity in the electric vehicle (EV) transition. Widespread SUI can be installed using sustainable materials without relying on computationally intensive infrastructure such as blockchain or generative AI technologies. There is an opportunity to avoid developing general AI, and doing so provides a way for humanity to release stress. We may be able to learn to live in harmony with the end of Moore's Law.

[0027] Can the poor afford to join this platform? This SUI has the potential to promote cross-training. Initial cost barriers exist, but payments can be made in installments over time, much like smartphone financing. Public policy can help make it as affordable as possible by allocating funds more efficiently. This could lead to a reduction in cross-border migration. The security measures implicit in this SUI invention can tangibly reduce human trafficking and fraud by setting software gating (protecting personally identifiable information from profiling), thus protecting the most vulnerable among us from harmful interactions. All virtual avatar characteristics can be securely stored on your local communication device. No open-source environment or standardization is needed because all operational choices are made on the edge computing side; only the results and actions are "published." Ultimately, this is your SUI, and how you use it is up to you.

[0028] Anyone skilled in designing and building user interfaces will immediately recognize that the success of this SUI requires no new technology. There is no need to invest in building a foundational technology layer. The platform is immediately available for building application layers on top of the hardware. There are no physical problems to study and circumvent, thus avoiding wasting valuable R&D funds. We can all immediately benefit from adopting this SUI. Technically, this invention itself is simply a new application that happens to spawn software development kits (SDKs) and new programming languages. In short, it permanently replaces the ubiquitous game controller. It replaces the QWERTY keyboard. It replaces the mouse. Eye-hand-body tracking is not mandatory in our shared future. It simplifies technology, facilitates ecosystem building, and accelerates the conception of the next "killer app."

[0029] Those well-versed in mixed reality devices will realize that investing in developing improved "pass-through" technology is no longer a priority. Nor will there be a monetization strategy inherent in this technology. We can limit advertising to in-app purchases. We can build on business models free from malicious motives. We probably can't eradicate adult content. We can democratize access to the most prestigious global sporting events and live performances (such as pay-per-view (PPV)) at a much lower price. We can revolutionize the art of filmmaking without the need for expensive pillar productions and sequels.

[0030] Storytellers will capture live-action footage, much like in James Cameron's Avatar series, but now audiences can move freely within the same space where the story unfolds. The possibilities are endless. Memories of loved ones can be preserved forever and feel like yesterday when filmed with Apple's newly released Vision-Pro. In fact, the technology to "film" in 3D already exists, achievable using small flying drones and post-production software like "BulletTime." We can strengthen our capacity for tolerance, viewing ourselves as a mirror image of other population groups; racial friction can become a relic of history. The tide truly does lift all boats.

[0031] Since many modifications, variations, and changes in detail can be made to the preferred embodiments of the invention described herein, it is intended that all matters in the foregoing description and the contents shown in the accompanying drawings be interpreted as illustrative rather than restrictive. Therefore, the scope of the invention should be determined by the appended claims and their legal equivalents.

Claims

1. A three-dimensional user interface system, characterized in that, include: A digital version of a real or imagined spatial map stored on a web server, accessible via a smartphone application paired with a goggle-style headgear, headphones, and a set of five gyroscope sensors (two for the hands, two for the feet, and one for the head) to capture input from four spatial quadrants, each tracking one of the operator's four limbs, while the operator is fixed in a supine position on a flat, well-padded surface, whereby the general up-and-down movements of their limbs are interpreted by the software and converted into virtual movement.

2. The system of claim 1, wherein a physical, electronically variable resistance four-pillar chassis movement controller is used, the controller having handles and foot support structures for the hands and feet respectively, which electronically track limb movements and the overall position of the chassis while the operator is comfortably supine; pneumatic valves determining the resistance of movement can be electronically adjusted to adapt to the virtual environment being explored; several small pillows are attached to the center of the chassis to allow the operator to release the grip of one or both hands; an adjustable-diameter swing ring is present at the second end of the chassis to provide additional support, and the foot support design includes a rotating heel stop that can be released by slightly twisting the ankle.

3. The system of claim 2, wherein there is a restraint strap fixed to the bed, or a cushion molded to the shape of the user, which holds the user's waist in place so that the user can be held in the center of the bed to maximize safety, but alternatively, a sophisticated power-assisted support lifts the user off the ground and senses the user's tilt in three-dimensional space.

4. The system of claim 1, wherein the operator can wear a VR headset with matching headphones and face the main menu of the operating system to navigate in a simulated three-dimensional world while always being able to communicate with colleagues in the work environment or family members in the home environment.

5. The system of claim 1, wherein the operator uses a four-piece 68-key interface, comprising 24 individual buttons on each hand and 10 controls on each foot.

6. The system of claim 1 or 2, wherein the use of a portable power source makes the entire component portable so that it can be used outdoors if needed; with efficient power management and lightweight construction materials, the operator can enjoy many hours of continuous use and physical activity that may not be noticed due to immersion in the immersive experience.

Images