Mobile platform as a physical interface for interaction

Abstract

The present application discloses systems and methods for providing a mobile computing platform as a physical interface to an interactive space. A user of a host device (e.g., a headset) can experience the interactive space. The interactive space can include a view of virtual content. A position and / or heading of the mobile computing platform relative to the virtual content of the interactive space can be determined. Remote command information can be determined based on the relative position information and / or user input information that conveys user entry and / or selection of one or more input elements of the mobile computing platform. The remote command information can be configured to enable user interaction with the virtual content in the interactive space based on user interaction with the mobile computing platform.

Description

Technical Field

[0001] The systems and methods described herein relate to providing a mobile computing platform as a physical interface tool for an interactive space. Background Art

[0002] There are many types of vision systems that can provide virtual and / or augmented reality (VR and / or AR) displays. In particular, wearable technologies such as head-mounted displays (HMDs) can be used. Wearable technologies can present an image of virtual content to a user, which can be perceived by the user as existing in the real world. The user can interact with the virtual content using gestures. Summary of the Invention

[0003] One or more implementations of the systems and methods described herein facilitate providing a mobile computing platform as a physical interface tool for an interactive space. The interactive space can include one or both of a virtual reality environment and / or an augmented reality environment. The augmented reality environment can include a view of an image of virtual content superimposed on a view of the real environment. The virtual reality environment can include a view of an image of virtual content within a virtual environment. The user can use the mobile computing platform as a remote control for interacting with the virtual content. For example, the mobile computing platform can be used to point to and / or select virtual content for interaction. The mobile computing platform can facilitate precise selection of virtual content, which may be difficult to achieve if such selection is attempted only through physical gestures. The mobile computing platform can present a user interface that includes input elements. The user can select one or more input elements to effect one or more interactions with the virtual content that the mobile computing platform may be pointing to. Other interactions are described herein.

[0004] A system configured to provide a mobile computing platform as a physical interface for an interactive space can include one or more of a host device, a mobile computing platform, and / or other components. The host device can be configured to be worn by a user. The host device can include one or more physical processors, one or more image forming components, and / or one or more other components. The one or more image forming components can be configured to generate light to form an image of virtual content. The one or more image forming components can be configured to present the image to the user. The image can be superimposed on a view of the real world to create an interactive space. By way of non-limiting illustration, the virtual content can be perceived by the user as existing in the real world. The virtual content can include one or more virtual objects.

[0005] One or more physical processors of the host device can be configured by machine-readable instructions. The machine-readable instructions can include one or more computer program components. The computer program components can include one or more of a content component, a control component, a relative position component, a communication component, and / or other components.

[0006] The content component can be configured to obtain information defining virtual content. The information defining virtual content can include information defining one or more of the visual appearance of the virtual content, one or more interaction capabilities of the virtual content, movement of the virtual content, and / or other information.

[0007] The communication component can be configured to establish a wireless connection between the host device and a mobile computing platform that is separate and different from the host device. In some implementations, user interaction with the mobile computing platform can facilitate user interaction with virtual content in the interactive space.

[0008] The relative position component can be configured to obtain relative position information. The relative position information can convey the position and / or orientation of the mobile computing platform relative to the perceived position and / or orientation of the virtual content.

[0009] The control component obtains user input information and / or other information. The user input information can convey user entry and / or selection of one or more input elements of the mobile computing platform. User entry and / or selection of one or more input elements can include a part of user interaction with the mobile computing platform.

[0010] The control component can be configured to determine remote command information and / or other information. The remote command information can be configured to enable user interaction with virtual content in the interactive space based on user interaction with the mobile computing platform. The remote command information can be determined based on one or more of the relative position information, user input information, and / or other information.

[0011] The control component can be configured to control one or more image forming components based on the remote command information and / or other information. By way of non-limiting illustration, one or more image forming components can be controlled to cause an image of the virtual content to reflect user interaction with the virtual content.

[0012] After considering the following description and the appended claims in conjunction with the accompanying drawings, these and other objects, features, and characteristics of the present disclosure, as well as the operating methods and functions of the structural related components and the combination of parts and manufacturing economy, will become more apparent, all of which form a part of this specification, wherein like reference numerals denote corresponding parts in the various drawings. However, it should be clearly understood that the drawings are for illustrative and descriptive purposes only and are not intended as a definition of any limitation. As used in the specification and claims, the singular forms "a", "an", and "the" include plural referents unless the context clearly dictates otherwise. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1Illustrates a system configured to provide a mobile computing platform as a physical interface for an interactive space, according to one or more implementations.

[0014] Figure 2 Illustrates a method of providing a mobile computing platform as a physical interface for an interactive space, according to one or more implementations.

[0015] Figure 3 Illustrates a method of providing a mobile computing platform as a physical interface for an interactive space, according to one or more implementations.

[0016] Figure 4 Illustrates an exemplary head-mounted display.

[0017] Figure 5 Illustrates a configuration of a light source and optical elements of an image forming component, according to one or more implementations.

[0018] Figure 6 Illustrates Figure 1 an implementation of a system Figure 1 that illustrates a view of an interactive space that includes a set of virtual objects and a mobile computing platform that provides a physical interface for the interactive space.

[0019] Figure 7 Illustrates Figure 1 an implementation of a system Figure 1 that illustrates a view of an interactive space that includes a set of virtual objects and a mobile computing platform that provides a physical interface for the interactive space.

[0020] Figure 8 Illustrates an exemplary user interface presented on a display of a mobile computing platform configured to have a physical interface for an interactive space.

[0021] Figure 9 Illustrates an exemplary user interface presented on a display of a mobile computing platform configured to have a physical interface for an interactive space.

[0022] Figure 10 Illustrates an exemplary user interface presented on a display of a mobile computing platform configured to have a physical interface for an interactive space.

[0023] Figure 11 Illustrates an exemplary user interface presented on a display of a mobile computing platform configured to have a physical interface for associating graphic images with real-world or virtual objects of an interactive space.

[0024] Figure 12a And Figure 12bShows another exemplary user interface presented on a display of a mobile computing platform configured as a physical interface for associating graphical images with real-world or virtual objects in an interactive space.

[0025] Specific implementation method

[0026] Figure 1 Shows a system 100 configured to provide a mobile computing platform as a physical interface for an interactive space, according to one or more implementations. The interactive space can include one or more of an augmented reality environment, a virtual reality environment, and / or other interactive spaces. The augmented reality environment can include a view of an image of virtual content formed as an overlay on a view of the real world. The virtual reality environment can include a view of an image of virtual content formed within a virtual environment. In the context of virtual reality and / or augmented reality, the terms "space" and "environment" can be used interchangeably herein. The virtual content can include one or more of virtual objects, virtual worlds, and / or other virtual content.

[0027] Interactions with virtual objects in the interactive space can be determined by hand movements and / or gestures. However, such hand-based interactions can be challenging. This can be due to the precision of hand movements that the user may need to make for interaction and / or due to the complexity of the virtual objects themselves. The mobile computing platform described herein can be configured to provide a handheld physical tool to generally interact with virtual objects in the interactive space and / or the system. Precise interactions and / or other interactions with virtual objects can be accomplished easily and intuitively via the mobile computing platform.

[0028] In some implementations, the mobile computing platform can act as a 3D controller and / or act as a pointer to control a cursor for selecting virtual objects perceived to be present in the interactive space. For example, sensors on the mobile computing platform can provide information indicating the position and / or heading of the mobile computing platform in the real world. In some implementations, a sensor 142, such as an image sensor on a host device, can identify the mobile computing platform and determine its position and / or heading via image-based techniques. The cursor can act as an extension of the mobile computing platform; for example, the user can point the mobile computing platform to the location where they want the cursor to move. The mobile computing platform can utilize existing and / or custom user interface elements / buttons to interact with virtual objects. By way of non-limiting illustration, text can be input into virtual objects (e.g., email, web browser, etc.) using an existing and / or custom keyboard. Custom user interface elements can allow certain interactions with virtual objects. The interactions can include one or more of selection, movement, scaling, rotation, changing the texture / color of the virtual object, resetting SLAM, surface scanning, and / or other types of interactions.

[0029] In Figure 1 , system 100 may include host device 102, one or more other host devices 123, mobile computing platform 126, one or more other mobile computing platforms 121, and / or one or more of other components. Note that although some descriptions herein may be directed to host device 102, this is for illustrative purposes only and should not be considered restrictive. For example, it should be understood that each of one or more other host devices 123 may be configured in the same or similar manner as host device 102. Further note that although some descriptions may be directed to mobile computing platform 126, this is for illustrative purposes only and should not be considered restrictive. For example, it should be understood that each of one or more other mobile computing platforms 121 may be configured in the same or similar manner as mobile computing platform 126.

[0030] In some implementations, one or more components of host device 102 may be included in and / or otherwise incorporated into host device 102. By way of non-limiting illustration, host device 102 may include and / or otherwise maintain one or more physical processors 104, one or more image forming components 120, one or more sensors 122, and / or one or more of other components. Host device 102 may be configured to be worn on a user's head. By way of non-limiting illustration, host device 102 may include a headset, such as a head-mounted display (HMD), glasses, goggles, and / or one or more of other devices.

[0031] In some implementations, one or more components of host device 102 may be included in one or more devices external to host device 102. For example, one or more components may be included in one or more of a desktop computer, a laptop computer, other mobile computer configurations, and / or other computing devices. By way of non-limiting illustration, one or more physical processors 104 and / or other components may be included in a mobile computing device external to host device 102 ( Figure 1 not shown). Host device 102 may be contacted and / or otherwise connected to one or more external devices. Such connections may be wired (e.g., USB, HDMI, and / or other wired connections) and / or wireless (e.g., Bluetooth, Wi-Fi, and / or other wireless connections).

[0032] In Figure 1Among them, each image forming component in one or more image forming components 120 may be configured to generate light rays to form an image of virtual content. The human perception system can effectively combine various sensory cues so as to perceive "physically reasonable" virtual content in a real-world environment. For example, the human perception system can integrate, among other things, one or more sensory cues such as brightness, depth, and / or shape information to form and / or perceive coherent virtual content. As a result, as described herein, the characteristics of human perception can be utilized by the visual system, and a hardware and / or software architecture can be adopted to form virtual content (e.g., digital images) that can be located and / or perceived as being in the real world through neuroscience principles. For example, the virtual content can be perceived as existing at a given location in the real world. Based on the viewpoint in the interactive space from which the virtual content is being viewed, the virtual content can be perceived as having a specific heading and / or orientation.

[0033] Each image forming component in one or more image forming components 120 may include one or more of the following: one or more light sources, one or more optical elements, and / or one or more other components. In some implementations, a single light source may be arranged on the host device 102 to direct light rays to one or more optical elements. The one or more image forming components 120 may be configured such that an image of the virtual content can be superimposed on a view of the real world to create a view of the interactive space. In some implementations, the images may be presented separately to each eye of the user as a stereoscopic image pair.

[0034] The light source may include one or more of the following: microelectromechanical systems (MEMS), RGB laser scanners, microLED microdisplays, LED light-emitting liquid crystal on silicon (LCOS) microdisplays, RGB laser LCOS microdisplays, digital light projectors (DLP), digital micromirror devices (DMD), liquid crystal displays (LCD), organic light-emitting diode (OLED) displays, OLED microdisplays, and / or other light sources.

[0035] Note that the use of the term "light ray" is not intended to limit the scope of the present disclosure to a single discrete photon and / or photon packet. Instead, in one or more implementations, the present disclosure contemplates that a light ray refers to a light beam including multiple and / or continuous photons.

[0036] In some implementations, one or more optical elements of one or more image forming components 120 may be disposed on the host device 102 such that when the host device 102 is worn on the user's head, the user's gaze may be directed towards the one or more optical elements. In some implementations, the optical elements may form at least a portion of the host device 102 through which the user may view the real world. In some implementations, the optical elements may include one or more reflective and / or partially reflective surfaces. The optical elements may be formed of reflective and / or partially reflective materials. The optical elements may be formed of transparent and / or partially transparent materials. The materials of the optical elements may include one or more of ZEONEX, cyclic olefin polymer (COP), cyclic olefin copolymer (COC), polycarbonate, poly(methyl methacrylate) (PMMA), and / or other materials.

[0037] In some implementations, the optical elements may include one or more of waveguides, optical coupling features, and / or other components. The waveguides may include one or more of layered waveguides, planar partially reflective mirror array waveguides, diffractive waveguides, diffractive waveguides including Bragg gratings, freeform surface prisms, and / or other waveguides. In some implementations, the waveguides may include optical coupling features and / or may be coupled to optical coupling features configured to couple light into the waveguides. The light coupled into the waveguides may propagate through the waveguides and be directed outwards from the waveguides towards one or both of the user's eyes.

[0038] In some implementations, one or more optical elements of one or more image forming components 120 may be disposed on the host device 102 such that light generated by one or more light sources may be directed to, coupled into, and / or extracted from the one or more optical elements towards the user's eyes. Images of virtual content may be projected onto the retinas of the eyes such that the images may be superimposed on the user's view of the real world.

[0039] Now referring Figure 4 , in some implementations, a head-mounted headset (e.g., Figure 1 the host device 102 therein) may include the HMD 400. The system 100 ( Figure 1One or more components of ( ) can be held by a part of the HMD 400, incorporated into a part of the HMD 400, and / or otherwise included as a part of the HMD 400. By way of non-limiting illustration, the optical elements of the image forming component can include at least a part of the visor portion 402 of the HMD 400. Components such as one or more physical processors, light sources of the image forming component, one or more sensors, and / or other components can be incorporated into the housing portion 404 and / or other parts of the HMD 400.

[0040] Figure 5 FIG. shows the configuration of the light source 504 and the optical element 506 of the image forming component according to one or more implementations. The optical element 506 can include the visor portion of the head-mounted headset and / or can be a part of the visor portion of the head-mounted headset. The light source 504 can be configured to emit a light beam including light rays 508. The light beam can form an image including an image 512. The light beam can be received by the optical element 506 and provided to the user's eye 510. The light rays received by the eye 510 can form virtual content 514 (e.g., a virtual object), which is perceived to be located within a three-dimensional space in the user's field of view or a part thereof.

[0041] Return Figure 1 , one or more sensors 122 of the host device 102 can include one or more of a depth sensor, an image sensor, a position sensor, a heading sensor, and / or other sensors.

[0042] The depth sensor can be configured to generate an output signal that conveys depth information and / or other information. The depth information can include the distance and / or range to a real-world surface and / or object from the depth sensor, and / or other information. In some implementations, the depth information can be provided in the form of a point cloud. The point cloud can include a set of points. Each point can represent an individual surface within the real world. The depth information can specify, for each point, one or more of a single distance of the point to the depth sensor, a single direction of the point relative to the depth sensor, and / or other information. In some implementations, the shape, surface, and / or heading of an object and / or other information can be determined from the depth information. In some implementations, the depth sensor can include one or more of a time-of-flight sensor, a structured light sensor, an unstructured light sensor, an active stereo pair, a passive stereo pair, and / or other depth sensing devices.

[0043] An image sensor can be configured to generate an output signal that conveys view information and / or other information. The view information can specify visual content within the field of view of the image sensor. The visual content can include real-world objects and / or surfaces that exist in the real world. The view information can specify the visual content in the form of pixels in an image. A pixel can be defined by one or more of a position (e.g., two-dimensional coordinates), color, transparency, and / or other information. In some implementations, the image sensor can include one or more of the following: an array of light sensors (e.g., an array of light sites), a charge-coupled device sensor, an active pixel sensor, a complementary metal-oxide-semiconductor sensor, an N-type metal-oxide-semiconductor sensor, and / or other image sensors.

[0044] A position sensor can be configured to generate an output signal that conveys the geographical location of the position sensor. The position sensor can include a Global Positioning System (GPS) and / or other position sensors.

[0045] A heading sensor can be configured to generate an output signal that conveys heading information and / or other information. The heading information can include orientation and / or heading. In some implementations, the heading information can be specified with respect to one or more degrees of freedom. By way of non-limiting illustration, the heading information can specify one or more of the pitch angle, roll angle, and / or yaw angle of the host device 102 over time. The heading sensor can include an Inertial Measurement Unit (IMU) and / or other sensors. In some implementations, the heading can be determined by using image processing techniques and / or other techniques that utilize the output signal that conveys the view information.

[0046] In Figure 1 an implementation, one or more processors 104, one or more image forming components 120, one or more sensors 122, and / or other components of the host device 102 can be operably linked via one or more electronic communication links. For example, such electronic communication links can be established at least in part via the network 118. The network 118 can include one or both of wired and / or wireless communication. It will be understood that this is not intended to be limiting, and the scope of the present disclosure includes implementations in which one or more processors 104, one or more image forming components 120, one or more sensors 122, and / or other components can be operably linked via some other communication medium.

[0047] In Figure 1Among them, the mobile computing platform 126 can be a handheld device. The mobile computing platform 126 can include one or more of a mobile phone (e.g., a smart phone), a tablet computing platform, and / or other devices. The mobile computing platform 126 can include one or more physical processors 128, a display 140, one or more sensors 142, a haptic actuator 144, and / or one or more of other components.

[0048] The display 140 can be configured to present visual content. The visual content can be in the form of an image, a video, text, a graphical user interface, and / or content. The display 140 can include one or more of a display screen, a touch screen display, a projector display, and / or other devices configured to present content. The user interface presented on the display 140 can be configured to facilitate interaction with virtual content in the interactive space (e.g., see the input component 134 and / or Figures 8 - 10 ).

[0049] In some implementations, each sensor of one or more sensors 142 of the mobile computing platform 126 can be the same as or similar to each sensor of one or more sensors 122 of the host device 102. By way of non-limiting illustration, one or more sensors 142 can include one or more of a depth sensor, an image sensor, a position sensor, a heading sensor, and / or other sensors.

[0050] In Figure 1 Among them, one or more physical processors 104 of the host device 102 can be configured by machine-readable instructions 106. Executing the machine-readable instructions 106 can cause one or more physical processors 104 to facilitate providing the mobile computing platform as a physical interface for the interactive space. The machine-readable instructions 106 can include a content component 108, a control component 110, a relative position component 112 (abbreviated as "Rel.Position Component 112" in Figure 1 ), a communication component 114 (abbreviated as "Comm.Component 114" in Figure 1 ), and / or one or more of other components.

[0051] The content component 108 can be configured to obtain information defining virtual content. The information defining virtual content can include information defining one or more of the visual appearance of the virtual content, one or more interaction capabilities of the virtual content, the movement of the virtual content, and / or other information. The visual appearance of the virtual content can include one or more of a two-dimensional representation of the virtual content, a three-dimensional representation of the virtual content, the color of the virtual content, and / or other information. Interaction capabilities can refer to attributes related to the ability to manipulate virtual content within an interactive space with a user. Manipulation can be provided via an input of the mobile computing platform 126. The movement of the virtual content can refer to the motion of the virtual content within the interactive space.

[0052] The control component 110 can be configured to control each of the one or more image forming components 120. Controlling the image forming components 120 can include controlling one or more of one or more light sources, one or more optical elements, and / or other components.

[0053] One or more light sources can be controlled to generate light based on the virtual content and / or other information to be presented to a user of the host device 102. The light sources can generate light to form a digital image that, when directed towards a user's eyes, can be perceived as virtual content within a three-dimensional space in a real-world user view. A viewing user can perceive the virtual content as existing at a location within the real world. In some implementations, a user can interact with the virtual content via the mobile computing platform 126, which may affect one or more aspects of the virtual content.

[0054] In some implementations, the control component 110 can be configured to control each of the one or more image forming components 120 based on one or more of user input information, remote command information, and / or other information (presented in more detail herein).

[0055] In some implementations, the relative position component 112 can be configured to obtain relative position information and / or other information. The relative position component 112 can be configured to obtain relative position information by determining relative position information and / or receiving information from one or more sources. The relative position information can convey the position and / or heading of the mobile computing platform 126 relative to the perceived position and / or heading of the virtual content. The relative position information can include one or more of the distance of the mobile computing platform 126 to the virtual content, the heading of the mobile computing platform 126 relative to the virtual content, and / or other information.

[0056] In some implementations, determining the relative position information may include one or more of the following: determining the main axis of the mobile computing platform 126, determining the main direction of the mobile computing platform 126 relative to the main axis, and / or other operations. In some implementations, the main axis and / or the main direction of the mobile computing platform 126 may indicate the pointing direction of the mobile computing platform 126. For example, the mobile computing platform 126 may act as a pointer to control a cursor for selecting a virtual object. In some implementations, the virtual cursor may be perceived as extending from the main axis along the main direction. The virtual cursor may be perceived as being located at the intersection of the main axis (along the main direction) and the perceived virtual object.

[0057] In some implementations, the main axis may include the longitudinal centerline of the mobile computing platform 126. In some implementations, the main direction may include the direction that can generally be understood as pointing to the "top" of the mobile computing platform 126. For example, the mobile computing platform 126 may include a smart phone and / or other devices. The smart phone and / or other devices may have a traditional orientation when in use. For example, its top can be distinguished from its bottom. The main direction may extend from the bottom to the top.

[0058] By way of non-limiting illustration, Figure 6 a view of an interactive space is shown. The interactive space may include a set of virtual objects, a virtual cursor 600, and / or the mobile computing platform 126 that provides the physical interface of the interactive space. The set of virtual objects may include one or more of a first virtual object 602, a second virtual object 604, a third virtual object 606, and / or other virtual objects. The mobile computing platform 126 may include a display surface 126a and a main axis 608. The dashed line representing the main axis 608 may be an imaginary line and / or may be shown as a virtual object within the interactive space. The main axis 608 may include the longitudinal centerline of the mobile computing platform 126. The cursor 600 may act as an extension of the mobile computing platform 126 and may be perceived as being present at the intersection of the main axis 608 and the first virtual object 602.

[0059] Figure 7 It shows how user interaction with the mobile computing platform 126 can facilitate the selection of other virtual objects such as the second virtual object 604. For example, the mobile computing platform 126 acting as a pointing device may allow the user to point to a desired virtual object to convey the selection of that virtual object. User interaction with the mobile computing platform 126 may include one or more of moving the mobile computing platform 126, rotating the mobile computing platform 126, and / or other interactions.

[0060] Return Figure 1, in some implementations, the relative position component 112 of the host device 102 can be configured to determine relative position information and / or other information. The relative position information can be determined based on output signals from one or more sensors 122 and / or 142 and / or other information sources.

[0061] In some implementations, the relative position component 112 can determine the relative position information based on depth information and / or other information conveyed by the output signals from the depth sensor of the host device 102. The depth information can include a point cloud and / or other information. The presence of the mobile computing platform 126 can be determined from the point cloud based on the recognition of the size and / or shape of the mobile computing platform 126 within the point cloud. The position and / or heading of the mobile computing platform 126 can be determined from the point cloud. By way of non-limiting illustration, one or more detection and / or recognition techniques can be utilized to identify and / or detect the shape and / or form of the mobile computing platform 126 within the point cloud.

[0062] In some implementations, the position and / or heading of the mobile computing platform 126 can be determined relative to one or more of the host device 102, the depth sensor, and / or other reference points. The position and / or heading of the mobile computing platform 126 relative to virtual content perceived to exist in the real world can be determined based on the perceived position and / or heading of the virtual content and the determined position and / or heading of the mobile computing platform 126. For example, the position and / or heading of the mobile computing platform 126 and the perceived position and / or heading of the virtual content can be determined relative to the same reference point (e.g., one or more of the host device 102, one or more sensors 122, and / or other references). Thus, conventional geometric considerations and / or other techniques can be used to determine and / or calculate the position and / or heading of the mobile computing platform 126 relative to the perceived position and / or heading of the virtual content.

[0063] In some implementations, the relative position information can be determined by the relative position component 112 based on view information conveyed by an output signal of an image sensor of the host device 102 and / or an image sensor on a wall or other fixed position in the interactive space (i.e., an "outside-in" external device capable of tracking the position and / or movement of the mobile computing platform 126). The viewing information can define an image and / or video. One or more image processing techniques (such as techniques in computer vision) can be used to determine the presence of the mobile computing platform 126 from the view information. One or more image processing techniques can be used to determine and / or track the position and / or heading of the mobile computing platform 126 relative to one or more references within the image and / or video. The one or more image processing techniques can include object detection, bundle adjustment, and / or other computer vision techniques. Based on the perceived position and / or heading of the virtual content and the determined position and / or heading of the mobile computing platform 126, the position and / or heading of the mobile computing platform 126 relative to the virtual content perceived to exist in the real world can be determined.

[0064] In some implementations, the relative position information can be determined by the relative position component 112 based on information received from the mobile computing platform 126 and / or other information. The information received from the mobile computing platform 126 can include one or more of the following: an output signal conveying heading information from a heading sensor of the mobile computing platform 126, an output signal conveying position information from a position sensor of the mobile computing platform 126, and / or other information. The position sensor can include one or more cameras or image sensors of the mobile computing platform 126, and / or one or more inertial measurement units (IMUs) and / or other position and / or motion-based sensors of the computing platform 126 (i.e., an "inside-out" internal device capable of tracking the position and / or movement of the mobile computing platform 126). The relative position component 112 can utilize the conveyed heading of the mobile computing platform 126 and / or the conveyed position of the mobile computing platform 126 to determine one or more of the heading of the mobile computing platform 126 relative to the perceived heading of the virtual content, the position of the mobile computing platform 126 relative to the perceived position of the virtual content, and / or other information.

[0065] The communication component 114 can be configured to establish a connection between the host device 102 and the mobile computing platform 126. The connection can be established through the network 124 and / or other communication networks. The connection can include a wired and / or wireless connection.

[0066] The communication component 114 can be configured to enable information communication to and / or from the mobile computing platform 126.

[0067] The control component 110 can be configured to obtain user input information and / or other information. The user input information can convey user entry and / or selection of one or more input elements of the mobile computing platform. The user entry and / or selection of one or more input elements can include a part of the user interaction with the mobile computing platform 126, which can facilitate interaction with virtual content in the interactive space. The user input information can be received and / or retrieved from the mobile computing platform 126. The user input information can be determined by the input component 134 of the mobile computing platform 126. In some implementations, the user input information can include interactions and / or actions that can be applied to the selected virtual content. By way of non-limiting illustration, each input element can correspond to each interaction and / or action within the interactive space. The interactions can include one or more of moving, scaling, rotating, changing the texture / color of the virtual object, and / or other types of interactions. The actions can include one or more of the following: resetting the map of the real-world environment, surface scanning, and / or other actions. The map can be provided by simultaneous localization and mapping (SLAM) computations that construct and / or update the map of the environment while keeping track of the position of the host device 102. Surface scanning can refer to scanning a surface to understand the topography of the surface and / or other information about the surface. Other interactions and / or actions can be performed.

[0068] The control component 110 can be configured to determine remote command information and / or other information. The remote command information can be configured to enable user interaction with virtual content in the interactive space based on user interaction with the mobile computing platform 126. In some implementations, the remote command information can include information that conveys selection of virtual content (e.g., via pointing at the mobile computing platform 126), interactions and / or actions applied to the selected virtual content (e.g., via user entry and / or selection of input elements of the mobile computing platform 126), instructions to control one or more image forming components 120 to reflect the interactions and / or actions applied to the selected virtual content, and / or other information. The remote command information can be determined based on one or more of relative position information, user input information, and / or other information. For example, it will be apparent to those of ordinary skill in the art that one or more of the relative position component 112 and the control component 110 can employ any combination of "outside-in" external sensor devices and "inside-out" internal sensor devices to determine relative position information and / or remote command information, respectively.

[0069] In some implementations, determining remote command information can include determining a perceived selection of virtual content based on relative position information and / or other information, such as a single virtual object and / or a group of virtual objects. In some implementations, the selection of virtual content can be determined based on one or more of the principal axis of the mobile computing platform 126, the principal direction of the mobile computing platform 126 relative to the principal axis, virtual content perceived to intersect the principal axis along the principal direction, and / or other information. In some implementations, virtual content perceived to intersect the principal axis along the principal direction can convey a selection of the virtual content.

[0070] In some implementations, the selection of virtual content can be facilitated by a virtual cursor. The virtual cursor can provide a visual indication of where the mobile computing platform 126 can be pointed relative to the principal direction of the principal axis. The control component 110 can be configured to control one or more image forming components 120 to generate light rays that form an image of the virtual cursor. The image of the virtual cursor can be configured such that the virtual cursor can be perceived to be located at the perceived intersection of the principal axis along the principal direction and the virtual content. The image of the virtual cursor can be configured such that the virtual cursor can track with the movement of the mobile computing platform 126.

[0071] In some implementations, determining remote command information can include determining an interaction with a selected virtual object in the interactive space and / or an action performed in the interactive space based on user entry and / or selection of one or more input elements of the mobile computing platform 126.

[0072] The control component 110 can be configured to control one or more image forming components 120 based on the remote command information and / or other information. One or more image forming components 120 can be controlled to cause the image of the virtual content to reflect the user interaction with the virtual content.

[0073] In Figure 1 this regard, one or more physical processors 128 of the mobile computing platform 126 can be configured by machine-readable instructions 130. Executing the machine-readable instructions 130 can cause the one or more physical processors 128 to facilitate providing the mobile computing platform as a physical interface to the interactive space. The machine-readable instructions 130 can include one or more of a communication component 132 (abbreviated as "Comm.Component 132" in Figure 1 this regard), an input component 134, and / or other components.

[0074] The communication component 132 can be configured to establish a connection between the host device 102 and the mobile computing platform 126. The connection can be established via the network 124 and / or other communication networks. The connection can include a wired and / or wireless connection.

[0075] The communication component 132 can be configured to implement information communication to and / or from the host device 102.

[0076] The input component 134 can be configured to obtain output signals from respective ones of the one or more sensors 142. By way of non-limiting illustration, the input component 134 can be configured to obtain output signals from the one or more sensors 142 that convey location information, heading information, and / or other information. The location information can convey the location of the mobile computing platform 126. The heading information can convey the heading of the mobile computing platform 126.

[0077] The input component 134 can be configured to obtain user input information. The user input information can be obtained by determining user input information based on user entry and / or selection of one or more input elements of the mobile computing platform 126. The user input information can convey user entry and / or selection of one or more input elements of the mobile computing platform 126. The user entry and / or selection of the one or more input elements can include a portion of the user interaction with the mobile computing platform 126 for determining interaction with virtual content.

[0078] In some implementations, the input component 134 can be configured to implement the presentation of one or more graphical user interfaces on the display 140 of the mobile computing platform 126. The user interface can include one or more input elements configured for user entry and / or selection. A single input element can correspond to one or more interactions with a virtual object and / or an action within the interactive space. By way of non-limiting illustration, a first input element can correspond to performing a color change, a second input element can correspond to performing a zoom operation, a third input element can correspond to providing text input (e.g., a keyboard), and / or other input elements can correspond to other interactions. A single input element can include one or more of a virtual button, a slider, a checkbox, a drop-down menu, a keyboard, a text input bar, and / or other elements. Once a virtual object is selected (e.g., by pointing at the mobile computing platform 126), one or more input elements can be selected to implement one or more interactions with the selected virtual object.

[0079] Figures 8 - 10 An example of a user interface presented on the display of a mobile computing platform configured to have a physical interface for an interactive space is shown. A single user interface can include one or more input elements.

[0080] In Figure 8In [reference], a user interface 800 is shown. The user interface 800 may include a grid of user interface elements. The grid of user interface elements may include one or more of a first user interface element 802, a second user interface element 804, a third user interface element 806, a fourth user interface element 808, and / or other user interface elements. A single user interface element may be formed as a square virtual button, and / or may have other shapes.

[0081] In Figure 9 In [reference], a user interface 900 is shown. The user interface 900 may be one or more of a first user interface element 902, a second user interface element 904, a third user interface element 906, and / or other user interface elements. The first user interface element 902 and / or the second user interface element 904 may be formed as circular virtual buttons and / or may have other shapes. The third user interface element 906 may be a slider user interface element. The third user interface element 906 may include a virtual button that may slide along a slider bar 908. Interaction with the third user interface element 906 may provide an increase and / or decrease in aspects of a virtual object. By way of non-limiting illustration, the slider mechanism may facilitate increasing and / or decreasing the size (or scale), brightness, hue, and / or other aspects of a virtual object.

[0082] In Figure 10 In [reference], a user interface 1000 is shown. The user interface 1000 may include one or more of a first user interface element 1002, a second user interface element 1004, and / or other user interface elements. The first user interface element 1002 may include a text viewing window. The second user interface element 1004 may include a keyboard. Inputs in the second user interface element 1004 may be reflected on a selected virtual object and / or within the first user interface element 1002 as a reference.

[0083] Return Figure 1, the host device 102, one or more other host devices 123, the mobile computing platform 126, one or more other mobile computing platforms 121, the external resource 125, and / or other components may be operably linked via one or more electronic communication links. For example, such electronic communication links may be established at least in part via the network 124. The network 124 may include one or both of wired and / or wireless communication (e.g., Bluetooth, Wi-Fi, etc.). It will be understood that this is not intended to be limiting, and the scope of the present disclosure includes implementations in which one or more host devices 102, one or more other host devices 123, the mobile computing platform 126, one or more other mobile computing platforms 121, the external resource 125, and / or other components may be operably linked via some other communication medium. In some implementations, the network 118 may be the same as the network 124, or the networks may be separate and different networks.

[0084] The external resource 125 may include an information source, a host, an external entity participating in the system 100, a provider of virtual content, and / or other resources. In some implementations, some or all of the functions attributed herein to the external resource 125 may be provided by resources included in the system 100.

[0085] The processor 104 may include and / or may have access to the electronic storage device 116 and / or other components. The processor 104 may include communication lines or ports to enable the exchange of information with the network and / or other components of the system 100. Figure 1 The illustration of the processor 104 in is not intended to be limiting. The processor 104 may include multiple hardware, software, and / or firmware components that operate together to provide the functions attributed herein to the processor 104. For example, the processor 104 may be implemented by a cloud of computing platforms operating as the processor 104.

[0086] The electronic storage device 116 may include a non-transitory electronic storage medium that stores information electronically. The electronic storage device 116 may store the machine-readable instructions 106. The electronic storage medium of the electronic storage device 116 may include one or both of a system storage device provided integrally (i.e., substantially immovably) with the processor 104 and a removable storage device removably connected to the processor 104 via, for example, a port or a drive. The port may include a USB port, a FIREWIRE port, and / or other ports. The drive may include a disk drive and / or other drives. The electronic storage device 116 may include one or more of an optically readable storage medium (e.g., an optical disc, etc.), a magnetically readable storage medium (e.g., a magnetic tape, a magnetic hard disk drive, a floppy disk drive, etc.), a charge-based storage medium (e.g., an EEPROM, a RAM, etc.), a solid-state storage medium (e.g., a flash drive, etc.), and / or other electronically readable storage media. The electronic storage device 116 may include one or more virtual storage resources (e.g., cloud storage, a virtual private network, and / or other virtual storage resources). The electronic storage device 116 may store software algorithms, information determined by the processor 104, information received from other components of the system 100, and / or other information that enables the processor 104 to function as described herein.

[0087] The processor 104 is configured to provide information processing capabilities for the host device 102. Thus, the processor 104 may include one or more of a physical processor, a digital processor, an analog processor, a digital circuit designed to process information, an analog circuit designed to process information, a state machine, and / or other mechanisms for electronically processing information. Although the processor 104 is shown as a single entity in Figure 1 for illustrative purposes only. In some implementations, the processor 104 may include one or more processing units. These processing units may be physically located within the same device, or the processor 104 may represent the processing functions of multiple devices operating in cooperation. The processor 104 may be configured to execute components 108, 110, 112, 114, and / or other components. The processor 104 may be configured to execute components 108, 110, 112, 114, and / or other components by: software; hardware; firmware; some combination of software, hardware, and / or firmware; and / or other mechanisms for configuring processing capabilities on the processor 104.

[0088] It should be understood that although components 108, 110, 112, and 114 are shown in Figure 1are shown as co - located within a single processing unit, but in implementations where processor 104 includes multiple processing units, one or more components may be remote from other components. The description of the functionality provided by a single component is for illustrative purposes and is not intended to be limiting, as a given component may provide more or less functionality than described. For example, a given component may be eliminated and some or all of its functionality may be provided by another component. As another example, processor 104 may be configured to execute one or more additional components that may perform some or all of the functionality attributed to each of components 108, 110, 112, 114, and / or other components.

[0089] The processor 128 of the mobile computing platform 126 may include and / or may access an electronic storage device 138 and / or other components. The processor 128 may include communication lines or ports to enable the exchange of information with the network of system 100 and / or other components. Figure 1 The illustration of the processor 128 in is not intended to be restrictive. The processor 128 may include multiple hardware, software, and / or firmware components that operate together to provide the functionality attributed to the processor 128 herein. For example, the processor 128 may be implemented by a cloud of computing platforms that operate as the processor 128.

[0090] The electronic storage device 138 may include a non - transient electronic storage medium that stores information electronically. By way of non - limiting illustration, the electronic storage device 138 may store machine - readable instructions 130. The electronic storage medium of the electronic storage device 138 may include one or both of a system storage device provided integrally (i.e., substantially immovable) with the processor 128 and / or a removable storage device removably connected to the processor 128 via, for example, a port or a drive. The port may include a USB port, a FIREWIRE port, and / or other ports. The drive may include a disk drive and / or other drives. The electronic storage device 138 may include one or more of an optically readable storage medium (e.g., optical disc, etc.), a magnetically readable storage medium (e.g., magnetic tape, magnetic hard disk drive, floppy disk drive, etc.), a charge - based storage medium (e.g., EEPROM, RAM, etc.), a solid - state storage medium (e.g., flash drive, etc.), and / or other electronically readable storage media. The electronic storage device 138 may include one or more virtual storage resources (e.g., cloud storage, virtual private network, and / or other virtual storage resources). The electronic storage device 138 may store software algorithms, information determined by the processor 128, information received from other components of the system 100, and / or other information that enables the processor 128 to function as described herein.

[0091] The processor 128 is configured to provide information processing capabilities in the mobile computing platform 126. As such, the processor 128 can include one or more of a physical processor, a digital processor, an analog processor, digital circuitry designed to process information, analog circuitry designed to process information, a state machine, and / or other mechanisms for electronically processing information. Although the processor 128 is shown as a single entity in Figure 1 , this is for illustrative purposes only. In some implementations, the processor 128 can include one or more processing units. These processing units can be physically located within the same device, or the processor 128 can represent the processing functionality of multiple devices operating in concert. The processor 128 can be configured to execute components 132, 134, and / or other components. The processor 128 can be configured to execute components 132, 134, and / or other components by: software; hardware; firmware; some combination of software, hardware, and / or firmware; and / or other mechanisms for configuring processing capabilities on the processor 128.

[0092] It should be understood that although components 132 and / or 134 are shown as co-located within a single processing unit in Figure 1 , in implementations where the processor 128 includes multiple processing units, one or more components can be remote from other components. The description of the functionality provided by a single component is for illustrative purposes and is not intended to be limiting, as a given component can provide more or less functionality than described. For example, a given component can be eliminated, and some or all of its functionality can be provided by another component. As another example, the processor 128 can be configured to execute one or more additional components that can perform some or all of the functionality attributed to each of components 132, 134, and / or other components.

[0093] Figure 2 and Figure 3 respectively illustrate methods 200 and 300 for providing a mobile computing platform as a physical interface to an interactive space according to one or more implementations. The operations of each of the methods 200 and / or 300 presented below are intended to be illustrative. In some implementations, methods 200 and / or 300 can be implemented with one or more additional operations not described and / or one or more operations not discussed. Additionally, in Figure 2 and Figure 3 , the order of operations of methods 200 and 300 is shown respectively, and the order described below is not intended to be limiting.

[0094] In some implementations, methods 200 and / or 300 can be in such as Figure 1implemented in and / or using the system 100 shown and described herein. By way of non-limiting illustration, method 200 may be implemented by one or more physical processes that are the same as or similar to one or more physical processors 104 of the host device 102 in Figure 1 By way of non-limiting illustration, method 300 may be implemented by one or more physical processors that are the same as or similar to one or more physical processors 128 of the mobile computing platform 126 in Figure 1 The one or more physical processors may include one or more devices that respectively perform one or more operations of method 200 and / or method 300 in response to instructions electronically stored on an electronic storage medium. The one or more processing devices may include one or more devices configured by hardware, firmware, and / or software specifically designed to perform one or more operations of method 200 and / or method 300.

[0095] Referring Figure 2 to method 200 in, at operation 202, a wireless connection may be established between the device and the mobile computing platform. The mobile computing platform may be separate and distinct from the device. The device may be configured to be worn on a user's head. The device may include one or more image forming components and / or other components. The one or more image forming components may be configured to generate light to form an image of virtual content. The one or more image forming components may be configured to present the image to the user and superimpose the image on a view of the real world to create an interactive space. The virtual content may be perceived as existing in the real world. User interaction with the mobile computing platform may facilitate user interaction with the virtual content in the interactive space. In some implementations, operation 202 may be performed by one or more physical processors that perform a communication component that is the same as or similar to communication component 114 (shown in Figure 1 and described herein).

[0096] At operation 204, relative position information may be determined. The relative position information may convey the position and / or heading of the mobile computing platform relative to the perceived position and / or heading of the virtual content. In some implementations, operation 204 may be performed by one or more physical processors that perform a relative position component that is the same as or similar to relative position component 112 (shown in Figure 1 and described herein).

[0097] At operation 206, user input information can be obtained. The user input information can convey user entry and / or selection of one or more input elements of the mobile computing platform. The user entry and / or selection of one or more input elements can include a part of the user interaction with the mobile computing platform. In some implementations, operation 206 can be performed by one or more physical processors that perform a communication component that is the same as or similar to communication component 114 (shown in Figure 1 and described herein).

[0098] At operation 208, remote command information can be determined. The remote command information can be determined based on one or more of relative position information, user input information, and / or other information. The remote command information can be configured to enable user interaction with virtual content in an interactive space based on the user interaction with the mobile computing platform. In some implementations, operation 208 can be performed by one or more physical processors that perform a control component that is the same as or similar to control component 110 (shown in Figure 1 and described herein).

[0099] At operation 210, one or more image forming components can be controlled based on the remote command information and / or other information. By way of non-limiting illustration, one or more image forming components can be controlled so that the image of the virtual content reflects the user interaction with the virtual content. In some implementations, operation 210 can be performed by one or more physical processors that perform a control component that is the same as or similar to control component 110 (shown in Figure 1 and described herein).

[0100] Referring to Figure 3 method 300 therein, at operation 302, a wireless connection can be established between the device and the mobile computing platform. The mobile computing platform can be separate and different from the device. The device can be configured to be worn on the user's head. The device can include one or more image forming components and / or other components. The one or more image forming components can be configured to generate light to form an image of the virtual content. The one or more image forming components can be configured to present the image to the user and superimpose the image on a view of the real world to create an interactive space. The virtual content can be perceived as existing in the real world. The user interaction with the mobile computing platform can facilitate the user interaction with the virtual content in the interactive space. In some implementations, operation 302 can be performed by one or more physical processors that perform a communication component that is the same as or similar to communication component 132 (shown in Figure 1 and described herein).

[0101] When virtual content is superimposed on the display 140 of the mobile computing platform 126 while the display is in an idle state and the color is black, the display will essentially appear invisible to the user (appearing like a piece of glass or a window into the virtual world). On the other hand, when an image is rendered on the display 140 in a non-black color (or fixed on other parts of the mobile computing platform 126), these elements can enhance the user's perception of the virtual content (e.g., by creating 2D floating elements and / or 3D elements in the virtual space). This shared or composite rendering of virtual content can, for example, advantageously integrate the high-resolution and tactile features of the mobile computing platform with the extended field of view (FOV) provided in the interactive space generated by the host device 102.

[0102] At operation 304, an output signal that conveys location information and / or heading information can be obtained. The location information and / or heading information can convey the location and / or heading of the mobile computing platform. In some implementations, operation 304 can be performed by one or more physical processors that execute an input component that is the same as or similar to the input component 134 (shown in Figure 1 and described herein).

[0103] At operation 306, user input information can be obtained. The user input information can convey user entry and / or selection of one or more input elements of the mobile computing platform. The user entry and / or selection of one or more input elements can include a part of the user interaction with the mobile computing platform. In some implementations, operation 306 can be performed by one or more physical processors that execute an input component that is the same as or similar to the input component 134 (shown in Figure 1 and described herein).

[0104] At operation 308, communication of the output signal, the user input information, and / or other information to the device can be implemented. The device can utilize the output signal, the user input information, and / or other information to control one or more image forming components so that the image of the virtual content reflects the user interaction with the virtual content. In some implementations, operation 308 can be performed by one or more physical processors that execute a communication component that is the same as or similar to the communication component 132 (shown in Figure 1 and described herein).

[0105] Referring again to Figure 6 and Figure 7, it should be readily understood that the principal axis 608 of the mobile computing platform 126 extending along the centerline from the bottom to the top of the mobile computing platform 126 is for illustrative purposes only, and the principal axis according to the present disclosure may extend along or through any axis that intersects the mobile computing device 126 (including within a plane of any outer surface of the mobile computing platform 126 or the volume therebetween or perpendicular to any outer surface or the volume therebetween). For example, a suitable principal axis may extend perpendicular to the display surface 126a of the mobile computing platform 126, and the heading may be, for example, in a direction extending outward from the surface opposite the display surface 126a of the mobile computing platform 126, so that the user can point to or interact with virtual objects by orienting the mobile computing platform 126 (such as a mobile phone) in an orientation conventionally used to capture video or images with the image sensor 142, and the image sensor is disposed on the surface opposite the display surface 126a in such mobile computing platforms 126.

[0106] In addition, for Figures 8 - 10 , it should be readily understood that user interface elements (such as interface elements 80X, 90X, 100X) presented on the display 140 of the mobile computing platform 126 may facilitate user interaction with virtual content projected in the interactive space, and the virtual content may be generated by, for example, a software application or application software stored in the memory 138 and executed by the processor 128 of such mobile computing platform 126 ( Figure 1 as shown). Such software applications or application software may also be used alone or in combination with previously displayed interface elements selected, deselected, activated, and / or deactivated by the user, for example, to depict different selectable interface elements, such as elements 80X, 90X, 100X, based on motion and / or changes in the orientation or heading of such mobile computing platform 126 relative to the position of one or more virtual objects in the interactive space.

[0107] Referring to Figure 1 , the processor 128 of the mobile computing platform 126 may also, based on the occurrence of a specific event, including for example when the principal axis 608 of the mobile computing platform 126 is in a heading or orientation to select or deselect a virtual object displayed in the interactive space, such as Figure 7 the virtual object 604 depicted in, to confirm the selection or deselection of such virtual object 604, or provide an alert regarding remote command information or the occurrence of other events, control the haptic actuator 144 to selectively vibrate or provide other types of haptic feedback to the user.

[0108] In addition, if the mobile computing platform 126 includes Figure 1For the image sensor 142, it is advantageous to use the image sensor 142 to facilitate the determination of the physical orientation or heading based on the processing of the captured images of the real-world environment at a known position relative to the mobile computing platform 126 or otherwise. The processor 128 can be employed to use such captured images either alone or in combination with other techniques for determining the physical orientation or heading, including, for example, gyroscopes or angle determination techniques.

[0109] Figure 11 Illustrates an exemplary operation of the mobile computing platform 126 for causing the selective association of specific graphical information 1101, such as alphanumeric, emoji, image, or video, as a virtual image with an object, such as the virtual object 602. Such an association can be represented, for example, by the projection of a corresponding guide line 1110. According to one exemplary operation for generating and associating a graphical image with an object (e.g., the virtual object 602), the user will position the mobile computing platform 126 to make a selection using, for example, the virtual cursor 600 according to one of the object selection techniques described herein or otherwise. The user further inputs graphical image information of the mobile computing platform 126 through, for example, the interface element 1120 displayed on the display 126a, and such input information indicates the graphical image that the user desires to associate with the virtual object 602. After the entry and selection of the information of the object associated with the corresponding graphical information, the user selects an appropriate interface element 1130 to perform the association step, where the graphical information 1101 is projected as a virtual image in the interactive space near the virtual object 602, preferably with a guide line 1110 projected therebetween. According to the present disclosure, alternatively, it is possible to associate the graphical image with a real-world object (not shown) instead of the virtual object 602 located within the interactive space. Additionally, the graphical image 1101 can be associated with the intended object without using the guide line 1110 by using, for example, a dashed line, a corresponding line projection, or the same or complementary color of the graphical image (or at least the border surrounding a portion of the graphical image) and / or the associated object image (or the border surrounding at least a portion of the object), or the use of some other association indicator.

[0110] Figure 12a and Figure 12b Illustrates another exemplary operation of the mobile computing platform 126 for causing the generation of specific graphical information 1201 as a virtual image that can be selectively associated with an object, such as Figure 12a and Figure 12b the virtual object 602 as shown in. The association can be determined, for example, by determining that the distance between the virtual object 602 and the mobile computing platform 126 is below a predetermined threshold.

[0111] As Figure 12bAs shown, the association can be represented, for example, by projecting a corresponding virtual guide line 1210 between the virtual object 602 and the graphical information 1201. In this operation, as shown in Figure 12a , the mobile computing platform 126 is not used to project the graphical image 1201 to the position indicated by the virtual cursor 600 as shown in Figure 11 , but is positioned by the user such that the position of the graphical information 1201 on the display surface 126a is consistent with the position where the graphical information 1201 appears in the interactive space. Once the mobile computing platform 126 is positioned to place the graphical information 1201 at this position, the user can select the interface element 1230 as shown in Figure 12a , and then withdraw the mobile computing platform 126 as shown in Figure 12b to generate the virtual graphical information 1201 at the desired position together with the guide line 1210. Although the graphical information 1210 is shown as having a solid rectangular border, it should be understood that the graphical information 1210 can be generated in any of a variety of border shapes and shadings, or may be generated without a border.

[0112] In addition, in the present disclosure, the reference to wireless communication, for example, with respect to step 302 in Figure 3 is intended to represent direct or indirect wireless communication between the first and second components. By indirect wireless communication is meant that the communication between the first and second components is via one or more intermediate components, where at least one communication path between any such first, second, or intermediate components is by wireless communication, and the other communication paths may be wired communication paths.

[0113] Although the disclosed technology has been described in detail for purposes of illustration based on the currently considered most practical and preferred implementations, it should be understood that such details are for that purpose only and the present disclosure is not limited to any particular implementation. Instead, it is intended to cover modifications and equivalent arrangements within the spirit and scope of the appended claims. For example, it should be understood that the present disclosure contemplates that, to the extent possible, one or more features of any embodiment can be combined with one or more features of any other implementation.

Claims

1. A system configured to provide a mobile computing platform as a physical interface to an interactive space, the system comprising: a device configured to be worn on a user's head; one or more image forming components held by the device, the one or more image forming components configured to generate light to form an image of virtual content, the one or more image forming components configured to present the image to the user and superimpose the image on a view of the real-world environment to create an interactive space such that the virtual content is perceived to exist in the real-world environment, the virtual content including one or more virtual objects; one or more physical processors configured by machine-readable instructions to: establish a wireless connection between the device and a mobile computing platform that is separate and distinct from the device, wherein user interaction with the mobile computing platform facilitates user interaction with the virtual content in the interactive space; obtain relative position information that conveys the position of the mobile computing platform relative to the perceived position of the virtual content, wherein the relative position information is determined at least in part based on the pointing direction of the mobile computing platform indicated by one or more of a main axis of the mobile computing platform or a main direction relative to the main axis; obtain user input information that conveys user interaction with one or more input elements of the mobile computing platform; determine remote command information based on the relative position information and the user input information, the remote command information configured to effect the user interaction with the virtual content in the interactive space; and control the one or more image forming components based on the remote command information such that the one or more image forming components are controlled to cause the image of the virtual content to reflect the user interaction with the virtual content, and the one or more image forming components are controlled to generate light that forms an image of a virtual cursor, the image of the virtual cursor being configured such that the virtual cursor is perceived to be located at the intersection of the main axis and the perceived virtual object, wherein the virtual cursor serves as an extension of the mobile computing platform.

2. The system of claim 1, wherein determining the remote command information based on the relative position information and the user input information includes: determining a perceived selection of a virtual object based on the relative position information; and determining an interaction with the selected virtual object in the interactive space based on the user interaction with the one or more input elements of the mobile computing platform.

3. The system of claim 1, wherein determining the relative position information includes: determining the main axis of the mobile computing platform; and determining the main direction of the mobile computing platform relative to the main axis.

4. The system of claim 3, wherein determining the remote command information includes: identifying virtual content perceived to intersect the main axis along the main direction.

5. The system of claim 3, wherein The spindle extends perpendicular to the display surface of the mobile computing platform.

6. The system according to claim 4, wherein, the recognition that the virtual content perceived to intersect the spindle along the main direction conveys the user's selection of the virtual content.

7. The system according to claim 1, wherein, one or more input elements of the mobile computing platform include one or both of a physical element or a virtual element, and the virtual element is presented on a display of the mobile computing platform.

8. The system according to claim 7, wherein, each input element corresponds to an interaction with the virtual content.

9. The system according to claim 1, wherein, the interactive space includes an augmented reality space.

10. The system according to claim 1, wherein, a view superimposed on the virtual content is presented by the display.

11. The system according to claim 1, wherein, the one or more physical processors are further configured by machine-readable instructions to: identify graphical information to be associated with a virtual object of the virtual content, wherein the graphical information is input by interaction with the user of the mobile computing platform; and present the graphical information as a virtual image in the interactive space and proximate to the virtual object.

12. The system according to claim 11, wherein, the one or more physical processors are further configured to: visually associate the virtual image with the virtual object by a guiding line.

13. The system according to claim 1, wherein, the virtual cursor being perceived as located at the perceived intersection of the spindle and the virtual object conveys the user's selection of the virtual object.

14. The system according to claim 3, wherein, the mobile computing platform further includes an image sensor communicatively coupled to the one or more physical processors, and wherein the image sensor facilitates the one or more physical processors to determine the main direction of the mobile computing platform relative to the spindle.

15. The system according to claim 3, wherein, the mobile computing platform further includes a haptic feedback actuator communicatively coupled to the one or more physical processors, and wherein the one or more physical processors are configured to control the haptic feedback actuator to alert the user of the occurrence of the event in combination with at least one of remote command information or the position of the mobile computing platform relative to the displayed virtual content.

16. A method of providing a physical interface of a mobile computing platform as an interactive space, the method comprises: Establish a wireless connection between a device and a mobile computing platform that is separate from and different from the device, the device being configured to be worn on a user's head, the device including one or more image forming components configured to generate light to form an image of virtual content, the one or more image forming components being configured to present the image to the user and superimpose the image on a view of the real-world environment to create an interactive space such that the virtual content is perceived to exist in the real-world environment, wherein user interaction with the mobile computing platform facilitates user interaction with the virtual content in the interactive space; Obtain relative position information that conveys the position of the mobile computing platform relative to the perceived position of the virtual content; Obtain user input information that conveys user interaction with one or more input elements of the mobile computing platform; Determine remote command information based on the relative position information and the user input information, the remote command information being configured to effect the user interaction with the virtual content in the interactive space; and Control the one or more image forming components based on the remote command information such that the one or more image forming components are controlled to cause the image of the virtual content to reflect the user interaction with the virtual content, wherein determining the remote command information based on the relative position information and the user input information includes: Based on the relative position information, determine a perceived selection of a virtual object; and Based on the user interaction with the one or more input elements of the mobile computing platform, determine an interaction with the selected virtual object in the interactive space, and Based on the distance between the virtual object and the mobile computing platform being less than a predetermined threshold, determine that the virtual object is associated with the virtual content.

17. The method according to claim 16, wherein, The view superimposed with the virtual content is presented by a display.

18. The method according to claim 16, wherein, Determining the relative position information includes: Determine the main axis of the mobile computing platform; and Determine the main direction of the mobile computing platform relative to the main axis.

19. The method according to claim 18, wherein, Determining the remote command information includes: Identify virtual content that is perceived to intersect the main axis along the main direction.

20. The method according to claim 19, further including: Control the one or more image forming components to generate light that forms an image of a virtual cursor, the image of the virtual cursor being configured such that the virtual cursor is perceived to be located at the intersection of the main axis and the perceived position of the virtual object.

21. The method according to claim 19, wherein, The identification of virtual content that is perceived to intersect the main axis along the main direction conveys the user's selection of the virtual content.

22. The method according to claim 16, wherein, One or more input elements of the mobile computing platform include one or both of physical elements or virtual elements, and the virtual elements are presented on a display of the mobile computing platform.

23. The method according to claim 16, wherein, the main axis extends perpendicular to a display surface of the mobile computing platform.

24. The method according to claim 23, wherein, determining the remote command information includes: identifying virtual content perceived to intersect the main axis along the main direction.

25. The method according to claim 24, further including: controlling the one or more image forming components to generate light rays forming an image of a virtual cursor, the image of the virtual cursor being configured such that the virtual cursor is perceived to be located at the perceived intersection.

26. The method according to claim 25, wherein, the identification conveys the user's selection of the virtual content with respect to the virtual content perceived to intersect the main axis along the main direction.

27. The method according to claim 25, wherein, the virtual cursor acts as an extension of the mobile computing platform.

28. The method according to claim 18, wherein, the relative position information is determined at least in part by image data generated by an image sensor disposed within the mobile computing platform.

29. The method according to claim 18, further including the steps of: generating haptic feedback to a user of the mobile computing platform to alert the user of the occurrence of the event in combination with at least one of the remote command information or the position of the mobile computing platform relative to the displayed virtual content.

30. The method according to claim 16, further including: identifying graphic information to be associated with the virtual content; presenting the graphic information as virtual content in the interactive space; and visually associating the virtual graphic information with the virtual object.

31. The method according to claim 30, wherein, the visual association is provided by virtual guide lines.

32. The method according to claim 30, wherein, the graphic information is displayed on the mobile computing platform and is thereby positioned in the interactive space adjacent to its position on the display of the mobile computing platform.

33. A system configured to provide a mobile computing platform as a physical interface to an interactive space, the system comprising: a device configured to be worn on a user's head; one or more image forming components held by the device, the one or more image forming components being configured to generate light rays to form an image of virtual content, the one or more image forming components being configured to present the image to the user and superimpose the image on a view of the real-world environment to create an interactive space such that the virtual content is perceived to exist in the real-world environment, the virtual content including one or more virtual objects; One or more physical processors configured by machine-readable instructions to: establish a wireless connection between the device and a mobile computing platform separate and distinct from the device, wherein user interaction with the mobile computing platform facilitates user interaction with the virtual content in the interactive space; Obtain relative position information that conveys the position of the mobile computing platform relative to the perceived position of the virtual content; Obtain user input information that conveys user interaction with one or more input elements of the mobile computing platform; Determine remote command information based on the relative position information and the user input information, the remote command information being configured to effect the user interaction with the virtual content in the interactive space; and Control the one or more image forming components based on the remote command information such that the one or more image forming components are controlled to cause the image of the virtual content to reflect the user interaction with the virtual content, wherein determining the remote command information based on the relative position information and the user input information includes: Based on the relative position information, determine a perceived selection of a virtual object; and Based on the user interaction with the one or more input elements of the mobile computing platform, determine an interaction with the selected virtual object in the interactive space, and Based on the distance between the virtual object and the mobile computing platform being less than a predetermined threshold, determine that the virtual object is associated with the virtual content.

34. A non-transitory electronic storage medium storing machine-readable instructions that, when executed by a mobile computing platform, cause the mobile computing platform to: Establish a wireless connection between the mobile computing platform and a device configured to be worn on a user's head, the device including one or more image forming components configured to generate light to form an image of virtual content, the one or more image forming components being configured to present the image to the user and overlay the image on a view of the real-world environment to create an interactive space such that the virtual content is perceived to exist in the real-world environment, wherein, User interaction with the mobile computing platform facilitates user interaction with the virtual content in the interactive space; Obtain an output signal that conveys position information that conveys the position of the mobile computing platform; Obtain user input information that conveys user interaction with one or more input elements of the mobile computing platform; and Effect communication of the output signal and the user input information to the device, wherein the device utilizes the output signal and the user input information to control the one or more image forming components to cause the image of the virtual content to reflect the user interaction with the virtual content, wherein determining the remote command information based on the relative position information and the user input information includes: Based on the relative position information, determine a perceived selection of a virtual object; and Determine an interaction with a selected virtual object in the interactive space based on the user interaction with the one or more input elements of the mobile computing platform, and Determine that the virtual object and the virtual content are associated based on the distance between the virtual object and the mobile computing platform being less than a predetermined threshold.

35. The non-transitory electronic storage medium according to claim 34, wherein, A view superimposed with the virtual content is presented by a display.

36. The non-transitory electronic storage medium according to claim 34, wherein, The method further includes: Implement the presentation of the one or more input elements on a display of the mobile computing platform.

37. The non-transitory electronic storage medium according to claim 34, wherein, The output signal that conveys the position information is generated by one or more sensors included in the mobile computing platform.

38. The non-transitory electronic storage medium according to claim 34, wherein, The mobile computing platform is a mobile phone or a tablet computing platform.

39. The non-transitory electronic storage medium according to claim 34, further including: Identify graphic information to be associated with the virtual content; Present the graphic information as virtual content in the interactive space; and Visually associate the virtual graphic information with the virtual object.

40. The non-transitory electronic storage medium according to claim 39, wherein, The visual association is provided by a virtual guide line.

41. The non-transitory electronic storage medium according to claim 39, wherein, The graphic information is displayed on the mobile computing platform and is thus positioned in the interactive space adjacent to its position on the display of the mobile computing platform.

42. A method for providing a mobile computing platform as a physical interface for an interactive space, the method includes: Establish a wireless connection between a device and the mobile computing platform that is separate and different from the device, wherein the device is configured to be worn on a user's head, the device includes one or more image forming components configured to generate light to form an image of virtual content, the one or more image forming components are configured to present the image to the user and superimpose the image on a view of the real-world environment to create an interactive space such that the virtual content is perceived to exist in the real-world environment, wherein user interaction with the mobile computing platform facilitates user interaction with the virtual content in the interactive space; Obtain relative position information that conveys the position of the mobile computing platform relative to the perceived position of the virtual content, wherein the relative position information is determined at least in part based on an indication of the pointing direction of the mobile computing platform by one or more of the principal axis of the mobile computing platform or the principal direction relative to the principal axis; Obtain user input information that conveys user interaction with one or more input elements of the mobile computing platform. Determine remote command information based on the relative position information and the user input information, the remote command information being configured to implement the user interaction with the virtual content in the interactive space; and Control the one or more image forming components based on the remote command information such that the one or more image forming components are controlled to cause the image of the virtual content to reflect the user interaction with the virtual content, and control the one or more image forming components to generate light rays that form an image of a virtual cursor, the image of the virtual cursor being configured such that the virtual cursor is perceived to be located at the intersection of the main axis and the perceived virtual object, wherein the virtual cursor serves as an extension of the mobile computing platform.

Images