Holographic Enhanced Creation

Abstract

Holographically enhanced authoring provides an extension of the personal computing experience for general-purpose or conventional productivity applications. The user interface of a productivity application executed on a personal computing device can be switched from touch or conventional mode to a holographic mode, which enables communication between the personal computing device and a holographic-enabled device providing a mixed reality system. Semantic representations of commands in the productivity application are generated as holograms in the mixed reality system, and changes to content files made by executing commands in the mixed reality system do not require a holographic-enabled device to view or even further edit.

Description

Background Art

[0001] Various technologies have emerged that allow users to experience a fusion of the real and virtual worlds. For example, a handheld mobile device (e.g., a smartphone, tablet computer, etc.) can be configured to display a virtual reality environment to the user within the user's field of view and / or the field of view of the device's camera. In some cases, a camera viewfinder window can be used to display information.

[0002] As another example, a head mounted display (HMD) device may include a high-definition camera, a depth camera, a rangefinder, a gyroscope, an accelerometer, and other technologies that allow the HMD device to map the real world and display a fusion of virtual objects and reality on the HMD device. Some HMD devices, including the Hololens, may include a stereoscopic display that uses stereoscopic vision to achieve 3D vision or holograms by displaying separate images with positions appropriately translated for each of the user's eyes. Summary of the Invention

[0003] Holographically enhanced authoring is disclosed. A mixed reality system is provided that includes a holographic mode for productivity applications. Additional functionality and commands are included in productivity applications that enable authoring of content persisted in documents consumable by productivity applications that do not support holography.

[0004] In one embodiment, a holographically enabled device and a second computing device communicate over a wired or wireless connection to execute a user session of a productivity application. The holographically enabled device may include at least a software component of the productivity application, and the second computing device may include the productivity application. The second computing device, while executing the productivity application, displays a content file in an authoring interface of the productivity application; and in response to receiving an indication of a holographic mode, enables communication between the second computing device and the holographic computing device to provide holographically enhanced authoring of the content file.

[0005] The holographically enabled device, while executing the software component of the productivity application, receives at least one content component of the content file and transmits information corresponding to at least one of new or modified content, command characteristics, application configuration, interface configuration, and layout characteristics. The software component of the productivity application executing on the holographically enabled device and the productivity application executing on the second computing device are capable of communicating with each other via an application programming interface.

[0006] Communication between the holographic-enabled device and a second computing device enables manipulation and movement of a holographic image into and out of a graphical user interface displayed at the second computing device. In many cases, the holographic-enabled device executing a software component of the productivity application provides at least one holographic authoring command corresponding to a semantic representation of an authoring command executable on at least one content component of the content file.

[0007] In another embodiment, an instance of a productivity application can be executed on a holographic-enabled device. The holographic-enabled device executing the instance of the productivity application provides at least one holographic authoring command corresponding to a semantic representation of an authoring command executable on at least one content component of a content file. The content file can be modified by executing one or more of the at least one holographic authoring command corresponding to the voice representation of the authoring command. The modified content file can be saved locally at the holographic-enabled device and uploaded for synchronization with a persisted content file at a remote storage location.

[0008] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figures 1A-1B Mixed reality technology is shown in which holographic augmented creation can be implemented.

[0010] Figure 2A An operating environment for an example implementation of holographically enhanced authoring is shown.

[0011] Figure 2B Another operating environment for an example implementation of holographically enhanced authoring is shown.

[0012] Figures 3A-3E An example of a holographically enhanced creation is shown.

[0013] Figures 4A-4C An example of a holographically enhanced creation is shown.

[0014] Figure 5 An example head-mounted display-based holographic-enabled device is shown that can be used in certain implementations described herein.

[0015] Figure 6 An example computing system of a holographically enabled device is shown.

[0016] Figure 7 Components of a computing device that may be used in certain implementations described herein are shown.

[0017] Figure 8 Components of a computing system that can be used to implement certain methods and services described herein are shown. DETAILED DESCRIPTION

[0018] Holographically enhanced creations revealed.

[0019] Virtual reality, augmented reality, and mixed reality technologies have typically focused on entertainment and graphics-intensive scenarios. These technologies have also been expanded to include training in medical and military scenarios. However, one area where these technologies have yet to impact is general productivity, such as in content creation applications.

[0020] Virtual reality is a fully immersive experience, i.e., a user puts on a headset and everything in their field of view is pixels.

[0021] Augmented reality refers to an experience that supplements reality with additional information, such as that available on a heads-up display, a simple overlay, or a mobile device. For example, a transparent heads-up display that includes a high-quality projector and an IR camera could provide odometer and speed data in a vehicle. As another example of augmented reality, a user could hold up their phone running an augmented reality application in front of a store and, based on the device's location, receive the store's phone number or additional information. The digital information is displayed on a specific screen.

[0022] Mixed reality takes augmented reality a step further by not only being able to access the real world and the digitally represented world, but also enabling digitally represented objects to interact with elements of the real world. That is, holograms can be rendered in the real world and appear to interact with objects in the physical world. For example, in a game, a character can follow the physics of physical items in the environment. In this example, a table or sofa in the real world would be recognized by the mixed reality system and taken into account when moving the character around the room, allowing the character to sit on the sofa and avoid walking into the table. Another example of mixed reality is the ability to pick up a component and secure it to a point, such as against a wall.

[0023] Productivity applications can run on a holographic-enabled device in a manner similar to any other computing device; however, on a holographic-enabled device, the graphical user interface for the productivity application can be pinned to an object in the room and made to follow the user of the holographic-enabled device. When the holographic-enabled device is implemented as a head-mounted display system, gaze, gestures, and / or voice can be used instead of a mouse, keyboard, or touch.

[0024] A combination of natural user interface (NUI) elements and holographic capabilities is applied in this article to enhance productivity and content creation.

[0025] A mixed reality system is provided that includes a holographic mode for productivity applications. Additional functionality and commands are included in productivity applications that enable authoring of content persisted in documents consumable by productivity applications that do not support holography.

[0026] Figures 1A-1B As an initial scenario, as shown in FIG1 , a user can sit down to use a computer program such as a computer program about Figure 7 The computing device 100 is depicted as being at work at its desk. Here, a productivity application, such as a presentation application, may be running on the computing device 100. The graphical user interface 101 of the productivity application is displayed on the display 102 of the computing device 100 and shows a normal editing mode for a slideshow 103.

[0027] This general productivity app experience can then be Figure 1B The ground shown is enhanced. Figure 1B By using a holographic-enabled device, users can elevate the traditional two-dimensional experience and extend productivity application features into their real, physical workspace 110, which includes surfaces such as a wall 111 and a desk 112. Visual components can be moved off the computing device 100 and into the workspace 110, and vice versa. Individual holographic slides 113 and other content components can be arranged and pinned in the workspace. Additionally, segment components 114 (which can contain subcomponents including content components) can be arranged and pinned in the workspace. Certain commands can be displayed in the workspace, for example, navigation commands 115 can be included to control visual components such as a segment component labeled "Appendix" 116, thereby causing appendix 116 to expand or collapse in view. Expanding appendix 116 can collapse segment component 114. Alternatively, another navigation command (not shown) can be included to collapse / expand segment component 114 independently of other segments such as appendix 116.

[0028] As an illustrative scenario, user Angela is working on a presentation using a presentation application running on her computing device 100. She can wear a holographically enabled device implemented as a head-mounted device (HMD) and look at the display 102 of her computing device 100 and say, "Expand presentation." To her left are individual slides 113D, E, and F, while to her right are the slides in the segment component 114 of segment 2. Slides 103A, B, and C can remain visible in the graphical user interface 101 of the application running on her computing device 100. In some cases, the previous and / or next slides can "surface" around the area of ​​primary focus (e.g., the actual display 102 of the computing device 100). For example, with slide "B" 103 visible on the display 102, slide "A" can be to her left, where, for example, slide "F" is provided along with the individual slide 113, and slides "C," "D," and so on can be to her right (off-screen). In one such case, thumbnail slides A, B, and C shown in graphical user interface 101 are omitted because those slides are unfolded off display 102. As part of the holographic enhancement creation, Angela can move the holographic slide to a stack 120 indicating discarded slides. From this stack 120, she can retrieve a slide or leave the slide as if it were "on the cutting room floor."

[0029] Some of the functions available in the illustrated scenario include using the computing device 100 normally, and then pushing the mouse off the screen and into the real world to grab and rearrange holograms. User Angela can look at a slide and say copy, delete, move, or even "slide sorter." She can use the real whiteboard in the workspace to annotate the current slide pinned to the whiteboard, scroll to the next slide, and have it all synced back to the saved presentation. She can pick up segment 2 of segment assembly 114 from wall 111 and spread it on desk 112. She can even "throw" one of the slides in the trash (and this action can be interpreted as "delete").

[0030] In addition to components from a single application, other software features can be brought into the workspace by using a holographically enabled device. For example, documents, to-do lists 130, or even additional content creation tools 140 such as networking and / or searching of Internet resources that support productivity tools can be provided in the extended work environment supported by the holographically enabled device.

[0031] Holographically enabled devices and communications between the holographically enabled devices and the computing device 100 may boost productivity and expand the workspace.

[0032] Holographic-enabled devices can allow users to perform certain productivity tasks using holograms instead of printouts. Pages, slides, and other content pieces are not only designed to be interacted with and moved around the workspace just like they would on physical paper, but changes made to them are also reflected and understood by productivity apps, resulting in changes to the content file.

[0033] Figure 2A An operating environment for an example implementation of holographic enhanced authoring is shown. Figure 6 The described computing system 600 and the holographic-enabled device 200 may implement Figure 7 The computing devices 210 of the described computing system 700 can communicate with each other to implement Figure 1A and 1B Example scenario shown and described.

[0034] The holographic-enabled device 200 may include at least a holographic display 201, a processor 202, a storage medium 203, a communication interface 204 (e.g., supporting wired communication or wireless communication such as Bluetooth, Wi-Fi, or other near-field communication); and at least one software component 205 of a productivity application stored on the storage medium 203. The holographic-enabled device 200 may generate holographic images that can be viewed via the holographic display 201.

[0035] The computing device 210 may include at least a processor 211 , a storage medium 212 , a communication interface 213 (eg, supporting wired communication or wireless communication such as Bluetooth, Wi-Fi, or other near field communication), and a productivity application 214 stored on the second storage medium 212 .

[0036] In some cases, software component 205 is part of productivity application 214 executing on holographic-enabled device 200. In some cases, software component 205 includes instructions for communicating with a productivity application running on another device, for example, in the form of a small application programming interface. In some cases, software component 205 is a separate instance from productivity application 214 and includes all or only some of the features available in productivity application 214, except for holographic commands.

[0037] When executed by computing device 210, productivity application 212 directs processor 211 to display the content file in an authoring interface of productivity application 214. A user can interact with the authoring interface and use productivity application 214 to author (e.g., create, read, update, delete) content in the content file. In addition to traditional authoring interfaces typically associated with mouse and keyboard input, there may be additional authoring interfaces optimized for devices of different form factors and different input capabilities. For example, a productivity application may have a touch-mode interface for use with a touch screen, which may change the arrangement and / or availability of certain commands.

[0038] For holographically enhanced authoring, productivity application 212 may include a holographic mode interface. In some cases, in response to receiving an indication of holographic mode, such as by receiving a command from an input device to computing device 210 or even by receiving a command from holographic-enabled device 200, communication between computing device 210 and holographic-enabled device 200 may be initiated to provide holographically enhanced authoring of a content file.

[0039] Thus, if a selection of mixed reality augmented mode (holographic mode) is made at computing device 210, productivity application 214 may be ready to communicate with a holographically enabled device and send information, such as content and content files or other aspects of the productivity application, to the holographically enabled device. In some cases, a user may send commands to the holographically enabled device via productivity application 214.

[0040] The software on the holographic-enabled device 200 (as part of component 205) and the software on the computing device 210 (as part of application 214) can communicate (in some manner through communication interfaces 204 and 213) to transform content components (for both dynamic and static content) in content files to the domain / workspace around the user. Dynamic content includes content that updates over time (and changes based on another program as opposed to changes made by the user), such as from an RSS feed or video. Static content includes content that will remain unchanged unless actively modified by the user within the content file.

[0041] Specific components from a content file that can be moved from the computing device 210 to the holographic-enabled device 200 and shown in a workspace (e.g., workspace 110) can be specifically selected portions of the content and / or atomic fragments that will be identified as objects in a particular productivity application. In some cases, a separate service may be used to identify specific components and sub-components in the content that can be pulled out as complete fragments and interacted with as virtual components in a mixed reality system. In some cases, the productivity application software can indicate and / or identify the available components. In some cases, an intermediary application such as a clipboard can be moved from the computing device 210 to the holographic-enabled device 200 and shown in a workspace (e.g., workspace 110). About Figures 3A-3E An example clipboard implementation is shown and described.

[0042] The software component 205, when executed by a holographically enabled device, provides at least one holographic authoring command corresponding to a semantic representation of an authoring command executable on at least one content component of a content file. Figures 4A-4C Examples and further explanation of the semantic representation of authoring commands are shown and described.

[0043] Software component 205 can recognize sensor data commands (from a user interacting with a holographic representation or otherwise providing command input) and can transmit information to computing device 210 (and productivity application 214) to affect the user interface of productivity application 214 or content files at computing device 210. For example, a holographically enabled device can transmit information 230 to computing device 210 that a command has been executed and / or a change has been made to the content of a content file, where relevant.

[0044] The information about the commands and / or content can change something about the content files at the computing device 210, and can even change the layout or application configuration. For example, the holographic-enabled device 200 can transmit information that a navigation pane is represented holographically to the computing device 210. This information can be used by the productivity application 214 to remove the pane from the user interface.

[0045] Instead, computing device 210 may transmit information 220 to holographic-enabled device 200 regarding aspects of a content file or productivity application that are to be displayed holographically by holographic-enabled device 200 (e.g., because the aspects have been changed or removed from the user interface while the user is interacting with productivity application 214 executing on computing device 210). The aspects of the content file or productivity application include, but are not limited to, content components of the content file, command features, and other features of the productivity application or interface (including graphical icons and functionality of the application or interface).

[0046] Examples of features that may be modified, eg, via an application programming interface of productivity application 214 and / or software component 205 , include, but are not limited to, content files (modifications to actual content or content components), application configuration, interface configuration, and layout.

[0047] As about Figure 2A As described, for seamless operation from the user's perspective, the two devices communicate directly with each other. However, in some cases, direct communication between the two devices is not necessary. In fact, the user can utilize a second computing device in the workspace to perform holographic augmented creation independent of the environment.

[0048] Figure 2B Another operating environment for an example implementation of holographic enhanced authoring is shown. Figure 2B , operating environment 250 may include a holographically enabled device 251 on which a productivity application 252 (a full, partial, or specialized version) can be executed; a computing device 261 on which another instance of a productivity application 262 can be executed; and a remote or cloud-based service 270.

[0049] The holographic-enabled device 251 may be implemented as Figure 6 The computing device 600 described above, the computing device 261 may be implemented as Figure 7 The computing device 700 described; and the service 270 can be implemented as Figure 8 The computing system 800 is described.

[0050] According to an example scenario that can be implemented in environment 250, collaboration on content file D1 (or subsequent consumption or editing) can be performed using one or both of a holographically enabled device and other computing devices. A local copy 280-A of content file D1 can be stored at holographically enabled device 251, and a local copy 280-B of content file D1 can be stored at computing device 261. In some cases, a master copy 280-C of D1 can be stored remotely. For some of the file changes, one or both of devices 251, 261 can communicate with service 270 to update document D1 with the changes. Deltas or entire documents can be transmitted to enable synchronization between documents. Changes made to content files using a holographically enabled device are persistent, which enables a content file viewed at another device to include the changes made to the content file when the holographic authoring commands were used.

[0051] Figures 3A-3E An example of holographic enhancement creation is shown. Figure 3A, in holographic mode, a user works on a document 300 shown in a graphical user interface 301 of a word processing application running on her computing device 302 (configured as device 210 or device 261), and also uses the holographic-enabled device (configured as device 200 or device 251) to view a clipboard 303 of content that the user may have saved from one or more file types (e.g., presentation slides, text clippings, image objects). The user can see both the graphical user interface 301 displayed at computing device 302 and the clipboard 303, which is shown as appearing adjacent to the display screen of computing device 302.

[0052] In this example illustration of holographically enhanced authoring, a clipboard 303 has four items displayed for insertion / pasting into a document 300. Here, the items include an image 305, a balloon for adding a footnote 306, a text snippet 307, and a presentation slide 308. Each item may be shown with a thumbnail 309 and, in some cases, an icon 310 indicating the file type. For example, the item for image 305 may include a thumbnail of the image and an icon 310 indicating that the file type is an image. Of course, the actual arrangement, icons, and information transferred on the clipboard may vary depending on the specific implementation, and this illustration should not be construed as limiting.

[0053] With computing device 302 displaying document 300 and holographic clipboard 303 in view, a user can perform various content creation tasks involving a holographic-enabled device and computing device 302. For example, a user can be adding content to document 300, which is displayed in the Figure 3A 31 and some text 312. A mouse icon 313 is shown on the graphical user interface of computing device 302. In this state, mouse icon 313 is rendered by computing device 302. In some cases, in holographic mode, when the user moves the mouse to the left (e.g., using mouse input to computing device 302), mouse icon 313 is shown moving across the display of computing device 302, and when the mouse reaches a position where mouse icon 313 disappears from the left side of the screen (or at some other specified position), computing device 302 can communicate with the holographic-enabled device to begin displaying the mouse icon holographically, as shown. Figure 3B As shown.

[0054] The holographically enabled device can present a holographic mouse icon 323 that can be controlled by any of a variety of input devices to the holographically enabled device. In the example illustration, the user may have decided to use the clipboard 303 to insert some previously saved or cut content into the document 300. The user may, for example, provide verbal and / or eye gaze input to the holographically enabled device to select a text snippet 307 from the clipboard 303, and then Figure 3B and 3C The selection (text snippet 307 and optionally the holographic mouse icon 323) is shown moved 330 to a position on the document 300 in front of the graphical user interface 301 of the computing device 302. The holographically enabled device can then communicate to insert the text snippet 307 following an appropriate command from the user, causing the computing device to Figure 3E Rendering of appropriate content is shown as 340. This and other seamless manipulation and interaction with the user can be performed through direct and indirect communication between the holographic-enabled device (e.g., 200, 251) and the computing device (e.g., 210, 261).

[0055] The method performed by the holographic-enabled device 200, 251 may include providing a set of holographic authoring commands. The holographic authoring commands are special commands that derive characteristics corresponding to a voice representation of an authoring command that can be performed on at least one content component of a content file. The holographic authoring commands are not only actions that are conventionally available from various menus found in productivity applications. Although these commands can be presented (as part of the virtualization of the menu or simply understood by the instance of the productivity application 252 at the holographic-enabled device 251 or the component 205 of the device 200), the holographic authoring commands are an additional or enhanced set of commands that utilize the holographic capabilities of the holographic-enabled device.

[0056] In response to the holographic-enabled device 200, 251 receiving an instruction to execute a holographic creation command selected from the set of holographic creation commands, the holographic-enabled device 200, 251 can execute the selected holographic creation command to modify the content file. The modification of the content file can be performed on a specific content component and saved at the holographic-enabled device, for example, as a local copy 280-A. The modified content file or an indication of what has been changed (e.g., a delta) can be uploaded to synchronize with the retained content file (e.g., 280-C) at the remote storage location.

[0057] Figures 4A-4C An example of the holographic authoring command "Slideshow Order" is shown in FIG.

[0058] Figures 4A-4C An example of holographic enhancement creation is shown. Figure 4A , a user 400 working on a presentation using a presentation application running on a computing device 410 can enhance her authoring experience holographically by using an HMD device 420 that implements a holographically enabled device. In her field of view 421, she can have her computing device 410, which can display a graphical user interface 412 for the presentation application on its monitor 411. Additionally, as a result of one or more holographic authoring commands that can be part of or precede a command for a slide sequence, slides A, B, C, D, E of her presentation can be represented as holographic slides that are capable of being independently moved within at least an action area 430 defined for the slide sequence. The action area refers to the physical area in which the action corresponding to a particular command is expected to be performed. Here, the action area 430 is shown as being fixed to a desk 440.

[0059] In one implementation, the user 400 requests a holographic slideshow sort via the holographic-enabled device 420, which then communicates with the computing device 410 to obtain the content components of the slideshow, converts the content components into holographic representations, and assigns or directs the user anchor to a specific area in the environment. In another implementation, the user 400 requests a holographic slideshow sort via the user interface 412 at the computing device 410, at which point the productivity application transmits the content components to the HMD device 420 and a slideshow sort command is requested.

[0060] refer to Figure 4B , while in the holographic authoring command function of the slide sequence, the user 400 is shown using eye gaze to move slide E between B and C within the action area 430 .

[0061] Then, as in Figure 4C , the new arrangement can be transmitted to the productivity application running on the computing device 410 (and saved in the content file at the computing device). As can be seen, as a result of the saving, the sequential arrangement of the slides (A, B, C, D) is shown in the general view of the graphical user interface 412.

[0062] Thus, when the selected holographic authoring command includes a slide sorting function and the content file includes a time-lapse file of a presentation application having a content component including at least slides, the semantic representation of the slide sorting function includes a holographic image of each slide on a real surface in the surrounding environment of a user of a holographic-enabled device, the holographic images being capable of being rearranged on the real surface, and the change in the arrangement of the holographic images corresponds to a modification to the order of the slides in the content file.

[0063] Although eye gaze is specifically shown, interaction may be provided using voice or hand tracking.

[0064] In some cases, the phonetic meaning may be what is encoded in a file format.

[0065] While presentation applications are discussed in detail herein, it should be understood that other productivity applications can include holographically enhanced authoring. For example, spreadsheet applications, word processing applications, notebook applications, and the like can have software components for executing on holographically enabled devices and include application programming interfaces for communicating therebetween. Thus, in addition to slides, other content components that can be independent of content files and acted upon in holographic mode include, but are not limited to, worksheet tabs, pivot tables, charts (surrounded by named objects) or pages, images, and clips.

[0066] For the described holographically enhanced authoring, the interactions take place in a holographic arena, but the results of the interactions can be captured in existing formats without requiring partners or consumers to use holographic-enabled devices.

[0067] Component objects from productivity apps (menus, content) can be transformed by holographically enabled devices to generate small interactive surfaces that can then be moved around and attached to physical objects in the environment. Changes to content within and between these component objects can be written back to content files understood by non-holographically enabled devices.

[0068] Holographically enhanced authoring provides a richer authoring experience. Once changes are made using holographically enhanced authoring, they can be appropriately expressed and interpreted within the desktop experience. In this way, a holographically enabled device, such as an HMD, does not need to be worn all day. Changes can be made to content files using holographically enhanced features, and the benefits of this functionality in terms of speed, ease of use, and / or reduced use of paper products do not require the possession of a holographically enabled device in order to view content authored during a session with the HMD.

[0069] In a mixed reality system provided by a holographically enabled device, certain actions taken by a user can be mapped to dedicated commands. For example, a user can verbally, gesture, or through eye contact take a representation of a slide or document and place the representation in (or near) a physical filing cabinet, where this activity can be mapped to physically moving the document, a component of a document, or a copy thereof to a specific file location on a specific device (or cloud storage). As another example, an action that appears to drag a representation of a document to a person in a room can be mapped to a command for messaging a copy of the document to a specific person.

[0070] Holographic-enabled devices can be implemented as see-through, mixed reality display devices. Figure 5 An example head-mounted display-based holographic-enabled device that can be used in certain implementations described herein is shown; and Figure 6 An example computing system is shown in which a holographic-enabled device may be implemented.

[0071] As in Figure 5 As shown in FIG, an example head-mounted display-based holographic-enabled device can take the form of an HMD display 500. The HMD display 500 shown implements a computing system 600 in the form of wearable glasses or eyepieces, but it will be understood that other forms are possible. The HMD device 500 includes a see-through display subsystem 606 (e.g., an at least partially see-through stereoscopic display) having one or more lenses 501 that can be configured to visually enhance the appearance of a physical environment viewed by a user through the see-through display subsystem such that an image can be displayed using the lenses 501 (e.g., using projection onto the lenses 501, one or more waveguide systems incorporated into the lenses 501, and / or in any other suitable manner).

[0072] In some examples, the see-through display subsystem can include one or more transparent (e.g., optically clear) areas and can include one or more opaque or translucent areas. In other examples, the see-through display subsystem can be transparent (e.g., optically clear) across the entire available display surface of the see-through display subsystem.

[0073] The HMD display 500 includes an optical sensor system 502, which may include one or more optical sensors. In one example, the optical sensor system 502 may include one or more outward-facing optical sensors that may be configured to acquire and detect the real-world background and / or physical space from a vantage point (e.g., line of sight) similar to that observed by the user through the lens 501. The optical sensor system 502 may include a variety of sensors, such as one or both of a depth camera / sensor and an RGB camera / sensor. In some cases, a high-definition camera or other resolution may be used for image sensing.

[0074] Sensors included with HMD device 500 may support various functions, including head tracking for determining the 3D (three-dimensional) position and orientation of a user's head within physical real-world space; and gaze tracking for determining the direction of the user's gaze.

[0075] For example, the HMD device 500 may include a position sensor system 503, which may include one or more sensors 504, such as accelerometers, gyroscopes, magnetometers, a global positioning system (GPS) 505, a multilateration tracker (not shown), and / or other sensors that output position sensor information that can be used as a correlation sensor for position, orientation, and / or movement.

[0076] When the position sensor system 503 includes one or more motion sensors 504 (eg, inertial, multi-axis gyroscope, or accelerometer), the movement and position / orientation / posture of the user's head can be detected while the user is wearing the system.

[0077] The HMD device 500 may also include a gaze detection subsystem 506 that is configured to detect the direction of the gaze of each eye of the user or the direction or position of the focus. The gaze detection subsystem 506 can be configured to determine the gaze direction of each eye of the user in any suitable manner. For example, in the illustrative example shown, the gaze detection subsystem 506 includes one or more flash sources 507, such as infrared light sources, that are configured to cause flashes of light to be reflected from each eye of the user, and one or more image sensors 508, such as inward-facing sensors, that are configured to capture images of each eye of the user. Changes in the flash changes from the user's eye and / or the user's pupil position, as determined from image data collected using the image sensor 508, can be used to determine the direction of the gaze.

[0078] Additionally, the location where the line of sight projected from the user's eye intersects the external display can be used to determine the object at which the user is looking (e.g., a displayed virtual object and / or a real background object). The line of sight detection subsystem 506 can have any suitable number and arrangement of light sources and image sensors. In some implementations, the line of sight detection subsystem 506 can be omitted.

[0079] The HMD device 500 may include one or more microphones 509 configured to detect sounds, such as voice commands from a user.

[0080] The position sensor system 503 (including the motion sensor 504), as well as the microphone 505 and the gaze detection subsystem 506, can also be used as user input devices to allow the user to interact with the HMD device 500 via gestures of the eyes, neck, and / or head, and in some cases, via spoken commands. Figure 6 denoted as input subsystem 608 .

[0081] The HMD device 500 can be configured with one or more audio transducers 510 (eg, speakers, headphones, etc.) to enable audio to be used as part of the user experience.

[0082] The HMD device 500 may also include a controller 511 that includes the components described with respect to the computing system 600, such as the processing system 602 and the storage system 604. The controller 511 may communicate with the sensor 502, the gaze detection subsystem 506, the lens 501, and / or other components, for example, via a communication subsystem 610. The communication subsystem 610 may operate using a network interface and subsystem 612 to facilitate a display system that operates in conjunction with remote resources (e.g., processing, storage, power, data, and services). That is, in some implementations, the HMD device may operate as part of a system that distributes resources and capabilities among different components and subsystems.

[0083] The storage subsystem 604 may include instructions stored thereon that are executable by the processing system 602 to, for example, receive and interpret input from sensors, identify the position and movement of a user, use surface reconstruction and other techniques to identify real objects, and dim / fade the display based on the distance to an object so that the object can be seen by the user, etc.

[0084] The HMD device 500 may include instructions for image generation stored on the storage system 604. These instructions direct the HMD device 500 to display virtual objects to the user, visually superimposed on the physical environment so as to be perceived at various depths and positions. The HMD device 500 may utilize stereoscopic vision to visually place virtual objects at desired depths by displaying separate images of the virtual object to the user's two eyes. To achieve depth perception, the image generation instructions may render two images of the virtual object in the focal plane of the HMD device 500, resulting in binocular parallax between the relative positions of the virtual object in the two images. For example, this binocular parallax may be horizontal parallax, where the relative positions of the virtual object in the two images are separated by a distance along the x-axis. Here, the x-axis may be defined as the axis extending left and right relative to the user, the y-axis as the axis extending upward and downward relative to the user, and the z-axis as the axis extending forward and backward relative to the user.

[0085] The horizontal disparity between the relative positions of the virtual objects in the two images will cause the user to perceive the virtual objects as being at a certain depth within the viewed physical environment due to stereoscopic vision. Using this stereoscopic vision technology, the HMD device 500 can control the displayed images of the virtual objects so that the user will perceive the virtual objects as existing at a desired depth and position within the viewed physical environment.

[0086] The optical sensor information received from the optical sensor system 502 and / or the position sensor information received from the position sensor system 503 can be used to estimate the position and orientation of the vantage point of the at least partially see-through stereoscopic display relative to environmental objects. In some embodiments, the position and orientation of the vantage point can be characterized using six degrees of freedom (e.g., world space X, Y, Z, pitch, bank, and yaw). The vantage point can be characterized globally or independently of the real-world context. The position and / or orientation can be determined using an onboard computing system (e.g., controller 511) and / or an offboard computing system.

[0087] Furthermore, the optical sensor information and the positional sensor information can be used by the computing system to perform analysis of the real-world context, such as depth analysis, surface reconstruction, ambient color and lighting analysis, or other suitable operations. In particular, the optical and positional sensor information can be used to create a virtual model of the real-world context. In some embodiments, the position and orientation of the vantage point can be characterized relative to the virtual space. Furthermore, the virtual model can be used to determine the position of virtual objects in the virtual space and to add additional virtual objects to be displayed to the user at a desired depth and position within the virtual world.

[0088] When the position sensor system 503 includes GPS 505, the system can use GPS 505 to determine the location of the HMD device 500. This can help identify real-world objects, such as buildings, that may be located in the user's immediate physical environment.

[0089] A power management subsystem (not shown) may include one or more batteries and / or protection circuit modules (PCMs) and associated charger interfaces and / or remote power interfaces for powering components in the HMD device 500 .

[0090] It is understandable that in Figure 5 As shown in and about Figure 5 The described sensors are included for purposes of example and are not intended to be limiting in any way, as any other suitable sensors and / or combinations of sensors may be utilized to meet the needs of a particular implementation of a head-worn holographic-enabled device. For example, biometric sensors (e.g., for detecting heart and breathing rates, blood pressure, brain activity, body temperature, etc.) or environmental sensors (e.g., for detecting temperature, humidity, altitude, UV (ultraviolet) light levels, etc.) may be employed in some implementations.

[0091] It should be understood that the display device may include additional and / or alternative sensors, cameras, microphones, input devices, output devices, etc. than those shown without departing from the scope of the present arrangement. Additionally, the physical configuration of the display device and its various sensors and subassemblies may take a variety of different forms without departing from the scope of the present arrangement.

[0092] As mentioned above, computing system 600 can implement holographic-enabled devices, including but not limited to HMD device 500. Computing system 600 includes a processing system 602, which can include a logical processor (and can even include multiple processors of the same or different types) and a storage system 604, which can include volatile and non-volatile memory.

[0093] The processing system 602 may include one or more physical processors (hardware) configured to execute software instructions. Additionally or alternatively, the logical processor may include one or more hardware logic circuits or firmware devices configured to execute hardware-implemented logic or firmware instructions. The processor of the processing system 602 may be single-core or multi-core, and the instructions executed thereon may be configured for sequential, parallel and / or distributed processing. The individual components of the processing system 602 may optionally be distributed in two or more separate devices that can be remotely located and / or configured for collaborative processing. Various aspects of the processing system 602 may be virtualized and executed by a remotely accessible networked computing device configured in a cloud computing configuration. In such a case, it should be understood that these virtualized aspects are run on different physical logical processors of various different machines.

[0094] Processing system 602 includes one or more physical devices configured to execute instructions. For example, processing system 602 can be configured to execute instructions that are part of one or more applications, programs, routines, libraries, objects, components, data structures, or other logical constructs. Such instructions can be implemented to perform a task, implement a data type, convert the state of one or more components, achieve a technical effect, or otherwise achieve a desired result. When the instructions are software-based (as opposed to hardware-based, such as implemented in a field programmable gate array (FPGA) or digital logic), the instructions can be stored as software 605 in storage system 604.

[0095] The storage system 604 may include removable and / or built-in physical devices. The storage system 604 may include one or more volatile and non-volatile storage devices, such as optical storage (e.g., CD, DVD, HD-DVD, Blu-ray Disc, etc.), semiconductor memory (e.g., RAM, SRAM, DRAM, ROM, EPROM, EEPROM, FLASH memory, etc.), and / or magnetic storage (e.g., hard disk drive, floppy disk drive, tape drive, MRAM, etc.), or other mass storage device technologies. The storage system 604 may include dynamic, static, read / write, read-only, random access, sequential access, location addressable, file addressable, and / or content addressable devices. It should be understood that the storage device or storage medium of the storage system includes one or more physical devices and excludes the propagation signal itself. It should be understood that, in contrast to being stored on a storage device or medium, various aspects of the instructions described herein may be propagated by pure signals using a communication medium (e.g., electromagnetic signals, optical signals, etc.). In addition, data and / or other forms of information related to the present arrangement may be propagated by pure signals.

[0096] Various aspects of the processing system 602 and the memory system 604 may be integrated together into one or more hardware logic components. Such hardware logic components may include, for example, field programmable gate arrays (FPGAs), program and application specific integrated circuits (PASIC / ASIC), program and application specific standard products (PSSP / ASSP), systems on chips (SOCs), and complex programmable logic devices (CPLDs).

[0097] The terms "module," "program," and "engine" may be used to describe aspects of computing system 600 that are implemented to perform specific functions. In some cases, a module, program, subsystem, or engine may be instantiated via processing system 602 executing instructions held by non-volatile storage of storage system 604 or by using a portion of the volatile storage of storage system 604. It will be understood that different modules, programs, and / or engines may be instantiated from the same application, service, code block, object, library, routine, API, function, etc. Likewise, the same module, program, and / or engine may be instantiated by different applications, services, code blocks, objects, libraries, routines, APIs, functions, etc. The terms "module," "program," and "engine" may encompass individual or grouped executable files, data files, libraries, drivers, scripts, database records, and the like.

[0098] When included, the display subsystem 606 can be used to present a visual representation of the data stored by the storage system 604. This visual representation can take the form of a graphical user interface (GUI). Since the methods and processes described herein change the data stored by the storage system and thus transform the state of the storage system, the state of the display subsystem 606 can also be transformed to visually represent the changes in the underlying data. The display subsystem 606 can include one or more display devices, such as those discussed above, utilizing virtually any type of technology. Such a display device can be combined with a logic device and / or storage device in a shared enclosure, or such a display device can be a peripheral display device. The at least partially see-through display of the HMD 500 described above is an example of a display subsystem 606.

[0099] When included, the input subsystem 608 may include or interface with selected natural user input (NUI) components. Such components may be integrated or peripheral, and the conversion and / or processing of input actions may be on-board or off-board. Example NUI components may include microphones for voice and / or speech recognition; infrared, color, stereo, and / or depth cameras for machine vision and / or gesture recognition; head trackers, eye trackers, accelerometers, and / or gyroscopes for motion detection and / or intent recognition; and electric field sensing components for assessing brain activity; as discussed above with respect to Figure 5 Any of the sensors described and / or other suitable sensors.

[0100] When included, the network interface and subsystems 612 can be configured to communicatively couple the computing system 600 with one or more other computing devices. For example, the network interface and subsystems 612 can include wired and / or wireless communication devices compatible with one or more different communication protocols. As non-limiting examples, the network interface and subsystems 612 can be configured to communicate via a wireless telephone network, or a wired or wireless, near-field local area network or wide area network. In some embodiments, the network interface and subsystems 612 can allow the computing system 600 to send and / or receive messages to and from other devices via a network such as the Internet.

[0101] Figure 7 illustrates components of a computing device that may be used in certain implementations described herein; and Figure 8 Shown are components of a computing system that can be used to implement certain methods and services described herein.

[0102] refer to Figure 7, system 700 can represent a computing device such as, but not limited to, a personal computer, a reader, a mobile device, a personal digital assistant, a wearable computer, a smartphone, a tablet computer, a laptop computer (notebook or netbook), a gaming device or console, an entertainment device, a hybrid computer, a desktop computer, or a smartphone. Thus, more or fewer elements described with respect to system 700 can be incorporated to implement a particular computing device.

[0103] System 700 includes a processing system 720 of one or more processors for converting or manipulating data according to instructions of software 710 stored on a storage system 715. Examples of processors of processing system 705 include general-purpose central processing units, special-purpose processors and logic devices, and any other type of processing unit, combination or variation thereof. Processing system 705 can be a system on a chip (SoC) or included in an SoC along with one or more other components such as network connectivity components, sensors, video display components, and the like.

[0104] Software 710 may include an operating system and application programs such as productivity applications 720, which may include components for communicating with a holographically enabled device such as those described herein. A device operating system generally controls and coordinates the functions of various components in a computing device, providing an easier way for applications to connect to lower-level interfaces such as networking interfaces. Non-limiting examples of operating systems include Window, Windows, and Windows XP from Microsoft Corporation. From Apple iOS TM , from Google and the Ubuntu variant of the Linux OS from Canonical.

[0105] It should be noted that the operating system can be implemented natively on the computing device or on a software virtualization layer running on top of the native device operating system (OS). Figure 7 , but the virtualized OS layers can be thought of as additional, nested groups within the operating system space, each containing the OS, applications, and APIs.

[0106] Storage system 715 may include any computer-readable storage medium that can be read by processing system 705 and that can store software 710 , including productivity application 720 .

[0107] The storage system 715 may include volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable instructions, data structures, program modules, or other data. Examples of storage media for the storage system 715 include random access memory, read-only memory, magnetic disks, optical disks, CDs, DVDs, flash memory, cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other suitable storage medium. In either case, the storage medium is not a propagating signal or carrier wave.

[0108] Storage system 715 may be implemented as a single storage device, but may also be implemented across multiple storage devices or subsystems that are co-located or distributed relative to each other. Storage system 715 may include additional elements such as a controller that can communicate with processing system 705.

[0109] The software 710 may be implemented as program instructions and, when executed by the system 700 in general, and the processing system 705 in particular, directs the system 700 or one or more processors of the processing system 705 to operate as described herein.

[0110] Generally speaking, the software, when loaded into the processing system 705 and executed, can transform the computing system 700 as a whole from a general-purpose computing system into a specialized computing system customized to retrieve and process information used to facilitate content creation, as described herein for each implementation. In practice, the encoded software on the storage system 715 can transform the physical structure of the storage system 715. The specific transformation of the physical structure can depend on various factors in different implementations of this description. Examples of such factors can include, but are not limited to, the technology of the storage medium used to implement the storage system 715 and whether the computer storage medium is characterized as primary or secondary storage.

[0111] The system may also include a user interface system 730, which includes input / output (I / O) devices and components that enable communication between a user and the system 700. The user interface system 730 may include input devices such as a mouse 731, a trackpad (not shown), a keyboard 732, a touch device 733 for receiving touch gestures from a user, a motion input device 734 for detecting non-touch gestures and other motions of a user, a microphone for detecting voice (not shown), and other types of input devices capable of receiving user input and their associated processing elements.

[0112] The user interface system 730 may also include output devices, such as a display screen (735), a speaker (not shown), a tactile device (not shown) for tactile feedback, and other types of output devices. In some cases, input and output devices can be combined in a single device, such as a touch screen display that both depicts an image and receives touch gesture input from a user. A touch screen (which may be associated with a display or form part of a display) is an input device configured to detect the presence and location of a touch. The touch screen may be a resistive touch screen, a capacitive touch screen, a surface acoustic wave touch screen, an infrared touch screen, an optical imaging touch screen, a scattered signal touch screen, a sound pulse recognition touch screen, or any other touch screen technology may be used. In some embodiments, the touch screen is included in the top layer of the display as a transparent layer to enable a user to use one or more touches to interact with objects or other information presented on the display.

[0113] Visual output may be depicted on display 735 in a variety of ways, such as graphical user interface elements, text, images, video, notifications, virtual buttons, virtual keyboards, or any other type of information capable of being depicted in visual form.

[0114] The user interface system 730 may also include user interface software and associated software (e.g., for a graphics chip and input device) executed by the OS with the support of various user input and output devices. Associated software assists the OS when using a defined mechanism to transmit user interface hardware events to an application. The user interface system 730 including user interface software may support a graphical user interface, a natural user interface, or any other type of user interface. For example, the interface of the productivity application and / or productivity tool for auxiliary content creation (and corresponding functions) described herein may be presented by the user interface system 730.

[0115] The communication interface 740 may include communication connections and devices that allow communication with other computing systems via one or more communication networks (not shown). Examples of connections and devices that collectively allow inter-system communication may include network interface cards, antennas, power amplifiers, RF circuits, transceivers, and other communication circuits. The connections and devices may communicate via a communication medium (e.g., metal, glass, air, or any other suitable communication medium) to communicate with other computing systems or network exchanges of the system. Transmissions to and from the communication interface are controlled by the OS, which notifies applications of communication events when necessary.

[0116] Computing system 700 is generally intended to represent a computing system with which software is deployed and executed to implement the applications, components, or services for productivity tools described herein. In some cases, aspects of computing system 700 may also represent a computing system on which software may reside and from which it may be distributed, transferred, downloaded, or otherwise provided to another computing system for deployment, execution, or further distribution.

[0117] Certain aspects described herein may be used in, for example, Figure 8 Executed on the system shown in . Figure 8 , system 800 can be implemented within a single computing device or distributed across multiple computing devices or subsystems that cooperate in executing program instructions. System 800 may include one or more blade server devices, stand-alone server devices, personal computers, routers, hubs, switches, bridges, firewall devices, intrusion detection devices, mainframe computers, network attached storage devices, and other types of computing devices. The system hardware can be configured according to any suitable computer architecture, such as a symmetric multiprocessing (SMP) architecture or a non-uniform memory access (NUMA) architecture.

[0118] System 800 may include a processing system 810, which may include one or more processors and / or other circuitry that retrieves and executes software 820 from a storage system 830. Processing system 810 may be implemented within a single processing device, but may also be distributed across multiple processing devices or subsystems that cooperate in executing program instructions.

[0119] The storage system 830 may include any computer-readable storage medium that can be read by the processing system 810 and that can store the software 820. The storage system 830 may be implemented as a single storage device, but may also be implemented across multiple storage devices that are co-located or distributed relative to each other. The storage system 830 may include additional elements, such as a controller that can communicate with the processing system 810. The storage system 830 may also include storage devices and / or subsystems on which data, such as information related to an entity, is stored.

[0120] The software 820 may be implemented as program instructions and, when executed by the system 800 generally or the processing system 810 specifically, directs the system 800 or the processing system 810 to operate as described herein for receiving communications associated with, for example, the productivity applications described herein and performing certain functions including storing or managing content files. 845

[0121] System 800 may represent any computing system on which software 820 may reside and from which software 820 may be distributed, transferred, downloaded, or otherwise provided to another computing system for deployment, execution, or further distribution.

[0122] In embodiments where the system 800 includes multiple computing devices, the server may include one or more communication networks that facilitate communication between the computing devices. For example, the one or more communication networks may include a local area network and a wide area network that facilitate communication between the computing devices. One or more direct communication links may be included between the computing devices. Additionally, in some cases, the computing devices may be installed at geographically distributed locations. In other cases, multiple computing devices may be installed at a single geographic location, such as a server farm or an office.

[0123] A communications interface 850 may be included that provides communications connections and devices that allow communication between the system 800 and other computing systems (not shown) over a communications network or collection of networks (not shown) or over the air.

[0124] Certain techniques described herein for holographically enhanced authoring may be described in the general context of computer-executable instructions, such as program modules, executed by one or more computing devices. Generally speaking, program modules include routines, programs, components, and data structures that perform specific tasks or implement specific abstract data types.

[0125] Alternatively or additionally, the functions, methods, and processes described herein may be implemented, at least in part, by one or more hardware modules (or logic components). For example, the hardware modules may include, but are not limited to, application specific integrated circuit (ASIC) chips, field programmable gate arrays (FPGAs), SoC systems, complex programmable logic devices (CPLDs), and other programmable logic devices currently known or developed in the future. When a hardware module is activated, the hardware module executes the functions, methods, and processes included within the hardware module.

[0126] Embodiments may be implemented as computer processes, computing systems, or articles of manufacture such as computer program products or computer-readable media. Certain methods and processes described herein may be implemented as software, code, and / or data that can be stored on one or more storage media. Certain embodiments of the present invention contemplate the use of a machine in the form of a computer system having an instruction set within the computer system that, when executed, can cause the system to perform any one or more of the methods discussed above. Certain computer program products may be one or more computer-readable storage media that can be read by a computer system and encode a computer program of instructions to perform a computer process.

[0127] Computer-readable media can be any available computer-readable storage media or communication media that can be accessed by a computer system.

[0128] Communication media include media through which communication signals containing, for example, computer-readable instructions, data structures, program modules, or other data are transmitted from one system to another. Such communication media may include guided transmission media, such as cables and wires (e.g., fiber optic, coaxial, etc.), and wireless (unguided transmission) media, such as acoustic, electromagnetic, radio frequency, microwave, and infrared, capable of propagating energy waves. Although described in terms of communication media, carrier waves and other propagated signals containing data that can be used by a computer system are not considered computer-readable "storage media."

[0129] By way of example and not limitation, computer-readable storage media may include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable instructions, data structures, program modules, or other data. For example, computer-readable storage media include, but are not limited to, volatile memory, such as random access memory (RAM, DRAM, SRAM); and non-volatile memory, such as flash memory, various read-only memories (ROM, PROM, EPROM, EEPROM), phase-change memory, magnetic and ferromagnetic / ferroelectric memories (MRAM, FeRAM), and magnetic and optical storage devices (hard drives, magnetic tape, CDs, DVDs). As described herein, in either case, "computer-readable storage media" does not include carrier waves or propagated signals.

[0130] It should be understood that the examples and embodiments described herein are for illustrative purposes only and that various modifications and changes therefrom will be suggested to those skilled in the art and are intended to be included within the scope of this application.

[0131] Although the subject matter has been described in language specific to structural features and / or acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as examples of implementing the claims, and other equivalent features and acts are intended to be within the scope of the claims.

Claims

1. A system for holographic enhanced creation: A holographic-enabled device comprising at least a holographic display, a processor, a storage medium, and a communication interface; and at least one software component of a productivity application stored on said storage medium; as well as a second computing device comprising at least a second processor, a second storage medium, a second communication interface, and a productivity application stored on the second storage medium, When the productivity application is executed by the second computing device, the second processor is directed to perform the following operations: displaying a content file in an authoring interface for the productivity application; In response to receiving the indication of the holographic mode, enabling communication between the second computing device and the holographic-enabled device to provide holographically enhanced authoring of the content file; transmitting to the holographically enabled device the content file or the aspect of the productivity application to be holographically displayed by the holographically enabled device; displaying the aspect of the content file or the productivity application before transmitting to the holographically enabled device the aspect to be holographically displayed; and removing the aspect of the content file or the productivity application from display by the second computing device while the aspect is holographically displayed by the holographic-enabled device; wherein the software component of the productivity application, when executed by the holographic-enabled device, provides at least one holographic authoring command corresponding to a semantic representation of an authoring command executable on at least one content component of the content file, and Wherein, when the software component of the productivity application is executed by the holographic-enabled device, it further directs the processor to perform the following operations: In response to receiving a communication from the second computing device that the aspect of the content file or the productivity application is to be holographically displayed, the content file or the aspect of the productivity application is displayed.

2. The system according to claim 1, wherein: The productivity application includes instructions that direct the second processor to: In response to receiving the indication of the holographic mode, the at least one content component of the content file is transmitted to the holographic enabled device.

3. The system according to claim 1, wherein: The software component of the productivity application includes instructions that direct the processor to: Information corresponding to at least one of new or modified content, command characteristics, application configuration, interface configuration, and layout characteristics is transmitted and received.

4. The system according to claim 1, wherein: The productivity application, when executed by the second computing device, further directs the second processor to perform the following operations: wherein the aspect of the content file or the productivity application comprises one or more of at least one content component of the content file, a command feature, or a graphical icon; Wherein, when the software component of the productivity application is executed by the holographic-enabled device, it further directs the processor to perform the following operations: In response to receiving a communication from the second computing device that the aspect of the content file or the productivity application is to be holographically displayed, the content file or the aspect of the productivity application is displayed.

5. The system according to claim 1, wherein: The software component of the productivity application, when executed by the holographic-enabled device, further directs the processor to: transmitting to the second computing device: the content file or the aspect of the productivity application to be displayed by the second computing device; When the productivity application is executed by the second computing device, it further directs the second processor to perform the following operations: In response to receiving a communication from the holographic-enabled device that the aspect of the content file or the productivity application is to be displayed, the aspect of the content file or the productivity application is displayed.

6. The system according to claim 1, wherein: The software component of the productivity application, when executed by the holographic-enabled device, further directs the processor to: In response to receiving an instruction for executing a holographic creation command selected from the at least one holographic creation command, executing the selected holographic creation command to modify the content file; as well as transmitting the modified content file or an indication of what was changed to the second computing device; When the productivity application is executed by the second computing device, it further directs the second processor to perform the following operations: In response to receiving the modified content file or the indication of what has been changed, the content file stored at the second computing device is updated.

Images