Tangible instances of mapping documents

Abstract

A computer-implemented method includes: detecting first content of a tangible instance of a first document by a first computer system; generating a first hash using the first content by the first computer system, the first hash including first obfuscated content; sending the first hash by the first computer system for reception by a second computer system; and receiving a response by the first computer system to the first hash generated by the second computer system, the response including information corresponding to a second document associated with the first content.

Description

Technical Field

[0001] This document deals with tangible instances of mapped documents. Background Technology

[0002] Historically, documents were typically made from durable media such as paper. When two or more copies of the same document were created, these copies were usually distributed separately from each other, and tracking what happened to each individual copy was not always possible or practical. For example, once a person creates a paper document, many things may happen to that paper, and that person may never know about those things or be able to react to them.

[0003] The advent of computer technology has made it possible to maintain and edit documents electronically, and thus monitor their development and use over time in a more organized and reliable manner. However, this enhanced ability to preserve time and logistical records has only been applied to the electronic records themselves, which are digitally maintained through representations of 1s and 0s. On the other hand, the physical instantiation of such electronic documents—which is essential for anyone to directly perceive or understand the document—often encounters the same challenges historically associated with paper documents. Summary of the Invention

[0004] In a first aspect, a computer-implemented method includes: detecting first content of a tangible instance of a first document by a first computer system; generating a first hash using the first content by the first computer system, the first hash including first obfuscated content; sending the first hash by the first computer system for reception by a second computer system; and receiving a response by the first computer system to the first hash generated by the second computer system, the response including information corresponding to a second document associated with the first content.

[0005] The implementation may include any or all of the following features. A second computer system controls a collaborative program for a document set. The computer-implemented method further includes a first computer system sending first content to a second computer system for receipt based on a response. The second computer system uses the first content to generate a new document for the collaborative program. The computer-implemented method further includes a first computer system sending a tag change of a first document to a second computer system for receipt based on a response. The document set includes a second document. The second computer system generates a second hash using second content of the second document, the second hash including second obfuscated content, and wherein the second hash is included in the response. The computer-implemented method further includes the first computer system using the second hash to verify a correspondence between the first document and the second document. The computer-implemented method further includes the first computer system receiving the second document from the second computer system. The second computer system performs unauthorized access detection of the second content upon receiving the first hash. The computer-implemented method further includes the first computer system receiving the access history of its users, the history including entries for accessing the second document based on the second computer system having received the first hash. Entries for accessing the second document are based on detecting at least one of a user approaching a hard copy of the first document or a screen presentation of the first document. The second computer system generates an entry in response to determining that a user has access rights to a second document. The user does not have access rights to the second document, and the information corresponding to the second document includes controls for the user to request access rights to the second document. The user does not have access rights to the second document, and the second computer system grants access rights to the user in response to determining that the user has been accessing the first document for at least a predetermined time. The first computer system identifies the user of the first computer system to the second computer system, and the document set is defined based on the user's access rights to the document set. The computer-implemented method further includes the first computer system receiving a request from the user for a text-to-speech service for the first document, wherein the second document contains structural markup, and the text-to-speech service for the first document is provided using the second document. The first computer system detects first content based on the first document presented on a display device. The display device is controlled by the first computer system and presents a screen-sharing application on the desktop, wherein the screen-sharing application is used to share the screen of the first document with the first computer system. The first computer system further presents a browser on the desktop, and the browser provides controls for opening the second document on the desktop using a collaboration program. The computer-implemented method further includes a first computer system facilitating the dragging of a representation of a first document using a collaborative program to trigger the opening of a second document. The dragging causes the second document to be displayed on the desktop of the first computer system. Based on the detection of another computer system by the first computer system, the dragging causes the second document to be displayed on the other computer system.The first computer system includes an augmented reality (AR) head-mounted device, wherein the first computer system detects the first content based on the first content within the field of view of the AR head-mounted device. The computer-implemented method further includes the first computer system detecting a gesture performed by a user of the AR head-mounted device within the AR field of view, and, in response, a representation of moving a first document within the AR field of view according to the gesture. The movement of the representation facilitates opening a second document on the first computer system using a collaborative program. The movement of the representation facilitates opening a second document on another computer system detected by the first computer system using a collaborative program. The computer-implemented method further includes the first computer system using information to present a second content of the second document. The first computer system includes an augmented reality (AR) head-mounted device, wherein the first computer system detects the first content based on the first content within the field of view of the AR head-mounted device, and wherein presenting the second content includes applying a first virtual annotation to the first document. The computer-implemented method further includes the first computer system receiving audio input generated by a user corresponding to a change made in the first document, presenting a second virtual annotation for the first document in the AR field of view, and sending the change to a second computer system. The second document is publicly accessible, and the response includes the second document. Both the first and second documents are paper documents, and the second computer system uses a first hash and a second hash of the second document to detect the second document associated with the first document. The first and second hashes facilitate the sharing of virtual annotations about the first or second document between the first and second computer systems.

[0006] In a second aspect, a computer program product tangibly embodied in a non-transitory storage medium includes instructions that, when executed by one or more processors, cause the one or more processors to perform operations, the operations including: detecting first content of a tangible instance of a first document by a first computer system; generating a first hash using the first content by the first computer system, the first hash including first obfuscated content; sending the first hash by the first computer system for reception by a second computer system; and receiving a response by the first computer system to the first hash generated by the second computer system, the response including information corresponding to a second document associated with the first content.

[0007] The implementation may include the following aspects. The computer program product includes instructions that, when executed by one or more processors, cause the one or more processors to perform the method described according to any embodiment disclosed in this application.

[0008] An optional feature of one aspect can be combined with any other aspect. Attached Figure Description

[0009] Figure 1 An example of a computer system is shown.

[0010] Figure 2 It shows Figure 1 An example of storage in a computer system.

[0011] Figures 3A-3F Examples related to tangible instances of documents are shown.

[0012] Figures 4A-4B The concepts illustrate examples related to tangible instances and computer systems.

[0013] Figure 5 An example related to storage drives is shown.

[0014] Figure 6A It shows the relationship with Figure 5 Examples related to storage drives.

[0015] Figure 6B This shows what the user is viewing. Figures 3A-3D Tangible examples and AR fields of view.

[0016] Figures 7A-7B It shows the relationship with Figures 3A-3E Tangible instances and examples related to AR field of view.

[0017] Figures 8A-8E It shows the relationship with Figures 3A-3E Tangible instances and examples related to display devices.

[0018] Figures 9A-9B It shows the relationship with Figures 8A-8E Examples related to display devices.

[0019] Figure 10 An example of a computer-implemented method is shown.

[0020] Figure 11 Examples of computer devices and mobile computer devices that can be used to implement the technologies described herein are shown.

[0021] The same reference numerals in the various figures indicate the same elements. Detailed Implementation

[0022] This document describes examples of mapping tangible instances of a document (e.g., paper copies or on-screen representations). A tangible instance can be mapped to an electronic document from which it was created, or to another tangible instance of the same document (also when neither tangible instance was created from a public electronic original), to name just two examples. Such mapping can facilitate a better user experience, enhanced protection of personal or other sensitive information, improved record keeping, and / or increased information security.

[0023] The examples in this document relate to computer systems. As used herein, a computer system includes, but is not limited to, any suitable combination of one or more devices configured with hardware, firmware, and software to perform one or more of the computerization techniques described herein. A computer system as used herein can be a single computing device or multiple computing devices that work together and in which data storage and the execution of functions are distributed among various computing devices.

[0024] The examples in this document relate to operations performed by a client or server, or operations occurring on the client side or server side. The terms client and server are used for illustrative purposes only and are not limited to specific types of computer systems or devices. As used herein, a client or server can include any type of computer system unless otherwise indicated. For example, a server can include a physical server and / or a virtual machine. For example, a client can include wearable devices, tablets, digital audio players, video game consoles, desktop computers, laptops, projectors, televisions, electronic billboards, cellular phones, personal digital assistants, and / or smartphones.

[0025] The examples in this document relate to Augmented Reality (AR). As used herein, AR relates to a user experience in which a computer system facilitates sensory perception comprising at least one virtual aspect and at least one real aspect. AR can be provided by any of a variety of computer systems, including but not limited to wearable devices. As used herein, an AR headset refers to any computer system that facilitates AR. AR headsets can include, but are not limited to, AR glasses, another wearable AR device, a tablet, or a laptop. In some types of AR, a user can perceive aspects of reality directly with his or her senses without the mediation of a computer system. For example, some AR headsets are designed to project images (e.g., the virtual aspect to be perceived) onto the user's retina while also allowing the eyes to register other light not produced by the AR headset. As another example, an in-lens microdisplay can be embedded in a see-through lens, or a projection display can be overlaid on the see-through lens. In other types of AR, the computer system can enhance, supplement, alter, and / or realize the user's impression of reality (e.g., the real aspect to be perceived) in one or more ways. In some implementations, AR is perceived on the screen of a display device of the computer system. For example, some AR headsets are designed with camera feedthroughs to display camera images of the user's surroundings on a display device positioned in front of the user's eyes.

[0026] The examples in this article relate to gestures being detected by an AR headset. As used herein, gesture detection includes any means by which the AR headset can recognize input from a user. Gestures can include, but are not limited to, any of the following: visually detected movement of a body part (e.g., a hand, finger, or arm); movement of a body part (e.g., a head) detected by an accelerometer or other inertial measurement unit; or input made using a controller (e.g., a handheld controller) and detected by the AR headset.

[0027] The examples in this document relate to documents. As used herein, a document is information in a medium. For example, an electronic document relates to information in digital form (e.g., a computer-readable document on a storage medium). As another example, a hard copy document relates to printed information (e.g., information printed on paper). As another example, a presentation on a screen relates to information displayed on a display device. The examples in this document relate to tangible instances of documents. As used herein, a tangible instance of a document includes any embodiment of a document in a form that can be perceived (and possibly understood) by humans. As used herein, a tangible instance of a document includes at least a hard copy of a document or a presentation of a document on a screen. If a tangible instance is substantially identical to a document, then the tangible instance may correspond to the document. For example, if a tangible instance is generated from a document, then the tangible instance corresponds to the document.

[0028] The examples in this article refer to the content of a document. As used in this article, the content includes some or all of the document's information.

[0029] The examples in this document involve generating hashes using the content of a document. As used herein, generating a hash or hashing a document includes any and all methods of creating a virtual, unique representation of the document for identification purposes. Hashes can be used to verify that two or more documents correspond to each other (e.g., they are substantially the same) without conveying personal or other sensitive information to the verifier. In some forms of hashing, portions of text can be selected. Characters or words spaced at regular intervals can be selected to hash the document. In some implementations, an n-gram of words (n is an integer greater than 1) can be identified, and every m-th word of the n-gram can be selected for hashing (m is an integer greater than 1 and less than or equal to n) to form a word string. For example, each of a plurality of word strings can be generated by selecting every other word from the text of the document. Words selected for hashing can be looked up in at least one dictionary to avoid selecting words that constitute personal or other sensitive information. In some implementations, text characters at specific locations can be selected for hashing. For example, the first character of each line of text can be used. In some implementations, numerical information about the document can be converted into a hash. For example, the optical character recognition (OCR) of a document can be converted (e.g., compressed) into digital bits, and sub-sections of the digital information can be selected. In some implementations, hashing can be based on the morphology of the document. For example, hashing can be based on the shape, position, orientation, and / or organization of content within the document. In some implementations, hashing can be based at least in part on one or more characteristics related to the layout or font of the text. For example, fonts or font families, font sizes, glyph spacing such as letter spacing, word spacing, and / or line height can be used. In some implementations, so-called "lazy hashing" can be performed, which is more robust to minor changes made later in the document. For example, hashing can be selectively applied to some, but not all, of the document's content. In some implementations, the hash of a document can include a fingerprint of the document's content. A hash generated for one document does not substantially have the likelihood of conflicting with a hash generated from another document unless one document is a copy of the other. For example, the chance of two unrelated documents having conflicting hashes can be on the order of approximately one in a trillion.

[0030] The examples in this document involve obfuscated content. As used herein, obfuscated content is the content of a document that is not derived from a hash of the document. In some implementations, obfuscated content is content that lacks discernible meaning (e.g., meaningless text or other characters or symbols). Obfuscated content can be included in a hash to substantially eliminate the opportunity to disclose personal or other sensitive information by sending the hash to someone. In some implementations, hashing involves selecting strings of words from a document (e.g., by selecting every other word until a suitable string length is reached). The obfuscated content can then include strings of words of the same length that do not occur in the document. For example, obfuscated strings can include words randomly selected from a dictionary. In some implementations, a hash can include a specific proportion of obfuscated content. For example, when fewer than twelve strings of words are taken from a document, the hash can include obfuscated content of approximately one hundred or more strings of words.

[0031] The examples in this article relate to collaborative programs for document collections. As used herein, a collaborative program allows multiple computer systems to access the same document in a document collection simultaneously. In some implementations, a collaborative program allows users to create, view, and edit one or more files online while collaborating on the files in real time with other users. For example, a collaborative program may be designed for word processing, spreadsheet manipulation, and / or PowerPoint presentations.

[0032] The examples in this article relate to access rights to electronic documents on a computer system. As used herein, a user has access rights to an electronic document if they possess sufficient computer system credentials to be allowed to at least perceive a tangible instance of the document.

[0033] The examples in this article relate to browsers. As used herein, a browser refers to a software application that can retrieve and display information; a browser extension; and / or any suitable combination thereof. A browser can use a Uniform Resource Locator (URL) to retrieve the content of a document (e.g., a document for a collaboration program) and then present at least a portion of the document to the user.

[0034] The examples in this article relate to screen sharing applications. As used herein, a screen sharing application is software that allows a computer system to display at least a portion of its current screen content on the screen of another computer system, substantially in real time. Screen sharing applications can be included in video conferencing applications, to name just one example.

[0035] Figure 1 An example of a computer system 100 is shown. Computer system 100 can be used in conjunction with one or more other examples described elsewhere herein. Computer system 100 can be designed such that at least one processor executes instructions stored in a computer-readable medium. For example, computer system 100 may include the following references Figure 11 Some or all of the components described.

[0036] In some implementations, computer system 100 may be characterized as performing and / or implemented on the client side in a client role. For example, computer system 100 may be an AR head-mounted device (e.g., an AR glasses collection or another AR device) that detects tangible instances of documents and provides their hashes to a server. In some implementations, computer system 100 may be characterized as performing and / or implemented on the server side in a server role. For example, computer system 100 may perform a search based on the received hash; if the server finds a match, it may provide further information to the client.

[0037] Computer system 100 includes input device 102. In some embodiments, input device 102 may include a keyboard or other buttons, a mouse, a touchscreen, one or more sensors, a fingerprint reader, a scanner, a camera or other image sensor, optics, a microphone, a gaze tracking component, an inertial measurement unit, and / or a global positioning system (GPS) sensor. A camera may be used to capture views of tangible instances of a document and to detect its content. A touchscreen may allow a user to generate input as described herein. A microphone may allow a user to input voice commands, such as performing an action or making changes to a document. A gaze tracking component may allow an AR headset to determine where the user is currently looking (e.g., referring to a document). An inertial measurement unit may detect whether computer system 100 is being moved (e.g., to detect proximity to another computer system). A GPS sensor may detect the location of computer system 100 (e.g., to determine the presence of another computer system).

[0038] Computer system 100 may include output device 104. Output device 104 may include a display device, a retinal projector, a haptic component, and / or a speaker. The display device and / or retinal projector may be used to generate visual output that will provide virtual aspects that will be perceived by the user. For example, one or more documents, document editing, controls, pages, windows, and / or desktops may be presented. Speakers may be used to provide audio output, such as audio output generated by a text-to-speech application.

[0039] Computer system 100 may include at least one storage unit 106. Storage unit 106 may include: mapping functionality; AR components; documents; hashes; user identification records of at least one user; access permissions of at least one user; hashing components for hashing content; obfuscated content components for generating obfuscated content; hash combiners for including obfuscated content in hashes; hash comparators for comparing two or more hashes; a collaborative program for document collections; a document editing program (e.g., for non-collaborative documents); a browser; a screen sharing program (e.g., as part of a video conferencing application); an OCR program; a gesture recognition program; and / or, a text-to-speech service application.

[0040] For example, mapping functions can be programmed to perform some or all of the operations described herein (e.g., to provide links between tangible instances and electronic documents, and / or to provide links between two or more tangible instances).

[0041] As another example, AR components can be programmed to operate according to some or all of the examples described herein (e.g., to capture content, and / or send, receive, or present information based on mapping).

[0042] As another example, a document may be captured (e.g., scanned) by computer system 100 and / or received from another computer system.

[0043] As another example, the hash can be generated by computer system 100 and / or received from another computer system.

[0044] As another example, a user identification record can specify who the user is and / or identify one or more other computer systems associated with the user.

[0045] As another example, access permissions can specify whether a user is allowed to access electronic documents on computer system 100 or another computer system.

[0046] As another example, hashing components can hash the content of a document to prevent the leakage of personal information or other sensitive information.

[0047] As another example, obfuscated content components can generate content that is unrelated to the document content in order to avoid the disclosure of personal information or other sensitive information.

[0048] As another example, hash combiners can combine hashed content with obfuscated content to generate hashes that avoid revealing personal or other sensitive information.

[0049] As another example, a hash comparator can determine whether two or more hashes are related to a corresponding document.

[0050] As another example, a collaborative program can allow a user of computer system 100 and another user of a separate computer system to simultaneously access and edit electronic documents.

[0051] As another example, a document editing program can allow users of computer system 100 to edit documents separately from collaborative programs.

[0052] As another example, a browser can allow users of computer system 100 to view documents and / or run programs, whether on computer system 100 locally or from a remote location.

[0053] As another example, a screen sharing program may allow a user of computer system 100 to view a tangible instance of a document shared from another computer system, and / or share a tangible instance of a document with such another computer system.

[0054] As another example, an OCR program can capture content from tangible instances of a document.

[0055] As another example, gesture recognition programs can track the location of a user or part of their body in computer system 100, such as to control an AR headset.

[0056] As another example, a text-to-speech service application can provide voice output to a user of computer system 100 based on the content of a document (e.g., from an electronic document or its tangible instance).

[0057] Computer system 100 may include at least one network interface 108, which allows communication between computer system 100 and one or more other systems and / or devices. Network interface 108 may be configured for wireless and / or wired communication. For example, network interface 108 may facilitate communication for searching for corresponding documents. As another example, network interface 108 may facilitate the detection of computer systems associated with (e.g., in the vicinity of) computer system 100.

[0058] Figure 2 Shown in Figure 1 An example of storage 200 in computer system 100. Storage 200 may be used in conjunction with one or more other examples described elsewhere herein. For example, storage 200 may include Figure 1 Some or all of the components of storage 106 in the middle, or vice versa.

[0059] Storage 200 comprises N documents 202, where N is any integer. Here, documents 202 are schematically shown as document 202-1, document 202-2, ..., and document 202-N, respectively. Each of documents 202 may exist as one or more versions 204. Here, each of documents 202 is schematically shown as having versions 204-1, 204-2, 204-3, ..., and 204-M, respectively, where M is any integer. For example, when a change is made to one of documents 202, the revision can be merged into an existing version of the document, or the document including the change can be considered a new version of an earlier document, or a new document can be created. Other means for version management can be used. In some implementations, a server controlling the collaboration program makes document 202 available to users.

[0060] Storage 200 includes P hashes 206, where P is any integer. Here, hashes 206 are schematically shown as hashes 206-1, 206-2, 206-3, ..., and 206-P, respectively. In some implementations, one or more hashes 206 can be generated based on a tangible instance of a document. The hashes can be provided to another computer system (e.g., to a server) to determine if any other document corresponds to the document on which the hash is based. As another example, a computer system (e.g., a server) can generate a hash of a found document and provide that hash to another computer system (e.g., a client) to allow the other computer system to verify that the found document corresponds to a tangible instance of a document at that other computer system.

[0061] The examples below illustrate how to use the document mapping described in this article to improve document collaboration as an AR experience and other aspects. Figures 3A-3F Examples relating to tangible instances of a document are shown. Any example of a tangible instance can be used in conjunction with one or more other examples described elsewhere herein. In some implementations, a tangible instance may be a hard copy or a presentation on a screen. The document comprises text, which, for simplicity, is represented herein by placeholder Latin text within the tangible instance. The document may be longer than what is currently reflected by the tangible instance (e.g., including more content).

[0062] exist Figure 3AIn this context, the tangible instance 300 is perceived by the user. The AR field of view 302 is schematically represented here using a dashed circle. In some embodiments, the user positioned at the tangible instance 300 is wearing an AR headset that defines the AR field of view 302. Here, the user can see the tangible instance 300 both inside and outside the AR field of view 302 and can perceive virtual aspects within the AR field of view 302, here, controls 304. The content of a document currently within the AR field of view 302 is visible to the computer system of the AR headset and is therefore processable. For example, the AR headset can detect the content (e.g., text) of the tangible instance 300 based on the content within the AR field of view 302. For example, the AR headset includes a display device. The AR headset may include a camera, for example, for capturing the tangible instance. Optionally, the image captured by the camera can be displayed on the display device of the AR headset. Alternatively or additionally, virtual content can be displayed on the display device of the AR headset. For example, the display device of the AR headset can be shifted relative to the tangible instance. For example, in use, the display device of the AR head-mounted device can be positioned between one or both of the user's eyes and a tangible instance in the AR field of view 302.

[0063] Control 304 is generated by the computer system of the AR headset. Control 304 allows a user to trigger a mapping of tangible instance 300. For example, such a mapping can be performed to search for any electronic document that corresponds to tangible instance 300 (e.g., may already be its source). As another example, such a mapping can be performed to search for any other tangible instance of the same document.

[0064] Mapping can be performed without compromising personal or other sensitive information. Before contacting the server regarding tangible instance 300, a hash of the content of tangible instance 300 can be generated, which may also include obfuscated content. Users can activate control 304 by performing gestures within the AR field of view 302 (e.g., with their hands or fingers) or through dedicated input functions on the AR headset, to name just two examples. That is, user activation of control 304 causes the AR headset to send only the hash associated with tangible instance 300 (e.g., its hash and obfuscated content) to the server, which does not include sensitive information.

[0065] A server receiving a hash of tangible instance 300 sent by an AR headset can use the hash in one or more searches. In some implementations, the server searches a set of documents associated with a collaboration program. The scope of the set of documents to be included in the search can be defined based on the user's access permissions. For example, the search may only include documents to which the user has access permissions. As another example, the search may include publicly accessible documents (e.g., documents available to anyone on the Internet) and documents explicitly shared with the user. The search may involve ranking two or more search hits based on the server's confidence that the corresponding search hit is a document corresponding to tangible instance 300. For example, when the hash is a sufficiently unique representation of the document, the search may produce a hit if a corresponding document exists on the server, or no hit if a corresponding document does not exist on the server.

[0066] The server can generate a response to the received hash and send the response to the AR headset. This response may include information corresponding to at least one document found in the hash-based search. In some implementations, the server may perform its own hashing on the found documents. Such hashing may involve generating a hash of the document and including obfuscated content in the hash. For example, the server may perform the same or a different type of hashing as performed by the AR headset.

[0067] The above examples illustrate that a computer-implemented method may include: a first computer system (e.g., an AR head-mounted device) detecting first content (e.g., within an AR field of view 302) of a tangible instance (e.g., tangible instance 300) of a first document; the first computer system (e.g., in response to activation of control 304) generating a first hash using the first content, the first hash including first obfuscated content; the first computer system sending the first hash for a second computer system (e.g., a server) to receive; and the first computer system receiving a response to the first hash generated by the second computer system, the response including information corresponding to a second document associated with the first content (e.g., a hash of the server).

[0068] Upon receiving a response generated by the server, the AR headset can use that response to verify the correspondence between tangible instance 300 and the document found by the server (e.g., the documents are the same). For example, this allows the AR headset to determine whether the server has performed a reliable task in a hash-based search. The AR headset can check the server's hash relative to tangible instance 300.

[0069] After receiving a response generated by the server, the AR headset can offer the user the opportunity to send part or all of the content of the tangible instance 300 to the server. Figure 3BThe illustration shows that an AR headset can present control 306 within an AR field of view 302. A user can activate control 306 by performing a gesture (e.g., using their hand or finger) within the AR field of view 302 or through a dedicated input function on the AR headset, to name just two examples. For instance, activating control 306 could cause the AR headset to send one or more captured images of tangible instance 300 to a server. As another example, text of tangible instance 300 could be sent. In some implementations, the server can generate new electronic documents or new versions of found documents based on content provided by the AR headset. In some implementations, the AR headset sends a marker change of tangible instance 300 to the server, the marker change corresponding to a difference already detected by the AR headset between tangible instance 300 and a document found by the server.

[0070] In some implementations, the server sends the content of the found document to the AR headset. This can be done, for example, in plaintext if the found document is publicly available, or otherwise after the server obtains confirmation on behalf of the AR headset. Such confirmation may include verification that the AR headset user has access to the found document, and / or notification from the AR headset that the found document does indeed correspond to tangible instance 300 and that the AR headset wishes to access the found document.

[0071] Figure 3C An example is shown of an AR headset presenting virtual annotations of a tangible instance 300 within an AR field of view 302. That is, the virtual annotations presented to the user of the AR headset include virtual aspects that enhance the user's perception of the real-world field of view (here, the field of view of the tangible instance 300) by marking changes. In this example, the virtual annotation includes the deletion of a word 308 and the insertion of another word in place of the deleted word 310. In this example, the virtual annotation also includes the highlighting of a paragraph in the tangible instance 300 312 and a comment 314 associated with the highlighted paragraph.

[0072] Some virtual annotations are edits entered by another user into an electronic document that the server has already found based on a search using a hash of tangible instance 300. Gaze tracking can be performed to determine whether the other user is currently focusing on any particular part of the electronic document. Here, gaze indicator 316 can be presented in AR field of view 302 based on this gaze tracking of the other user. That is, in this example, the document found by the server is part of a collection of documents associated with the collaboration program. Note that tangible instance 300 in this example is not a live presentation of a document that the server will find later; rather, tangible instance 300 may already be a hard copy of the document or a presentation on a screen shown in a conference room, to name just two examples. Instead, the mapping performed on tangible instance 300 here allows the user who is perceiving tangible instance 300 to gain access to real-time virtual updates from an electronic document that has already been found to correspond to tangible instance 300.

[0073] In a different scenario than those just described, the document found by the server might not be part of the document set associated with the collaboration program. In some implementations, the server may have already found the document by performing a search using a hash received from an AR headset and another hash received from another computer system. For example, the other computer system might have uploaded its hash based on detecting content from another tangible instance (i.e., not tangible instance 300). Based on the server's determination that the two hashes correspond to each other, the server can assume that the corresponding tangible instances correspond to each other. This can also allow for a mapping to be established between two tangible instances, for example, when the server does not find any electronic document corresponding to a tangible instance.

[0074] The mappings described in this article can improve version management and / or the detection of unauthorized access. For example, suppose tangible instance 300 is an unpublished manuscript of a motion picture being developed by a film studio. The hash of tangible instance 300 could allow the server to detect that a user without access to the film manuscript is currently viewing it.

[0075] Users with AR headsets can perform edits on tangible instance 300. Once a mapping has been established, such edits can be shared with one or more other users, as will be described now. Figure 3DThe illustration shows a user with an AR headset marking the deletion of an expression (318) and the insertion of another expression to replace the deleted expression (320). If the tangible instance 300 is a hard copy, editing of the user's input with the AR headset can be done with a pen; or if the tangible instance 300 is a presentation on a screen, editing of the user's input with the AR headset can be done electronically. If the AR field of view 302 includes a portion of the tangible instance 300 in which editing is performed, sharing of the edits can be performed.

[0076] If an edit is shared with another user, that user can view the edits related to the corresponding tangible instance. Figure 3E-3F This illustrates a tangible instance 300' perceived by another user. Tangible instance 300' is a tangible instance found by the server using a hash-based search based on the example above. From Figure 3E Initially, another user is currently working electronically on the physical instance 300'. Therefore, this other user may not currently be perceiving AR. In this example, the annotation made by the other user on the physical instance 300' includes deleting 308 ( Figure 3C The corresponding deletion 308' and insertion 310' are... Figure 3C The corresponding Insert 310' and Highlight 312 ( Figure 3C The corresponding highlighted section 312' and comment 314 ( Figure 3C (See comment 314'.) These comments from another user were made electronically in the document under tangible instance 300', not in the virtual aspect.

[0077] exist Figure 3D One or more annotations input by the user, which are tangible instances 300, can also be relative to... Figure 3E The tangible instance 300' in the text is visible. Here, such comments include those related to deletion 318 ( Figure 3D The corresponding deletion 318', and the corresponding insertion 320' Figure 3D The corresponding insertion is 320'.

[0078] Turn now Figure 3F Tangible instance 300' here is a hard copy (e.g., a paper printout). Another user perceiving tangible instance 300' here uses an AR headset that defines an AR field of view 302' relative to tangible instance 300'. Another user's comment on tangible instance 300' includes the deletion of 308 (…). Figure 3C The corresponding deletion is 308”, and the insertion is 310”. Figure 3C The corresponding Insert 310” and Highlight 312” are also mentioned. Figure 3C The corresponding highlighted text is 312, and the comment is 314. Figure 3CThe corresponding comment 314”. These comments from another user were made by hand (e.g., using a pen or highlighter) on a hard copy as a tangible instance 300’, rather than in a virtual or electronic form.

[0079] Figure 3D The annotations made in can be displayed in Figure 3F In the AR field of view 302'. Here, such annotations include those related to deletion 318 ( Figure 3D The corresponding deletion 318”, and the insertion 320 ( Figure 3D The corresponding insertion is 320". This can be based on a tangible instance 300 ( Figures 3A-3D The user's gaze is tracked, and the gaze indicator 322 is displayed in the AR field of view 302'.

[0080] The examples above illustrate editing a document electronically in digital form or manually on a hard copy. Other methods of inputting changes to the electronic document, such as editing, annotations, or other markings, or any other commands, can be used. In some implementations, the audio interface can register user-made audio input and interpret it as voice commands to make changes in the electronic document. See, for example, [reference needed]. Figure 3E Instead of using keyboard input to delete '308' and insert '310', the user can say "line six, replace dolor with dolore (Sixth line, replace dolor with dolor)".

[0081] The mapping described in this article can improve access to visual information, for example, for people with low vision. For instance, assuming perception... Figure 3A A tangible instance 300 in the context of a user requesting content for a text-to-speech service. If the text-to-speech service is performed based on the text of the tangible instance 300 detected by the AR headset, the service may need to infer certain structural aspects of the document when generating speech, such as the order in which parallel paragraphs are read, or the position of the end of one paragraph and the beginning of the next. On the other hand, if the tangible instance 300 corresponds to an electronic document, that document may contain structural markup that can be used by the text-to-speech service. Structural markup may include elements according to any suitable markup language, including but not limited to Hypertext Markup Language. The text-to-speech service can then reference the structural concepts reflected by the markup elements of the electronic document to better render the text of the tangible instance 300 as speech.

[0082] Figures 4A-4B The examples shown are conceptually related to tangible instances 400 and 402 and computer system 404. Tangible instances 400 and 402 and computer system 404 can be used in conjunction with one or more other examples described elsewhere herein.

[0083] exist Figure 4A In this embodiment, computer system 404 includes an electronic document 406 corresponding to each of tangible instances 400 and 402. For example, each of tangible instances 400 and 402 is a hard copy or a screen representation of electronic document 406. Computer system 404 may have already identified electronic document 406 after receiving a hash from the computer system associated with tangible instance 400; similarly, computer system 404 may have already identified electronic document 406 after receiving a hash from the computer system associated with tangible instance 402. Therefore, each of tangible instances 400 and 402 is mapped to electronic document 406, and tangible instances 400 and 402 are linked to each other via electronic document 406. Link 408 can be used for identification and / or information transfer in one or both directions between tangible instances 400 and 402.

[0084] exist Figure 4B In this context, computer system 404 does not necessarily include any electronic documents corresponding to each tangible instance 400 and 402. Here, the computer system associated with tangible instance 400 has provided computer system 404 with a hash 410 for tangible instance 400. Similarly, the computer system associated with tangible instance 402 has provided computer system 404 with a hash 412 for tangible instance 402. Therefore, tangible instance 400 is mapped to hash 410, and tangible instance 402 is mapped to hash 412. Furthermore, computer system 404 can determine that hashes 410 and 412 correspond to each other (e.g., because they are the same). That is, computer system 404 can use hashes 410 and 412 to detect that tangible instance 402 is associated with tangible instance 400. Tangible instances 400 and 402 are thus linked 414 to each other through hashes 410 and 412. Link 414 can be used for identification and / or information transfer in either or both directions between tangible instances 400 and 402. That is, link 414 also provides a real-time dynamic connection between tangible instances 400 and 402 when they are hard copies (e.g., paper documents). For example, link 414 can facilitate the sharing of virtual annotations about tangible instances 400 or 402 between their respective computer systems.

[0085] The mapping described herein can improve the process of visually recording information by providing convenient ways to electronically capture the content of tangible instances. Figure 5An example related to storage drive 500 is shown. In some implementations, storage drive 500 can be provided as a central storage solution for files and folders of multiple users (e.g., a file backup service) and can allow access to files using various types of devices. For example, storage drive 500 can be made available in the cloud.

[0086] In storage drive 500, a user can activate control 502 to see the names of their files in pane 504. Control 506 can be used to access another drive different from storage drive 500 (e.g., a drive that does not belong to the user but is shared with them). Control 508 can be activated to see the history of documents recently accessed by the user.

[0087] Suppose a user controlling storage drive 500 begins to have paper documents that the user is deciding whether to keep or recycle. For example, the paper document could be a user manual for a product the user has already purchased. Instead of recycling the user manual and hoping it will no longer be needed in the future, the user could wear an AR headset and quickly flip through parts or all of the paper document. Similar to the reference above... Figures 3A-3B As described in the example, the AR headset can capture the contents of a user manual and provide that information to a server, in this example, a server that controls the storage drive 500. Therefore, when the user subsequently accesses the storage drive 500, pane 504 can include items 510 corresponding to the user manual the user is referring to, thus providing a convenient way to maintain that information.

[0088] The mapping described herein can improve access to visual information previously viewed by the user, for example, by providing a record of tangible instances that the user has already seen. Figure 6A It shows the relationship with Figure 5 Examples related to the 500 storage drive. See again briefly. Figure 3AThis example describes an AR headset that provides a hash of a tangible instance 300 to a server. This notifies the server that a user associated with the AR headset has perceived the tangible instance 300. This example relates to a situation where the server determines that a user has access to a document that the server has found using the hash. Therefore, the server adds the identifier of the tangible instance 300 as an entry in the user's document access history. In some implementations, storage drive 500 provides a control 508 in which the user can see a list of recently viewed documents. Upon activation of control 508, pane 504 can present the corresponding entries in the access history, including, here, entry 512 corresponding to the tangible instance 300. That is, because the user is wearing the AR headset while viewing the tangible instance 300 (e.g., a printed document or a presentation on a screen), storage drive 500 is updated to reflect the tangible instance 300 in the user's access history. In some implementations, entry 512 may be based on detecting that the user is near (e.g., viewing) the tangible instance 300. For example, the tangible instance 300 may be a hard copy document or a presentation of an electronic document on a screen.

[0089] exist Figure 3A In such cases, if the user does not have access to the electronic document found by the server using a hash, additional interactions may occur. Figure 6B This indicates that the user is viewing tangible instance 300 and AR field of view 302. As described above, when searching using a hash provided by the AR headset, the server identifies the corresponding electronic document and determines that the user does not have access rights to it. The server can now cause the AR headset to display a prompt 600, informing the user that they do not have access rights to the electronic document corresponding to tangible instance 300. Prompt 600 may provide the user with one or more actions that can be performed. In some implementations, control 602 allows the user to submit a request for permission to access the electronic document. For example, the server may forward such a request for consideration by the person who is the owner or otherwise controls the electronic document. If access rights are granted to the user, the server can then proceed according to the above reference. Figures 3A-3F One or more of the described examples continue. In some implementations, control 604 allows the user to log in to a different account that the user may have on the server; if the new account has access to electronic documents, the server can proceed according to the above reference. Figures 3A-3F One or more of the examples described continue.

[0090] In some implementations, practical document management techniques can be applied to improve information protection. Hypothetically, in an organization, the owner or controller of electronic documents may frequently be overwhelmed by a large number of access requests—such as those mentioned above. Furthermore, contrary to the organization's document protection policy, this person might choose to change a document from protected to freely accessible simply because they are unable to address all incoming access requests in a timely manner.

[0091] The practical methods mentioned above can be based on the following identification: Those who can perceive a tangible instance of a document have actually acquired at least minimal access to the document, even if they are not yet formally authorized to access it within the corresponding computer system. See again briefly... Figure 3A A user wearing an AR headset is currently facing a tangible instance 300 and is likely able to perceive its content. If the user is only present at the tangible instance 300 for a short period, they may have inadvertently come into contact with it and have no reason or opportunity to fully or deeply perceive its content. In this case, changing the user's status to not have access to the corresponding electronic document might be unreasonable. However, if it is determined that the user has faced (e.g., viewed using the AR headset) the tangible instance 300 for at least a predetermined time, then a better course of action would be to automatically grant the person access to the corresponding electronic document without requiring a formal request. The length of time considered sufficient can be defined based on the nature of the information in the document, the size or complexity of the document, the type of organization, or the user's role within the organization. When the server determines that the user has accessed (e.g., faced) the tangible instance 300 for at least a predetermined time, the server can automatically authorize the user to access the corresponding electronic document without presenting a prompt 600.

[0092] The mapping described in this article can facilitate easier access to collaboration programs for document collections, such as by providing convenient ways to find and access collaboration documents based on tangible instances of perceived collaboration documents. Figures 7A-7B It shows the relationship with Figures 3A-3E Examples related to tangible instance 300 and AR field of view 302. The examples described with reference to this illustration can be used in conjunction with one or more other examples described elsewhere in this document.

[0093] These examples involve the case where tangible instance 300 corresponds to a document in a document collection associated with the collaboration program. Figure 7A In the image, a user wearing an AR headset is currently viewing tangible instance 300 within AR field of view 302. (See again briefly.) Figure 3BIn this scenario, when the server determines, based on the provided hash, that a user has access to the corresponding electronic document, the server can make one or more additional functions available to the user. In some implementations, control 700 can be provided by an AR head-mounted device, such as by being visually included in the AR field of view 302. The user can activate control 700 to gain access to the corresponding electronic document.

[0094] In some implementations, a reference can be triggered by performing a gesture. Figure 7A The described functionality. For example, a user can perform a gesture within the AR field of view 302. The AR headset can interpret this input as a request to "drag" a corresponding electronic document from a tangible instance 300 viewed using the AR headset. Dragging can be spatially controlled by the positioning and movement of the user's hand or fingers. Figure 7B An image is shown where a user performs a gesture to drag a shape 702 away from a tangible instance 300 within the AR field of view 302. That is, shape 702 is rendered by the AR headset; on the other hand, tangible instance 300 is visible within the AR field of view 302 but is not rendered by the AR headset. Control 704 corresponds to the computer system of the AR headset. For example, a user can perform a gesture at control 704 corresponding to "dropping" shape 702. This allows the computer system of the AR headset to create access to a corresponding electronic document for a collaborative program. Access can be created in the form of an electronic document representation (e.g., a link or icon) on the graphical user interface controlling the computer system of the AR headset. Once access has been created, the user can open the corresponding electronic document on the computer system using the collaborative program.

[0095] Furthermore, control 706 corresponds to a nearby computer system detected by the AR headset. For example, via near-field communication, the AR headset can determine that a user's tablet or laptop is nearby, and this allows the AR headset to recognize the tablet or laptop as another computer system associated with the user. The user can perform a gesture at control 706 corresponding to the "drop" shape 702. This can trigger access to the corresponding electronic document of a collaborative program to be created on that other computer system. Access on the other computer system can be created on a graphical user interface in the form of an electronic document (e.g., a link or icon). Once access has been created, the user can use the collaborative program to open the corresponding electronic document on the other computer system.

[0096] The mapping described in this article can also, or alternatively, facilitate collaborative programs for document collections outside the AR domain by providing cross-computer document management. Figures 8A-8E It shows the relationship with Figures 3A-3EExamples relating to tangible instance 300 and display device 800. The examples described with reference to display device 800 may be used in conjunction with one or more other examples described elsewhere in this document.

[0097] Display device 800 is controlled by a computer system currently displaying screen sharing application 802. In some embodiments, a user of the computer system is currently participating in a video conference with another person using a separate computer system, and that person is sharing content using screen sharing application 802. For example, the person is sharing an electronic document from a separate computer system, which is represented as a tangible instance 804 on display device 800.

[0098] The computer system of display device 800 can detect the content of tangible instance 804 based on the content of tangible instance 804 being presented on display device 800. The computer system can provide a hash to a server controlling the collaborative program. In response to the server using the provided hash to determine that a user has access rights to the corresponding electronic document, one or more additional functions can be made available to the user. Figure 8B The illustration shows a user dragging a shape 806 representing a corresponding electronic document from a tangible instance 804 of a screen-sharing application 802 and placing the shape 806 on the desktop of a display device 800. This allows the computer system of the display device 800 to create access to the corresponding electronic document in the collaborative program. Figure 8C A representation 808 is shown of an electronic document located on the desktop. Representation 808 may include a link or icon that triggers a collaboration program to display the corresponding electronic document on the computer system.

[0099] Applications executed by computer programs of display device 800 may also, or alternatively, facilitate access to collaborative programs based on the detected tangible instance 804. Figure 8D The computer system of display device 800 presents browser 810. Browser 810 can detect the content of tangible instance 804 and provide one or more functions to the user in response. In some embodiments, browser 810 presents controls 812 for creating access to a corresponding electronic document of a collaborative program at the computer system of display device 800. This can represent 808 ( Figure 8C Place it on the desktop.

[0100] In some implementations, browser 810 presents control 814 for creating access to a corresponding electronic document for a collaborative program on another computer system. This other computer system may be a nearby computer system detected by the computer system of display device 800. For example, the computer system of display device 800 may determine that a user's tablet or laptop is nearby via near-field communication, and this may cause the computer system of display device 800 to recognize the tablet or laptop as another computer system associated with the user. Activation of control 814 may result in the creation of access to a corresponding electronic document for a collaborative program on that other computer system. Figure 8E Another system is shown with a display device 816, and a representation 818 of an electronic document is located on its desktop. The representation 818 may include a link or icon that triggers a collaboration program to display the corresponding electronic document on the other computer system.

[0101] As previously mentioned, mappings as described in this article can facilitate collaborative programs for document collections, such as by providing easier access to document management across computers. Figures 9A-9B It shows the relationship with Figures 8A-8E Examples related to display device 800 are provided. Display device 800 can be controlled by a computer system running a specific operating system, while another display device 900 can be controlled by separate computer systems running different operating systems. The operating system can be, for example, Chrome OS or Android designed by Google, Windows designed by Microsoft, macOS or iOS designed by Apple, or an open-source operating system (such as Linux).

[0102] Here, the tangible instance 300 is presented by the display device 800 and is currently within the AR field of view 302. Figure 9A This image shows a user dragging shape 902 away from tangible instance 300 within the AR field of view 302. For example, the user drags shape 902 by performing a gesture within the AR field of view 302, which is detected and interpreted by the AR headset. That is, shape 902 is rendered by the AR headset; on the other hand, tangible instance 300 is visible within the AR field of view 302 but is rendered by the display device 800.

[0103] The shape 902 rendered within the AR field of view 302 by the AR headset allows the user to look directly at the display device 900, such as... Figure 9BAs indicated. For example, a user can turn his or her head so that the AR field of view 302 is instead pointed at the display device 900. As another example, the AR field of view 302 can be large enough to accommodate both display devices 800 and 900. The AR headset is connected to each of the corresponding computer systems of display devices 800 and 900. For example, a near-field communication network can couple the AR headset and the computer systems to each other. The user can “place” shape 902 at display device 900 by performing a gesture in front of their desktop. Based on the AR headset’s knowledge of the identity, location, and network address of the computer system of display device 900, the AR headset can place a representation 904 of an electronic document on the desktop of display device 900. Representation 904 may include links or icons that trigger collaborative programs to display the corresponding electronic document on display device 900.

[0104] Figure 10 An example of a computer-implemented method 1000 is shown. The computer-implemented method 1000 may be used in conjunction with one or more examples described elsewhere herein. Unless otherwise indicated, more or fewer operations may be performed than shown, and / or two or more operations may be performed in a different order.

[0105] The computer-implemented method 1000 may include an operation 1002 of detecting first content of a tangible instance of a first document by a first computer system. For example, a computer system 100 operating on the client side ( Figure 1 An instance of ) can detect the contents of tangible instance 300 (Figure 3).

[0106] The computer-implemented method 1000 may include an operation 1004 in which a first computer system generates a first hash using first content, the first hash including first obfuscated content. For example, a computer system 100 operating on the client side ( Figure 1 An instance of ) can generate a hash of the contents of tangible instance 300 (Figure 3).

[0107] The computer-implemented method 1000 may include an operation 1006 in which a first computer system sends a first hash for a second computer system to receive. For example, the computer system 100 operating on the client side ( Figure 1 An instance of ) can send a hash of the contents of tangible instance 300 (Figure 3) to a computer system 100 operating on the server side. Figure 1 ) instance reception.

[0108] The computer-implemented method 1000 may include an operation 1008 in which a first computer system receives a response to a first hash generated by a second computer system, the response including information corresponding to a second document associated with the first content. For example, the computer system 100 operating on the client side ( Figure 1 An instance of ) can receive data from a computer system 100 operating on the server side. Figure 1 The response generated by the instance of ).

[0109] The computer-implemented method 1000 may include an operation 1010 in which a first computer system sends first content based on a response for a second computer system to receive. For example, the computer system 100 operating on the client side ( Figure 1 An instance of ) can send the contents of tangible instance 300 (Figure 3) to a computer system 100 operating on the server side. Figure 1 ) instance reception.

[0110] Figure 11 Examples of computer devices and mobile computer devices that can be used to implement the technologies described herein are shown. Figure 11 Examples of general-purpose computing device 1100 and general-purpose mobile computing device 1150 are shown, which can be used with the techniques described herein. Computing device 1100 is intended to represent various forms of digital computers, such as notebook computers, desktop computers, tablets, workstations, personal digital assistants, televisions, servers, blade servers, mainframes, and other suitable computing devices. Computing device 1150 is intended to represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely exemplary and are not intended to limit the implementation of the inventions described and / or claimed in this document.

[0111] The computing device 1100 includes at least one processor 1102, a memory 1104, a storage device 1106, a high-speed interface 1108 connected to the memory 1104 and a high-speed expansion port 1110, and a low-speed interface 1112 connected to a low-speed bus 1114 and the storage device 1106. The processor 1102 may be a semiconductor-based processor. The memory 1104 may be a semiconductor-based memory. Each of the components 1102, 1104, 1106, 1108, 1110, and 1112 is interconnected using various buses and may be mounted on a common motherboard or otherwise suitably mounted. The processor 1102 can process instructions for execution within the computing device 1100, including instructions stored on the memory 1104 or storage device 1106 for displaying graphical information for a GUI on an external input / output device—such as a display 1116 coupled to the high-speed interface 1108. In other embodiments, multiple processors and / or multiple buses, as well as multiple memories and various memory types, may be suitably used. In addition, multiple computing devices 1100 can be connected, each providing a portion of the necessary operation (e.g., as a server library, a set of blade servers, or a multiprocessor system).

[0112] The memory 1104 stores information within the computing device 1100. In one embodiment, the memory 1104 is one or more volatile memory cells. In another embodiment, the memory 1104 is one or more non-volatile memory cells. The memory 1104 may also be another form of computer-readable medium, such as a magnetic disk or optical disk.

[0113] Storage device 1106 provides mass storage for computing device 1100. In one embodiment, storage device 1106 may be or include computer-readable media such as floppy disk devices, hard disk devices, optical disk devices, or magnetic tape devices, flash memory or other similar solid-state storage devices, or device arrays, including devices in storage area networks or other configurations. A computer program product may be tangibly embodied in an information carrier. The computer program product may also contain instructions that, when executed, perform one or more methods, such as those described above. The information carrier is a computer or machine-readable medium, such as memory 1104, storage device 1106, or memory on processor 1102.

[0114] High-speed controller 1108 manages bandwidth-intensive operations of computing device 1100, while low-speed controller 1112 manages lower bandwidth-intensive operations. This functional allocation is merely exemplary. In one embodiment, high-speed controller 1108 is coupled to memory 1104, display 1116 (e.g., via a graphics processor or accelerator), and high-speed expansion port 1110, which can accept various expansion cards (not shown). In another embodiment, low-speed controller 1112 is coupled to storage device 1106 and low-speed expansion port 1114. The low-speed expansion port, which may include various communication ports (e.g., USB, Bluetooth, Ethernet, Wireless Ethernet), may be coupled to one or more input / output devices, such as keyboards, pointing devices, scanners, or networking devices—such as switches or routers—for example, via a network adapter.

[0115] As shown in the figure, computing device 1100 can be implemented in a variety of different forms. For example, it can be implemented as a standard server 1120, or multiple times in a group of such servers. It can also be implemented as part of a rack server system 1124. Furthermore, it can be implemented in a personal computer—such as a laptop computer 1122. Alternatively, components from computing device 1100 can be combined with other components in mobile devices (not shown), such as device 1150. Each of such devices can contain one or more of computing devices 1100, 1150, and the entire system can consist of multiple computing devices 1100, 1150 communicating with each other.

[0116] Computing device 1150 includes at least one processor 1152, memory 1164, input / output devices such as a display 1154, a communication interface 1166, and a transceiver 1168, as well as other components. Device 1150 may also be equipped with storage devices, such as microdrives or other devices, to provide additional storage. Each of components 1150, 1152, 1164, 1154, 1166, and 1168 is interconnected using various buses, and several of the components may be mounted on a common motherboard or otherwise suitably mounted.

[0117] Processor 1152 can execute instructions within computing device 1150, including instructions stored in memory 1164. The processor can be implemented as a chipset comprising multiple separate analog and digital processors. The processor can provide, for example, coordination of other components of device 1150, such as control of the user interface, applications running by device 1150, and wireless communication via device 1150.

[0118] Processor 1152 can communicate with the user via control interface 1158 and display interface 1156, which are coupled to display 1154. Display 1154 can be, for example, a TFT LCD (Thin Film Transistor Liquid Crystal Display) or OLED (Organic Light Emitting Diode) display or other suitable display technology. Display interface 1156 may include suitable circuitry for driving display 1154 to present graphics and other information to the user. Control interface 1158 can receive commands from the user and translate them for submission to processor 1152. Additionally, external interface 1162 may be provided to communicate with processor 1152 to enable near-field communication between device 1150 and other devices. External interface 1162 may provide wired communication in some embodiments, or wireless communication in others, and multiple interfaces may be used.

[0119] Memory 1164 stores information within computing device 1150. Memory 1164 can be implemented as one or more computer-readable media, one or more volatile memory cells, or one or more non-volatile memory cells. Extended memory 1174 can also be provided and connected to device 1150 via an extended interface 1172, which may include, for example, a SIMM (Single In-line Memory Module) card interface. Such extended memory 1174 can provide additional storage space for device 1150 or store applications or other information for device 1150. Specifically, extended memory 1174 may include instructions for performing or supplementing the above processes and may also include security information. Therefore, for example, extended memory 1174 can be provided as a security module of device 1150 and can be programmed using instructions that allow secure use of device 1150. Furthermore, secure applications and additional information, such as placing identification information on the SIMM card in an unbreakable manner, can be provided via a SIMM card.

[0120] The memory may include, for example, flash memory and / or NVRAM memory, as discussed below. In one embodiment, the computer program product is tangibly embodied in an information carrier. The computer program product contains instructions that, when executed, perform one or more methods, such as those described above. The information carrier is a computer or machine-readable medium, such as memory 1164, extended memory 1174, or memory on processor 1152, which may be received, for example, via transceiver 1168 or external interface 1162.

[0121] Device 1150 can communicate wirelessly via communication interface 1166, which may include a digital signal processing circuitry system if necessary. Communication interface 1166 can provide communication under various modes or protocols, such as GSM voice calls, SMS, EMS or MMS messaging, CDMA, TDMA, PDC, WCDMA, CDMA2000, or GPRS. Such communication can occur, for example, via radio frequency transceiver 1168. Furthermore, short-range communication can occur using transceivers such as Bluetooth, Wi-Fi, or others (not shown). Additionally, GPS (Global Positioning System) receiver module 1170 can provide device 1150 with additional navigation and location-related wireless data, which can be used as appropriate by applications running on device 1150.

[0122] Device 1150 can also use audio codec 1160 for audio communication, which can receive spoken information from a user and convert it into usable digital information. Audio codec 1160 can similarly generate audible sounds for the user, such as through a speaker in the mobile phone of device 1150. Such sounds can include sounds from voice phone calls, recorded sounds (e.g., voice messages, music files, etc.), and sounds generated by applications operating on device 1150.

[0123] As shown in the figure, the computing device 1150 can be implemented in a variety of different forms. For example, it can be implemented as a cellular phone 1118. It can also be implemented as part of a smartphone 1182, a personal digital assistant, or other similar mobile devices.

[0124] Various implementations of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, specially designed ASICs (Application-Specific Integrated Circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include implementations in one or more computer programs executable and / or interpretable on a programmable system, which includes at least one programmable processor, which may be specialized or general-purpose, coupled to receive data and instructions from a storage system, at least one input device, and at least one output device, and to transmit data and instructions to the storage system, at least one input device, and at least one output device.

[0125] These computer programs (also referred to as programs, software, software applications, or code) include machine instructions for a programmable processor and can be implemented in high-level procedural and / or object-oriented programming languages ​​and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" mean any computer program product, apparatus, and / or device (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, which includes a machine-readable medium that receives machine instructions as machine-readable signals. The term "machine-readable signal" means any signal used to provide machine instructions and / or data to a programmable processor.

[0126] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0127] The systems and technologies described herein can be implemented in computing systems that include back-end components (such as data servers), middleware components (such as application servers), front-end components (such as client computers having a graphical user interface or web browser through which users can interact with implementations of the systems and technologies described herein), or any combination of such back-end, middleware, or front-end components. Components of the system can be interconnected via digital data communication (e.g., a communication network) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), and the Internet.

[0128] A computing system may include clients and servers. Clients and servers are typically geographically separated and usually interact through a communication network. The relationship between clients and servers is generated by computer programs running on the respective computers and having a client-server relationship with each other.

[0129] In some implementations... Figure 11 The computing device depicted may include sensors connected to a virtual reality (VR headset 1190) interface. For example, in computing device 1150 or... Figure 11One or more sensors included on other computing devices depicted can provide input to the VR headset 1190, or generally, to the VR space. Sensors may include, but are not limited to, touchscreens, accelerometers, gyroscopes, pressure sensors, biometric sensors, temperature sensors, humidity sensors, and ambient light sensors. The computing device 1150 can use sensors to determine its absolute position and / or detected rotation in the VR space, which can then be used as input to the VR space. For example, the computing device 1150 can be incorporated into the VR space as a virtual object—such as a controller, laser pointer, keyboard, weapon, etc. The user's positioning of the computing device / virtual object when incorporating it into the VR space allows the user to position the computing device to view the virtual object in a specific way within the VR space. For example, if the virtual object represents a laser pointer, the user can manipulate the computing device as if it were an actual laser pointer. The user can move the computing device left and right, up and down, in circles, etc., and use the device in a manner similar to using a laser pointer.

[0130] In some implementations, one or more input devices included on or connected to computing device 1150 can be used as input to the VR space. Input devices may include, but are not limited to, touchscreens, keyboards, one or more buttons, trackpads, touchpads, pointing devices, mice, trackballs, joysticks, cameras, microphones, headphones or earbuds with input functionality, game controllers, or other connectable input devices. When the computing device is incorporated into the VR space, a user interacting with the input devices included on computing device 1150 can cause specific actions to occur in the VR space.

[0131] In some implementations, the touchscreen of computing device 1150 can be rendered as a touchpad in VR space. Users can interact with the touchscreen of computing device 1150. For example, in VR headset 1190, the interaction is rendered as movement on a touchpad rendered in VR space. The rendered movement can control objects in VR space.

[0132] In some implementations, one or more output devices included on the computing device 1150 may provide output and / or feedback to a user of the VR headset 1190 in the VR space. The output and feedback may be visual, tactile, or audio. The output and / or feedback may include, but is not limited to, vibration, turning on and off or flashing and / or blinking one or more lights or strobe lights, emitting alarm sounds, playing ringtones, playing songs, and playing audio files. Output devices may include, but are not limited to, vibration motors, vibration coils, piezoelectric devices, electrostatic devices, light-emitting diodes (LEDs), strobe lights, and speakers.

[0133] In some implementations, computing device 1150 may appear as another object in a computer-generated 3D environment. User interactions with computing device 1150 (e.g., rotating, shaking, touching a touchscreen, swiping a finger across the touchscreen) can be interpreted as interactions with objects in VR space. In the example of a laser pointer in VR space, computing device 1150 manifests as a virtual laser pointer in the computer-generated 3D environment. When a user manipulates computing device 1150, the user in VR space perceives the laser pointer as moving. The user receives feedback from interactions with computing device 1150 in VR space on computing device 1150 or VR headset 1190.

[0134] Several embodiments have been described. However, it will be understood that various modifications can be made without departing from the spirit and scope of the invention.

[0135] Furthermore, the logical flow depicted in the figures does not require the specific order or sequence shown to achieve the desired result. Additionally, other steps may be provided, or steps may be removed from the described flow, and other components may be added to or removed from the described system. Therefore, other embodiments are within the scope of the appended claims.

Claims

1. A computer-implemented method comprising: detecting, by a first computer system, first content of a hardcopy instance or onscreen presentation instance of a first document; generating, by the first computer system, a first hash using the first content, the first hash comprising content not originating from the first document; sending, by the first computer system, the first hash for lookup against a collection of documents controlled by a collaboration program; and receiving, by the first computer system, a response to the first hash, the response comprising information corresponding to a second document from the collection of documents that is associated with the first content, and the response comprising a second hash generated using second content of the second document, the second hash comprising content not originating from the second document; and verifying, by the first computer system, a correspondence between the first document and the second document using the second hash.

2. The computer-implemented method of claim 1, further comprising: sending, by the first computer system, the first content based on the response for a second computer system to receive.

3. The computer-implemented method of claim 2, wherein, the second computer system generating a new document for the collaboration program using the first content.

4. The computer-implemented method of claim 1, further comprising: sending, by the first computer system, a markup change of the first document based on the response for a second computer system to receive.

5. The computer-implemented method of claim 1, further comprising: receiving, by the first computer system, the second document from a second computer system.

6. The computer-implemented method of claim 1, wherein, the second computer system performing detection of unauthorized access to the second content upon receiving the first hash.

7. The computer-implemented method of claim 1, further comprising: receiving, by the first computer system, an access history of a user of the first computer system, the history comprising an entry for accessing the second document based on the second computer system having received the first hash.

8. The computer-implemented method of claim 7, wherein, the entry for accessing the second document is based on detecting that the user is in proximity to at least one of a hardcopy of the first document or a screen presentation of the first document.

9. The computer-implemented method of claim 7, wherein, the second computer system generating the entry in response to determining that the user has access to the second document.

10. The computer-implemented method of claim 7, wherein, the user does not have access to the second document, and wherein the information corresponding to the second document comprises a control for the user to request access to the second document.

11. The computer-implemented method of claim 7, wherein, the user does not have access to the second document, and wherein the second computer system grants the user access in response to determining that the user is accessing the first document for at least a predetermined time.

12. The computer-implemented method of claim 1, wherein, the first computer system identifying a user of the first computer system to a second computer system, and wherein the collection of documents is defined based on the user having access to the collection of documents.

13. The computer-implemented method of claim 1, further comprising: receiving, by the first computer system, a request from a user for a text-to-speech service of the first document, wherein the second document contains structural markup, and providing the text-to-speech service of the first document using the second document.

14. The computer-implemented method of claim 1, wherein, the first computer system detecting the first content based on the first document being presented on a display device.

15. The computer-implemented method of claim 14, wherein, The display device is controlled by the first computer system and presents a screen sharing application on a desktop, wherein the first document is shared for screen sharing with the first computer system using the screen sharing application.

16. The computer-implemented method of claim 15, wherein, The first computer system further presents a browser on the desktop, and wherein the browser provides a control for opening the second document on the desktop using the collaboration program.

17. The computer-implemented method of claim 15, further comprising: Dragging, facilitated by the first computer system, of a representation of the first document using the collaboration program to trigger opening of the second document.

18. The computer-implemented method of claim 17, wherein, The dragging causes the second document to be presented on the desktop of the first computer system.

19. The computer-implemented method of claim 17, wherein, The dragging causes the second document to be presented on another computer system based on detection, by the first computer system, of the other computer system.

20. The computer-implemented method of any one of claims 1 to 19, wherein, The first computer system comprises an augmented reality (AR) headset, and wherein the first computer system detects the first content based on the first content being within a field of view of the AR headset.

21. The computer-implemented method of claim 20, further comprising: A gesture is detected, by the first computer system, of a user of the AR headset being performed within the field of view of the AR headset, and in response the representation of the first document is moved within the field of view of the AR headset according to the gesture.

22. The computer-implemented method of claim 21, wherein, The movement of the representation facilitates opening of the second document on the first computer system using the collaboration program for the set of documents.

23. The computer-implemented method of claim 21, wherein, The movement of the representation facilitates opening of the second document on another computer system detected by the first computer system using the collaboration program for the set of documents.

24. The computer-implemented method of any one of claims 1 to 19, further comprising: The first computer system presents second content of the second document using the information.

25. The computer-implemented method of claim 24, wherein, The first computer system comprises an augmented reality (AR) headset, wherein the first computer system detects the first content based on the first content being within a field of view of the AR headset, and wherein presenting the second content comprises applying a first virtual annotation to the first document.

26. The computer-implemented method of claim 25, further comprising: An audio input is received, by the first computer system, generated by a user corresponding to a change made in the first document, a second virtual annotation is presented for the first document in the field of view of the AR headset, and the change is sent to a second computer system.

27. The computer-implemented method of any one of claims 1 to 19, wherein, The second document is publicly accessible, and wherein the response comprises the second document.

28. The computer-implemented method of any one of claims 1 to 19, wherein, The first and second documents are paper documents, and wherein a second computer system uses the first and second hashes of the second document to detect the second document associated with the first document.

29. The computer-implemented method of claim 28, wherein, The first and second hashes facilitate sharing of virtual annotations regarding the first or second document between the first and second computer systems.

30. A computer program product tangibly embodied in a non-transitory storage medium, the computer program product including instructions that, when executed by one or more processors, cause the one or more processors to perform the method of any of claims 1-29.

31. A computing device comprising: one or more processors; and memory storing instructions that, when executed by the one or more processors, cause the one or more processors to perform the method of any of claims 1-29. ​

Images