Extensible framework for executable annotation of electronic content

Abstract

Embodiments of the present disclosure relate to an extensible framework for executable annotation of electronic content. Insertion of executable commands or instructions in the form of annotating electronic content enables recipients to benefit from more accurate and detailed context when they interpret and / or take actions on the senders' requests. The system may be configured, upon detection of an appropriate trigger, to "translate" or translate annotations across multiple applications and platforms, and perform one or more tasks that support recipient access and interaction with items. As an electronic content item is shared, senders enjoy a manner of finer control of the electronic content item, presentation, and overall context, and can allow recipients to more accurately understand their own purpose of sharing the item. Similarly, a recipient may be directed to a desired or expected action or response of the received electronic content item.

Description

[0001] Related Applications Citations

[0002] This application is a divisional application of the invention patent application with international application number PCT / US2019 / 068881, international application date December 30, 2019, date of entry into the Chinese national phase July 6, 2021, invention name “Extensible framework for executable annotations of electronic content”, and Chinese national application number 201980088093.9. Technical Field

[0003] Embodiments of the present disclosure relate to an extensible framework for executable annotation of electronic content. Background Art

[0004] Computing devices including cameras and multimedia content support applications are becoming increasingly common in mobile devices, including laptops, tablet computers, digital cameras, smart phones, and other mobile data, messaging, and / or communication devices. Typically, users use cameras associated with computing devices to take various photos, such as images of landscapes, people, presentations, whiteboards, business cards, documents, sketches, drawings, etc. Users can refer to the captured images to recall the information contained therein (e.g., charts, pictures, lists, and other text) and / or electronically transmit them to other users, storage services, or devices.

[0005] Multimedia content is commonly shared between mobile users via Multimedia Messaging Service (MMS), and multimedia content is shared between Internet users and mobile users via email. However, sharing multimedia content (e.g., photos, video clips, audio clips, data, etc.) stored in a device with friends, colleagues, and other family members remains relatively delicate and cumbersome. Users may have specific goals related to content sharing that are lost in transit, or require time-intensive efforts to convey. Therefore, there remain significant new and improved ideas for effective and meaningful sharing of electronic content, and management of the types of responses expected by recipients. Summary of the invention

[0006] According to a first aspect of the present disclosure, a data processing system includes at least one processor and one or more computer-readable media. The computer-readable medium includes instructions executed by at least one processor, causing the at least one processor to: present a first user interface for a first content viewing application on a first client device through a communication network, the first user interface displays an option for associating a first tag with a first electronic content, and receive a first user input from a first user through the communication network, the first user input is used to select an option to associate a first tag with a first electronic content, and the first tag represents a first executable process for performing a first task. In addition, the instructions also cause the at least one processor to: record an association between the first electronic content and the first tag in response to receiving the first user input. In addition, the instructions cause the at least one processor to transmit the first electronic content to the first recipient through the communication network, and execute the first executable process in response to transmitting the first electronic content.

[0007] According to a second aspect of the present disclosure, a method performed on one or more computing devices includes presenting a first user interface for a first content viewing application on a first client device via a communication network, the first user interface displaying an option for associating a first tag with a first electronic content. The method also includes receiving a first user input from a first user via the communication network, the first user input being used to select an option to associate a first tag with a first electronic content, the first tag representing a first executable process for performing a first task. In addition, the method includes recording an association between the first electronic content and the first tag in response to receiving the first user input. In addition, the method includes transmitting the first electronic content to a first recipient via the communication network, and executing a first executable process in response to transmitting the first electronic content.

[0008] According to a third aspect of the present disclosure, a data processing system includes a device for presenting a first user interface for a first content viewing application on a first client device via a communication network, the first user interface displaying an option for associating a first tag with a first electronic content. The system also includes a device for receiving a first user input from a first user via the communication network, the first user input being used to select an option to associate a first tag with a first electronic content, the first tag representing a first executable process for performing a first task. In addition, the system also includes a device for recording an association between the first electronic content and the first tag in response to receiving the first user input. The system also includes a device for transmitting the first electronic content to a first recipient via the communication network, and executing a first executable process in response to transmitting the first electronic content.

[0009] This summary is provided to introduce some concepts in a simplified form, which will be further described in the detailed description below. This summary is neither intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. Furthermore, the claimed subject matter is not limited to implementations that solve any or all of the disadvantages noted in any part of this disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The accompanying drawings depict one or more implementations according to the present teachings, which are provided by way of example only and not limitation. In the various figures, the same reference numerals refer to the same or similar elements. In addition, it should be understood that the accompanying drawings are not necessarily drawn to scale.

[0011] Figure 1 illustrates the implementation of an image sharing and annotation application and environment;

[0012] Figure 2 is a conceptual diagram illustrating one implementation of a distributed computing environment for enabling executable annotations with electronic content items;

[0013] Figure 3 is a display diagram illustrating an implementation of a user interface of an application configured to provide annotation tools for electronic content items;

[0014] Figure 4 is a display diagram illustrating an implementation of a user interface of a communication application for communicating an enhanced electronic content item to a recipient;

[0015] Figure 5 is a display diagram illustrating an implementation of a user interface of a communication application for presenting a recipient's response to an interaction event;

[0016] Figure 6 is a display diagram illustrating an implementation of a user interface for a payment processing application;

[0017] Figure 7 is a diagram depicting an instant messaging application presenting an automatic notification notifying a sender that a purchase has occurred and / or providing details about the transaction;

[0018] Figure 8A-8F is a sequence of diagrams depicting examples of implementation of an image sharing interface and execution of instructions tagged with an image when being shared;

[0019] Fig. 9 is a display diagram illustrating an implementation of a user interface of an application configured to provide annotation tools for electronic content items;

[0020] Fig.10 It is a diagram showing Fig. 9A display diagram of an implementation of a user interface after execution of the instructions conveyed by the enhanced electronic content item;

[0021] Fig.11 is a diagram for a device configured to receive Fig. 9 A display diagram of an implementation of a user interface of an application for status information associated with an electronic content item;

[0022] Fig.12 is a flow chart for a user-customized implementation of an annotation tool;

[0023] Fig.13 is a flow chart illustrating an implementation of a process for sharing and managing annotated electronic content;

[0024] Fig.14 is a block diagram of an example computing device that may be used to provide implementations of the mechanisms described herein; and

[0025] Fig.15 is a block diagram illustrating components of an example machine configured to read instructions from a machine-readable medium. DETAILED DESCRIPTION

[0026] In the following detailed description, many specific details are set forth by way of example to provide a thorough understanding of the relevant teachings. However, it is apparent that the teachings can be practiced without these details. In other cases, well-known methods, steps, components and / or circuits have been described at a relatively high level, and in order to avoid unnecessarily obscuring various aspects of the teachings, no detailed description is provided.

[0027] The following implementation introduces a scanning application toolkit that is configured to enhance the user content sharing experience by allowing users to add executable labels or annotations to content before sharing. Traditionally, content sharing workflows include a series of cumbersome steps as users attempt to share their electronic content with a specific purpose of a mechanism, through which mechanism the mechanism should be communicated or to whom, or an expectation that further action is to be taken or initiated by the recipient. The sender must typically (1) identify the item; (2) insert the item into an application for sharing; (3) add some annotation or message indicating the response or result expected by the recipient of the shared item; and (4) hope that the recipient understands the request and takes the necessary steps to respond appropriately. Such notes are still typically embedded or visibly printed on the electronic content next to the electronic content, which may not be the preferred result for the sender, who adds the annotation to ensure that the recipient acts on the information as required. The disclosed implementation allows users to add executable commands or instructions to the electronic content in the form of annotations, so that the recipient can benefit from more precise and detailed context when interpreting and / or acting on the sender's request. In some implementations, the system can be configured to "translate" or convert annotations across multiple applications and platforms when appropriate triggers are detected, and perform one or more tasks that support recipients to access and interact with the project. When electronic content projects are shared, senders enjoy more refined control of the electronic content project's manner, presentation, and overall context, and can allow recipients to more accurately understand their own purpose of sharing the project. Similarly, recipients can be guided to the actions or responses that are expected or anticipated by the received electronic content project, while also finding that the process of responding or participating in the project is greatly simplified.

[0028] As described above, applications such as word processing applications, desktop publishing applications, electronic spreadsheets, presentation software, etc. can be used to generate electronic documents or content. Generally, the term "electronic document" or "document" includes any data that can be presented (e.g., visually, auditorily, based on tactile sensation, etc.), including but not limited to electronic content items and parts thereof, media items, media streams, web pages, hypertext documents, images, digital video or video recordings, digital audio or audio recordings, animations, markup language documents, such as hypertext markup language (HTML) or extensible markup language (XML) documents, forms with blank components for accepting input data, or data describing GUI applications, image documents containing images from which text can be extracted using optical character recognition (OCR) processes, documents that may contain a mixture of text and images, such as portable document format (PDF) documents or PowerPoint (PPT) documents, or any type or format of documents from which text can be extracted or which can be converted to text, and other digital data. For example, electronic content can include word processing documents, electronic spreadsheets, presentations, electronic books, images, slides, photos, or other digital-based media.

[0029] In addition, a user or participant generally refers to a person who views, creates, captures, modifies, develops, collaborates, suggests, listens to, receives, shares, comments, revise or promotes electronic content, including the creation, viewing or updating of annotations associated with the electronic content. Users include creators or sharers of electronic content and programs based on electronic content, as well as users of the devices and systems described herein. In addition, the terms "software application", "software" or "application" refer to computer programs that perform useful work, usually unrelated to the computer itself. Some non-limiting examples of software applications include speech to text applications, image capture applications, recording applications, word processing applications, spreadsheets, slide and presentation design applications, accounting systems and telecommunications programs, as well as game software, utilities and productivity tools, mobile applications, presentation graphics and other electronic content creation or sharing software.

[0030] In various implementations, a software application, such as Microsoft Office The procedures provided in ), Office Microsoft Microsoft Microsoft Office and other program groups or applications may provide various teleconferencing, text generation, presentation, and image capture or review tools. In other implementations, Adobe Google or another image capture or digital presentation program such as Clear Adobe Apple Corel Google IBM Lotus Word or other Microsoft Office or Software applications within the product portfolio, and any non- These are non-limiting examples, and any other electronic content creation, editing, or collaboration application can benefit from the disclosed implementations. Throughout the description, software applications that are used to initially create or normally view (e.g., on a desktop) electronic content will be referred to as native applications.

[0031] In some implementations, various image sharing and scanning applications allow end users to scan documents, presentations, real-world objects, and other objects using images captured by a camera associated with the device or via images stored or accessed from memory. In different implementations, software applications such as Microsoft Office Programs provided in (for example, Office ) and other apps that can provide a variety of image capture and editing tools, including scanning and identifying different areas in an image. Other examples include Microsoft Safety Windows Fax and Genius Scanner Google and Helmut Film Google Evernote These are non-limiting examples, and any other electronic content editing or viewing application may benefit from the disclosed implementations.

[0032] Software applications that may incorporate the disclosed features may be installed on a client's device, or associated with a third-party application, such as a web browser application configured to communicate with the device. These devices may include, for example, desktop computers, mobile computers, mobile communication devices (such as mobile phones, smart phones, tablet computers, etc.), smart televisions, gaming devices, set-top boxes, and / or any other computing device that includes a camera and / or image display capabilities.

[0033] In order to better introduce the system and method to readers, Figure 1 An example of a typical electronic content sharing environment for implementing executable tagging or annotation functionality is presented (in Figure 2 ). In various implementations, the environment may include multiple computing device end users, or simply "users," who may capture, view, edit, modify, and / or interact with electronic content (e.g., a first user, a second user, a third user, etc.). For simplicity, in the following scenarios, the user responsible for adding, attaching, linking, inserting, or otherwise associating an executable annotation with an electronic content item will be referred to as the sender, and will be referred to as the sender's user, and the user, group, or entity that receives or is otherwise affected by the annotated electronic content and / or any representation of the executed annotation will be referred to as the recipient. Thus, the recipient need not be limited to an individual or an individual communication address or contact point, but may be more broadly understood to include any sharing point, including an email account, instant messaging, text messaging, messages transmitted via a teleconferencing application, personal and / or group social media pages, collaborative applications, websites or organizational sites, cloud or network storage devices, or any other mechanism by which electronic content may be forwarded, shared, moved, or uploaded from a first digital location to a second digital location. In some implementations, the transfer of a file or the movement of a file from one location to another can be configured to trigger the execution of instructions carried by the annotation.

[0034] As will be described in further detail below, executable annotations may be understood to refer to any virtual note, tag, label, or information that is distinct from the data comprising the electronic content item itself and is paired with executable instructions or otherwise includes executable instructions or code that performs one or more tasks. Executable instructions may be configured to cause a computing or other processing device to perform a function or a set of functions. In some implementations, executable instructions may be executed by a computing system associated with a sender and / or a target recipient, while in other implementations, executable instructions may be executed via a cloud computing system or network.

[0035] As an example, Figure 1 Depicted is a sender 102, a first recipient 110, a second recipient 120, and a third recipient 130. In this case, the sender 102 is accessing an electronic content item (here, an image 104) on a mobile computing device 106, which is linked to other end users, data repositories, and entities via a network. When a user views the image 104, he or she may add or otherwise associate one or more executable annotations with the image 104. For simplicity, the term "tag" may be used interchangeably with the term "executable annotation," and the act of "tagging" also refers to the process of associating an executable annotation with an electronic content item.

[0036] In some implementations, selection of a tag may occur via a user interface 108 presented on a display. In general, an "interface" may be understood to refer to a mechanism for transmitting content to an application user via a client application. For example, an interface may include pop-ups that may be presented to a user via a native application user interface (UI), controls, actuable interfaces, interactive buttons, or other objects that may be displayed to a user via a native application UI, and an inherent mechanism of a particular application for presenting content associated with those native controls. Additionally, "actuation" or "actuation event" refers to an event (or a particular sequence of events) associated with a particular input or use of an application via an interface that may trigger a change in the display of the application.

[0037] In addition, "native controls" refers to mechanisms for communicating content to application users through client applications. For example, native controls may include actuatable or selectable options or "buttons" that may be presented to users via native application UIs, touch screen access points, menu items, or other objects that may be displayed to users through native application UIs, fragments of larger interfaces, and the inherent mechanisms of specific applications for presenting content associated with these native controls. The term "asset" refers to content that is presented, which may be associated with native controls in a native application. Thus, as non-limiting examples, assets may include text in an actuatable pop-up window, audio associated with interactive clicks on buttons or other native application objects, video associated with an instructional user interface, or other such information displays.

[0038] exist Figure 1 In the example of , sender 102 has selected and / or created three different tags, including a first tag 152, a second tag 154, and a third tag 156. In different implementations, tags can contain, include, or convey tag content. As an example, the tag content of the first tag 152 includes "at" mentions of personal user accounts and notes ("@Jason-Is this flower included in the list?"). In some implementations, the format and / or use of specific symbols or characters can determine or define the dynamics of a specific task to be performed when the tag is executed. For example, the use of "@" followed by an identifier can indicate that the electronic content item should be shared with a specified user in a specific manner. In addition, the use of a dotted line "-" (or other symbols, such as a colon, a comma, a quotation mark, a hash, etc.) can indicate that the text following the colon should be presented to the specified user as a message. These details are shown for illustrative purposes only, and it should be understood that a wide range of other tag content can represent or be used to convey different types of instructions.

[0039] The tag content for the second tag 154 includes a plus sign ("+") followed by a file name (here "Flowers.PPT," where the powerpoint file extension implies that the file is a slide show). In this example, the plus sign may indicate a request that the image 104 be shared or otherwise added to the named file. Again, this usage is for illustrative purposes only, and the specific format or character types used to express any instructions may vary. In a third example, the tag content for the third tag 156 includes "#MarylandPlants," which indicates a request by a user for the image to be shared on a social media webpage linked to the MarylandPlants identifier.

[0040] It is understood that each label has been inserted into image 104 or is associated with image 104. In different implementations, once a label has been associated with an electronic content item and the electronic content item through the label is shared or otherwise moved, the system can be configured to perform the instruction paired with the selected label or included in the selected label. In some implementations, the user can access or otherwise manage the image 104 through the label via the first application. It is understood that although the execution of the task associated with the label during the use of the first application can be triggered (for example, relative to the image 104 that triggers executable instructions, the user takes some actions), the task performed can also occur via the first application and / or through the resources of the second, different applications. In other words, the label can be configured to cross various platforms to perform the task type of a wide range. Therefore, the scope of the task requested by the insertion of the label need not be limited to the ability of the first application, but can be extended to multiple applications and functionality.

[0041] exist Figure 1 In , for simplicity, the sender 102 submits or uploads the tagged image to an executable annotated content manager application ("manager application") 160, which is configured to access a cloud-based annotation system via a network connection. In other implementations, the manager application 160 may be optional, with its functions being performed by software used to create, modify, view and / or share electronic content items, or the user may directly access the system online as a cloud-based application. Upon detecting and / or identifying submissions of tagged content, the manager application 160 may be configured to process any tags and / or forward the tagged image to another module or application for execution or presentation. In this case, the manager application 160 identifies and classifies each tag. Depending on the type of tag and its corresponding task, the manager application 160 may route the tagged content to an appropriate outlet or execution application.

[0042] In the first scenario 170, upon execution of the first tag 152, the image 104 has been transmitted in a message 172 to an email inbox for the first recipient 110, wherein the message subject line contains the same text as the note "Is this flower included in the list?" In the second scenario 180, upon execution of the second tag 154, the image 104 has been added and embedded in the file "Flowers.PPT" 182, wherein the selected file can be understood as acting as the second recipient 120. In the third scenario 190, the image 104 has been posted to a designated third recipient 130, which is a social media page 192 in this case. Therefore, it can be understood that in different implementations, each tag can be paired with a significantly different task. When the instructions are executed for each of these tags, multiple processes are initiated across several applications. In addition, in some implementations, some or all of the steps comprising these tasks are automatically performed without further user intervention or input. As will be appreciated, this type of system may provide an improved experience to users when they create and share content across multiple distribution paths.

[0043] Reference now Figure 2 , an example of a typical architecture for a content annotation and sharing management system ("system") 200 is depicted. In various implementations, the proposed system may include provisions for applying various annotation-task pair types to electronic content and / or presenting various user interfaces for tag customization. In some implementations, the user experience for inserting annotations may be relatively simple. Figure 2 , an example of a system configured to manage the use of executable annotations is illustrated. It should be understood that the system 200 presented here is merely an example implementation, only some aspects are presented for clarity, and a variety of other implementations are possible.

[0044] While the system 200 can be installed locally on a client device, in other implementations, as described herein, the system 200 is stored and / or configured for access via a network and / or cloud-based computing service. Figure 2, it can be seen that system 200 includes annotation input component 210. Annotation input component 210 is configured to store, update and provide or enable access to multiple annotation options that can be used for insertion or associated with electronic content items. As an example, annotation input component 210 can include a set of tags 212 (e.g., tag A, tag B, tag C, etc.), which can be selected to link or pair with one or more specific corresponding tasks 214 (e.g., task A, task B, task C, etc.). In some implementations, pairing can be established as a default through the system; for example, there can be a pairing that is specific to a specific user type and is expected to be generally used by a specific user type, such as a business, supplier, social media, family, individual, professional or personal network, travel, blog, school or other group. In other implementations, this pairing can occur as a result of the customized user-specified or created tag-task connection via a customization module 216.

[0045] The sender 202 accesses the electronic content item 204 via any type of device 228 capable of presenting images and / or digital content, such as a camera, mobile phone, tablet computer, laptop computer, desktop computer, gaming device, projector, and other such devices. In addition, software such as a content sharing application may be installed locally on the device, or some or all aspects or functions of the application may be accessed from another device or from a cloud storage computing service.

[0046] In different implementations, device 228 can access, store or receive electronic content item 204 for sender 202 to view and annotate on user device 228. In some implementations, sender 202 can attach or associate at least one mark-task pair 218 to electronic content item 204 to produce enhanced electronic content item 206. In this case, the term enhancement can be understood to refer to the result when core content item (e.g., electronic content item 204) is associated with added features, content, attributes or services such as tags. Additional functions give project a framework registered in the sharing process. As an example, sender 202 can access (multiple) mark-task pairs from online repository 290a, and mark-task pairs 218 can be updated or otherwise modified via network connection 290b to system 200.

[0047] User input can vary widely based on the type of input device used. Figure 2In the present invention, user input can be understood as being associated with or corresponding to the use of certain sharing applications 208, including but not limited to telecommunications software, storage synchronization services, social media platforms, and other such mechanisms for sharing content between one location and another. As an example, the user input can include a touch or mouse click on a touch screen interface that specifies a target recipient or location intended by the user for sharing the electronic content item 204.

[0048] In some implementations, the user input may also be identified as a trigger action event 226 by the trigger detection module 220 of the annotation management system 200. In general, a "trigger event" or "trigger" refers to an event (or a particular sequence of events) associated with a particular use of an application, the event corresponding to the selection of an option provided via a native control, or an event matching a condition. Figure 2 In the embodiment, the trigger event can be understood to include "clicks", switches, voice commands or other input actions (such as left or right mouse clicks, touch screen clicks, data selections or other input types), which lead to the sharing of electronic content items 204 or the activation of enhanced electronic content items 206. It can be seen that although the trigger action event 226 occurs through the user device 228, the occurrence of the event can be transmitted through the network connection 290c of the system 200.

[0049] Once the trigger is detected, the enhancement layer or content layer corresponding to the annotation added to the electronic content item 204 can be processed by the annotation translation module 222, which can be configured to interpret the instructions conveyed by the annotation and determine the manner in which the instructions are to be executed. This information can then be provided to the annotation execution module 230, which initiates execution via the designated task execution engine 232 and produces a result that aligns with the sender's request; an example of this process is shown below. Figure 3-Figure 11 is presented. For example, the annotation translation module 222 can allocate a display area in the client device display within which the context layer can create user interface (UI) controls. The annotation translation module 222 can execute the context layer rendered in the appropriate display area and adapt the performance of the task to the contents of the context layer. The allocated display area depends on the display mode of the layer. In one embodiment, the layer cannot modify the size of its display area, move the display area, or create UI elements outside the display area. In another embodiment, the layer can request the properties of its display environment (e.g., display mode, display area size, display area location, and fixed state) and / or be notified of any changes.

[0050] In some implementations, the result may include the transmission of electronic content item 204 within or alongside enhanced context 234. In other words, the result of execution of the instructions may include the presentation or delivery of electronic content item 204 to recipient 252 (e.g., via network connection 290d) with a second layer of content corresponding to the manner and / or mechanism by which the content requested by sender 202 should be displayed to recipient 252.

[0051] Additionally, in various implementations, the annotation management system 200 can be configured to direct or assist in the execution of additional actions resulting from the delivery of the enhanced context 234. For example, in response to various interaction events with the received content by the recipient 252 (e.g., via the network connection 290e), the system can manage subsequent actions. As a general matter, an "interaction event" as used herein refers to an event (or a particular sequence of events) associated with a particular use or access of electronic content or an application, which can then be used to determine whether an additional action should occur.

[0052] exist Figure 2 244 can be evaluated by the recipient interaction workflow module 240. Depending on the type of action that occurred, the system can generate a reaction 242 that can be communicated to the external application 246 and / or cause the system to record the action in the activity log 260. In some implementations, this information can be used to notify the sender 202 (e.g., via the network connection 290f) of the status of the annotation performed, for example, via the automatic response generator 270.

[0053] In general, the context layer generated by the annotation is implemented as an extensible framework that may include the following functions: (a) an XML file format for packaging the HTML, JavaScript, and application descriptors of the layer into a single file. The application descriptor indicates what types of electronic content the layer is applicable to; (b) a service that renders the layer XML file as HTML / JavaScript that can be displayed in a web browser; (c) client-side JavaScript for existing web applications (e.g., electronic content processing modules) to instantiate the layers and display them within a container element. The JavaScript creates a sandboxed iframe HTML element for each layer. The iframe uses a uniform resource locator (URL) pointing to the service to render the requested layer; (d) client-side JavaScript for a remote procedure call (RPC) framework that enables an application to register a layer that can call the JavaScript methods (layer API) of the layer from its sandboxed iframe. This enables the layer to call back to the application and send / receive data. In addition, there are provisions for preventing the context layer from interfering with the operation of electronic content presentation while executing instructions. In some implementations, the annotation can indicate in its application descriptor which functions it needs to perform.

[0054] For the sake of clarity for the readers, Figure 3-Figure 11 Some examples of user interfaces where enhanced electronic content sharing may occur are presented. Figure 3 , an implementation of display 300 for computing device 308 is illustrated. In different implementations, display 300 can present various icons, interfaces, graphics, applications, or other device state information. As an example, display 300 includes a representation of a user's access to a file associated with a first application. For simplicity, Figure 3 The first application in is an image processing and rendering program including a viewing interface 310, which is configured to support users to access various documents or files via the application. Here, the viewing interface 310 presents a first content item 312 for user viewing or modification.

[0055] Once a user has access to an electronic content item, they can access options or tools that can be used in association with the content item or application. Figure 3As shown, the first content item 312 is presented in conjunction with the first annotation interface ("first interface") 350 of the annotation management system, in this case identified as the title "TAGS". In some implementations, the first interface 350 can be separate from the viewing interface 310 of the first application, or placed or located near or close to the viewing interface 310 of the first application, or can extend outward from the viewing interface 310. In another implementation, the first interface 350 can be arranged in the area of ​​the viewing interface 310, and / or can be provided as a function of the first application itself. The first interface 350 can be configured to display or present various options through which the user can select or customize tags and / or associate the selected tags with content items. These options can be presented independently through the tag software application, or can be provided as part of the function of the first application itself. For example, the user can choose to print, save, share, close the currently accessed project, etc., as well as modify or associate one or more tags to the project through a single application.

[0056] In some implementations, the first interface 350 may include navigation tools or headers to assist the user in navigating the available annotation efforts. Figure 3 , a main title 352 ("Vendor Tags") is displayed, indicating that the user is currently viewing tags categorized or commonly used by customers who are primarily interested in transactional type services. In other words, the tag list can be modified or requested to display those tags that are frequently used by those involved in business transactions, merchant tasks, and other interactive related products. In other implementations, the tags can be sorted or filtered in other ways, or the user can customize the displayed tag list. In this case, multiple annotation options 354 are shown, with the first option 356 ("Buy") being selected. When selected, the tag can be associated with the electronic content currently displayed in the viewing interface 310. In this example, the illustrated content is a menu or catalog of cakes, including a description of each cake, its title, and its price.

[0057] As previously described, in some implementations, the annotation management system can be configured to identify one or more characters or symbols in a particular electronic content item as corresponding intended tags. In this particular example, when a user selects or activates the "Buy" tag, the system can be configured to detect all instances of a "$" (dollar sign) with a number and, in each of those instances, pair or add an executable instruction to the instance. In other implementations, a user can select (e.g., via a mouse click, mouse highlight, or other interactive event) each instance of an intended executable annotation and manually link or pair the tag with the instance. Once one or more tags are added or inserted into the electronic content, the sharing of the content or other transfer of the content to another location can automatically trigger the execution of a task, such as Figure 4and Figure 5 shown.

[0058] exist Figure 4 In the embodiment of the present invention, the first content item 312 has been communicated, transmitted, shared or sent to the second user (recipient) from the sender via the instant messaging application. In different implementations, the enhanced electronic content item presented to the recipient can provide a "call to action" button or an actionable option, if selected, the recipient can be routed to a predefined destination, receive, store and / or share information generated by the selection of the actionable option, and / or guide the recipient through the process specified by the sender. Because the first content item 312 is presented and displayed for the recipient (here is Martha Johnston) in the message transceiving interface 410, it can be seen that in addition to the original content layer of the electronic content item itself, the second contextual content layer is also performed. The contextual content layer refers to any content added in response to the execution of the instruction associated with the selected label. In this case, the contextual content layer is presented to the recipient in the form of multiple native controls or actionable options ("buy") 450 displayed adjacent to each product. In some implementations, each button can be configured to perform an action or initiate a process when selected. Thus, in some implementations, the "task" to be performed by the sender inserting the annotation includes the rendering of a new (contextual) content layer as well as the original electronic content.

[0059] In some implementations, the recipient client device interprets the annotated electronic content and rendering occurs within the messaging application main window or is displayed by a separate application (e.g., a web browser invoked using a URL or a map application invoked using an address). In one implementation, the context layer can be displayed in various modes, such as overlay (floating on top of the electronic content), replacement (floating on top of a specific portion of the electronic content, thereby "hiding" the content), side panel (presented in a side panel adjacent to the electronic content, perhaps in the form of a small preview that can be interacted with), and margin (presented in a margin area of ​​the electronic content, perhaps in the form of a very limited preview).

[0060] If the recipient clicks or otherwise selects one of the plurality of actionable options 450, such as Figure 4 As shown by the mouse cursor 452 in FIG. 1 , a new process can be initiated in conjunction with an instant messaging application. Figure 5 , the new visual (second content item 514) can be displayed in the messaging interface 410 to provide the recipient with a way to continue / confirm Figure 4 As the recipient continues to provide input to the system, additional actions may occur. Figure 6In response to the recipient's confirmation of the purchase, web page 600 has opened a payment process for the recipient on the same computing device. It can be seen that in some implementations, the system can be configured to initiate or execute a process involving multiple steps and / or applications, receive and process input from various sources, and / or integrate information occurring at different points in time to provide a seamless user experience.

[0061] As the receiver interacts with the contextual content layer and participates in any subsequent steps, in different implementations, the system can optionally be configured to update or notify the sender of the progress or status of the transaction. For example, in some implementations, such as Figure 7 As shown, the instant messaging application can present an automatic notification 700, notifying the sender that a purchase has occurred and / or providing details about the transaction. In some implementations, one or more suggested responses or options can also be automatically presented to the sender. In this example, a second interface 750 is displayed within the messaging interface 410, and the messaging interface provides multiple options 710 for responding to the notification 700. In other implementations, the second interface 750 can be independent of the messaging interface 410, or additional options may not be provided to the sender.

[0062] As mentioned previously, executable annotations can be configured to initiate a wide range of tasks in different implementations. Fig. 8A and Figure 8F , another example is presented, where a first user 810 is shown capturing a photo of a whiteboard 814 including a chart 816 on a mobile device 812. Figure 8B , the captured image (third content item 820) is viewed on the mobile device 812, and a second annotation interface ("second interface") 822 is displayed next to it. Figure 3 In contrast to the first interface 350, the tags provided in the second interface 822 are categorized under a "Share Tags" heading 824 and point to tasks that enable sharing of electronic content items. Figure 8B In the "Instant Message" option 826, the sender may be prompted to add details to form the execution of the selected task. Figure 8C , the sender has inserted a tag 830 with a message 834 (“@EricS should we include this in next week’s meeting?”), thereby creating an enhanced content item 832. In some implementations, the system can be configured to initiate a specific set of actions based on at least the data entered into the tag 830, as described below with reference to Figure 8F Being discussed.

[0063] exist Fig.8DIn , the first user opens the social media application 840 and identifies a recipient ("domain group") 842 for a third content item, in which case the third content item corresponds to a larger user group that may include multiple members. Fig. 8E In FIG. 8 , it can be seen that the publication of the third content item 820 has been successfully added to the recipient group page 850. In addition, Figure 8C The tag 830 is not visible, nor is there any indication to the group that the content is enhanced. In other words, in some implementations, the system can be configured to remove some or all traces of the tag 830 when the tag 830 is shared. In addition, when the third content item 820 is shared with the recipient 842, the system can interpret the tag content and perform the associated task. In this case, as shown in FIG. Figure 8F As shown, the third content item 820 automatically comes to the attention of the personal user account 862 associated with the "EricS" username via a separate instant message on the second mobile device 860. Thus, at the same time that the third content item 820 is shared on the recipient group page 850, the subsequent (or substantially simultaneous) presentation of the image on Eric Sho's channel 866, as a result of the manual action by the sender, and any optional message content 868, automatically occurs as a result of the execution of the instructions conveyed by the tag. Thus, in some implementations, the sender can specify or identify one or more individuals who should be specifically "summoned" or notified about the shared content item (and specific message content) via the tag, and the system can "translate" the tag and automatically deliver the message to the specified (multiple) persons.

[0064] For clarity, reference is now made to Figure 9-11 , additional examples of the disclosed system are presented. Fig. 9 , a sender (represented by a hand) 900 views a document (fourth content item 910). In this case, the document is an invoice for services. The sender 900 wishes to receive approval for the invoice from her manager. In different implementations, the system may include provisions for providing organizations or other users with the ability to customize the creation or designation of tags for everyday use. In this example, the sender 900 accesses a third annotation interface ("third interface") 930, which includes a title "Shezi Company, Ltd. Tags" 932. The title 932 identifies the listed options as being specifically provided for company use, and / or as being created by company members in anticipation of requests for tasks that are typically requested by employees. In Fig. 9 In the example, the sender 900 selects the "Approve Invoice" option 940, which triggers the automatic association of the predefined tag with the fourth content item 910. When the fourth content item 910 is shared with another user, the tag can be activated and its corresponding instruction can be executed. Fig.10 As shown, as a result of the execution of the tag associated with the fourth content item 910, the presentation of the fourth content item 910 can occur within a display context rendered by a specific user interface (here, a messaging application 1010) or a computing device 1000 of the recipient. In other words, although the tag itself is not shared, the content of the tag directly changes the conversion or reshapes the recipient's user experience of the document. The application can directly present the content in the context that best suits the sender's goals, rather than simply presenting the document, for example, as an attachment. In some other implementations, the message to the recipient can include an entry or link to an invoice approval system. When the recipient accesses the link, the document is displayed in a manner that immediately and clearly indicates the type of response expected by the sender, and provides a simple and effective means for submitting a response.

[0065] In some implementations, the recipient's selected response can be transmitted back to the sender in the same communication format that the sender used to share the electronic content. In other implementations, the sender can request notification of the response via specifying other communication mechanisms. In some implementations, the executable annotation can be configured with instructions that, when executed, automatically update a record or database, or merge information from the response into another application. Fig.11 , it can be seen that the invoice status database 1100 has been automatically updated after the recipient submits the response.

[0066] As previously described, in some implementations, the system may include provisions for enhanced specificity by which a user may select or assign particular annotation-task pairs to an electronic content item. In some implementations, a user may assign a classification, tag, or category directly to one or more intended tasks, and / or specify the type of annotation tools that should be provided or made available. As an example, Fig.12 A first customization menu 1210 is presented in which a plurality of possible tasks 1220 are shown. It should be understood that Fig.12 The specific tasks and actions depicted are shown for simplicity purposes only, and there are various other possible options (including tasks specifically described via manual user input, such as a series of building blocks or a custom-designed path for identifying a specific process that is expected to be triggered).

[0067] exist Fig.12In the example, tasks 1220 include a first task 1222 ("Offer Actionable Option") that has been selected by the user, indicating the purpose of adding an electronic content item, so that when the item is presented to the recipient, the presentation will also include an effective response mechanism in a way that bypasses many cumbersome user-intensive manual steps that would otherwise be required to perform the task. For example, this can be communicated through selectable (multiple) buttons or (multiple) options that, when selected by the recipient, will perform certain specified actions. In response, in some implementations, the system can present a second customization menu 1230, in which the task type selected in the previous step causes multiple actions 1232 to be displayed, which can be linked to selectable buttons and will be displayed to the recipient. In this example, the user has selected a first action 1234 ("Buy-Pay"), which indicates that the recipient's activation of this option should trigger a payment process associated with the electronic content item for all or a specified portion of the electronic content item. In some implementations, the user may wish to utilize the payment processing workflow already provided by the system (see first option 1250). However, in other implementations, the user may access another interface that will provide the user with an opportunity to manually identify and specify the process that should occur in order to elicit a payment from a recipient (see second option 1252).

[0068] In different implementations, the system can provide various types of annotated task-action pairs. Some examples are also shown in Fig.12 , including "Send Email" and "Send Text Message," the selection of which may indicate that the electronic content item is being annotated with the expectation that the system transmits the electronic content item to a specific recipient via one or more communication channels. Similarly, "Add to File" may indicate an expectation that the electronic content item is inserted or added to an existing or new file located at a specific location or path. The "Embed in Web Page" selection may indicate that the electronic content item should be inserted or added to a specific web page, such as a social media page or blog. Many other tasks are possible, including but not limited to providing earlier information about Figure 1-Figure 11 Functionality as described.

[0069] In addition, in one embodiment, the manner in which the enhanced content is displayed can be configured from a fixed set of options. The options include, for example, simple format, display mode, and automatic layer activation. The simple format includes, for example, text style options (e.g., font, color, highlight, italic, bold, underline, and outline) and an indicator (e.g., a mini icon or keypad) that will be displayed to the recipient along with the particular electronic content item being shared.

[0070] Fig.13is a flow chart illustrating an implementation of a method 1300 for annotating an electronic content item and initiating a task associated with the annotation. A first step 1310 of the method 1300 includes presenting a first user interface for a first content viewing application on a first client device, the first user interface displaying first electronic content, a second step 1320 includes receiving a first user input from a first user for associating a first tag with the first electronic content, the first tag representing a first executable process for performing a first task; in a third step 1330, the method includes recording an association between the first electronic content and the first tag in response to receiving the first user input; and a fourth step 1340 includes receiving a second user input from the first user, the second user input requesting that the first electronic content be transferred to a first recipient. A fifth step 1350 includes transferring the first electronic content to the first recipient in response to receiving the second user input, and a sixth step 1360 involves executing the first executable process in response to transferring the first electronic content.

[0071] In other implementations, the method may include additional steps or aspects. In some implementations, the method may also include presenting, on the first client device, an annotation menu providing a plurality of selectable tags including a first tag. In some cases, each of the plurality of tags represents a different type of executable process, the executable process may be associated with the electronic content, and the first user input results from a selection of the first tag from the annotation menu. In another implementation, the method also includes presenting a first selectable option to the first recipient during execution of the first executable process, the first selectable option being used to initiate execution of an action in conjunction with the first electronic content. In some implementations, the action includes providing a payment information interface to the second user for receiving payment method details, while in other implementations, the action includes modifying a record associated with the first electronic content.

[0072] As another example, the method may include, during execution of the first executable process, automatically generating a first message to a second recipient, wherein the first message includes a copy of the first electronic content. In some cases, the transmission to the first recipient occurs via a first telecommunication service and the transmission to the second recipient occurs via a second telecommunication service that is different from the first telecommunication service. As a further example, the method may include receiving a third user input from a first user, the third user input requesting that a second tag be associated with the first electronic content item, the second tag indicating a second task to be performed, and in response to receiving the third user input, recording a second executable process of an association between the first electronic content and the second tag. In addition, the method may include: in response to transmitting the first electronic content, executing a second executable process, and in response to execution of the second executable process, automatically generating a second message to a third recipient, the third recipient being a member of the first recipient.

[0073] In addition, the method may include receiving a third user input from the second user selecting a first selectable option, thereby triggering the execution of the action, and then in response to receiving the third user input, presenting a second selectable option to the first user for accessing details associated with the execution of the action. In one implementation, the method also includes automatically generating a message to the first user indicating that the action has occurred. In some implementations, the first user input includes at least a first content element configured to trigger tag generation, and the method also includes determining that the first user input includes the first content element, and then in response to determining that the first user input includes the first content element, determining that the first user input corresponds to a request to generate a tag associated with the first electronic content. In another example, the method may involve receiving a third user input from the second user, the third user input specifying which selectable tags are to be provided in the annotation menu. In some cases, the virtual tag may be incorporated into or added to electronic content that is visible only to the first user. In another example, the user may provide input that specifically specifies a pairing between tag types, tasks, and / or actions.

[0074] Use of the disclosed systems and methods can enable users to share electronic content in a manner that is more aligned with their purpose for the project. In addition, organizations can define preferred tags that senders can use, as well as define the behavior of the listed tags, and the way each tag is displayed on the receiving end. In addition, the behavior can be defined to be presented in a manner that effectively utilizes native functions on the receiving end. In different implementations, the system can be configured to interpret various enhancement layers and be able to retain, execute and render them on the receiving end endpoint. If the recipient interacts with the context layer that is presented in a manner that triggers the corresponding process, a service that aggregates these actions and triggers subsequent workflows based on these interactions can be initiated.

[0075] Combination Figure 1-Figure 13 Detailed examples of the described systems, devices, and techniques are presented herein to illustrate the present disclosure and its benefits. Such usage examples should not be construed as limitations on the logical process implementations of the present disclosure, nor should variations from those user interface methods described herein be considered beyond the scope of the present disclosure. In some implementations, Figure 1-Figure 13 The various features described in the present invention are implemented in respective modules, which may also be referred to as and / or include logic, components, units and / or mechanisms. A module may constitute a software module (eg, code contained on a machine-readable medium) or a hardware module.

[0076] In some examples, the hardware module can be implemented mechanically, electronically, or in any suitable combination thereof. For example, the hardware module may include a dedicated circuit or logic configured to perform certain operations. For example, the hardware module may include a dedicated processor, such as a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC). The hardware module may also include a programmable logic or circuit temporarily configured by software to perform certain operations, and may include a portion of machine-readable medium data and / or instructions for such configuration. For example, the hardware module may include software contained in a programmable processor configured to execute a set of software instructions. It should be understood that the decision to mechanically implement the hardware module in a dedicated and permanently configured circuit or in a temporarily configured circuit (e.g., configured by software) may be driven by cost, time, support, and engineering considerations.

[0077] Therefore, the phrase "hardware module" should be understood to include a tangible entity that can perform certain operations and can be configured or arranged in a certain physical manner, an entity that is physically constructed, permanently configured (e.g., hardwired), and / or temporarily configured (e.g., programmed) to operate in a certain way or perform certain operations described herein. As used herein, a "hardware-implemented module" refers to a hardware module. Considering the example of a temporarily configured (e.g., programmed) hardware module, it is not necessary for each hardware module to be configured or instantiated in any one instance in a timely manner. For example, in the case where a hardware module includes a programmable processor that is configured to be a special-purpose processor by software, the programmable processor can be configured as different special-purpose processors (e.g., including different hardware modules) at different times. The software can configure one or more specific processors accordingly, for example, to constitute a specific hardware module at one time instance and to constitute different hardware modules at different time instances. A hardware module implemented using one or more processors can be referred to as a "processor implementation" or a "computer implementation."

[0078] Hardware modules can provide information to other hardware modules and receive information from other hardware modules. Therefore, the described hardware modules can be considered to be communicatively coupled. Where there are multiple hardware modules, communication can be achieved by signal transmission between two or more hardware modules (e.g., through appropriate circuits and buses). In embodiments where multiple hardware modules are configured or instantiated at different times, communication between these hardware modules can be achieved, for example, by storage and retrieval of information in memory devices accessible to multiple hardware modules. For example, one hardware module can perform an operation and store the output in a memory device, and then another hardware module can access the memory device to retrieve and process the stored output.

[0079] In some examples, at least some operations of the method may be performed by one or more processors or processor-implemented modules. In addition, one or more processors may also be operable to support the performance of related operations in a "cloud computing" environment or as "software as a service" (SaaS). For example, at least some operations may be performed by and / or on multiple computers (as an example of a machine including a processor), which may be accessed via a network (e.g., the Internet) and / or via one or more software interfaces (e.g., application program interface (API)). The performance of certain operations may be distributed between processors, not only residing in a single machine, but also deployed on multiple machines. A processor or a processor-implemented module may be located in a single geographic location (e.g., in a home or office environment, or in a server farm), or may be distributed in multiple geographic locations.

[0080] Fig.14 Block diagram 1400 illustrates an example software architecture 1402, portions of which may be used in conjunction with the various hardware architectures described herein that may implement any of the above-described functionality. Fig.14 is a non-limiting example of a software architecture, and it should be understood that many other architectures may be implemented to support the functionality described herein. The software architecture 1402 may be executed on hardware, such as Figure 1 1070, processors, memory, and input / output (I / O) components. A typical hardware layer 1404 is illustrated and may represent, for example, Figure 1 The exemplary hardware layer 1404 includes a processing unit 1406 and associated executable instructions 1408. The executable instructions 1408 represent executable instructions of the software architecture 1402, including implementations of the methods, modules, etc. described herein. The hardware layer 1404 also includes a memory / storage device 1410, which also includes executable instructions 1408 and accompanying data. The hardware layer 1404 may also include other hardware modules 1412. The instructions 1408 held by the processing unit 1406 may be part of the instructions 1408 held by the memory / storage device 1410.

[0081] The example software architecture 1402 can be conceptualized as layers, each providing various functions. For example, the software architecture 1402 may include layers and components, such as an operating system (OS) 1414, a library 1416, a framework 1418, an application 1420, and a presentation layer 1444. In operation, the application 1420 and / or other components within a layer may invoke API calls 1424 to other layers and receive corresponding results 1426. The illustrated layers are typical in nature, and other software architectures may include additional or different layers. For example, some mobile or special-purpose operating systems may not provide a framework / middleware.

[0082] The operating system 1414 can manage hardware resources and provide public services. The operating system 1414 can include, for example, a kernel 1428, a service 1430, and a driver 1432. The kernel 1428 can act as an abstraction layer between the hardware layer 1404 and other software layers. For example, the kernel 1428 can be responsible for memory management, processor management (e.g., scheduling), component management, network, security settings, etc. The service 1430 can provide other public services for other software layers. The driver 1432 can be responsible for controlling the underlying hardware layer 1404 or interfacing with the underlying hardware layer 1404. For example, the driver 1432 can include a display driver, a camera driver, a memory / storage device driver, a peripheral device driver (e.g., via a universal serial bus (USB)), a network and / or wireless communication driver, an audio driver, etc., depending on the hardware and / or software configuration.

[0083] Libraries 1416 may provide common infrastructure that may be used by applications 1420 and / or other components and / or layers. Libraries 1416 typically provide functionality for other software modules to use to perform tasks, rather than interacting directly with operating system 1414. Libraries 1416 may include system libraries 1434 (e.g., C standard libraries) that may provide functionality such as memory allocation, string manipulation, file manipulation, etc. In addition, libraries 1416 may include API libraries 1436, such as media libraries (e.g., supporting the presentation and manipulation of image, sound, and / or video data formats), graphics libraries (e.g., OpenGL libraries for rendering 2D and 3D graphics on a display), database libraries (e.g., SQLite or other relational database functions), and network libraries (e.g., a browser that may provide web browsing functionality). Libraries 1416 may also include a variety of other libraries 1438 to provide many functions for applications 1420 and other software modules.

[0084] Framework 1418 (sometimes also referred to as middleware) provides a higher level common infrastructure that can be used by applications 1420 and / or other software modules. For example, framework 1418 can provide various graphical user interface (GUI) functions, advanced resource management, or advanced positioning services. Framework 1418 can provide a wide range of other APIs for applications 1420 and / or other software modules.

[0085] Applications 1420 include built-in applications 1440 and / or third-party applications 1442. Examples of built-in applications 1440 may include, but are not limited to, contact applications, browser applications, location applications, media applications, messaging applications, and / or game applications. Third-party applications 1442 may include any application developed by an entity other than the vendor of a particular platform. Applications 1420 may use the functionality available via operating system 1414, library 1416, framework 1418, and presentation layer 1444 to create a user interface to interact with a user.

[0086] Some software architectures use virtual machines, such as virtual machine 1448. Virtual machine 1448 provides an execution environment where applications / modules can run as if they were on a hardware machine (e.g., Fig.10 The virtual machine 1448 may be hosted by a host operating system (e.g., operating system 1414) or a hypervisor and may have a virtual machine monitor 1446 that manages the operation of the virtual machine 1448 and interoperation with the host operating system. Software architectures other than the software architecture 1402 external to the virtual machine may be executed within the virtual machine 1448, such as an operating system 1450, libraries 1452, frameworks 1454, applications 1456, and / or presentation layer 1458.

[0087] Fig.15 A block diagram of components of an example machine 1500 is illustrated, which is configured to read instructions from a machine-readable medium (e.g., a machine-readable storage medium) and perform any of the functions described herein. The example machine 1500 is in the form of a computer system in which instructions 1516 (e.g., in the form of software components) for causing the machine 1500 to perform any of the functions described herein can be executed. Therefore, the instructions 1516 can be used to implement the modules or components described herein. The instructions 1516 cause the unprogrammed and / or unconfigured machine 1500 to operate as a specific machine configured to perform the described features. The machine 1500 can be configured to operate as a standalone device or can be coupled (e.g., networked) to other machines. In a networked deployment, the machine 1500 can operate in the capacity of a server machine or a client machine in a server-client network environment, or as a node in a peer-to-peer or distributed network environment. The machine 1500 may be embodied as, for example, a server computer, a client computer, a personal computer (PC), a tablet computer, a laptop computer, a netbook, a set-top box (STB), a gaming and / or entertainment system, a smart phone, a mobile device, a wearable device (e.g., a smart watch), and an Internet of Things (IoT) device. Furthermore, while only a single machine 1500 is shown, the term "machine" includes a collection of machines that individually or jointly execute instructions 1516.

[0088] The machine 1500 may include a processor 1510, a memory 1530, and I / O components 1550, which may be communicatively coupled via, for example, a bus 1502. Bus 1502 may include multiple buses that couple various elements of the machine 1500 via various bus technologies and protocols. In an example, processor 1510 (including, for example, a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), an ASIC, or a suitable combination thereof) may include one or more processors 1512a to 1512n, which may execute instructions 1516 and process data. In some examples, one or more processors 1510 may execute instructions provided or identified by one or more other processors 1510. The term "processor" includes a multi-core processor, which includes cores that can execute instructions simultaneously. Although Fig.15 Multiple processors are shown, and machine 1500 may include a single processor with a single core, a single processor with multiple cores (e.g., a multi-core processor), multiple processors each with a single core, multiple processors each with multiple cores, or any combination thereof. In some examples, machine 1500 may include multiple processors distributed across multiple machines.

[0089] The memory 1530 / storage device may include a main memory 1532, a static memory 1534 or other memory, and a storage unit 1536, both of which may be accessed by the processor 1510, such as via the bus 1502. The storage unit 1536 and the memory 1532, the memory 1534 store instructions 1516 that embody any one or more of the functions described herein. The memory 1530 / storage device may also store temporary, intermediate and / or long-term data for the processor 1510. The instructions 1516 may also reside, in whole or in part, within the memory 1532, the memory 1534, the storage unit 1536, within at least one of the processors 1510 (e.g., within a command buffer or cache memory), within a memory of at least one I / O component, or any suitable combination thereof, during execution thereof. Therefore, the memory 1532, the memory 1534, the storage unit 1536, the memory in the processor 1510, and the memory in the I / O component 1550 are examples of machine-readable media.

[0090] As used herein, a "machine-readable medium" refers to a device capable of temporarily or permanently storing instructions and data that cause the machine 1500 to operate in a particular manner. The term "machine-readable medium" as used herein does not include transient electrical or electromagnetic signals per se (e.g., on a carrier wave propagating through the medium); thus, the term "machine-readable medium" may be considered to be tangible and non-transitory. Non-limiting examples of non-transitory tangible machine-readable media may include, but are not limited to, non-volatile memory (such as flash memory or read-only memory (ROM)), volatile memory (e.g., static random access memory (RAM) or dynamic RAM), buffer memory, cache memory, optical storage media, magnetic storage media and devices, network accessible or cloud storage, other types of storage devices, and / or any suitable combination thereof. The term "machine-readable medium" applies to a single medium or a combination of multiple media for storing instructions (e.g., instructions 1516) executed by the machine 1500, such that the instructions, when executed by one or more processors 1510 of the machine 1500, cause the machine 1500 to perform one or more functions described herein. Thus, a "machine-readable medium" may refer to a single storage device, as well as a "cloud-based" storage system or storage network comprising multiple storage devices or apparatuses.

[0091] I / O components 1550 may include a variety of hardware components suitable for receiving input, providing output, generating output, transmitting information, exchanging information, capturing measurements, etc. The specific I / O components 1550 included in a particular machine will depend on the type and / or function of the machine. For example, a mobile device such as a mobile phone may include a touch input device, while a leaderless server or an IoT device may not include such a touch input device. Fig.15 The specific examples of I / O components shown in are by no means limiting, and other types of components may be included in the machine 1500. The grouping of the I / O components 1550 is merely to simplify the discussion, and the grouping is by no means limiting. In various examples, the I / O components 1550 may include a user output component 1552 and a user input component 1554. The user output component 1552 may include, for example, a display component (e.g., a liquid crystal display (LCD) or a projector) for displaying information, an acoustic component (e.g., a speaker), a tactile component (e.g., a vibration motor or a force feedback device), and / or other signal generators. The user input component 1554 may include, for example, an alphanumeric input component (e.g., a keyboard or a touch screen), a pointing component (e.g., a mouse device, a touch pad, or another pointing tool), and / or a tactile input component (e.g., a physical button or touch screen that provides a position and / or a touch force or a touch gesture), configured to receive various user inputs, such as user commands and / or selections.

[0092] In some examples, I / O components 1550 may include biometric components 1556 and / or location components 1562, as well as a wide array of other environmental sensor components. Biometric components 1556 may include, for example, components for detecting body expressions (e.g., facial expressions, vocal expressions, hand or body gestures, or eye tracking), measuring biosignals (e.g., heart rate or brain waves), and identifying a person (e.g., through voice, retinal and / or facial based identification). Location components 1562 may include, for example, a location sensor (e.g., a global positioning system (GPS) receiver), an altitude sensor (e.g., a barometric pressure sensor from which altitude can be derived), and / or an orientation sensor (e.g., a magnetometer).

[0093] I / O components 1550 may include communication components 1564 that implement a variety of technologies operable to couple machine 1500 to network 1570 and / or device 1580 via respective communication couplings 1572 and 1582. Communication components 1564 may include one or more network interface components or other suitable devices to interface with network(s) 1570. Communication components 1564 may include, for example, components suitable for providing wired communication, wireless communication, cellular communication, near field communication (NFC), Bluetooth communication, Wi-Fi, and / or communication via other modalities. Device 1580 may include other machines or various peripherals (e.g., via USB coupling).

[0094] In some examples, the communication component 1564 can detect an identifier or include a component suitable for detecting an identifier. For example, the communication component 1564 can include a radio frequency identification (RFID) tag reader, an NFC detector, an optical sensor (e.g., a one-dimensional or multi-dimensional barcode, or other optical code), and / or an acoustic detector (e.g., a microphone to identify a tagged audio signal). In some examples, the location information can be determined based on information from the communication component 1564, such as, but not limited to, a geographic location via an Internet Protocol (IP) address, through Wi-Fi, cellular, NFC, Bluetooth or other wireless station identification and / or signal triangulation.

[0095] Although various implementations have been described, the description is intended to be exemplary rather than restrictive, and it should be understood that within the scope of these implementations, more implementations and implementations are possible. Although many possible feature combinations are shown in the drawings and discussed in this detailed description, many other combinations of the disclosed features are possible. Any feature of any implementation can be used in combination with or replace any other feature or element in any other implementation, unless specifically limited. Therefore, it should be understood that any feature shown and / or discussed in this disclosure can be implemented together in any suitable combination. Therefore, these implementations are not limited unless in view of the attached claims and their equivalents. In addition, various modifications and changes can be made within the scope of the attached claims.

[0096] Although what is considered to be the best mode and / or other examples have been described above, it is to be understood that various modifications may be made therein and that the subject matter disclosed herein may be implemented in various forms and examples, and that the teachings may be applied in many applications, only some of which are described herein. The following claims are intended to claim any and all applications, modifications, and variations that fall within the true scope of the present teachings.

[0097] Unless otherwise indicated, all measurements, values, ratings, positions, amplitudes, sizes and other specifications set forth in this specification (including the claims that follow) are approximate and not exact. They are intended to have a reasonable range consistent with the functions to which they are related and with the customary practices in the art to which they belong.

[0098] The scope of protection is limited only by the claims that now follow. When interpreted in light of this specification and the subsequent prosecution history, the scope is intended and should be interpreted to be consistent with the ordinary meaning of the language used in the claims and to encompass all structural and functional equivalents. Any unintended inclusion of such subject matter is hereby disclaimed.

[0099] Except as stated above, nothing stated or described is intended or should be construed as causing any component, step, feature, object, benefit, advantage, or equivalent to be dedicated to the public, regardless of whether or not recited in the claims.

[0100] It should be understood that the terms and expressions used herein have the ordinary meaning consistent with these terms and expressions with respect to their corresponding corresponding research and research fields, unless otherwise specified herein. Relational terms such as first and second can be used alone to distinguish one entity or action from another entity or action without requiring or implying any actual such relationship or order between these entities or actions. The terms "comprises", "comprising" or any other variant thereof are intended to cover non-exclusive inclusions, so that the process, method, article or device including a list of elements includes not only those elements, but may also include other elements that are not explicitly listed or inherent to such process, method, article or device. Elements starting with "a" or "an" do not exclude the presence of additional identical elements in the process, method, article or device including the element without further limitation.

[0101] An abstract of the disclosure is provided to allow the reader to quickly ascertain the nature of the technical disclosure. The abstract is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. Furthermore, in the above detailed description, it can be seen that various features are grouped together in various examples for the purpose of simplifying the disclosure. This disclosure method should not be interpreted as reflecting an intent that the claims require more features than those expressly stated in each claim. On the contrary, as reflected in the following claims, the inventive subject matter lies in less than all the features of a single disclosed example. Therefore, the following claims are hereby incorporated into the detailed description, with each claim standing alone as a separately claimed subject matter.

Claims

1. A data processing system, comprising: at least one processor; as well as One or more computer-readable media comprising instructions that, when executed by the at least one processor, cause the at least one processor to: receiving, over a communications network, a first request to associate a first tag from among a plurality of different tags with a first electronic content, wherein each tag in the plurality of tags: is configured to mark electronic content, and being associated with a corresponding task from among the plurality of tasks; In response to receiving the first request, recording a first association between the first electronic content and the first tag; as well as Transmitting to the first receiving client via the communication network: First Electronic Content, and Instructions to the first receiving client, so that the first receiving client, in response to receiving the first electronic content, performs the corresponding task associated with the first tag on the first electronic content.

2. The data processing system of claim 1 , wherein the instructions further cause the at least one processor to: causing presentation of a first user interface for a content viewing application on a first client device, the first user interface comprising an annotation menu including the plurality of tags, and Wherein the first request originates from a selection of the first tab of the annotation menu from the first user interface.

3. The data processing system of claim 2, wherein the instructions further cause the at least one processor to: automatically analyzing the first electronic content to identify one or more characters, symbols, or both, including the first electronic content associated with the first tag at one or more locations within the first electronic content; and The first tag is automatically associated with the one or more locations within the first electronic content.

4. The data processing system of claim 2, wherein the instructions further cause the at least one processor to: automatically analyzing the first electronic content to identify one or more characters, symbols, or both, including the first electronic content associated with the first tag at one or more locations within the first electronic content; causing presentation of the one or more locations within the first electronic content on the first client device; receiving, from the first client device, a second request to associate the first tag with a selected location from the one or more locations; as well as In response to receiving the second request, a second association between the first tag and the selected location is recorded.

5. The data processing system of claim 1 , wherein the instructions further cause the at least one processor to: causing presentation of a first user interface for a content viewing application on a first client device, the first user interface comprising a customization menu for customizing a tab from the plurality of tabs; receiving, from the first client device, a second request to associate a second tag from the plurality of tags with a second task from the plurality of tasks; as well as In response to receiving the second request, a second association between the second tag and the second task is recorded in a tag repository maintained by the data processing system.

6. The data processing system of claim 5, wherein the instructions further cause the at least one processor to: receiving, from the first client device over the communications network, a third request to associate the second tag with second electronic content; and In response to receiving the second request, a third association between the second electronic content and the second tag is recorded.

7. The data processing system of claim 6, wherein the instructions further cause the at least one processor to: Transmitting to the second receiving client via the communication network: Second electronic content, and An instruction to the second receiving client, so that the second receiving client performs the second task on the second electronic content in response to receiving the second electronic content.

8. A method for annotating an electronic content item, the method comprising: Receiving, over a communication network, a first request to associate a first tag from a plurality of different tags with first electronic content, wherein each tag from the plurality of tags: is configured to mark electronic content, and being associated with a corresponding task from the plurality of tasks; In response to receiving the first request, recording a first association between the first electronic content and the first tag; as well as Transmitting to the first receiving client via the communication network: First Electronic Content, and Instructions to the first receiving client, so that the first receiving client, in response to receiving the first electronic content, performs the corresponding task associated with the first tag on the first electronic content.

9. The method according to claim 8, further comprising: causing presentation of a first user interface for a content viewing application on a first client device, the first user interface comprising an annotations menu, the annotations menu comprising a plurality of tabs, and Wherein, the first request originates from a selection of the first tab in the annotation menu from the first user interface.

10. The method according to claim 9, further comprising: automatically analyzing the first electronic content to identify one or more characters, symbols, or both, including the first electronic content associated with the first tag at one or more locations within the first electronic content; as well as The first tag is automatically associated with the one or more locations in the first electronic content.

11. The method according to claim 9, further comprising: automatically analyzing the first electronic content to identify one or more characters, symbols, or both, including the first electronic content associated with the first tag at one or more locations within the first electronic content; causing presentation of the one or more locations in the first electronic content on the first client device; receiving, from a first client device, a second request to associate the first tag with a selected location of the one or more locations; as well as In response to receiving the second request, a second association between the first tag and the selected location is recorded.

12. The method according to claim 8, further comprising: causing presentation of a first user interface for a content viewing application on a first client device, the first user interface comprising a customization menu for customizing a tab in the plurality of tabs; receiving, from the first client device, a second request to associate a second tag in the plurality of tags with a second task in the plurality of tasks; as well as In response to receiving the second request, a second association between the second tag and the second task is recorded in a tag repository.

13. The method according to claim 12, further comprising: receiving, from the first client device via the communications network, a third request to associate the second tag with second electronic content; as well as In response to receiving the second request, a third association between the second electronic content and the second tag is recorded.

14. The method according to claim 13, further comprising: Transmitting to the second receiving client via the communication network: Second electronic content, and Instructions to the second receiving client to cause the second receiving client to perform the second task on the second electronic content in response to receiving the second electronic content.

15. A computer-readable storage medium having stored thereon instructions which, when executed, cause a processor of a programmable device to: Receiving, over a communication network, a first request to associate a first tag from a plurality of different tags with first electronic content, wherein each tag from the plurality of tags: is configured to mark electronic content, and being associated with corresponding tasks in a plurality of different tasks; In response to receiving the first request, recording an association between the first electronic content and the first tag; as well as Transmitting to the first receiving client via the communication network: the first electronic content, and Instructions to the first receiving client, so that the first receiving client, in response to receiving the first electronic content, performs the corresponding task associated with the first tag on the first electronic content.

16. The computer-readable storage medium of claim 15, wherein the instructions further cause the processor of the programmable device to: causing presentation of a first user interface for a content viewing application on the first client device, the first user interface comprising an annotation menu including the plurality of tags, and in, The first request results from a selection of the first tab in the annotation menu from the first user interface.

17. The computer-readable storage medium of claim 16, wherein the instructions further cause the processor of the programmable device to: automatically analyzing the first electronic content to identify one or more characters, symbols, or both, including the first electronic content associated with the first tag at one or more locations within the first electronic content; and The first tag is automatically associated with the one or more locations in the first electronic content.

18. The computer-readable storage medium of claim 16, wherein the instructions further cause the processor of the programmable device to: automatically analyzing the first electronic content to identify one or more characters, symbols, or both, including the first electronic content associated with the first tag at one or more locations within the first electronic content; causing presentation of the one or more locations within the first electronic content on the first client device; receiving, from the first user device, a second request to associate a first tag with a selected location of the one or more locations; as well as In response to receiving the second request, a second association between the first tag and the selected location is recorded.

19. The computer-readable storage medium of claim 15, wherein the instructions further cause the processor of the programmable device to: causing presentation of a first user interface for a content viewing application on a first client device, the first user interface comprising a customization menu for customizing a tab from the plurality of tabs; receiving, from the first client device, a second request to associate a second tag in the plurality of tags with a second task in the plurality of tasks; as well as In response to receiving the second request, a second association between the second tag and the second task is recorded in a tag repository.

20. A computer-readable storage medium according to claim 19, wherein the instructions further cause the processor of the programmable device to perform the following operations: receive a third request for associating the second tag with second electronic content from the first client device via the communication network; and in response to receiving the second request, record a third association between the second electronic content and the second tag.

Images