Inter-application transfer of format-specific data objects

Abstract

Generally, the present disclosure relates to techniques that can enable a computing device to recreate a data object that is specially formatted for use in a currently executing application based on other data objects that are specially formatted for a second application in different ways. For example, a computing device can initially execute a first application. The computing device can execute a second application that contains one or more data objects that are specially formatted for use and display in the second application. The first application can receive a first data object, where the format of the first data object is specific to the second application. The first application can determine the content of the first data object and create a second data object that includes at least a portion of the content of the first data object, where the format of the second data object is specific to the first application.

Description

[0001] This application is a divisional application of the following application. International application number of the original application: PCT / US2018 / 037549 The original international filing date was June 14, 2018. The original application's national application number was 201880043389.4. Original application title: Transfer of format-specific data objects between applications Background Technology

[0002] While multiple applications can run concurrently, applications are designed to run as isolated instances of their specific applications, changing state as events (such as user input or fulfilling other requirements) occur within that specific instance. Some applications can communicate with other applications, but this typically requires explicit navigation to relevant data within the respective application and additional user input to initiate data exchange between applications. Users of mobile computing devices may have to provide input to switch between different application graphical user interfaces (GUIs) to complete specific tasks. For example, a user of a computing device might have to stop typing text in a slideshow application, provide input to switch the device to a note-taking application, manually navigate within the note-taking application to locate any note objects that might be related to the slide object being edited in the slideshow application, and provide additional input at the note-taking application's GUI to select or recreate information that the user might want to complete the editing of the slide object. Providing several inputs required for a computing device to perform various tasks can be tedious, repetitive, and time-consuming. Summary of the Invention

[0003] Generally, the present invention relates to techniques for automatically modifying data objects having application-specific formats when moving information contained in data objects between different applications, each of which is associated with a different application-specific data object format. For example, a computing device may execute a first application, such as a slideshow application. The computing device may also execute a second application, which may be associated with one or more data objects, each of which is specifically formatted for use and display in the graphical user interface of the second application. For example, the second application may be a note-taking application, and one or more note documents of the note-taking application may be stored in memory for use within the note-taking application. The first application (e.g., the slideshow application) may receive a first data object having a note-taking application-specific format. Furthermore, the data object may include one or more data structures. For example, the computing device may receive user input instructing the computing device to send a note document from the note-taking application to the slideshow application, the note document being specifically formatted for use in the note-taking application. The first application may create a second data object (e.g., a graphical slide), wherein the format of the second data object is specific to the first application (e.g., the slideshow application). For at least one of one or more data structures of a first data object, a first application can extract the content of the data structure, select a specific data structure of a second data object to place the extracted content, and insert the value of the corresponding data structure of the first data object into the specific data structure of the second data object. In this way, the second application delivers information to the first application in a manner different from the previous copy and paste operation, in which the user must select the specific text to be copied from the first document, select a specific location to place the text in the second document, and manually format the pasted text.

[0004] Different applications typically have different file formats, data formats, and ways of creating data objects with unique structures and layouts. By automatically modifying data objects in this way, the technology disclosed herein provides users with a way to quickly transfer documents, files, or any other data stored in a data object on a second application to a data object in a first application, without requiring the user to manually navigate to the relevant content page within the respective application, manually recreate the data in the first application, or manually reformat the transferred data using common copy and paste techniques for storing content in the memory of a computing device. In this way, the technology disclosed herein can reduce the amount of time and user input required to transfer data objects between applications, which can simplify the user experience and reduce the power consumption of computing devices.

[0005] In one example, this disclosure relates to a method comprising receiving a first data object from a second application executing on a computing device by a first application executing on a computing device. The format of the first data object is specific to the second application. The first data object includes a first set of one or more data structures. The method further comprises creating a second data object by the first application. The format of the second data object is specific to the first application and differs from the format of the first data object. The second data object includes a second set of one or more data structures. The method further comprises, for at least one data structure in the first set of one or more data structures, extracting one or more values ​​from a corresponding data structure of the first data object by the first application; selecting a specific data structure from the second set of one or more data structures selected by the second data object based on metadata associated with the corresponding data structure of the first data object; and inserting one or more values ​​from the corresponding data structure of the first data object into the specific data structure of the second data object by the first application.

[0006] In another example, the invention relates to a computing device including at least one processor and a memory storing instructions associated with a first application. When executed, the instructions cause the at least one processor to receive a first data object from a second application executing on the computing device. The format of the first data object is specific to the second application. The first data object includes a first set of one or more data structures. The instructions, when executed, also cause the at least one processor to determine the contents of the first data object. The instructions, when executed, also cause the at least one processor to create a second data object. The format of the second data object is specific to the first application and differs from the format of the first data object. The second data object includes a second set of one or more data structures. The instructions, when executed, also cause the at least one processor to: extract one or more values ​​from a corresponding data structure of the first data object for at least one data structure in the first set of one or more data structures; select a specific data structure in a second set of one or more data structures of the second data object based on metadata associated with the corresponding data structure of the first data object; and insert one or more values ​​from the corresponding data structure of the first data object into the specific data structure of the second data object.

[0007] In another example, the invention relates to a non-transitory computer-readable storage medium storing instructions that, when executed, cause at least one processor of a computing device to receive a first data object from a second application executing on the computing device. The format of the first data object is specific to the second application. The first data object includes a first set of one or more data structures. When executed, the instructions also cause the at least one processor to determine the contents of the first data object. When executed, the instructions also cause the at least one processor to create a second data object. The format of the second data object is specific to the first application and differs from the format of the first data object. The second data object includes a second set of one or more data structures. When executed, the instructions also cause the at least one processor to: extract one or more values ​​from a corresponding data structure of the first data object for at least one data structure in the first set of one or more data structures; select a specific data structure in a second set of one or more data structures of the second data object based on metadata associated with the corresponding data structure of the first data object; and insert one or more values ​​from the corresponding data structure of the first data object into the specific data structure of the second data object.

[0008] In another example, the invention relates to an apparatus comprising units for receiving a first data object from a second application executing at the computing device. The format of the first data object is specific to the second application. The first data object includes a first set of one or more data structures. The apparatus further includes units for determining the contents of the first data object. The apparatus further includes units for creating a second data object. The format of the second data object is specific to the first application and differs from the format of the first data object. The second data object includes a second set of one or more data structures. The apparatus further includes, for at least one data structure in the first set of one or more data structures, units for extracting one or more values ​​from a corresponding data structure of the first data object, units for selecting a specific data structure in the second set of one or more data structures of the second data object based on metadata associated with the corresponding data structure of the first data object, and units for inserting one or more values ​​from the corresponding data structure of the first data object into the specific data structure of the second data object.

[0009] In another example, the invention relates to a technique that enables a computing device to recreate data objects specifically formatted for use in a currently executing application based on other data objects formatted differently for a second application. For example, the computing device may initially execute a first application. The computing device may execute a second application that includes one or more data objects specifically formatted for use and display in the second application. The first application may receive a first data object, wherein the format of the first data object is specific to the second application. The first application may determine the content of the first data object and create a second data object that includes at least a portion of the content of the first data object, wherein the format of the second data object is specific to the first application.

[0010] Details of one or more examples of this disclosure are set forth in the following drawings and description. Other features, objects, and advantages of this disclosure will be apparent from the specification, drawings, and claims. Attached Figure Description

[0011] Figure 1 This is a conceptual diagram illustrating an example computing device configured to automatically analyze and modify data objects transferred between applications according to one or more aspects of the present invention.

[0012] Figure 2 This is a block diagram illustrating an example computing device configured to automatically analyze and modify data objects transferred between applications according to one or more aspects of the present invention.

[0013] Figures 3A-3B These are conceptual diagrams illustrating a series of exemplary user interfaces of an exemplary computing device according to one or more aspects of this disclosure, the example computing device being configured to automatically analyze data objects formatted for use in a first application and modify the data objects for a second application with a different format.

[0014] Figures 4A-4B This is a conceptual diagram illustrating another series of example user interfaces for an example computing device according to one or more aspects of this disclosure, the example computing device being configured to automatically analyze data objects formatted for use in a first application and modify the data objects for a second application with a different format.

[0015] Figures 5A-5B This is a conceptual diagram illustrating another series of example user interfaces for an example computing device according to one or more aspects of this disclosure, the example computing device being configured to automatically analyze data objects formatted for use in a first application and modify the data objects for a second application with a different format.

[0016] Figures 6A-6BThis is a conceptual diagram illustrating another series of example user interfaces for an example computing device according to one or more aspects of this disclosure, the example computing device being configured to automatically analyze data objects formatted for use in a first application and modify the data objects for a second application with a different format.

[0017] Figure 7 This is a flowchart illustrating example operation of an example computing device according to one or more aspects of the present invention, the example computing device being configured to automatically analyze a data object formatted for use in a first application and modify the data object for a second application with a different format.

[0018] Figure 8 This is a conceptual diagram illustrating an example computing device configured to identify and display information determined to be related to a task being performed, according to one or more aspects of the present invention.

[0019] Figures 9A-9E This is a conceptual diagram illustrating a series of user interfaces for an example computing device according to one or more aspects of this disclosure, the example computing device being configured to identify and display information determined to be related to a task being performed.

[0020] Figure 10 This is a flowchart illustrating further example operations of an example computing device configured to identify and display information determined to be related to a task being performed, according to one or more aspects of the present invention. Detailed Implementation

[0021] Figure 1 This is a conceptual diagram illustrating an example computing device 110 according to one or more aspects of the present invention, which is configured to automatically analyze and modify data objects transferred between applications. Figure 1 In the examples, computing device 110 refers to a standalone mobile or non-mobile computing device. Examples of computing device 110 include mobile phones, tablet computers, laptop computers, desktop computers, servers, host computers, set-top boxes, televisions, wearable devices (e.g., computerized watches, computerized glasses, computerized gloves, etc.), home automation devices or systems (e.g., smart thermostats or home assistant devices), personal digital assistants (PDAs), gaming systems, media players, e-book readers, mobile TV platforms, car navigation or infotainment systems, or any other type of mobile, non-mobile, wearable, and non-wearable computing device configured to execute multiple applications (e.g., applications 118A and 118B) or otherwise access applications related to the task currently being performed by computing device 110.

[0022] The computing device 110 includes a user interface (UID) 112. The UID 112 of the computing device 110 can be used as an input and / or output device for the computing device 110. The UID 112 can be implemented using various technologies. For example, the UID 112 can be used as an input device using a presence-sensitive input screen, such as a resistive touchscreen, a surface acoustic wave touchscreen, a capacitive touchscreen, a projected capacitive touchscreen, a pressure-sensitive screen, an acoustic impulse recognition touchscreen, or another presence-sensitive display technology.

[0023] UID 112 can be used as an input device employing microphone technology, infrared sensor technology, or other input device technologies to receive user input. For example, UID 112 can use built-in microphone technology to detect voice input processed by UI module 120 and / or context module 122 to complete a task. As another example, UID 112 may include a presence-sensitive display that can receive tactile input from a user of computing device 110. UID 112 can receive tactile input indications by detecting one or more gestures from the user (e.g., the user touching or pointing at one or more locations on UID 112 with a finger or stylus).

[0024] UID 112 can be used as an output (e.g., display) device to present output to a user. UID 112 can be used as an output device using any one or more display devices, such as a liquid crystal display (LCD), a dot matrix display, a light-emitting diode (LED) display, an organic light-emitting diode (OLED) display, electronic ink, or a similar monochrome or color display capable of outputting visual information to a user of computing device 110. UID 112 can be used as an output device using speaker technology, haptic feedback technology, or other output device technologies to output information to a user. UID 112 can present a user interface (e.g., user interface 102A or 102B) associated with applications 118A and 118B and format module 121. UID 112 can present a user interface associated with other features of the computing platform, operating system, applications, and / or services performed at and / or accessible from computing device 110 (e.g., email, chat, online services, telephone, games, etc.).

[0025] The computing device 110 also includes applications 118A and 118B, a user interface (UI) module 120, and a formatting module 121. Applications 118A and 118B, as well as modules 120 and 121, can perform the described operations using software, hardware, firmware, or a mixture of hardware, software, and firmware residing in and / or executing on the computing device 110. One or more processors of the computing device 110, or processors of multiple devices, can execute instructions stored in the memory or other non-transitory storage medium of the computing device 110 to perform the operations of applications 118A and 118B, as well as modules 120 and 121. The computing device 110 can execute applications 118A and 118B, as well as modules 120 and 121, as virtual machines executing on the underlying hardware. Applications 118A and 118B, as well as modules 120 and 121, can execute as one or more services of an operating system or computing platform. Applications 118A and 118B, as well as modules 120 and 121, can be executed as one or more executable programs at the application layer of the computing platform.

[0026] UI module 120 manages user interactions with UID 112 and other components of computing device 110. In other words, UI module 120 can act as an intermediary between various components of computing device 110 to determine user input detected by UID 112 and generate output at UID 112 in response to user input. UI module 120 can receive instructions from applications, services, platforms, or other modules of computing device 110 to cause UID 112 to output a user interface (e.g., user interface 14A). UI module 120 can manage input received by computing device 110 as a user view and interactions with the user interface presented at UID 112, and update the user interface in response to additional instructions received from applications, services, platforms, or other modules of computing device 110 that are processing user input.

[0027] like Figure 1 As shown, user interface 102A is a slideshow-style user interface. However, user interface 102A can be any graphical user interface generated during the execution of the first application (e.g., application 118A). Figure 1 In the example, user interface 102A includes a graphical slide object 104A, a title data structure 106A, and a body data structure 108A. The graphical user interface 102A also includes a note object 109A, which the UI module 120 can generate based on the execution of a second application (e.g., application 118B).

[0028] According to the technology disclosed herein, UI module 120 can generate content pages for application 118A currently running on computing device 110. Figure 1In the example, application 118A is a slideshow application. UI module 120 can also execute application 118B to generate content pages for application 118B, which is currently also running on computing device 110. Figure 1 In the example, the content page of application 118A includes a graphic slide object 104A, and the content page of application 118B includes a note object 109A. The content pages of the respective applications can all be included in the user interface 102A.

[0029] Formatting module 121 can receive a first data object from application 118B, and forward the first data object to application 118A. The format of the first data object is specific to application 118B, and the first data object includes a first set of one or more data structures. For example, application 118A can receive a note object 109A from application 118B. Figure 1 As shown, a note object is illustrated in a specific format specific to application 118B. In other words, note object 109A can be constructed such that application 118B can process note object 109A, create note objects structurally similar to note object 109A, and use note object 109A to perform specific functions, wherein other applications such as application 118A may not be configured to perform these same actions. Other applications such as application 118A can access the content stored within such a data object, but in a read-only manner, wherein other applications can extract content from the data structure of the data object, but may not modify or delete the existing data object, or create additional data objects.

[0030] In response to receiving the first data object, application 118A creates a second data object (e.g., a graphic slide object 104B). The second data object includes a second set of one or more data structures. The format of the graphic slide object 104B is application-specific and differs from the format of the note object 109B. Similar to the relationship between the note object 109A and application 118B, the graphic slide object 104B can be configured such that application 118A can process the graphic slide object 104B, create a graphic slide object structurally similar to the graphic slide object 104B, and use the graphic slide object 104B to perform specific functions, wherein other applications, such as application 118B, may not be configured to perform these same actions on the graphic slide object 104B.

[0031] In the case of a first data object and a second data object, application 118A can process the first data object to extract content from one or more data structures of the first data object and place the content within the second data object. For example, for at least one data structure in a first group of one or more data structures of the first object, application 118A can extract one or more values ​​from the corresponding data structure of the first data object. Using metadata associated with the corresponding data structure of the first data object (e.g., Extensible Markup Language (XML) tags), application 118A can select a specific data structure from a second group of one or more data structures for the second data object and insert one or more values ​​from the corresponding data structure of the first data object into the specific data structure of the second data object.

[0032] exist Figure 1 In the example, note object 109A includes a header reading "process for success" and a series of bullet points. All this data can be contained in a single data structure (e.g., a long text string with newline characters, specific spacing characters, bullet point characters, and text characters), two data structures (e.g., a data structure for the header and a data structure for the series of bullet points), or more than two data structures (e.g., a data structure for the header and a corresponding data structure for each bullet point). The formatting module 121 can process the content differently depending on the specific data structure within note object 109A. In any case, the formatting module 121 can extract content from one or more data structures of note object 109A.

[0033] like Figure 1As shown, format module 121 can retrieve the extracted content from note object 109A and populate the data structure of graphic slide object 104B to create a graphic slide object in application 118A. Whether using metadata associated with various data structures of note object 109A or by parsing the extracted content, format module 121 can determine that the phrase "process for success" is being used as a header or title for note object 109A. Application 118A can determine whether the phrase is a header or title using hard-coded information from note object 109A (e.g., header or title XML tags) or inferred data indicating how headers and titles typically appear in various documents (e.g., font size, font weight, text alignment, text position, etc.). Thus, format module 121 can insert the phrase into title data structure 106B. Furthermore, the formatting module 121 can determine whether the bullet list is used as the body of the note object 109A using metadata associated with various data structures (e.g., body XML tags) or by parsing the extracted content using inferred data, where the inferred data indicates how the body text or bullet list typically appears in various documents (e.g., font size, font weight, text alignment, text position, presence of bullet points, dashes, etc.). In this way, the formatting module 121 can insert the bullet list into the body data structure 108B. The UI module 120 can output a user interface 102B for display at UID 112, showing a newly created graphical slide object 104B for application 118A, and the note object 109A from application 118B.

[0034] As understood from the techniques described herein, different applications typically have different file formats, data formats, and ways of creating data objects with unique structures and layouts. Often, in simple file conversions (e.g., from an editable document to a printable document format), an image of the editable file is obtained without considering the specific data structure within either file. In copy and paste operations, the user must explicitly select the specific content to transfer, manually navigate to the specific section of the target destination document where that content should be placed, and manually reformat the transferred content to fit the new application. Some other applications allow users to embed application-unique objects into the application's files, but these require access to external applications. While many applications attempt to provide a universal experience, more specialized applications typically offer users greater control over the content and more application-specific features. However, a problem inherent in computing is that recreating content across different specialized applications is a tedious and imprecise process that requires significant manual input from the user.

[0035] By transferring data objects between applications 118A and 118B, the technology of this disclosure provides a way for users to quickly transfer documents, files, or any other data stored in a data object within a file in application 118B to a data object within a file in application 118A, without requiring the user to manually navigate to the relevant content page within the respective application, recreate the data in application 118A, or reformat the transferred data using common copy and paste techniques for storing content in the memory of computing device 110. In this way, the technology of this disclosure can reduce the amount of time required to transfer data objects between applications and the amount of user input, which can simplify the user experience and reduce the power consumption of computing device 110.

[0036] Figure 2 This is a block diagram illustrating an example computing device according to one or more aspects of this disclosure, the computing device being configured to automatically analyze and modify data objects passed between applications. The following will... Figure 2 The computing device 210 is described as follows Figure 1 Examples of computing devices 110A and / or 110B. Figure 2 This is only one specific example of computing device 210, and many other examples of computing device 210 may be used in some instances and may include a subset of the components contained in the example computing device 210 or may include Figure 2 Additional components not shown in the image.

[0037] like Figure 2 As shown in the example, computing device 210 includes a user interface device (UID) 212, one or more processors 240, one or more communication units 242, one or more input components 244, one or more output components 246, and one or more storage components 248. UID 212 includes a display component 202 and an presence-sensitive input component 204. The storage component 248 of computing device 210 includes applications 218A and 218B, a UI module 220, a context module 222, one or more associated applications 224, and associated object data storage 226.

[0038] One or more processors 240 may implement functions associated with computing device 210 and / or execute instructions associated with computing device 210 to proactively direct a user to information related to a task performed on computing device 210 in application 218A. Examples of processors 240 include application processors, display controllers, auxiliary processors, one or more sensor hubs, and any other hardware configured to function as a processor, processing unit, or processing device. Modules 218A, 218B, 220, and 222 may be operated by processor 240 to perform various actions, operations, or functions of computing device 210. For example, processor 240 of computing device 210 may retrieve and execute instructions stored by storage component 248 that cause processor 240 to perform operations on modules 218A, 218B, 220, and 222. When executed by processor 240, the instructions can cause computing device 210 to provide application 218B with information about application 218A, application 218B determining the context of application 218A and retrieving relevant data objects based on the context.

[0039] UI module 220 may include Figure 1 The UI module 220 of computing device 110 has all the functions of the UI module 120 and can perform operations similar to the UI module 120, such as managing the user interface (e.g., user interfaces 102A and 102B) provided by computing device 210 at UID 212 to facilitate interaction between the user of computing device 110 and applications 218A and 218B. For example, the UI module 220 of computing device 210 can receive information from applications 218A and 218B, including instructions for outputting (e.g., displaying) the accompanying user interface (e.g., user interface 102B). The UI module 220 can receive information from application 218B via communication channel 250 and use the data to generate the user interface. The UI module 220 can send display commands and associated data on communication channel 250 to cause UID 212 to render the user interface at UID 212.

[0040] Format module 221 may include Figure 1 The format module 221 of the computing device 210 has all the functions of the format module 121 and can perform operations similar to those of the format module 121 used to execute application 218B. In some examples, the format module 221 may execute locally (e.g., at processor 240) to provide functionality associated with creating data objects within application 218A. In some instances, the format module 221 may act as an interface to remote companion services accessible to the computing device 210. For example, the format module 221 may be an interface to a remote version of application 218B or an application programming interface (API).

[0041] One or more related applications 224 represent all the various individual applications and services that execute at and are accessible from computing device 210, and that are accessible by application 218B and context module 222 to provide information to the user and / or perform tasks. Numerous examples of related applications 224 may exist and include email applications, chat or messaging applications, calendar applications, document processing applications, contact storage applications, search applications, map or navigation applications, transportation service applications (e.g., bus or train tracking applications), social media applications, internet browser applications, or any and all other applications that can execute at computing device 210. Typically, one or more related applications 224 may include any application in the same suite as application 218A, any application that can exchange data with application 218A, any application developed by the same company as application 218A, or any application that can interact with the same content as application 218A in a data storage device.

[0042] As used throughout this disclosure, the term "context" is used to describe any information that can be used by format module 221, context module 222, and application 218B to define a task or details about that task that the computing device and its user can perform using application 218A at a specific time. For example, after receiving explicit permission from the user, application 218B may receive information about the current state or current task being performed using application 218A and determine the context based on that information. Typically, application 218B may determine the context based on information in the content pages of application 218A, at least a portion of which may be output for display in a graphical user interface. The content pages may include various content data objects or content data structures, each of which may include text, images, media, audio, metadata, computer code, or Uniform Resource Locator (URL) information that application 218B can analyze to determine the context of application 218A.

[0043] Furthermore, throughout the description herein, users may be provided with controls that allow them to select, when analyzing the application context, whether and when the system, program, or feature described herein can collect user information (e.g., information about the user's social networks, social actions or activities, occupation, user preferences, or the user's current location), and if so, the user sends content or communications from the server. Additionally, certain data may be processed in one or more ways before being stored or used, thereby removing personally identifiable information. For example, a user's identity may be processed so that personally identifiable information about the user cannot be determined, or the user's geographic location may be generalized at the place where location information is obtained (e.g., city, zip code, or state level), thereby preventing the determination of the user's specific location. Therefore, users can control what information about themselves is collected, how that information is used, and what information is provided to them.

[0044] One or more storage components 248 within computing device 210 may store information for processing during operation of computing device 210 (e.g., computing device 210 may store data accessed by modules 218A, 218B, 220, 221, and 222 during execution at computing device 210). In some examples, storage component 248 is temporary memory, meaning that the primary purpose of storage component 248 is not long-term storage. Storage component 248 on computing device 210 may be configured to store information temporarily as volatile memory, and therefore the stored contents are not retained when power is lost. Examples of volatile memory include random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), and other forms of volatile memory known in the art.

[0045] In some examples, storage component 248 also includes one or more computer-readable storage media. In some examples, storage component 248 includes one or more non-transitory computer-readable storage media. Storage component 248 may be configured to store a larger amount of information than is typically stored by volatile memory. Storage component 248 may be further configured to permanently store information in non-volatile memory space and retain the information after power-on / power-off cycles. Examples of non-volatile memory include magnetic hard disks, optical disks, floppy disks, flash memory, or electrically programmable memory (EPROM) or electrically erasable programmable memory (EEPROM). Storage component 248 may store program instructions and / or information (e.g., data) associated with modules 218A, 218B, 220, 221, and 222 and data memory 226. Storage component 248 may include memory configured to store data or other information associated with modules 218A, 218B, 220, 221, and 222 and data memory 226.

[0046] Communication channel 250 may interconnect each of components 212, 240, 242, 244, 246, and 248 for inter-component communication (physical, communicative, and / or operational). In some examples, communication channel 250 may include a system bus, a network connection, an inter-process communication data structure, or any other method for transmitting data.

[0047] One or more communication units 242 of computing device 210 can communicate with external devices via one or more wired and / or wireless networks by transmitting and / or receiving network signals on one or more networks. Examples of communication units 242 include network interface cards (e.g., such as Ethernet cards), optical transceivers, radio frequency transceivers, GPS receivers, or any other type of device capable of transmitting and / or receiving information. Other examples of communication units 242 may include shortwave radios, cellular data radios, wireless network radios, and Universal Serial Bus (USB) controllers.

[0048] One or more input components 244 of computing device 210 can receive input. Examples of input are tactile, audio, and video input. In one example, input component 242 of computing device 210 includes presence-sensitive input devices (e.g., touch-sensitive screen, PSD), mice, keyboards, voice response systems, cameras, microphones, or any other type of device for detecting input from a person or machine. In some examples, input component 242 may include one or more sensor components, one or more position sensors (GPS components, Wi-Fi components, cellular components), one or more temperature sensors, one or more motion sensors (e.g., accelerometers, gyroscopes), one or more pressure sensors (e.g., barometers), one or more ambient light sensors, and one or more other sensors (e.g., infrared proximity sensors, hygrometer sensors, etc.). Several other non-limiting examples include heart rate sensors, magnetometers, glucose sensors, olfactory sensors, compass sensors, and step counter sensors.

[0049] One or more output components 246 of computing device 210 may produce output in a selected manner. Examples of the manner may include tactile notification, auditory notification, visual notification, machine-generated voice notification, or other forms. In one example, the output component 246 of computing device 210 includes a presentation-sensitive display, sound card, video graphics adapter card, speaker, cathode ray tube (CRT) monitor, liquid crystal display (LCD), or any other type of device for producing output to a person or machine in a selected manner.

[0050] The UID 212 of computing device 210 may be similar to the UID 112 of computing device 110A, and includes a display component 202 and a presence-sensitive input component 204. The display component 202 may be a screen that displays information (e.g., visual indications) by UID 212, while the presence-sensitive input component 204 may detect data objects at and / or near the display component 202.

[0051] Although shown as an internal component of computing device 210, UID 212 can also represent an external component that shares a data path with computing device 210 for sending and / or receiving input and output. For example, in one example, UID 212 represents a built-in component of computing device 210 that is located within and physically connected to the external package of computing device 210 (e.g., a screen on a mobile phone). In another example, UID 212 represents an external component of computing device 210 that is located outside the package or housing of computing device 210 and physically separate from the package or housing (e.g., a monitor, projector, etc. that shares a wired and / or wireless data path with computing device 210).

[0052] The UID 212 of computing device 210 can detect two-dimensional and / or three-dimensional gestures as input from a user of computing device 210. For example, the sensor of UID 212 can detect user movement (e.g., movement of hand, arm, pen, pointer, etc.) within a threshold distance of the sensor of UID 212. UID 212 can determine a two-dimensional or three-dimensional vector representation of the motion and correlate the vector representation with a gesture input having multiple dimensions (e.g., hand wave, contraction, clamp, stroke, etc.). In other words, UID 212 can detect multidimensional gestures without requiring the user to perform the gesture at or near the screen or surface on which UID 212 outputs information for display. Alternatively, UID 212 can detect multidimensional gestures performed at or near a sensor, which may or may not be located near the screen or surface on which UID 212 outputs information for display.

[0053] According to the technology disclosed herein, UI module 220 can generate content pages for application 218A currently running on computing device 210. Figure 2 In the example, application 218A is a note-taking application. UI module 220 can also execute application 218B to generate content pages for application 218B (e.g., a slideshow application). Figure 2 In the examples, the content page of application 218A may include one or more note objects, and the content page of application 218B may include one or more graphic slide objects. The content pages of the respective applications can both be included in the user interface.

[0054] Format module 221 can receive a first data object from application 218B and forward the first data object to application 218A. The format of the first data object is application 218B specific, and the first data object includes a first set of one or more data structures. For example, application 218A can receive one or more graphic slide objects from application 218B. Figure 2 In the example, one or more graphic slide objects may be represented in a specific format specific to application 218B. In other words, graphic slide objects can be constructed such that application 218B can process them, create graphic slide objects structurally similar to them, and use them to perform specific functions, where other applications such as application 218A may not be configured to perform these same actions. Other applications such as application 218A can access the content stored within the graphic slide objects, but in a read-only manner, whereby other applications can extract content from the data structure of the graphic slide objects without altering or deleting existing data objects, or creating additional data objects.

[0055] In response to receiving the first data object, application 218A creates a second data object (e.g., one or more note objects). The second data object includes a second set of one or more data structures. The format of the one or more created note objects is specific to application 218A and differs from the format of the one or more graphic slide objects. Similar to the relationship between the one or more graphic slide objects and application 218B, the one or more note objects can be configured such that application 218A can process the one or more note objects, create note objects structurally similar to the one or more note objects, and use the one or more note objects to perform specific functions, wherein other applications such as application 218B may not be configured to perform these same actions.

[0056] Having a first data object and a second data object, application 218A can process the first data object to determine and extract its content (e.g., one or more graphic slide objects). For example, for at least one data structure in a first set of one or more data structures of the first object, application 218A can extract one or more values ​​from the corresponding data structure of the first data object. Using metadata associated with the corresponding data structure of the first data object (e.g., Extensible Markup Language (XML) tags or inferred data about the structure, location, or other characteristics of the content), application 218A can select specific data from a second set of one or more data structures for the second data object corresponding to the corresponding data, and insert one or more values ​​from the corresponding data structure of the first data object into the specific data structure of the second data object.

[0057] In various examples, each of one or more graphic slide objects may include various data, such as a topic sequence of text, a main sequence of text, one or more images, one or more videos, mathematical equations, and at least one of a formatted list. Furthermore, each graphic slide object may include different data structures. All this data may be contained within a single data structure or multiple data structures. Depending on the specific data structure within one or more graphic slide objects, the formatting module 221 may process the content differently. In any case, the formatting module 221 may extract content from one or more data structures within one or more graphic slide objects.

[0058] Application 218A can extract content from one or more graphic slide objects and populate the data structure of one or more note objects to create one or more note objects in application 218A based on the content of the data object in application 218B. Alternatively, using metadata associated with the various data structures of one or more graphic slide objects, or by parsing the extracted content, format module 221 can select the appropriate data structure suitable for the extracted content in one or more note objects. For example, application 218A can use hard-coded information (e.g., header or heading XML tags) of note object 109A, or inferred data indicating how headers and headings typically appear in various documents (e.g., font size, font weight, text alignment, text position, etc.) to determine whether a phrase is a header or heading. In some examples, when creating a second data object, format module 221 can read the metadata associated with the corresponding data structure of the first data object (e.g., one or more graphic slide objects) for each data structure of one or more data structures of the first data object. Then, the format module 221 can select a specific data structure of one or more data structures of the created note object based on the metadata associated with the corresponding data structure, and insert one or more values ​​from the corresponding data structure of the graphic slide object into the specific data structure of the note object.

[0059] For example, one or more data structures for a specific graphic slide object may include a title data structure and a body data structure. For the title data structure of the graphic slide object, format module 221 can select the title data structure of the created note object and insert one or more values ​​from the title data structure of the graphic slide object into the title data structure of the created note object. For the body data structure of the graphic slide object, format module 221 can select the body data structure of the created note object and insert one or more values ​​from the body data structure of the graphic slide object into the body data structure of the created note object.

[0060] In some other examples, a particular graphic slide object may include a list of values. Thus, formatting module 221 can create note objects to include an image of the value list or a diagram depicting the value list. This feature can also be used in an example where application 218B is a spreadsheet application that includes a list of values ​​(e.g., a table). If the list of values ​​is chosen to be sent to application 218A, formatting module 221 can change the list of values ​​to an image depicting the values ​​as shown in the spreadsheet application, or create a table based on the values ​​in the list. In other examples, formatting module 221 can reproduce an editable table of values ​​in a format usable by application 218A.

[0061] In some cases, a graphic slide object may include an image. In such an example, formatting module 221 may conditionally format the image of the first data object based on one or more requirements for the image in application 218A. For example, when outputting for display on UID 212, the size of the note object may be smaller than the graphic slide object. Thus, formatting module 221 may crop the selected image or shrink the entire image so that the image will fit the smaller display size. Formatting module 221 may also reduce quality, color characteristics, or perform any other image editing functions to meet any size, color, file size, quality, or any other characteristic requirements. The formatting module 221 may then insert the conditionally formatted image into the note object during creation.

[0062] When creating the second data object, format module 221 can use application 218A to create multiple data objects, each data object including at least a portion of the corresponding data structure of the first data object. The multiple data objects may include the second data object, and the format of each data object may be application-specific and different from the format of the first data object. For example, in an example where application 218A is a slideshow application and application 218B is a note-taking application, the note object in application 218B may include headers and bulleted lists (e.g., similar to...). Figure 1(Note object 109A). In such a case, the formatting module 221 can create multiple graphic slide objects, each including a header and / or a portion of a bulleted list, instead of creating a single graphic slide object including a header and a bulleted list. For example, the first graphic slide object may include the header of the note object as the main title slide, with each item from the bulleted list associated with a different graphic slide object, which is displayed as the header of the corresponding graphic slide object with an empty body portion that the user can fill in as they see fit. In other examples, the formatting module 221 may create a number of graphic slide objects equal to the number of items in the bulleted list of the note object, where the header of the note object is the header of each graphic slide object, and where a single item from the bulleted list fills the body portion of a particular graphic slide object.

[0063] In some examples, a first data object can be included within multiple data objects, where each data object includes corresponding content and corresponding metadata. For example, the first data object could be a first value within a spreadsheet, and each of the multiple data objects could be a value under the same header as the first value. In some instances, within application 218B, one or more data objects within the application can be tagged using a descriptive language that is considered metadata for the data objects, where each data object having a common label as the first data object is included within multiple data objects. Formatting module 221 can receive the first data object (e.g., the first value) using application 218A. Formatting module 221 can select each value having the same corresponding metadata as the first value (i.e., each value under the same header as the first value). Formatting module 221 can insert each value under the header including the first value into a second data object. In this way, formatting module 221 can efficiently move multiple pieces of data between applications 218A and 218B with only a single user interaction, further reducing the time spent reproducing data in alternative applications.

[0064] In some examples, context module 222 may use application 218B to receive information associated with a content page of application 218A before format module 221 receives the first data object. At least a first portion of the content page of the first application may be included in the graphical user interface of UID 212. Context module 222 may use application 218B and based on the information associated with the content page of application 218A to determine the context of application 218A. Context module 222 may use application 218B and based at least in part on the context of application 218A to determine a corresponding set of data objects from each of one or more applications associated with application 218A (e.g., related application 224). The corresponding set of data objects may include the first data object. UI module 220 may use application 218B and based at least in part on the corresponding set of data objects to generate a content page for application 218B, the content page including one or more graphical indications of at least one data object from at least one of the corresponding set of data objects. The at least one data object may include the first data object. Although at least a second portion of the content page for application 218A is included in the graphical user interface, UI module 220 can use application 218B to output at least a portion of the content page of application 218B for displaying within the graphical user interface of computing device 210.

[0065] In other words, application 218B can be a companion application that retrieves data objects from related application 224. Format module 221 can receive one of the retrieved data objects, determine the content of the received data object, and create a data object within application 218A based on the content of the received data object. Further details regarding how application 218B retrieves these data objects will be referenced below. Figure 8-10 Describe it.

[0066] As another example, application 218A may be a spreadsheet application, and the second data object may be one or more cells used in the spreadsheet application. In some such examples, at least one data structure of the first data object may include an equation. Application 218A may also determine that one or more headers of the current document used in the spreadsheet application match one or more variables in the equation of the first data object. Based on this match, application 218A may then format the second data object such that one or more variables in the equation matching the corresponding one or more headers of the current document are replaced by the matching one or more headers in the equation of the second data object.

[0067] While note-taking, spreadsheet, accompaniment, and slide presentation applications have been described above, many other types of applications can utilize the techniques described herein. For example, application 218A or application 218B could be a word processing application, an email application, a calendar application, a file storage application, a messaging application, or any other application that creates data objects based on specific data that can be included in data objects used in other applications.

[0068] Figures 3A-3B These are conceptual diagrams illustrating a series of exemplary user interfaces of an exemplary computing device according to one or more aspects of this disclosure, the example computing device being configured to automatically analyze data objects formatted for use in a first application and modify those data objects for a second application with a different format. Below... Figure 1 In the context of computing device 110 Figures 3A-3B For example, according to one or more aspects of this disclosure, when executed on one or more processors of computing device 110, format module 121 can execute applications 118A and 118B to generate graphical user interfaces 310 and 320. For illustrative purposes only, the following... Figure 1 In the context of computing device 110 Figures 3A-3B .

[0069] exist Figure 3A In the example, computing device 110 can output GUI 310. When computing device 110 outputs GUI 310, it can execute two different applications. The first application could be a slideshow application, and the second application could be a note-taking application. GUI 310 can display content pages associated with each application. The slideshow application can include slide objects, where titles and bodies can be optionally inserted into the slides. Figure 3A In the example, a note-taking application can include a single note object with a header and a list of text.

[0070] A slideshow application running on computing device 110 can receive note objects from a note-taking application. Note objects specifically formatted for use by the note-taking application may not be in a form usable by the slideshow application. Thus, the slideshow application can determine the content of the note objects and create multiple slide objects based on that content. For example, each of the multiple created slide objects may have the same header as the note object. Furthermore, the body of each created slide object may include the content of one of the items in the list within the note object. In other words, the number of slide objects created may equal the number of items in the list contained in the note, thereby creating an entire slideshow based on the content of a single note object. In this way, the slideshow application reformats the note objects so that their content is in a form usable by the slideshow application, thereby creating multiple slide objects based on a single note object. Figure 3B In this context, the computing device 110 can output a GUI 320, which displays a slide object containing the created notes objects.

[0071] Figures 4A-4B These are conceptual diagrams illustrating another series of example user interfaces for an example computing device according to one or more aspects of this disclosure, the example computing device being configured to automatically analyze data objects formatted for use in a first application and modify those data objects for a second application with a different format. Below... Figure 1 In the context of computing device 110 Figures 4A-4B For example, according to one or more aspects of this disclosure, when executed on one or more processors of computing device 110, format module 121 can execute applications 118A and 118B to generate graphical user interfaces 410 and 420. For illustrative purposes only, the following... Figure 1 In the context of computing device 110 Figures 4A-4B .

[0072] exist Figure 4A In the example, computing device 110 can output GUI 410. When computing device 110 outputs GUI 410, it can execute two different applications. The first application could be a slideshow application, and the second application could be a note-taking application. GUI 410 can display content pages associated with each application. The slideshow application can include slide objects, where titles and bodies can be optionally inserted into the slides. Figure 4A In the example, the note-taking application can include multiple note objects, some of which have headers and content, while others only have content. Furthermore, each note object is categorized under the main tag "meeting".

[0073] A slideshow application running on computing device 110 can receive multiple note objects from a note-taking application. Each note object, specifically formatted for use in the note-taking application, may not be in a format usable by the slideshow application. Thus, the slideshow application can determine the corresponding content for each note object and create a corresponding slide object for each note object, where the corresponding slide object is based on the content of each note object.

[0074] For example, the first slide object could simply include a title based on the tag "meeting," used to group multiple data objects. Each subsequent slide could be based on the note object itself, including a header for the corresponding note object (if it exists) that serves as the title of that specific slide. Furthermore, the body of each created slide object could include the content of one note object from multiple note objects. In other words, the number of slide objects created could equal the number of note objects associated with a specific tag, plus any additional slide objects used for the tag itself, thus creating the entire slide presentation from the content of a single note object. In this way, the slide presentation application reformatts the note objects so that their content is in a form usable by the slide presentation application, thereby creating multiple slide objects based on multiple note objects. Figure 4B In this context, the computing device 110 can output a GUI 420 that displays a slide object containing the created notes objects.

[0075] Figures 5A-5B These are conceptual diagrams illustrating another series of example user interfaces for an example computing device according to one or more aspects of this disclosure, the example computing device being configured to automatically analyze data objects formatted for use in a first application and modify those data objects for a second application with a different format. The following... Figure 1 In the context of computing device 110 Figures 5A-5B For example, according to one or more aspects of this disclosure, when the format module 121 executes on one or more processors of the computing device 110, it can execute application 118B to produce graphical user interfaces 510 and 520. For illustrative purposes only, the following... Figure 1 In the context of computing device 110 Figures 5A-5B .

[0076] While each of the preceding examples illustrates the creation of slide objects based on note objects, computing device 110 can also perform the inverse function of this. In other words, computing device 110 can create one or more note objects based on one or more slide objects. Figure 5AIn the example, computing device 110 can output GUI 510. When computing device 110 outputs GUI 510, it can execute two different applications. The first application could be a slideshow application, and the second application could be a note-taking application. GUI 510 can display content pages associated with each application. Figure 5A In the example, a slideshow application can include multiple slide objects, some of which have titles and body content, while others only have content. Furthermore, the first slide object only includes a title.

[0077] A note-taking application running on computing device 110 can receive multiple slide objects from a slide presentation application. Each slide object, specifically formatted for use in the slide presentation application, may not be in a format available to the note-taking application. Thus, the note-taking application can determine the appropriate content for each slide object and create a corresponding note object for each slide object, where the corresponding note object is based on the content of each slide object.

[0078] For example, the first slide object could simply include the title "meeting". The note-taking application can then tag each of the created note objects named "meeting". Each note object included under this tag can be based on the remaining slide objects, including the title of the corresponding slide object (if it exists) as the header for that specific note object. Furthermore, the body of each created note object can include the content from one of the multiple slide objects. This allows the note-taking application to reformat the slide objects so that the content of the slide objects is in a form usable by the note-taking application, thereby creating multiple note objects based on multiple slide objects. Figure 5B In this context, the computing device 110 can output a GUI 520, which displays a slide object containing the created notes objects.

[0079] Figures 6A-6B This is a conceptual diagram illustrating another series of example user interfaces for an example computing device according to one or more aspects of this disclosure, the example computing device being configured to automatically analyze data objects formatted for use in a first application and modify the data objects for a second application with a different format. Figures 6A-6B It is below Figure 1 The description is within the context of a computing device 110. For example, according to one or more aspects of this disclosure, when executed on one or more processors of the computing device 110, the format module 121 can execute applications 118A and 118B to generate graphical user interfaces 610 and 620. For illustrative purposes only, the following description... Figure 1 In the context of computing device 110 Figures 6A-6B .

[0080] like Figure 6A As shown, applications other than slideshow or note-taking apps can utilize the techniques described herein. For example, in Figure 6A In the example, the first application could be a spreadsheet application, while the second application could be a note-taking application.

[0081] exist Figure 6A In the example, computing device 110 can output GUI 610. GUI 610 can display content pages associated with each application. Figure 6A In the example, the spreadsheet application can include multiple cell objects, in which content has already been placed. The four cell objects in GUI 610 are header cell objects (i.e., cells containing the text "Object", "Mass(kg)", "Acceleration (m / s2)", and "Force (N)"). Furthermore, in... Figure 6A In the example, a note-taking application can include a single note object containing the equation "Force = Mass x Acceleration".

[0082] A spreadsheet application running on computing device 110 can receive note objects from a note-taking application. Note objects specifically formatted for use by the note-taking application may not be in a form usable by the spreadsheet application. Thus, the spreadsheet application can determine the content of the note objects and create one or more cell objects based on that content.

[0083] For example, in Figure 6B In the example, computing device 110 can create three cell objects based on the equation in the note object. Computing device 110 can determine that the variables in the equation of the note object match two of the headers present in the already filled cell objects (i.e., mass and acceleration). Thus, computing device 110 can replace the variables in the equation with references to columns in a spreadsheet that match the listed variables. Furthermore, computing device 110 can determine that the product in the equation matches a third header (i.e., force). Thus, computing device 110 can insert the reformatted equation into the "force" column in one or more rows of cells containing content. In this way, the spreadsheet application reformatts the note object so that its content is in a form usable by the spreadsheet application, thereby converting the equation into a form recognizable by the spreadsheet application. Computing device 110 can output a GUI 620 showing the created cell objects including the content of the note object.

[0084] Figure 7This is a flowchart illustrating further example operation of an example computing device according to one or more aspects of the present invention, the example computing device being configured to transfer data objects formatted for use in a first application to a second application having a different format. Below... Figure 1 In the context of computing device 110 Figure 7 For example, according to one or more aspects of this disclosure, when executed on one or more processors of computing device 110, format module 121 can execute application 118B to perform operations 700-720. For illustrative purposes only, the following... Figure 1 In the context of computing device 110 Figure 10 .

[0085] According to the technology disclosed herein, computing device 110 uses a first application executing at computing device 110 and receives a first data object (700) from a second application executing at computing device 110. The format of the first data object may be specific to the second application, and the first data object may include a first set of one or more data structures. In some cases, one or more values ​​of at least one data structure of the first data object may include at least one of the following: a sequence of titles of text, a main sequence of text, one or more images, one or more videos, mathematical equations, and formatted lists. Using the first application, computing device 110 may classify content into one of these types of content based on metadata associated with at least one data structure of the first object, the metadata including hard-coded tags (e.g., XML tags) or inferred data corresponding to the structure, layout, or visual characteristics of the content.

[0086] The computing device 110 uses a first application to create a second data object (710). The second data object may be specifically formatted for the first application and is different from the first data object. The second data object may include a second set of one or more data structures.

[0087] For at least one data structure in the first group of one or more data structures, computing device 110 can extract one or more values ​​from the corresponding data structure of the first data object (720). Based on the metadata associated with the corresponding data structure of the first data object, computing device 110 can select a specific data structure from the second group of one or more data structures for the second data object (730). Then, computing device 110 can insert one or more values ​​from the corresponding data structure of the first data object into the specific data structure of the second data object (740). In some examples, computing device 110 can output a graphical user interface for display, which includes at least a portion of the content of the second data object.

[0088] The computing device 110 can determine whether another data structure of the first data object will be included in the second data object (750). If the computing device 110 determines that another data structure of the first data object exists to be included in the second data object (the "yes" branch of 750), the computing device 110 can repeat the process (steps 720-740) to extract content from the data structure and insert that content into the data structure of the second data object. If the computing device 110 determines that no other data structure of the first data object exists to be included in the second data object (the "no" branch of 750), the computing device 110 can terminate the process.

[0089] For example, one or more data structures of the first data object may include a header data structure and a body data structure. For the header data structure of the first data object, the computing device 110 can select the header data structure of the second data object and insert one or more values ​​from the header data structure of the first data object into the header data structure of the second data object. For the body data structure of the first data object, the computing device 110 can select the body data structure of the second data object and insert one or more values ​​from the body data structure of the first data object into the body data structure of the second data object.

[0090] As another example, the first application is a spreadsheet application, and the second data object can be one or more cells used in the spreadsheet application. In some such examples, when determining the content of the first data object, the computing device 110 determines that the content of the first data object includes equations. The computing device 110 can determine that one or more headers of the current document used in the spreadsheet application match one or more variables in the equations. The computing device 110 can then format the second data object such that one or more variables in the equations that match the corresponding one or more headers of the current document are replaced by the matching one or more headers in the equations of the second data object.

[0091] The first data object may also include a data structure for the list of values. Thus, the computing device 110 can create the second data object as an image of the list of values ​​or a graph depicting the list of values.

[0092] The first data object may also include a data structure for an image. In such an example, the computing device 110 may conditionally format the image of the first data object based on one or more requirements for the image in the first application. The computing device 110 may then insert the conditionally formatted image into the second data object during creation.

[0093] When creating a second data object, computing device 110 may use a first application to create multiple data objects, each data object including at least a portion of the corresponding data structure of the first data object. The multiple data objects may include the second data object, and the corresponding format for each of the multiple data objects may be application-specific and may differ from the format of the first data object.

[0094] The first application may also include a slideshow application, wherein the second data object is one or more graphic slides for the slideshow application. The second application may be a note-taking application, and the first data object is one or more note objects for the note-taking application. In other examples, the first application may be a note-taking application, the second data object may be one or more note objects for the note-taking application, the second application may be a slideshow application, and the first data object may be one or more note graphic slides for the note-taking application.

[0095] The first data object may be included in a plurality of data objects, wherein each of the plurality of data objects includes corresponding content and corresponding metadata. The computing device 110 may also receive the plurality of data objects using a first application. The computing device 110 may use the first application to select each of the plurality of data objects having the same corresponding metadata as the first data object. The computing device 110 may use the first application to insert one or more values ​​of at least one data structure of the first data object and one or more corresponding values ​​of one or more corresponding data structures of each of the plurality of data objects into a second data object, wherein one or more corresponding values ​​of one or more corresponding data structures of each of the plurality of data objects have the same corresponding metadata as the first data object.

[0096] Computing device 110 may receive information associated with a content page of a first application executed on the computing device using a second application before receiving a first data object. At least a first portion of the content page for the first application may be included in the graphical user interface of computing device 110. Computing device 110 may use the second application and, based on the information associated with the content page of the first application, determine the context of the first application. Computing device 110 may use the second application and, at least in part, based on the context of the first application, determine a corresponding group of data objects for each of one or more applications associated with the first application. The corresponding group of data objects may include the first data object. Computing device 110 may use the second application and, at least in part, based on the corresponding group of data objects to generate a content page for the second application, the content page including one or more graphical indications of at least one data object from at least one group of data objects in the corresponding group. The at least one data object may include the first data object. When at least a second portion of the content page for the first application is included in the graphical user interface, computing device 110 may use the second application to output at least a portion of the content page for the second application within the graphical user interface of computing device 110 for display.

[0097] When determining the context of the second application, computing device 110 may use the second application to identify one or more content objects in a content page for the first application, extract at least one corresponding attribute of each of the one or more content objects, and determine the context of the first application based at least in part on one or more of the corresponding attributes of each of the one or more content objects. The one or more content objects may include one or more of text, images, metadata, application status, application code, field identifiers, filenames, or file types.

[0098] When determining the corresponding group of data objects for each of one or more applications related to the first application, the computing device 110 may use a second application to construct a corresponding search query for each of the one or more applications related to the first application based on the context, and execute each corresponding search query from the corresponding search query to retrieve the corresponding group of data objects from the data of the corresponding application of one or more applications related to the first application.

[0099] One or more applications associated with the first application may include at least two applications associated with the first application, and when generating a content page for the first application, the computing device 110 may use a second application and generate a content page for the second application based at least in part on data objects from corresponding groups of data objects for the at least two applications associated with the first application, the content page for the second application including at least one graphical indication of at least one data object from corresponding group of data objects for each of the at least two applications associated with the first application.

[0100] The computing device 110 can also use a second application to receive user input instructions, which select a first graphical indicator of a first data object from one or more graphical indicators of at least one data object included in the content page of the second application. Using the second application, the computing device 110 can send the first data object to the first application.

[0101] The computing device 110 may also use a second application to receive an instruction for selecting one or more graphical indications representing one or more content objects included in a content page of a first application, determine an update context for the first application based on one or more content objects, determine, at least in part based on the update context of the first application, determine data objects of a corresponding update group for each of the one or more applications associated with the first application, generate an update content page for the first application based at least in part based on the data objects of the corresponding update group, the updated content page including one or more graphical indications of at least one data object from at least one group of data objects in the corresponding update group, and output a portion of the updated content page of the second application for display within the graphical user interface of the computing device 110 when at least a third portion of the content page for the first application is included in the graphical user interface.

[0102] When outputting a portion of the content page for the second application, the computing device 110 may use the second application to replace a portion of the graphical user interface with a portion of the content page for the first application, while at least a second portion of the content page for the first application is included in the graphical user interface.

[0103] A portion of the content page for displaying a second application includes one or more graphical indicators from at least one data object in a first set of data objects for an application from one or more applications associated with the first application. The computing device 110 can also receive user input indicating that the user selects an application different from one or more applications associated with the first application. The computing device 110 can generate an updated content page for the first application, at least in part, based on the corresponding set of data objects, the updated content page including one or more graphical indicators from at least one data object in a second set of data objects for an application different from one or more applications associated with the first application. When at least a second portion of the content page for the first application is included in the graphical user interface, the computing device 110 can output a portion of the updated content page for the second application for display within the graphical user interface of the computing device.

[0104] The computing device 110 may receive a user input instruction to begin executing a second application before determining the context, and in response to receiving the user input instruction, use the second application to determine the context. In some examples, the second application may be a companion application.

[0105] Figure 8 This is a conceptual diagram illustrating an example computing device 810 according to one or more aspects of the present invention, which is configured to identify and display information determined to be related to a task being performed. Figure 8 In the example, computing device 810 may be and include similar Figure 1 The computing device 110 is a component of the computing device 810. The computing device 810 includes a UID 812, which can be similar to... Figure 1 UID 112. Figure 8 The example shows an instance where the data object passed to application 818A originates from an accompanying application that can identify and display information from one or more related applications identified as being associated with the task being performed.

[0106] The computing device 810 also includes applications 818A and 818B, a user interface (UI) module 820, and a context module 822. Applications 818A and 818B, as well as modules 820 and 822, can perform the described operations using software, hardware, firmware, or a mixture of hardware, software, and firmware residing in and / or executing on the computing device 810. One or more processors of the computing device 810, or processors of multiple devices, can execute instructions stored in the memory or other non-transitory storage medium of the computing device 810 to perform the operations of applications 818A and 818B, as well as modules 820 and 822. The computing device 810 can execute applications 818A and 818B, as well as modules 820 and 822, as virtual machines executing on the underlying hardware. Applications 818A and 818B, as well as modules 820 and 822, can execute as one or more services of an operating system or computing platform. Applications 818A and 818B, as well as modules 820 and 822, can be executed as one or more executable programs at the application layer of the computing platform.

[0107] UI module 820 manages user interactions with UID 812 and other components of computing device 810. In other words, UI module 820 can act as an intermediary between various components of computing device 810 to determine user input detected by UID 812 and generate output at UID 812 in response to user input. UI module 820 can receive instructions from applications, services, platforms, or other modules of computing device 810 to cause UID 812 to output a user interface (e.g., user interface 802A). UI module 820 can manage input received by computing device 810 as a user view and interact with the user interface presented at UID 812, and update the user interface in response to additional instructions received from applications, services, platforms, or other modules of computing device 810 that are processing user input.

[0108] like Figure 8 As shown, user interface 802A is an email user interface. However, user interface 802A can be any graphical user interface generated during the execution of the first application (e.g., application 818A). Figure 8 In the example, user interface 802 A includes content object 804A (i.e., the "to" field of the email communication), content object 804B (i.e., the "from" field of the email communication), and content object 804C (i.e., the body of the email communication).

[0109] According to the technology disclosed herein, the UI module 820 can execute application 818A to generate content pages for application 818A currently running on computing device 810. Figure 8In the example, application 818A is an email application, which is part of the application's web suite. The content page of the email application may include a "To" field, a "From" field, a subject line, an email body area, an email list, additional content in an email chain, or any additional content related to the user's email inbox or outbox, including text, images, metadata, application status, application code, field identifiers, filenames, or file types.

[0110] UI module 820 can output a graphical user interface 802A for display via UID 812, the graphical user interface 802A including at least a portion of the content page of application 818A. Figure 8 In the example, the portion of the content page included in the graphical user interface 802A consists of content object 804A (i.e., the "to" field of the email communication), content object 804B (i.e., the "from" field of the email communication), and content object 804C (i.e., the body of the email communication). However, in other instances, UI module 820 may output more or fewer content objects based on the graphical real estate available in UID 812 and the amount of content contained within each content object.

[0111] After receiving explicit permission from the user to do so, context module 822 can execute application 818B to receive information associated with the content page of application 818A. Context module 822 can then execute application 818B to determine the context of application 818A based on the content page of application 818A. The context may include any information about the current state of application 818A, which may provide indications of the current task being performed within application 818A, future tasks to be performed within application 818A, or any other information related to application 818A. For example, before executing application 818B, UI module 820 may output a request for explicit user permission to execute application 818B and transmit the information associated with the content page to 818A. After receiving explicit permission to do so, context module 822 may begin executing application 818B. If explicit permission is denied, context module 822 may instead prevent the execution of application 818B.

[0112] exist Figure 8In the example, context module 822 can determine that content object 804A indicates that the current email communication is an email received by a user addressed to computing device 810. Context module 822 can also determine that content object 804B indicates that the user's boss, "Manager John," is the author of the current email communication, and therefore the email may be work-related. Finally, context module 822 can also determine that content object 804C indicates that the user's boss is requesting a document co-authored by both the user and the boss regarding a transaction with company X.

[0113] although Figure 8 The example describes a context module 822 using a content object that UI module 820 is currently outputting to be displayed via UID 812. However, context module 822 can analyze any content object within the content page during the context determination process, including content objects that UI module 820 is not currently outputting for display but are on the same content page. For example, content object 804C could be a response to a previous email chain that is included on the content page but not on the screen. When determining the context for applying 818A, context module 822 can use the previous email chain as additional data.

[0114] Based at least in part on the context of application 818A, context module 822 can execute application 818B to determine the corresponding group of data objects for each of one or more applications associated with application 818A. For example, application 818A (an email application) may be part of a network suite of applications. Application 818B may be configured to access data from other applications within the same network suite, which will be associated with application 818A. In some cases, any application that can exchange data with application 818A can be considered an associated application. Furthermore, any application developed by the same company as application 818A can be considered an associated application. In some instances, any application that can interact with the same content as application 818A in a data storage device can be considered an associated application.

[0115] exist Figure 8 In the example, application 818B determines the context of application 818A as a user viewing an email exchange with the user's boss in order to find a specific document that the user and the boss collaborated on during drafting. Then, context module 822 can execute application 818B to search for documents within a document editing / storage application related to application 818A based on the determined context (e.g., documents about company X and transactions occurring with said company X). Application 818B can retrieve a list of documents associated with the relevant document editing / storage application, including references to company X, parties involved in the user's business, documents co-authored by the user and the user's boss, etc.

[0116] Application 818B can forward the list of retrieved data objects to UI module 820, which can generate a content page for application 818B. This content page includes one or more graphical representations of at least one data object from at least one group of data objects in the corresponding set. The content page may include a portion or all of the set of data objects retrieved by application 818B. UI module 820 can then output a graphical user interface 802B, which includes at least a portion of the content page for application 818B, while simultaneously displaying at least a portion of graphical user interface 802A.

[0117] For example, application 818B can retrieve a set of ten documents related to company X, a transaction where the user is one party, or a transaction co-created by the user and their boss. Application 818B can then rank the documents based on how well they match a given context. Figure 8 In the example, document 806A, titled "Corporation X Deal.doc", could be a document co-authored by the user and the user's boss regarding a transaction between the user's company and company X. Document 806B, titled "Corporation X Fee Agreement.doc", could be a document co-authored by the user and the user's boss regarding a previous fee agreement between the user's company and company X. Document 806C, titled "Corporation R Deal.doc", could be a document created by the user regarding a transaction between the user's company and company R. UI module 820 can determine that UID 812 is large enough to display three of the ten documents. UI module 820 can output a graphical user interface 802B, such as displaying the first three documents out of a total of ten retrieved documents, or the three documents ranked highest in the sorted list of the previously generated application 818B (i.e., documents 806A-806C). The graphical user interface 802B can be scrollable, allowing the user to see each of the other seven documents not initially shown in the graphical user interface 802B.

[0118] By utilizing application 818B to determine contextual information about application 818A executed on computing device 810, computing device 810 can provide a user with a way to quickly access documents, files, or any other stored data objects identified as related to tasks performed on computing device 810 by application 818A, without requiring the user to switch between several different applications, manually navigate to relevant content pages within the respective applications, or understand what input is needed to move data objects between two different applications. For example, in Figure 8In the example, a user can drag document 806A (a possible document requested by the user's boss) into the reply field of the graphical user interface of application 818A to attach the requested document to the reply email. In this way, the technology of the present invention reduces the amount of time required to retrieve relevant data objects in other applications and the amount of user input, which simplifies the user experience and reduces the power consumption of computing device 810.

[0119] By providing application 818B with contextual information associated with application 818A, the technology of this disclosure can provide users with a way to quickly obtain documents, files, or any other data identified as being related to a task performed using application 818A, without requiring users to switch between several different applications, manually navigate to relevant content pages within the respective applications, or understand what input is required to move data objects between two different applications. In this way, the technology of the present invention can reduce the amount of time required to retrieve relevant data objects from other applications and the amount of user input, which simplifies the user experience and reduces the power consumption of computing device 810.

[0120] Figures 9A-9C This is a conceptual diagram illustrating a series of operations of an exemplary computing device according to one or more aspects of this disclosure, the exemplary computing device being configured to execute accompanying applications to identify and display information from various applications that may be related to tasks performed using the currently executing applications. The following... Figure 2 FIG9 is described in the context of computing device 210. For example, according to one or more aspects of this disclosure, when executed on one or more processors of computing device 210, context module 222 can execute application 218B to generate graphical user interface 960A-960E. For illustrative purposes only, the following description... Figure 1 Figure 9 describes the scenario of computing device 210.

[0121] exist Figure 9A In this context, computing device 210 can execute application 218A to generate a graphical user interface 960A. Figures 9A-9C In the example, application 218A could be a word processing application. The graphical user interface 960A shows a user editing a text document about a client meeting of Company Z on August 8, 2017. Further details of the meeting, including the agenda, are included in the graphical user interface 960A. The content page of application 218A can be the entire text document, even if the graphical user interface 960A only includes a portion of the text document.

[0122] At a certain point, computing device 210 can begin executing application 218B. Application 218B can determine the context of application 218A based on the content pages (i.e., the text document as a whole). Application 218B can extract information about the text document, including any additional author information and the content of the text document. Based on this context, application 218B can retrieve the corresponding group of data objects from applications related to application 218A. Figure 9A In the example, application 218B can retrieve data objects from an email application, a calendar application, and a contacts application, each of which is associated with application 218A. This list of applications can be described by graphical element 964, which includes a composite page containing data objects associated with all applications.

[0123] exist Figure 9A In the example, application 218B can determine that the title of the text document is "Client Meeting Kick-Off". Application 218B can also determine that the relevant email application includes an email from a collaborator of the text document with the subject line "Re: Client Meeting Kick-Off Agenda". Application 218B can determine that the email may include information useful to the user in the given current context and output a graphical indication 962A of the email message.

[0124] Application 218B can also find calendar events for meetings discussed in word documents within the calendar application, as well as contact information for one of the meeting attendees. Thus, application 218B can also output graphical indicators 962B and 962C for these objects respectively in the graphical user interface 960B.

[0125] exist Figure 9B In the graphical user interface 960C, the user can select an email application from a list of graphical indicators 964, as the user may be looking for a specific email discussing a text document. Thus, application 218B can generate an updated content page that includes only emails retrieved from the email application's data storage. Graphical indicator 966A may be the same email message initially shown in the graphical user interface 960B. However, the graphical user interface 960C may also include two additional graphical indicators 966B and 966C associated with additional emails not initially displayed in the graphical user interface 960B.

[0126] Then, application 218B can receive user input indicating that the user selects graphical instruction 966A and the associated email. In this way, application 218B can access the relevant email application to update the graphical user interface to display additional information about the selected email message. For example, application 218B can output this information for display in the graphical user interface 960D, the subject line 968A, the contact line 968B, and the email body 968C.

[0127] exist Figure 9C In this context, application 218B can receive instructions to select further user input for the email body 968C. Thus, within the graphical user interface 960E, application 218B can use the selected email body 968C to perform the functions found in application 218A. Figure 9C In the example, this feature is related to the "copy and paste" function, which pastes the body of an email into a text document. In this way, users can quickly navigate between multiple applications that can communicate with each other and transfer relevant information between them, without having to completely navigate away from the initial application.

[0128] Figure 10 This is a flowchart illustrating further example operation of an example computing device according to one or more aspects of the present invention, the example computing device being configured to execute accompanying applications to identify and display information from various applications that may be related to tasks performed using the currently executing applications. The following... Figure 8 In the context of computing device 810 Figure 10 For example, according to one or more aspects of this disclosure, when scenario module 822 executes at one or more processors of computing device 810, application 818B may be executed to perform operations 1000-1040. For illustrative purposes only, the following... Figure 8 In the context of computing device 810 Figure 10 .

[0129] According to the technology of the present invention, the context module 822 can execute application 818B to receive information (1000) associated with the content page of application 818A for execution at computing device 810. The context module 822 can then execute application 818B to determine the context of application 818A based on the content page of application 818A (1010). The context may include any information about the current state of application 818A, which may provide indication of the current task being performed within application 818A, future tasks to be performed within application 818A, or any other information related to application 818A.

[0130] Although Figure 8The example describes a context module 822 using the content object currently being output by UI module 820 for display via UID 812. However, context module 822 can analyze any content object within the content page during the context determination process, including content objects not currently output by UI module 820 for display but on the same content page. For example, content object 804C could be a response to a previous email chain that is included on the content page but not on the screen. When determining the context for application 818A, context module 822 can use the previous email chain as additional data.

[0131] Based at least in part on the context of application 818A, context module 822 can execute application 818B to determine the corresponding group of data objects (1020) for each application from one or more applications associated with application 818A. For example, application 818A (an email application) may be part of a network suite of applications. Application 818B may be configured to access data from other applications within the same network suite, which will be associated with application 818A. In some cases, any application that can exchange data with application 818A may be considered an associated application. Furthermore, any application developed by the same company as application 818A may be considered an associated application. In some examples, any application that can interact with the same content of application 818A in a data storage device may be considered an associated application.

[0132] Application 818B can forward the list of retrieved data objects to UI module 820, which can generate a content page for application 818B, the content page including one or more graphical indicators (1030) of at least one data object from at least one group of data objects in the corresponding group. The content page may include part or all of the group of data objects retrieved by application 818B. UI module 820 can then output a graphical user interface 802B, which includes at least a portion of the content page for application 818B, while displaying at least a portion (1040) of graphical user interface 802A.

[0133] By utilizing application 818B to determine contextual information about application 818A executed on computing device 810, computing device 810 can provide a user with a way to quickly access documents, files, or any other stored data objects identified as related to tasks performed on computing device 810 by application 818A, without requiring the user to switch between several different applications, manually navigate to relevant content pages within the respective applications, or understand what input is needed to move data objects between two different applications. For example, in Figure 8In the example, a user can drag document 806A (a possible document requested by the user's boss) into the reply field of the graphical user interface of application 818A to attach the requested document to the reply email. In this way, the technology of the present invention reduces the amount of time required to retrieve relevant data objects in other applications and the amount of user input, which simplifies the user experience and reduces the power consumption of computing device 810.

[0134] Typically, the techniques described above describe the transfer of information between two applications running on a computing device, allowing one application to utilize information that is normally only available in the other application. However, different applications often have different data objects with different formats, meaning that the transfer of such data objects is not always possible. Additional editing of the transferred content or additional manual input is required to update the format of the copied data objects.

[0135] like Figure 10 As further shown, application 818A of computing device 810 can receive a data object (1050) from application 818B. The format of the first data object may be specific to the second application, and the first data object may include a first set of one or more data structures. In some cases, one or more values ​​of at least one data structure of the first data object may include at least one of the following: a sequence of text headings, a main sequence of text, one or more images, one or more videos, mathematical equations, and a formatted list. Application 818A may classify the content into one of these types of content based on metadata associated with at least one data structure of the first object, the metadata containing hard-coded tags (e.g., XML tags) or inferred data corresponding to the structure, layout, or visual characteristics of the content.

[0136] The application 818A creates a second data object (1060). The second data object may be specifically formatted for the first application and is different from the first data object. The second data object may include a second set of one or more data structures.

[0137] For at least one data structure in the first group of one or more data structures, application 818A can extract one or more values ​​from the corresponding data structure of the first data object (1070). Based on the metadata associated with the corresponding data structure of the first data object, application 818A can select a specific data structure from the second group of one or more data structures for the second data object (1080). Then, application 818A can insert one or more values ​​from the corresponding data structure of the first data object into the specific data structure of the second data object (1090).

[0138] Application 818A determines whether another data structure of the first data object will be included in the second data object (1100). If application 818A determines that another data structure of the first data object exists to be included in the second data object (the "Yes" branch of 1100), application 818A can repeat the process (steps 1070-1090) to extract content from the data structure and insert that content into the data structure of the second data object. If application 818A determines that no other data structure of the first data object exists to be included in the second data object (the "No" branch of 1100), application 818A can terminate the process.

[0139] As understood from the techniques described herein, different applications typically have different file formats, data formats, and ways of creating data objects with unique structures and layouts. Often, in simple file conversions (e.g., from editable documents to printable document format files), an image of an editable file is obtained without considering the specific data structures within either file. In copy and paste operations, the user must explicitly select the specific content to transfer, manually navigate to the specific section of the target destination document where that content should be placed, and manually reformat the transferred content to fit the new application. Some other applications allow users to embed application-unique objects into the application's files, but these require access to external applications. While many applications attempt to provide a universal experience, more specialized applications typically offer users greater control over the content and more application-specific features. However, an inherent problem with computing is that recreating content across different specialized applications is a cumbersome and imprecise process that requires significant manual input from the user.

[0140] By transferring data objects between applications, the technology of this disclosure provides a way for users to quickly transfer documents, files, or any other data stored in a data object within a file of a second application to a data object within a file of a first application, without requiring the user to manually navigate to the relevant content page within the respective application, recreate the data in the first application, or reformat the transferred data using common copy and paste techniques for storing content in the memory of a computing device. Different applications may serve different functions, but a single user may want to use the functions of multiple applications to complete a task. By reducing the digital and user effort required to transfer information between applications, computing devices can perform tasks with greater efficiency, a simplified user experience, and lower power consumption. In this way, the technology of this disclosure can reduce the amount of time and user input required to transfer data objects between applications, which simplifies the user experience and reduces the power consumption of computing devices.

[0141] Clause 1. A method comprising: receiving a first data object from a second application executing on a computing device by a first application, wherein the format of the first data object is specific to the second application, and wherein the first data object includes a first set of one or more data structures; creating a second data object by the first application, the second data object being specific to the first application and having a different format from the first data object, wherein the second data object includes a second set of one or more data structures; and for at least one data structure in the first set of one or more data structures: extracting one or more values ​​from a corresponding data structure of the first data object by the first application; selecting a specific data structure in the second set of one or more data structures of the second data object by the first application based on metadata associated with the corresponding data structure of the first data object; and inserting the one or more values ​​from the corresponding data structure of the first data object into the specific data structure of the second data object by the first application.

[0142] Clause 2. The method according to Clause 1 further includes: outputting a graphical user interface for display by the first application, the graphical user interface including at least a portion of the specific data structure of the second data object.

[0143] Clause 3. The method of any one of Clauses 1-2, wherein creating the second data object comprises: creating a plurality of data objects by the first application, each data object including at least a portion of a corresponding data structure of the first data object, wherein the plurality of data objects includes the second data object, and wherein the corresponding format of each of the plurality of data objects is specific to the first application and different from the format of the first data object.

[0144] Clause 4. The method of any one of Clauses 1-3, wherein the first application includes a slide presentation application, wherein the second data object includes one or more graphic slides for the slide presentation application, wherein the second application includes a note-taking application, and wherein the first data object includes one or more note documents for the note-taking application.

[0145] Clause 5. The method of any one of Clauses 1-4, wherein extracting the content of the corresponding data structure of the first data object comprises: retrieving the values ​​of XML tags of the first data object by the first application using Extensible Markup Language (XML), and wherein inserting the one or more values ​​from the corresponding data structure of the first data object into the specific data structure of the second data object comprises: setting the XML tags of the second data object to be equal to the values ​​of the XML tags of the first data object by the first application.

[0146] Clause 6. The method according to Clause 5, wherein the one or more data structures of the first data object include a header data structure and a body data structure, wherein creating the second data object includes: for the header data structure of the first data object: the first application selects the header data structure of the second data object; the first application inserts one or more values ​​from the header data structure of the first data object into the header data structure of the second data object; for the body data structure of the first data object: the first application selects the body data structure of the second data object; the first application inserts one or more values ​​from the body data structure of the first data object into the body data structure of the second data object.

[0147] Clause 7. The method of any one of Clauses 1-6, wherein the first application includes a note-taking application, wherein the second data object includes one or more note documents of the note-taking application, wherein the second application includes a slideshow application, and wherein the first data object includes one or more note graphic slides of the slideshow application.

[0148] Article 8. The method of any one of Articles 1-7, wherein the first application comprises a spreadsheet application, and wherein the second data object comprises one or more cells of the spreadsheet application.

[0149] Article 9. The method of any one of Articles 1-8, wherein the first data object is included in a plurality of data objects, wherein each data object in the plurality of data objects includes corresponding content and corresponding metadata, and wherein the method further comprises: receiving the first data object from the plurality of data objects by the first application; retrieving each data object in the plurality of data objects having the same corresponding metadata as the first data object by the first application; and inserting by the first application the one or more values ​​of the one or more data structures of the first data object, and the one or more values ​​of the corresponding one or more data structures of each data object in the plurality of data objects having the same corresponding metadata as the first data object, into the second data object.

[0150] Article 10. The method of any one of Articles 1-9, wherein the one or more values ​​of the at least one data structure of the first data object include at least one of the following: a title sequence of text, a main sequence of text, one or more images, one or more videos, mathematical equations, and formatted lists.

[0151] Article 11. The method of any one of Articles 1-10, further comprising: before receiving the first data object: receiving by the second application information associated with a content page of the first application executed on the computing device, wherein at least a first portion of the content page of the first application is included in a graphical user interface of the computing device; determining a context of the first application by the second application based on the information associated with the content page of the second application; determining by the second application, at least partially based on the context of the first application, a corresponding group of data objects for each of one or more applications associated with the first application, wherein the corresponding group of data objects includes the first data object; generating a content page of the second application by the second application at least partially based on the corresponding group of data objects, the content page including one or more graphical indications of at least one data object from at least one group of the corresponding group of data objects, wherein the at least one data object includes the first data object; and When at least a second portion of the content page of the first application is included in the graphical user interface, the second application outputs at least a portion of the content page of the second application within the graphical user interface of the computing device for display.

[0152] Clause 12. The method according to Clause 11, wherein determining the context of the second application comprises: identifying one or more content objects in the content page of the first application by the second application; extracting at least one corresponding attribute of each of the one or more content objects by the second application; and determining the context of the first application by the second application at least in part based on one or more of the at least one corresponding attribute of each of the one or more content objects.

[0153] Article 13. The method as described in Article 12, wherein the one or more content objects include one or more of text, images, metadata, application state, application code, field identifiers, filenames, or file types.

[0154] Clause 14. The method of any one of Clauses 11-13, wherein determining the data objects of the corresponding group for each of the one or more applications associated with the first application comprises: the second application constructing a corresponding search query for each of the one or more applications associated with the first application based on the context; and the second application executing each corresponding search query from the corresponding search query to retrieve the data objects of the corresponding group from the data of the corresponding application in the one or more applications associated with the first application.

[0155] Clause 15. The method of any one of Clauses 11-14, wherein the one or more applications associated with the first application include at least two applications associated with the first application, and wherein generating the content page for the first application includes generating the content page of the second application by the second application based at least in part on data objects of corresponding groups of the at least two applications associated with the first application, the content page including at least one graphical indication of at least one data object from at least one group of data objects of each of the at least two applications associated with the first application.

[0156] Article 16. The method of any one of Articles 11-15 further comprises: The second application receives an instruction from user input, which selects a first graphic instruction of a first data object from one or more graphic instructions of at least one data object included in the content page of the second application; and the second application sends the first data object to the first application.

[0157] Clause 17. The method of any one of Clauses 11-16, further comprising: receiving by the second application an instruction for selecting one or more graphical indications representing one or more content objects included in a content page of the first application; determining by the second application an update context of the first application based on the one or more content objects; determining by the second application, at least in part, data objects of a corresponding update group for each of the one or more applications associated with the first application, based on the update context of the first application; generating by the second application an updated content page of the first application, at least in part, based on the data objects of the corresponding update group, the updated content page including one or more graphical indications of at least one data object from at least one group of data objects in the corresponding update group; and when at least a third portion of the content page of the first application is included in a graphical user interface, outputting a portion of the updated content page of the second application for display within a graphical user interface of a computing device.

[0158] Article 18. The method of any one of Articles 11-17, wherein outputting the portion of the content page of the second application comprises: when at least the second portion of the content page of the first application is included in the graphical user interface, the second application uses the portion of the content page of the first application to replace a portion of the graphical user interface.

[0159] Clause 19. The method of any one of Clauses 11-18, wherein the portion of the content page of the second application output for display includes one or more graphical indications of at least one data object from a first set of data objects of an application in one or more applications associated with the first application, the method further comprising: receiving an indication input by the second application, the user input indicating selection of an application different from the one or more applications associated with the first application; generating an updated content page of the first application by the second application at least in part based on the corresponding set of data objects, the updated content page including one or more graphical indications of at least one data object from a second set of data objects of a different application in one or more applications associated with the first application; and outputting a portion of the updated content page of the second application for display within the graphical user interface of a computing device when at least a second portion of the content page of the first application is included in a graphical user interface.

[0160] Article 20. The method of any one of Articles 11-19, further comprising: receiving, before determining the situation, an instruction from user input by the computing device to begin execution of the second application; and, in response to receiving the instruction from user input, determining the situation by the second application.

[0161] Article 21. The method of any one of Articles 11-20, wherein the second application includes an adjunct application.

[0162] Article 22. The method of any one of Articles 1-21, wherein the first data object comprises a list of values, and wherein the second data object comprises one of an image of the list of values ​​and a graph depicting the list of values.

[0163] Article 23. The method of any one of Articles 1-22, wherein the first data object comprises an image, and wherein the method further comprises: conditionally formatting the image of the first data object by the first application based on one or more requirements for an image in the first application; and inserting the conditionally formatted image into a second data object by the first application.

[0164] Item 24. A computing device comprising: at least one processor; and a memory storing instructions associated with a first application, the instructions, when executed, causing the at least one processor to: receive a first data object from a second application executing on the computing device, wherein the format of the first data object is specific to the second application and wherein the first data object includes a first set of one or more data structures; create a second data object, the format of the second data object being specific to the first application and different from the format of the first data object, the second data object including a second set of one or more data structures; and for at least one data structure in the first set of one or more data structures: extract one or more values ​​from a corresponding data structure of the first data object; select a specific data structure in the second set of one or more data structures of the second object based on metadata associated with the corresponding data structure of the first data object; and insert one or more values ​​from the corresponding data structure of the first data object into the specific data structure of the second data object.

[0165] Clause 25. The computing device according to Clause 24, wherein the memory further stores instructions associated with the first application, which, when executed, cause the at least one processor to output a graphical user interface containing at least a portion of the particular data structure of the second data object for display.

[0166] Clause 26. The computing device of any one of Clauses 24-25, wherein the instructions causing the at least one processor to create the second data object include instructions that, when executed, cause the at least one processor to perform the following operations: create a plurality of data objects, each of the plurality of data objects including at least a portion of a corresponding data structure of the first data object, wherein the plurality of data objects includes the second data object, and wherein the corresponding format of each of the plurality of data objects is specific to the first application and is different from the format of the first data object.

[0167] Article 27. The computing device of any one of Articles 24-26, wherein the instructions for causing the at least one processor to extract the contents of a corresponding data structure of the first data object include, when executed, instructions for causing the at least one processor to use Extensible Markup Language (XML) to retrieve the values ​​of XML tags of the first data object, and wherein the instructions for causing the at least one processor to insert the one or more values ​​from the corresponding data structure of the first data object into the particular data include, when executed, instructions for causing the at least one processor to set the XML tags of the second data object to be equal to the values ​​of the XML tags of the first data object.

[0168] Article 28. A non-transitory computer-readable storage medium storing instructions associated with a first application, the instructions, when executed, causing at least one processor of a computing device to: receive a first data object from a second application executing at the computing device, wherein the format of the first data object is specific to the second application and wherein the first data object includes a first set of one or more data structures; create a second data object, the format of which is specific to the first application and differs from the format of the first data object, the second data object including a second set of one or more data structures; and for at least one data structure in the first set of one or more data structures: extract one or more values ​​from a corresponding data structure of the first data object; select a specific data structure in the second set of one or more data structures of the second object based on metadata associated with the corresponding data structure of the first data object; and insert one or more values ​​from the corresponding data structure of the first data object into the specific data structure of the second data object.

[0169] Clause 29. The non-transitory computer-readable storage medium according to Clause 28, wherein the instructions causing the at least one processor to create the second data object include instructions that, when executed, cause the at least one processor to perform the following operations: create a plurality of data objects, each of the plurality of data objects including at least a portion of a corresponding data structure of the first data object, wherein the plurality of data objects includes the second data object, and wherein the corresponding format of each of the plurality of data objects is specific to the first application and differs from the format of the first data object.

[0170] Article 30. The non-transitory computer-readable storage medium as described in any one of Articles 28-29, wherein the instructions for causing the at least one processor to extract the contents of a corresponding data structure of the first data object include, when executed, instructions for causing the at least one processor to retrieve the values ​​of XML tags of the first data object using Extensible Markup Language (XML), and wherein the instructions for causing the at least one processor to insert the one or more values ​​from the corresponding data structure of the first data object into a specific data structure of the second data object include, when executed, instructions for causing the at least one processor to set the XML tags of the second data object to be equal to the values ​​of the XML tags of the first data object.

[0171] Article 31. A computing device configured to perform any one of the methods in Articles 1-23.

[0172] Article 32. A computing device comprising means for performing any one of the methods in Articles 1-23.

[0173] Article 33. A computer-readable storage medium encoded with instructions for causing one or more programmable processors to perform any one of the methods in Articles 1-23.

[0174] In one or more examples, the described functionality may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functionality may be stored or transmitted on or as one or more instructions or code on a computer-readable medium and executed by a hardware-based processing unit. A computer-readable medium may include a computer-readable storage medium corresponding to a tangible medium such as a data storage medium, or a communication medium that includes any medium that facilitates, for example, the transfer of a computer program from one place to another according to a communication protocol. In this way, a computer-readable medium may generally correspond to (1) a non-transitory tangible computer-readable storage medium or (2) a communication medium such as a signal or carrier wave. A data storage medium may be any available medium accessible by one or more computers or one or more processors to retrieve instructions, code, and / or data structures to implement the techniques described in this disclosure. Computer program products may include computer-readable media.

[0175] By way of example and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disc storage, disk storage, or other magnetic storage devices, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. Furthermore, any connection is appropriately referred to as a computer-readable medium. For example, coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium if instructions are transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave. However, it should be understood that computer-readable storage media and data storage media do not include connections, carrier waves, signals, or other transient media, but rather refer to non-transient, tangible storage media. As used herein, disks and discs include optical discs (CDs), laser discs, optical discs, digital universal discs (DVDs), floppy disks, and Blu-ray discs, where disks typically magnetically reproduce data, while optical discs optically reproduce data using lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0176] Instructions can be executed by one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable arrays (FPGAs), or other equivalent integrated or discrete logic circuits. Therefore, the term "processor" as used herein can refer to any of the foregoing structures or any other structure suitable for implementing the techniques described herein. Furthermore, in some aspects, the functionality described herein may be provided within dedicated hardware and / or software modules. Moreover, these techniques can be fully implemented in one or more circuit or logic elements.

[0177] The techniques disclosed herein can be implemented in various devices or apparatuses, including wireless mobile phones, integrated circuits (ICs), or a set of ICs (e.g., chipsets). Various components, modules, or units are described in this disclosure to emphasize functional aspects of a device configured to perform the disclosed techniques, but they do not necessarily need to be implemented through different hardware units. Rather, as described above, various units can be combined in a hardware unit or provided by a combination of interoperable hardware units including one or more processors as described above, combined with appropriate software and / or firmware.

[0178] Various examples of this disclosure have been described. Any combination of the systems, operations, or functions described is contemplated. These and other examples are within the scope of the appended claims.

Claims

1. A method for automatically analyzing and modifying data objects passed between applications, characterized in that, include: An email data object is received by a first application, which is executed on a computing device and configured to render calendar entries, from a second application, which is executed on the computing device and configured to render emails. The format of the email data object is specific to the second application, and the email data object includes a first set of one or more data structures. Without receiving user input instructing modification of the email data object's format, the first application automatically creates a calendar data object, wherein the calendar data object's format is application-specific and differs from the email data object's format, and wherein the calendar data object includes a second set of one or more data structures; and For at least one data structure in the first group of one or more data structures: The first application extracts one or more values ​​from the corresponding data structure of the email data object; The first application selects a specific data structure from one or more data structures in the second group of the calendar data object based on metadata representing the corresponding data structure of the email data object; and The first application inserts one or more values ​​from the corresponding data structure of the email data object into the specific data structure of the calendar data object, such that the title of the email data object identified by the metadata is inserted as the title data structure of the second set of one or more data structures of the calendar data object, and the body text associated with the title of the email data object identified by the metadata is inserted as a different body data structure of the second set of one or more data structures of the calendar data object.

2. The method according to claim 1, characterized in that, Also includes: Before receiving the email data object: The second application receives information associated with a content page of the first application executed on the computing device, wherein the graphical user interface of the computing device includes at least a first portion of the content page of the first application; The second application determines the context of the first application based on the information associated with the content page of the first application; The second application, at least in part based on the context of the first application, determines a corresponding set of data objects for each of one or more applications related to the first application, the corresponding set of data objects including the email data objects; The second application generates its content page based at least in part on the corresponding set of data objects, the content page including one or more graphical representations of at least one data object from at least one set of data objects in the corresponding set of data objects, wherein the at least one data object includes the email data object; and While at least a second portion of the content page of the first application is included in the graphical user interface, the second application outputs at least a portion of the content page of the second application to the graphical user interface of the computing device for display.

3. The method according to claim 2, characterized in that, The metadata that marks the corresponding data structure of the email data object includes one or more of Extensible Markup Language (XML) tags or inferred data about the structure, location, or characteristics of the corresponding data structure; The context of the first application includes one or more of the following: the current state of the first application, the current task being executed within the first application, and the future task to be executed within the first application.

4. The method according to claim 1, characterized in that, Creating the calendar data object includes: The first application creates multiple data objects, each data object including at least a portion of the corresponding data structure of the email data object, wherein the multiple data objects include the calendar data object, and wherein the corresponding format of each of the multiple data objects is specific to the first application and different from the format of the email data object.

5. The method according to claim 1, characterized in that, Extracting the one or more values ​​from the corresponding data structure of the email data object includes: the first application using Extensible Markup Language (XML) to retrieve the values ​​of the XML tags of the email data object; Inserting one or more values ​​from the corresponding data structure of the email data object into the specific data structure of the calendar data object includes: the first application setting the XML tag of the calendar data object to be equal to the value of the XML tag of the email data object.

6. The method according to claim 1, characterized in that, The one or more data structures of the email data object include a header data structure and a body data structure, wherein creating the calendar data object includes: For the header data structure of the email data object: The first application selects the title data structure of the calendar data object; and The first application inserts one or more values ​​from the header data structure of the email data object into the header data structure of the calendar data object; and For the main data structure of the email data object: The first application selects one or more main data structures of the calendar data object; and The first application inserts one or more values ​​from the body data structure of the email data object into the body data structure of the calendar data object.

7. The method according to claim 1, characterized in that, The first application includes an email application; The calendar data object includes one or more emails from the email application. The second application includes a calendar application; The email data object includes one or more calendar entries of the calendar application.

8. The method according to claim 1, characterized in that, The email data object is included in a plurality of email data objects, wherein each of the plurality of email data objects includes corresponding content and corresponding metadata, and wherein the method further includes: The first application receives the email data object from the plurality of email data objects; The first application retrieves each of the plurality of email data objects that has the same corresponding metadata as the email data object; and The first application inserts the one or more values ​​of the one or more data structures of the email data object, and the corresponding one or more values ​​of the corresponding one or more data structures of each of the plurality of data objects having the same corresponding metadata as the email data object, into the calendar data object.

9. The method according to claim 1, characterized in that, The one or more values ​​of the at least one data structure of the email data object include at least one of the following: The title sequence of the text, The main sequence of the text, One or more images, One or more videos, Mathematical equations, and A formatted list.

10. The method according to claim 1, characterized in that, The email data object includes a list of values, and the calendar data object includes either an image of the list of values ​​or a chart depicting the list of values.

11. The method according to claim 1, characterized in that, The email data object includes an image, and the method further includes: The image of the email data object is conditionally formatted by the first application based on one or more requirements for the image in the first application; and The first application inserts a conditionally formatted image into the calendar data object.

12. A computing device for automatically analyzing and modifying data objects transferred between applications, characterized in that, include: At least one processor; and A memory storing instructions associated with a first application and configured to present calendar entries, which, when executed, cause the at least one processor to: Receive an email data object from a second application that is executed on the computing device and configured to render emails, the email data object being formatted specifically for the second application, the email data object including a first set of one or more data structures; Without receiving user input instructing modification of the email data object's format, a calendar data object is automatically created, wherein the format of the calendar data object is specific to the first application and differs from the format of the email data object, wherein the calendar data object includes a second set of one or more data structures; and For at least one data structure in the first group of one or more data structures: Extract one or more values ​​from the corresponding data structure of the email data object; Based on the metadata of the corresponding data structure that marks the email data object, a specific data structure is selected from one or more data structures in the second group of the calendar data object; and The values ​​of one or more from the corresponding data structure of the email data object are inserted into the specific data structure of the calendar data object, such that the title of the email data object identified by the metadata is inserted as the title data structure of the second set of one or more data structures of the calendar data object, and the body text associated with the title of the email data object identified by the metadata is inserted as a different body data structure of the second set of one or more data structures of the calendar data object.

13. The computing device according to claim 12, characterized in that, The memory also stores instructions associated with the second application, which, when executed, cause the at least one processor to: Before receiving the email data object: Receive information associated with a content page of the first application executed on the computing device, wherein the graphical user interface of the computing device includes at least a first portion of the content page of the first application; Based on the information associated with the content page of the first application, the context of the first application is determined; Based at least in part on the context of the first application, determine a corresponding set of data objects for each of one or more applications associated with the first application, the corresponding set of data objects including the email data objects; The content page of the second application is generated at least partially based on the corresponding set of data objects, the content page including one or more graphical representations of at least one data object from at least one set of data objects in the corresponding set of data objects, wherein the at least one data object includes the email data object; and While at least a second portion of the content page of the first application is included in the graphical user interface, at least a portion of the content page of the second application is output to the graphical user interface of the computing device for display.

14. The computing device according to claim 12, characterized in that, The instructions that cause the at least one processor to create the calendar data object include, when executed, instructions that cause the at least one processor to perform the following operations: Create multiple data objects, each data object including at least a portion of the corresponding data structure of the email data object, wherein the multiple data objects include the calendar data object, and wherein the corresponding format of each of the multiple data objects is specific to the first application and different from the format of the email data object.

15. The computing device according to claim 12, characterized in that, Instructions for the at least one processor to extract the one or more values ​​from the corresponding data structure of the email data object include: instructions for the at least one processor, when executed, to use Extensible Markup Language (XML) to retrieve the values ​​of XML tags of the email data object; The instruction that causes the at least one processor to insert one or more values ​​from the corresponding data structure of the email data object into the specific data structure of the calendar data object includes, when executed, an instruction that causes the at least one processor to set the XML tag of the calendar data object to be equal to the value of the XML tag of the email data object.

16. A non-transitory computer-readable storage medium for storing instructions associated with a first application, characterized in that, The first application is configured to display calendar entries, and when the instruction is executed, it causes at least one processor of the computing device to perform the following operations: Receive an email data object from a second application that is executed on the computing device and configured to render emails, the email data object being formatted specifically for the second application, the email data object including a first set of one or more data structures; Without receiving user input instructing modification of the email data object's format, a calendar data object is automatically created, wherein the format of the calendar data object is specific to the first application and differs from the format of the email data object, wherein the calendar data object includes a second set of one or more data structures; and For at least one data structure in the first group of one or more data structures: Extract one or more values ​​from the corresponding data structure of the email data object; Based on the metadata of the corresponding data structure that marks the email data object, a specific data structure is selected from one or more data structures in the second group of the calendar data object; and The values ​​of one or more from the corresponding data structure of the email data object are inserted into the specific data structure of the calendar data object, such that the title of the email data object identified by the metadata is inserted as the title data structure of the second set of one or more data structures of the calendar data object, and the body text associated with the title of the email data object identified by the metadata is inserted as a different body data structure of the second set of one or more data structures of the calendar data object.

17. The non-transitory computer-readable storage medium according to claim 16, characterized in that, The non-transitory computer-readable storage medium also stores instructions associated with the second application, which, when executed, cause the at least one processor to: Before receiving the email data object: Receive information associated with a content page of the first application executed on the computing device, wherein the graphical user interface of the computing device includes at least a first portion of the content page of the first application; Based on the information associated with the content page of the first application, the context of the first application is determined; Based at least in part on the context of the first application, determine a corresponding set of data objects for each of one or more applications associated with the first application, the corresponding set of data objects including the email data objects; The content page of the second application is generated at least partially based on the corresponding set of data objects, the content page including one or more graphical representations of at least one data object from at least one set of data objects in the corresponding set of data objects, wherein the at least one data object includes the email data object; and While at least a second portion of the content page of the first application is included in the graphical user interface, at least a portion of the content page of the second application is output to the graphical user interface of the computing device for display.

18. The non-transitory computer-readable storage medium according to claim 16, characterized in that, Instructions for the at least one processor to extract the one or more values ​​from the corresponding data structure of the email data object include: instructions for the at least one processor, when executed, to retrieve the values ​​of XML tags of the email data object using Extensible Markup Language (XML); The instruction that causes the at least one processor to insert one or more values ​​from the corresponding data structure of the email data object into the specific data structure of the calendar data object includes, when executed, an instruction that causes the at least one processor to set the XML tag of the calendar data object to be equal to the value of the XML tag of the email data object.

Images