Generation of Snippet Packets Based on Selection of a Portion of a Web Page

Abstract

A system and method for generating snippet packets can include obtaining input data (e.g., input data describing a gesture). The input data can be processed to determine a content item selected by the input. Based on the content item, a snippet packet can be generated, which can include the content item, address data, and location data. The snippet packet can be configured to be interacted with to navigate to the source web page of the content item, including navigating to a specific portion of the web page that includes the content item.

Description

Technical Field

[0001] Related Applications This application claims the benefit and priority of U.S. Non-Provisional Application No. 18 / 081,814, filed Dec. 15, 2022, which claims the benefit and priority of U.S. Provisional Application No. 63 / 344,783, filed May 23, 2022. U.S. Non-Provisional Application No. 18 / 081,814 and U.S. Provisional Application No. 63 / 344,783 are hereby incorporated by reference in their entirety.

[0002] The present disclosure generally relates to generating interactive snippet packets in response to user input. More specifically, the present disclosure relates to obtaining user input that selects content items for saving with snippet packets that can be selected later, and providing a portion of a web page that includes the content items.

Background Art

[0003] When saving text, images, and / or audio from a web page, the user may be able to experience the text, images, and / or audio locally again without being connected to the Internet. However, the saving process may provide a limited context of the data's origin, and if the user wants to see the context of the saved data, the user has to use the data as a search query, navigate from browsing history, or even try to remember how they got to that web page in the first place. Further, when the source of the web page is found, the user may still have to review most of the web page to accurately find where the saved data was originally on the web page.

Summary of the Invention

[0004] Aspects and advantages of embodiments of the present disclosure are shown in part in the following description, or can be learned from the description, or can be learned through the practice of the embodiments.

[0005] An exemplary aspect of the present disclosure is directed to a computing system. The system can include one or more processors and one or more non-transitory computer-readable media that collectively store instructions that, when executed by the one or more processors, cause the computing system to perform operations. The operations can include providing data that describes a graphical user interface. The graphical user interface can include a graphical window for displaying a web page. In some embodiments, the web page can include a plurality of content items. The operations can include obtaining input data. The input data can include a request to save one or more of the plurality of content items. The operations can include generating a snippet packet. In some embodiments, the snippet packet can include one or more content items. The snippet packet can include address data. The address data can describe the web address of the web page. The snippet packet can include location data. The location data can describe the location of one or more content items within the web page. The operations can include storing the snippet packet. The snippet packet can be associated with a particular user.

[0006] In some embodiments, the operation can include receiving a snippet request to provide a snippet interface. The snippet interface can include interactive elements associated with a snippet packet. The operation can include providing the snippet interface for display, receiving an interface selection that selects an interactive element, and providing a portion of a web page for display. The portion of the web page can include the location of one or more content items within the web page. In some embodiments, the operation can include receiving an insertion input. The insertion input can include user input that requests insertion of one or more content items into a different interface. The operation can include providing the snippet packet to a third-party server computing system.

[0007] In some embodiments, generating the snippet packet can include obtaining one or more content items and generating a graphic card. The graphic card can describe one or more content items. Generating the snippet packet can include storing the graphic card as a graphic representation of the snippet packet. In some embodiments, the location data can include at least one of a scroll position, a start node, or an end node. The scroll position can describe the location of one or more content items relative to other portions of the web page. The start node can describe the start position of one or more content items. The end node can describe the end position of one or more content items.

[0008] In some embodiments, one or more content items can include at least one of an image, a video, a graphic depiction of a product, or audio. The operation can include processing one or more content items to determine an entity associated with the one or more content items and generating an entity tag based on the entity. The snippet packet can include the entity tag. In some embodiments, storing the snippet packet can include locally storing the snippet packet on a mobile computing device.

[0009] Other exemplary aspects of the present disclosure are directed to a computer-implemented method. The method can include obtaining input data by a computing system including one or more processors. The input data can describe a selection of content items associated with a snippet packet. The method can include obtaining address data and location data associated with the snippet packet by the computing system. The address data can be associated with a web page. The content item can be associated with the web page. In some embodiments, the location data can describe the location of the content item within the web page. The method can include obtaining web page data by the computing system. The web page data can be obtained based at least in part on the address data. The method can include determining, by the computing system, the location within the web page associated with the content item and providing, by the computing system, a portion of the web page. The portion of the web page can include the location of the content item.

[0010] In some embodiments, the address data can include a Uniform Resource Locator. The location data can include a text fragment. Determining, by a computing system, a location within a web page associated with a content item can include, by the computing system, adding a text fragment to a Uniform Resource Locator to generate a shortcut link, and, by the computing system, entering the shortcut link into a browser.

[0011] In some embodiments, the snippet packet may be generated by a computing system by providing a graphical user interface. The graphical user interface can include a graphical window for displaying a web page. The web page can include a plurality of content items. The snippet packet may be generated by a computing system by obtaining selection data, by the computing system generating the snippet packet, and by the computing system storing the snippet packet in a user database. The selection data can include a request to save a content item out of a plurality of content items. In some embodiments, providing, by a computing system, a portion of a web page can include providing one or more indicators to the portion of the web page. The one or more indicators can indicate a content item associated with the snippet packet. The one or more indicators can include highlighting text associated with the content item. In some embodiments, the snippet packet can be associated with a user account of a particular user. The user account can be associated with one or more platforms.

[0012] Other exemplary aspects of the disclosure are directed to one or more non-transitory computer-readable media that collectively store instructions that, when executed by one or more computing devices, cause the one or more computing devices to perform operations. The operations can include providing a graphical user interface. The graphical user interface can include a graphical window for displaying a web page. In some embodiments, the web page can include a plurality of content items. The operations can include receiving gesture data. In some embodiments, the gesture data can describe a gesture associated with a portion of the web page. The operations can include processing the gesture data to determine a selected content item. The selected content item can be associated with a portion of the web page. The operations can include generating a snippet packet based on the gesture data. In some embodiments, the snippet packet can include the selected content item.

[0013] In some embodiments, a snippet packet can include address data and location data. The address data can be associated with a web page. The location data can describe the location of a selected content item within the web page. A gesture can include a circular gesture that surrounds a portion of the web page. In some embodiments, processing gesture data to determine a selected content item can include determining a portion of the web page surrounded by the gesture, determining a focus of the portion, and determining that the selected content item is associated with the focus of the portion. In some embodiments, processing gesture data to determine a selected content item can include processing the gestured data with a machine-learned model to determine the selected content item. Gesture data can describe a touch input to a touch screen display of a mobile computing device.

[0014] Other aspects of the disclosure are directed to various systems, devices, non-transitory computer-readable media, user interfaces, and electronic devices.

[0015] These and other features, aspects, and advantages of the various embodiments of the present disclosure will be better understood with reference to the following description and the appended claims. The accompanying drawings, which are incorporated herein and constitute a part of this specification, illustrate exemplary embodiments of the disclosure and, together with the description, serve to explain the relevant principles for carrying out the invention.

[0016] A detailed description of embodiments directed to those of ordinary skill in the art is set forth in this specification with reference to the accompanying drawings.

Brief Description of the Drawings

[0017]

Figure 1A

Figure 1B

Figure 1C

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10A

Figure 10B

Figure 10C

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

DETAILED DESCRIPTION

[0018] Reference numerals repeated across multiple drawings are intended to identify the same features in various embodiments.

[0019] Overview The present disclosure generally relates to generating interactive snippet packets in response to user input. More specifically, the present disclosure relates to obtaining user input that selects a content item for saving with a snippet packet that can be selected later, and providing a portion of a web page that includes the content item. For example, a user can select a portion of a web page and / or a data file. Next, the data that describes the selected portion can be stored along with the information associated with the web page / data file and the location of the portion with respect to the web page / data file. The stored data set can be a snippet packet that includes a graphic representation of the selected portion, which can include a graphic card with text and / or images from the selected portion. The snippet packet can be stored for later reference and / or shared with other users. The snippet packet enables the user to navigate to a specific location of the selected portion of the original web page / data file at the time of selection after viewing the selected portion. The system and method can include providing a graphical user interface. The graphical user interface can include a graphical window for displaying a web page. In some embodiments, the web page can include a plurality of content items. The system and method can include obtaining input data. The input data can include a request to save one or more of the plurality of content items. A snippet packet can be generated. The snippet packet can include one or more content items, address data, and location data. In some embodiments, the address data can describe the web address of the web page. The location data can describe the location of one or more content items within the web page. The snippet packet can be stored in a user database.

[0020] For example, the systems and methods disclosed herein can include providing a graphical user interface. The graphical user interface can include a graphical window for displaying a web page. In some embodiments, the web page can include a plurality of content items. The displayed web page can include text, one or more images, one or more interactive user interface elements, one or more videos, and / or one or more audio clips. The graphical user interface can be part of a browser application.

[0021] The systems and methods can obtain input data. The input data can include a request to save one or more of the plurality of content items. In some embodiments, the one or more content items can include at least one of an image, a video, a graphic depiction of a product, or audio. The graphical user interface may be updated to include one or more user interface elements for interacting with the one or more content items, and can include saving the one or more content items, and may include different options for the format in which to save. Alternatively and / or additionally, the overlay interface may provide a pop-up interface in response to the request. The input data may describe a gesture associated with one or more of the content items (e.g., a circle around one or more of the content items).

[0022] A snippet packet can be generated. The snippet packet can be generated based on input data. The snippet packet can include one or more content items, address data, and location data. Generating the snippet packet can include processing one or more content items with one or more machine-learned models to generate a semantic understanding output that can be used for summarization, annotation, and / or classification.

[0023] One or more content items can include text data (e.g., text data describing words, sentences, quotes, and / or paragraphs), image data (e.g., an image of an object (e.g., a product)), video data (e.g., a video and / or one or more video frames), audio data (e.g., waveform data), and / or latent encoded data. One or more content items can include multimodal data. The address data can include a resource locator and / or a file address associated with a web page (e.g., a web address). The location data can describe where one or more content items are located within a web page and / or a file. For example, the location data can indicate the start and end of one or more content items within a web page and / or a file.

[0024] In some embodiments, generating a snippet packet can include obtaining one or more content items and generating a graphic card. The graphic card can describe one or more content items. The graphic card can include text data overlaid on a color and / or an image. The color can be determined based on the primary color of the web page. In some embodiments, the color can be predefined and / or determined based on surrounding content items. The image can be an image determined based on the determined topic of the content item. Alternatively and / or additionally, the image can be an image close to the content item. In some embodiments, the graphic card can include a font determined based on the font used in the web page. The graphic card can include a background, text describing the content item (e.g., the content item and / or a summary of the content item), and / or text and / or a logo describing the source and / or the determined entity. In some embodiments, the text size of the text in the graphic card can be based on the amount of text in the content item.

[0025] The address data can describe the web address of the web page. The address data can include a uniform resource identifier and / or a uniform resource locator. In some embodiments, the address data can include data describing the source of the content item.

[0026] Location data can describe the location of one or more content items within a web page. In some embodiments, the location data can include at least one of a scroll position, a start node, or an end node. The scroll position can describe the location of one or more content items relative to other parts of the web page. In some embodiments, the start node can describe the starting position of one or more content items. The end node can describe the ending position of one or more content items. The location data can include a text fragment (Tomayac et al., “Scroll to Text Fragment,” GITHUB, (May 20, 2022, 9:40PM) https: / / github.com / WICG / scroll-to-text-fragment) that can be used to indicate the location of the content item. In some embodiments, the text fragment can include one or more text directives associated with the location. The text directive can include a string of start data (e.g., the first text associated with the content item and / or the first pixel associated with the content item) and / or end data (e.g., the last text associated with the content item and / or the last pixel associated with the content item) of the code. The text directive can be used to search the web page for a set of data that matches the start and / or end of the content item.

[0027] In some embodiments, the system and method can include processing one or more content items to determine an entity associated with the one or more content items. An entity tag can be generated based on the entity. A snippet packet can include the entity tag.

[0028] Snippet packets can be stored in a user database. In some embodiments, storing the snippet packets in a user database can include storing the snippet packets locally on a mobile computing device. Additionally and / or alternatively, a graphics card can store the snippet packets as a graphic display. The graphics card can be automatically generated and may be customizable by a user. The graphics card can start with a template that can be customized based on other content of the web page, based on user input, and / or based on other contexts. In some embodiments, the graphics card can include multimodal data. Alternatively and / or additionally, the snippet packets can be stored in a server computing system. In some embodiments, when a content item references other content items, the system and method can obtain the additional content items and save the additional content items in the snippet packets.

[0029] In some embodiments, the system and method can include receiving a snippet request to provide a snippet interface. The snippet interface can include interactive elements associated with the snippet packets. The snippet interface can be provided for display. An interface selection can be received, and the interface selection can describe a selection to select an interactive element. Next, a portion of the web page can be provided for display. In some embodiments, the portion of the web page can include the location of one or more content items within the web page.

[0030] In some embodiments, the system and method can include receiving an insertion input. The insertion input can include user input that requests insertion of a content item into a different interface. A snippet packet can be provided to a third-party server computing system.

[0031] Additionally and / or alternatively, the system and method can include adding a snippet packet to a collection. The snippet packet can be added to the collection based on received user input. Additionally and / or alternatively, the snippet packet can be automatically added to the collection based on a determined entity associated with the content item. In some embodiments, the snippet packet can be added to the collection based on the source of the content item (e.g., based on the type of web page, type of media provider, and / or type of content item).

[0032] In some embodiments, the snippet packet can be generated based on content items obtained from sources other than web pages (e.g., mobile applications, large data files (e.g., downloaded videos or books), and / or data from other sources).

[0033] The system and method can include providing a particular portion of a web page for display in response to an interaction with a snippet packet. For example, the system and method can include obtaining input data. The input data can describe a selection of a content item associated with the snippet packet. Address data and location data associated with the snippet packet can be obtained. The address data can be associated with a web page. The content item can be associated with a web page. Additionally and / or alternatively, the location data can describe the location of the content item within the web page. The system and method can obtain web page data. The web page data can be obtained at least partially based on the address data. The location of the content item within the web page associated with the content item can be determined. Next, a portion of the web page can be provided. The portion of the web page can include the location of the content item.

[0034] The system and method can obtain input data. In some embodiments, the input data can describe a selection of content items associated with a snippet packet. The snippet packet can be associated with a user account of a particular user. Additionally and / or alternatively, the user account can be associated with one or more platforms. The snippet packet can include content items and deep links. Additionally and / or alternatively, the snippet packet can include snippets (e.g., content items and / or media data generated based on content items (e.g., graphic cards and / or summaries of content items)), address data (e.g., uniform resource locators and / or uniform resource identifiers), and / or metadata. The snippet packet can include location data (e.g., metadata indicating the location of a content item within a web page, text fragments for identifying start data and end data, one or more pointers, and / or one or more scroll position data).

[0035] In some embodiments, the snippet packet may be generated by providing a graphical user interface for display. The graphical user interface can include a graphical window for displaying a web page. In some embodiments, the web page can include a plurality of content items. Snippet packet generation can include obtaining selection data. The selection data can include a request to save a content item among the plurality of content items. In some embodiments, snippet packet generation can include generating a snippet packet and storing the snippet packet in a user database.

[0036] The system and method can obtain address data and location data associated with a snippet packet based on input data. The address data can be associated with a web page. A content item can be associated with a web page. Additionally and / or alternatively, the location data can describe the location of a content item within a web page.

[0037] Next, web page data can be obtained. The web page data can be obtained at least in part based on the address data (e.g., by navigating to the web page using a uniform resource locator). Additionally and / or alternatively, a file may be obtained based on the address data (e.g., the address data can describe a file location and may be used to obtain that particular file).

[0038] The location within the web page (and / or file) associated with the content item can be determined based on the obtained location data. The location may be determined based on text fragments, one or more pointers, and / or via web page processing.

[0039] In some embodiments, the address data can include a uniform resource locator. Additionally and / or alternatively, the location data can include one or more text fragments. Determining the location within a web page associated with a content item can then include adding a text fragment to the uniform resource locator to generate a shortcut link and entering that shortcut link into a browser.

[0040] Next, it can be provided to display a portion of a web page. The portion of the web page can include the positions of content items. In some embodiments, providing a portion of the web page can include providing one or more indicators on the portion of the web page. The one or more indicators can indicate content items associated with the snippet packet. In some embodiments, the one or more indicators can include highlighting text associated with the content item.

[0041] The system and method can include processing gesture data. For example, the system and method can include providing a graphical user interface. The graphical user interface can include a graphical window for displaying a web page. In some embodiments, the web page can include a plurality of content items. The system and method can receive gesture data. The gesture data can describe a gesture associated with a portion of the web page. The gesture data can be processed to determine a selected content item. The selected content item can be associated with the portion of the web page. The snippet packet can be generated based on the gesture data. In some embodiments, the snippet packet can include the selected content item.

[0042] In particular, a graphical user interface can be provided for display. The graphical user interface can include a graphical window for displaying a web page. In some embodiments, the web page can include a plurality of content items. The web page can be associated with a uniform resource locator and / or source code. The plurality of content items can include structured text data (e.g., text paragraphs and / or one or more titles), whitespace, one or more images, and / or audio content items.

[0043] Next, gesture data can be received. The gesture data can describe a gesture associated with a portion of the web page. In some embodiments, the gesture can include a circular gesture surrounding a portion of the web page. The gesture data can describe a touch input to a touch screen display of a mobile computing device.

[0044] The gesture data can be processed to determine a selected content item. The selected content item can be associated with a portion of the web page. For example, the gesture can surround one or more lines of text and an image in a web page that includes a plurality of lines and / or a plurality of images.

[0045] In some embodiments, processing the gesture data to determine the selected content item can include determining a portion of the web page surrounded by the gesture, determining a focus of the portion, and determining that the selected content item is associated with the focus of the portion.

[0046] Alternatively and / or additionally, processing gesture data to determine a selected content item can include processing the gestured data with a machine-learned model to determine the selected content item. The machine-learned model can be trained to determine the start and end of the selected content item based on proximity to gesture boundaries, syntax, structural data, whitespace, and / or semantic cohesion.

[0047] In some embodiments, the selected content item can be determined based on a determined match area associated with a rectangle determined based on a circle. The rectangle can be a rectangle of data based on the syntactic composition of the web page. In some embodiments, the selected content item can be determined based on determined word boundaries and / or based on determined media content boundaries. The determination can be based on the code boundaries of a hypertext markup language. In some embodiments, the source code of the web page can be parsed and the parsed data can be processed.

[0048] Alternatively and / or additionally, the determination can include calculating the area of a gesture rectangle associated with the gesture. The area of one or more content item elements can be determined. The intersecting area between the area of the gesture rectangle and the area of each different content item element can be determined. The element with the highest degree of intersection may have the highest probability of being selected and can thus be determined as the selected content item.

[0049] Next, a snippet packet can be generated based on the gesture data. The snippet packet can include the selected content item. In some embodiments, the snippet packet can include address data and location data. The address data can be associated with the web page and the location data can describe the location of the content item within the web page.

[0050] Next, it can be provided to display a save interface. The save interface can provide an interactive interface element that can be selected to add a snippet packet to a collection. Next, a collection interface can be provided. Next, a drag input can be received that drags the graphic display of the snippet packet to a graphic tile that describes a particular collection. Next, the snippet packet can be stored in a collection (e.g., the snippet packet can be stored along with a relationship tag that links the snippet packet to a particular collection). In some embodiments, the graphic display of the snippet packet can change in size and / or ratio when the graphic display is dragged to a particular collection. The change in size and ratio can provide an intuitive indication of collection addition while providing a visually appealing display.

[0051] In some embodiments, a content item can be processed to determine an entity associated with the content item. The entity can be determined by processing the content item (e.g., text data, image data, audio data, video data, latent encoded data, and / or link data) with a machine-learned model (e.g., an image classification model, an object classification model, a text classification model (e.g., a natural language processing model), a segmentation model, a semantic model, and / or a detection model) to generate entity data (e.g., classification). Next, relationship data can be generated and added to the snippet packet based on the entity data. The relationship data can include one or more entity tags and / or references to other related snippet packets and / or related web pages or content items.

[0052] Snippet packets can be searchable within one or more applications and / or databases. Additionally and / or alternatively, snippet packets may be shareable via a messaging application, a social media application, and / or via other applications.

[0053] In some embodiments, content items can be processed with one or more machine-learned models to generate tags that can be stored in a snippet packet. The tags can then be utilized as searchable tags to surface the snippet packet in response to a search query. The tags can be determined based on the content of the content item (e.g., recognized words, recognized objects within an image, characteristics of a video frame or an audio stream, etc.).

[0054] In some embodiments, snippet packet generation occurs after selection of a snippet packet generation interface element that can open a snippet packet generation interface. Alternatively and / or additionally, a user can select a content item and provide various options for interacting with the content item in one or more (e.g., two) pop-ups, one of the options can include snippet packet generation. Alternatively and / or additionally, search results associated with the content item may be provided.

[0055] In some embodiments, a screenshot request can be received. A screenshot can be generated, uploaded to a new client, and a token can be generated. The token can be utilized to receive data associated with the screenshot. The token, the screenshot, and details of the screenshot can be utilized to generate a snippet packet.

[0056] Systems and methods can be implemented to generate snippet packets based not only on web pages but also on other data sources. For example, the systems and methods can be utilized to generate snippet packets based on content items of data files stored locally and / or on a server computing system. The generated snippet packets can include content items (and / or graphic cards), address data, and location data. The address data can describe where the data file is stored (e.g., the name of the drive and the name of any folder (e.g., G:\ResearchPapers\Quantum\Spin)). The location data can describe where in the data file the content item is located. The location data can include start data and end data that can be utilized to find matching data within the data file and then navigate to and highlight the data file. Alternatively and / or additionally, the location data can include one or more pointers.

[0057] The systems and methods can enable user selection of subsets, excerpts, and / or portions of objects (e.g., text, images, and / or video that can be part of a larger web page). In some embodiments, the systems and methods can segment a portion of text from a larger body of text, segment a portion of an image, separate frames within a video, and / or segment a portion of an audio file.

[0058] In some embodiments, the system and method can be utilized as an extension of a browser application and / or as a feature positioned on top of other applications, and snippet packet generation can be utilized for content displayed in various different application types. For example, the system and method can be incorporated into the operating system of a computing device to enable snippet packet generation for selections made in various different applications (e.g., map applications, browser applications, social media applications, etc.).

[0059] Additionally and / or alternatively, the system and method can be utilized by a plurality of different types of computing devices. For example, the system and method can be utilized by mobile computing devices, desktop computing devices, smart wearables (e.g., smart glasses), and / or other computing devices. The system and method may be utilized in virtual reality interfaces and augmented reality interfaces.

[0060] In some embodiments, the snippet packet can include user context data that describes the user's context when the snippet packet was generated. For example, the computing device can include a plurality of sensors that can collect data regarding the user's context. In some embodiments, physical location data of the user computing device can be obtained and stored in the snippet packet to provide additional context to the snippet. The physical location data can be provided within the graphics card and / or as an optional dataset that can be viewed during snippet packet interaction.

[0061] The system and method can locally store the snippet packets on the user's device and / or store the snippet packets in a server computing system. The local storage of the snippet packets can be utilized to ensure that the snippet packets remain private to the user and can provide offline access. Further, the metadata related to the collection and generation of the snippet packets can be kept private and secure.

[0062] The system and method can be provided via a browser extension, via an overlay application, and / or via the functionality of a built-in application. The system and method can be utilized on mobile devices, desktop devices, smart wearables, and / or other computing devices.

[0063] The snippet packets may include a selected portion, a web page, and / or other metadata associated with one or more contexts of the user (e.g., the application in use, the time, the user's geographical location, and / or user profile data).

[0064] The system and method may be executed in a server computing system. Alternatively and / or additionally, the system and method can be executed locally on a user computing device. In some embodiments, the user computing device can be communicatively connected via a network and can transmit data to perform cloud-based computing. The snippet packets may be stored locally and / or stored in a server.

[0065] The systems and methods of the present disclosure provide several technical effects and advantages. As an example, the systems and methods can generate and store snippet packets. In particular, the systems and methods disclosed herein can obtain input data, determine content items (e.g., text, images, videos, and / or audio) associated with the input data, generate a snippet packet, and store the snippet packet. The snippet packet can include a graphic representation of the content item, and this graphic representation of the content item can, when selected, lead the user to a portion of the web page from which the content item originated. The generation and storage of the snippet packet can enable easy access to the stored content while maintaining more links to the context regarding the content item.

[0066] Other technical advantages of the systems and methods of the present disclosure are that snippet packets can be utilized to share hierarchical level information at relatively low transmission costs. For example, the systems and methods can generate snippet packets. The snippet packets can be shared with a second user who can first view the content item. Next, the second user can select the content item, navigate to a web page, and be routed to a specific portion of the web page from which the content item originated, enabling the second user to obtain more context regarding the content item. Since the content item, web address, and text fragment can be transmitted, the provision of hierarchical information can be accomplished at relatively low transmission costs. The second user can interact with the snippet packet, view the content item separately, and then select the snippet packet and use the web address in combination with the text fragment to navigate to a portion of the web page where the content item is highlighted or otherwise indicated. Transmitting the entire web page file highlighted may involve more uploads and downloads during transmission.

[0067] Other exemplary technical effects and advantages relate to improved computational efficiency and improvements in the functionality of computing systems. For example, the systems and methods disclosed herein can utilize snippet buckets to reduce the amount of data stored to save content items and related web page context. In particular, the snippet packet may include a compressed version of the content item, web address, and text fragment instead of saving a compressed version of the entire web page that may include a large number of content items and embedded data. Further, searching a collection of snippet packets can require less computation than searching multiple compressed web pages.

[0068] Reference will now be made to the drawings, in which exemplary embodiments of the present disclosure will be described in more detail.

[0069] Exemplary Devices and Systems FIG. 1A shows a block diagram of an exemplary computing system 100 that performs snippet packet generation, according to an exemplary embodiment of the present disclosure. The system 100 includes a user computing device 102, a server computing system 130, and a training computing system 150 that are communicatively connected via a network 180.

[0070] The user computing device 102 can be any type of computing device, such as, for example, a personal computing device (e.g., a laptop or desktop), a mobile computing device (e.g., a smartphone or tablet), a gaming console or controller, a wearable computing device, an embedded computing device, or any other type of computing device.

[0071] The user computing device 102 includes one or more processors 112 and a memory 114. The one or more processors 112 can be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and can be one processor or multiple processors operably connected. The memory 114 can include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, and combinations thereof. The memory 114 can store data 116 and instructions 118 that are executed by the processor 112 to cause the user computing device 102 to perform operations.

[0072] In some embodiments, the user computing device 102 can store or include one or more packet generation modules 120. For example, the packet generation module 120 can be or include various machine-learned models such as a neural network (e.g., a deep neural network) or other types of machine-learned models including non-linear models and / or linear models. The neural network can include a feed-forward neural network, a recurrent neural network (e.g., a long short-term memory recurrent neural network), a convolutional neural network, or other forms of neural networks. Exemplary packet generation model 120 is described with reference to FIGS. 2-4.

[0073] In some embodiments, one or more packet generation models 120 can be received from the server computing system 130 via the network 180, stored in the user computing device memory 114, and then used or otherwise implemented by one or more processors 112. In some embodiments, the user computing device 102 can implement multiple parallel instances of a single packet generation model 120 (e.g., to perform parallel snippet packet generation across multiple instances of snippet selection).

[0074] More specifically, the packet generation model can receive input data, determine one or more selected content items, generate a snippet packet, and / or determine one or more tags. In some embodiments, the packet generation model can process the selected content items and generate a summary to be added to the snippet packet.

[0075] Additionally and / or alternatively, one or more packet generation models 140 can be included in the server computing system 130 that communicates with the user computing device 102 according to a client-server relationship, or otherwise can be stored and implemented by the server computing system 130. For example, the packet generation module 140 can be implemented by the server computing system 140 as part of a web service (e.g., a snippet packet generation service). Accordingly, one or more models 120 can be stored and implemented on the user computing device 102, and / or one or more models 140 can be stored and implemented on the server computing system 130.

[0076] The user computing device 102 can also include one or more user input components 122 that receive user input. For example, the user input component 122 can be a touch sensor-based component (e.g., a touch sensor-based display screen or a touch pad) that is sensitive to the touch of a user input object (e.g., a finger or a stylus). The touch sensor-based component can serve to implement a virtual keyboard. Other exemplary user input components include a microphone, a conventional keyboard, or other means by which a user can provide user input.

[0077] The server computing system 130 includes one or more processors 132 and a memory 134. The one or more processors 132 can be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.), and can be a single processor or multiple processors operably connected. The memory 134 can include one or more non-transitory computer-readable storage media such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, and combinations thereof. The memory 134 can store data 136 and instructions 138 that are executed by the processor 132 to cause the server computing system 130 to perform operations.

[0078] In some embodiments, the server computing system 130 includes or is implemented by one or more server computing devices. When the server computing system 130 includes multiple server computing devices, such server computing devices can operate according to a sequential computing architecture, a parallel computing architecture, or a combination of either.

[0079] As described above, the server computing system 130 can store or include one or more machine-learned packet generation models 140. For example, the model 140 can be or include various machine-learned models. Exemplary machine-learned models include neural networks or other multi-layer non-linear models. Exemplary neural networks include feed-forward neural networks, deep neural networks, recurrent neural networks, and convolutional neural networks. Exemplary model 140 is described with reference to FIGS. 2-4.

[0080] The user computing device 102 and / or the server computing system 130 can train the models 120 and / or 140 through interaction with a training computing system 150 communicatively connected via a network 180. The training computing system 150 can be separate from the server computing system 130 or can be a part of the server computing system 130.

[0081] The training computing system 150 includes one or more processors 152 and a memory 154. The one or more processors 152 can be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and can be one processor or multiple processors operably connected. The memory 154 can include one or more non-transitory computer-readable storage media such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, and combinations thereof. The memory 154 can store data 156 and instructions 158 to be executed by the processor 152 to cause the training computing system 150 to perform operations. In some embodiments, the training computing system 150 includes or is implemented by one or more server computing devices.

[0082] The training computing system 150 can include, for example, a model trainer 160 that trains the machine-learned models 120 and / or 140 stored in the user computing device 102 and / or the server computing system 130 using various training techniques or learning techniques such as backpropagation of error. For example, a loss function can backpropagate through the model(s) to update one or more parameters of the model(s) (e.g., based on the gradient of the loss function). Various loss functions can be used, such as mean squared error, likelihood loss, cross-entropy loss, hinge loss, and / or various other loss functions. Gradient descent techniques can be used to iteratively update the parameters over a number of training iterations.

[0083] In some embodiments, performing backpropagation of error can include performing truncated through-time backpropagation of error. The model trainer 160 can perform various generalization techniques (e.g., weight decay, dropout, etc.) to improve the generalization ability of the model being trained.

[0084] In particular, the model trainer 160 can train the packet generation models 120 and / or 140 based on a set of training data 162. The training data 162 can include, for example, training inputs (e.g., training gestures), training web pages, training text data, training image data, ground truth graphics cards, ground truth snippet packets (e.g., ground truth snippets, ground truth address data, and / or ground truth location data), and / or ground truth entity labels.

[0085] In some embodiments, if the user consents, training examples can be provided by the user computing device 102. Thus, in such embodiments, the model 120 provided to the user computing device 102 can be trained with user-specific data received from the user computing device 102 by the training computing system 150. In some cases, this process may be referred to as personalizing the model.

[0086] The model trainer 160 includes computer logic utilized to provide the desired functionality. The model trainer 160 can be implemented in hardware, firmware, and / or software that controls a general-purpose processor. For example, in some embodiments, the model trainer 160 includes program files stored on a storage device, loaded into memory, and executed by one or more processors. In other embodiments, the model trainer 160 includes one or more sets of computer-executable instructions stored on a tangible computer-readable storage medium such as a RAM hard disk or optical or magnetic media.

[0087] The network 180 can be any type of communication network, such as a local area network (e.g., intranet), wide area network (e.g., Internet), or any combination thereof, and can include any number of wired or wireless links. Generally, communication via the network 180 can be performed over any type of wired and / or wireless connection using a variety of communication protocols (e.g., TCP / IP, HTTP, SMTP, FTP), encoding or formatting (e.g., HTML, XML), and / or protection schemes (e.g., VPN, secure HTTP, SSL).

[0088] The machine-learned models described herein can be used in a variety of tasks, applications, and / or use cases.

[0089] In some embodiments, the input to the machine-learned model(s) of the present disclosure can be image data. The machine-learned model(s) can process the image data to generate an output. As an example, the machine-learned model(s) can process the image data to generate an image recognition output (e.g., recognition of the image data, latent embedding of the image data, encoded representation of the image data, hash of the image data, etc.). As another example, the machine-learned model(s) can process the image data to generate an image segmentation output. As another example, the machine-learned model(s) can process the image data to generate an image classification output. As another example, the machine-learned model(s) can process the image data to generate an image data modification output (e.g., modification of the image data, etc.). As another example, the machine-learned model(s) can process the image data to generate an encoded image data output (e.g., encoded representation and / or compressed representation of the image data, etc.). As another example, the machine-learned model(s) can process the image data to generate an upscaled image data output. As another example, the machine-learned model(s) can process the image data to generate a prediction output.

[0090] In some embodiments, the input to the machine-learned model(s) of the present disclosure can be text or natural language data. The machine-learned model(s) can process the text or natural language data to generate an output. As another example, the machine-learned model(s) can process natural language data to generate a language encoding output. As another example, the machine-learned model(s) can process text or natural language data to generate a latent text embedding output. As another example, the machine-learned model(s) can process text or natural language data to generate a translation output. As another example, the machine-learned model(s) can process text or natural language data to generate a classification output. As another example, the machine-learned model(s) can process text or natural language data to generate a text segmentation output. As another example, the machine-learned model(s) can process text or natural language data to generate a semantic intent output. As another example, the machine-learned model(s) can process text or natural language data to generate an upscaled text or natural language output (e.g., text or natural language data of higher quality than the input text or natural language). As another example, the machine-learned model(s) can process text or natural language data to generate a prediction output.

[0091] In some embodiments, the input to the machine-learned model(s) of the present disclosure can be audio data. The machine-learned model(s) can process the audio data to generate an output. As an example, the machine-learned model(s) can process the audio data to generate an audio recognition output. As another example, the machine-learned model(s) can process the audio data to generate an audio translation output. As another example, the machine-learned model(s) can process the audio data to generate a latent embedding output. As another example, the machine-learned model(s) can process the audio data to generate an encoded audio output (e.g., an encoded representation and / or a compressed representation of the audio data, etc.). As another example, the machine-learned model(s) can process the audio data to generate an upscaled audio output (e.g., audio data of higher quality than the input audio data, etc.). As another example, the machine-learned model(s) can process the audio data to generate a text display output (e.g., a text display of the input audio data, etc.). As another example, the machine-learned model(s) can process the audio data to generate a prediction output.

[0092] In some embodiments, the input to the machine-learned model(s) of the present disclosure can be latent encoded data (e.g., a latent space representation of the input, etc.). The machine-learned model(s) can process the latent encoded data to generate an output. As an example, the machine-learned model(s) can process the latent encoded data to generate a recognition output. As another example, the machine-learned model(s) can process the latent encoded data to generate a reconstruction output. As another example, the machine-learned model(s) can process the latent encoded data to generate a search output. As another example, the machine-learned model(s) can process the latent encoded data to generate a reclustering output. As another example, the machine-learned model(s) can process the latent encoded data to generate a prediction output.

[0093] In some embodiments, the input to the machine-learned model(s) of the present disclosure can be statistical data. The machine-learned model(s) can process the statistical data to generate an output. As an example, the machine-learned model(s) can process the statistical data to generate a recognition output. As another example, the machine-learned model(s) can process the statistical data to generate a prediction output. As another example, the machine-learned model(s) can process the statistical data to generate a classification output. As another example, the machine-learned model(s) can process the statistical data to generate a segmentation output. As another example, the machine-learned model(s) can process the statistical data to generate a visualization output. As another example, the machine-learned model(s) can process the statistical data to generate a diagnostic output.

[0094] In some cases, the machine-learned model(s) can be configured to perform tasks including encoding the input data for reliable and / or efficient transmission or storage (and / or corresponding decoding). For example, the task can be an audio compression task. The input can include audio data, and the output can include compressed audio data. In other examples, the input includes visual data (e.g., one or more images or videos), the output includes compressed visual data, and the task is a visual data compression task. In other examples, the task can include generating an embedding for the input data (e.g., input audio or visual data).

[0095] In some cases, the input includes visual data and the task is a computer vision task. In some cases, the input includes pixel data of one or more images and the task is an image processing task. For example, the image processing task can be image classification, and the output is a set of scores, where each score corresponds to a different object class and represents the likelihood that one or more images contain an object belonging to the object class. The image processing task can be object detection, and the image processing output identifies one or more regions in one or more images and, for each region, the likelihood that the region represents an object of interest. As another example, the image processing task can be image segmentation, and the image processing output defines, for each pixel of one or more images, each likelihood for each category of a set of predetermined categories. For example, the set of categories can be foreground and background. As another example, the set of categories can be object classes. As another example, the image processing task can be depth estimation, and the image processing output defines, for each pixel of one or more images, each depth value. As another example, the image processing task can be motion estimation, the network input includes multiple images, and the image processing output defines, for each pixel of one of the input images, the motion of the scene depicted in the pixels between the images in the network input.

[0096] In some cases, the task includes encrypting or decrypting the input data. In some cases, the task includes microprocessor performance tasks such as branch prediction or memory address translation.

[0097] FIG. 1A shows one exemplary computing system that can be used to implement the present disclosure. Other computing systems can be used as well. For example, in some embodiments, user computing device 102 can include model trainer 160 and training dataset 162. In such embodiments, model 120 can perform both training and use locally on user computing device 102. In some of such embodiments, user computing device 102 can implement model trainer 160 for personalizing model 120 based on user-specific data.

[0098] FIG. 1B shows a block diagram of an exemplary computing device 10 executing in accordance with an exemplary embodiment of the present disclosure. Computing device 10 can be a user computing device or a server computing device.

[0099] Computing device 10 includes several applications (e.g., applications 1 - N). Each application includes its own machine learning library and one or more machine-learned models. For example, each application can include a machine-learned model. Exemplary applications include a text messaging application, an email application, a dictation application, a virtual keyboard application, a browser application, and the like.

[0100] As shown in FIG. 1B, each application can communicate with some other components of a computing device, such as, for example, one or more sensors, a context manager, a device state component, and / or additional components. In some embodiments, each application can communicate with each device component using an API (e.g., a public API). In some embodiments, the API used by each application is specific to that application.

[0101] FIG. 1C represents a block diagram of an exemplary computing device 50 operating in accordance with an exemplary embodiment of the present disclosure. The computing device 50 can be a user computing device or a server computing device.

[0102] The computing device 50 includes several applications (e.g., applications 1 to N). Each application communicates with a central intelligence layer. Exemplary applications include a text messaging application, an email application, a dictation application, a virtual keyboard application, a browser application, and the like. In some embodiments, each application can communicate with the central intelligence layer (and the model(s) stored therein) using an API (e.g., a common API across all applications).

[0103] The central intelligence layer includes several machine - learned models. For example, as shown in FIG. 1C, each machine - learned model (e.g., the model) can be provided for each application and can be managed by the central intelligence layer. In other embodiments, two or more applications can share a single machine - learned model. For example, in some embodiments, the central intelligence layer can provide a single model (e.g., a single model) for all of the applications. In some embodiments, the central intelligence layer is included within or otherwise implemented by the operating system of the computing device 50.

[0104] The central intelligence layer can communicate with the central device data layer. The central device data layer can be a centralized repository of data for the computing device 50. As shown in FIG. 1C, the central device data layer can communicate with some other components of the computing device, such as, for example, one or more sensors, a context manager, a device state component, and / or additional components. In some embodiments, the central device data layer can communicate with each device component using an API (e.g., a private API).

[0105] Model Placement Examples Figure 2 shows a diagram of an exemplary snippet packet generation interface 200 according to an exemplary embodiment of the present disclosure. The snippet packet generation interface can be configured to receive text selection, image selection, screenshots, video selection, and / or audio selection. Next, an option menu can pop up with a plurality of options including save options. Since the save option can be selected, it can prompt to generate a graphics card and provide it for display. A snippet packet can be generated. The user can then select in which collection to save the snippet packet.

[0106] For example, in 202, a display window of a web page is provided for display, a portion of the web page including a text content item is selected, and a save user interface element is selected. In 204, a snippet packet is generated, and a generated graphic display (e.g., a graphic card with a portion of text with a title and position indicated) is provided for display. Further, in 204, the user interface includes a pop-up window for providing options to add to a collection, which can include adding to an existing collection and / or generating a new collection. The collection can be automatically generated and / or manually generated.

[0107] In 206, a portion of the search results of a web page can be selected (e.g., a portion of a knowledge graph can be selected), and a save option can be selected. In 208, a graphic card is provided for display, and the graphic card is generated based on the selected portion of the search results page. For example, the graphic card can include a picture associated with the selected portion as a background, and the selected text is displayed in the foreground. Additionally and / or alternatively, the graphic card can include the search query and related search engines. Options to add to a collection can be provided.

[0108] At 210, a screenshot input is received and save options for display are provided. At 212, a save input is received and options for the generated graphic display and addition to a collection are provided for display. The graphic display can include at least a portion of the screenshot and a banner including source information (e.g., the title of the web page, entities associated with the screenshot information, and / or the source of the screenshot information).

[0109] The generated snippet packet can be stored along with the generated graphic display. Next, the user may select the snippet packet to view an enlarged graphic display, view the saved content item(s), and / or navigate to a specific location on the web page where the content item(s) in the resource are located.

[0110] FIG. 3 shows a diagram of an exemplary gesture interaction 300 according to an exemplary embodiment of the present disclosure. For example, user input requesting a clipping interface can be received (e.g., selection of the "clip" user interface element 302). A snippet packet generation interface can be provided, and a graphic animation 304 can be provided to demonstrate a method of selecting a content item. The graphic animation can demonstrate that a swipe gesture in the movement of a circle can be utilized to select a content item. The instruction user interface element 306 may be provided as a pop-up to provide instructions regarding how to clip and save.

[0111] Figure 4 shows a diagram of an exemplary snippet packet generation interface 400 according to an exemplary embodiment of the present disclosure. In some embodiments, a snippet packet generation interface can be requested based on received user input, and a snippet packet generation interface can be provided. The snippet packet generation interface can include clipping options or summarization options that can enable the generation of snippet packets based on selected content items. The snippet packet generation interface 400 can include a display window 402 and an interaction pane 404 of a web page, which can be provided in response to web page interactions and can provide a plurality of web page interaction options for display. The interaction options can be predefined and can be unified across the web page. Alternatively and / or additionally, the interaction options can be varied based on the content of the web page, based on selected content items, and / or based on entities associated with the web page. The interaction options can include adding to a collection, summarizing a selected portion and / or the entire web page, clipping a portion of the web page, saving an image, finding more web pages like the current web page, comparing products on the web page to other similar products, and / or tracking price options.

[0112] FIG. 5 shows an exemplary snippet packet generation and collection addition interface 500 according to an exemplary embodiment of the present disclosure. The systems and methods disclosed herein can be utilized to generate and save snippet packets. Snippet packets can be added to a collection and the snippet packets may be searchable for later use. The snippet packets can include the generated graphic card. In some embodiments, the selected content item can be processed by one or more machine learned models to generate a summary of the content item, and this can be utilized to generate the graphic card. The graphic card can be customizable. The graphic card may include a background color and / or background image determined based on the source web page. Alternatively and / or additionally, the background color and / or background image may be selected by the user. Similarly, the font may be automatically determined, predefined, and / or selected by the user.

[0113] At 502, a snippet packet (including a graphic display) is generated based on a selected portion of a web page. Next, the generated snippet packet is added to the media content item and the "Inspo" collection of the snippet packet.

[0114] At 504, a summary is generated for a portion of the web page and a graphic card is generated based on semantic understanding and / or determined entities. The generated graphic card can be saved as part of the generated snippet packet and added to a collection. The collection may be associated with a particular entity and / or a particular type of entity.

[0115] At 506, different options are provided for sharing and / or customizing the generated snippet packet for display. For example, the template, text, and / or background of the graphic display can be customized. The sharing options can include adding to a collection, adding to a memo, sending by text message, sending by email, copying, and / or performing an AirDrop.

[0116] At 508, the snippet packet can be published to social media and / or may be published to a localized search interface. For example, the user may utilize a search application, which can surface multiple web search results in response to a query and / or surface one or more generated snippet packets (e.g., the generated snippet packets of a particular user and / or the generated snippet packets of related users (e.g., friends or users close to a particular user)) in response to a query.

[0117] Next, the generated snippet packet can be shared via a messaging application, a social media application, and / or various other means. In some embodiments, the snippet packet can be published to the web and may be utilized as a new format for web search results.

[0118] FIG. 9 shows a diagram of an exemplary collection addition interface 900 according to an exemplary embodiment of the present disclosure. The collection addition interface can provide a graphic display (e.g., a graphic card) of content items and / or snippet packets for display. In response to a save element being selected, a plurality of collections can be provided for display. Next, the user can add the snippet packet to a particular collection. Next, the snippet packet can be stored in a particular collection.

[0119] Next, a specific collection can be opened and provided for displaying the graphic representation of the generated snippet packet together with other graphic representations associated with other snippet packets. The collection addition interface 900 may include displaying the graphics card (at a first size) for display during snippet packet generation 902. Next, a pop-up window 904 for collection addition can be provided for display upon selection of a user interface element. When a snippet packet is added to a specific collection, the collection 906 may be provided for display with a plurality of thumbnails describing different snippet packets in a collection that includes a thumbnail describing the generated graphics card (at a second size).

[0120] FIG. 10A shows a diagram of an exemplary snippet packet interaction 1020 according to an exemplary embodiment of the present disclosure. When a snippet packet is generated, the snippet packet can be interacted with to navigate to a web page associated with a content item. The position data of the snippet packet can be utilized to navigate to a specific portion of the web page that includes the source of the content item. The content item can be highlighted when displayed. For example, the graphics card of 1022 can be selected. Next, the address data and position data of the snippet packet associated with the graphics card can be obtained. Next, the address data and position data can be utilized to open the web page of 1024 at the exact position of one or more content items of the selected snippet packet that highlights one or more content items.

[0121] FIG. 10B shows a diagram of an exemplary snippet packet search 1040 according to an exemplary embodiment of the present disclosure. The generated snippet packets can be provided as search results when searching locally and / or when searching the web. For example, a user can enter a search query, the search query can be processed, a plurality of proposed queries and a plurality of proposed snippet packets can be determined, and further input can be received to provide those snippet packets for display (e.g., as shown at 1042).

[0122] FIG. 10C shows a diagram of an exemplary highlight interface 1080 according to an exemplary embodiment of the present disclosure. In some embodiments, the highlight (e.g., as shown at 1082) can be removed (e.g., as shown at 1084) in response to receiving user input requesting removal of the highlight.

[0123] FIG. 11 shows a diagram of an exemplary graphics card 1100 according to an exemplary embodiment of the present disclosure. The graphics card can be automatically generated and / or generated based on one or more user inputs. The graphics card can include a portion assigned to a content item and / or a portion assigned to an attribution to the source of the content item. The graphics card can include text, image(s), video(s), and / or audio. The graphics card can vary based on the selected content item. The text can be the selected text and / or a summary of data retrieved from a web page. In some embodiments, the background and / or image can be automatically selected / generated and / or selected by the user. In particular, FIG. 11 shows two formats of a text-based graphics card 1102 having an attribution of a resource and a related title / search query. That is, an image-based graphics card 1104 having an image, an entity thumbnail, a title or caption, and an attribution of a resource, and a screenshot-based graphics card 1106 including a screenshot, an entity logo, a title of each article, and an attribution of a resource.

[0124] FIG. 12 shows a diagram of an exemplary summary snippet packet generation interface 1200 according to an exemplary embodiment of the present disclosure. Snippet packet generation can include generating a graphic card. The graphic card can include text that describes a summary of selected data within a web page. The summary can be generated by processing the selected data with a machine-learned model. For example, at 1202, a portion of the web page is selected. The user can select a summary user interface element to process the selected portion and generate a summary. The summary may include words from a section and / or different words that summarize a section in simple terms or user-specific terms. At 1204, a graphic card is generated using the summarized text and stored together with a generated snippet packet that includes information associated with a specific location within the web page and the selected portion within the web page.

[0125] FIG. 13 shows a diagram of an exemplary snippet packet proposal interface 1300 according to an exemplary embodiment of the present disclosure. In some embodiments, the snippet packet proposal interface can include providing a proposed graphic card associated with the proposed snippet packet (e.g., may be displayed in response to a selection of a bookmark option as shown at 1302, as shown at 1304). The proposed snippet packet can be based on past user interactions by a particular user and / or based on past interactions by other users (e.g., popular snippets). The proposed snippet packet can be locally selected and stored by a user computing system.

[0126] FIG. 14 shows a diagram of an exemplary snippet packet generation and sharing interaction 1400 according to an exemplary embodiment of the present disclosure. The snippet packet can be generated in response to a user selection of a content item (e.g., the selection at 1402). Next, the generated snippet packet can be shared (e.g., via the text message option of a sharing interface 1404 that provides a plurality of sharing options). The snippet packet can be shared via a messaging application, a social media application, and / or inserted into other data files (e.g., a memo, a text document, and / or a slide deck). The shared snippet packet can be transmitted with a graphics card and a download option for local storage (e.g., as shown at 1406). Alternatively and / or additionally, the graphics card may be selectable to navigate to one or more content items of the snippet packet of a native web page.

[0127] FIG. 15 shows a diagram of an exemplary graphics card customization interface 1500 according to an exemplary embodiment of the present disclosure. The graphics card customization interface can be accompanied by a plurality of templates that can be selected to customize the graphics card. The templates can include different images, different colors, different fonts, and / or different layouts. For example, the initial graphics card 1502 can include text in a first font, text in a first size, and a first background. A change template request can be received, and an extended graphics card 1504 can be generated with a different text font, a different text size, and / or a different background.

[0128] FIG. 16 shows a block diagram of an exemplary snippet packet generation system 1600 according to an exemplary embodiment of the present disclosure. The snippet packet generation system 1600 can include obtaining input data 1602 (e.g., input data describing a selection of one or more content items). The input data (1602) can be processed to determine the selected content items (1604). Based on the determined selection, content items can be obtained and a graphic card can be generated (1606). Address data can be generated and / or obtained (1608). The address data can include uniform resource locator data (1608). Location data can be generated and / or obtained (1610). The location data can include text fragment data that can be used to find and highlight content items within a source page, including a scroll position, a start of a content item, and an end of a content item (1610). Using the graphic card (1606), the content items, the address data (1608), and the location data (1610), a snippet packet (1612) can be generated. The snippet packet (1612) can be processed to determine one or more entity tags for the snippet packet (1612) based on the content items and / or based on the source of the content items (1614). The entity tag (1614) can include relationship data that links the snippet packet (1612) to other snippet packets associated with the same entity. The snippet packet (1612) having the entity tag(s) (1614) can then be stored (e.g., locally and / or on a server computing system) (1616).

[0129] One or more of the decisions and / or one or more of the generations can be performed at least in part based on one or more machine-learned models. For example, determining a selected content item (1604), obtaining a content item and / or generating a graphics card (1606), generating location data (1610), and / or determining an entity tag (1614) can be performed by one or more machine-learned models.

[0130] Exemplary method FIG. 6 shows a flowchart diagram of an exemplary method for performing in accordance with an exemplary embodiment of the present disclosure. FIG. 6 shows steps performed in a particular order for purposes of illustration and explanation, but the methods of the present disclosure are not limited to the particular order or arrangement shown. The various steps of method 600 can be omitted, rearranged, combined, and / or adapted in various ways without departing from the scope of the present disclosure.

[0131] At 602, a computing system can provide data that describes a graphical user interface. The graphical user interface can include a graphical window for displaying a web page. In some embodiments, the web page can include a plurality of content items.

[0132] At 604, a computing system can obtain input data. The input data can include a request to save one or more of the plurality of content items. In some embodiments, the one or more content items can include at least one of an image, a video, a graphic depiction of a product, or an audio.

[0133] In 606, the computing system can generate snippet packets. The snippet packets can be generated based on input data. The snippet packets can include one or more content items, address data, and location data.

[0134] One or more content items can include text data (e.g., text data describing words, sentences, quotes, and / or paragraphs), image data (e.g., an image of an object (e.g., a product)), video data (e.g., a video and / or one or more video frames), audio data (e.g., waveform data), and / or latent encoded data. One or more content items can include multimodal data.

[0135] In some embodiments, generating the snippet packets can include obtaining one or more content items and generating a graphic card. The graphic card can describe one or more content items. The graphic card can include text data overlaid on a color and / or an image. The color can be determined based on the primary color of the web page. In some embodiments, the color can be predefined and / or determined based on surrounding content items. The image can be an image determined based on the determined topic of the content item. Alternatively and / or additionally, the image can be an image close to the content item. In some embodiments, the graphic card can include a font determined based on the font used in the web page. The graphic card can include a background, text describing the content item (e.g., the content item and / or a summary of the content item), and / or text and / or logo describing the source and / or the determined entity. In some embodiments, the text size of the text in the graphic card can be based on the amount of text in the content item.

[0136] Address data can describe the web address of a web page. The address data can include a Uniform Resource Identifier and / or a Uniform Resource Locator.

[0137] Location data can describe the location of one or more content items within a web page. In some embodiments, the location data can include at least one of a scroll position, a start node, or an end node. The scroll position can describe the location of one or more content items relative to other parts of the web page. In some embodiments, the start node can describe the starting position of one or more content items. The end node can describe the ending position of one or more content items. The location data can include a text fragment (Tomayac et al., “Scroll to Text Fragment,” GITHUB, (May 20, 2022, 9:40PM) https: / / github.com / WICG / scroll-to-text-fragment) that can be used to indicate the location of a content item. In some embodiments, the text fragment can include one or more text directives associated with the location. The text directive can include a string of start data (e.g., the first text associated with the content item and / or the first pixel associated with the content item) and / or end data (e.g., the last text associated with the content item and / or the last pixel associated with the content item). The text directive can be used to search the web page for a set of data that matches the beginning and / or end of a content item.

[0138] In some embodiments, a computing system can include processing one or more content items to determine entities associated with the one or more content items. An entity tag can be generated based on the entity. A snippet packet can include the entity tag.

[0139] At 608, the computing system can store the snippet packet. The snippet packet can be associated with a particular user. Associating with a particular user can include storing the snippet packet with metadata indicating the user and / or associating the snippet packet with a particular user profile. The particular user can be a user who provides input data that is processed to determine a snippet packet generation request. The snippet packet can be stored in a user database. In some embodiments, storing the snippet packet in a user database can include locally storing the snippet packet on a mobile computing device. Additionally and / or alternatively, a graphics card can store the snippet packet as a graphic display. The graphics card can be automatically generated and may be customizable by a user. The graphics card can begin with a template that can be customized based on other content of a web page, based on user input, and / or based on other context. In some embodiments, the graphics card can include multimodal data. Alternatively and / or additionally, the snippet packet can be stored in a server computing system. In some embodiments, when a content item references another content item, the system and method can obtain the additional content item and save the additional content item in the snippet packet.

[0140] In some embodiments, the system and method can include receiving a snippet request to provide a snippet interface. The snippet interface can include interactive elements associated with a snippet packet. The snippet interface can be provided for display. An interface selection can be received, and the interface selection can describe a selection to select an interactive element. Next, a portion of a web page can be provided for display. In some embodiments, the portion of the web page can include the location of one or more content items within the web page.

[0141] In some embodiments, the system and method can include receiving an insertion input. The insertion input can include user input requesting the insertion of a content item into a different interface. The snippet packet can be provided to a third-party server computing system.

[0142] Additionally and / or alternatively, the system and method can include adding a snippet packet to a collection. The snippet packet can be added to the collection based on received user input. Alternatively and / or additionally, the snippet packet can be automatically added to the collection based on a determined entity associated with the content item. In some embodiments, the snippet packet can be added to the collection based on the source of the content item (e.g., based on the type of web page, the type of media provider, and / or the type of content item).

[0143] In some embodiments, the snippet packet can be generated based on content items obtained from sources other than a web page (e.g., mobile applications, large data files (e.g., downloaded videos or books), and / or data from other sources).

[0144] FIG. 7 shows a flowchart diagram of an exemplary method for execution in accordance with an exemplary embodiment of the present disclosure. FIG. 7 shows steps executed in a particular order for purposes of illustration and explanation, but the method of the present disclosure is not limited to the particular order or arrangement shown. The various steps of method 700 can be omitted, rearranged, combined, and / or adapted in various ways without departing from the scope of the present disclosure.

[0145] At 702, the computing system can obtain input data. In some embodiments, the input data can describe a selection of content items associated with a snippet packet. The snippet packet can be associated with a particular user's user account. Additionally and / or alternatively, the user account can be associated with one or more platforms. The snippet packet can include content items and deep links. Alternatively and / or additionally, the snippet packet can include snippets (e.g., content items and / or media data generated based on content items (e.g., graphic cards and / or summaries of content items)), address data (e.g., uniform resource locators and / or uniform resource identifiers), and / or metadata. The snippet packet can include location data (e.g., metadata indicating the location of a content item within a web page, text fragments for identifying start data and end data, one or more pointers, and / or one or more scroll position data).

[0146] In some embodiments, a snippet packet may be generated by providing a graphical user interface for display. The graphical user interface can include a graphical window for displaying a web page. In some embodiments, the web page can include a plurality of content items. Snippet packet generation can include obtaining selection data. The selection data can include a request to save a content item among the plurality of content items. In some embodiments, snippet packet generation can include generating a snippet packet and storing the snippet packet in a user database.

[0147] At 704, the computing system can obtain address data and location data associated with the snippet packet. The address data can be associated with a web page. The content item can be associated with a web page. Additionally and / or alternatively, the location data can describe the location of the content item within the web page.

[0148] At 706, the computing system can obtain web page data. The web page data can be obtained at least in part based on the address data (e.g., by navigating to the web page using a uniform resource locator).

[0149] At 708, the computing system can determine the location within the web page associated with the content item. The location can be determined based on a text fragment, one or more pointers, and / or via web page processing.

[0150] In some embodiments, the address data can include a Uniform Resource Locator. Additionally and / or alternatively, the location data can include one or more text fragments. Determining the location within a web page associated with a content item can then include adding the text fragment to the Uniform Resource Locator to generate a shortcut link and entering the shortcut link into a browser.

[0151] At 710, the computing system can provide a portion of a web page. The portion of the web page can include the location of the content item. In some embodiments, providing a portion of the web page can include providing one or more indicators on the portion of the web page. The one or more indicators can indicate the content item associated with the snippet packet. In some embodiments, the one or more indicators can include highlighting the text associated with the content item.

[0152] FIG. 8 shows a flowchart diagram of an exemplary method for performing in accordance with an exemplary embodiment of the present disclosure. FIG. 8 shows steps performed in a particular order for purposes of illustration and explanation, but the methods of the present disclosure are not limited to the particular order or arrangement shown. The various steps of method 800 can be omitted, rearranged, combined, and / or adapted in various ways without departing from the scope of the present disclosure.

[0153] At 802, the computing system can provide a graphical user interface. The graphical user interface can include a graphical window for displaying a web page. In some embodiments, the web page can include a plurality of content items.

[0154] At 804, the computing system can receive gesture data. The gesture data can describe a gesture associated with a portion of a web page. In some embodiments, the gesture can include a circular gesture surrounding a portion of the web page. The gesture data can describe touch input to a touch screen display of a mobile computing device.

[0155] At 806, the computing system can process the gesture data to determine a selected content item. The selected content item can be associated with a portion of the web page.

[0156] In some embodiments, processing the gesture data to determine the selected content item can include determining the portion of the web page surrounded by the gesture, determining a focus of the portion, and determining that the selected content item is associated with the focus of the portion.

[0157] Alternatively and / or additionally, processing the gesture data to determine the selected content item can include processing the gestured data with a machine-learned model to determine the selected content item.

[0158] In some embodiments, the selected content item can be determined based on a determined match area associated with a rectangle determined based on a circle. The rectangle can be a rectangle of data based on the syntactic structure of the web page. In some embodiments, the selected content item can be determined based on determined word boundaries and / or determined media content boundaries. The determination can be based on code boundaries of a hypertext markup language. In some embodiments, the source code of the web page can be parsed and the parsed data can be processed.

[0159] Alternatively and / or additionally, the determination can include calculating the area of a gesture rectangle associated with the gesture. The area of one or more content item elements can be determined. The intersecting area between the area of the gesture rectangle and the area of each different content item element can be determined. The element with the highest degree of intersection may have the highest probability of being selected and thus can be determined as the selected content item.

[0160] At 808, the computing system can generate a snippet packet based on the gesture data. The snippet packet can include the selected content item. In some embodiments, the snippet packet can include address data and location data. The address data can be associated with a web page, and the location data may describe the location of the content item within the web page.

[0161] Next, a save interface can be provided for display. The save interface can provide an interactive interface element that can be selected to add the snippet packet to a collection. Next, a collection interface can be provided. Next, a drag input can be received that drags the graphic display of the snippet packet to a graphic tile that describes a particular collection. Next, the snippet packet can be stored in the collection (e.g., the snippet packet can be stored with a relationship tag that links the snippet packet to a particular collection). In some embodiments, the graphic display of the snippet packet can change in size and / or ratio when the graphic display is dragged to a particular collection. The change in size and ratio can provide an intuitive indication of collection addition while providing an aesthetically pleasing display.

[0162] In some embodiments, content items can be processed to determine entities associated with the content items. The entities can be determined by processing the content items (e.g., text data, image data, audio data, video data, latent encoded data, and / or link data) with a machine-learned model (e.g., an image classification model, an object classification model, a text classification model (e.g., a natural language processing model), a segmentation model, a semantic model, and / or a detection model) to generate entity data (e.g., classification). Next, relationship data can be generated and added to the snippet packet based on the entity data. The relationship data can include one or more entity tags and / or references to other related snippet packets and / or related web pages or content items.

[0163] The snippet packet can be searchable within one or more applications and / or databases. Additionally and / or alternatively, the snippet packet can be shareable via a messaging application, a social media application, and / or other applications.

[0164] In some embodiments, content items can be processed with one or more machine-learned models to generate tags that can be stored in a snippet packet. Next, the tags can be utilized as searchable tags to surface the snippet packet in response to a search query. The tags can be determined based on the content of the content item (e.g., recognized words, recognized objects in an image, characteristics of a video frame or audio stream, etc.).

[0165] In some embodiments, snippet packet generation occurs after selection of a snippet packet generation interface element that can open a snippet packet generation interface. Alternatively and / or additionally, a user can select a content item and provide various options for interacting with the content item in one or more (e.g., two) pop-ups, one of the options can include snippet packet generation. Alternatively and / or additionally, search results associated with the content item may be provided.

[0166] Additional Disclosure The techniques described herein refer to servers, databases, software applications, and other computer-based systems, as well as actions taken and information sent between such systems. The inherent flexibility of computer-based systems allows for a wide variety of possible configurations, combinations, and divisions of tasks and functionality between components. For example, the processes described herein can be implemented using a single device or component, or multiple devices or components operating in combination. Databases and applications can be implemented in a single system or distributed across multiple systems. Distributed components can operate sequentially or in parallel.

[0167] Although the present subject matter has been described in detail with respect to its various specific and exemplary embodiments, each example is provided for illustrative purposes and is not intended to limit the disclosure. Those skilled in the art will readily appreciate, upon understanding the foregoing, that modifications, variations, and equivalents to such embodiments can be readily made. Accordingly, the disclosure is not intended to exclude such modifications, variations, and / or additions to the subject matter that would be readily apparent to those skilled in the art. For example, features shown or described as part of one embodiment can be used with other embodiments to yield further embodiments. Accordingly, the disclosure is intended to embrace such modifications, variations, and equivalents.

Claims

1. A computing system comprising: one or more processors; and one or more non-transitory computer-readable media that, when executed by the one or more processors, collectively store instructions that cause the computing system to perform operations, the operations including: providing data that describes a graphical user interface, the graphical user interface including a graphical window for displaying a web page, the web page including a plurality of content items; acquiring input data, the input data including a request to save one or more of the plurality of content items; generating a snippet packet, the snippet packet including: the one or more content items; address data that describes a web address of the web page; and location data that describes a location of the one or more content items within the web page; storing the snippet packet, the snippet packet being associated with a particular user.

2. The operations further include: receiving a snippet request to provide a snippet interface, the snippet interface including interactive elements associated with the snippet packet; providing the snippet interface for display; receiving an interface selection to select one of the interactive elements; and providing a portion of the web page for display, the portion of the web page including the location of the one or more content items within the web page.

3. The operations further include: receiving an insertion input, the insertion input including user input requesting insertion of the one or more content items into a different interface. The system according to claim 1 or 2, further comprising providing the snippet packet to a third-party server computing system.

4. Generating the snippet packet comprises: obtaining the one or more content items; generating a graphic card that describes the one or more content items; and storing the graphic card as a graphic representation of the snippet packet. The system according to any one of claims 1, 2, and 3.

5. The position data includes at least one of a scroll position, a start node, or an end node, wherein the scroll position describes the position of the one or more content items relative to other portions of the web page, the start node describes the start position of the one or more content items, and the end node describes the end position of the one or more content items. The system according to one of claims 1 to 4.

6. The one or more content items include at least one of an image, a video, a graphic depiction of a product, or an audio. The system according to one of claims 1 to 5.

7. The operations include: processing the one or more content items to determine an entity associated with the one or more content items; and generating an entity tag based on the entity. The snippet packet includes the entity tag. The system according to one of claims 1 to 6.

8. Storing the snippet packet includes locally storing the snippet packet on a mobile computing device. The system according to one of claims 1 to 7.

9. A computer-implemented method, comprising: obtaining, by a computing system including one or more processors, input data that describes a selection of content items associated with a snippet packet. A step of obtaining address data and location data associated with the snippet packet by the computing system, wherein the address data is associated with a web page, the content item is associated with the web page, and the location data describes the location of the content item within the web page; A step of obtaining web page data by the computing system, wherein the web page data is obtained at least partially based on the address data; A step of determining the location within the web page associated with the content item by the computing system; A computer-implemented method including: a step of providing a portion of the web page by the computing system, wherein the portion of the web page includes the location of the content item.

10. The address data includes a uniform resource locator, the location data includes a text fragment, and the step of determining the location within the web page associated with the content item by the computing system includes: A step of adding the text fragment to the uniform resource locator by the computing system to generate a shortcut link; The method according to claim 9, further including: a step of inputting the shortcut link into a browser by the computing system.

11. The snippet packet is A step of providing a graphical user interface by the computing system, wherein the graphical user interface includes a graphical window for displaying the web page, and the web page includes a plurality of content items; A step of obtaining selection data by the computing system, wherein the selection data includes a request for saving the content item among the plurality of content items; A step of generating the snippet packet by the computing system. The method according to claim 9 or 10, generated by the computing system storing the snippet packet in a user database.

12. The step of providing, by the computing system, the portion of the web page is a step of providing one or more indicators to the portion of the web page, the one or more indicators indicating the content item associated with the snippet packet, the method according to any one of claims 9, 10, and 11.

13. The method according to claim 12, wherein the one or more indicators include a step of highlighting text associated with the content item.

14. The method according to any one of claims 9 to 13, wherein the snippet packet is associated with a user account of a specific user, and the user account is associated with one or more platforms.

15. One or more non-transitory computer-readable media that collectively store instructions that, when executed by one or more computing devices, cause the one or more computing devices to perform operations, the operations being providing a graphical user interface, the graphical user interface including a graphical window for displaying a web page, the web page including a plurality of content items; receiving gesture data, the gesture data describing a gesture associated with a portion of the web page; processing the gesture data to determine a selected content item, the selected content item being associated with the portion of the web page; generating a snippet packet based on the gesture data, the snippet packet including the selected content item, one or more non-transitory computer-readable media.

16. The snippet packet includes address data and location data, the address data is associated with the web page, and the location data describes the location of the selected content item within the web page, one or more non-transitory computer-readable media according to claim 15.

17. The gesture includes a circular gesture surrounding the portion of the web page, one or more non-transitory computer-readable media according to claim 15 or 16.

18. Processing the gesture data to determine the selected content item includes determining the portion of the web page surrounded by the gesture, determining the focus of the portion, and determining that the selected content item is associated with the focus of the portion, one or more non-transitory computer-readable media according to one of claims 15 to 17.

19. Processing the gesture data to determine the selected content item includes processing the gestural data with a machine-learned model to determine the selected content item, one or more non-transitory computer-readable media according to one of claims 15 to 18.

20. The gesture data describes a touch input to a touch screen display of a mobile computing device, one or more non-transitory computer-readable media according to one of claims 15 to 19.

Images