Generating snippet packets based on a selection of portions of a web page

Abstract

A system and method for generating snippet packets can include obtaining input data (e.g., input data describing a gesture). The input data can be processed to determine a content item selected by the input. Based on the content item, a snippet packet can be generated, which can include the content item, address data, and location data. The snippet packet can be configured to be interacted with to navigate to the source web page of the content item, including navigating to a specific portion of the web page that includes the content item.

Description

[Technical Field]

[0001] Related Applications This application claims priority to and the benefit of U.S. Non-provisional Patent Application No. 18 / 081,814, filed December 15, 2022, which claims priority to and the benefit of U.S. Provisional Patent Application No. 63 / 344,783, filed May 23, 2022. U.S. Non-provisional Patent Application No. 18 / 081,814 and U.S. Provisional Patent Application No. 63 / 344,783 are incorporated herein by reference in their entireties.

[0002] The present disclosure relates generally to generating interactive snippet packets in response to user input, and more particularly to obtaining user input to select content items for saving with snippet packets that can be selected at a later time, and providing portions of web pages that include the content items. [Background technology]

[0003] Saving text, images, and / or audio from a web page may allow a user to re-experience the text, images, and / or audio locally without an Internet connection. However, the saving process may provide limited context of the data's origin, and if a user wants to see the context of the saved data, the user must use the data as a search query, navigate from their browsing history, or try to remember how they got to the web page in the first place. Furthermore, once the source of the web page is found, the user may still have to revisit large portions of the web page to find exactly where the saved data was originally on the web page. Summary of the Invention

[0004] Aspects and advantages of embodiments of the present disclosure will be set forth in part in the description that follows, or may be learned from the description, or may be learned by practice of the embodiments.

[0005] One exemplary aspect of the present disclosure is directed to a computing system. The system may include one or more processors and one or more non-transitory computer-readable media that collectively store instructions that, when executed by the one or more processors, cause the computing system to perform operations. The operations may include providing data describing a graphical user interface. The graphical user interface may include a graphical window for displaying a web page. In some implementations, the web page may include a plurality of content items. The operations may include obtaining input data. The input data may include a request to save one or more content items of the plurality of content items. The operations may include generating a snippet packet. In some implementations, the snippet packet may include one or more content items. The snippet packet may include address data. The address data may describe a web address of the web page. The snippet packet may include location data. The location data may describe the location of the one or more content items within the web page. The operations may include storing the snippet packet. The snippet packet may be associated with a particular user.

[0006] In some implementations, the operations include receiving a snippet request to provide a snippet interface. The snippet interface can include an interactive element associated with the snippet packet. The operations can include providing the snippet interface for display, receiving an interface selection selecting the interactive element, and providing a portion of the web page for display. The portion of the web page can include the location of one or more content items within the web page. In some implementations, the operations can include receiving an insertion input. The insertion input can include user input requesting insertion of one or more content items into a different interface. The operations can include providing the snippet packet to a third-party server computing system.

[0007] In some implementations, generating the snippet packet may include obtaining one or more content items and generating a graphics card. The graphics card may describe the one or more content items. Generating the snippet packet may include storing the graphics card as a graphical representation of the snippet packet. In some implementations, the position data may include at least one of a scroll position, a start node, or an end node. The scroll position may describe a position of one or more content items relative to other portions of the web page. The start node may describe a start position of one or more content items. The end node may describe an end position of one or more content items.

[0008] In some implementations, the one or more content items may include at least one of an image, a video, a graphical depiction of a product, or audio. The operations may include processing the one or more content items to determine an entity associated with the one or more content items and generating an entity tag based on the entity. The snippet packet may include the entity tag. In some implementations, storing the snippet packet may include storing the snippet packet locally on the mobile computing device.

[0009] Another exemplary aspect of the present disclosure is directed to a computer-implemented method. The method may include obtaining, by a computing system including one or more processors, input data. The input data may describe a selection of a content item associated with the snippet packet. The method may include obtaining, by the computing system, address data and location data associated with the snippet packet. The address data may be associated with a web page. The content item may be associated with the web page. In some implementations, the location data may describe a location of the content item within the web page. The method may include obtaining, by the computing system, web page data. The web page data may be obtained based at least in part on the address data. The method may include determining, by the computing system, a location within the web page associated with the content item and providing, by the computing system, a portion of the web page. The portion of the web page may include the location of the content item.

[0010] In some implementations, the address data can include a uniform resource locator. The location data can include a text fragment. Determining, by the computing system, a location within a web page associated with the content item can include, by the computing system, adding, by the computing system, the text fragment to the uniform resource locator to generate a shortcut link, and inputting, by the computing system, the shortcut link into a browser.

[0011] In some embodiments, the snippet packet may be generated by the computing system by providing a graphical user interface. The graphical user interface may include a graphical window for displaying a web page. The web page may include multiple content items. The snippet packet may be generated by the computing system by obtaining selection data, generating a snippet packet, and storing the snippet packet in a user database. The selection data may include a request to save a content item of the multiple content items. In some embodiments, providing the portion of the web page by the computing system may include providing one or more indicators in the portion of the web page. The one or more indicators may indicate a content item associated with the snippet packet. The one or more indicators may include highlighting text associated with the content item. In some embodiments, the snippet packet may be associated with a user account of a particular user. The user account may be associated with one or more platforms.

[0012] Another example aspect of the present disclosure is directed to one or more non-transitory computer-readable media that collectively store instructions that, when executed by one or more computing devices, cause the one or more computing devices to perform operations. The operations can include providing a graphical user interface. The graphical user interface can include a graphical window for displaying a web page. In some implementations, the web page can include multiple content items. The operations can include receiving gesture data. In some implementations, the gesture data can describe a gesture associated with a portion of the web page. The operations can include processing the gesture data to determine a selected content item. The selected content item can be associated with the portion of the web page. The operations can include generating a snippet packet based on the gesture data. In some implementations, the snippet packet can include the selected content item.

[0013] In some implementations, the snippet packet may include address data and location data. The address data may be associated with a web page. The location data may describe a location of a selected content item within the web page. The gesture may include a circular gesture encircling a portion of the web page. In some implementations, processing the gesture data to determine the selected content item may include determining a portion of the web page enclosed by the gesture, determining a focus of the portion, and determining that the selected content item is associated with the focus of the portion. In some implementations, processing the gesture data to determine the selected content item may include processing the gestured data with a machine-learned model to determine the selected content item. The gesture data may describe touch input to a touchscreen display of the mobile computing device.

[0014] Other aspects of the present disclosure are directed to various systems, apparatus, non-transitory computer-readable media, user interfaces, and electronic devices.

[0015] These and other features, aspects, and advantages of various embodiments of the present disclosure will become better understood with reference to the following description and appended claims. The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate exemplary embodiments of the present disclosure and, together with the detailed description, serve to explain the associated principles.

[0016] Detailed descriptions of embodiments directed to those skilled in the art are set forth herein with reference to the accompanying drawings. [Brief explanation of the drawings]

[0017] [Figure 1A] 1 illustrates a block diagram of an exemplary computing system that performs snippet packet generation, according to an exemplary embodiment of the present disclosure. [Figure 1B] 1 illustrates a block diagram of an exemplary computing device that performs snippet packet generation, according to an exemplary embodiment of the present disclosure. [Figure 1C] 1 illustrates a block diagram of an exemplary computing device that performs snippet packet generation, according to an exemplary embodiment of the present disclosure. [Figure 2] 1 illustrates a diagram of an exemplary snippet packet generation interface, according to an exemplary embodiment of the present disclosure. [Figure 3] 1 illustrates an exemplary gesture interaction diagram according to an exemplary embodiment of the present disclosure. [Figure 4] 1 illustrates a diagram of an exemplary snippet packet generation interface, according to an exemplary embodiment of the present disclosure. [Figure 5] 10 illustrates a diagram of an exemplary snippet packet generation and collection addition interface, according to an exemplary embodiment of the present disclosure. [Figure 6] 1 illustrates a flowchart diagram of an exemplary method for performing snippet packet generation, according to an exemplary embodiment of the present disclosure. [Figure 7] 1 illustrates a flowchart diagram of an exemplary method for performing snippet packet interaction, according to an exemplary embodiment of the present disclosure. [Figure 8] 1 illustrates a flowchart diagram of an exemplary method for performing snippet packet generation based on gesture input, according to an exemplary embodiment of the present disclosure. [Figure 9] 10 illustrates a diagram of an exemplary add collection interface, according to an exemplary embodiment of the present disclosure. [Figure 10A] 1 illustrates a diagram of an exemplary snippet packet interaction according to an exemplary embodiment of the present disclosure. [Figure 10B] 1 illustrates a diagram of an exemplary snippet packet search according to an exemplary embodiment of the present disclosure. [Figure 10C] 10A-10C illustrate diagrams of exemplary highlighting interfaces according to exemplary embodiments of the present disclosure. [Figure 11]1 illustrates a diagram of an exemplary graphics card, according to an exemplary embodiment of the present disclosure. [Figure 12] 1 illustrates a diagram of an exemplary summary snippet packet generation interface, according to an exemplary embodiment of the present disclosure. [Figure 13] 10 illustrates a diagram of an exemplary snippet packet proposal interface, according to an exemplary embodiment of the present disclosure. [Figure 14] 1 illustrates a diagram of an exemplary snippet packet generation and sharing interaction, according to an exemplary embodiment of the present disclosure. [Figure 15] 1 illustrates a diagram of an exemplary graphics card customization interface, according to an exemplary embodiment of the present disclosure. [Figure 16] 1 illustrates a block diagram of an exemplary snippet packet generation system, in accordance with an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0018] Reference numbers repeated among the drawings are intended to identify like features in the various embodiments.

[0019] overview The present disclosure is generally directed to generating interactive snippet packets in response to user input. More specifically, the present disclosure relates to obtaining user input selecting a content item for saving with a snippet packet that can be later selected, and providing a portion of a web page including the content item. For example, a user can select a portion of a web page and / or data file. Data describing the selected portion can then be stored along with information associated with the web page / data file and the portion's location relative to the web page / data file. The stored data set can be a snippet packet including a graphical representation of the selected portion, which can include a graphics card with text and / or images from the selected portion. The snippet packet can be stored for later reference and / or shared with other users. The snippet packet can enable a user to view the selected portion and then, upon selection, navigate to a specific location of the selected portion of the original web page / data file. The system and method can include providing a graphical user interface. The graphical user interface can include a graphical window for displaying the web page. In some implementations, the web page can include multiple content items. The system and method can include obtaining input data. The input data may include a request to save one or more content items of the plurality of content items. A snippet packet may be generated. The snippet packet may include one or more content items, address data, and location data. In some implementations, the address data may describe a web address of a web page. The location data may describe a location of the one or more content items within the web page. The snippet packet may be stored in a user database.

[0020] For example, the systems and methods disclosed herein may include providing a graphical user interface. The graphical user interface may include a graphical window for displaying a web page. In some implementations, the web page may include multiple content items. The displayed web page may include text, one or more images, one or more interactive user interface elements, one or more videos, and / or one or more audio clips. The graphical user interface may be part of a browser application.

[0021] The system and method may obtain input data. The input data may include a request to save one or more content items of the plurality of content items. In some implementations, the one or more content items may include at least one of an image, a video, a graphical depiction of a product, or audio. The graphical user interface may be updated to include one or more user interface elements for interacting with the one or more content items, including saving the one or more content items, and may include different options for saving formats. Alternatively and / or additionally, the overlay interface may provide a pop-up interface in response to the request. The input data may describe a gesture associated with one or more content items (e.g., circling one or more content items).

[0022] A snippet packet can be generated. The snippet packet can be generated based on input data. The snippet packet can include one or more content items, address data, and location data. Generating the snippet packet can include processing the one or more content items with one or more machine-learned models to generate semantic understanding output that can be utilized for summarization, annotation, and / or classification.

[0023] The one or more content items may include text data (e.g., text data describing a word, a sentence, a quote, and / or a paragraph), image data (e.g., an image of an object (e.g., a product)), video data (e.g., a video and / or one or more video frames), audio data (e.g., waveform data), and / or latent coding data. The one or more content items may include multimodal data. The address data may include a resource locator and / or a file address associated with a web page (e.g., a web address). The location data may describe where within a web page and / or file the one or more content items are located. For example, the location data may indicate the start and end of the one or more content items within a web page and / or file.

[0024] In some implementations, generating a snippet packet may include obtaining one or more content items and generating a graphics card. The graphics card may describe the one or more content items. The graphics card may include color and / or text data overlaid on an image. The color may be determined based on a predominant color of the web page. In some implementations, the color may be predetermined and / or determined based on surrounding content items. The image may be an image determined based on a determined topic of the content item. Alternatively and / or additionally, the image may be an image close to the content item. In some implementations, the graphics card may include a font determined based on a font used on the web page. The graphics card may include a background, text describing the content item (e.g., the content item and / or a summary of the content item), and / or text and / or logos describing the source and / or the determined entity. In some implementations, the text size of the text in the graphics card may be based on the amount of text in the content item.

[0025] The address data may describe a web address of a web page. The address data may include a uniform resource identifier and / or a uniform resource locator. In some implementations, the address data may include data describing the source of a content item.

[0026] The position data can describe the position of one or more content items within a web page. In some implementations, the position data can include at least one of a scroll position, a start node, or an end node. The scroll position can describe the position of one or more content items relative to other portions of the web page. In some implementations, the start node can describe the start position of one or more content items. The end node can describe the end position of one or more content items. The position data can include a text fragment (Tomayac et al., “Scroll to Text Fragment,” GITHUB, (May 20, 2022, 9:40 PM) https: / / github.com / WICG / scroll-to-text-fragment) that can be utilized to indicate the position of the content item. In some implementations, the text fragment can include one or more text directives associated with the position. The text directive can include a string of code start data (e.g., the first text associated with the content item and / or the first pixel associated with the content item) and / or end data (e.g., the last text associated with the content item and / or the last pixel associated with the content item). Text directives can be used to search a web page for a set of data that matches the beginning and / or end of a content item.

[0027] In some implementations, systems and methods can include processing one or more content items to determine entities associated with the one or more content items. Entity tags can be generated based on the entities. Snippet packets can include entity tags.

[0028] The snippet packets can be stored in a user database. In some embodiments, storing the snippet packets in a user database can include storing the snippet packets locally on the mobile computing device. Additionally and / or alternatively, a graphics card can be stored as a graphical representation of the snippet packets. The graphics card can be automatically generated and can be customizable by a user. The graphics card can start with a template that can be customized based on other content of the web page, based on user input, and / or based on other context. In some embodiments, the graphics card can include multimodal data. Alternatively and / or additionally, the snippet packets can be stored on a server computing system. In some embodiments, if the content item references other content items, the systems and methods can obtain the additional content items and store the additional content items in the snippet packets.

[0029] In some implementations, the system and method may include receiving a snippet request to provide a snippet interface. The snippet interface may include an interactive element associated with the snippet packet. The snippet interface may be provided for display. An interface selection may be received, where the interface selection may describe a selection of the interactive element. A portion of the web page may then be provided for display. In some implementations, the portion of the web page may include the location of one or more content items within the web page.

[0030] In some implementations, the systems and methods can include receiving an insertion input. The insertion input can include user input requesting insertion of a content item into a different interface. The snippet packet can be provided to a third-party server computing system.

[0031] Additionally and / or alternatively, the systems and methods may include adding snippet packets to a collection. The snippet packets may be added to the collection based on received user input. Alternatively and / or additionally, the snippet packets may be automatically added to the collection based on determined entities associated with the content item. In some implementations, the snippet packets may be added to the collection based on the source of the content item (e.g., based on the type of web page, the type of media provider, and / or the type of content item).

[0032] In some implementations, snippet packets can be generated based on content items obtained from sources other than web pages (e.g., mobile applications, large data files (e.g., downloaded videos or books), and / or data from other sources).

[0033] The systems and methods can include providing a particular portion of a web page for display in response to an interaction with the snippet packet. For example, the systems and methods can include obtaining input data. The input data can describe a selection of a content item associated with the snippet packet. Address data and location data associated with the snippet packet can be obtained. The address data can be associated with a web page. The content item can be associated with the web page. Additionally and / or alternatively, the location data can describe a location of the content item within the web page. The systems and methods can obtain web page data. The web page data can be obtained based at least in part on the address data. A location within the web page associated with the content item can be determined. A portion of the web page can then be provided. The portion of the web page can include the location of the content item.

[0034] The system and method can obtain input data. In some implementations, the input data can describe a selection of a content item associated with the snippet packet. The snippet packet can be associated with a user account for a particular user. Additionally and / or alternatively, the user account can be associated with one or more platforms. The snippet packet can include a content item and a deep link. Alternatively and / or additionally, the snippet packet can include a snippet (e.g., a content item and / or media data generated based on the content item (e.g., a graphics card and / or a summary of the content item)), address data (e.g., a uniform resource locator and / or a uniform resource identifier), and / or metadata. The snippet packet can include location data (e.g., metadata indicating the location of the content item within a web page, a text fragment for identifying start and end data, one or more pointers, and / or one or more scroll position data).

[0035] In some embodiments, the snippet packet may be generated by providing a graphical user interface for display. The graphical user interface may include a graphical window for displaying a web page. In some embodiments, the web page may include a plurality of content items. The snippet packet generation may include obtaining selection data. The selection data may include a request to save a content item of the plurality of content items. In some embodiments, the snippet packet generation may include generating a snippet packet and storing the snippet packet in a user database.

[0036] The systems and methods can obtain address data and location data associated with the snippet packet based on the input data. The address data can be associated with a web page. The content item can be associated with the web page. Additionally and / or alternatively, the location data can describe a location of the content item within the web page.

[0037] Web page data can then be retrieved. The web page data can be retrieved based at least in part on the address data (e.g., by navigating to the web page using a uniform resource locator). Alternatively and / or additionally, a file can be retrieved based on the address data (e.g., the address data can describe a file location and can be utilized to retrieve that particular file).

[0038] A location within a web page (and / or file) associated with a content item can be determined based on the obtained location data. The location may be determined based on a text fragment, one or more pointers, and / or via web page processing.

[0039] In some implementations, the address data can include a uniform resource locator. Additionally and / or alternatively, the location data can include one or more text fragments. Determining a location within a web page associated with the content item can then include adding the text fragment to the uniform resource locator to generate a shortcut link and entering the shortcut link into a browser.

[0040] A portion of the web page can then be provided for display. The portion of the web page can include a location of the content item. In some implementations, providing the portion of the web page can include providing one or more indicators in the portion of the web page. The one or more indicators can indicate the content item associated with the snippet packet. In some implementations, the one or more indicators can include highlighting text associated with the content item.

[0041] The systems and methods can include processing gesture data. For example, the systems and methods can include providing a graphical user interface. The graphical user interface can include a graphical window for displaying a web page. In some implementations, the web page can include multiple content items. The systems and methods can receive gesture data. The gesture data can describe a gesture associated with a portion of the web page. The gesture data can be processed to determine a selected content item. The selected content item can be associated with the portion of the web page. A snippet packet can be generated based on the gesture data. In some implementations, the snippet packet can include the selected content item.

[0042] In particular, a graphical user interface can be provided for display. The graphical user interface can include a graphical window for displaying a web page. In some implementations, the web page can include multiple content items. The web page can be associated with a uniform resource locator and / or source code. The multiple content items can include structured text data (e.g., body paragraphs and / or one or more titles), white space, one or more images, and / or audio content items.

[0043] Gesture data can then be received. The gesture data can describe a gesture associated with a portion of the web page. In some implementations, the gesture can include a circular gesture that encircles the portion of the web page. The gesture data can describe touch input to a touchscreen display of the mobile computing device.

[0044] The gesture data can be processed to determine a selected content item. The selected content item can be associated with a portion of a web page. For example, the gesture can encircle one or more lines of an image and text in a web page that includes multiple lines and / or multiple images.

[0045] In some implementations, processing the gesture data to determine the selected content item may include determining a portion of the web page enclosed by the gesture, determining a focus of the portion, and determining that the selected content item is associated with the focus of the portion.

[0046] Alternatively and / or additionally, processing the gesture data to determine the selected content items may include processing the gestured data with a machine-learned model to determine the selected content items. The machine-learned model may be trained to determine the beginning and end of the selected content items based on proximity to gesture boundaries, syntax, structural data, white space, and / or semantic cohesion.

[0047] In some implementations, the selected content item can be determined based on a determined matching area associated with a rectangle determined based on the circle. The rectangle can be a rectangle of data based on a syntactic configuration of the web page. In some implementations, the selected content item can be determined based on determined word boundaries and / or determined media content boundaries. The determination can be based on hypertext markup language code boundaries. In some implementations, the source code of the web page can be analyzed and the analyzed data can be processed.

[0048] Alternatively and / or additionally, the determining may include calculating an area of ​​a gesture rectangle associated with the gesture. The area of ​​one or more content item elements may be determined. The area of ​​intersection of the area of ​​the gesture rectangle with each area of ​​the different content item elements may be determined. The element with the highest degree of intersection may have the highest probability of being selected and may therefore be determined as the selected content item.

[0049] A snippet packet can then be generated based on the gesture data. The snippet packet can include the selected content item. In some implementations, the snippet packet can include address data and location data. The address data can be associated with a web page, and the location data may describe the location of the content item within the web page.

[0050] A save interface can then be provided for display. The save interface can provide interactive interface elements that can be selected to add the snippet packet to a collection. A collection interface can then be provided. A drag input can then be received to drag a graphical representation of the snippet packet to a graphical tile that describes a particular collection. The snippet packet can then be stored in the collection (e.g., the snippet packet can be stored with a relationship tag that links the snippet packet to a particular collection). In some implementations, the graphical representation of the snippet packet can change size and / or proportion as the graphical representation is dragged to a particular collection. The change in size and proportion can provide an intuitive indication of collection addition while providing an aesthetically pleasing display.

[0051] In some implementations, a content item can be processed to determine entities associated with the content item. The entities can be determined by processing the content item (e.g., text data, image data, audio data, video data, latent coding data, and / or link data) with a machine-learned model (e.g., an image classification model, an object classification model, a text classification model (e.g., a natural language processing model), a segmentation model, a semantic model, and / or a detection model) to generate entity data (e.g., a classification). Relationship data can then be generated and added to the snippet packet based on the entity data. The relationship data can include one or more entity tags and / or references to other related snippet packets and / or related web pages or content items.

[0052] The snippet packets may be searchable within one or more applications and / or databases. Additionally and / or alternatively, the snippet packets may be shareable via messaging applications, social media applications, and / or other applications.

[0053] In some implementations, content items can be processed with one or more machine-learned models to generate tags that can be stored in snippet packets. The tags can then be utilized as searchable tags to surface snippet packets in response to search queries. Tags can be determined based on the content of the content item (e.g., recognized words, recognized objects in an image, characteristics of a video frame or audio stream, etc.).

[0054] In some implementations, snippet packet generation occurs after selection of a snippet packet generation interface element that can open a snippet packet generation interface. Alternatively and / or additionally, a user can select a content item and one or more (e.g., two) popups can be provided with various options for interacting with the content item, one of the options can include snippet packet generation. Alternatively and / or additionally, search results associated with the content item can be provided.

[0055] In some implementations, a screenshot request can be received. A screenshot can be generated and uploaded to a new client, and a token can be generated. The token can be utilized to receive data associated with the screenshot. The token, screenshot, and screenshot details can be utilized to generate a snippet packet.

[0056] Systems and methods can be implemented to generate snippet packets based on other data sources, not just web pages. For example, systems and methods can be used to generate snippet packets based on content items in data files stored locally and / or on a server computing system. The generated snippet packets can include the content item (and / or graphics card), address data, and location data. The address data can describe where the data file is stored (e.g., the name of the drive and any folder names (e.g., G:\ResearchPapers\Quantum\Spin)). The location data can describe where the content item is located within the data file. The location data can include start and end data that can be used to find matching data within the data file, which can then be navigated to and highlighted. Alternatively and / or additionally, the location data can include one or more pointers.

[0057] The systems and methods may allow user selection of a subset, excerpt, and / or portion of an object (e.g., text, an image, and / or a video that may be part of a larger web page). In some implementations, the systems and methods may segment a portion of text from a larger body of text, may segment a portion of an image, may separate frames within a video, and / or may segment a portion of an audio file.

[0058] In some implementations, the systems and methods may be utilized as an extension to a browser application and / or as functionality that sits on top of other applications, and snippet packet generation may be utilized for content displayed in a variety of different application types. For example, the systems and methods may be incorporated into a computing device's operating system to enable snippet packet generation to occur in response to selections made in a variety of different applications (e.g., a map application, a browser application, a social media application, etc.).

[0059] Additionally and / or alternatively, the systems and methods may be utilized by a variety of different types of computing devices. For example, the systems and methods may be utilized by mobile computing devices, desktop computing devices, smart wearables (e.g., smart glasses), and / or other computing devices. The systems and methods may also be utilized in virtual reality and augmented reality interfaces.

[0060] In some implementations, a snippet packet may include user context data describing the user's context when the snippet packet was generated. For example, a computing device may include multiple sensors that can collect data regarding the user's context. In some implementations, physical location data of the user's computing device may be obtained and stored in the snippet packet to provide further context to the snippet. The physical location data may be provided within a graphics card and / or may be provided as an optional data set that is viewable during snippet packet interaction.

[0061] The systems and methods can store snippet packets locally on a user's device and / or may store snippet packets on a server computing system. Local storage of snippet packets can be utilized to ensure that the snippet packets remain private to the user, and offline access can be provided. Additionally, metadata associated with the collection and generation of snippet packets can be kept private and secure.

[0062] The systems and methods may be provided via browser extensions, overlay applications, and / or embedded application functionality. The systems and methods may be utilized on mobile devices, desktop devices, smart wearables, and / or other computing devices.

[0063] The snippet packet may include other metadata associated with the selected portion, web page, and / or one or more contexts of the user (e.g., the application in use, the time of day, the user's geographic location, and / or user profile data).

[0064] The systems and methods may be executed on a server computing system. Alternatively and / or additionally, the systems and methods may be executed locally on a user computing device. In some implementations, the user computing device may be communicatively connected over a network to transmit data and perform cloud-based computing. The snippet packets may be stored locally and / or on a server.

[0065] The systems and methods of the present disclosure provide several technical effects and advantages. As an example, the systems and methods can generate and store snippet packets. In particular, the systems and methods disclosed herein can obtain input data, determine content items (e.g., text, images, video, and / or audio) associated with the input data, generate snippet packets, and store the snippet packets. The snippet packets can include a graphical representation of the content item that, when selected, can direct a user to a portion of a web page from which the content item originated. The generation and storage of snippet packets can enable easy access to stored content while maintaining links to more context about the content item.

[0066] Another technical advantage of the systems and methods of the present disclosure is that snippet packets can be utilized to share hierarchical level information with relatively low transmission costs. For example, the systems and methods can generate snippet packets. The snippet packets can be shared with a second user who can first view the content item. The second user can then select the content item, navigate to a web page, and be routed to a specific portion of the web page from which the content item originated, allowing the second user to obtain more context about the content item. Because the content item, web address, and text fragment can be transmitted, providing hierarchical information can be completed with relatively low transmission costs. The second user can interact with the snippet packets, view the content item in isolation, and then select the snippet packets to use the web address in combination with the text fragment to navigate to a portion of the web page where the content item was highlighted or otherwise indicated. Highlighting and transmitting an entire web page file may involve even more uploading and downloading during transmission.

[0067] Other exemplary technical effects and advantages relate to improved computational efficiency and improved functionality of computing systems. For example, the systems and methods disclosed herein can leverage snippet buckets to reduce the amount of data stored to store content items and associated web page context. In particular, snippet packets may include compressed versions of content items, web addresses, and text fragments, instead of storing compressed versions of entire web pages, which may contain large amounts of content items and embedded data. Furthermore, searching a collection of snippet packets can be less computationally intensive than searching multiple compressed web pages.

[0068] Referring now to the drawings, exemplary embodiments of the present disclosure will be described in more detail.

[0069] Exemplary Devices and Systems 1A illustrates a block diagram of an exemplary computing system 100 for performing snippet packet generation in accordance with an exemplary embodiment of the present disclosure. System 100 includes a user computing device 102, a server computing system 130, and a training computing system 150, communicatively connected via a network 180.

[0070] The user computing device 102 can be any type of computing device, such as, for example, a personal computing device (e.g., a laptop or desktop), a mobile computing device (e.g., a smartphone or tablet), a gaming console or controller, a wearable computing device, an embedded computing device, or any other type of computing device.

[0071] The user computing device 102 includes one or more processors 112 and memory 114. The one or more processors 112 can be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and can be a single processor or multiple operatively connected processors. The memory 114 can include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. The memory 114 can store data 116 and instructions 118 that are executed by the processor 112 to cause the user computing device 102 to perform operations.

[0072] In some implementations, the user computing device 102 can store or include one or more packet generation modules 120. For example, the packet generation modules 120 can be or otherwise include various machine-learned models, such as neural networks (e.g., deep neural networks) or other types of machine-learned models including nonlinear and / or linear models. The neural networks can include feed-forward neural networks, recurrent neural networks (e.g., long short-term memory recurrent neural networks), convolutional neural networks, or other types of neural networks. Exemplary packet generation models 120 are described with reference to FIGS. 2-4.

[0073] In some implementations, one or more packet generation models 120 may be received from server computing system 130 over network 180, stored in user computing device memory 114, and then used or otherwise implemented by one or more processors 112. In some implementations, user computing device 102 may implement multiple parallel instances of a single packet generation model 120 (e.g., to perform parallel snippet packet generation across multiple instances of snippet selection).

[0074] More specifically, the packet generation model may receive input data, determine one or more selected content items, generate snippet packets, and / or determine one or more tags. In some implementations, the packet generation model may process the selected content items and generate summaries that are added to the snippet packets.

[0075] Additionally and / or alternatively, one or more packet generation models 140 may be included in or otherwise stored and implemented by a server computing system 130 that communicates with the user computing device 102 according to a client-server relationship. For example, the packet generation module 140 may be implemented by the server computing system 140 as part of a web service (e.g., a snippet packet generation service). Thus, one or more models 120 may be stored and implemented at the user computing device 102 and / or one or more models 140 may be stored and implemented at the server computing system 130.

[0076] The user computing device 102 may also include one or more user input components 122 that receive user input. For example, the user input component 122 may be a touch-sensitive component (e.g., a touch-sensitive display screen or touchpad) that is sensitive to the touch of a user input object (e.g., a finger or stylus). The touch-sensitive component may serve to implement a virtual keyboard. Other exemplary user input components include a microphone, a traditional keyboard, or other means by which a user can provide user input.

[0077] The server computing system 130 includes one or more processors 132 and memory 134. The one or more processors 132 can be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and can be a single processor or multiple operably connected processors. The memory 134 can include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. The memory 134 can store data 136 and instructions 138 that are executed by the processor 132 to cause the server computing system 130 to perform operations.

[0078] In some implementations, server computing system 130 includes or is otherwise implemented by one or more server computing devices. If server computing system 130 includes multiple server computing devices, such server computing devices may operate according to a sequential computing architecture, a parallel computing architecture, or any combination thereof.

[0079] As described above, the server computing system 130 can store or otherwise include one or more machine-learned packet generation models 140. For example, the models 140 can be or otherwise include various machine-learned models. Exemplary machine-learned models include neural networks or other multi-layer nonlinear models. Exemplary neural networks include feedforward neural networks, deep neural networks, recurrent neural networks, and convolutional neural networks. Exemplary models 140 are described with reference to FIGS. 2-4.

[0080] The user computing device 102 and / or the server computing system 130 can train the models 120 and / or 140 by interacting with a training computing system 150 that is communicatively connected via a network 180. The training computing system 150 can be separate from the server computing system 130 or can be part of the server computing system 130.

[0081] Training computing system 150 includes one or more processors 152 and memory 154. The one or more processors 152 can be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and can be a single processor or multiple operably connected processors. Memory 154 can include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. Memory 154 can store data 156 and instructions 158 that are executed by processor 152 to cause training computing system 150 to perform operations. In some implementations, training computing system 150 includes or is otherwise implemented by one or more server computing devices.

[0082] The training computing system 150 may include a model trainer 160 that trains the machine-learned models 120 and / or 140 stored on the user computing device 102 and / or the server computing system 130 using various training or learning techniques, such as, for example, backpropagation. For example, a loss function may be backpropagated through the model(s) to update one or more parameters of the model(s) (e.g., based on the gradient of the loss function). Various loss functions may be used, such as mean squared error, likelihood loss, cross-entropy loss, hinge loss, and / or various other loss functions. Gradient descent techniques may be used to iteratively update parameters over several training iterations.

[0083] In some implementations, performing backpropagation may include performing truncated backpropagation through time. Model trainer 160 may implement several generalization techniques (e.g., weight decay, dropout, etc.) to improve the generalization ability of the trained model.

[0084] In particular, model trainer 160 can train packet generation models 120 and / or 140 based on a set of training data 162. Training data 162 can include, for example, training inputs (e.g., training gestures), training web pages, training text data, training image data, ground truth graphics cards, ground truth snippet packets (e.g., ground truth snippets, ground truth address data, and / or ground truth location data), and / or ground truth entity labels.

[0085] In some implementations, if the user consents, training examples may be provided by the user computing device 102. Thus, in such implementations, the model 120 provided to the user computing device 102 may be trained by the training computing system 150 with user-specific data received from the user computing device 102. In some cases, this process may be referred to as personalizing the model.

[0086] Model trainer 160 includes computer logic utilized to provide desired functionality. Model trainer 160 can be implemented in hardware, firmware, and / or software controlling a general-purpose processor. For example, in some embodiments, model trainer 160 includes program files stored on a storage device, loaded into memory, and executed by one or more processors. In other embodiments, model trainer 160 includes one or more sets of computer-executable instructions stored on a tangible computer-readable storage medium, such as RAM, a hard disk, or an optical or magnetic medium.

[0087] Network 180 can be any type of communications network, such as a local area network (e.g., an intranet), a wide area network (e.g., the Internet), or any combination thereof, and can include any number of wired or wireless links. Generally, communications over network 180 can occur over any type of wired and / or wireless connection, using a wide variety of communications protocols (e.g., TCP / IP, HTTP, SMTP, FTP), encodings or formats (e.g., HTML, XML), and / or security schemes (e.g., VPN, Secure HTTP, SSL).

[0088] The machine-learned models described herein may be used in a variety of tasks, applications, and / or use cases.

[0089] In some implementations, the input to the machine-learned model(s) of the present disclosure can be image data. The machine-learned model(s) can process the image data to generate an output. As an example, the machine-learned model(s) can process the image data to generate an image recognition output (e.g., a recognition of the image data, a latent embedding of the image data, an encoded representation of the image data, a hash of the image data, etc.). As another example, the machine-learned model(s) can process the image data to generate an image segmentation output. As another example, the machine-learned model(s) can process the image data to generate an image classification output. As another example, the machine-learned model(s) can process the image data to generate an image data modification output (e.g., an alteration of the image data, etc.). As another example, the machine-learned model(s) can process the image data to generate an encoded image data output (e.g., an encoded and / or compressed representation of the image data, etc.). As another example, the machine-learned model(s) can process the image data to generate an upscaled image data output. As another example, the machine-learned model(s) can process the image data to generate a prediction output.

[0090] In some implementations, input to the machine-learned model(s) of the present disclosure can be text or natural language data. The machine-learned model(s) can process the text or natural language data to generate an output. As another example, the machine-learned model(s) can process the natural language data to generate a language-encoded output. As another example, the machine-learned model(s) can process the text or natural language data to generate a latent text embedding output. As another example, the machine-learned model(s) can process the text or natural language data to generate a translation output. As another example, the machine-learned model(s) can process the text or natural language data to generate a classification output. As another example, the machine-learned model(s) can process the text or natural language data to generate a text segmentation output. As another example, the machine-learned model(s) can process the text or natural language data to generate a semantic intent output. As another example, the machine-learned model(s) may process text or natural language data to generate upscaled text or natural language output (e.g., text or natural language data of higher quality than the input text or natural language). As another example, the machine-learned model(s) may process text or natural language data to generate predicted outputs.

[0091] In some implementations, the input to the machine-learned model(s) of the present disclosure can be audio data. The machine-learned model(s) can process the audio data to generate an output. As an example, the machine-learned model(s) can process the audio data to generate a speech recognition output. As another example, the machine-learned model(s) can process the audio data to generate a speech translation output. As another example, the machine-learned model(s) can process the audio data to generate a latent embedding output. As another example, the machine-learned model(s) can process the audio data to generate an encoded audio output (e.g., an encoded and / or compressed representation of the audio data, etc.). As another example, the machine-learned model(s) can process the audio data to generate an upscaled audio output (e.g., audio data of higher quality than the input audio data, etc.). As another example, the machine-learned model(s) can process the audio data to generate a text representation output (e.g., a text representation of the input audio data, etc.). As another example, the machine-learned model(s) can process the audio data to generate a predicted output.

[0092] In some implementations, input to the machine-learned model(s) of the present disclosure can be latent-coded data (e.g., a latent space representation of the input, etc.). The machine-learned model(s) can process the latent-coded data to generate an output. As an example, the machine-learned model(s) can process the latent-coded data to generate a recognition output. As another example, the machine-learned model(s) can process the latent-coded data to generate a reconstruction output. As another example, the machine-learned model(s) can process the latent-coded data to generate a search output. As another example, the machine-learned model(s) can process the latent-coded data to generate a reclustering output. As another example, the machine-learned model(s) can process the latent-coded data to generate a prediction output.

[0093] In some implementations, input to the machine-learned model(s) of the present disclosure can be statistical data. The machine-learned model(s) can process the statistical data to generate an output. As an example, the machine-learned model(s) can process the statistical data to generate a recognition output. As another example, the machine-learned model(s) can process the statistical data to generate a prediction output. As another example, the machine-learned model(s) can process the statistical data to generate a classification output. As another example, the machine-learned model(s) can process the statistical data to generate a segmentation output. As another example, the machine-learned model(s) can process the statistical data to generate a segmentation output. As another example, the machine-learned model(s) can process the statistical data to generate a visualization output. As another example, the machine-learned model(s) can process the statistical data to generate a diagnostic output.

[0094] In some cases, the machine learning model(s) can be configured to perform a task that includes encoding input data for reliable and / or efficient transmission or storage (and / or corresponding decoding). For example, the task can be an audio compression task. The input can include audio data, and the output can include compressed audio data. In another example, the input can include visual data (e.g., one or more images or videos), and the output can include compressed visual data, and the task is a visual data compression task. In another example, the task can include generating an embedding for the input data (e.g., input audio or visual data).

[0095] In some cases, the input includes visual data and the task is a computer vision task. In some cases, the input includes pixel data of one or more images and the task is an image processing task. For example, the image processing task can be image classification, and the output is a set of scores, each score corresponding to a different object class and representing the likelihood that one or more images show an object belonging to the object class. The image processing task can be object detection, and the image processing output identifies one or more regions in one or more images and, for each region, the likelihood that the region shows an object of interest. As another example, the image processing task can be image segmentation, and the image processing output defines, for each pixel of one or more images, a respective likelihood for each category of a set of predetermined categories. For example, the set of categories can be foreground and background. As another example, the set of categories can be object classes. As another example, the image processing task can be depth estimation, and the image processing output defines, for each pixel of one or more images, a respective depth value. As another example, the image processing task can be motion estimation, where the network input includes multiple images and the image processing output defines, for each pixel of one of the input images, the motion of the scene depicted in pixels between the images in the network input.

[0096] In some cases, the tasks include encrypting or decrypting input data. In some cases, the tasks include microprocessor performance tasks such as branch prediction or memory address translation.

[0097] 1A illustrates one exemplary computing system that can be used to implement the present disclosure. Other computing systems can be used as well. For example, in some implementations, a user computing device 102 can include a model trainer 160 and a training dataset 162. In such implementations, the model 120 can be both trained and used locally on the user computing device 102. In some such implementations, the user computing device 102 can implement the model trainer 160 to personalize the model 120 based on user-specific data.

[0098] 1B illustrates a block diagram of an exemplary computing device 10 for performing according to an exemplary embodiment of the present disclosure. The computing device 10 can be a user computing device or a server computing device.

[0099] Computing device 10 includes several applications (e.g., applications 1-N). Each application includes its own machine learning library and machine-learned model(s). For example, each application may include a machine-learned model. Exemplary applications include a text messaging application, an email application, a dictation application, a virtual keyboard application, a browser application, etc.

[0100] 1B , each application may communicate with several other components of the computing device, such as, for example, one or more sensors, a context manager, a device state component, and / or additional components. In some implementations, each application may communicate with each device component using an API (e.g., a public API). In some implementations, the API used by each application is specific to that application.

[0101] 1C depicts a block diagram of an exemplary computing device 50 for performing according to an exemplary embodiment of the present disclosure. Computing device 50 can be a user computing device or a server computing device.

[0102] Computing device 50 includes several applications (e.g., applications 1-N). Each application communicates with a central intelligence layer. Exemplary applications include a text messaging application, an email application, a dictation application, a virtual keyboard application, a browser application, etc. In some implementations, each application can communicate with the central intelligence layer (and the model(s) stored therein) using an API (e.g., a common API across all applications).

[0103] The central intelligence layer includes several machine-learned models. For example, as shown in FIG. 1C , each machine-learned model (e.g., model) can be provided for each application and managed by the central intelligence layer. In other embodiments, two or more applications can share a single machine-learned model. For example, in some embodiments, the central intelligence layer can provide a single model (e.g., single model) for all of the applications. In some embodiments, the central intelligence layer is included within or otherwise implemented by the operating system of computing device 50.

[0104] The central intelligence layer may communicate with a central device data layer. The central device data layer may be a centralized repository of data for the computing device 50. As shown in FIG. 1C , the central device data layer may communicate with several other components of the computing device, such as, for example, one or more sensors, a context manager, a device state component, and / or additional components. In some implementations, the central device data layer may communicate with each device component using an API (e.g., a private API).

[0105] Model layout example 2 illustrates a diagram of an exemplary snippet packet generation interface 200, according to an exemplary embodiment of the present disclosure. The snippet packet generation interface can be configured to receive a text selection, an image selection, a screenshot, a video selection, and / or an audio selection. An options menu can then pop up with multiple options, including a save option. The save option can be selected, which can prompt a graphics card to generate and provide for display. A snippet packet can be generated. The user can then select which collection to save the snippet packet to.

[0106] For example, at 202, a display window of the web page is provided for viewing, a portion of the web page including a textual content item is selected, and a save user interface element is selected. At 204, a snippet packet is generated, and the generated graphical representation (e.g., a graphics card with the portion of text with its title and location indicated) is provided for viewing. Further, at 204, the user interface includes a pop-up window to provide options for adding to a collection, which may include adding to an existing collection and / or creating a new collection. Collections may be automatically generated and / or manually generated.

[0107] At 206, a portion of the search results of the web page may be selected (e.g., a portion of the knowledge graph may be selected), and a save option may be selected. At 208, a graphics card is provided for display, the graphics card being generated based on the selected portion of the search results page. For example, the graphics card may include a picture associated with the selected portion as a background, with selected text displayed in the foreground. Additionally and / or alternatively, the graphics card may include the search query and associated search engines. An add collection option may be provided.

[0108] A screenshot input is received and a save option is provided for display at 210. A save input is received and the generated graphical display and an add to collection option are provided for display at 212. The graphical display can include at least a portion of the screenshot and a banner including source information (e.g., the title of the web page, an entity associated with the screenshot information, and / or the source of the screenshot information).

[0109] The generated snippet packet can be stored along with the generated graphical representation, and the user can then select the snippet packet to view an enlarged graphical representation, view the saved content item(s), and / or navigate to a particular point on the web page where the content item(s) in the resource resides.

[0110] 3 illustrates an exemplary gesture interaction 300, according to an exemplary embodiment of the present disclosure. For example, a user input requesting a clipping interface can be received (e.g., selection of a "clip" user interface element 302). A snippet packet generation interface can be provided, and a graphic animation 304 can be provided to demonstrate how to select a content item. The graphic animation can demonstrate that a swipe gesture in a circular motion can be utilized to select a content item. An instruction user interface element 306 can be provided as a pop-up to provide instructions on how to clip and save.

[0111] FIG. 4 illustrates a diagram of an exemplary snippet packet generation interface 400, according to an exemplary embodiment of the present disclosure. In some implementations, a snippet packet generation interface can be requested and provided based on received user input. The snippet packet generation interface can include clipping or summarization options, which can enable generation of snippet packets based on a selected content item. The snippet packet generation interface 400 can include a web page display window 402 and an interaction pane 404, which can be provided in response to a web page interaction and can provide multiple web page interaction options for display. The interaction options can be predetermined and uniform across the web page. Alternatively and / or additionally, the interaction options can vary based on the content of the web page, based on the selected content item, and / or based on an entity associated with the web page. Interaction options may include adding to a collection, summarizing a selected portion and / or the entire web page, clipping a portion of the web page, saving an image, finding more web pages like the current web page, comparing the products on the web page with other similar products, and / or tracking pricing options.

[0112] FIG. 5 illustrates a diagram of an exemplary snippet packet generation and collection addition interface 500, according to an exemplary embodiment of the present disclosure. The systems and methods disclosed herein can be utilized to generate and store snippet packets. The snippet packets can be added to a collection, and the snippet packets may be searchable for later use. The snippet packets can include generated graphics cards. In some implementations, a selected content item can be processed by one or more machine-learned models to generate a summary of the content item, which can be used to generate the graphics card. The graphics card can be customizable. The graphics card can include a background color and / or background image determined based on the source web page. Alternatively and / or additionally, the background color and / or background image can be selected by a user. Similarly, the font can be automatically determined, predetermined, and / or selected by a user.

[0113] At 502, a snippet packet (including a graphical representation) is generated based on the selected portion of the web page. The generated snippet packet is then added to an "Inspo" collection of media content items and snippet packets.

[0114] At 504, a summary is generated for the portion of the web page and a graphic card is generated based on the semantic understanding and / or the determined entities. The generated graphic card can be saved as part of the generated snippet packet and added to a collection. A collection may be associated with a particular entity and / or a particular type of entity.

[0115] At 506, different options for sharing and / or customizing the generated snippet packet are provided for display. For example, the template, text, and / or background of the graphical display can be customized. Sharing options can include adding to a collection, adding to a note, sending via text message, sending via email, copying, and / or airdropping.

[0116] The snippet packet may be published to social media and / or may be published to a localized search interface at 508. For example, a user may utilize a search application, which may surface multiple web search results in response to a query and / or may surface one or more generated snippet packets in response to the query (e.g., a generated snippet packet for a particular user and / or generated snippet packets for related users (e.g., friends or users close to the particular user)).

[0117] The generated snippet packets can then be shared via messaging applications, social media applications, and / or various other methods. In some implementations, the snippet packets can be published to the web and may be utilized as a new format for web search results.

[0118] 9 illustrates a diagram of an exemplary add collection interface 900, according to an exemplary embodiment of the present disclosure. The add collection interface can provide a graphical representation (e.g., a graphics card) of content items and / or snippet packets for display. In response to a storage element being selected, multiple collections can be provided for display. A user can then add snippet packets to a particular collection. The snippet packets can then be stored in the particular collection.

[0119] The particular collection can then be opened, and a graphical representation of the generated snippet packet can be provided for display along with other graphical representations associated with other snippet packets. The collection addition interface 900 can include displaying a graphics card (at a first size) for display during snippet packet generation 902. A pop-up window 904 for adding to the collection can then be provided for display upon selection of the user interface element. When a snippet packet is added to a particular collection, a collection 906 can be provided for display along with multiple thumbnails describing different snippet packets in the collection, including a thumbnail describing the generated graphics card (at a second size).

[0120] FIG. 10A shows a diagram of an exemplary snippet packet interaction 1020, according to an exemplary embodiment of the present disclosure. Once a snippet packet is generated, the snippet packet can be interacted with to navigate to a web page associated with a content item. Location data in the snippet packet can be utilized to navigate to a specific portion of the web page that contains the source of the content item. The content item can be highlighted when displayed. For example, a graphics card 1022 can be selected. Address data and location data of the snippet packet associated with the graphics card can then be obtained. The address data and location data can then be utilized to open the web page 1024 to the exact location of the content item(s) in the selected snippet packet that highlighted the content item(s).

[0121] 10B shows a diagram of an example snippet packet search 1040, according to an example embodiment of the present disclosure. The generated snippet packets can be provided as search results when searching locally and / or when searching the web. For example, a user can enter a search query, the search query can be processed, a number of suggested queries and a number of suggested snippet packets can be determined, and the snippet packets can be provided for display (e.g., as shown at 1042) as further input can be received.

[0122] 10C shows a diagram of an exemplary highlighting interface 1080, according to an exemplary embodiment of the present disclosure. In some implementations, a highlighting (e.g., as shown at 1082) can be removed (e.g., as shown at 1084) in response to receiving user input requesting removal of the highlighting.

[0123] FIG. 11 illustrates a diagram of an exemplary graphics card 1100, according to an exemplary embodiment of the present disclosure. The graphics card may be automatically generated and / or generated based on one or more user inputs. The graphics card may include a portion assigned to a content item and / or a portion assigned to attribution to the source of the content item. The graphics card may include text, image(s), video(s), and / or audio. The graphics card may vary based on the content item selected. The text may be selected text and / or a summary of data retrieved from a web page. In some implementations, the background and / or image may be automatically selected / generated and / or selected by the user. In particular, FIG. 11 illustrates two formats of a text-based graphics card 1102 with resource attribution and associated title / search query: an image-based graphics card 1104 with an image, entity thumbnail, title or caption, and resource attribution; and a screenshot-based graphics card 1106 with a screenshot, entity logo, title of each article, and resource attribution.

[0124] FIG. 12 shows a diagram of an exemplary summary snippet packet generation interface 1200, according to an exemplary embodiment of the present disclosure. Snippet packet generation can include generating a graphics card. The graphics card can include text describing a summary of selected data within a web page. The summary can be generated by processing the selected data with a machine-learned model. For example, at 1202, a portion of a web page is selected. A user can select a summary user interface element to process the selected portion and generate a summary. The summary can include words from the passage and / or different words that summarize the passage in simpler terms or user-specific terms. At 1204, the summarized text is used to generate a graphics card that is stored with the generated snippet packet, which includes information associated with the web page and a specific location of the selected portion within the web page.

[0125] 13 illustrates a diagram of an exemplary snippet packet suggestion interface 1300, according to an exemplary embodiment of the present disclosure. In some implementations, the snippet packet suggestion interface may include providing a suggested graphic card associated with the suggested snippet packet (e.g., may be displayed in response to selecting a bookmark option, such as shown at 1302, as shown at 1304). The suggested snippet packet may be based on past user interactions by a particular user and / or based on past interactions by other users (e.g., popular snippets). The suggested snippet packet may be selected and stored locally by a user computing system.

[0126] FIG. 14 shows a diagram of an exemplary snippet packet generation and sharing interaction 1400, according to an exemplary embodiment of the present disclosure. A snippet packet can be generated in response to a user selection of a content item (e.g., selection at 1402). The generated snippet packet can then be shared (e.g., shared via a text message option of a sharing interface 1404, which provides multiple sharing options). The snippet packet can be shared via a messaging application, a social media application, and / or inserted into other data files (e.g., notes, text documents, and / or slide decks). The shared snippet packet can be sent to a graphics card (e.g., as shown at 1406) with a download option for local storage. Alternatively and / or additionally, the graphics card may be selectable to navigate to one or more content items of the snippet packet in a native web page.

[0127] FIG. 15 shows a diagram of an example graphics card customization interface 1500, according to an example embodiment of the present disclosure. The graphics card customization interface can include multiple templates from which to select to customize the graphics card. The templates can include different images, different colors, different fonts, and / or different layouts. For example, an initial graphics card 1502 can include text in a first font, text in a first size, and a first background. A change template request can be received, and an enhanced graphics card 1504 can be generated with a different text font, a different text size, and / or a different background.

[0128] FIG. 16 shows a block diagram of an exemplary snippet packet generation system 1600, according to an exemplary embodiment of the present disclosure. The snippet packet generation system 1600 may include obtaining input data 1602 (e.g., input data describing a selection of one or more content items). The input data (1602) may be processed to determine the selected content items (1604). Based on the determined selection, the content items may be obtained, and a graphics card may be generated (1606). Address data may be generated and / or obtained (1608). The address data may include uniform resource locator data (1608). Location data may be generated and / or obtained (1610). The location data may include text fragment data (1610), which may include a scroll position, a start of the content item, and an end of the content item, which may be utilized to locate and highlight the content item within a source page. The graphics card (1606), the content items, the address data (1608), and the location data (1610) may be utilized to generate a snippet packet (1612). The snippet packet 1612 may be processed to determine 1614 one or more entity tags for the snippet packet 1612 based on the content item and / or based on the source of the content item. The entity tag 1614 may include relationship data linking the snippet packet 1612 with other snippet packets associated with the same entity. The snippet packet 1612 with the entity tag(s) 1614 may then be stored 1616 (e.g., locally and / or on a server computing system).

[0129] One or more of the determining and / or generating may be performed based at least in part on one or more machine-learned models. For example, determining the selected content item (1604), acquiring the content item and / or generating the graphics card (1606), generating the location data (1610), and / or determining the entity tag (1614) may be performed by one or more machine-learned models.

[0130] Exemplary Methods 6 shows a flowchart diagram of an exemplary method for performing in accordance with an exemplary embodiment of the present disclosure. While FIG. 6 shows steps performed in a particular order for purposes of illustration and explanation, the methods of the present disclosure are not limited to the particularly shown order or arrangement. Various steps of method 600 can be omitted, rearranged, combined, and / or adapted in various ways without departing from the scope of the present disclosure.

[0131] At 602, a computing system can provide data describing a graphical user interface. The graphical user interface can include a graphical window for displaying a web page. In some implementations, the web page can include multiple content items.

[0132] At 604, the computing system may obtain input data. The input data may include a request to store one or more content items of the plurality of content items. In some implementations, the one or more content items may include at least one of an image, a video, a graphical depiction of a product, or audio.

[0133] At 606, the computing system can generate a snippet packet. The snippet packet can be generated based on the input data. The snippet packet can include one or more content items, address data, and location data.

[0134] The one or more content items may include text data (e.g., text data describing a word, a sentence, a quote, and / or a paragraph), image data (e.g., an image of an object (e.g., a product)), video data (e.g., a video and / or one or more frames of a video), audio data (e.g., waveform data), and / or latent coded data. The one or more content items may include multimodal data.

[0135] In some implementations, generating a snippet packet may include obtaining one or more content items and generating a graphics card. The graphics card may describe the one or more content items. The graphics card may include color and / or text data overlaid on an image. The color may be determined based on a predominant color of the web page. In some implementations, the color may be predetermined and / or determined based on surrounding content items. The image may be an image determined based on a determined topic of the content item. Alternatively and / or additionally, the image may be an image close to the content item. In some implementations, the graphics card may include a font determined based on a font used on the web page. The graphics card may include a background, text describing the content item (e.g., the content item and / or a summary of the content item), and / or text and / or logos describing the source and / or the determined entity. In some implementations, the text size of the text in the graphics card may be based on the amount of text in the content item.

[0136] The address data may describe a web address of a web page. The address data may include a uniform resource identifier and / or a uniform resource locator.

[0137] The position data can describe the position of one or more content items within a web page. In some implementations, the position data can include at least one of a scroll position, a start node, or an end node. The scroll position can describe the position of one or more content items relative to other portions of the web page. In some implementations, the start node can describe the start position of one or more content items. The end node can describe the end position of one or more content items. The position data can include a text fragment (Tomayac et al., “Scroll to Text Fragment,” GITHUB, (May 20, 2022, 9:40 PM) https: / / github.com / WICG / scroll-to-text-fragment) that can be utilized to indicate the position of the content item. In some implementations, the text fragment can include one or more text directives associated with the position. The text directive can include a string of code start data (e.g., the first text associated with the content item and / or the first pixel associated with the content item) and / or end data (e.g., the last text associated with the content item and / or the last pixel associated with the content item). Text directives can be used to search a web page for a set of data that matches the beginning and / or end of a content item.

[0138] In some implementations, a computing system can process one or more content items to determine entities associated with the one or more content items. Entity tags can be generated based on the entities. A snippet packet can include the entity tag.

[0139] At 608, the computing system may store the snippet packet. The snippet packet may be associated with a particular user. Associating with a particular user may include storing the snippet packet with metadata indicative of the user and / or associating the snippet packet with a particular user profile. The particular user may be a user who provides input data that is processed to determine the snippet packet generation request. The snippet packet may be stored in a user database. In some implementations, storing the snippet packet in the user database may include storing the snippet packet locally on the mobile computing device. Additionally and / or alternatively, a graphics card may be stored as a graphical representation of the snippet packet. The graphics card may be automatically generated and may be customizable by a user. The graphics card may start with a template that can be customized based on other content of the web page, based on user input, and / or based on other context. In some implementations, the graphics card may include multimodal data. Alternatively and / or additionally, the snippet packet may be stored on a server computing system. In some implementations, if a content item references other content items, the systems and methods can obtain the additional content items and store the additional content items in snippet packets.

[0140] In some embodiments, the system and method may include receiving a snippet request to provide a snippet interface. The snippet interface may include an interactive element associated with the snippet packet. The snippet interface may be provided for display. An interface selection may be received, where the interface selection may describe a selection of the interactive element. A portion of the web page may then be provided for display. In some embodiments, the portion of the web page may include the location of one or more content items within the web page.

[0141] In some implementations, the systems and methods can include receiving an insertion input. The insertion input can include user input requesting insertion of a content item into a different interface. The snippet packet can be provided to a third-party server computing system.

[0142] Additionally and / or alternatively, the systems and methods may include adding snippet packets to a collection. The snippet packets may be added to the collection based on received user input. Alternatively and / or additionally, the snippet packets may be automatically added to the collection based on determined entities associated with the content item. In some implementations, the snippet packets may be added to the collection based on the source of the content item (e.g., based on the type of web page, the type of media provider, and / or the type of content item).

[0143] In some implementations, snippet packets can be generated based on content items obtained from sources other than web pages (e.g., mobile applications, large data files (e.g., downloaded videos or books), and / or data from other sources).

[0144] 7 shows a flowchart diagram of an exemplary method for performing in accordance with an exemplary embodiment of the present disclosure. While FIG. 7 shows steps performed in a particular order for purposes of illustration and explanation, the methods of the present disclosure are not limited to the particularly shown order or arrangement. Various steps of method 700 can be omitted, rearranged, combined, and / or adapted in various ways without departing from the scope of the present disclosure.

[0145] At 702, a computing system may obtain input data. In some implementations, the input data may describe a selection of a content item associated with the snippet packet. The snippet packet may be associated with a user account for a particular user. Additionally and / or alternatively, the user account may be associated with one or more platforms. The snippet packet may include a content item and a deep link. Alternatively and / or additionally, the snippet packet may include a snippet (e.g., a content item and / or media data generated based on the content item (e.g., a graphics card and / or a summary of the content item)), address data (e.g., a uniform resource locator and / or a uniform resource identifier), and / or metadata. The snippet packet may include location data (e.g., metadata indicating the location of the content item within a web page, a text fragment for identifying start and end data, one or more pointers, and / or one or more scroll position data).

[0146] In some embodiments, the snippet packet may be generated by providing a graphical user interface for display. The graphical user interface may include a graphical window for displaying a web page. In some embodiments, the web page may include a plurality of content items. The snippet packet generation may include obtaining selection data. The selection data may include a request to save a content item of the plurality of content items. In some embodiments, the snippet packet generation may include generating a snippet packet and storing the snippet packet in a user database.

[0147] At 704, the computing system can obtain address data and location data associated with the snippet packet. The address data can be associated with a web page. The content item can be associated with the web page. Additionally and / or alternatively, the location data can describe a location of the content item within the web page.

[0148] At 706, the computing system may obtain web page data. The web page data may be obtained based at least in part on the address data (e.g., by navigating to the web page using a uniform resource locator).

[0149] At 708, the computing system can determine a location within the web page associated with the content item. The location can be determined based on the text fragment, one or more pointers, and / or via web page processing.

[0150] In some implementations, the address data can include a uniform resource locator. Additionally and / or alternatively, the location data can include one or more text fragments. Determining a location within a web page associated with the content item can then include adding the text fragment to the uniform resource locator to generate a shortcut link and entering the shortcut link into a browser.

[0151] At 710, the computing system may provide a portion of a web page. The portion of the web page may include a location of a content item. In some implementations, providing the portion of the web page may include providing one or more indicators in the portion of the web page. The one or more indicators may indicate a content item associated with the snippet packet. In some implementations, the one or more indicators may include highlighting text associated with the content item.

[0152] 8 shows a flowchart diagram of an exemplary method for performing in accordance with an exemplary embodiment of the present disclosure. While FIG. 8 shows steps performed in a particular order for purposes of illustration and explanation, the methods of the present disclosure are not limited to the particularly shown order or arrangement. Various steps of method 800 can be omitted, rearranged, combined, and / or adapted in various ways without departing from the scope of the present disclosure.

[0153] At 802, a computing system may provide a graphical user interface. The graphical user interface may include a graphical window for displaying a web page. In some implementations, the web page may include multiple content items.

[0154] At 804, the computing system may receive gesture data. The gesture data may describe a gesture associated with a portion of a web page. In some implementations, the gesture may include a circular gesture that encircles the portion of the web page. The gesture data may describe touch input to a touchscreen display of a mobile computing device.

[0155] At 806, the computing system can process the gesture data to determine a selected content item. The selected content item can be associated with a portion of a web page.

[0156] In some implementations, processing the gesture data to determine the selected content item may include determining a portion of the web page enclosed by the gesture, determining a focus of the portion, and determining that the selected content item is associated with the focus of the portion.

[0157] Alternatively and / or additionally, processing the gesture data to determine the selected content item may include processing the gestured data with a machine-learned model to determine the selected content item.

[0158] In some implementations, the selected content item can be determined based on a determined matching area associated with a rectangle determined based on the circle. The rectangle can be a rectangle of data based on a syntactic configuration of the web page. In some implementations, the selected content item can be determined based on determined word boundaries and / or determined media content boundaries. The determination can be based on hypertext markup language code boundaries. In some implementations, the source code of the web page can be analyzed and the analyzed data can be processed.

[0159] Alternatively and / or additionally, the determining may include calculating an area of ​​a gesture rectangle associated with the gesture. The area of ​​one or more content item elements may be determined. The area of ​​intersection of the area of ​​the gesture rectangle with each area of ​​the different content item elements may be determined. The element with the highest degree of intersection may have the highest probability of being selected and may therefore be determined as the selected content item.

[0160] At 808, the computing system can generate a snippet packet based on the gesture data. The snippet packet can include the selected content item. In some implementations, the snippet packet can include address data and location data. The address data can be associated with a web page, and the location data can describe the location of the content item within the web page.

[0161] A save interface can then be provided for display. The save interface can provide interactive interface elements that can be selected to add the snippet packet to a collection. A collection interface can then be provided. A drag input can then be received to drag a graphical representation of the snippet packet to a graphical tile that describes a particular collection. The snippet packet can then be stored in the collection (e.g., the snippet packet can be stored with a relationship tag that links the snippet packet to a particular collection). In some implementations, the graphical representation of the snippet packet can change size and / or proportion as the graphical representation is dragged to a particular collection. The change in size and proportion can provide an intuitive indication of collection addition while providing an aesthetically pleasing display.

[0162] In some implementations, a content item can be processed to determine entities associated with the content item. The entities can be determined by processing the content item (e.g., text data, image data, audio data, video data, latent coding data, and / or link data) with a machine-learned model (e.g., an image classification model, an object classification model, a text classification model (e.g., a natural language processing model), a segmentation model, a semantic model, and / or a detection model) to generate entity data (e.g., a classification). Relationship data can then be generated and added to the snippet packet based on the entity data. The relationship data can include one or more entity tags and / or references to other related snippet packets and / or related web pages or content items.

[0163] The snippet packets may be searchable within one or more applications and / or databases. Additionally and / or alternatively, the snippet packets may be shareable via messaging applications, social media applications, and / or other applications.

[0164] In some implementations, content items can be processed with one or more machine-learned models to generate tags that can be stored in snippet packets. The tags can then be utilized as searchable tags to surface snippet packets in response to search queries. Tags can be determined based on the content of the content item (e.g., recognized words, recognized objects in an image, characteristics of a video frame or audio stream, etc.).

[0165] In some implementations, snippet packet generation occurs after selection of a snippet packet generation interface element that can open a snippet packet generation interface. Alternatively and / or additionally, a user can select a content item and one or more (e.g., two) popups can be provided with various options for interacting with the content item, one of the options can include snippet packet generation. Alternatively and / or additionally, search results associated with the content item can be provided.

[0166] Additional Disclosures The technology described herein refers to servers, databases, software applications, and other computer-based systems, as well as actions taken and information sent to and from such systems. The inherent flexibility of computer-based systems allows for a wide variety of possible configurations, combinations, and divisions of tasks and functionality among components. For example, the processes described herein can be implemented using a single device or component, or multiple devices or components working in combination. Databases and applications can be implemented on a single system or distributed across multiple systems. Distributed components can operate sequentially or in parallel.

[0167] While the present subject matter has been described in detail with reference to various specific exemplary embodiments thereof, each example is provided for purposes of illustration and not limitation of the present disclosure. Those skilled in the art, upon understanding the foregoing, will readily make modifications, variations, and equivalents to such embodiments. Accordingly, the present disclosure does not exclude the inclusion of such modifications, variations, and / or additions to the present subject matter as would be readily apparent to one skilled in the art. For example, features illustrated or described as part of one embodiment can be used with other embodiments to yield a further embodiment. Accordingly, the present disclosure is intended to encompass such modifications, variations, and equivalents.

Claims

1. 1. A computing system comprising: one or more processors; and one or more non-transitory computer-readable storage media that collectively store instructions that, when executed by the one or more processors, cause the computing system to perform operations, wherein the operations include: providing data describing a graphical user interface, the graphical user interface including a graphical window for displaying a web page, the web page including a plurality of content items; obtaining input data, the input data including a request to store one or more content items of the plurality of content items; generating a snippet packet, the snippet packet comprising: the one or more content items; and address data, the address data describing a web address of the web page; location data describing a location of the one or more content items within the web page; Generating a snippet packet involves determining an image associated with the one or more content items; determining text describing the one or more content items; generating a graphics card including the text overlaid on the image. and storing the snippet packet, the snippet packet being associated with a particular user; obtaining a selection of a particular sharing option from the one or more sharing options; transmitting the snippet packet to one or more computing systems based on the selection of the particular sharing option, the snippet packet including the graphics card, the graphics card being selectable to automatically navigate to the location of the selected content within the web page based on the address data and the location data; a computing system including:

2. The operation is receiving an insertion input, the insertion input comprising a user input requesting insertion of the one or more content items into a different interface; and providing the snippet packet to a third-party server computing system.

3. 2. The system of claim 1, wherein the position data includes at least one of a scroll position, a start node, or an end node, wherein the scroll position describes a position of the one or more content items relative to other portions of the web page, the start node describes a starting position of the one or more content items, and the end node describes an ending position of the one or more content items.

4. The system of claim 1 , wherein the one or more content items include at least one of an image, a video, a product graphic depiction, or audio.

5. The operation is processing the one or more content items to determine entities associated with the one or more content items; generating an entity tag based on the entity; The system of claim 1 , wherein the snippet packet includes the entity tag.

6. The system of claim 1 , wherein storing the snippet packet comprises storing the snippet packet locally on a mobile computing device.

7. 1. A computer-implemented method comprising: obtaining, by a computing system including one or more processors, input data describing a selection of a graphics card associated with a snippet packet, the graphics card including text data and an image, the text data describing a summary of a content item, the image associated with the content item, and the graphics card including the summary overlaid on the image; obtaining, by the computing system, address data and location data associated with the snippet packet, the address data being associated with a web page, the content item being associated with the web page, and the location data describing a location of the content item within the web page; obtaining, by the computing system, web page data, the web page data being obtained based at least in part on the address data; determining, by the computing system, the location within the web page associated with the content item based at least in part on the location data; providing, by the computing system, a portion of the web page, the portion of the web page including the location of the content item, the providing of the portion of the web page comprising: providing the web page for display in a graphical window; automatically navigating to the location within the web page associated with the content item based on the location data; providing one or more indicators of the location of the content item within the web page being offered for display; A method comprising:

8. The address data includes a uniform resource locator and the location data includes a text fragment, and determining, by the computing system, the location within the web page associated with the content item includes: adding, by the computing system, the text fragment to the uniform resource locator to generate a shortcut link; and inputting, by the computing system, the shortcut link into a browser.

9. The snippet packet comprises: providing, by the computing system, a graphical user interface, the graphical user interface including the graphical window for displaying the web page, the web page including a plurality of content items; obtaining, by the computing system, selection data, the selection data including a request to store the content item of the plurality of content items; generating, by the computing system, the snippet packet; and storing, by the computing system, the snippet packet in a user database.

10. The method of claim 7 , wherein the one or more indicators include highlighting text associated with the content item.

11. The method of claim 7 , wherein the snippet packets are associated with a user account of a particular user, the user account being associated with one or more platforms.

12. One or more non-transitory computer-readable storage media that collectively store instructions that, when executed by one or more computing devices, cause the one or more computing devices to perform operations, the operations including: providing a graphical user interface, the graphical user interface including a graphical window for displaying a web page, the web page including a plurality of content items; receiving gesture data, the gesture data describing a gesture associated with a portion of the web page; processing the gesture data to determine a selected content item, the selected content item being associated with the portion of the web page; generating a snippet packet based on the gesture data, the snippet packet including the selected content item, the snippet packet including a graphics card associated with the selected content item, the graphics card presenting data describing the selected content item associated with the portion of the web page, the graphics card being selectable to automatically navigate to a location of the selected content within the web page based on address data and location data; processing the selected content items to determine topics associated with the selected content items; determining an image based on the topic associated with the selected content item; generating the graphics card, the graphics card including the image as a background, the graphics card including text associated with at least a subset of the selected content items overlaid on the image; and generating a snippet packet, wherein the graphics card generates the snippet packet by:

13. 13. The one or more non-transitory computer-readable media of claim 12, wherein the snippet packet includes address data and location data, the address data being associated with the web page and the location data describing the location of the selected content item within the web page.

14. The one or more non-transitory computer-readable storage media of claim 12 , wherein the gesture comprises a circular gesture that encircles the portion of the web page.

15. Processing the gesture data to determine the selected content item comprises: determining the portion of the web page enclosed by the gesture; determining a focus of the portion; and determining that the selected content item is associated with the focal point of the portion.

16. Processing the gesture data to determine the selected content item comprises:

13. The one or more non-transitory computer-readable storage media of claim 12, comprising processing the gestured data with a machine-learned model to determine the selected content item.

17. The one or more non-transitory computer-readable media of claim 12 , wherein the gesture data describes touch input to a touchscreen display of a mobile computing device.

Images