Digital Asset Search User Interface

Abstract

Embodiments of the present disclosure present devices, methods, and computer-readable media for presenting a collection of digital assets that allows a user to quickly and easily filter and search. The disclosed technology allows for quick invocation of desired digital assets, linking of assets into logical collections, and an overall improved user experience. The zero keyword / contextual keyword feature presents multimedia content icons and searchable keywords to allow a user to search the collection of digital assets by simply clicking on one of these keywords. The top autocomplete feature automatically completes suggestions in the search field based on various heuristics to ensure that the method produces diverse and relevant results. The next keyword suggestion feature predicts the next search term based on learned attributes about the collection of digital assets.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. Provisional Application No. 62 / 668,181, filed May 7, 2018, entitled “Digital Asset Search UserInterface”; and U.S. Patent Application No. 16 / 147,067, filed September 28, 2018, entitled “DigitalAsset Search User Interface”; both of which are hereby incorporated by reference in their entirety for all purposes. This patent application incorporates by reference in its entirety commonly owned U.S. Patent Application Serial No. 15,391,276, filed December 27, 2016, entitled “Knowledge Graph Metadata Network Based on Notable Moments” for all purposes. Background Art

[0003] Modern computing devices provide the opportunity to store thousands of digital assets (e.g., digital photos, digital videos, etc.) in electronic devices. Users typically display their digital assets to others by presenting images on a display screen of a computing device. Finding a specific digital asset or a group of related digital assets can be time consuming and result in a poor user experience. Manually sorting thousands or tens of thousands of digital assets into digital albums or folders can be time consuming, and it can be difficult to link a single digital asset with multiple collections. Managing digital asset collections can be a resource intensive exercise for users. Before finding a digital asset of interest, a user may have to sort through many unrelated digital assets. Scrolling through hundreds or thousands of photos is cumbersome and results in a poor user experience. Summary of the invention

[0004] Embodiments of the present disclosure may provide devices, methods, and computer-readable media for implementing a user interface for searching for digital assets in a digital asset collection. The present disclosure describes the implementation of one or more embodiments of a user interface that enables a user to quickly and easily filter and search for digital assets in a digital asset collection. The disclosed technology allows for quick recall of desired assets with minimal keystroke input, links assets to logical collections, and provides an overall improved user experience.

[0005] In various embodiments, a graphical user interface helps overcome the difficulties in searching for a digital asset collection using the technology described in the present disclosure. In various embodiments, a computing device may include one or more memories, one or more processors communicating with the one or more memories, and a display having a graphical user interface. The graphical user interface may be configured to display a search field in a first area of ​​a display of a digital asset collection page of a computing device. The search field may receive text associated with a search for a digital asset collection of a computing device. The graphical user interface may also be configured to display at least one representative multimedia icon including a first keyword tag, wherein the keyword tag is derived by the computing device according to a knowledge graph of the digital asset collection. Multimedia icons and keyword tags may be displayed for at least one search category in a second area of ​​the display, wherein the representative multimedia icon represents at least one asset associated with the first keyword tag within the digital asset collection. The graphical user interface may receive a selection of at least one representative multimedia icon of a desired search category corresponding to the first keyword tag. The graphical user interface may allow a user to initiate a search of the digital asset collection of a computing device to search for relevant digital assets of the desired search category. The graphical user interface may display a subset of the digital asset collection on the display of the computing device according to the desired search category.

[0006] In various embodiments, a graphical user interface of a computing device may be configured to display at least one suggested search term based at least in part on a correlation between a first metadata set for a desired search category and a second metadata set for digital assets. The graphical user interface may display a second keyword tag associated with the at least one suggested search term. The technology may further filter the digital assets in the collection of digital assets to exclude certain digital assets that are not associated with the at least one suggested search term, the further filtering creating a further modified collection of digital assets; and displaying the further modified collection of assets on the display.

[0007] In various embodiments, the at least one suggested search term may include identifying text and a collection icon. The collection icon may identify a collection of suggested search terms in the collection of digital assets.

[0008] In various embodiments, a graphical user interface of a computing device may be configured to display at least one suggested search term based at least in part on a correlation between one or more text entries in a search field and a metadata set of digital assets. The graphical user interface may display a second keyword tag associated with the at least one suggested search term to further filter the digital assets in the set of digital assets to exclude certain digital assets that are not associated with the at least one suggested search term. The further filtering may create a further modified set of digital assets, and the graphical user interface may display the further modified set of assets on the display.

[0009] In various embodiments, a graphical user interface of a computing device is configured to automatically complete a text entry of a portion of a search term entered into a search field depicted in an area of ​​the user interface, wherein the automatic completion is based at least in part on metadata and weighted criteria stored in a knowledge graph. The weighted criteria may take into account at least one of: the number of digital assets with keyword tags that match the completed search term; the position of the matching term in a keyword tag of a multi-term keyword tag, where the matched term matches the completed search term; the asset class of the keyword tag corresponding to the completed search term; the position of the keyword tag of the completed search term, where the position is in the ranking of the keyword tags in the selected asset class; or the number of matching keyword tags for the completed search term; displaying the completed search term in the search field on the user interface.

[0010] In various embodiments, a graphical user interface of a computing device may be configured to display one or more multimedia icons determined at least in part by keyword tags associated with the multimedia icons in multiple sets, with each set displayed in a separate area of ​​the display.

[0011] In some embodiments, these collections may include events, people, places, categories, and groups.

[0012] In various embodiments, a graphical user interface of the computing device may be configured to scroll a list of multimedia icons in a set of at least one multimedia icons in response to at least recognition of a gesture.

[0013] In various embodiments, the multimedia icon may include a depiction of a representative digital asset from the collection of digital assets that corresponds to the first keyword tag.

[0014] In various embodiments, a technique for displaying a graphical user interface on a computing device having a display may include displaying a search field in a first area of ​​a display of a digital asset collection page of the computing device. The search field may receive text associated with the digital asset collection of the computing device. The technique may also include displaying at least one user interface element including a first keyword tag. The computing device may generate keyword tags based on a knowledge graph of the digital asset collection for at least one search category in a second area of ​​the display, wherein the user interface element represents at least one asset associated with the first keyword tag within the digital asset collection. The technique includes receiving a selection of at least one user interface element of a desired search category corresponding to the first keyword tag. Next, the technique includes initiating a search of the digital asset collection of the computing device to search for relevant content of the desired search category. The technique includes displaying a subset of the digital asset collection on the display according to the desired search category. In various embodiments, each suggested search term may include an asset count to notify the user of the number of digital assets in response to the search request.

[0015] In various embodiments, the technique may also include displaying suggested search terms in a graphical user interface based at least in part on a correlation between the first metadata set of the desired search category and the second metadata set of the digital asset. The technique includes preparing to display a second keyword tag associated with the suggested search term. Next, the technique includes further filtering the digital assets in the set of digital assets to exclude certain digital assets that are not relevant to the suggested search term. The further filtering may create a further modified set of digital assets. The technique may include displaying the further modified set of assets on a display.

[0016] In various embodiments, the suggested search terms may include identifying text and a collection icon, where the collection icon identifies a collection of suggested search terms in the collection of digital assets.

[0017] In various embodiments, the technique may include displaying suggested search terms on a graphical user interface based at least in part on a correlation between one or more text entries in a search field and a metadata set of a digital asset. The technique may include displaying a second keyword tag associated with the suggested search term. The technique may also include further filtering the digital assets in the set of digital assets to exclude certain digital assets that are not associated with the suggested search term. The further filtering may create a further modified set of digital assets. The technique may include displaying the further modified set of assets on a display.

[0018] In various embodiments, the technology may include automatically completing a text entry of a portion of a search term entered into a search field depicted in an area of ​​a user interface, wherein the automatic completion is based at least in part on metadata and weighted criteria stored in a knowledge graph. The weighted criteria may consider at least one of: the number of digital assets with keyword tags that match the completed search term; the position of the matching term in a keyword tag of a multi-term keyword tag, where the matched term matches the completed search term; the asset class of the keyword tag corresponding to the completed search term; the position of the keyword tag of the completed search term, where the position is in the ranking of the keyword tags in the selected asset class; or the number of matching keyword tags for the completed search term; displaying the completed search term in the search field on the user interface.

[0019] In various embodiments, the technique may further include the graphical user interface displaying one or more multimedia icons in a plurality of sets determined at least in part by a keyword tag associated with the graphical user interface, wherein each set may be displayed in a separate area of ​​the display.

[0020] In various embodiments, a computer-readable medium storing a plurality of instructions that, when executed by one or more processors of a computing device having a display, configure the one or more processors to perform operations including generating a graphical user interface for searching a collection of digital assets on the computing device. The operations may include providing a search field in a first area of ​​the graphical user interface for receiving text associated with the collection of digital assets of the computing device. The operations may also include providing at least one user interface element including a first keyword tag for at least one search category in a second area of ​​the display, wherein the at least one user interface element is generated by the computing device based on a knowledge graph of the collection of digital assets, and initiating a search of the collection of digital assets of the computing device by activating the desired user interface element to search for relevant content of the desired search category.

[0021] In various embodiments, the computer-readable medium may also provide instructions for operating to display suggested search terms on a graphical user interface based at least in part on the correlation between the first metadata set of the desired search category and the second metadata set of the digital asset. The operation may include displaying a second keyword tag associated with the suggested search term. The operation may include further filtering the digital assets in the set of digital assets to exclude certain digital assets that are not related to the suggested search term, the further filtering creating a further modified set of digital assets. Finally, the operation may also include displaying the further modified set of assets on a display.

[0022] In various embodiments, the suggested search terms may include identifying text and a collection icon, where the collection icon may identify a collection of suggested search terms in a collection of digital assets.

[0023] In various embodiments, the operations may further include displaying suggested search terms based at least in part on a correlation between one or more text entries in the search field and a metadata set of the digital asset. The operations may include displaying a second keyword tag associated with the suggested search term. The operations may further include further filtering the digital assets in the set of digital assets to exclude certain digital assets that are not associated with the suggested search term. The further filtering may create a further modified set of digital assets. The operations may include displaying the further modified set of assets on a display.

[0024] In various embodiments, operations may include automatically completing a text entry of a portion of a search term entered into a search field depicted in an area of ​​a user interface, wherein the automatic completion is based at least in part on metadata and weighted criteria stored in a knowledge graph. The weighted criteria may consider at least one of: the number of digital assets with keyword tags that match the completed search term; the position of the matching term in a keyword tag of a multi-term keyword tag, where the matched term matches the completed search term; the asset class of the keyword tag corresponding to the completed search term; the position of the keyword tag of the completed search term, where the position is in the ranking of the keyword tags in the selected asset class; or the number of matching keyword tags for the completed search term; displaying the completed search term in the search field on the user interface.

[0025] In various embodiments, the graphical user interface may display thumbnails of the top 8 search results along with metadata responsive to the search query.

[0026] The following detailed description together with the accompanying drawings will provide a better understanding of the nature and advantages of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 An example user interface, specifically a digital asset search page, is shown according to at least one embodiment.

[0028] Figure 2 Another example user interface for digital asset searching, specifically another example of a digital asset search page, is shown according to at least one embodiment.

[0029] Figure 3 Another example user interface for digital asset searching, specifically a suggestions page, is shown according to at least one embodiment.

[0030] Figure 4Another example user interface for digital asset searching, and specifically another example of a suggestions page, is shown according to at least one embodiment.

[0031] Figure 5 Another example user interface for digital asset searching, and specifically another example of a suggestions page, is shown according to at least one embodiment.

[0032] Figure 6 Another example user interface for digital asset searching, specifically an example results page, is shown according to at least one embodiment.

[0033] Figure 7 Another example user interface for digital asset searching, and specifically another example results page, is shown according to at least one embodiment.

[0034] Figure 8 Another example user interface for digital asset searching, particularly an example of auto-completion, is shown according to at least one embodiment.

[0035] Fig. 9 Another example user interface for digital asset searching, and specifically another example of next keyword suggestion, is shown in accordance with at least one embodiment.

[0036] Fig.10 Another example user interface for digital asset searching, specifically a suggestions page, is shown according to at least one embodiment.

[0037] Fig.11 Another example user interface for digital asset searching, specifically a results page, is shown according to at least one embodiment.

[0038] Fig.12 An example flow diagram of a technique for searching a collection of digital assets is shown.

[0039] Fig.13 is a flow diagram illustrating providing a user interface for searching for digital assets in a collection of digital assets as described herein, according to at least one example.

[0040] Fig.14 is another flow diagram illustrating providing a user interface for searching for digital assets in a collection of digital assets as described herein, according to at least one example.

[0041] Fig.15 is a simplified block diagram illustrating a computer architecture for providing the audio and / or video effects techniques described herein, according to at least one example. DETAILED DESCRIPTION

[0042] Certain embodiments of the present disclosure relate to devices, computer-readable media, and methods for generating a graphical user interface for implementing various technologies for searching digital assets in a computing device. In the following description, various embodiments will be described. For the purpose of explanation, many specific configurations and details are set forth in order to provide a thorough understanding of the embodiments. However, it will also be apparent to those skilled in the art that these embodiments can be implemented without these specific details. In addition, well-known features may be omitted or simplified to prevent confusion with the embodiments described herein. The present disclosure describes a user interface (UI) to allow searching for various digital assets (e.g., digital photos, digital videos, etc.) stored in a collection of digital assets on a computing device.

[0043] Embodiments of the present disclosure relate to improving user experience regarding accessing and searching digital content collections, etc. As used herein, "digital assets" may include data that can be stored in digital form or stored in digital form (e.g., digital images, digital videos, digital voice recordings). As used herein, a "digital asset collection" refers to a plurality of digital assets that can be stored in one or more storage locations. One or more storage locations may be separated spatially or logically. As used herein, a "knowledge graph" refers to a metadata network associated with a digital asset collection that includes related metadata assets that describe features associated with digital assets in the digital asset collection. As used herein, a "node" in a metadata network refers to a metadata asset associated with one or more digital assets in a digital asset collection. The examples provided herein and the context of such examples are intended for illustrative purposes, rather than limiting the scope of the present disclosure.

[0044] Disclosed herein is a user interface that implements several different features for improving the search process for a collection of digital assets. These features include zero keyword / contextual keyword, top autocomplete, and next keyword suggestion. The zero keyword / contextual keyword feature may present multimedia content icons and searchable keywords in the user interface to allow a user to search for digital assets by simply clicking on one of these keywords. The top autocomplete feature may enable text suggestions to be automatically completed in the search field based on various heuristics to ensure that the search produces diverse and relevant results. The next keyword suggestion feature may predict and present the next search term in the user interface based on learned attributes about the collection of digital assets.

[0045] In one embodiment, the disclosed technology may generate a user interface for generating a zero-keyword or contextual keyword search for digital assets. In the technology, one or more user interface elements in combination with keyword tags may be displayed, and these keyword tags describe features associated with digital assets. The technology may be performed by one or more processors of a computing device using a knowledge graph. The knowledge graph may include multiple nodes representing associations between digital assets and asset categories. The digital assets may be stored in a digital asset collection of a computing device, wherein each category in the asset category corresponds to a corresponding keyword tag in a plurality of keyword tags. According to various embodiments, these technologies may involve accessing a knowledge graph to retrieve a plurality of keyword tags based at least in part on an action defined by a computing device. A specific digital asset in a digital asset collection may be selected for each keyword tag in the plurality of keyword tags based at least in part on a specific digital asset associated with a specific node in the plurality of nodes of the knowledge graph. These technologies may involve preparing to display a user interface including user interface elements. Each of the user interface elements may include a keyword tag in the plurality of keyword tags and a corresponding multimedia icon representing the corresponding selected digital asset. The technology may also involve receiving a selection of at least one user interface element, the selection indicating a desired search category based at least in part on a corresponding keyword tag for the selection. The technology may also include filtering the digital assets in the digital asset set to exclude certain digital assets that are not related to the desired search category. The filtering may be implemented by the one or more processors of the computing device. The filtering may create a modified digital asset set. The technology may include initiating a search of the modified digital asset set of the computing device to search for digital assets having metadata corresponding to the corresponding keyword tag, and preparing to display a second user interface including a second user interface element corresponding to the modified digital asset set. Each second user interface element of the plurality of second user interface elements may include a second keyword tag based at least in part on the desired search category and a second corresponding multimedia icon.

[0046] In various embodiments, the technology may include calculating a priority score for a plurality of keyword tags represented by items including multimedia icons and keyword tags for each asset set. The priority scores may be based on criteria. According to various embodiments, the multimedia icons and keyword tags are stored and displayed by sorting the priority scores. In addition to heuristics (e.g., a context location based on the current location of the computing device may be displayed before a context location before a recent event), the context keywords may be displayed before zero keywords using the temporal distance between the current date and the date of the context event.

[0047] The priority scores for keyword tag display with zero keyword suggestions may vary depending on the set of keyword tags. For a "people" set, the priority scores may be based on criteria that may include: a number of assets / collections related to a person in the digital asset set; the relationship of the device owner to a person (e.g., spouse, friend, family member, colleague, collaborator, etc.); if a person may be associated with one or more social groups, the person may be displayed with other characters from those social groups. For a social "group" set, the priority scores may be based on criteria that may include: a number of assets / collections associated with a social group; and a social group consistency score based on the fact that characters belonging to a social group mostly appear in that social group in the data asset set. For the "places" collection, the priority score may be based on criteria that may include: multiple assets / collections at the location (i.e., time / distance clustering of collections may be used to approximate the location); if the place is a common location in the digital asset collection; if the place is identified as a home / work location in the user's contact card; if the place is a location of interest detected by the device's GPS data (i.e., no digital asset collection is required); various heuristics may allow for appropriate location scales (e.g., city, state, country) to be supported based on the digital asset collection. These factors take into account the hierarchy of locations in the knowledge graph and may be based on how many children / inheritances a location node has in the hierarchy of the knowledge graph. For the category ("scenes") collection, the priority score may be based on criteria that may include: multiple assets / collections with the scene; the level of the scene in the scene taxonomy; and a good whitelist of scenes that appear as zero keywords. For the "events" (knowledge graph meanings) collection, the priority score may be based on criteria that may include: multiple assets / collections with the meaning or any parent with the meaning in the graph meaning hierarchy; and a good whitelist of meanings that appear as zero keywords. Once the digital asset management module / logic component determines the priority score for each set in the collection, each set of zero keywords is based on its score, and we use heuristics based on the mean and standard deviation to cut the top selection of zero keywords for display.

[0048] The priority scores for the display of contextual keyword tags may differ based on the set of keyword tags. For the "People" set, the priority scores for contextual keyword suggestions may be based on criteria that may include: people currently located near the computing device; people who recently had birthdays or are about to have birthdays; and people recently seen by the user. For the social "Groups" set, the priority scores for contextual keyword suggestions may be based on criteria that may include: social groups whose members are currently located near the computing device; social groups recently seen by the user. For the "Places" set, the priority scores for contextual keyword suggestions may be based on criteria that may include: places close to the computing device; and meaningful places that the user has recently visited. For the "Events" (Meanings) set, the priority scores for contextual keyword suggestions may be based on criteria that may include: meanings from the user's recent events (e.g., museums, theme parks, weddings, etc.). For the "Scenes" set, the priority scores for contextual keyword suggestions may be based on criteria that may include: scenes from the user's recent events (and a whitelist of scenes). For the "Holidays" set, the priority scores for contextual keyword suggestions may be based on criteria that may include: past and upcoming holidays that the user frequently celebrates (e.g., celebrations are inferred in the knowledge graph). For the "Season" collection, the priority scores for contextual keyword suggestions may be based on criteria that may include: the current season and the season that just passed. For the "One Year Ago" collection, the priority scores for contextual keyword suggestions may be based on criteria that may include: if the digital asset was captured on this day one year ago (a week span), it will be displayed.

[0049] In various embodiments, the action of causing the knowledge graph to retrieve the plurality of keyword tags may include at least one of the following events: a change to a digital asset in the set of digital assets; a change to a face in the set of digital assets. A face represents a connection between an image of a person and an identity of the person. The action may include adding or deleting a selected node from a plurality of nodes of the knowledge graph. The action may also include synchronizing the set of digital assets to a cloud storage application. The action may also include a change in a relationship between a first digital asset and a second digital asset.

[0050] In various embodiments, these techniques may also include generating a multimedia icon including a depiction of a representative digital asset in the digital asset collection corresponding to the keyword tag. In the case where the digital asset is a video in the digital asset collection, the depiction of the representative asset may be a video frame.

[0051] In various embodiments, the next keyword suggestion function infers the next keyword that the user may want to search for in order to refine the search query. This technique has the following two advantages: optimizing the search query and obtaining the best search results for the user in the shortest number of steps while displaying search terms that should be familiar to the user because they are based on assets in the user's digital asset collection. The next keyword suggestion feature is run by a system that accesses the knowledge graph to access multiple metadata associated with the keyword tags.

[0052] In various embodiments, the next keyword suggestion technique may derive at least one suggested search term based at least in part on the correlation between a first metadata set (selected multimedia icons and / or keyword tags) of a desired search category and a second metadata set of digital assets of the asset category. The technique may prepare to display a second keyword tag associated with at least one suggested search term and further filter the digital assets in the digital asset collection to exclude certain digital assets that are not associated with at least one suggested search term. Further filtering may create a further modified digital asset collection. The next keyword suggestion feature may also include a classic autocomplete function found in most search engines. However, the present disclosure may teach an innovative suggestion method to minimize the number of characters entered by the user while encouraging the user to refine his or her search as much as possible to find his or her search results. The digital asset management module / logic component may refine his or her search query by trying to infer the next keyword that the user may want to search for to achieve the next keyword suggestion. However, most search engines may achieve this by looking at the statistical correlation between search terms or trying to predict the next search term based on learned existing information. The digital asset management module / logic component can analyze the domain space covered by the keywords that have been entered in the search query to infer the next keyword of the best separation space and, in theory, guide the user to his or her results faster. Each time the user enters a keyword, the digital asset management module / logic component can find a set of next potential keywords that will maximize the coverage of the current searchable domain space (for example, possibly up to 100%), while minimizing the overlap between the subdomains covered by the updated search query as each suggested next keyword is added to the existing query. This technology has the following advantages: the search query is optimized to get the user to his or her next result as quickly as possible (in terms of steps) while displaying these next keyword suggestions that are familiar to him or her because they come from his or her own collection.

[0053] In various embodiments, the at least one suggested search term includes a collection icon. The collection icon identifies a collection of digital assets associated with the suggested search term in the collection of digital assets. For example, the collection icon may include people, places, categories, moments, and events.

[0054] Autocomplete suggestions and search fields are usually based on statistical data. For example, the suggestions may be learned from the frequency of terms searched by a set of users or extracted from a large text corpus. These classic approaches are not applicable or may have limited success in the present case, where the search is specific to each user's collection of digital assets. In this case, the digital asset management module / logic component still wants to ensure that the user has the best search results. This approach combines different heuristics to ensure that the user gets diverse and relevant results.

[0055] In various embodiments, the top auto-complete function automatically completes suggestions in the search field based on weighted criteria so as to provide both diverse and relevant search results. The top auto-complete technology may include automatically completing a portion of the text input of the search term in the search field depicted in the area of ​​the user interface, wherein the auto-completion is based at least in part on metadata stored in the knowledge graph and weighted criteria that consider at least one of the following: the number of digital assets with keyword tags that match the completed search terms; the position of the matching term in the keyword tag of the multi-term keyword tag, where the matched term matches the completed search term. The asset category of the keyword tag that can correspond to the completed search term; the position of the keyword tag of the completed search term, where the position is in the ranking of the keyword tag in the selected asset category; or the number of matching keyword tags of the completed search term. In addition to these criteria, the digital asset management module / logic component may also add classic criteria, such as search history. Finally, the digital asset management module / logic component establishes a score by applying weights to all criteria in order to return the best results. The criteria for the position of the matching terms in the keyword tag for the multi-term keyword tag where the matching terms match the completed search terms can create diversity in auto-completion because each part can be associated with a different category. In addition, for the purpose of auto-completion, the first position means a higher score, so even without considering the position in the score, the first element of a part can be picked even if that part is not as good as the last element of another part. The technology also includes displaying the completed search terms in the search field on the user interface.

[0056] In various embodiments, the weighting criteria may take into account a search history of a plurality of historical search terms of a historical search, where the search history may be stored on a computing device.

[0057] In various embodiments, the techniques may combine a top autocomplete feature and a next keyword suggestion feature. The techniques may be implemented by first deriving at least one suggested search term based at least in part on a correlation between a first metadata set corresponding to the completed search term and a second metadata set of digital assets of the asset class. Next, the techniques may include preparing to display a second keyword tag associated with the at least one suggested search term. Finally, the techniques may include further filtering the digital assets in the set of digital assets to exclude certain digital assets that are not associated with the at least one suggested search term, the further filtering creating a further modified set of digital assets.

[0058] In some embodiments, the digital asset management module / logic component may filter the search results to display the top digital assets in response to the search request. In some embodiments, the user interface may display one or more thumbnails of those top images. In some embodiments, the top digital assets may be the top 2, top 4, top 6, top 8, or any defined number of "top" assets. In some embodiments, the thumbnails may be displayed in one or more rows of thumbnails. The thumbnails may display the top 8 digital assets in two rows, with four thumbnails in each row. In some embodiments, the assets may be displayed in chronological order, with the oldest thumbnails displayed first and the newest thumbnails displayed last. In some embodiments, the assets may be displayed in reverse chronological order (from newest to oldest). In some embodiments, the assets may be displayed in order of asset scores, with the first position being the asset with the highest score and the last position being the asset with the lowest score. In some embodiments, the first position may be the leftmost thumbnail in the first row of two rows of thumbnails. In some embodiments, the last position may be the rightmost thumbnail in the second row of two rows of thumbnails. In some embodiments, the digital asset thumbnails may be displayed in order of score, with thumbnails for the top assets displayed first and thumbnails for the highest-scoring assets displayed last. If the search query includes a greater number of assets (e.g., eight assets), a "Show All" button may be displayed, which, if selected, displays all assets. If the user selects one of the top digital assets, the asset may be displayed in a carousel, allowing the user to scroll through the assets using gestures on the display screen.

[0059] In some embodiments, the user interface may calculate and display a suggestion count for search results. For example, in addition to displaying the suggested search terms, if additional search terms are selected, the digital asset management module / logic component may display the number of digital asset results. In some embodiments, the suggestion count may be displayed on the right side of the display. In some embodiments, the digital asset management module / logic component may calculate and display the total number of digital assets (e.g., digital photos or videos) in the digital assets in response to the search request. The suggestion count may be calculated by calculating the number of suggested results for a given query. When multiple suggestions are collapsed together, the digital asset management module / logic component may deduplicate the number of digital assets in the search results. Deduplication helps ensure that the count can always represent the number of results that the user will see after selecting a result.

[0060] As used herein, "knowledge graph", "knowledge graph metadata network", "metadata network" and their variations refer to a dynamically organized collection of metadata that describes one or more digital assets (e.g., one or more digital asset groups in a digital asset set, one or more digital assets in a digital asset set, etc.) used by one or more computer systems for deductive reasoning. A knowledge graph is created using multiple metadata assets. In some embodiments, the digital asset management module / logic component enables the system to generate and use a knowledge graph of digital asset metadata and treat it as a multidimensional network. Each metadata asset may describe a feature associated with one or more digital assets in a digital asset set. Each metadata asset may be represented as a node in a metadata network. A metadata asset may be associated with at least one other metadata asset. Each correlation between metadata assets may be represented as an edge between nodes in the metadata network representing related metadata assets. Keyword tags are generated using a knowledge graph consisting of the multiple metadata associated with the digital assets in the digital asset set. Further description of using or generating knowledge graphs may be found in U.S. patent application serial number 15,391,276, entitled “Knowledge Graph Metadata Network Based on Notable Moments,” filed on December 27, 2016, which is hereby incorporated by reference in its entirety for all purposes.

[0061] Figure 1An example user interface according to at least one embodiment of the present disclosure is shown, specifically a digital asset search page. The user interface 100 displays a search field 102 in a first area of ​​the digital asset search page. The search field 102 is configured to receive one or more characters of text for searching for digital assets in a digital asset collection. The search field 102 is depicted at the top of the digital asset search page, but can be configured to be displayed in any part of the page. In the second area of ​​the display, multiple digital asset collections 103 marked with a collection identifier 104 can be displayed. In some embodiments, the collection 103 may include photos, people, places, events, categories, and groups. Each collection 103 displays multiple multimedia content icons (user interface elements) 106 with associated keyword tags 108. The multimedia content icon 106 displays a thumbnail image of an asset in the search results representing the digital assets associated with the associated keyword tag 108. Therefore, at least one digital asset in the digital asset collection associated with the multimedia content icon search results will contain a digital image / video frame displayed as a thumbnail. The user interface 100 allows the multimedia content icon 106 to be scrolled at least in response to the recognition of a gesture. In some embodiments, the gesture may be received via a touch screen display of the computing device. In some embodiments, the gesture causes the multimedia content icons 106 and the associated keyword tags 108 to scroll horizontally. In some embodiments, the vertical gesture causes the digital asset search page to scroll vertically, thereby providing a display of the additional collection 103. In addition, although the following examples are given primarily with reference to finger inputs (e.g., finger contact, single-finger tap gestures, finger swipe gestures), it should be understood that in some embodiments, one or more of these finger inputs are replaced by input from another input device (e.g., mouse-based input or stylus input). For example, a swipe gesture is optionally replaced by a mouse click (e.g., instead of contact), followed by movement of the cursor along the path of the swipe (e.g., instead of movement of contact). As another example, a tap gesture is optionally replaced by a mouse click when the cursor is located above the location of the tap gesture (e.g., instead of detecting contact, followed by stopping detecting contact). Similarly, when multiple user inputs are detected simultaneously, it should be understood that multiple computer mice are optionally used simultaneously, or a mouse and a finger contact are optionally used simultaneously.

[0062] The event collection 103 presents a plurality of multimedia content icons 106 and associated keyword tags 108, wherein each multimedia content icon 106 and associated keyword tags 108 represent a collection of digital assets stored in the digital asset collection. Each digital asset includes corresponding metadata that associates the digital asset with the event described by the associated keyword tag 108. For example, the first keyword tag is labeled "Easter", and selecting the associated multimedia content icon 106 for "Easter" will result in filtering the digital assets to exclude any assets that are not related to the event labeled "Easter". The search engine references the knowledge graph to determine the multimedia content icons 106 and keyword tags 108 based on the plurality of metadata for each digital asset in the digital asset collection. The keyword tags 108 are associated by the metadata of the digital assets in the digital asset collection.

[0063] The character collection 140 with the associated collection identifier 104 presents a plurality of multimedia content icons 106 and associated keyword tags 108, wherein each multimedia content icon 106 and associated keyword tag 108 represents a collection of digital assets stored in the digital asset collection. Each digital asset contains metadata associating the digital asset with one or more characters depicted in the multimedia content icon 106 and the associated keyword tag 108. For example, Figure 1 , has an associated keyword tag 108 labeled "Mary," and selecting the associated multimedia content icon 106 will result in filtering the digital assets to exclude any assets not associated with "Mary." In some embodiments, the keyword tag 108 is associated with an image of a person stored in a contact list of the computing device. In some embodiments, the keyword tag receives information from the image analysis and correlation with other tagged digital assets stored in the digital asset collection. In some embodiments, the user can identify the asset by storing the asset in a digital asset folder tagged with the identity of the person in the available image.

[0064] The place collection 150 will be described further below. The user interface 100 depicts a plurality of icons, allowing for additional organizational tools for managing a collection of digital assets. The icons may include an "album" icon, a "favorites" icon (labeled "personal"), and a "photos" icon. Figure 1 As shown, selecting the "Search" function. Selecting the "Album" icon will direct the user to the album feature of the digital asset. Selecting the "Collections" icon will direct the user to the "Collections" page. Selecting "Photos" will direct the user to a page that lists digital assets in chronological order.

[0065] Figure 2Another example user interface for digital asset searching, specifically another example of a digital asset search page, is shown according to at least one embodiment. Figure 2 Shows Figure 2 The digital asset management user interface is a continuation of the . Figure 2 The user interface 200 depicted in FIG. 1 is scrolled down to access Figure 2 . The user can return to the user interface 100 by scrolling up on the display. In some embodiments, the scrolling is in response to recognition of a gesture by a touch screen display of the computing device. The user interface 200 presents a search field 202 in a first area of ​​the display. The user interface 200 depicts a collection of places, categories, and groups.

[0066] The place collection 250 with associated collection identifier 204 presents a plurality of multimedia content icons 206 and associated keyword tags 208, wherein each multimedia content icon 206 and associated keyword tag 208 represents a collection of digital assets stored in the digital asset collection. Each digital asset includes metadata associating the digital asset with a geographical place depicted in the multimedia content icon 206 and the associated keyword tag 208. The place collection 250 depicts an exemplary collection of keyword tags 208 labeled "home," "Paris," and "San Francisco." The digital asset management module / logic component will recognize a "home" identifier for a user of a computing device based on the user's activities. Figure 2 The exemplary "home" depicted in the is San Jose, CA. Instead of displaying a thumbnail representing a representative asset in the collection for each multimedia content icon 206 depicted for a place icon, the disclosed technology generates an image of a map of the location. Therefore, in the "home" collection, there is no image of a digital asset with a map of San Jose, CA. Selection of the associated multimedia content icon 206 for "home" will result in filtering the digital assets to exclude any assets that do not contain location metadata associated with San Jose, CA. "Paris" and "San Francisco" are exemplary locations, and the user will associate additional multimedia content icons 206 and keyword tags 208 with the relevant locations where the digital assets have been captured. The capture location of the digital assets is identified by the location metadata associated with each digital asset. The digital asset management module / logic component determines which multimedia content icons 206 and associated keyword tags 208 are displayed, and in what order the multimedia content icons 206 and associated keyword tags 208 are displayed.

[0067] The category set 260 with the associated set identifier 204 presents a plurality of multimedia content icons 206 and associated keyword tags 208, wherein each multimedia content icon 206 and associated keyword tag 208 represents a set of digital assets stored in the digital asset collection. Each digital asset includes metadata associating the digital asset with the category depicted in the multimedia content icon 206 and the associated keyword tag 208. Figure 2 The user interface 200 depicted in the figure depicts exemplary topics of keyword tags 208 labeled "animals", "snow", and "sports". The digital asset management module / logic component generates representative multimedia content icons 206 and associated keyword tags 208 related to digital assets stored in the digital asset collection by accessing the knowledge graph and based on multiple metadata of the digital assets. For example, the selection of the associated multimedia content icon 206 for "animals" will result in filtering the digital assets to exclude any assets that do not contain metadata related to "animals". Similarly, the selection of the associated multimedia content icon 206 for "snow" or "sports" will result in filtering the digital assets to exclude any assets that do not contain metadata related to "snow" or "sports". The digital asset management module / logic component determines which multimedia content icons 206 and associated keyword tags 208 to display, and in what order to display the multimedia content icons 206 and associated keyword tags 208 by accessing the knowledge graph.

[0068] The social group collection 270 with the associated collection identifier 204 presents a plurality of multimedia content icons 206 and associated keyword tags 208, wherein each multimedia content icon 206 and associated keyword tag 208 represents a collection of digital assets stored in the digital asset collection. Each digital asset includes metadata associating the digital asset with a group of characters depicted in the multimedia content icon 206 and the associated keyword tag 208. Figure 2The user interface 200 depicted in the example multimedia content icon 206 presents an exemplary multimedia content icon that depicts multiple characters in the same image or includes a collage of images depicting multiple characters in the same image. Unlike most other collections, the group collection may not depict the keyword tag 208 associated with each group. The multimedia content icon 206 for each group may have multiple different images, which represent characters associated with each other in different digital assets contained in the group. The social group multimedia content icon 306 may have two to nine people associated with the group. In some embodiments, the multimedia content icon may present any one of the two to nine images depicted. The collage of multiple images in the multimedia content icon may depend on the number of digital assets responsive to the character metadata of the group. The selection of the associated multimedia content icon 206 for a group in the group will result in the return of digital assets containing all characters, and all asset collections containing all characters or associated with a social group composed of all these characters. The digital asset management module / logic component decides which multimedia content icons 206 are displayed for the group by accessing the knowledge graph, and in what order the multimedia content icons 206 are displayed.

[0069] Similar to user interface 100, user interface 200 allows for scrolling of multimedia content icons 206 in each collection in response to at least recognition of a gesture to display additional multimedia content icons 206. In some embodiments, the gesture may be received via a touch screen display of a computing device. In some embodiments, the gesture causes horizontal scrolling of multimedia content icons 206 and associated keyword tags 208. In some embodiments, a vertical gesture causes vertical scrolling of a digital asset search page, thereby providing display of additional collections.

[0070] Figure 3 Another example user interface for digital asset searching, specifically a suggestion page, is shown according to at least one embodiment. User interface 300 allows a user to further refine a search for a collection of digital assets. Similar to user interfaces 200 and 300, a search field 302 is depicted in an area of ​​the display. Upon selecting one of the multimedia content icons 306, the digital asset management module / logic component presents one or more additional search categories 314 that can be selected to further refine the search for digital assets. For example, in user interface 300, a group multimedia content icon is selected as the primary search category 312, resulting in the group icon being depicted in search field 302. The digital asset search module / logic component presents additional suggested search categories 314 that include the names of people, places, scenes, and motion pictures that are associated with the selected group 306 as identified by metadata. As shown in FIG. Figure 3 As shown, a "person" icon may be depicted next to each person's name. The place icon is located next to "Santa Clara, CA". The scene icon is located next to the "beach" keyword tag. There is an icon next to the "dynamic photo" icon, which indicates that there is a dynamic photo with related metadata. Dynamic photos record what happened 1.5 seconds before and after the user took the photo. The name of the person can be depicted from the metadata of the digital asset or the name of the person can be identified through the user's contact list. Selecting any of these characters will further limit the search of the digital asset collection and return only digital assets related to the selected designated person. For example, selecting "Marie Smith" will result in returning only digital assets identified as being associated with the person "Marie Smith". By accessing the knowledge graph, the digital asset management module / logic component determines which additional search categories and search category identifiers are displayed and the order in which they are displayed. Using this feature, a user may be able to search for assets associated with a specific person, even if the user forgets the name of the specific person but remembers that the person is associated with an individual group.

[0071] Figure 3The user interface 300 depicted in allows selection of one or more moment multimedia content icons 306. A moment is a collection of digital assets that are temporally and geographically related to each other. The user interface 300 may depict multiple moment multimedia suggestions 310. Each moment multimedia content icon 306 depicts a representative image from a digital asset associated with the moment and a keyword tag 308 identifier that identifies the location and date associated with the digital asset associated with the moment. The selection of any of these moment multimedia content icons 306 will further restrict the search for the digital asset collection and only return digital assets related to both the location and time restrictions associated with the moment. For example, selecting "Santa Clara, CA, February 2" will result in only returning digital assets associated with both the location metadata associated with "Santa Clara, CA" and the time metadata associated with "February 2". By accessing the knowledge graph, the digital asset management module / logic component determines which multimedia content icons 306 and associated keyword tags 308 are displayed, and in what order the multimedia content icons 306 and associated keyword tags 308 are displayed. In addition, by accessing the knowledge graph, the digital asset management module / logic component determines the scope of the temporal metadata to include a specific moment and the scope of the location metadata for identifying the location of the moment. Additional moments can be displayed by selecting the "Show All" option 332. Additional information about the generation of moments can be found in U.S. Patent Application Serial No. 15,391,276, filed on December 27, 2016, entitled "Knowledge Graph Metadata Network Based on Notable Moments," which is hereby incorporated by reference in its entirety for all purposes.

[0072] Figure 4Another example user interface for digital asset search according to at least one embodiment is shown, specifically another example of a suggestion page. User interface 400 allows a user to further refine the search for a digital asset collection. Similar to user interface 100, user interface 200, and user interface 300, a search field 402 is depicted in an area of ​​the display. When one of the multimedia content icons 406 is selected, the digital asset management module / logic component presents one or more additional search categories 414, which can be selected to further refine the search. For example, in user interface 400, a place (here, "Paris"), a primary search term 412 and a date (here, "2015"), a secondary search term 420 are selected and depicted in search field 402. As shown, both the place and date search terms can have associated icons. The selection of these primary search terms and secondary search terms further refines the search for digital assets in the digital asset collection to limit the return of digital assets with metadata associated with "Paris" and "2015". The digital asset search module / logic component further suggests additional additional search terms 414 to further refine the search. Additional search term icons 416 and associated search terms 418 include the names "Marie Smith," "home," "burst," and "video." Selection of any of these additional search icons 416 or search terms 418 will further limit the display of digital assets with metadata that are associated with the selected terms. For example, selecting the "Marie Smith" icon will result in further filtering of the digital asset collection to display only digital assets with metadata associated with "Paris," "2015," and "Marie Smith."

[0073] Similar to user interface 300, Figure 4The user interface 400 depicted in allows selection of one or more moment multimedia content icons. A moment is a collection of digital assets that are temporally and geographically related to each other. A plurality of moment multimedia content icons 410. Each moment multimedia content icon 406 depicts a representative image and a keyword tag 408 identifier from a digital asset associated with the moment, which identifies the location and date associated with the digital asset, which digital assets are associated with the moment. Selecting any of these moment multimedia content icons 406 will further limit the search for the digital asset collection and only return digital assets related to both the location and time restrictions associated with the moment. For example, selecting "Paris-Marais, 15th Anniversary" will result in only returning digital assets associated with both the location metadata associated with "Paris" and the time metadata associated with "December 15, 2015". The digital asset management module / logic decides which multimedia content icons 406 and associated keyword tags 408 to display, and in what order to display the multimedia content icons 406 and associated keyword tags 408 by accessing the knowledge graph. Furthermore, by accessing the knowledge graph, the digital asset management module / logic determines the scope of the temporal metadata to include a specific moment and the scope of the location metadata for identifying the location of the moment. Additional moments may be displayed by selecting the "Show All" option 432.

[0074] Figure 4 The user interface 400 depicted in FIG. 4 also allows the results to be further sorted by additional categories. The number and type of additional categories are determined by the digital asset search module / logic component and depend on the metadata associated with the digital assets stored in the digital asset collection. In the example, Figure 4 As shown, digital assets can be further filtered by date or location. For example, the digital asset search module / logic component presents additional multimedia content icons 406 and keyword tags 408 for additional search categories for the user to select. In this example, the digital asset management module / logic component determines the various dates and specific locations to be included in the category by accessing the knowledge graph.

[0075] Figure 5Another example user interface for digital asset search, specifically another example of a suggestion page, according to at least one embodiment is shown. User interface 500 allows a user to further refine a search for a collection of digital assets. Similar to user interface 100, user interface 200, user interface 300, and user interface 400, a search field 502 is depicted in an area of ​​the display. Upon selecting one of the multimedia content icons 506, the digital asset management module / logic component presents one or more additional search categories 514 that can be selected to further refine the search. For example, in user interface 500, a place (here "Paris") has been selected as a primary search term 512, a date (here "2015") has been selected as a secondary search term 520, and a person (here "Marie Smith") has been selected as a tertiary search term 522. These terms are displayed in search field 502. Each of the search terms is depicted with an associated icon. Selecting these primary terms, secondary search terms, and tertiary terms further refines the search for digital assets in the digital asset collection to limit the return of digital assets with metadata associated with "Paris", "2015", and "Marie Smith". The digital asset search module / logic component further suggests additional additional search terms 514 to further refine the search. Additional search term icons 516 and associated search terms 518 include the names "Jack Delaney", "Albert Brassier", "AnatolBoxeur", and the location "home". The selection of any of these additional search icons 516 or search terms 518 will further limit the display of digital assets with metadata associated with the selected terms. For example, selecting the "Jack Delaney" icon will result in further filtering of the digital asset collection to only display digital assets with metadata associated with "Paris", "2015", "Marie Smith", and "Jack Delaney".

[0076] User interface 500 also presents time of day and additional suggested search categories that function similarly to the time of day and additional search categories described for user interface 400. Figure 5 , the additional search categories for searching are “date”, “person”, and “place”. Scrolling down the user interface page 500 may present additional multimedia content icons 506 and associated keyword tags 508.

[0077] Figure 6Another example user interface for digital asset search, specifically an example results page, according to at least one embodiment is shown. User interface 600 allows a user to further refine a search for a collection of digital assets. Similar to user interface 100, user interface 200, user interface 300, user interface 400, and user interface 500, a search field 602 is depicted in an area of ​​the display. When "Jack Delaney" is selected as an additional search term 624, exemplary user interface 600 presents results from user interface 600. Here, the user interface only displays moments with metadata associated with "Paris", "2015", "MarieSmith", and "Jack Delaney". This illustrates how the selection of numerous search categories can narrow the search scope, perhaps to a single moment, making it easier for a user to find a specific digital asset in a collection with a few keystrokes. Exemplary user interface 600 depicts only a single moment identified with a keyword tag 608 "Paris - Trocadero Square, December 21, 2015" using an associated multimedia content icon 606. Selecting the moment multimedia content icon 606 will return the digital assets associated with that moment.

[0078] Figure 7 Another example user interface for digital asset search, specifically another example results page, is shown according to at least one embodiment. User interface 700 depicts a search result page from a search result page. Figure 6 The multimedia content icon 706 and the keyword tag 708 of the search "Paris-Trocadero Square" are described. Figure 7 As shown, a related search can be performed by selecting one of the multimedia content icons 706 for the collection of groups and people. This allows additional searches for digital assets related to the selected moment. For example, "Show nearby photos" allows the digital asset search module / logic component to search for digital assets with metadata stored on adjacent nodes.

[0079] Figure 8 Another example user interface for digital asset search, specifically an example of auto-completion, is shown according to at least one embodiment. User interface 800 allows a user to further refine a search for a collection of digital assets. Similar to other user interfaces disclosed herein, a search field 802 is depicted in a first area of ​​the display. User interface 800 shows a next keyword suggestion function. Figure 8As shown, a string of text (here, the letter "l") is entered into the search field 802. The digital asset search module / logic component returns a suggested search category 814 and an associated collection icon 816 and a keyword tag 818 to help suggest another search category. The user interface 800 also presents a moment of functioning similar to that described for the user interface 400 and a moment of additional search categories. For example, a multimedia content icon 806 and an associated keyword tag 808 may be depicted in the user interface 800. Figure 8 Also shown is a virtual keyboard 830 for entering text into the search field. The virtual keyboard 830 may be displayed by entering into the selected search field without first selecting the multimedia content icon 806. Additional moments may be displayed by selecting the "Show All" option 832.

[0080] Fig. 9 Another example user interface for digital asset search, specifically another example of a next keyword suggestion function, is shown according to at least one embodiment. User interface 900 allows a user to further refine a search for a collection of digital assets by entering text after selecting one or more search categories 922. Similar to other disclosed user interfaces herein, a search field 902 is depicted in an area of ​​the display. User interface 900 also shows a next keyword suggestion function. Fig. 9 As shown, after the character "Bao Thieu" search category, a string of text (here, the letters "ch") is entered into the search field 902. The digital asset search module / logic component returns a suggested search category 914 with an associated collection icon 916 and a keyword tag 918 to help suggest another search category. The user interface 900 also presents moments and additional search categories that function similarly to those described for the user interface 800. For example, the user interface 900 displays a moment additional content icon 906 and an associated keyword tag 906. Fig. 9 Also shown is a virtual keyboard 930 for entering text into the search field. Additional times may be displayed by selecting the "Show All" option 932.

[0081] Fig.10 Another example user interface for digital asset searching, specifically a suggestions page, is shown according to at least one embodiment. User interface 1000 allows a user to further refine a search for a collection of digital assets. Similar to user interfaces 300 and 400, a search field 1002 is depicted in an area of ​​the display. Upon selecting a category 1036, such as "dining," into search field 1002, the digital asset search technology will suggest possible search terms to return digital assets in response to category 1036. Suggested search terms 1022 provide the user with the ability to further refine the search based on metadata of the suggested search terms 1022. Fig.10 , the suggested search terms 1022 include: a person icon and "Sonia". The suggestion count 1040 indicates that there are 74 digital assets with metadata associated with "meals" and "Sonia". Selecting the "Sonia" search term will return digital assets associated with "meals" and "Sonia".

[0082] exist Fig.10 , example user interface 1000 suggests the following: the location "Santa Clara" with 1,606 digital assets and the location "Los Gatos" with 434 digital assets; the search term "dinner," with 2,001 digital assets; and a calendar for "winter," with 483 digital assets. Photo identifier 1024 lists the type of digital asset having metadata associated with search term entry 1018. Photo identifier 1024 lists the type of digital asset having metadata associated with the "dining" category 1036. Asset count number 1025 indicates the number of digital assets for photo identifier 1024. Fig.10 In the example of , there are 2,238 photos responsive to the category 1036 of "meals". One exemplary digital asset 1026 is a digital photo of a meal. Fig.10 Also depicted are exemplary digital asset thumbnails in two rows below the photo identifier 1024. Below the exemplary digital asset thumbnails is a virtual keyboard 1020 for entering text into the search field 1002.

[0083] In some embodiments, the top 8 features may display thumbnails 1038 of the top digital assets in response to the search query. In various embodiments, the top 8 thumbnail results are displayed as two rows, each row being four thumbnails. In some embodiments, the digital assets may be sorted chronologically first. In some embodiments, the chronological order may be from the oldest asset based on the asset creation date to the newest asset based on the asset creation date. In some embodiments, the chronological order may be reversed. In some embodiments, the digital asset management module / logic component may access the knowledge graph of the digital assets to classify the digital assets into clusters. If the number of digital assets is less than a set amount (e.g., 160 assets), the assets may be indexed into different numbers of groups, each with the same number of assets. For example, for the top 8 presentations with 160 digital assets, the digital asset management module / logic component may index the digital assets into eight groups, each with 20 digital assets. For another example, for the top 8 presentations and 80 assets, the digital asset management module / logic component may index the digital assets into eight groups, each with 10 assets. If the number of digital assets is greater than a set amount (e.g., 160 assets), multiple digital asset clusters may be used to sample the assets to reduce run time. For example, if there are 1,200 digital assets in response to a search request, the digital asset management module / logic component may find the top 8 asset clusters and divide the top 8 clusters into 8 groups, each containing 20 digital assets. Therefore, the greater the number of digital assets, the greater the number of digital assets that will not be sampled in the top 8 results. If there are less than a set number of digital assets (e.g., 160 digital assets), the technique samples all digital assets.

[0084] Digital asset management / logic can use the knowledge graph to determine if there is sufficient temporal distance between digital assets. For example, search results should not return multiple assets from the same cluster or event. This provides some diversity for the thumbnail display of digital assets.

[0085] In some embodiments, a set number of assets (e.g., 20 assets) from each index location may be sampled. The digital asset management / logic may access the content score of each sampled asset from the knowledge graph. In some embodiments, empirical rules and thresholds may be used to calculate content scores. For example, based on these rules, these techniques may select a range of possible scores, including: [0.00; 0.25] for junk digital assets, screenshots, documents, blurry assets, assets with very low aesthetic scores, and very short (e.g., only a few seconds) videos; [0.60; 0.75] for edited assets, assets that are frequently viewed or played, or outstanding assets such as panoramas or portraits; [0.75; 1.00] for digital assets marked as favorites, digital assets with excellent aesthetic scores, frequently shared, and digital assets with many people identified in the digital assets; and [0.50; 0.75] for all remaining digital assets. After assigning scores to digital assets, an exact score may be calculated within a specified range based on the number of people in the digital asset, faces in the digital asset, smiles in the digital asset, winks in the digital asset, the number of shares of the digital asset, and the type of digital asset (e.g., photo, video, screenshot, dynamic photo, burst, panorama, portrait, selfie, long exposure, screenshot, animation, slow motion, etc.) The highest scoring asset for each asset sample is available for display in the top 8 assets.

[0086] The digital asset with the highest content score in each index group may be selected as one of the top 8 thumbnails for display. In various embodiments, if there is a connection between two or more digital assets of the index group, the digital asset management / logic may access the aesthetic score of each digital asset as a tiebreaker for selecting one of the top 8 digital assets. In some embodiments, the aesthetic score may be calculated by assigning weights to multiple attributes of the digital asset. A deep neural network may be used to calculate the aesthetic score. In various embodiments, the technology curates a data set and then trains a model. The technology may first create a data set of pictures that are independent of the digital assets on the user's device. These techniques may train the model by asking human annotators to rate the global aesthetics and photo attributes of the picture (e.g., on a scale from 1 to 5) such as framing, focus, lighting, composition, timing, blur, etc. In various embodiments, there may be a total of 20 such attributes. Subsequently, the neural network may be trained to predict these attribute scores for other digital assets. The training process may take an image as input and predict the final aesthetic score based on the training. This formula may be abstracted by a deep learning model that maps the image to a predicted aesthetic score through multiple nonlinear transformations. Inference of this model on various devices can be done through a web editing application for on-device inference (e.g., Espresso).

[0087] In some embodiments, the search results may include a suggestion count 1040. The suggestion count may be determined by counting the digital assets or may be determined in response to the search results. The suggestion count may include multiple types of digital assets (e.g., photos, videos, panoramas, etc.) in the suggestion count. In some embodiments, the suggestion count may be displayed to the right of the suggested search category on the display and include the number of digital assets. In some embodiments, when multiple suggestions are collapsed together, the digital asset management / logic may deduplicate the digital assets to ensure that the suggestion count may represent the number of results that the user will see after selecting the search term.

[0088] Fig.11 An example user interface for digital asset search is shown, specifically Fig.10 1100. The user interface 1100 is scrolled downward using a gesture on the touch screen display of the electronic device to present an additional user interface 1100. In addition to depicting the search field 1102 in the area of ​​the display, the exemplary digital asset thumbnails in two rows below the photo identifier 1124, the user interface 1100 presents a moment indicator 1128. When a category 1136 such as "dining" is selected into the search field 1102, the digital asset search technology will display the moment in response to the category 1136. A moment is a collection of digital assets that are temporally and geographically related to each other. A plurality of moment multimedia content icons 1130 are also depicted in the user interface 1100. Each moment multimedia content icon 1130 depicts a representative image from the digital assets associated with the moment and a keyword tag 1132 identifier that identifies the location and date associated with the digital assets associated with the moment. Selection of any of these moment multimedia content icons 1130 will further limit the search of the digital asset collection and return only digital assets related to both the location and time restrictions associated with the moment. For example, selecting the moment content icon 1130 for “Cupertino Village” will return a digital asset having metadata associated with the moment for Cupertino Village captured on August 22, 2018. Fig.11 Additional moments are presented for "San Francisco Marriott Marquis" on July 27, 2018 and "San Francisco" on July 26, 2018. The digital asset management module / logic determines which multimedia content icons 1130 and associated keyword tags 1132 to display and in what order to display the multimedia content icons 1130 and associated keyword tags 1132 by accessing the knowledge graph. In addition, by accessing the knowledge graph, the digital asset management module / logic component determines the scope of the temporal metadata to include a specific moment and the scope of the location metadata for identifying the location of the moment.

[0089] User interface 1100 depicts a number of icons, allowing for additional organizational tools for managing a collection of digital assets. The icons may include a “Photos” icon 1110, a “Favorites” icon (labeled “Personal”) 1112, an “Albums” icon 1114, and a “Search” icon 1116. Fig.11 As shown, selecting the "Search" function. Selecting the "Album" icon will direct the user to the album feature of the digital asset. Selecting the "Collections" icon will direct the user to the "Collections" page. Selecting "Photos" will direct the user to a page that lists digital assets in chronological order.

[0090] Fig.12 An example process flow diagram for searching for digital assets in a digital asset collection of a computing device is shown. The search process 1200 shows a digital asset search module / logic component 1202, a knowledge graph 1204, an asset collection 1206, and a display 1208. The digital asset search module / logic component 1202, the knowledge graph 1204, the asset collection 1206, and the display 1208 indicate elements of a system that performs the processes listed below each heading. The digital asset search module / logic component 1202 may be stored in one or more memories and executed by the one or more processors of the computing device. The knowledge graph 1204 is a logical collection of metadata associated with digital assets. The knowledge graph 1204 establishes links and correlations between metadata, which can be used to generate keyword tags 1216 for searching digital assets in a digital asset collection. The display 1208 can be an LCD, OLED, AMOLED, Super AMOLED, TFT, IPS, or TFT-LCD that can be commonly found in computing devices. The display 1208 can be a touch screen display of a computing device.

[0091] Process 1200 is illustrated as a logical flow diagram, each operation of which represents a series of operations that can be implemented in hardware, computer instructions, or a combination thereof. In the context of computer instructions, an operation represents a computer-executable instruction stored on one or more computer-readable storage media that performs the operation when executed by one or more processors. Generally speaking, computer-executable instructions include routines, programs, objects, components, data structures, etc. that perform specific functions or implement specific data types. The order in which the operations are described is not intended to be construed as a limitation, and any number of the operations may be combined in any order and / or in parallel to implement the processes.

[0092] Some, any, or all of process 1200 (or any other process described herein or variations and / or combinations thereof) may be performed under the control of one or more computer systems configured with computer executable instructions and may be implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) that is executed collectively on one or more processors. The code may be stored on a computer-readable storage medium, such as in the form of a computer program that includes a plurality of instructions executable by one or more processors. The computer-readable storage medium may be non-transitory.

[0093] At box 1200, the search process 1210 is started by opening an asset page. The asset page can be opened by selecting an icon on the page of the computing device. The icon can be labeled "photo" or another similar descriptive term for a digital asset stored in the digital asset collection. Opening the asset page causes a user interface for searching digital assets to be displayed. The user interface can be presented on the display of the computing device. Next, at box 1212, the digital asset search module / logic component retrieves keyword tags associated with the digital assets stored in the digital asset collection. At box 1214, the knowledge graph is updated to include keyword tags for all digital assets stored in the asset collection 1206 at box 1216. As additional assets are added to the digital asset collection, the knowledge graph can also be updated to include new keyword tags associated with the new digital assets.

[0094] At box 1218, the digital search module / logic component selects a plurality of selected keyword tags 1220 for display. The knowledge graph prioritizes a series of keyword tags based on criteria. The digital asset collection may contain tens of thousands of assets, and each asset may have multiple metadata stored for each asset in the knowledge graph. In order to make it easier for users to search, the digital search module / logic component selects the best keywords that are most likely to be searched by users. The keyword tags can be divided into various sets (parts) related to the searchable dimensions in the digital asset collection. These sets (parts) may include people, locations, times, events. Next, at box 1222, the digital asset search module / logic component prepares the user interface for display. Select the selected keyword tag 1220 from the keyword tags 1216 in the asset collection 1206 stored in the memory of the computing device. This step involves selecting keywords suitable for the user and generating related multimedia content icons / user interface elements for display. The keyword tags selected for display may change with changes in the digital asset collection or user activities on the computing device.

[0095] At box 1224, the display presents a user interface for searching a collection of digital assets. A user interface is generated on the display of the computing device for viewing by the user. At box 1226, the display is configured to receive a user's selection of a search category. The user's selection of a search category can be performed by simply clicking one of the displayed multimedia content icons / user interface elements. Other means of selecting one of the multimedia content icons / user interface elements can be used by the user through a device button or other gestures. At box 1226, upon receiving the user's selection of a search category, the knowledge graph 1204 receives a keyword tag associated with the selected search category at box 1228. The digital asset search module / logic component uses the selected search category to identify metadata related to the assets in the digital asset collection. The metadata can be used to find digital assets, or can be used to further generate the suggested search terms to further limit the search for digital assets. At box 1230, the digital asset search module / logic component filters digital assets that are not related to the selected search category. At box 1232, the digital asset search module / logic component creates a modified asset collection. At box 1232, the modified asset set includes the selected keyword tag 1234 from the asset set 1206. The selected keyword tag 1234 is associated with the selected category from box 1226. Next, the digital asset search module / logic component prepares a second user interface at box 1236. The second user interface can present further search suggestions on the display to allow the user to further refine the search for digital assets. The second user interface can be one or more moments corresponding to the selected search category. These moments correspond to asset categories that correspond to the same geographic location and time range. The second user interface can display images from the search terms. At box 1240, the display 1208 presents the second user interface.

[0096] Fig.13 1300 is a flow chart illustrating a process 1300 for searching a collection of digital assets as described herein, according to at least one example. The process 1300 is illustrated as a logical flow chart, each operation of which represents a series of operations that can be implemented in hardware, computer instructions, or a combination thereof. In the context of computer instructions, an operation represents a computer-executable instruction stored on one or more computer-readable storage media that performs the operation when executed by one or more processors. In general, computer-executable instructions include routines, programs, objects, components, data structures, etc. that perform a specific function or implement a specific data type. The order in which the operations are described is not intended to be construed as a limitation, and any number of the operations may be combined in any order and / or in parallel to implement the processes.

[0097] Some, any, or all of process 1300 (or any other process described herein or variations and / or combinations thereof) may be performed under the control of one or more computer systems configured with computer executable instructions and may be implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) that is executed collectively on one or more processors. The code may be stored on a computer-readable storage medium, such as in the form of a computer program that includes a plurality of instructions executable by one or more processors. The computer-readable storage medium may be non-transitory.

[0098] At 1302 , process 1300 includes displaying a search field in a first area of ​​a display of a digital asset collection page of a computing device, the search field for receiving text associated with a digital asset collection of the computing device.

[0099] At 1304, process 1300 includes displaying at least one user interface element including a first keyword tag for at least one search category in a second area of ​​the display, the first keyword tag generated by a computing device based on a knowledge graph of a set of digital assets, wherein the user interface element represents at least one asset within the set of digital assets that is associated with the first keyword tag.

[0100] At 1306 , process 1300 includes receiving a selection of at least one user interface element of a desired search category corresponding to the first keyword tag.

[0101] At 1308 , process 1300 includes initiating a search of the computing device's collection of digital assets to search for relevant content of the desired search category.

[0102] At 1310, process 1300 includes displaying on a display a subset of the collection of digital assets according to a desired search category.

[0103] Fig.14 An example flow chart is shown that illustrates a process (e.g., a computer-implemented technique) 1400 for implementing a digital asset search, according to at least some embodiments. The process 1400 is shown as a logical flow chart, each operation of which represents a series of operations that can be implemented in hardware, computer instructions, or a combination thereof. In the context of computer instructions, an operation represents a computer-executable instruction stored on one or more computer-readable storage media that performs the operation when executed by one or more processors. In general, computer-executable instructions include routines, programs, objects, components, data structures, etc. that perform specific functions or implement specific data types. The order in which the operations are described is not intended to be construed as a limitation, and any number of the operations may be combined in any order and / or in parallel to implement the processes.

[0104] Some, any, or all of process 1400 (or any other process described herein, or variations and / or combinations thereof) may be performed under the control of one or more computer systems configured with computer executable instructions that execute collectively on one or more processors and may be implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications). The code may be stored on a computer-readable storage medium, such as in the form of a computer program that includes a plurality of instructions executable by one or more processors. The computer-readable storage medium may be non-transitory.

[0105] At 1402 , operations include generating a graphical user interface for searching a collection of digital assets on a computing device.

[0106] At 1404, operations include providing a search field in a first area of ​​a graphical user interface for receiving text associated with a collection of digital assets of a computing device; and

[0107] At 1406, the operation includes providing at least one user interface element including a first keyword tag for at least one search category in a second area of ​​the display, wherein the at least one user interface element is generated by the computing device based on a knowledge graph of the digital asset collection, and initiating a search of the digital asset collection of the computing device by activating the desired user interface element to search for relevant content of the desired search category.

[0108] Fig.15 1 is a simplified block diagram illustrating an example architecture 1500 for implementing the functionality described herein according to at least one embodiment. In some examples, a computing device 1502 having the example architecture 1500 can be configured to present a relevant user interface, capture audio and video information, search a collection of digital assets, display relevant results on a display, receive tactile input, receive touch screen input, and execute logic.

[0109] The computing device 1502 may be configured to execute or otherwise manage applications or instructions for performing the techniques described, such as, but not limited to, providing a user interface for searching for digital assets (e.g., Figures 1 to 11 1500). Computing device 1502 may receive input from one or more users 1508 at the user interface (e.g., using one or more I / O devices 1504, such as at touch screen 1506), capture information, process the information, and then present the asset also using one or more I / O devices 1504 (e.g., a speaker of computing device 1502). Computing device 1502 may be configured to search for data assets stored in data asset collection 1520.

[0110] The computing device 1502 may be any type of computing device, such as, but not limited to, a mobile phone (e.g., a smart phone), a tablet computer, a personal digital assistant (PDA), a laptop computer, a desktop computer, a thin client device, a smart watch, a wireless headset, etc. According to some embodiments, the computing device 1502 may be a portable multifunction device with a touch screen 1506. The touch screen optionally displays one or more graphics in a user interface (UI). In this embodiment, and in other embodiments described below, a user can select one or more of these graphics by, for example, making gestures on the graphics with one or more fingers or with one or more styluses. In some embodiments, when the user interrupts contact with one or more graphics, selection of one or more graphics will occur. In some embodiments, gestures optionally include one or more taps, one or more swipes (from left to right, from right to left, up and / or down) and / or rolling of fingers that have been in contact with the computing device 1502 (from right to left, from left to right, up and / or down). In some specific implementations or in some cases, inadvertent contact with a graphic will not select a graphic. For example, a swipe gesture that sweeps over an application icon may not select the corresponding application when the gesture corresponding to selection is a tap. Computing device 1502 may optionally also include one or more physical buttons, such as a "home" button or a menu button. Optionally, the menu button is used to navigate to any application in a set of applications that are optionally executed on computing device 1502. Alternatively, in some embodiments, the menu button is implemented as a soft key in a GUI displayed on touch screen 1506.

[0111] In one embodiment, the computing device 1502 includes a touch screen 1506, a menu button, a push button for turning the device on and off and locking the device, one or more volume adjustment buttons, a user identity module (SIM) card slot, a headset jack, and a docking / charging external port. The push button is optionally used to turn the device on / off by pressing the button and keeping the button in a depressed state for a predefined time interval; lock the device by pressing the button and releasing the button before the predefined time interval passes; and / or unlock the device or initiate an unlocking process. In an alternative embodiment, the device 1506 also accepts voice input for activating or deactivating certain functions through a microphone. The computing device 1502 also optionally includes one or more contact strength sensors for detecting the strength of contact on the touch screen 1506, and / or one or more tactile output generators for generating tactile output for a user of the device 1502.

[0112] In an exemplary configuration, the computing device 1502 may include at least one memory 1512 and one or more processing units (or one or more processors) 1514. The one or more processors 1514 may be suitably implemented in hardware, software, or a combination thereof. Computer executable instructions or firmware implementations of the one or more processors 1514 include computer executable instructions written in any suitable programming language to perform the various functions described.

[0113] The memory 1512 may store program instructions that can be loaded and executed on one or more processors 1514 and data generated during the execution of these programs. Depending on the configuration and type of the computing device 1502, the memory 1512 may be a volatile memory (such as a random access memory (RAM)) and / or a non-volatile memory (such as a read-only memory (ROM), flash memory, etc.). The computing device 1502 may also include additional removable storage devices and / or non-removable storage devices, including but not limited to magnetic storage devices, optical disks, and / or tape storage devices. The disk drive and its associated non-transitory computer-readable medium may provide a non-volatile storage device for computer-readable instructions, data structures, program modules, and other data for the computing device. In some specific implementations, the memory 1512 may include a variety of different types of memory, such as static random access memory (SRAM), dynamic random access memory (DRAM), or ROM. Although the volatile memory described herein may be referred to as RAM, any volatile memory that will not retain the data stored therein after being unplugged from the host and / or power supply is appropriate.

[0114] Memory 1512 and additional storage are removable or non-removable, and are examples of non-transient computer-readable storage media. For example, non-transient computer-readable storage media may include volatile or non-volatile, removable or non-removable media, which are implemented by any method or technology to store information such as computer-readable instructions, data structures, program modules or other data. Memory 1512 and additional storage devices are examples of non-transient computer storage media. Other types of computer storage media that may be present in computing device 1502 may include, but are not limited to: phase change RAM (PRAM), SRAM, DRAM, RAM, ROM, electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital video disk (DVD) or other optical storage device, cassette, tape, magnetic disk storage magnetic storage device or other magnetic storage device or any other medium that can be used to store the required information and can be accessed by computing device 1502. Any combination of the above should also be included in the scope of non-transient computer-readable storage media.

[0115] Alternatively, the computer-readable communication medium may include computer-readable instructions, program modules, or other data transmitted within a data signal such as a carrier wave or other transmission means. However, as used herein, computer-readable storage media does not include computer-readable communication media.

[0116] The computing device 1502 may also include a communication connection 1524 that allows the computing device 1502 to communicate with a data repository, another computing device or server, a user terminal, and / or other devices via one or more networks. Such networks may include any one or combination of many different types of networks, such as a wired network, the Internet, a wireless network, a cellular network, a satellite network, other private networks, and / or public networks, or any combination thereof. The computing device 1502 may also include one or more I / O devices 1504, such as a touch input device, a keyboard, a mouse, a pen, a voice input device, a display, a speaker, a printer, and the like.

[0117] Turning to the contents of memory 1512 in more detail, memory 1512 may include an operating system 1516 and / or one or more applications or services for implementing the functionality disclosed herein, including a user interface module 1518, a digital asset collection 1520, and a knowledge graph 1510. Memory 1512 may also be configured to store one or more audio and video files as digital assets in the digital asset collection 1520 and the knowledge graph 1510. In this way, computing device 1502 may perform all of the operations described herein.

[0118] In some examples, user interface module 1518 can be configured to manage a user interface of computing device 1502. For example, user interface module 1518 can present any number of various UIs requested by computing device 1502. In particular, user interface module 1518 can be configured to present Figures 1 to 11 The UIs 100 to 1100 enable the functionality described herein, specifically searching a collection of digital assets and providing a user 1508 with a simple way to search for digital assets.

[0119] The computing device 1502 also includes a graphical user interface (GUI) module 1518. The GUI module 1518 is used to output a signal of a display device associated with the client computing device 1502. The client computing device 1502 also includes a display interface for interacting with the display device. The client computing device 1502 can request data for a video information window from a media server (e.g., a media server) via a network interface. In one embodiment, the media server provides the requested data in XML format. The client computing device 1506 can process the requested data and cause the GUI module 1518 to present the video information window. In some embodiments, an object updater creates and updates objects used in a graphical user interface. For example, an object updater creates a new user interface object or updates the position of a user interface object. The GUI updater updates the GUI. For example, the GUI updater prepares display information and sends the display information to a graphics module to display on a display such as a touch-sensitive display.

[0120] In some embodiments, the digital asset management module / logic component obtains or generates a knowledge graph 1510 metadata network associated with a collection of digital assets (hereinafter referred to as a "knowledge graph"). The metadata network may include related metadata assets that describe features associated with digital assets in the digital asset collection. Each metadata asset may describe features associated with one or more digital assets in the digital asset collection. In a non-limiting example, a metadata asset may describe features associated with multiple digital assets in the digital asset collection. Each metadata asset may be represented as a node in the metadata network. A metadata asset may be associated with at least one other metadata asset. Each correlation between metadata assets may be represented as an edge between nodes representing related metadata assets in the metadata network. In some embodiments, the digital asset management module / logic component identifies a first metadata asset in the metadata network. The digital asset management module / logic component may also identify a second metadata asset based at least on the first metadata asset. In some embodiments, the digital asset management module / logic component causes one or more digital assets having a first metadata asset and / or a second metadata asset to be presented via an output device.

[0121] In some embodiments, the digital asset management module / logic component 1522 may enable the system to generate a knowledge graph 1510 of digital asset metadata and present it as a multidimensional network. The digital asset management module / logic component 1522 may obtain or receive a collection of digital asset metadata associated with a collection of digital assets. The digital assets stored in the collection of digital assets include, but are not limited to, the following: image media (e.g., still or animated images, etc.); audio media (e.g., digital sound files); text media (e.g., e-books, etc.); video media (e.g., movies, etc.); and tactile media (e.g., vibrations or movements provided in combination with other media, etc.). The above examples of digitized data may be combined to form multimedia (e.g., animated films, video games, etc.). A single digital asset refers to a single instance of digitized data (e.g., an image, a song, a movie, etc.).

[0122] As used herein, "metadata" and "digital asset metadata" refer collectively to information about one or more digital assets. Metadata can be: (i) a single instance of information about digitized data (e.g., a timestamp associated with one or more images, etc.); or (ii) a metadata grouping, which refers to a group consisting of multiple instances of information about digitized data (e.g., several timestamps associated with one or more images, etc.). There are different types of metadata. Each type of metadata describes one or more characteristics or attributes associated with one or more digital assets. Each metadata type can be classified as raw metadata or inferred metadata, as further described below.

[0123] In some embodiments, the digital asset management module / logic component 1522 may identify raw metadata associated with one or more digital assets within the digital asset metadata. In some embodiments, the digital asset management module / logic component 1522 may determine inferred metadata based at least on the raw metadata. As used herein, "raw metadata" refers to metadata that describes one or more features or attributes associated with one or more digital assets. That is, raw metadata includes obtained metadata that describes one or more digital assets. In some cases, the raw metadata may be extracted from the inferred metadata, as further described below.

[0124] The main raw metadata may include one or more of the following: time metadata, geolocation metadata; geographic location metadata; person metadata; scene metadata; content metadata; object metadata; and sound metadata. Time metadata refers to the time associated with one or more digital assets (e.g., a timestamp associated with a digital asset, the time when the digital asset was generated, the time when the digital asset was modified, the time when the digital asset was stored, the time when the digital asset was transmitted, the time when the digital asset was received, etc.). Geolocation metadata refers to geographic or spatial attributes associated with one or more digital assets using a geographic coordinate system (e.g., latitude, longitude, and / or altitude, etc.). Geographic location metadata refers to one or more meaningful locations associated with one or more digital assets, rather than geographic coordinates associated with the digital assets. Examples include beaches (and their names), street addresses, country names, regions, buildings, landmarks, etc. For example, geographic location metadata can be determined by processing geographic location information together with data from a map application to determine the geographic location of a scene in a set of images. Person metadata refers to at least one detected or known person associated with one or more digital assets (e.g., a known person in an image detected by facial recognition technology, etc.). Scene metadata refers to an overall description of an activity or situation associated with one or more digital assets. For example, if the digital assets include a set of images, the detected objects in the images can be used to determine the scene metadata of the set of images. For a more specific example, the presence of a large cake with candles and balloons in at least two images in the set can be used to determine that the scene of the set of images is a birthday celebration. Object metadata refers to one or more detected objects associated with one or more digital assets (e.g., detected animals, detected company logos, detected furniture, etc.). Content metadata refers to characteristics of digital assets (e.g., pixel characteristics, pixel intensity values, luminescence values, brightness values, loudness levels, etc.). Sound metadata refers to one or more detected sounds associated with one or more digital assets (e.g., the detected sound is a human voice, the detected sound is a fire truck siren, etc.).

[0125] Auxiliary primitive metadata includes, but is not limited to, the following: (i) conditions associated with capturing one or more digital assets; (ii) conditions associated with modifying one or more digital assets; and (iii) conditions associated with storing or retrieving one or more digital assets. As used herein, "inferred metadata" refers to additional information about one or more digital assets beyond the information provided by the original metadata. One difference between original metadata and inferred metadata is that original metadata represents an initial set of descriptions of one or more digital assets, while inferred metadata provides additional descriptions of one or more digital assets based on processing of one or more of the original metadata and contextual information. For example, original metadata can be used to identify detected people in a set of images, such as John Doe and Jane duo, and an inferred metadata can identify John Doe and Jane Doe as a married couple based on processing one or more original metadata (i.e., the initial set of descriptions and contextual information). In some embodiments, the inferred metadata is formed by at least one of the following: (i) a combination of different types of original metadata; (ii) a combination of different types of contextual information; (iii) or a combination of original metadata and contextual information. As used herein, "contacts" and variations thereof refer to any or all attributes of a user device that includes or has access to a collection of digital assets associated with a user, such as physical, logical, social, and / or other contact context information. As used herein, "context information" and variations thereof refer to metadata assets that describe or define the context of a user or the context of a user device that includes or has access to a collection of digital assets associated with a user. Exemplary context information includes, but is not limited to, the following: a predetermined time interval; an event scheduled to occur at a predetermined time interval; a geographic location visited at a predetermined time interval; one or more identified persons associated with a predetermined time; an event scheduled within a predetermined time, or a geographic location to be visited at a predetermined time; whether metadata describes whether a particular time period is associated (e.g., rainy, snowy, windy, cloudy, sunny, hot, cold, etc.); season-related metadata describing the season associated with the captured image. For some embodiments, the context information may be obtained from an external source, a social networking application, a weather application, a calendar application, and a contact book application, any other type of application, or from any type of data storage accessible via a wired or wireless network (e.g., the Internet, a private intranet, etc.).

[0126] The primary inferred metadata may include event metadata describing one or more events associated with one or more digital assets. For example, if the digital asset includes one or more images, the primary inferred metadata may include event metadata describing one or more events that captured the one or more images (e.g., vacation, birthday, sporting event, concert, graduation ceremony, dinner, project, fitness class, traditional festival, etc.). In some embodiments, the primary inferred metadata may be determined by clustering one or more primary original metadata, auxiliary original metadata, and contextual metadata. Auxiliary inferred metadata includes, but is not limited to, the following: (i) geographic location relationship metadata; (ii) character relationship metadata; (iii) object relationship metadata; space and (iv) sound relationship metadata. Geographic location relationship metadata refers to the relationship between one or more known characters associated with one or more digital assets and one or more meaningful locations associated with one or more digital assets. For example, an analysis engine or data conferencing technology can be used to determine that a scene associated with one or more images of John Doe represents John Doe's home. Character relationship metadata refers to the relationship between one or more known characters associated with one or more digital assets and one or more other known characters associated with one or more digital assets. For example, an analysis engine or data mining technology can be used to determine that Jane Doe (who appears in multiple images with John Doe) is John Doe's wife. Object relationship metadata refers to the relationship between one or more known objects associated with one or more digital assets and one or more known persons associated with one or more digital assets. For example, an analysis engine or data mining technique may be used to determine that a boat appearing in one or more images of John Doe is owned by John Doe. Sound relationship metadata refers to the relationship between one or more known sounds associated with one or more digital assets and one or more known persons associated with one or more digital assets. For example, an analysis engine or data mining technique may be used to determine that a voice appearing in one or more videos with John Doe is John Doe's voice.

[0127] As described above, inferred metadata can be determined or inferred from raw metadata and / or context information by performing at least one of the following: (i) data mining raw metadata and / or context information; (ii) analyzing raw metadata and / or context information; (iii) applying logical rules to raw metadata and / or context information; or (iv) any other known method for inferring new information from provided or acquired information. Moreover, raw metadata can be extracted from inferred metadata. For specific implementations, primary raw metadata (e.g., time metadata, geographic location metadata, scene metadata, etc.) can be extracted from primary inferred metadata (e.g., event metadata, etc.). The techniques for determining inferred metadata and / or extracting raw metadata from inferred metadata can be iterative. For the first example, inferred metadata can trigger the inference of other metadata, etc., and raw metadata from inferred metadata can trigger the inference of additional inferred metadata or the extraction of additional raw metadata.

[0128] The above-mentioned original metadata and inferred metadata are collectively referred to as digital asset metadata. In some embodiments, the digital asset maintenance module / logic component uses digital asset metadata to generate knowledge graph 1510. All or some metadata networks may be stored in one or more processing units and / or memories. As used herein, "knowledge graph", "knowledge graph metadata network", "metadata network" and their variants refer to a dynamically organized metadata collection that describes one or more digital assets (e.g., one or more digital asset groups in a digital asset collection, one or more digital assets in a digital asset collection, etc.) used for deductive reasoning by one or more computer systems. In the metadata network, there is no metadata that is only for digital assets (e.g., metadata associated with one or more digital asset groups, metadata associated with one or more digital assets, etc.). The metadata network is different from a database because the metadata network generally allows deep connections between metadata of multiple dimensions, which can be traversed for additional inference relevance. In the absence of loading a large number (e.g., hundreds, thousands, etc.) of database tables, this deductive reasoning is generally not feasible in conventional relational databases. Therefore, conventional databases may require a large amount of computing resources (e.g., external data storage, remote servers and their associated communication technologies, etc.) to perform deductive reasoning. In contrast, metadata networks can be viewed, manipulated, and / or stored with fewer computing resource requirements than the aforementioned database example. Furthermore, metadata networks are dynamic resources that have the ability to learn, grow, and adapt to new information added to them. This is different from databases, which are useful for accessing cross-reference information. While a database can be extended with additional information, it is still a tool for accessing cross-reference information placed into it. Metadata networks are not just about accessing cross-referenced information, but go far beyond that and involve extrapolation of data for inferring or determining additional data.

[0129] As explained in the preceding paragraphs, the metadata network enables deep connections between metadata using multiple dimensions in the metadata network, which can be traversed for additional inferred correlations. Each dimension in the metadata network can be considered as a metadata grouping based on metadata type. For example, a metadata grouping can be all time metadata assets in a metadata collection, and another grouping can be all geolocation metadata assets in the same metadata collection. Therefore, for this example, the time dimension refers to all time metadata assets in the metadata collection, and the geolocation dimension refers to all geolocation metadata assets in the same metadata collection. In addition, the number of dimensions can vary based on constraints. Constraints include, but are not limited to, the desired use of the metadata network, the desired level of detail, and / or the available metadata or computing resources for implementing the metadata network. For example, the metadata network may include only the time dimension, the metadata network may include all types of raw metadata dimensions, etc. With respect to the desired level of detail, each dimension can be further refined based on the specificity of the metadata. That is, each dimension in the metadata network is a metadata grouping based on the type of metadata and the granularity of the information described by the metadata. For the first example, there may be two time dimensions in the metadata network, wherein the first time dimension includes all time metadata assets classified by week, and the second time dimension includes all time metadata assets classified by month. For the second example, there may be two geolocation dimensions in the metadata network, where the first geolocation dimension includes all geolocation metadata assets categorized by establishment type (e.g., home, business, etc.), and the second geolocation dimension includes all geolocation metadata assets categorized by country. The foregoing examples are illustrative and non-limiting. It should be understood that the level of detail of the dimensions may vary depending on designer selection, application, available metadata, and / or available computing resources.

[0130] The digital asset management module / logic component 1522 can be configured to generate a metadata network as a multidimensional network of digital asset metadata. As used herein, "multidimensional network" and its variations refer to a complex graph with multiple relationships. A multidimensional network typically includes multiple nodes and edges. For one embodiment, the nodes represent metadata, and the edges represent the relationships or correlations between the metadata. Exemplary multidimensional networks include, but are not limited to, edge-labeled multigraphs, multi-part edge-labeled multigraphs, and multi-layer networks.

[0131] For one embodiment, the nodes in the metadata network represent metadata assets found in the digital asset metadata. For example, each node represents a metadata asset associated with one or more digital assets in a digital asset collection. For another example, each node represents a metadata asset associated with a set of digital assets in a digital asset collection. As used herein, "metadata assets" and variations thereof refer to metadata (e.g., a single instance of metadata, a set of multiple metadata instances, etc.) that describes one or more features of one or more digital assets in a digital asset collection. Thus, there may be original metadata assets, inferred metadata assets, primary original metadata assets, and exciting original metadata assets, primary inferred metadata assets, and / or exciting inferred metadata assets. For the first example, the original metadata asset refers to a time metadata asset that describes the time interval when one or more digital assets were captured between June 1, 2016 and June 3, 2016. For the second example, the original metadata asset refers to a geolocation metadata asset that describes one or more latitudes and / or longitudes at which one or more digital assets were captured. As another example, an inferred metadata asset is an event metadata asset that describes a vacation in Paris, France between June 5, 2016 and June 30, 2016 when one or more digital assets were captured.

[0132] In some embodiments, the metadata network includes two types of nodes: (i) moment nodes; and (ii) non-moment nodes. As used herein, a "moment" refers to a single event (as described by an event metadata asset) associated with one or more digital assets. For example, a moment refers to a vacation in Paris, France between June 1, 2016 and June 9, 2016. For example, the moment can be used to identify one or more digital assets (e.g., an image, a group of images, a video, a group of videos, a song, a group of songs, etc.) associated with a vacation in Paris, France between June 1, 2016 and June 9, 2016 (rather than associated with any other event). As used herein, a "moment node" refers to a node in a multidimensional network that represents a moment. Therefore, a moment node refers to a primary inferred metadata asset that represents a single event associated with one or more digital assets. Primary inferred metadata as described above. As used herein, a "non-moment node" refers to a node in a multidimensional network that does not represent a moment. Thus, a non-moment node refers to at least one of: (i) an original metadata asset associated with one or more digital assets; or (ii) an inferred metadata asset associated with one or more digital assets that are not moments (i.e., not event metadata assets).

[0133] As used herein, the term "event" in its various forms refers to an occurrence or activity that occurs at one or more locations during a specific time interval. Events include, but are not limited to, the following: a gathering of one or more people to perform an activity (e.g., a festival, vacation, birthday, dinner, project, fitness class, etc.); sporting events (e.g., sports games, etc.); ceremonies (e.g., rituals of cultural significance performed on special occasions, etc.); meetings (e.g., gatherings of individuals participating in some common interest, etc.); festivals (e.g., gatherings to celebrate some aspect of a community, etc.); concerts (e.g., art performances, etc.); media events (e.g., events created for publicity, etc.); and parties (e.g., large social or entertainment gatherings, etc.).

[0134] According to the embodiment, the knowledge graph 1510 can be generated by the processing system and used to perform digital asset management. Generating a metadata network by a digital asset management module / logic component can include defining nodes based on original metadata and / or inferred metadata associated with one or more digital assets in a digital asset set. When the digital asset management module / logic component 1522 utilizes metadata associated with a digital asset set to identify more original metadata and / or infer metadata at least from the original metadata, the digital asset management module / logic component 1522 can generate additional nodes to represent the original metadata and / or inferred metadata. In addition, when the digital asset management module / logic component 1522 determines the correlation between nodes, the digital asset management module / logic component 1522 can create edges between nodes. Two generation processes can be used to generate a metadata network. Use metadata assets that do not describe a moment (e.g., primary original metadata assets, auxiliary original metadata assets, and auxiliary inferred metadata assets, etc.) to initiate a first generation process. Use metadata assets that describe a moment (e.g., event metadata) to initiate a second generation process. Each of these generation processes is described below.

[0135] For the first generation process, the digital asset management module / logic component 1522 can generate non-time nodes to represent metadata associated with a user, consumer, or owner of a digital asset collection associated with a metadata network. For example, a user can be identified as Jean DuPont. In one embodiment, the digital asset management module / logic component 1522 generates non-time nodes to represent metadata provided by a user (e.g., Jean DuPont, etc.) via an input device. For example, a user can add at least some metadata about him or herself to the metadata network via an input device. In this way, the digital asset management module / logic component 1522 can use metadata to associate the user with other metadata obtained from the digital asset collection, for example, the metadata provided by the user Jean DuPont can include one or more of his birthplaces (Paris, France), his date of birth (May 27, 1991), his gender (male), his relationship status (married), his significant other or spouse (Marie Dupont) and his current residence (located in Key West, Florida, USA).

[0136] With respect to the first generation process, at least some metadata may be predicted based on processing performed by the digital asset management module / logic component. The digital asset management module / logic component 1522 may predict metadata based on analysis of a metadata access application or a metadata and data repository (e.g., a memory). For example, the digital asset management module / logic component 1522 may predict metadata based on analysis of information obtained by accessing a user's contacts (via a contacts application), activities (accounts or applications or organizational applications), contextual information (via sensors or peripherals), and / or social network data (via a social network application).

[0137] In some embodiments, the metadata includes, but is not limited to, other metadata such as the user's relationships with others (e.g., family members, friends, colleagues, etc.), the user's workplace (e.g., past workplace, current workplace, etc.), places visited by the user (e.g., previous places visited by the user, places that the user will visit, etc.). For one embodiment, metadata 210 may be used alone or in combination with other data to determine or infer at least one of the following: (i) vacations or trips taken by Jean Dupont; days of the week (e.g., weekends, holidays, etc.); locations associated with Jean Dupont; social groups of Jean Dupont (e.g., his wife Marie Dupont); professional groups or other groups of Jean Dupont (e.g., groups based on his profession, etc.); types of places visited by Jean Dupont (e.g., Prime 114 restaurant, home); activities performed (e.g., fitness classes, etc.); etc. The foregoing examples are illustrative and not limiting.

[0138] For the second generation process, the metadata network may include at least one moment node. For the second generation process, the digital asset management module / logic component 1522 generates a moment node to represent one or more primary inferred metadata assets (e.g., event metadata assets, etc.). The digital asset management module / logic component 1522 may determine or infer primary inferred metadata (e.g., event metadata assets, etc.) based on one or more information, metadata, or other data received from an external source (e.g., an application, a calendar application, a social networking application, an address book, etc.). Moreover, the digital asset management module / logic component 1522 may receive the primary inferred metadata assets, generate the metadata as a moment node, and extract primary raw metadata from the primary inferred metadata assets represented as the moment node.

[0139] The knowledge graph 1510 may be obtained from a memory. In addition or alternatively, a metadata network may be generated by a processing unit. When a first metadata asset (e.g., a time node, a non-time node, etc.) is identified in a multidimensional network representing a metadata network, a knowledge graph 1510 is created. For one embodiment, the first metadata may be represented as a time node. For this embodiment, the first metadata asset represents a first event associated with one or more digital assets. A second metadata asset is identified or detected based on at least the first metadata asset. In some embodiments, a second metadata asset may be identified or detected as a second node (e.g., a time node in a time node, etc.) based on a first node used to represent the first metadata asset in the metadata network, and the second metadata asset is represented as a second time node different from the first time node. This is because the first time node represents a first event metadata asset, which describes a first second event associated with one or more digital assets, wherein the second time node represents a second event metadata asset describing a second event associated with one or more digital assets.

[0140] In some embodiments, even if some metadata of the first metadata asset and the second metadata asset are different, identifying the second metadata asset (e.g., a moment node, etc.) can be performed by determining that the first metadata asset and the second metadata asset share a primary original metadata asset, a primary inferred metadata asset, an auxiliary original metadata asset, and / or an auxiliary inferred metadata asset. Further description of using or generating the knowledge graph 1510 can be found in U.S. patent application serial number 15,391,276, filed on December 27, 2016, entitled "Knowledge Graph Metadata Network Based on Notable Moments," which is hereby incorporated by reference herein in its entirety for all purposes.

[0141] The foregoing describes illustrative methods, computer-readable media, and systems for providing various techniques for adjusting audio and / or video content based at least in part on speech and / or facial feature characteristics. Some or all of these systems, media, and methods may, but need not, be at least partially provided by architectures and processes (such as at least those described above). Figures 1 to 11Although many embodiments are described above with reference to messaging applications, it should be understood that any of the above techniques can be used in any type of application, including real-time video playback or real-time video messaging applications. For purposes of explanation, many specific configurations and details are set forth in order to provide a thorough understanding of the examples. However, it will also be apparent to those skilled in the art that some examples can be implemented without these specific details. In addition, well-known features are sometimes omitted or simplified to prevent confusion with the examples described herein.

[0142] Various embodiments may also be implemented in various operating environments, which in some cases may include one or more user computers, computing devices, or processing devices that may be used to operate any of a number of applications. User devices or client devices may include any of a number of general-purpose personal computers, such as desktop computers or laptop computers running standard operating systems 1516, and cellular devices, wireless devices, and handheld devices running mobile software and capable of supporting multiple networking protocols and instant messaging protocols. This system may also include multiple workstations running various commercially available operating systems and any of other known applications for purposes such as development and database management. These devices may also include other electronic devices, such as virtual terminals, thin clients, gaming systems, and other devices capable of communicating via a network.

[0143] Most embodiments utilize at least one network familiar to those skilled in the art to support communications using any of a variety of commercial protocols such as TCP / IP, OSI, FTP, UPnP, NFS, CIFS, and AppleTalk. The network can be, for example, a local area network, a wide area network, a virtual private network, the Internet, an intranet, an extranet, a public switched telephone network, an infrared network, a wireless network, and any combination thereof.

[0144] In embodiments utilizing a web server, the web server may run any of a variety of server or middle-tier applications, including HTTP servers, FTP servers, CGI servers, data servers, Java servers, and business application servers. One or more servers may also be capable of executing programs or scripts in response to requests from user devices, such as by executing one or more applications, which may be implemented in any programming language such as One or more scripts or programs written in C, C# or C++ or any scripting language such as Perl, Python or TCL and their combinations. The one or more servers may also include a database server, including but not limited to and Those commercially available.

[0145] Environment may include various data repositories and other memory and storage media, as described above. These may reside in various locations, such as on a storage medium local to one or more computers or away from the storage medium of any or all computers on the network (and / or reside in one or more computers). In a specific set of embodiments, information may reside in a storage area network (SAN) familiar to those skilled in the art. Similarly, any necessary files for performing functions belonging to a computer, server or other network device may be stored locally and / or remotely as required. When the system includes a computerized device, each such device may include a hardware element electrically coupled via a bus, the element including, for example, at least one central processing unit (CPU), at least one input device (e.g., mouse, keyboard, controller, touch screen or keypad), and at least one output device (e.g., display device, printer or speaker). Such systems may also include one or more storage devices, such as disk drives, optical storage devices and solid-state storage devices such as RAM or ROM, and removable media devices, memory cards, flash memory cards, etc.

[0146] Such devices may also include a computer-readable storage medium reader, a communication device (e.g., a modem, a network card (wireless or wired), an infrared communication device, etc.), and a working memory as described above. A computer-readable storage medium reader may be connected to or configured to receive a non-transitory computer-readable storage medium representing a remote, local, fixed, and / or removable storage device, and a storage medium for temporarily and / or more permanently containing, storing, transmitting, and retrieving computer-readable information. The system and various devices will typically also include many software applications, modules, services, or other elements located within at least one working memory device, including an operating system 1516 and an application, such as a client application or a browser. It should be understood that alternative embodiments may have many variations according to those described above. For example, custom hardware may also be used, and / or specific elements may be implemented in hardware, software (including portable software, such as applets), or both. In addition, connections with other computing devices such as network input / output devices may be used.

[0147] Non-transitory storage media and computer-readable storage media for containing code or parts of code may include any suitable media known or used in the art (except transient media such as carrier waves), such as but not limited to volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information such as computer-readable instructions, data structures, program modules or other data, including RAM, ROM, electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, CD-ROM, DVD or other optical storage device, cassette, tape, disk storage device or other magnetic storage device, or any other medium that can be used to store the required information and can be accessed by system devices. Based on the disclosure and teachings provided herein, a person of ordinary skill in the art will understand other ways and / or methods of implementing various embodiments. However, as described above, computer-readable storage media do not include transient media such as carrier waves.

[0148] Accordingly, the specification and drawings are to be regarded in an illustrative rather than a restrictive sense. It will, however, be evident that various modifications and changes may be made thereto without departing from the broader spirit and scope of the present disclosure as set forth in the claims.

[0149] As described above, one aspect of the present technology is to collect and use data from various sources to search for digital assets. The present disclosure contemplates that, in some instances, these collected data may include personal information data that uniquely identifies or can be used to contact or locate a specific person. Such personal information data may include demographic data, location-based data, phone numbers, email addresses, twitter IDs, home addresses, data or records related to the user's health or fitness level (e.g., vital sign measurements, medication information, exercise information), date of birth, or any other identifying information or personal information.

[0150] The present disclosure recognizes that the use of such personal information data in the present technology can be used to benefit the user. For example, the personal information data can be used to search and display digital assets and information about digital assets. Therefore, the use of such personal information data can be presented to the user on a display. In addition, the present disclosure also anticipates other uses of personal information data that benefit the user. For example, health and fitness data can be used to provide insights into the user's overall health status, or can be used as positive feedback to individuals who use technology to pursue health goals.

[0151] This disclosure envisions that entities responsible for collecting, analyzing, disclosing, transmitting, storing or otherwise using such personal information data will comply with established privacy policies and / or privacy practices. Specifically, such entities should implement and adhere to privacy policies and practices that are recognized as meeting or exceeding industry or government requirements for maintaining the privacy and security of personal information data. Such policies should be easily accessible to users and should be updated as the collection and / or use of data changes. Personal information from users should be collected for legitimate and reasonable purposes of the entity and not shared or sold outside of these legitimate uses. In addition, such collection / sharing should be performed after receiving informed consent from the user. In addition, such entities should consider taking any necessary steps to defend and secure access to such personal information data and ensure that others who have access to personal information data comply with their privacy policies and processes. In addition, such entities may subject themselves to third-party assessments to demonstrate their compliance with widely accepted privacy policies and practices. In addition, policies and practices should be adjusted to specific types of personal information data collected and / or accessed, and to applicable laws and standards including specific considerations of jurisdiction. For example, in the United States, the collection or access of certain health data may be governed by federal and / or state laws, such as the Health Insurance Portability and Accountability Act (HIPAA), while health data in other countries may be subject to other regulations and policies and should be handled accordingly. Therefore, different privacy practices should be maintained in each country for different types of personal data.

[0152] Regardless of the foregoing, the present disclosure also contemplates implementation schemes in which users selectively block the use or access of personal information data. That is, the present disclosure contemplates that hardware elements and / or software elements may be provided to prevent or block access to such personal information data. For example, with respect to digital asset search technology, the technology of the present invention may be configured to allow users to choose to "opt in" or "opt out" at any time during or after registration for a service to participate in the collection of personal information data. As another example, a user may choose not to provide personal information to be displayed on a display. As another example, a user may choose to limit the amount of personal data to maintain or completely prohibit the display of personal data. In addition to providing "opt-in" and "opt-out" options, the present disclosure contemplates providing notifications related to access or use of personal information. For example, a user may be notified that their personal information data will be accessed when downloading an application, and then the user may be reminded again just before the personal information data is accessed by the application.

[0153] In addition, it is an object of the present disclosure that personal information data should be managed and processed to minimize the risk of unintentional or unauthorized access or use. Risks can be minimized by limiting data collection and deleting data once it is no longer needed. In addition, and when applicable, including in certain health-related applications, data de-identification can be used to protect the privacy of users. De-identification can be facilitated by removing specific identifiers (e.g., date of birth, etc.), controlling the amount or specificity of stored data (e.g., collecting location data at the city level rather than at the address level), controlling how data is stored (e.g., aggregating data between users), and / or other methods when appropriate.

[0154] Thus, while the present disclosure broadly covers the use of personal information data to implement one or more of the various disclosed embodiments, the present disclosure also contemplates that various embodiments may also be implemented without access to such personal information data. That is, various embodiments of the present technology will not fail to function properly due to the lack of all or a portion of such personal information data. For example, content may be selected and delivered to a user by inferring preferences based on non-personal information data or an absolute minimum amount of personal information, such as content requested by a device associated with the user, other non-personal information available to the boundary path technology, or publicly available information.

[0155] Other variations are within the spirit of the present disclosure. Thus, while the disclosed technology is susceptible to various modifications and alternative constructions, certain illustrative embodiments thereof are shown in the drawings and have been described in detail above. However, it should be understood that it is not intended to limit the present disclosure to the particular forms disclosed, but on the contrary, it is intended to cover all modifications, alternative constructions, and equivalent forms that fall within the spirit and scope of the present disclosure as defined by the appended claims.

[0156] The use of the terms "one", "an" and "the" and similar indicators in the context of describing the disclosed embodiments (particularly in the context of the claims below) will be interpreted as covering the singular and plural, unless otherwise stated or clearly contradicted by the context. Unless otherwise stated, the terms "comprise", "have", "include" and "contain" should be interpreted as open terms (i.e., meaning "including but not limited to"). The term "connected" is interpreted as being partially or completely included, attached or joined together even if there are intervening things. The phrase "based on" should be understood as open, and is not limited in any way, and is intended to be interpreted or otherwise understood as "at least partially based on" where appropriate. Unless otherwise stated herein, the description of the numerical range herein is intended only to be used as a simple method of individually referring to each individual value falling within the range, and each individual value is incorporated into the specification, as if quoted separately in this article. All methods described herein can be performed in any suitable order, unless otherwise stated herein or otherwise clearly contradicted by the context. Unless otherwise stated, the use of any and all examples or exemplary language (e.g., "such as") provided herein is intended only to better illustrate the embodiments of the present disclosure and will not limit the scope of the present disclosure. No language in the specification should be construed as indicating that any unclaimed element is essential to the practice of the present disclosure.

[0157] Unless otherwise specifically stated, disjunctive language such as the phrase "at least one of X, Y, or Z" is understood in context as generally used to present an item, term, etc., which can be X, Y, or Z, or any combination thereof (e.g., X, Y, and / or Z). Thus, such disjunctive language is generally not intended to, and should not, imply that certain embodiments require that at least one of X, at least one of Y, or at least one of Z each be present. Additionally, conjunctive language such as the phrase "at least one of X, Y, and Z" is also understood to mean X, Y, Z, or any combination thereof, including "X, Y, and / or Z," unless otherwise specifically stated.

[0158] Various embodiments of the present disclosure are described herein, including the best mode known to the inventor for carrying out the present disclosure. After reading the foregoing description, variations of those embodiments may become apparent to those of ordinary skill in the art. The inventor expects that the technician will appropriately adopt such variations, and the inventor intends to practice the present disclosure in a manner different from that specifically described herein. Therefore, as permitted by applicable law, the present disclosure includes all modifications and equivalents of the subject matter recorded in the appended claims. In addition, unless otherwise noted herein or clearly contradictory to the context, the present disclosure includes any combination of all possible variations of the above elements.

[0159] All references, including publications, patent applications, and patents, cited herein are hereby incorporated by reference to the same extent as if each reference were individually and specifically indicated to be incorporated by reference and were set forth in its entirety herein.

Claims

1. A computing device, comprising: one or more memories; one or more processors in communication with the one or more memories; and A display having a graphical user interface, the display being configured to: displaying a search field in a first area of ​​the display of a digital asset collection page of the computing device, the search field for receiving text associated with the digital asset collection of the computing device; displaying, in a second area of ​​the display, at least one representative multimedia icon including a first keyword tag for at least one search category, the first keyword tag being derived by the computing device from a knowledge graph of the collection of digital assets, wherein the representative multimedia icon represents at least one asset within the collection of digital assets associated with the first keyword tag; receiving a selection of the at least one representative multimedia icon corresponding to a desired search category for the first keyword tag; initiating a search of the digital asset collection of the computing device to search for relevant digital assets of the desired search category; displaying on the display a subset of the collection of digital assets according to the desired search category; displaying at least one suggested search term on the graphical user interface based at least in part on the knowledge graph of the collection of digital assets and the determined relevance between the first metadata set of the desired search category and the second metadata set of the digital assets; displaying a second keyword tag associated with the at least one suggested search term; In response to receiving a selection of the at least one suggested search term, further filtering the digital assets in the set of digital assets to exclude specific digital assets not associated with the at least one suggested search term, the further filtering creating a further modified set of digital assets; and The further modified set of assets is displayed on the display.

2. The computing device of claim 1, wherein the at least one suggested search term comprises identifying text and a collection icon, wherein the collection icon identifies a collection of the at least one suggested search term in the collection of digital assets.

3. The computing device of claim 1 , further comprising: displaying at least one suggested search term on the graphical user interface based at least in part on a correlation between one or more text entries in the search field and the metadata set for the digital asset; displaying a second keyword tag associated with the at least one suggested search term; further filtering the digital assets in the set of digital assets to exclude specific digital assets not associated with the at least one suggested search term, the further filtering creating a further modified set of digital assets; and The further modified set of assets is displayed on the display.

4. The computing device of claim 1 , further comprising: and automatically completing a text entry of a portion of a search term entered into a search field depicted in an area of ​​the user interface, wherein the auto-completion is based at least in part on metadata stored in a knowledge graph for the collection of digital assets and weighted criteria that consider at least one of: the number of said digital assets of said keyword tag that match the completed search term; the location of a matched term in the keyword tag for a multi-term keyword tag, wherein the matched term matches the completed search term; an asset category of the keyword tag corresponding to the completed search term; a position of the keyword tag for the completed search term, wherein the position is in a ranking of the keyword tag within a selected asset category; or the number of matching keyword tags for the completed search term; and The completed search term in the search field is displayed on the user interface.

5. The computing device of claim 1 , further comprising the graphical user interface displaying one or more multimedia icons in a plurality of sets determined at least in part by the keyword tags associated with the multimedia icons, wherein each set is displayed in a separate area of ​​the display. The computing device of claim 5 , wherein the collection comprises events, people, places, categories, and groups. 7 . The computing device of claim 5 , further comprising the graphical user interface scrolling a list of multimedia icons in the set of at least one multimedia icons in response to at least recognizing a gesture.

8. The computing device of claim 1, wherein the multimedia icon comprises a depiction of a representative asset in the collection of digital assets that corresponds to the first keyword tag.

9. A method for displaying a graphical user interface on a computing device having a display, comprising: displaying a search field in a first area of ​​the display of a digital asset collection page of the computing device, the search field for receiving text associated with the digital asset collection of the computing device; displaying, in a second area of ​​the display, at least one user interface element including a first keyword tag for at least one search category, the first keyword tag generated by the computing device from a knowledge graph of the collection of digital assets, wherein the user interface element represents at least one asset within the collection of digital assets that is associated with the first keyword tag; receiving a selection of the at least one user interface element of a desired search category corresponding to the first keyword tag; Initiating a search of the collection of digital assets of the computing device to search for content related to the desired search category; displaying on the display a subset of the collection of digital assets according to the desired search category; displaying suggested search terms on the graphical user interface based at least in part on the knowledge graph of the collection of digital assets and the determined relevance between the first metadata set of the desired search category and the second metadata set of the digital assets; displaying a second keyword tag associated with the suggested search term; In response to receiving a selection of the suggested search term, further filtering the digital assets in the set of digital assets to exclude specific digital assets not associated with the suggested search term, the further filtering creating a further modified set of digital assets; and The further modified set of assets is displayed on the display.

10. The method of claim 9, wherein the suggested search terms include identifying text and a collection icon, wherein the collection icon identifies a collection of the suggested search terms in the collection of digital assets.

11. The method according to claim 9, further comprising: displaying suggested search terms on the graphical user interface based at least in part on a correlation between one or more text entries in the search field and a metadata set for the digital asset; displaying a second keyword tag associated with the suggested search term; further filtering the digital assets in the set of digital assets to exclude specific digital assets not associated with the suggested search terms, the further filtering creating a further modified set of digital assets; and The further modified set of assets is displayed on the display.

12. The method according to claim 9, further comprising: and automatically completing a text entry of a portion of a search term entered into a search field depicted in an area of ​​the user interface, wherein the auto-completion is based at least in part on metadata stored in a knowledge graph for the collection of digital assets and weighted criteria that consider at least one of: the number of said digital assets of said keyword tag that match the completed search term; the location of a matched term in the keyword tag for a multi-term keyword tag, wherein the matched term matches the completed search term; an asset category of the keyword tag corresponding to the completed search term; a position of the keyword tag for the completed search term, wherein the position is in a ranking of the keyword tag within a selected asset category; or the number of matching keyword tags for the completed search term; and The completed search term in the search field is displayed on the user interface.

13. The method of claim 9 further comprises the graphical user interface displaying one or more multimedia icons in a plurality of sets determined at least in part by the keyword tags associated with the user interface elements, wherein each set is displayed in a separate area of ​​the display.

14. A computer-readable medium storing a plurality of instructions that, when executed by one or more processors of a computing device having a display, configure the one or more processors to perform operations comprising: generating a graphical user interface for searching a collection of digital assets on the computing device; providing a search field in a first area of ​​the graphical user interface, the search field for receiving text associated with the collection of digital assets of the computing device; providing at least one user interface element including a first keyword tag for at least one search category in a second area of ​​the display, wherein the at least one user interface element is generated by the computing device based on a knowledge graph of the collection of digital assets, and initiating a search of the collection of digital assets of the computing device for relevant content of the desired search category by activating the desired user interface element; displaying suggested search terms on the graphical user interface based at least in part on the knowledge graph of the collection of digital assets and the determined relevance between the first metadata set of the desired search category and the second metadata set of the digital assets; displaying a second keyword tag associated with the suggested search term; In response to receiving a selection of the suggested search term, further filtering the digital assets in the set of digital assets to exclude specific digital assets not associated with the suggested search term, the further filtering creating a further modified set of digital assets; and The further modified set of assets is displayed on the display.

15. The computer-readable medium of claim 14, wherein the suggested search terms include identifying text and a collection icon, wherein the collection icon identifies a collection of the suggested search terms in the collection of digital assets.

16. The computer-readable medium of claim 14, further comprising: displaying suggested search terms on the graphical user interface based at least in part on a correlation between one or more text entries in the search field and a metadata set for the digital asset; displaying a second keyword tag associated with the suggested search term; further filtering the digital assets in the set of digital assets to exclude specific digital assets not associated with the suggested search terms, the further filtering creating a further modified set of digital assets; and The further modified set of assets is displayed on the display.

17. The computer-readable medium of claim 14, further comprising: and automatically completing a text entry of a portion of a search term entered into a search field depicted in an area of ​​the user interface, wherein the auto-completion is based at least in part on metadata stored in a knowledge graph for the collection of digital assets and weighted criteria that consider at least one of: the number of said digital assets of said keyword tag that match the completed search term; the location of a matched term in the keyword tag for a multi-term keyword tag, wherein the matched term matches the completed search term; an asset category of the keyword tag corresponding to the completed search term; a position of the keyword tag for the completed search term, wherein the position is in a ranking of the keyword tag within a selected asset category; or the number of matching keyword tags for the completed search term; and The completed search term in the search field is displayed on the user interface.

Images