Automatic digital asset sharing suggestions

Abstract

Techniques for digital asset management (DAM) are described. A DAM system may obtain a knowledge graph metadata network that describes relationships between metadata associated with a user's collection of digital assets (DAs), such as images, videos, music, and the like. Based on information obtained, for example, from a user's collection of DAs and / or the knowledge graph metadata network, the DAM system may provide the user with smarter (and automated) DA sharing suggestions that are as relevant as possible for a given context. In some embodiments, the sharing suggestions may be based on one or more DAs from third parties that were recently shared with the user. In other embodiments, proactive sharing suggestions may be presented to the user based on detected indications of intent to share the DA, such as based on extracting relevant features from an incoming message from a third party (or an outgoing message from the user to a third party).

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This patent application claims priority to U.S. Provisional Patent Application No. 62 / 668,077, filed on May 7, 2018, entitled “Automatic Digital AssetSharing Suggestions” (“Patent Application No. 077”). This patent application is related to the following patent applications: (i) U.S. Non-Provisional Patent Application No. 15 / 391,269, filed on December 27, 2016, entitled “Notable Moments in a Collection of Digital Assets” (“269 Patent Application”); (ii) U.S. Non-Provisional Patent Application No. 15 / 391,276, filed on December 27, 2016, entitled “Knowledge Graph Metadata Network Based on Notable Moments” (“276 Patent Application”); (iii) U.S. Non-Provisional Patent Application No. 15 / 391,280, filed on December 27, 2016, entitled “Relating Digital Assets Using Notable Moments” (“280 Patent Application”); and (iv) U.S. Non-Provisional Patent Application No. 14 / 733,663, filed on June 8, 2015, entitled “Using Locations to Define Moments” (“663 Patent Application”). Each of the foregoing patent applications is incorporated herein by reference in its entirety. Technical Field

[0003] Embodiments described herein relate to digital asset management (also referred to as DAM). More specifically, embodiments described herein relate to organizing, storing, describing, and / or retrieving digital assets (also referred to herein as "DAs") so that they can be presented to a user of a computing system, for example, in the form of a suggestion to share one or more DAs from a collection of DAs with one or more third parties based on contextual analysis. Background Art

[0004] Modern consumer electronics have enabled users to create, purchase, and accumulate large numbers of digital assets or "DAs." For example, a computing system (e.g., a smartphone, a stationary computer system, a portable computer system, a media player, a tablet computer system, a wearable computer system or device, etc.) may store or have access to a collection of digital assets (also referred to as a DA collection) that includes hundreds or thousands of DAs (e.g., images, videos, music, etc.).

[0005] Managing DA collections can be a resource-intensive exercise for users. For example, retrieving multiple DAs representing important moments or events in a user's life from a fairly large DA collection may require the user to sift through many unrelated DAs. This process can be difficult and unpleasant for many users. A digital asset management (DAM) system can help manage DA collections. A DAM system represents an interwoven system that combines software, hardware, and / or other services to manage, store, ingest, organize, and retrieve DAs in a DA collection. For at least one commonly available DAM system, an important building block is a database. A database includes a collection of data organized into schemas, tables, queries, reports, views, and other objects. Exemplary databases include relational databases (e.g., tabular databases, etc.), distributed databases that can be dispersed or replicated between different points in a network, and object-oriented programming databases that can be consistent with data defined in object classes and subclasses.

[0006] However, one problem associated with using databases for digital asset management is that DAM systems can become resource intensive in terms of storage, management, and updating. That is, a large amount of computing resources may be required to manage the DAs in a DA collection (e.g., processing power to perform queries or transactions, storage memory space to store the necessary databases, etc.). Another related problem associated with using databases is that DAM cannot be easily implemented on computing systems with limited storage capacity (e.g., portable or personal computing systems such as smart phones or wearable devices) without directly managing the assets. Therefore, the functionality of the DAM system is often provided by a remote device (e.g., an external data repository, an external server, etc.), where a copy of the DA is stored and the results are transmitted back to the computing system with limited storage capacity.

[0007] Therefore, according to some DAM implementations, the DAM may also include a knowledge graph metadata network (also referred to herein as a "knowledge graph" or "metadata network") associated with a collection of digital assets (i.e., a DA collection). The metadata network may include related metadata assets that describe characteristics associated with digital assets in the DA collection. Each metadata asset may describe characteristics associated with one or more DAs in a digital asset (DA) collection. For example, a metadata asset may describe characteristics associated with multiple DAs in a DA collection, such as the location, day of the week, event type, etc. of one or more associated DAs. Each metadata asset may be represented as a node in the metadata network. A metadata asset may be associated with at least one other metadata asset. Each correlation between metadata assets may be represented as an edge between nodes in the metadata network representing related metadata assets. According to some implementations, the metadata network may define multiple types of nodes and edges based on the needs of a given specific implementation, for example, each node and edge has its own properties.

[0008] In addition to the aforementioned difficulties that users may face in managing large DA collections (e.g., locating and / or retrieving multiple DAs that represent important moments or events in the user's life), users may have difficulty determining (or be unable to take the time to determine) which DAs would make sense to share with third parties (e.g., other users of similar DAM systems and / or the user's social contacts). Further, users may have difficulty determining which third parties may be interested in which DAs and which events in the user's life may be of interest (or may not even recognize which third parties may be interested). Therefore, there is a need for methods, apparatus, computer-readable media, and systems that provide users with more intelligent and automated DA sharing suggestions based, for example, on contextual analysis of the user's DA collection and / or the nature of the user's relationship with one or more third parties with whom the user may wish to share DAs. Summary of the invention

[0009] Methods, apparatus, computer-readable media, and systems for providing smarter and automated DA sharing suggestions to users are described herein. Such embodiments enable sharing of DAs from a user's DA collection in an intelligent (e.g., context-aware) and user-friendly (e.g., automated) manner, while leveraging information provided in a knowledge graph metadata network describing the user's DA collection (and / or from other information sources) to make the DA sharing suggestions as relevant as possible for a given context, and sufficiently salient / compelling that the user may actually decide to share the suggested DA.

[0010] For one embodiment, a process is described that includes obtaining a metadata set associated with a user's DA set. In addition to obtaining information describing the DA set, the process may also obtain a knowledge graph metadata network for the DA set. Within the DA set, one or more unique "moments" may be identified based at least in part on the knowledge graph metadata network (as will be further described below). Because each moment may be associated with one or more digital assets, the process may next determine one or more of the associated digital assets for at least one identified moment to suggest sharing with one or more third parties. The determination of which third parties to suggest sharing with may be informed by the potential relationship of the one or more third parties to the at least one identified moment (e.g., whether the third party appears in the DA associated with the moment, whether the third party is in a particular social group with the user, etc.). Finally, the process may provide a suggestion to the user to share the determined one or more associated digital assets with the one or more third parties. After or in response to receiving an indication from the user as to which of the determined one or more associated digital assets are to be shared with the one or more third parties, the process may continue to share the determined one or more associated digital assets with the one or more third parties, for example, by sending the DA directly to the third party (e.g., via email, text message, instant message, or other proximity-based communication protocol, etc.), or indirectly, such as via a server holding a copy or reference to the DA.

[0011] For another embodiment, identifying relevant moments for sharing DAs from the user's DA set may be based on one or more DAs (and / or associated metadata) received from a third party, such as a DA recently received from the third party, such as in a message thread. Specifically, a process may identify one or more moments within the user's DA set to "share back" to the third party based at least in part on the user's knowledge graph and the one or more DAs received from the third party. The identification may include analyzing location and time metadata for the one or more DAs received from the third party, and performing a search against the user's knowledge graph using the received metadata for the DAs shared from the third party. In some embodiments, the search against the user's knowledge graph may include a "fuzzy" search, for example, by matching DAs from a larger time window and / or a larger geographic area than the DA originally shared by the third party and / or by matching DAs associated with moments that the knowledge graph is able to infer are related to the moments that matched the initial search for the user's DA set, thereby allowing for inexact matching of DAs in the DA set. Next, one or more of the digital assets associated with the matching moments from the user's DA set may be determined to share back to one or more third parties. The determination of which DAs to share back may also be informed by the exact DA that was originally shared by the third party and / or the relationship of the third party to the at least one identified match moment. Finally, the process may provide a suggestion to the user to share the determined one or more associated digital assets with the initially sharing third party. After or in response to receiving an indication from the user which of the determined one or more associated digital assets are to be shared with the third party, the process may continue to share the determined one or more associated digital assets with the third party.

[0012] For yet another embodiment, a process is described that includes obtaining a metadata set associated with a set of digital assets, wherein the set of digital assets includes one or more moments, and wherein each of the one or more moments is associated with one or more digital assets in the set of digital assets. In addition to obtaining information describing the DA set, the process may also obtain a knowledge graph metadata network for the DA set. The process may then receive an incoming message from a sender via a first device, detect a sharing intent in the incoming message, and then extract one or more features from the content of the incoming message. Based on a comparison of the one or more extracted features with the one or more moments of the set of digital assets and the knowledge graph metadata network, the process may then determine at least one moment of the one or more moments that matches the one or more extracted features, and one or more digital assets of the digital assets associated with the at least one moment to share with the sender in response to the incoming message. Finally, the process may provide a suggestion to the user via the first device to share the determined one or more associated digital assets with the sender. After receiving an indication from the user as to which of the determined one or more associated digital assets are to be shared with the sender or in response to receiving an indication from the user as to which of the determined one or more associated digital assets are to be shared with the sender, the process may continue to share the determined one or more associated digital assets with the sender.

[0013] Other features or advantages attributable to the embodiments described herein will be apparent from the accompanying drawings and from the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The embodiments described herein are illustrated by way of example and not limitation in the accompanying drawings, in which like reference numerals represent similar features. In addition, some conventional details have been omitted from the drawings so as not to obscure the inventive concepts described herein.

[0015] Figure 1A An asset management processing system including electronic components for performing digital asset management (DAM) according to one embodiment is shown in block diagram form.

[0016] Figure 1B An example of a moment view user interface for presenting a collection of digital assets based on the moment period during which the digital assets were captured is shown according to one embodiment.

[0017] Figure 2A The sharing of a set of DAs from a first user to multiple DAs of a second user is shown according to one embodiment.

[0018] Figure 2BSharing of multiple DAs from a second user's DA set back to a first user based on a DA shared by a first user according to one embodiment is shown.

[0019] Figure 3 An exemplary knowledge graph metadata network according to one embodiment is shown in block diagram form. Figure 3 The exemplary metadata network shown may be represented by Figure 1A The DAM system shown is generated and / or used.

[0020] Figure 4A The operations for providing content sharing suggestions according to one embodiment are shown in flow chart form.

[0021] FIG. 4B to FIG. 4C Operations for providing context-aware content sharing suggestions according to one embodiment are illustrated in flow chart form.

[0022] Figure 5 An exemplary user interface is shown for providing context-aware content sharing suggestions in a messaging application according to one embodiment.

[0023] Figure 6 Operations for providing context-aware content sharing suggestions in a messaging application are illustrated in flow chart form, according to one embodiment.

[0024] Figure 7 A simplified functional block diagram of an illustrative programmable electronic device for performing a DAM is shown according to one embodiment. DETAILED DESCRIPTION

[0025] The present invention describes methods, apparatus, computer-readable media, and systems for organizing, storing, describing, and / or retrieving digital assets (also referred to herein as "DAs") so that they can be presented to a user of a computing system in the form of a suggestion to share one or more DAs from a DA collection with one or more third parties, for example based on contextual analysis. Such embodiments can enable digital asset management (DAM), and in particular, enable sharing of DAs from a DA collection, in a more seamless and relevant manner.

[0026] Embodiments described herein may help improve computer functionality by enabling computing systems that use one or more embodiments of the digital asset management (DAM) system described herein. Such computing systems may implement DAM to help reduce or eliminate the need for users to manually determine what, when, and with whom to share with a DA. This reduction or elimination, in turn, may help minimize wasted computing resources (e.g., memory, processing power, computing time, etc.) that may be associated with using a unique relational database for DAM. For example, performing DAM via a relational database may include an external data repository and / or a remote server (as well as a network, communication protocol, and other components required to communicate with the external data repository and / or remote server). In contrast, DAM performed as described herein (i.e., utilizing a knowledge graph metadata network) may occur locally on a device (e.g., a portable computing system, a wearable computing system, etc.) without the need for an external data repository, a remote server, a network, a communication protocol, and / or other components required to communicate with an external data repository and / or remote server. In addition, by automating the process of content sharing suggestions in a context-sensitive manner, users do not have to perform excessive manual inspections of their (usually sizable) DA collection to determine which DAs may be appropriate to share with a given third party in a given context. Thus, at least one embodiment of the DAM described herein may help reduce or eliminate additional computing resources (e.g., memory, processing power, computing time, etc.) that may be associated with a user searching, storing, and / or obtaining a DA from an external relational database in order to determine whether to share such a DA with one or more third parties.

[0027] Figure 1A A processing system 100 is shown in block diagram form, including electronic components for performing digital asset management (DAM) according to one or more embodiments described in the present disclosure. The system 100 may be housed in a single computing system (such as a desktop computer system, a laptop computer system, a tablet computer system, a server computer system, a mobile phone, a media player, a personal digital assistant (PDA), a personal communicator, a gaming device, a network router or hub, a wireless access point (AP) or repeater, a set-top box, or a combination thereof). The components in the system 100 may be spatially separated and implemented on separate computing systems connected by a communication technology 110, as described in further detail below.

[0028] For one embodiment, the system 100 may include a processing unit 104, a memory 110, a DA capture device 102, a sensor 122, and a peripheral device 118. For one embodiment, one or more components in the system 100 may be implemented as one or more integrated circuits (ICs). For example, at least one of the processing unit 104, the communication technology 120, the DA capture device 102, the peripheral device 118, the sensor 122, or the memory 110 may be implemented as a system on chip (SoC) IC, a three-dimensional (3D) IC, any other known IC, or a combination of any known ICs. For another embodiment, two or more components in the system 100 are implemented together as one or more ICs. For example, at least two of the processing unit 104, the communication technology 120, the DA capture device 102, the peripheral device 118, the sensor 122, or the memory 110 may be implemented together as a SoC IC. Each component of the system 100 is described below.

[0029] like Figure 1A As shown, the system 100 may include a processing unit 104, such as a CPU, GPU, other integrated circuits (ICs), memory, and / or other electronic circuits. For one embodiment, the processing unit 104 manipulates and / or processes DA metadata associated with a digital asset 112 or optional data 116 associated with the digital asset (e.g., data objects such as nodes reflecting one or more people, places, points of interest, scenes, meanings, and / or events associated with a given DA). The processing unit 104 may include a digital asset management (DAM) system 106 for executing one or more embodiments of DAM, as described herein. For one embodiment, the DAM system 106 is implemented as hardware (e.g., electronic circuits, circuits, dedicated logic components, etc. associated with the processing unit 104), software (e.g., one or more instructions associated with a computer program executed by the processing unit 104, software running on a general-purpose computer system or a dedicated machine, etc.), or a combination thereof.

[0030] The DAM system 106 can enable the system 100 to generate and use a knowledge graph metadata network (also more simply referred to herein as a "knowledge graph" or "metadata network") 114 of the DA metadata 112 as a multidimensional network. Metadata networks and multidimensional networks that can be used to implement the various techniques described herein are described in further detail, for example, in patent application Ser. No. 269, which is incorporated by reference above. Figure 3 (which is described below) provides additional details regarding an exemplary metadata network 114.

[0031] In one embodiment, the DAM system 106 may perform one or more of the following operations: (i) generate a metadata network 114; (ii) associate and / or present at least two DAs based on the metadata network 114, for example, as part of a moment; (iii) determine an interesting DA in a DA set based on the metadata network 114 and one or more other criteria and / or present the interesting DA to the user as a sharing suggestion; and (iv) select and / or present the suggested DA to be shared with one or more third parties, for example, based on context analysis. Figures 1B to 6 Additional details regarding the immediately preceding operations that may be performed by the DAM system 106 are described.

[0032] The DAM system 106 may obtain or receive a collection of DA metadata 112 associated with a DA collection. As used herein, "digital assets," "DAs," and variations thereof refer to data that can be stored in or as a digital form (e.g., digital files, etc.). The digitized data includes, but is not limited to, the following: image media (e.g., still or animated images, etc.); audio media (e.g., songs, etc.); text media (e.g., e-books, etc.); video media (e.g., movies, etc.); and tactile media (e.g., vibrations or movements provided in conjunction with other media, etc.). The examples of digitized data above may be combined to form multimedia (e.g., computer-animated cartoons, video games, etc.). A single DA refers to a single instance of digitized data (e.g., an image, a song, a movie, etc.). Multiple DAs or a group of DAs refers to multiple instances of digitized data (e.g., multiple images, multiple songs, multiple movies, etc.). Throughout this disclosure, the use of "DA" refers to "one or more DAs," including a single DA and a group of DAs. For simplicity, the concepts described in this document use examples of the operation of a DA as one or more images. It should be understood that the DA is not limited in this regard, and that the concepts set forth in this document are applicable to other DAs (eg, different media described above, etc.).

[0033] As used herein, "digital asset collection", "DA collection" and their variations refer to a plurality of DAs that can be stored in one or more storage locations. As is known, the one or more storage locations can be separated spatially or logically.

[0034] As used herein, "metadata," "digital asset metadata," "DA metadata," and variations thereof, collectively refer to information about one or more DAs. Metadata may be: (i) a single instance of information about digitized data (e.g., a timestamp associated with one or more images, etc.); or (ii) a metadata grouping, which refers to a group consisting of multiple instances of information about digitized data (e.g., several timestamps associated with one or more images, etc.). There may also be many different types of metadata associated with a DA collection. Each type of metadata (also referred to as a "metadata type") describes one or more characteristics or attributes associated with one or more DAs. More details on the various types of metadata that may be stored in a DA collection and / or used in conjunction with a knowledge graph metadata network are described in more detail, for example, in patent application Ser. No. 269, which is incorporated by reference above.

[0035] As used herein, "context" and variations thereof refer to any or all attributes of a user device that includes a set of DAs associated with a user or that may have access thereto, such as physical, logical, social, and other contextual information. As used herein, "context information" and variations thereof refer to metadata that describes or defines the context of a user or the context of a user device that includes a set of DAs associated with a user or that may have access thereto. Exemplary context information includes, but is not limited to, the following: predetermined time intervals; events scheduled to occur at predetermined time intervals; geographic locations visited during specific time intervals; one or more identified persons associated with specific time intervals; events occurring during specific time intervals, or geographic locations visited during specific time intervals; weather metadata describing the weather associated with a specific time period (e.g., rain, snow, sun, temperature, etc.); season metadata describing the season associated with the capture of one or more DAs; relationship information describing the nature of a social relationship between a user and one or more third parties; or natural language processing (NLP) information describing the nature and / or content of an interaction between a user and one or more third parties. For some embodiments, contextual information may be obtained from an external source such as a social networking application, a weather application, a calendar application, a contact book application, any other type of application, or from any type of data repository accessible via a wired or wireless network (e.g., the Internet, a private intranet, etc.).

[0036] See again Figure 1A For one embodiment, the DAM system 106 uses the DA metadata 112 to generate a metadata network 114. Figure 1AAs shown, all or some of the metadata networks 114 may be stored in the processing unit 104 and / or the memory 110. As used herein, "knowledge graph", "knowledge graph metadata network", "metadata network" and their variations refer to a dynamically organized metadata collection that describes one or more DAs used by one or more computer systems (e.g., one or more DA groups in a DA set, one or more DAs in a DA set, etc.). In the metadata network, actual DAs are not stored, but only metadata (e.g., metadata associated with one or more DA groups, metadata associated with one or more DAs, etc.). The metadata network is different from a database because the metadata network generally allows deep connections between metadata using multiple dimensions, which can be traversed for additional inferred correlations. Such deductive reasoning is generally not feasible in conventional relational databases without loading a large number (e.g., hundreds, thousands, etc.) of database tables. Therefore, as described above, conventional databases may require a large amount of computing resources (e.g., external data repositories, remote servers and their associated communication technologies, etc.) to perform deductive reasoning. In contrast, the metadata network can be viewed, operated and / or stored using fewer computing resource requirements than the conventional databases described above. Furthermore, the metadata network is a dynamic resource that has the ability to learn, grow, and adapt to new information that is added to it. This is different from a database, which is useful for accessing cross-referenced information. While a database can be extended with additional information, it is still a tool for accessing the cross-referenced information placed therein. The metadata network is not just about accessing cross-referenced information, but goes far beyond that and involves extrapolation of data for inferring or determining additional data. As described above, the DA itself may be stored, for example, on one or more servers remote from the system 100, with thumbnail versions of the DA stored in the system memory 110 and the full version of a particular DA only downloaded and / or stored in the memory 110 of the system 100 as needed (e.g., when a user wishes to view or share a particular DA). However, in other embodiments, for example, when the amount of onboard storage space and processing resources at the system 100 is large enough and / or the size of the user's DA collection is small enough, the DA itself may also be stored within the memory 110, for example, in a separate database (such as the conventional database described above).

[0037] The DAM system 106 can generate a metadata network 114 as a multidimensional network of the DA metadata 112. As used herein, "multidimensional network" and variations thereof refer to a complex graph having multiple relationships. A multidimensional network typically includes multiple nodes and edges. For one embodiment, the nodes represent metadata, and the edges represent relationships or correlations between the metadata. Exemplary multidimensional networks include, but are not limited to, edge-labeled multigraphs, multi-part edge-labeled multigraphs, and multi-layer networks.

[0038] In some embodiments, the metadata network 114 includes two types of nodes: (i) moment nodes; and (ii) non-moment nodes. As used herein, "moment" shall refer to a contextual organizational pattern used to group one or more digital assets based on inferred or explicitly defined associations between such digital assets (e.g., for the purpose of displaying the group of digital assets to a user). For example, a moment may refer to a visit to a coffee shop in Cupertino, California, USA on March 26, 2018. For example, the moment can be used to identify one or more DAs (e.g., an image, a group of images, a video, a group of videos, a song, a group of songs, etc.) associated with the visit to the coffee shop on March 26, 2018 (rather than with any other event).

[0039] As used herein, a "time node" refers to a node in a multidimensional network that represents a time (as described above). As used herein, a "non-time node" refers to a node in a multidimensional network that does not represent a time. Thus, a non-time node may refer to a metadata asset associated with one or more DAs that are not times. More details on possible types of "non-time" nodes that may be found in an exemplary metadata network may be found, for example, in Patent Application No. 269, which is incorporated by reference above.

[0040] As used herein, "event" and variations thereof refer to a situation or activity that occurs at one or more locations during a specific time interval. Examples of events may include, but are not limited to, the following: a gathering of one or more people to perform an activity (e.g., a holiday, vacation, birthday, dinner, project, fitness class, etc.); sporting events (e.g., sports games, etc.); ceremonies (e.g., rituals of cultural significance performed on special occasions, etc.); meetings (e.g., gatherings of individuals participating in some common interest, etc.); festivals (e.g., gatherings celebrating some aspect of a community, etc.); concerts (e.g., art performances, etc.); media events (e.g., events created for publicity, etc.); and parties (e.g., large social or entertainment gatherings, etc.). According to some embodiments, an event may include a single moment identified in a given user's DA set. According to other embodiments, an event may include two or more related identified moments in a given user's DA set.

[0041] For one embodiment, edges between nodes in the metadata network 114 represent relationships or dependencies between the nodes. For one embodiment, the DAM system 106 updates the metadata network 114 when it obtains or receives new metadata 112 and / or determines new metadata 112 for DAs in the user's DA set.

[0042] The DAM system 106 may use the metadata network 114 in various ways to manage DAs associated with the DA metadata 112. For a first example, the DAM system 106 may use the metadata network 114 to identify and present an interesting group of one or more DAs in a DA set based on the DA metadata (i.e., nodes in the metadata network 114) and correlations between one or more criteria (i.e., edges in the metadata network 114). For this first example, the DAM system 106 may select the DAs of interest based on the moment nodes in the metadata network 114. In some embodiments, the DAM system 106 may suggest that the user share the one or more identified DAs with one or more third parties. For a second example, the DAM system 106 may use the metadata network 114 and other contextual information collected from the system (e.g., the user's relationship with one or more third parties, the subject of the conversation in a messaging thread, an inferred intent to share DAs associated with one or more moments, etc.) to select and present a representative group of one or more DAs that the user may want to share with one or more third parties.

[0043] The system 100 may also include a memory 110 for storing and / or retrieving metadata 112, a metadata network 114, and / or optional data 116 described by or associated with the metadata 112. The metadata 112, the metadata network 114, and / or the optional data 116 may be generated, processed, and / or captured by other components in the system 100. For example, the metadata 112, the metadata network 114, and / or the optional data 116 may include data generated, captured, processed, or associated with one or more peripheral devices 118, the DA capture device 102, or the processing unit 104. The system 100 may also include a memory controller (not shown) including at least one electronic circuit that manages data flowing to and / or out of the memory 110. The memory controller may be a separate processing unit or integrated in the processing unit 104.

[0044] The system 100 may include a DA capture device 102 (e.g., an imaging device for capturing images, an audio device for capturing sounds, a multimedia device for capturing audio and video, any other known DA capture device, etc.). The device 102 is shown in a dashed box to illustrate that it is an optional component of the system 100. For one embodiment, the DA capture device 102 may also include a signal processing pipeline implemented as hardware, software, or a combination thereof. The signal processing pipeline may perform one or more operations on data received from one or more components in the device 102. The signal processing pipeline may also provide processed data to the memory 110, the peripheral device 118 (as further discussed below), and / or the processing unit 104.

[0045] The system 100 may also include peripherals 118. For one embodiment, the peripherals 118 may include at least one of the following: (i) one or more input devices that interact with or send data to one or more components in the system 100 (e.g., a mouse, keyboard, etc.); (ii) one or more output devices that provide output from one or more components in the system 100 (e.g., a monitor, printer, display device, etc.); or (iii) one or more storage devices that store data in addition to the memory 110. The peripherals 118 are shown in dashed boxes to show that they are optional components of the system 100. The peripherals 118 may also refer to a single component or device that can be used as both an input device and an output device (e.g., a touch screen, etc.). The system 100 may include at least one peripheral control circuit (not shown) for the peripherals 118. The peripheral control circuit may be a controller (e.g., a chip, an expansion card, or a separate device, etc.) that interacts with the peripherals 118 and is used to instruct operations performed by the peripherals. The peripheral controller may be a separate processing unit or integrated in the processing unit 104. Peripheral devices 118 may also be referred to as input / output (I / O) devices 118 throughout this document.

[0046] The system 100 may also include one or more sensors 122, which are shown in dashed boxes to illustrate that the sensors may be optional components of the system 100. For one embodiment, the sensors 122 may detect one or more characteristics of the environment. Examples of sensors include, but are not limited to, light sensors, imaging sensors, accelerometers, sound sensors, air pressure sensors, proximity sensors, vibration sensors, gyroscope sensors, compasses, barometers, thermal sensors, rotation sensors, speed sensors, and inclinometers.

[0047] For one embodiment, system 100 includes communication mechanism 120. Communication mechanism 120 can be, for example, a bus, a network, or a switch. When technology 120 is a bus, technology 120 is a communication system that transmits data between components in system 100, or a communication system that transmits data between components in system 100 and other components associated with other systems (not shown). As a bus, technology 120 includes all relevant hardware components (wires, optical fibers, etc.) and / or software, including communication protocols. For one embodiment, technology 120 may include an internal bus and / or an external bus. In addition, technology 120 may include a control bus, an address bus, and / or a data bus for communication associated with system 100. For one embodiment, technology 120 can be a network or a switch. As a network, technology 120 can be any wired or wireless network, such as a local area network (LAN), a wide area network (WAN) such as the Internet, a fiber optic network, a storage network, or a combination thereof. When technology 120 is a network, the components in system 100 do not have to be physically co-located. When technology 110 is a switch (e.g., a "crossbar" machine), separate components in system 100 may be directly linked by a network even though the components may not be physically in close proximity to one another. For example, two or more of processing unit 104, communication technology 120, memory 110, peripherals 118, sensors 122, and DA capture device 102 are in different physical locations from one another and are communicatively coupled via communication technology 120, which is a network or switch that directly links the components by a network.

[0048] Figure 1B An example of a moment view user interface 130 for presenting a collection of digital assets based on the moments during which the digital assets were captured is shown, according to one embodiment. The interface 130 includes a list view of DA collections, in this case image collections 132, 134, and 136. Each such image collection may represent a unique moment in the user's DA collection. The image collections 132, 134, 136 include thumbnail versions of the images presented along with a description of the location at which the image was captured and the date (or date range) at which the image was captured. The temporal data and location data may be used to improve the definition and boundaries between moments, to more precisely define moments and to divide moment collections into more specific moments, as described in more detail, for example, in patent application No. 663, which is incorporated by reference above.

[0049] In one example, reference will be made to the following Figure 2A and Figure 2BDescribed in more detail, a subset of DAs from a user's DA set (e.g., DA set 138) may be selected by a user of the device to be shared with one or more third parties, where the DA set is part of image set 134 and was captured in and around Cupertino and San Francisco, California, United States on March 26, 2018.

[0050] Figure 2A FIG. 4 shows the sharing of a set of DAs from a first user to multiple DAs of a second user according to one embodiment. Figure 2A As shown, a first user (user A) holds a digital asset collection 200a, which includes, in addition to other digital assets, Figure 1B 130. In this particular example, user A has chosen to share (202) a subset of his DAs, namely DA set 138, with a third-party user B. It should be understood that after sharing (202), the DAs in DA set 138 will also appear in user B's digital asset collection 200b, for example, alongside user B's other pre-existing DAs.

[0051] In some cases, the decision of user A to initially share the DA set 138 with user B may be made by manual determination. In other words, user A may remember that he went to a coffee shop with user B last week, but user B did not take a picture of the coffee ordered by user A or the exterior of the coffee shop. Therefore, user A may manually determine that he wants to share the relevant set of images in the DA set 138 with user B.

[0052] However, as will be explained in further detail below, according to some embodiments described herein, user A's DAM system may automatically and in an intelligent (e.g., context-aware) manner suggest which DAs to share, with whom to share, and / or when to share such DAs. For example, if user A's knowledge graph indicates that user B is a close social contact of user A, the DAM may suggest sharing one or more of user A's DAs with user B, particularly those DAs where user A's DAM system may determine, for example via DA metadata or one or more other information sources, that user B was with user A during the moment when the images in the DA set 138 were captured (e.g., via detection of user B's face in one or more of the images).

[0053] In other embodiments, as will be described in further detail below, user A's DAM system may apply contextual analysis to determine if there is an indication of an intent to share certain assets in user A's DA collection (or a request to share these assets). For example, user B may have recently sent a message to user A saying, "Can you send me pictures of the coffee shop from last week?" Once the sharing intent is determined, user A's knowledge graph can quickly apply search heuristics to date ranges and points of interest (such as "restaurants" or "coffee shops") in the past week, can quickly identify relevant (or potentially relevant) DAs that user B is requesting, and automatically present user A with suggestions to share one or more of the matching DAs with user B. In other embodiments, the user's knowledge graph may be further utilized to determine (e.g., proactively determine) whether / when the user has a DA in his or her DA collection in a messaging thread that is related to the topic being discussed (and / or a party involved), and the user may be interested in sharing the DA with one or more third parties.

[0054] Figure 2B Yet another example of a content sharing scenario is shown, where content sharing suggestions are determined by a user's DAM system performing contextual analysis. Figure 2B In the example of Figure 2A In the example of DAs shared in user A's DA set 138, user B's DAM system has suggested "sharing back" (208) multiple DAs 204 from user B's DA set 200b. Specifically, user B's DAM system identifies that user A's DAs 204 may be "shared back" (208) based on identifying moments in user B's DA set that occur in approximately the same geographic location and / or approximately the same time interval as the initial shared DAs from user A's DA set 138. In some embodiments, the magnitude (e.g., in terms of geographic scope) and / or duration (e.g., in terms of time scope) of the suggested set of DAs to be shared back may be scaled directly and proportionally to the magnitude and duration of the initial DAs shared from the third party. Thus, if Figure 2B As shown, the plurality of DAs 204 from the DA set 200b of user B have been suggested for sharing back (208) based on the fact that they were captured on the same day and in the same coffee shop as the DAs in the initial shared DA set 138. In contrast, the DAs 206 in the DA set of user B represent DAs captured at different locations and / or during different time intervals than the DAs in the initial shared DA set 138, and therefore are not part of the exemplary suggested shared back DAs 204.

[0055] Figure 3 An exemplary knowledge graph metadata network 300 is shown in block diagram form, according to one embodiment. Figure 3The exemplary metadata network shown may be represented by Figure 1A The DAM system shown generates and / or uses. For one embodiment, Figure 3 The metadata network 300 shown is combined with the above Figure 1A The metadata network 114 is similar or identical to the metadata network 114 described above. It should be understood that Figure 3 The metadata network 300 described and illustrated in FIG. 1 is exemplary and does not illustrate every type of node or edge that the DAM system 106 may generate. For example, even if Figure 3 Not every possible node is shown in the figure, and the DAM system 106 may also generate nodes to represent Figure 2A In the exemplary scenario shown, a shared DA set 138 is associated with several metadata assets.

[0056] exist Figure 3 In the illustrated metadata network 300, nodes representing metadata are shown as circles, and edges representing correlations between metadata are shown as connections or edges between circles. In addition, some nodes are labeled with the type of metadata they represent (e.g., region, city, state, country / region, year, day, week, month, point of interest (POI), area of ​​interest (AOI), region of interest (ROI), person, event type, event name, event performer, event venue, merchant name, merchant category, etc.). Figure 3 In the example metadata network 300 shown, an "Event" node is shown as linking various other metadata nodes together. In some implementations, an event may simply include a moment in time, as discussed previously herein. However, in other implementations, an event may be considered a higher level DA association than a moment in time, for example, two or more related moments in time may be identified and collectively referred to as an event. In other embodiments, for example, where a user may have a DA group involving assets other than images captured at a specific time and location, an event may refer to all DAs related to circumstances or activities occurring at one or more locations within a time interval (e.g., a video recorded at a concert, a digital ticket stub for a concert, a music file of an artist performing at a concert, etc.).

[0057] For one embodiment, the metadata represented in the nodes of the metadata network 300 may include, but is not limited to: other metadata, such as the user's relationship with other people (e.g., family members, friends, colleagues, etc.), the user's work place (e.g., past work place, current work place, etc.), the user's interests (e.g., hobbies, DAs held, DAs consumed, DAs used, etc.), places visited by the user (e.g., previous places visited by the user, places that the user will visit, etc.). Such metadata information can be used alone (or in combination with other data) to determine or infer at least one of the following: (i) the user is on vacation or traveling; the days of the week (e.g., weekends, holidays, etc.); the location associated with the user; the user's social group; the type of place visited by the user (e.g., restaurant, coffee shop, etc.); the category of event (e.g., cooking, sports, travel, etc.); and the like. The foregoing examples are intended to illustrate, but not limit, the types of metadata information that can be captured in the metadata network 300.

[0058] Figure 4AThe operation 400 of providing content sharing suggestions according to one embodiment is shown in flowchart form. First, the operation may begin at step 402 by obtaining a metadata set associated with a user's DA set. Next, at step 404, the method may also obtain a knowledge graph metadata network for the DA set. At step 406, one or more unique moments may be identified within the DA set based at least in part on the knowledge graph metadata network, as described above. According to some embodiments, the identification of moments within the user's DA set may optionally include analyzing at least location-related metadata of the DA in the user's DA set to determine significant locations where the user has spent time (step 407). In some embodiments, determining that a location is significant involves determining that the location is a location that has been visited for at least a predetermined period of time or that the location is a familiar location (e.g., the user's home) or a priori significant location (e.g., a well-known landmark). In other embodiments, determining that a location is significant may involve determining that the location is a location that the user frequently visits. Determining that a location is frequently visited may involve collecting information including location coordinates, location name, counts indicating the number of times an electronic device has visited the location, dates associated with each visit in the visit, duration indications associated with each visit in the visit, and the like. According to other embodiments, frequently visited places may also relate to more precise sub-locations included in the initially identified location. Next, according to some embodiments, moments within the user's DA set may optionally be identified based at least in part on the time period that the user spent at the prominent location (step 408). In other words, any DA captured or created while the user is at a particular prominent location may each be tagged as part of the same unique moment. Once the DA set has been divided into moments (e.g., using any desired method), the identification of one or more moments within the DA set from which content is suggested to be shared may be based on any of a number of factors, such as factors that may be collected from a knowledge graph. For example, a moment for suggested sharing may be identified based on one or more of the following factors: the significance of the moment (e.g., related to the category of the DA event associated with the moment), a point of interest associated with the moment, a holiday event associated with the moment, a specific location associated with the moment, a type of scene identified at the moment, a date or time associated with the moment, a specific person or group of people associated with the moment, whether a group of moments can be inferred to be related to each other as part of a larger event, etc.

[0059] Because each moment may be associated with one or more digital assets, operation 400 may next determine, for at least one identified moment, one or more of the associated digital assets to suggest sharing with one or more third parties (step 410). Such determination of specific associated digital assets to suggest sharing may be based on, for example, selecting: only DAs above a particular quality threshold (e.g., based on focus, exposure level, saturation, color balance, user rating, threshold number of detected faces, etc.); only DAs that are not duplicated; only DAs that are not screenshots, etc. The determination to suggest sharing with the one or more third parties may be informed by the relationship of the one or more third parties to the at least one identified moment (e.g., whether the third party appears in the DA associated with the moment, whether the third party appears at the same location during the identified moment, whether the third party is in a particular social group with the user, etc.). In some embodiments, the one or more third parties may also be determined based, at least in part, on the current proximity of the one or more third parties to the user at the time of the sharing suggestion.

[0060] In some embodiments, the determination of the one or more third parties with which the DAM suggests the user may share the DA may be filtered according to one or more filtering options. For example, in some cases, it may be desirable to filter out third parties that are otherwise determined to be suggested sharing targets (e.g., based on the various factors listed above), but for which it may be inappropriate or undesirable to suggest the user as a sharing target.

[0061] For example, in some cases, the determined third-party sharing targets may be filtered from the suggested recipient list based on: (i) their character type; (ii) the scene type reflected in one or more DAs in the DA to be shared; and / or (iii) the third party's current relationship with the user (e.g., as determined from the user's knowledge graph metadata network). For example, in some embodiments, it may be desirable to employ an age-based filtering option on the suggested sharing targets. The age-based filtering option may be used, for example, to filter out sharing targets that are below a minimum age threshold, above a maximum age threshold, deceased, etc. In other embodiments, the filtering options may be based on whether the suggested sharing target is: a current social contact of the user, a blocked (or previous) contact of the user, a holder of a device that uses a DAM system similar to the user, or a specific type of contact of the user (e.g., a subordinate in the user's workplace, a manager in the user's workplace, a spouse / partner of the user, an ex-spouse / partner of the user, etc.). It should also be mentioned that the user's social contacts that simply do not currently exist (or do not hold a device that uses a DAM system similar to the user) may not necessarily be the basis for filtering out the determined third party as a suggested sharing target. For example, in some embodiments, the DAM system may provide the user with an opportunity to name a third party and / or create a social connection for the third party before sharing the DA with the third party (or, alternatively, continue to filter out third parties as sharing targets).

[0062] In other embodiments, for example, as mentioned in (ii) above, the scene types determined to be reflected in one or more DAs to be shared may be used to filter out suggested third-party sharing targets. For example, if a DA is determined to represent a "pet" scene or a "nature" scene, it may not be appropriate to suggest sharing the DA with any animal whose face may be located within the DA. As another example, if a DA represents a "child" or "baby" scene, it may not be appropriate to suggest sharing the DA with any child or baby who may be located within the DA (because they are unlikely to be in contact with the user or hold / use a device that employs a DAM system similar to the user). In some embodiments, for example, if such information is available in the user's knowledge graph network, parents, guardians, or other relevant persons of the children or babies located in the DA may alternatively be suggested as third-party sharing targets for the DA, including representations of the children or babies (i.e., instead of the children or babies themselves).

[0063] In other embodiments, a filter score may be determined for each of the one or more third parties initially determined as the suggested sharing targets of the DA, which filter score may be used to help the DAM determine whether to filter out any of the one or more third parties determined as the suggested sharing targets. The filter score may be based on any desired number of filter options for a given specific implementation. For example, if the initially determined third-party sharing target is classified as an infant or child, +100 points may be added to its filter score; if the initially determined third-party sharing target is not a current contact of the user, +50 points may be added to its filter score; if the initially determined third-party sharing target is not a contact of the user in any external social network (or social group identified in the user's knowledge graph), +25 points may be added to its filter score, etc. In other embodiments, for example, the filtering options may also reduce the filter score of the third party (e.g., -25 points for each social network of the user where the third party is a contact). In this example, the filter score of the initially determined third-party sharing target may be 175 (i.e., 100+50+25). In some embodiments, a filter score threshold may be employed, e.g., above which the initially identified third party may be filtered out as a potential sharing target. For example, if the filter score threshold in a given embodiment is 150, then the initially identified third party having a filter score of 175 may be filtered out from the list of sharing targets. If another third party has a filter score below 150, then they may not be filtered out by the DAM, i.e., they may still be a suggested sharing target for the DA.

[0064] Finally, at step 412, the method may provide a suggestion to the user to share the determined one or more associated digital assets with the one or more third parties, e.g., subject to any third-party filtering options (e.g., including the various potential filtering options described above). After receiving an indication from the user of which of the determined one or more associated digital assets are to be shared with the one or more third parties, or in response to receiving an indication from the user of which of the determined one or more associated digital assets are to be shared with the one or more third parties, the method may proceed to step 414 and actually share one or more of the suggested one or more associated digital assets with the one or more third parties. The sharing may occur, for example, by sending the DA directly to the third party (e.g., via email, text message, instant message, or other proximity-based communication protocol, etc.), or indirectly to the third party, such as via a server holding a copy or reference to the DA. Once the desired DA has been shared, operation 400 may end.

[0065] FIG. 4B to FIG. 4COperation 450 for providing context-aware content sharing recommendations according to one embodiment is shown in flowchart form. As with other embodiments described herein, before being able to provide context-aware content sharing recommendations, the user's device may first obtain a metadata set associated with a DA set (step 452), for example, where the digital asset set includes one or more moments, and where each of the one or more moments is associated with one or more digital assets in the digital asset set. The user's device may also a priori obtain a knowledge graph metadata network for the user's DA set (step 454). Operation 450 may then continue at step 456 by receiving one or more DAs (and their associated metadata) from a third party. In operation 450, the content sharing recommendations will be based at least in part on content and / or metadata of DAs from third parties that were recently shared with the user, for example, as previously referenced. Figure 2B discussed.

[0066] Next, at step 458, operation 450 may proceed to identify relevant moments to share DAs from the user's DA set. This determination may be based at least in part on the user's knowledge graph and one or more DAs (and / or associated metadata) received from a third party, e.g., a DA recently received from a third party, such as in a messaging thread. Specifically, operation 450 may identify one or more moments within the user's DA set to "share back" to a third party, i.e., in response to an initial share by the third party. According to some embodiments, such identification of moments to consider a "share back" function may optionally include analyzing location and time metadata of the one or more DAs received from the third party (step 459) and performing a search against the user's knowledge graph by matching the received metadata of the DAs shared from the third party to the user's knowledge graph (step 460). In some embodiments, the search against the user's knowledge graph may optionally include a "fuzzy" search (step 461), e.g., a search that allows for inexact matches of DAs in the DA set by matching DAs from a larger time window and / or a larger geographic area than the DA originally shared by the third party. In some such embodiments, the amount of fuzziness allowed in the search is based at least in part on the density of the DA set. In other words, if the DA set includes a relatively small number of relevant DAs (i.e., is relatively sparse over the relevant time period), the method may allow for a less precise match with the DAs that were originally shared. In contrast, if the DA set includes a large number of relevant DAs (i.e., is relatively dense over the relevant time period), the method may require a relatively more precise match with the DAs that were originally shared. The fuzzy search may also consider larger DA sets based on reasoning that can be obtained from the knowledge graph (e.g., if it can be inferred that the vacation occurred within a larger time interval that overlaps with the time window of the search, then include additional content from the suggestion set to vacation). At step 462, operation 450 may be performed at step 463. Figure 4CContinue at step 464.

[0067] Next, go to Figure 4C At step 466, operation 450 may determine, for at least one of the identified moments from step 458, one or more of the digital assets associated with the matching moments from the user's DA set to be "shared back" to one or more third parties. Again, this determination may be based on, for example, selecting: only DAs above a certain quality threshold (e.g., based on focus, exposure level, saturation, color balance, user rating, threshold number of detected faces, etc.); only DAs that are not duplicated; only DAs that are not screenshots, etc. It may also be further informed by the actual DA (and its associated metadata) originally shared by the third party and / or the relationship of the third party to the at least one identified matching moment. Finally, at step 468, operation 450 may provide the user with a suggestion to share the determined one or more associated digital assets with the originally shared third party. After receiving an indication from the user of which of the determined one or more associated digital assets are to be shared with the third party or in response to receiving an indication from the user of which of the determined one or more associated digital assets are to be shared with the third party, operation 450 may proceed to share the determined one or more associated digital assets with the third party (step 470). According to some embodiments, the magnitude (e.g., in geographic scope) and / or duration (e.g., in time scope) of the suggested "shared back" DA set will scale with the magnitude and duration of the initial DA sharing from the third party. In other words, for example, the greater the time period (or location) that the third party shares the DA with the user, the greater the time period (or location) that the share-back suggestion logic will consider the DA from the user set to be likely to match the shared-back DA. Conversely, the smaller the time period (or location) that the third party shares the DA with the user, the smaller the time period (or location) that the share-back suggestion logic will consider the DA from the user set to be likely to match the shared-back DA.

[0068] Figure 5 5 is an exemplary user interface 500 for providing context-aware content sharing suggestions in a messaging application according to one embodiment. Figure 5 In the example of , exemplary user interface 500 shows a conversation thread (502) occurring on the computing device of user B. In this example, user A's initial message says: "Hey, user B! Can you send me the pictures you took at the coffee shop last week?" According to some embodiments, a process may be running in the background of the messaging application to continuously analyze incoming (or outgoing) messages in the messaging application for sharing intent, such as via natural language processing (NLP), word graphs, or other artificial intelligence-based language processing techniques. Figure 5In the example shown, user B's use of the terms "send me," "picture," "coffee shop," and "last week" may, in combination, suggest to the intent determination process that user B has indicated a desire for user A to share with him certain DAs from user B's DA set. In response to such a determination, the messaging application may display a quick suggestion (504) of the one or more DAs from user B's DA set that are believed to best match the sharing intent of the incoming message from the user. In this example, the matching DAs include the same two images from DA set 204, as previously described with reference to FIG. Figure 2B The two images may, for example, have been taken by user B at a time that occurred last week, relate to a location that is known to be a coffee shop (or other type of restaurant), and / or relate to user A in some way (e.g., the time of the image including the image in which user A's face was detected). It should be understood that the quick suggestion (504) may appear only on user B's device (i.e., the owner of DA), and that the suggestion may appear in any desired user interface element on user B's device, e.g., in a "pop-up" message box, notification, within a messaging thread, within a message input box, etc., and Figure 5 The location of the quick suggestions 504 in are merely illustrative. In some embodiments, user B will then be presented with an option 506 to share all, none, or some of the automatically suggested DAs. Assuming user B agrees to share the DA in response to the sharing request from user A, the DA may be sent to user A (508), for example, via the same messaging application that received the original incoming message from user A. In other embodiments, the selected suggested DA may be sent via some other messaging application (e.g., via email, text message, instant message or other proximity-based communication protocol, etc.), or indirectly, such as by providing a link or reference to a location on a server that holds a copy or reference of a copy of the shared DA.

[0069] Figure 6Operation 600 of providing context-aware content sharing suggestions in a messaging application according to one embodiment is shown in flowchart form. As with other embodiments described herein, before being able to provide context-aware content sharing suggestions in a messaging application, a user's device may first obtain a metadata set associated with a DA set, wherein the digital asset set includes one or more moments, and wherein each of the one or more moments is associated with one or more digital assets in the digital asset set. The user's device may also a priori obtain a knowledge graph metadata network for the user's DA set. Operation 600 may then continue at step 602 by, for example, receiving an incoming message from a sender at a first device of the user. Next, at step 604, the DAM system on the first device may detect a sharing intent in the incoming message. According to some embodiments, determining the sharing intent from the incoming message may be achieved by performing natural language processing (NLP) on the content of the incoming message.

[0070] Next, at step 606, operation 600 may extract one or more features from the content of the incoming message. In some embodiments, extracting the one or more features from the content of the incoming message may also include enhancing the extracted features to allow for "fuzzy" (i.e., inexact) matching against the user's knowledge graph. According to some embodiments, enhancing the extracted features from the incoming message may be achieved by using at least one of: synonyms of the extracted features, word embeddings based on the extracted features, and NLP on the extracted features. In some embodiments, the distance between the extracted features and the generated synonyms / embeddings (e.g., a measure of the string difference between two character sequences) may be used as an additional heuristic when attempting to perform and / or characterize the results of a fuzzy search against the user's knowledge graph.

[0071] Next, at step 608, operation 600 may perform a comparison of the one or more extracted features with the one or more moments and knowledge graph metadata network identified within the user's digital asset collection. Then, at step 610, the operation may determine at least one moment of the one or more moments that matches the one or more extracted (and optionally enhanced) features. In some embodiments, the match of the at least one moment determined may be optionally further enhanced based at least in part on the relationship of the sender of the message to the identified moment (e.g., whether the sender appears in a DA associated with the moment, whether the sender appears at the same location during the identified moment, whether the sender is in a particular social group with the user, etc.).

[0072] Next, at step 612, the operation 600 may determine, in response to the incoming message, for the at least one determined moment, one or more of the digital assets associated with the at least one moment to share with the sender. For example, the operation 600 may determine that: only DAs above a certain quality threshold (e.g., based on focus, exposure level, saturation, color balance, user rating, a threshold number of detected faces, etc.); only DAs that are not duplicates; only DAs that are not screenshots; only DAs that match the detected intent of the incoming message by more than a threshold amount should be shared with the sender, etc.

[0073] Finally, at step 614, operation 600 may provide a suggestion to the user, e.g., via the first device, to share the determined one or more associated digital assets with the sender. After receiving an indication from the user of which of the determined one or more associated digital assets are to be shared with the sender or in response to receiving an indication from the user of which of the determined one or more associated digital assets are to be shared with the sender, operation 600 may proceed to share the determined one or more associated digital assets with the sender (step 616). As previously described, the determined one or more associated digital assets may be shared with the sender, e.g., by directly sending the DA back to the sender via the same messaging application as the incoming message was received, via some other messaging application (e.g., via email, text messaging, instant messaging, or other proximity-based communication protocol, etc.), or indirectly sending the DA back to the sender, such as via providing a link or reference to a location on a server that holds a copy or reference of a copy of the shared DA.

[0074] See now Figure 7 , which shows a simplified functional block diagram of an exemplary programmable electronic device 700 for performing DAM according to one embodiment. The electronic device 700 can be, for example, a mobile phone, a personal media device, a portable camera, or a system of a tablet computer, a laptop computer, or a desktop computer. As shown, the electronic device 700 may include a processor 705, a display 710, a user interface 715, a graphics hardware 720, a device sensor 725 (e.g., a proximity sensor / ambient light sensor, an accelerometer and / or a gyroscope), a microphone 730, an audio codec 735, a speaker 740, a communication circuit 745, an image capture circuit or unit 750 (e.g., it may include a plurality of camera units / optical sensors with different characteristics (and a camera unit housed outside the device 700 but in electronic communication with the device)), a video codec 755, a memory 760, a storage device 765, and a communication bus 770.

[0075] The processor 705 may execute instructions necessary to implement or control the operation of the various functions performed by the device 700 (e.g., such as the generation and / or processing of DA according to various embodiments described herein). The processor 705 may, for example, drive the display 710 and may receive user input from a user interface 715. The user interface 715 may take a variety of forms, such as buttons, keypads, dials, click wheels, keyboards, display screens, and / or touch screens. The user interface 715 may, for example, be a conduit through which a user can view a captured video stream and / or indicate a particular image that the user wants to capture or share (e.g., by clicking a physical or virtual button at the moment the desired image is being displayed on the device's display screen).

[0076] In one embodiment, the display 710 may display a video stream captured while the processor 705 and / or the graphics hardware 720 and / or the image capture circuitry simultaneously stores the video stream (or a single image frame from the video stream) in the memory 760 and / or the storage device 765. The processor 705 may be a system on a chip such as those present in mobile devices, and may include one or more dedicated graphics processing units (GPUs). The processor 705 may be based on a reduced instruction set computer (RISC) or complex instruction set computer (CISC) architecture or any other suitable architecture, and may include one or more processing cores. The graphics hardware 720 may be dedicated computing hardware for processing graphics and / or assisting the processor 705 in performing computing tasks. In one embodiment, the graphics hardware 720 may include one or more programmable graphics processing units (GPUs).

[0077] For example, according to the present disclosure, the image capture circuit 750 may include one or more camera units configured to capture images, such as images that can be managed by the DAM system. The output from the image capture circuit 750 may be processed at least in part by the following devices: a video codec 755 and / or a processor 705 and / or graphics hardware 720 and / or a dedicated image processing unit incorporated into the circuit 750. The images captured thereby may be stored in a memory 760 and / or a storage device 765. The memory 760 may include one or more different types of media used by the processor 705, the graphics hardware 720, and the image capture circuit 750 to perform device functions. For example, the memory 760 may include a memory cache, a read-only memory (ROM), and / or a random access memory (RAM). The storage device 765 may store media (e.g., audio files, image files, and video files), computer program instructions or software, preference information, device profile information, and any other suitable data. The storage device 765 may include one or more non-transitory storage media, including, for example, magnetic disks (fixed hard disks, floppy disks, and removable disks) and tapes, optical media such as CD-ROMs and digital video disks (DVDs), and semiconductor memory devices such as electrically programmable read-only memories (EPROMs), and electrically erasable programmable read-only memories (EEPROMs). The memory 760 and the storage device 765 may be used to hold computer program instructions or codes organized into one or more modules and written in any desired computer programming language. When executed by, for example, the processor 705, such computer program codes may implement one or more of the methods described herein.

[0078] In the foregoing description, many specific details such as specific configurations, attributes and processes, etc. are described to provide a thorough understanding of the embodiments. In other examples, well-known processes and manufacturing techniques have not been described in particular detail so as not to unnecessarily obscure the embodiments. Throughout this specification, references to "one embodiment", "embodiment", "another embodiment", "other embodiments", "some embodiments" and their variations mean that the specific features, structures, configurations or characteristics described in conjunction with the embodiment are included in at least one embodiment. Therefore, the phrases "for an embodiment", "for an embodiment", "for another embodiment", "in other embodiments", "in some embodiments" or their variations in various positions that appear throughout this specification do not necessarily refer to the same embodiment. In addition, specific features, structures, configurations or characteristics can be combined in one or more embodiments in any appropriate manner.

[0079] In the following description and claims, the terms "coupled" and "connected" and their derivatives may be used. It should be understood that these terms are not intended to be synonymous with each other. "Coupled" is used herein to indicate that two or more elements or components that may or may not be in direct physical or electrical contact with each other cooperate or interact with each other. "Connected" is used to indicate the establishment of communication between two or more elements or components that are coupled to each other.

[0080] Certain portions of the foregoing detailed description have been presented in terms of algorithms and symbolic representations of operations on data bits in a computer memory. These algorithmic descriptions and representations are the means used by those skilled in the art of data processing to most effectively convey the substance of their work to others skilled in the art. An algorithm is here and generally refers to a self-consistent sequence of operations leading to a desired result. Operations are those operations requiring physical manipulation of physical quantities. However, it should be kept in mind that all of these and similar terms are associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. Unless otherwise specifically stated, it should be apparent from the above discussion that it should be understood that throughout this specification, discussions utilizing terms such as those set forth in the following claims refer to operations and processes of a computer system or similar electronic computing system that can manipulate data represented as physical (electronic) quantities in computer system registers and memories and convert them into other data that are also displayed as physical quantities in computer memories, registers, or other such information storage, transmission, or display devices.

[0081] The embodiments described herein may relate to a device for executing a computer program (e.g., the operations described herein, etc.). Such a computer program may be stored in a non-temporary computer-readable medium. A machine-readable medium includes any mechanism for storing information in a machine (e.g., computer) readable form. For example, a machine-readable (e.g., computer-readable) medium includes a machine (e.g., computer) readable storage medium (e.g., read-only memory ("ROM"), random access memory ("RAM"), magnetic disk storage medium, optical storage medium, flash memory device).

[0082] Although the above describes operations or methods according to some sequential operations, it should be understood that some of the operations described can be performed in different orders. In addition, some operations can also be performed in parallel rather than in order. The embodiments described herein are not described with reference to any specific programming language. It should be understood that various embodiments of the disclosed subject matter can be implemented using various programming languages. In utilizing the various aspects of the embodiments described herein, it will be apparent to those skilled in the art that the combination, modification or variation of the above embodiments is possible for the components of the management processing system to increase the power and performance of at least one component in those components. Therefore, it is obvious that various modifications can be made to it without departing from the broad spirit and scope of at least one disclosed concept in the disclosed concepts set forth in the appended claims. Therefore, the specification and the accompanying drawings are regarded as illustrative rather than restrictive.

[0083] In the development of any actual implementation of one or more of the disclosed concepts (e.g., such as a software and / or hardware development project, etc.), many decisions must be made to achieve the developer's specific goals (e.g., to comply with system-related constraints and / or business-related constraints). These goals may vary from implementation to implementation, and such variations may affect the actual implementation of one or more of the disclosed concepts set forth in the embodiments described herein. Such development work may be complex and time consuming, but may still be a daily routine for those of ordinary skill in the art in the design and / or implementation of one or more of the inventive concepts set forth in the embodiments described herein.

[0084] As described above, one aspect of the present technology is to collect and use data from various sources to improve the delivery of content sharing suggestions to users. The present disclosure contemplates that in some instances, such collected data may include personal information data that uniquely identifies or can be used to contact or locate a specific person. Such personal information data may include demographic data, location-based data, phone numbers, email addresses, Twitter accounts, home addresses, data or records related to the user's health or fitness level (e.g., vital sign measurements, medication information, exercise information), date of birth, or any other identifying or personal information.

[0085] The present disclosure recognizes that the use of such personal information data in the present technology can be used to benefit users. For example, personal information data can be used to deliver targeted content sharing suggestions that are of greater interest and / or greater contextual relevance to the user. Therefore, the use of such personal information data enables users to more simply and meaningfully control the content they share with others. In addition, the present disclosure also anticipates other uses of personal information data that benefit users. For example, health and fitness data can be used to gain insight into the user's overall health, or their health during various moments or events in their lives.

[0086] This disclosure envisions that entities responsible for collecting, analyzing, disclosing, transmitting, storing or otherwise using such personal information data will comply with established privacy policies and / or privacy practices. Specifically, such entities should implement and adhere to privacy policies and practices that are recognized as meeting or exceeding industry or government requirements for maintaining the privacy and security of personal information data. Such policies should be easily accessible to users and should be updated as the collection and / or use of data changes. Personal information from users should be collected for legitimate and reasonable purposes of the entity and not shared or sold outside of these legitimate uses. In addition, such collection / sharing should be performed after receiving informed consent from the user. In addition, such entities should consider taking any necessary steps to defend and secure access to such personal information data and ensure that others who have access to personal information data comply with their privacy policies and processes. In addition, such entities may subject themselves to third-party assessments to demonstrate their compliance with widely accepted privacy policies and practices. In addition, policies and practices should be adjusted to specific types of personal information data collected and / or accessed, and to applicable laws and standards including specific considerations of jurisdiction. For example, in the United States, the collection or access of certain health data may be governed by federal and / or state laws, such as the Health Insurance Portability and Accountability Act (HIPAA); while health data in other countries may be subject to other regulations and policies and should be handled accordingly. Therefore, different privacy practices for different types of personal data should be maintained in each country.

[0087] Regardless of the foregoing, the present disclosure also contemplates implementation schemes in which users selectively block the use or access of personal information data. That is, the present disclosure contemplates providing hardware elements and / or software elements to prevent or block access to such personal information data. For example, with respect to a content sharing suggestion service, the technology of the present invention may be configured to allow users to choose to "opt in" or "opt out" at any time during or after registration for the service to participate in the collection of personal information data. As another example, users may choose not to provide their content and other personal information data to improve the content sharing suggestion service. As another example, users may be able to choose to limit the length of time that a third party maintains their personal information data, limit the length of time that content sharing suggestions can be extracted from it in the past, and / or completely prohibit the development of knowledge graphs or other metadata profiles. In addition to providing "opt-in" and "opt-out" options, the present disclosure contemplates providing notifications related to access or use of personal information. For example, users may be notified that their personal information data will be accessed when downloading an application, and then reminded again just before the personal information data is accessed by the application.

[0088] In addition, it is an object of the present disclosure that personal information data should be managed and processed to minimize the risk of unintentional or unauthorized access or use. Once the data is no longer needed, the risk can be minimized by limiting data collection and deleting the data. In addition, and when applicable, including in certain health-related applications, data de-identification can be used to protect the privacy of users. De-identification can be facilitated by removing specific identifiers (e.g., date of birth, etc.), controlling the amount or specificity of stored data (e.g., collecting location data at the city level rather than at the address level), controlling how the data is stored (e.g., aggregating data between users), and / or other methods when appropriate.

[0089] Thus, while the present disclosure broadly covers the use of personal information data to implement one or more of the various disclosed embodiments, the present disclosure also contemplates that various embodiments may also be implemented without access to such personal information data. That is, the various embodiments of the present technology will not fail to function properly due to the lack of all or a portion of such personal information data. For example, a preference may be inferred based on non-personal information data or an absolute minimum of personal information such as the quality level of the content (e.g., focus, exposure level, etc.) or the fact that a device associated with the user's contacts, other non-personal information available to the DAM system, or publicly available information is requesting certain content, thereby suggesting that content be shared with a user.

[0090] As used above and in the claims below, the phrases "at least one of A, B, or C" and "one or more of A, B, or C" include A alone, B alone, C alone, a combination of A and B, a combination of B and C, a combination of A and C, and a combination of A, B, and C. That is, the phrases "at least one of A, B, or C" and "one or more of A, B, or C" mean A, B, C, or any combination thereof, so that one or more of a group of elements consisting of A, B, and C should not be interpreted as requiring at least one of each of the listed elements A, B, and C, regardless of whether A, B, and C are related to a category or otherwise. In addition, the use of the articles "a" or "the" when introducing an element should not be interpreted as excluding multiple elements. Moreover, the expression "A, B, and / or C" is equivalent to "at least one of A, B, or C". Moreover, "a" as used in the present disclosure means "one or more". For example, "DA" means "a DA" or "a group of DAs".

Claims

1. A non-transitory computer-readable medium comprising computer-executable instructions stored thereon, the computer-executable instructions being configured to cause one or more processors to: Obtaining a metadata collection associated with a collection of digital assets; Obtaining a knowledge graph metadata network for the collection of digital assets; identifying one or more moments within the collection of digital assets based at least in part on the knowledge graph metadata network, wherein each moment is associated with one or more digital assets; determining, for at least one identified moment, one or more of the associated digital assets to share with one or more third parties, wherein the one or more third parties with which to share the one or more associated digital assets are determined based at least in part on a relationship between the one or more third parties and the at least one identified moment; as well as A suggestion is provided to the first user to share the determined one or more associated digital assets with the one or more third parties.

2. The non-transitory computer-readable medium of claim 1 , further comprising computer-executable instructions stored thereon, the computer-executable instructions being configured to cause the one or more processors to: receiving an indication from the first user to share the determined one or more associated digital assets with the one or more third parties; and The determined one or more associated digital assets are shared with the one or more third parties.

3. The non-transitory computer-readable medium of claim 2, wherein the instructions for causing the one or more processors to share the determined one or more associated digital assets with the one or more third parties further comprise instructions for causing the one or more processors to: An indication to share the determined one or more associated digital assets with the one or more third parties is transmitted to a server holding a copy or reference to the determined one or more associated digital assets.

4. The non-transitory computer-readable medium of claim 1 , wherein the instructions for causing the one or more processors to identify one or more moments in time within the collection of digital assets further comprise instructions for causing the one or more processors to: analyzing location data for one or more digital assets within the collection of digital assets to determine significant locations; and Digital assets captured during the time period spent at the prominent location are identified as belonging to the corresponding moment.

5. The non-transitory computer-readable medium of claim 1, wherein the instructions for causing the one or more processors to determine, for at least one time instant, one or more of the associated digital assets to share with one or more third parties further comprise instructions for causing the one or more processors to: The one or more third parties are determined based at least in part on: a relationship of the one or more third parties to the identified one or more moments in time; or a proximity of the one or more third parties to the first user.

6. The non-transitory computer-readable medium of claim 5, wherein at least one of the determined one or more third parties is present in at least one of the digital assets associated with the identified one or more moments in time.

7. The non-transitory computer-readable medium of claim 1, wherein the instructions for causing the one or more processors to determine, for at least one time instant, one or more of the associated digital assets to share with one or more third parties further comprise instructions for causing the one or more processors to: The one or more third parties are determined to be subject to the one or more filtering options.

8. A non-transitory computer-readable medium comprising computer-executable instructions stored thereon, the computer-executable instructions being configured to cause one or more processors to: Obtaining a metadata collection associated with a collection of digital assets; Obtaining a knowledge graph metadata network for the collection of digital assets; receiving one or more first digital assets from a third party; identifying one or more moments within the collection of digital assets based at least in part on the knowledge graph metadata network and the one or more first digital assets received from the third party, wherein each moment is associated with one or more second digital assets; determining, for at least one identified moment in time, one or more of the associated second digital assets to share with the third party based at least in part on the one or more first digital assets received from the third party; as well as A suggestion is provided to the first user to share the determined one or more associated second digital assets with the one or more third parties, wherein the sharing suggestion is determined based on a relationship of the one or more third parties and the at least one identified moment in time.

9. The non-transitory computer-readable medium of claim 8, further comprising computer-executable instructions stored thereon, the computer-executable instructions for causing one or more processors to: receiving an indication from the first user to share the determined one or more associated second digital assets with the one or more third parties; and The determined one or more associated second digital assets are shared with the one or more third parties.

10. The non-transitory computer-readable medium of claim 9, wherein the instructions for causing the one or more processors to share the determined one or more associated second digital assets with the one or more third parties further comprise instructions for causing the one or more processors to: An indication to share the determined one or more associated second digital assets with the one or more third parties is transmitted to a server holding a copy or reference to the determined one or more associated second digital assets.

11. The non-transitory computer-readable medium of claim 8, wherein the instructions for causing the one or more processors to identify one or more moments in time within the collection of digital assets further comprise instructions for causing the one or more processors to: analyzing location metadata of the one or more first digital assets; analyzing temporal metadata of the one or more first digital assets; and The location metadata and the time metadata of the one or more first digital assets are matched to the knowledge graph metadata network.

12. The non-transitory computer-readable medium of claim 11, wherein the instructions for causing the one or more processors to match the location metadata and the time metadata of the one or more first digital assets to the knowledge graph metadata network further comprise instructions for causing the one or more processors to: A fuzzy search is performed on the knowledge graph metadata network, wherein the fuzzy search allows for inexact matching with the knowledge graph metadata network.

13. The non-transitory computer-readable medium of claim 12, wherein the degree to which the fuzzy search allows for inexact matches with the knowledge graph metadata network is based at least in part on a density of the collection of digital assets.

14. The non-transitory computer-readable medium of claim 12, wherein the fuzzy search allows the degree of inexact matching with the knowledge graph metadata network to scale in proportion to the magnitude of the location metadata analyzed.

15. The non-transitory computer-readable medium of claim 12, wherein the fuzzy search allows the degree of inexact matching with the knowledge graph metadata network to scale in proportion to the duration of the temporal metadata being analyzed.

16. A non-transitory computer readable medium comprising computer executable instructions stored thereon, the computer executable instructions for causing one or more processors to: Obtaining a metadata set associated with a collection of digital assets, wherein the collection of digital assets includes one or more moments in time, and wherein each of the one or more moments in time is associated with one or more digital assets in the collection of digital assets; Obtaining a knowledge graph metadata network for the collection of digital assets; receiving an incoming message from a sender via a first device; detecting a sharing intent in the incoming message; extracting one or more features from the content of the incoming message; comparing one or more extracted features to the one or more moments of the collection of digital assets and the knowledge graph metadata network; determining, based at least in part on the act of comparing, at least one of the one or more moments in time that matches the one or more extracted features; determining, in response to the incoming message, for at least one determined moment in time, one or more of the digital assets associated with the at least one moment in time to share with the sender; as well as A suggestion is provided, via the first device, to share the determined one or more associated digital assets with the sender, wherein the sharing suggestion is determined based on a relationship of the sender to at least one identified moment in time.

17. The non-transitory computer readable medium of claim 16, further comprising computer executable instructions stored thereon, the computer executable instructions for causing one or more processors to: receiving, via the first device, an indication to share the determined one or more associated digital assets with the sender; and The determined one or more associated digital assets are sent to the sender via an outgoing message.

18. The non-transitory computer-readable medium of claim 17, wherein the instructions for causing the one or more processors to send the determined one or more associated digital assets to the sender via an outgoing message further comprise instructions for causing the one or more processors to: An indication to share the determined one or more associated digital assets with the sender is transmitted to a server holding a copy or reference to the determined one or more associated digital assets.

19. The non-transitory computer-readable medium of claim 16, wherein the instructions for causing the one or more processors to extract one or more features from the incoming message further comprise instructions for causing the one or more processors to: At least one of the one or more features is enhanced using at least one of: synonyms, word embeddings, and natural language processing (NLP).

20. The non-transitory computer-readable medium of claim 16, wherein the instructions for causing the one or more processors to determine, for the at least one time instant, one or more of the associated digital assets to share with the sender further comprise instructions for causing the one or more processors to: The one or more associated digital assets are determined based at least in part on a relationship of the sender to the identified one or more moments in time.

Images