Method and system for managing media content associated with message context on mobile computing device

Abstract

Content matching for use in a hash tag-centric messaging platform may include a system comprising: a computer-to-computer interface configured to facilitate communication between a messaging platform adapted to associate a hash tag with a message and a content matching platform, the communication between the messaging platform and the content matching platform includes at least one of a message and a hash tag. The system may include a natural language processing facility of a content matching platform that generates an output including at least one of an understanding, a topic, a sentiment, and an intent of a message delivered from a messaging platform. The system may also include a metadata matching facility in communication with the content matching platform, the metadata matching facility configured to identify candidate hash tags by determining a similarity of hash tags in the pool of hash tags to an output of the natural language processing, the content matching facility transmits the at least one candidate hash tag to a messaging platform over a computer-to-computer interface.

Description

[0001] Related Applications

[0002] This application claims priority to U.S. Provisional Patent Application No. 62 / 597,309, filed on December 11, 2017, which is hereby incorporated by reference in its entirety as if fully set forth herein.

[0003] This application is related to U.S. Patent Application No. 16 / 158,000, filed on October 11, 2018, which is a continuation of U.S. Patent Application No. 15 / 725,645, filed on October 5, 2017 (now U.S. Patent No. 10,104,228), which claims the benefit of U.S. Provisional Patent Application No. 62 / 459,742, filed on February 16, 2017, and U.S. Provisional Patent Application No. 62 / 486,495, filed on April 18, 2017. U.S. Patent Application No. 15 / 725,645, filed on October 5, 2017, is a continuation-in-part of International Application No. PCT / US2016 / 30083, filed on April 29, 2016, and published as WO2016 / 178984, which claims priority to U.S. Provisional Patent Application No. 62 / 155,706, filed on May 1, 2015. Each of the above-identified applications is hereby incorporated by reference in its entirety as if fully set forth herein. Technical Field

[0004] This application relates to the field of mobile devices, and more particularly to the management of content such as ringtones and videos that can be played when one user contacts another user via a mobile device. Background Art

[0005] The proliferation of mobile devices has enabled a wide range of highly personalized social interactions, such as sharing photos, videos, and music, among other types of content. However, the contacts functionality of a typical mobile device is fairly generic, typically providing an alphabetical list of names with phone numbers and other basic contact information. Calls and texts between users typically identify the users, and may even play a selected ringtone, selected by the recipient of the call or text, to play when a particular individual calls or texts, but the event of the call or text is otherwise non-customized.

[0006] Users can create content that can be played at contact events, but in current systems, another user would need to undertake a lot of interaction in order to play them, and without a lot of effort on the part of the recipient of the call or text, the content would often become stale.

[0007] There exists a need for improved methods and systems for managing contacts modules and functionality of mobile devices. Summary of the invention

[0008] In an embodiment, a method and system for content matching may include a system, the system including: a computer-to-computer interface configured to facilitate communication between a messaging platform adapted to associate a hash tag with a message and a content matching platform, the communication between the messaging platform and the content matching platform including at least one of a message and a hash tag. The system may include: a natural language processing facility of the content matching platform, which generates an output including at least one of an understanding, a theme, a sentiment, and an intent of a message transmitted from the messaging platform. The system may also include a metadata matching facility in communication with the content matching platform, the metadata matching facility configured to identify a candidate hash tag by determining a similarity between a hash tag in a pool of hash tags and the output of the natural language processing, the content matching facility transmitting at least one candidate hash tag to the messaging platform via the computer-to-computer interface. The system may further include: an electronic user interface facilitating a user to select one or more of the candidate hash tags transmitted by the content matching facility to the messaging platform for association with a message processed by the natural language processing facility. In the system, the computer-to-computer interface may include an application programming interface (API). In the system, determining the similarity of a hash tag to the output of the natural language processing of the message may include: determining the similarity of metadata associated with the hash tag. In the system, metadata associated with the hash tag may be generated by processing the hash tag using a natural language processing facility. In the system, at least one of the candidate hash tags may indicate a sentiment of the message. In the system, at least one of the candidate hash tags may indicate an intent of the message. In the system, a content matching platform may tag the message using at least one candidate hash tag and transmit the tagged message to a messaging platform. The system may further include: a content matching platform that transmits an output of the natural language processing of the message to the messaging platform.

[0009] In an embodiment, a method and system for content matching may include a system, the system including: a computer-to-computer interface configured to facilitate communication between a messaging platform adapted to associate a hash tag with a message and a content matching platform, the communication between the messaging platform and the content matching platform including at least one of the message and the hash tag. The system may further include: a natural language processing facility of the content matching platform, which generates an output, the output including at least one of the understanding, topic, sentiment, and intent of the message transmitted from the messaging platform, and at least one of a plurality of hash tags in a pool of hash tags. The system may further include: a metadata matching tool configured to determine a candidate hash tag by determining a similarity between the output of the natural language processing for the plurality of hash tags and the output of the natural language processing for the message transmitted from the messaging platform, the content matching facility transmitting at least one candidate hash tag based on the determined similarity to the messaging platform via the computer-to-computer interface. In the system, the output of the natural language processing for at least one candidate hash tag is more similar to the output of the natural language processing for the message than the output of the natural language processing for any other hash tag in the pool of hash tags. The system may further include: an electronic user interface that facilitates user selection of one or more of the candidate hash tags that the content matching facility transmits to the messaging platform for association with the message processed by the natural language processing facility. In the system, the computer-to-computer interface may be an application programming interface (API). In the system, determining the similarity of the hash tag to the output of the natural language processing of the message may include: determining the similarity of metadata associated with the hash tag. In the system, the metadata associated with the hash tag may be generated by processing the hash tag using the natural language processing facility.

[0010] In an embodiment, a method and system for content matching may include: transmitting a portion of a message from a messaging platform to a content matching platform. The method may also include: processing the text in the portion of the message using natural language processing, thereby generating an NLP output including at least one of the subject, understanding, intent, and sentiment of the message. The message may also include: determining the similarity of the NLP output to a plurality of hash tags. The method may also include: transmitting at least one candidate hash tag of the message to the messaging platform based on the determined similarity. In the method, determining the similarity may include: comparing a portion of the NLP output to the hash tag. In the method, determining the similarity may include: comparing a portion of the NLP output to metadata describing the hash tag. The method may further include: facilitating user selection of at least one candidate hash tag by presenting a plurality of hash tags in an electronic user interface based on the determined similarity. In the method, transmitting a portion of the message and transmitting at least one candidate hash tag may utilize an application programming interface between the content matching platform and the messaging platform.

[0011] Provided herein are methods and systems for sender-controlled contact media content types (referred to as "SCCMC" in some cases herein) that can be sent among mobile devices, including addressing contact systems and methods of such devices to provide rich interactions among users when contact events occur. Such methods and systems can occur at the level of the contact functionality of the mobile device, such as being initiated directly upon contact and independent of the needs of an application on the recipient's mobile device. In an embodiment, the SCCMC can be played, for example, on the lock screen of the recipient's mobile device when an incoming call occurs, or the SCCMC can be displayed on the contact screen when the phone is unlocked and active, including but not limited to on the home screen of the recipient's mobile device, when the recipient is on a call, and / or when the recipient is using an application on the recipient's mobile device. The methods and systems described herein can allow a first user (user A) to create customized photos, videos, ringtones, etc. that are played on another user's (user B) mobile device under the control of user A, such as when user A calls or texts user B's device. For example, user A can create a video with associated audio content and pass that content to user B's phone, and have the video and audio play on user B's phone when user A contacts user B, such as by text or call. The content can include both voice layer and data layer content in a single broadcast action, and use voice layer and data layer transmission media in a coordinated manner. Such content is referred to herein as sender controlled contact media content item ("SCCMC"), or in some cases "Ving" or "Vyng".

[0012] Provided herein are methods and systems for providing a sender-controlled contact media content data structure, which is adapted to be created using a sender's mobile device and sent to at least one mobile device of at least one recipient; and a host computing system, which includes at least one server for managing the sender-controlled contact media content data structure; wherein the sender-controlled contact media data structure is adapted to be displayed via a contact function of the recipient's mobile device under the control of the sender of the sender-controlled contact media content data structure.

[0013] Methods and systems are provided herein, including the following methods and systems, which include: a sender-controlled contact media content data structure, which is adapted to be created using a sender's mobile device and sent to at least one mobile device of at least one recipient; and a host computing system, which includes at least one server for managing the sender-controlled contact media content data structure; wherein the sender-controlled contact media data structure is adapted to be displayed on the screen of the recipient's mobile device when the recipient's device is turned on and capable of receiving at least one of a data signal and a voice signal, wherein the sender-controlled contact media data structure displays content under the control of the sender of the sender-controlled contact media content data structure.

[0014] In an embodiment, a sender-controlled contact media content data structure uses voice layer and data layer transmission media in a single broadcast action of the sender. In an embodiment, the screen of the recipient's mobile device is the screen displayed when a contact event occurs at the recipient's mobile device. In an embodiment, the screen of the recipient's mobile device is the lock screen of the recipient's mobile device. In an embodiment, the screen of the recipient's mobile device is the home screen of the recipient's mobile device. In an embodiment, the screen of the recipient's mobile device is the active screen of the recipient's mobile device, which is displayed when the mobile device is unlocked. In an embodiment, the screen of the recipient's mobile device is the screen displayed when the recipient is on a call using the recipient's mobile device.

[0015] Methods and systems disclosed herein may include an installable application running on a host computing system for establishing a data connection between a sender's mobile device and at least one mobile device of at least one recipient. In an embodiment, the installable application is configured to allow discovery of nearby peer devices to allow exchange of sender-controlled contact media content data structures between the sender's mobile device and at least one mobile device of at least one recipient. In an embodiment, the installable application enables a user interface that allows creation of a sender-controlled contact media content data structure on the sender's mobile device.

[0016] In an embodiment, a host computing system enables downloading of a sender-controlled contact media content data structure on at least one mobile device of at least one recipient.

[0017] In an embodiment, the sender's mobile device and at least one mobile device of at least one recipient are communicatively coupled to a server, and wherein the server is a content server. In an embodiment, the sender's mobile device is communicatively coupled to a plurality of recipients' mobile devices such that each recipient device receives a contact media content data structure controlled by the sender. In some cases, this is described as a "bomb," "Vyng bomb," or the like.

[0018] In an embodiment, multiple sender-controlled contact media data structures are organized into channels so that a mobile device of at least one recipient plays a series of sender-controlled contact media data structures upon multiple contact events from at least one sender. In an embodiment, the channel is associated with at least one of the following: a topic of interest to the sender, a topic of interest to at least one recipient, and a social graph of at least one of the sender and at least one recipient. In an embodiment, the sender-controlled contact media data structure is selected by the sender from an organized library of sender-controlled contact media data structures. In an embodiment, the library is algorithmically organized based on topics of interest indicated by at least one of the sender and the recipient. In an embodiment, the library is organized by editing based on the topics of the channel.

[0019] The methods and systems described herein may include methods and systems for employing a sender-controlled contact media content data structure created using a sender's mobile device; providing a host system that enables the sender-controlled contact media content data structure to be transmitted from the sender's mobile device to at least one mobile device of at least one recipient; and causing the sender-controlled contact media content data structure to be displayed on a screen of at least one mobile device of at least one recipient when the at least one recipient's device is turned on and is capable of receiving at least one of a data signal and a voice signal, wherein the sender-controlled contact media data structure displays content under the control of the sender of the sender-controlled contact media content data structure. In an embodiment, the sender-controlled contact media content data structure uses voice layer and data layer transmission media in a single broadcast action of the sender. In an embodiment, the screen of the recipient's mobile device is a screen displayed when a contact event occurs at the recipient's mobile device. In an embodiment, the screen of the recipient's mobile device is a lock screen of the recipient's mobile device. In an embodiment, the screen of the recipient's mobile device is a home screen of the recipient's mobile device. In an embodiment, the screen of the recipient's mobile device is an active screen of the recipient's mobile device that is displayed when the mobile device is unlocked. In an embodiment, the screen of the recipient's mobile device is a screen displayed when the recipient is on a call using the recipient's mobile device. In an embodiment, the sender-controlled contact media content data structure includes one or more of audio, video, contact data, and metadata. In an embodiment, sending the sender-controlled contact media content data structure includes sending a text message, wherein the text message includes an embedded link so that at least one recipient can install an application on at least one mobile device of at least one recipient when at least one recipient clicks on the embedded link, wherein the installable application establishes a data connection between the sender's mobile phone and the at least one recipient's mobile phone.

[0020] In an embodiment, the methods and systems described herein may further include downloading a sender-controlled contact media content data structure on at least one mobile device of at least one recipient using an embedded link, wherein the downloading of the sender-controlled contact media content data structure enables display of the sender's contacts as associated with the sender-controlled contact media content data structure on at least one mobile device of at least one recipient.

[0021] In an embodiment, the methods and systems described herein may further include: initiating contact by the sender's mobile device with at least one mobile device of at least one recipient, such that when contact between the sender's mobile device and at least one mobile device of at least one recipient is initiated, a contact media content data structure controlled by the sender is displayed on at least one mobile device of at least one recipient. In an embodiment, contact is initiated between the sender's mobile device and at least one mobile device of at least one recipient when the sender places a phone call. In an embodiment, contact is initiated between the sender's mobile device and at least one mobile device of at least one recipient when the sender sends a text message to at least one mobile device of at least one recipient.

[0022] In an embodiment, the methods and systems described herein may further include enabling a user interface on a sender's mobile device to allow the creation of a sender-controlled contact media content data structure through the user interface. In an embodiment, the user interface further allows editing of a sender-controlled contact media content data structure from an external source that stores the sender-controlled contact media content data structure. In an embodiment, the external source includes one of a cloud repository and a library of a third-party content provider. In an embodiment, the user interface allows the creation of a sender-controlled contact media content data structure associated with the sender's emotional state. In an embodiment, the emotional state is indicated by a data field in the metadata associated with the sender-controlled contact media content data structure. In an embodiment, the emotional state is indicated by one or more of a facial expression, an image, a sound, and an emoticon.

[0023] In an embodiment, the methods and systems described herein may further include setting privacy controls via the installable application to limit the release of the sender-controlled contact media content data structure to a selected audience.

[0024] The methods and systems disclosed herein may include methods and systems for isolating a created sender-controlled contact media content data structure into one or more of audio, video, and metadata components; associating each component of the created sender-controlled contact media content data structure with a specific tone; and storing the tone of each component locally on the sender's mobile device. In an embodiment, the methods and systems described herein may further include uploading the tone of each component on an application server or a content server. In an embodiment, the methods and systems described herein may further include: allowing preview of the sender-controlled contact media content data structure; and adding customization details to the sender-controlled contact media content data structure before or after sending the sender-controlled contact media content data structure to at least one recipient. In an embodiment, causing the sender-controlled contact media content data structure to be displayed on the screen of at least one mobile device is triggered by detecting that the recipient's mobile device enters an area defined by a geo-fence.

[0025] The methods and systems disclosed herein may include establishing a personal channel with at least one media item by accessing an installable application from a mobile device and selecting at least one preconfigured menu item that grants access to at least one media item as part of the personal channel. The personal channel may include at least one media item selected from at least one media content source. The at least one media content source may include at least one of sender-controlled contact media content received from other devices, content locally available on the mobile device, content created or captured by the mobile device, content obtained from a social network server, and combinations thereof. The methods and systems disclosed herein may further include sharing the sender's personal channel with at least one recipient of the sender-controlled contact media content data structure.

[0026] The method and system may further include initiating contact with a mobile device of at least one recipient for notifying the mobile device of the at least one recipient to provide media content for use as a sender-controlled contact media content data structure associated with the at least one recipient. In some examples, the at least one recipient may be selected from a contact list or phone book available on the sender's mobile device. The method and system may enable providing a link to a website of the at least one recipient for uploading some media content, which may be used as a sender-controlled contact media content data structure associated with the at least one recipient.

[0027] Provided herein are methods and systems for establishing a personal channel of media content items to be displayed on a lock screen of a user's mobile device in response to a triggering contact event. These may include: providing a host system that enables processing of a contact media content data structure for controlled display of the contact media content data structure on a lock screen of at least one mobile device; and enabling the contact media content data structure to be displayed on a screen of at least one mobile device of at least one recipient when the device of at least one recipient is turned on and is capable of receiving at least one of a data signal and a voice signal, wherein upon receiving a triggering contact event for at least one mobile device, the contact media data structure displays the media content on the lock screen of at least one mobile device; and enabling a user to establish a personal channel of contact media content with at least one media item by accessing an installable application on at least one mobile device and selecting at least one preconfigured menu item, the preconfigured menu item designating at least one media item as part of a personal channel. In an embodiment, the personal channel includes: at least one media item selected from at least one media content source, wherein at least one media content source includes at least one of sender-controlled contact media content received from another device, content locally available on at least one mobile device, content created using the mobile device, and content obtained from a server. The method and system may further include: enabling a user to share a personal channel with at least one other user of a mobile application configured to interact with a host system. The method and system may further include: initiating contact with a mobile device of at least one other user of a mobile device, the mobile device configured to interact with the host system to request a contact media content item to be displayed when another user initiates a triggered contact to a user of the at least one mobile device. In an embodiment, at least one other user is selected from a contact list available on the sender's mobile device. In an embodiment, notifying the mobile device of the at least one other user includes: sending a link to a website that invites the other user to upload media content to be displayed when the other user initiates a triggered contact with the user of the at least one mobile device.

[0028] In an embodiment, the methods and systems described herein may include a system that may include an application with a lock screen function executed on a recipient computing device, the device with a lock screen function facilitating user interface access to communication functions of the recipient computing device when the computing device remains in a locked state, wherein the application with a lock screen function intercepts communications received by the recipient device, the communications being associated with at least one communication application or service operable on the recipient computing device. The system may further include: a content association application executed on the recipient computing device, wherein the content association application facilitates association of content items with communications based on a relevance of the content items to the communications, the relevance being determined from processing the intercepted communications to identify associations based on metadata describing the content items stored in a library of content items. The system may further include: automatically associating content items in response to detecting an indication of the intercepted communications. The automatic association may include: associating at least one of a GIF and a video with the communications. The automatic association may further include: determining an emotional state of the communications, and associating content in response to the determined emotional state. The automatic association may further include: determining a context of the communications, and associating content in response to the determined context.

[0029] The methods and systems described herein may include a system that may include a sender computing device configured with a sender-controlled contact media content-based application (SCCMC application), the SCCMC application executing on the sender device and interfacing with a sender state detection application executing on the sender device, wherein the sender state detection application detects at least one of a sender's emotion, a sender's physical state, a sender's emotional state, and a sender's psychological state, and wherein the SCCMC application uses the sender's detected state to determine a SCCMC structure corresponding to the detected state, and associates information identifying the determined SCCMC structure with an outgoing message of a recipient. In the system, the outgoing message is associated with a telephone call from the sender to the recipient. In the system, the SCCMC application further determines a content channel associated with at least one of the sender and the recipient, and bases the determined SCCMC structure on the determined content channel.

[0030] The methods and systems described herein may include a method that may include adapting a sender-controlled contact media content structure in response to receiving an indication of a current state of a sender of the sender-controlled contact media content structure, wherein the state is one of a mood, a physical state, an emotional state, and a psychological state. In the method, the state is determined in response to facial recognition of the sender simultaneously with the sender sending a message from the sender's computing device to the recipient's computing device. In the method, the adapting occurs while the recipient computing device is processing a contact event resulting from receiving a message from the sender's computing device.

[0031] The methods and systems described herein may include a method that may include: operating a sender-controlled contact media content structure creation platform on a computing device, which, in response to intercepting a message transmitted between a messaging application resident on a sender's mobile device and a recipient's mobile device, provides customized services of the platform to a user of the mobile device on which the message was intercepted. In the method, the message is intercepted on the sender's mobile device before it is sent to the recipient. In the system, the message is intercepted on the recipient's mobile device before the messaging application resident on the recipient's mobile device receives the message. In the system, interception includes: utilizing natural language processing to develop an understanding of the message, which is further used to facilitate suggestions for video / image / audio of the SCCMC associated with the message.

[0032] Methods and systems described herein may include a method that may include: operating a sender controlled contact media content (SCCMC) structure contact event processing platform. The platform may perform the following steps, including intercepting a message associated with other messaging applications residing on a mobile device, wherein the message is directed to a first messaging application of the other messaging applications; processing the message using natural language processing to determine at least one SCCMC structure; forwarding the determined at least one SCCMC structure to a SCCMC unified messaging user interface; processing the SCCMC structure using the unified messaging interface; presenting at least one of text and an image determined from processing the SCCMC structure in the unified messaging interface; and delivering the message compatible with the other messaging applications to the first messaging application. In the method, the message is intercepted on the recipient mobile device. In the method, the platform operation may be performed while locking the recipient's mobile device. In the method, the message is intercepted on the sender's mobile device.

[0033] The methods and systems described herein may include a method that may include, in response to detecting a contact event moving to a mobile device, integrating at least one of audio, video, image, and text indicated by a sender controlled contact media content (SCCMC) structure into an augmented reality environment, the integration indicating the contact event.

[0034] The method and system described herein include: an application with a lock screen function executed on a recipient computing device. The application with a lock screen function is configured to facilitate user interface access to the communication function of the recipient computing device when the recipient computing device remains in a locked state. The application with a lock screen function is configured to intercept communications received by the recipient computing device. The intercepted communications are associated with at least one of the communication applications or communication services operable on the recipient computing device. The method and system further include: a content association application executed on the recipient computing device. The content association application is configured to facilitate associating the media content items with the intercepted communications based on the correlation of the media content items with the intercepted communications. The correlation is determined by automatically processing the intercepted communications to identify the association based on metadata describing the media content items stored in the library of the media content items.

[0035] In an embodiment, the associating of the media content item is automatically performed by the content association application in response to detecting an indication of an intercepted communication.

[0036] In an embodiment, the content association application is configured to associate at least one of a GIF and a video with the intercepted communication.

[0037] In an embodiment, automatically processing the intercepted communication to identify the association includes automatically determining an affective state of the intercepted communication, and automatically associating at least one of the media content items in response to the automatically determined affective state.

[0038] In an embodiment, automatically processing the intercepted communication to identify the association includes determining a context of the intercepted communication and associating at least one of the media content items in response to the determined context.

[0039] The methods and systems described herein include: a sender computing device configured with a sender-controlled contact media content-based application (SCCMC application). The SCCMC application is configured to execute on the sender device and interface with a sender state detection application executed on the sender device. The sender state detection application is configured to detect a detected state, the detected state comprising at least one of a sender's emotion, a sender's physical state, a sender's emotional state, and a sender's psychological state. The SCCMC application uses the detected sender's state to determine a SCCMC structure corresponding to the detected state, and associates information identifying the determined SCCMC structure with a recipient's outgoing message.

[0040] In an embodiment, the outgoing message is associated with a telephone call from the sender to the recipient.

[0041] In an embodiment, the SCCMC application is configured to determine a content channel associated with at least one of the sender and the receiver, and to base the determined SCCMC structure on the determined content channel.

[0042] Methods and systems described herein include adapting a sender-controlled contact media content structure in response to receiving an indication of a sender's current state of the sender-controlled contact media content structure. The sender's current state is one of an emotion, a physical state, an emotional state, and a psychological state.

[0043] In an embodiment, concurrently with the sender sending the message from the sender's computing device to the recipient's computing device, the sender's current status is determined in response to facial recognition of the sender.

[0044] In an embodiment, the adaptation of the sender-controlled contact media content occurs while the recipient's computing device is processing a contact event resulting from receipt of a message from the sender's computing device.

[0045] The methods and systems described herein include operating a sender-controlled contact media content structure creation platform on a computing device, which, in response to intercepting a message transmitted between messaging applications residing on the sender's mobile device and the recipient's mobile device, provides customized services of the platform to a user of the mobile device on which the message was intercepted.

[0046] In an embodiment, the message is intercepted on the sender's mobile device before being sent to the recipient.

[0047] In an embodiment, the message is intercepted on the recipient's mobile device before the message is received by a messaging application resident on the recipient's mobile device.

[0048] In an embodiment, interception comprises utilizing natural language processing to develop an understanding of the message, and wherein the understanding is further utilized to facilitate suggesting video / image / audio of sender-controlled contact media content associated with the message.

[0049] The methods and systems described herein include: operating a sender controlled contact media content (SCCMC) structure contact event processing platform, which includes intercepting messages associated with other messaging applications residing on a mobile device, wherein the message is directed to a first messaging application in another messaging application. The methods and systems also include: processing the message using natural language processing to determine at least one SCCMC structure, and forwarding the determined at least one SCCMC structure to a SCCMC unified messaging user interface. The methods and systems further include: processing the SCCMC structure using a unified messaging interface; presenting at least one of text and an image determined from processing the SCCMC structure in the unified messaging interface; and delivering the message compatible with the other messaging application to the first messaging application.

[0050] In an embodiment, the message is intercepted on the recipient's computing device.

[0051] In an embodiment, the message is intercepted on the recipient's computing device while the recipient's computing device is locked.

[0052] In an embodiment, the message is intercepted on the sender's computing device.

[0053] The methods and systems described herein include: in response to detecting a contact event of a mobile device to a mobile device, integrating at least one of audio, video, image, and text indicated by a sender controlled contact media content (SCCMC) structure into an augmented reality environment. The integration of at least one of the audio, video, image, and text is an indicator of a contact event.

[0054] In an embodiment, the method and system include: in response to receiving an indication of a current state of a sender of the sender-controlled contact media content structure, adapting the sender-controlled contact media content structure. The sender's current state is one of an emotion, a physical state, an emotional state, and a psychological state.

[0055] In an embodiment, concurrently with the sender sending the message from the sender's computing device to the recipient's computing device, the sender's current status is determined in response to facial recognition of the sender.

[0056] In an embodiment, the integration includes utilizing natural language processing to develop an understanding of a message from a sender, and wherein the understanding is further utilized to facilitate suggesting video / image / audio of sender-controlled contact media content associated with the message. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In the accompanying drawings, like reference numerals refer to identical or functionally similar elements throughout the separate views and, together with the following detailed description, are incorporated into and form a part of the specification to further illustrate various embodiments and explain various principles and advantages according to the systems and methods disclosed herein.

[0058] Figure 1 The architecture of the method and system for SCCMC according to embodiments of the systems and methods disclosed herein is illustrated.

[0059] Figure 2A and Figure 2B A process for creating and sharing a SCCMC according to an embodiment of the systems and methods disclosed herein is illustrated.

[0060] Figure 2C A process for initiating a telephone call according to an embodiment of the systems and methods disclosed herein is illustrated.

[0061] Figures 3A-3G Illustrated is the creation of a SCCMC in accordance with embodiments of the systems and methods disclosed herein.

[0062] Figures 4A-4F An example of a SCCMC according to embodiments of the systems and methods disclosed herein is illustrated.

[0063] Figure 5A , 5B 5C illustrates user A sharing a SCCMC with user B according to embodiments of the systems and methods disclosed herein.

[0064] Figure 6 User A is illustrated adding customization details to the SCCMC according to an embodiment of the systems and methods disclosed herein.

[0065] Figure 7 It illustrates user B setting the SCCMC received from user A according to an embodiment of the systems and methods disclosed herein.

[0066] Figure 8 It is illustrated that user B creates a SCCMC and shares the SCCMC with user A and others according to an embodiment of the systems and methods disclosed herein.

[0067] Fig. 9Illustrated is a SCCMC created by user B for his or her significant other in accordance with an embodiment of the systems and methods disclosed herein.

[0068] Fig.10 Illustrated is a screenshot of the home page - feed of the SCCMC, in accordance with an embodiment of the systems and methods disclosed herein.

[0069] Fig.11A and 11B Illustrated is a notification screenshot of the SCCMC in accordance with an embodiment of the systems and methods disclosed herein.

[0070] Fig.12 Illustrated is a menu screenshot of the SCCMC in accordance with an embodiment of the systems and methods disclosed herein.

[0071] Fig.13 Illustrated are setup screenshots of the SCCMC in accordance with an embodiment of the systems and methods disclosed herein.

[0072] Figures 14A-14I Illustrated are screenshots of a process that enables contact editing between users in accordance with an embodiment of the systems and methods disclosed herein.

[0073] Fig.15A , 15B and 15C illustrate screen shots of a process for transferring contacts with SCCMC that is updated on another user's phone contacts automatically or under user control in accordance with an embodiment of the systems and methods disclosed herein.

[0074] Fig.16 A process for transferring a SCCMC from user A to user B according to an embodiment of the systems and methods disclosed herein is illustrated.

[0075] Figures 17A-17D Illustrated is the growth of the user community of SCCMC through the viral sharing effect according to an embodiment of the systems and methods disclosed herein.

[0076] Fig.18A , 18B and 18C illustrate an example of monetization of a SCCMC according to an embodiment of the systems and methods disclosed herein.

[0077] Figure 19-21 An example of triggering of a SCCMC according to an embodiment of the systems and methods disclosed herein is illustrated.

[0078] Fig. 22 The process of recording the second user's reaction to the SCCMC and sharing it with another party is illustrated.

[0079] Fig.23 The diagram illustrates a process in which the playback of the SCCMC can be triggered by a trigger event.

[0080] Fig.24 A computer system is illustrated.

[0081] Fig.25 A mobile device is illustrated.

[0082] Fig.26 Illustrated are screenshots of a user interface for setting up a personal channel for a SCCMC in accordance with an embodiment of the systems and methods disclosed herein.

[0083] Fig. 27 and 28 Illustrated are screenshots of a user interface for enabling media content based on a user's identity in accordance with an embodiment of the systems and methods disclosed herein.

[0084] Fig.29 Illustrated is a screenshot of a user interface for creating a video for a SCCMC in accordance with an embodiment of the systems and methods disclosed herein.

[0085] Fig.30 A flow chart of a method for providing settings for contextual video messages according to an embodiment of the systems and methods disclosed herein is illustrated.

[0086] Figures 31A-31E Illustrated are screenshots of a user interface for messaging interactions in accordance with an embodiment of the systems and methods disclosed herein.

[0087] Fig.32 Illustrated is a block diagram of a unified messaging platform in accordance with an embodiment of the systems and methods disclosed herein.

[0088] Fig.33 An embodiment of a media content library according to embodiments of the systems and methods disclosed herein is illustrated.

[0089] Fig.34 An embodiment of a composite media content library according to embodiments of the systems and methods disclosed herein is illustrated.

[0090] Fig.35 Illustrated are embodiments of separate media and SCCMC structure libraries in accordance with embodiments of the systems and methods disclosed herein.

[0091] Fig.36 An embodiment of a sender device configuration according to an embodiment of the systems and methods disclosed herein is illustrated.

[0092] Fig.37Illustrated is an embodiment of a unified messaging user interface for media content selection in accordance with an embodiment of the systems and methods disclosed herein.

[0093] Fig.38 Embodiments of recipient device configurations according to embodiments of the systems and methods disclosed herein are illustrated.

[0094] Fig.39

[0013] An embodiment of a unified messaging interface for responding to a modified media content message in accordance with an embodiment of the systems and methods disclosed herein is illustrated.

[0095] Fig.40 Pictured Fig.32 An embodiment of a system further comprising speech-to-text to transcribe audio messages for use with NLP.

[0096] Fig.41 The diagram shows the use of Fig.40 A flow chart of an embodiment of a platform for processing audio messages.

[0097] Fig.42 A flow diagram is illustrated of communicating between a messaging platform and a content matching platform via an API to recommend hash tags instead of image content.

[0098] Fig.43 Exemplary screens for generating and playing back a visual caller ID are illustrated.

[0099] Fig.44 An exemplary screen for leaving a video voicemail is illustrated.

[0100] Fig.45 Illustrated is the process for handling unknown callers.

[0101] Fig.46 Illustration of the post call experience user screen.

[0102] Fig.47 The diagram illustrates the architecture and process of calling a community firewall.

[0103] Fig.48 Illustrate the call participant verification user screen.

[0104] Fig.49 Illustrated is a user screen for saving and playing back visual / video voicemails.

[0105] Those skilled in the art will appreciate that the elements in the figures are illustrated for simplicity and clarity and are not necessarily drawn to scale. For example, the size of some elements in the figures may be enlarged relative to other elements to help improve the understanding of the embodiments of the systems and methods disclosed herein. DETAILED DESCRIPTION

[0106] The present disclosure will now be described in detail by referring to the accompanying drawings and showing various illustrative non-limiting embodiments that describe the present disclosure. However, the present disclosure can be embodied in many different forms and should not be construed as limited to the illustrative embodiments set forth herein; rather, these embodiments are provided so that the present disclosure will be thorough and will fully convey the concept of the present disclosure to those skilled in the art. The claims should be consulted to determine the true scope of the present disclosure.

[0107] Before describing in detail embodiments in accordance with the systems and methods disclosed herein, it should be observed that the embodiments reside primarily in combinations of method steps and / or system components related to enabling a user to associate a service with a product in a manner that enables the user to more conveniently and effectively utilize the benefits of the service associated with the product. Accordingly, system components and method steps are represented by conventional symbols in the accompanying drawings, where appropriate, and only those specific details relevant to understanding the embodiments of the systems and methods disclosed herein are shown so as not to obscure the disclosure with details that would be readily apparent to one skilled in the art.

[0108] All documents mentioned herein are incorporated herein by reference in their entirety. Unless otherwise expressly stated or clear from the context, references to items in the singular should be understood to include items in the plural, and vice versa. Grammatical conjunctions are intended to express any and all transitional conjunctions and transitional conjunction combinations of clauses, sentences, words, etc., that are combined, unless otherwise stated or clear from the context. Thus, the term "or" should generally be understood to mean "and / or" and the like.

[0109] Unless otherwise indicated herein, the description of numerical ranges herein is not intended to be restrictive, but refers individually to any and all numerical values ​​falling within the range, and each individual value in such a range is incorporated into the specification, just as it is recorded individually herein. When accompanied by numerical values, the words "approximately", "approximately", etc. are to be interpreted as indicating deviations, as will be appreciated by those skilled in the art, to operate satisfactorily for the intended purpose. The range of values ​​and / or numerical values ​​is provided herein only as an example, and does not constitute a limitation on the scope of the described embodiments. The use of any and all examples or exemplary language ("for example", "such as", etc.) provided herein is intended only to better illustrate the embodiments, and does not limit the scope of the embodiments or claims. Any language in the specification should not be interpreted as indicating any unclaimed elements that are essential to the practice of the embodiments.

[0110] In the following description, it is understood that terms such as "first", "second", "third", "above", "below" and the like are words of convenience and are not to be construed as implying a chronological order or otherwise limiting any corresponding elements, unless expressly stated otherwise.

[0111] As used herein, the term "tone" can be used to refer to a single instance of a sender-controlled contact media content type, which can include one or more of audio, video (which can include video captured by a camera), voice, computer-generated video, animation, or other video-type content (which can be displayed in any format on a mobile device (such as a mobile phone)), contact data, and / or metadata. A single tone can be adapted to be broadcast in a coordinated manner by the sender in a single action over both the voice layer and data layer transmission media. As described above, for convenience, SCCMC is also referred to as "Vyng" in this document (including in the various figures), which is an example of a brand name for SCCMC that users can understand to refer to this novel type of content item.

[0112] Figure 1 An embodiment of an architecture 100 for SCCMC is illustrated. The architecture 100 includes: a mobile phone 102 of user A, alternatively referred to herein as a sender mobile phone 102. The architecture 100 further includes: a mobile phone 104 of user B, alternatively referred to as a recipient mobile phone 104. The sender mobile phone 102 and the recipient mobile phone 104 are each communicatively connected to a content server 112. The architecture 100 can run an application (in some cases referred to herein as an App) that can establish a data connection or tunnel between the two users' phones and can control the creation and display of SCCMC-type content between the two devices 102 and 104. The architecture 100 and the SCCMC-type content data structure can be used with any type of mobile device or mobile device operating system or platform, such as Android TM Operating system, iOS TM operating system or other operating systems. Unless the context indicates otherwise (such as where a particular operating system lacks functionality described in conjunction with a particular embodiment), the embodiments provided herein that describe a particular operating system should be understood to be applicable to other operating systems. In addition to using and integrating with mobile operating system platforms, architecture 100 and SCCMC-type content items can also be integrated with and layered within other applications, including applications that host developer platforms, such as Facebook Messenger TM and Viber TMIntegration with apps that have calling features, including applications that use voice over IP, allows SCCMC-type content items to take advantage of platforms that support such applications, including generating SCCMC-type content items to be delivered upon an incoming call. This can be accomplished by, among other things, using developer platform resources associated with such applications, such as developer user interfaces, application programming interfaces, developer tools, and the like, including integration with the content server 112 described in conjunction with architecture 100.

[0113] Figure 2A A process for creating and sharing SCCMC tones (in some cases simply referred to as "tones") is illustrated. First, at step 1, the architecture 100 can enable a user interface on a sender mobile phone 102 to allow the sender to create video and / or audio content, or a SCCMC tone. The user can edit audio and / or video content from external sources (e.g., stored in the cloud or from a library of third-party content providers), internal sources (e.g., from audio / video saved on the phone), and can create new video / audio. The user can set privacy controls via the App to limit the publication of the SCCMC tone to a selected audience.

[0114] The user's composite creation may be separated into audio, video, and metadata components. These components may be associated with a particular tone, such as by being assigned attributes of the tone. The tone may then be stored locally on the device 102. The tone may be uploaded to an application server or content server 112 and may be persistently stored, such as in a storage device associated with the server 112 or in a cloud-based storage device.

[0115] In an exemplary embodiment, a first user may create a SCCMC at step 1, such as using an App. At step 2, the first user may then share the SCCMC with a friend (such as with a second user), such as via a text / SMS message (such as in the form of a snippet, link, etc. in the text). At step 3, the first user may preview the SCCMC and / or add other customization details before or after sharing the SCCMC with one or more other users. At step 4, the second user may click on the link and install the App, which may set up the first user's SCCMC on the second user's phone 104. Either user may send the SCCMC to multiple people, and may use the community around the App, such as within a group of friends, within a school district, within a work group or enterprise, etc. When the second user first downloads the App, the first user's contacts may now be shown to be associated with the SCCMC tone created by the first user. The second user may further create a tone at step 5A. At step 6, for example, the second user may share this SCCMC with the first user and / or other friends. For example, the second user may create a customized SCCMC for his spouse at step 5B.

[0116] Figure 2B A process 200 for creating and sharing SCCMC tones (referred to simply as "tones" in some cases, or as "Vings" or "Vyngs" in other cases) is illustrated. First, at step 202, the architecture 100 can enable a user interface on a sender mobile phone 102 to allow the sender to create video and / or audio content or SCCMC tones. The user can edit audio and / or video content from external sources (e.g., stored in the cloud or from a library of third-party content providers), internal sources (e.g., from audio / video saved on the phone), and can create new video / audio. The user can set privacy controls via the App to limit the publication of the SCCMC tones to a selected audience.

[0117] The user's composite creation may be separated into audio, video, and metadata components. These components may be associated with a particular tone, such as by being assigned attributes to the tone. The tone may then be stored locally on the device 102. The tone may be uploaded to an application server or content server 112 and may be persistently stored, such as in a storage device associated with the server 112 or in a cloud-based storage device.

[0118] exist Figure 2BIn the exemplary embodiment illustrated in FIG. 2 , a first user may create a SCCMC at a first step 202, such as using an App. The SCCMC may be created using a standard encoding protocol. The standard encoding protocol may include HTML5, Flash, MP4 / H.264, WebM, Ogg, 3GP / MPEG4, etc. At a second step 204, the first user may upload the SCCMC to a server. The server may be a server hosted on a cloud hosting service. Cloud hosting services may include Amazon Web Services TM (AWS), Microsoft Azure infrastructure services TM 、Google CloudPlatform TM 、CenturyLinkCloud TM (CLC), VMware vCloud Air TM 、IBM SoftLayer TM , Rackspace TM etc.

[0119] At the third step 208, a dual push notification may be sent from the server to the second user. The dual push notification may be sent via the first network 220, the second network 222, or both. The first network 220 and the second network 222 may be on separate infrastructures operated by different operators, such as Amazon Web Services. TM SMS infrastructure, Google TM Cloud messaging infrastructure, Pushy TM MQTT infrastructure, mobile telecommunications operator's infrastructure, etc. Sending push notifications through multiple separate infrastructures can improve the reliability of the push notifications sent from the first user received by the second user.

[0120] At a fourth step 210, the second user may receive a push notification on the second user's mobile phone 104. The push notification may include information telling the second user's mobile phone 104 to prepare to play the SCCMC created by the first user. The second user's mobile phone 104 may download the SCCMC. The SCCMC may be installed on the second user's mobile phone 104 to play on the screen of the second user's mobile phone 104, such as at the lock screen, on the home screen, in an application, or on a screen displayed when the recipient is on a call. At a fifth step 212, the second user may send a push notification to the first user. The push notification sent by the second user to the first user may notify the first user that the second user received the SCCMC sent by the first user. The push notification sent by the second user to the first user may also notify the first user that the SCCMC sent by the first user to the second user is ready to play on the second user's mobile phone 104.

[0121] Figure 2C A process and flow 228 for initiating a phone call is illustrated. As illustrated in the first step 238, a phone call may be initiated when a first user receives a push notification from a second user. In a second step 240, the first user may initiate a phone call from the first user's mobile phone 102 to the second user's mobile phone 104. The phone call may be initiated over a voice network 230, a data network 232, or both, and may be associated with a message from a sender to a receiver. In an embodiment, the call may use two transport layers (voice and data). The voice network 230 may be a cellular telephone network. The phone call initiated over the data network 232 may be a voice over internet protocol (VOIP) phone call. The VOIP phone call may be initiated using a second push notification sent from the first user to the second user. The second push notification may include a session ID. A third party service may be used to initiate the VOIP call. Top Box TM , Skype TM , Google Hangouts TM , WhatsApp TM 、WeChat TM Wait can provide third-party services.

[0122] At a third step 242, the second user may receive a phone call from the first user's mobile device 102 on the second user's mobile device 104. Upon receiving the phone call from the first user's mobile device 102, the second user's mobile device 104 may play the SCCMC, as illustrated in subsequent steps 244 and 248.

[0123] Figures 3A-3GDetailed screenshots of the user interface of the App are illustrated, through which the first user can create a new SCCMC. The first user can create a SCCMC with customized ringtones, text tones, and photos as part of his or her contact card, which can be instantly controlled by the first user himself on his or her friend's phone (recipient mobile phone 104). The App can provide an interface for starting with the App, a brief tutorial, the ability to record videos, the ability to select friends, the ability to push videos to other people's phones, and the like. Ideas can be provided to users as prompts. The App can allow the creation and sharing of videos and tunes. The App can allow listening to or viewing other ringtones that have been created. The App can allow the creation of a first SCCMC for the user. The App can also allow various video options (such as slow motion, the use of timers, etc.) to be included in or with the SCCMC. The App can allow the user to edit the page, and / or request additional tools or advanced features. In an example, the source file can be removed, and the App can retain the video, and play other content of the audio, and / or split the video and audio and change either one. The App may allow adding filters, subtitles, messages or music, customization of messages, etc. The App may allow sharing of pages to a list of friends, etc. The App may allow sharing via text message, such as via a button that allows the user to send a preview of the SCCMC with video and / or audio and a link to download the App. If the App is not already installed on the recipient's phone, the App may enable the store to download the App. If the second user has the App, a message may trigger the delivery of the SCCMC tone to the queue.

[0124] For authoring purposes, a button in the App may allow the first user to discover and pick audio, video, or SCCMC tones that he may already own. Videos or other forms of content to be used in the SCCMC may be stored in a library, and the App may facilitate discovery from the library. The App may also allow browsing of other videos from other websites that enable sharing or downloading of videos. In an embodiment, the SCCMC may be selected from an organized library of content of the SCCMC type. The library may be automatically organized under computer control via an algorithm based on topics of interest indicated by the sender or recipient of the SCCMC. The library may be organized by an editor based on topics, such as the topics of channels to which the sender or recipient subscribes.

[0125] The App may allow searching for keywords, and it may automatically upload videos searched for those keywords. The App may also allow the use of freeze-frame photos in the SCCMC, and may allow video or audio payloads. The App may allow adding audio to videos, etc. Audio files may be obtained from already created content (user's library), or from the sender mobile phone 102, or from other users, albums (such as music stores), etc.

[0126] Figure 4A and 4B A call is illustrated being received on a second user's mobile device 104. The call may automatically display a SCCMC tone created by a first user on the second user's phone. In an embodiment, the first user may wish to convey a current emotional state. For example, a user interface on the first user's phone may allow the first user to create multiple tones, each associated with an emotional state. The emotional state may be indicated in the metadata of the tone. When a user initiates a call, the user may indicate the current emotional state, and when the current emotional state is transmitted to the second user's phone, the following process may be initiated, by which the App selects and displays the correct SCCMC tone associated with the current emotional state of the first user. The emotional state may be conveyed through facial expressions, images, sounds, emoticons, or other characteristics associated with a particular tone.

[0127] As shown, the recipient mobile phone 104 may display a SCCMC in which the first user indicates that the recipient mobile phone 104 is ringing / shows a video reflecting the current emotion of the first user. The App may be able to pick what the first user sends or what message is displayed on the second user's phone by selecting an emotion, etc., such as indicated by an emoticon. In an embodiment, each time the first user calls or texts the second user, the correct SCCMC tone may be associated with it and thus shared. In other embodiments, one or more tones are stored on the second user's mobile device 104 and triggered by receiving a call from the number of the first user's device 102, possibly carrying other information (which may be transmitted via the App through the server 112) that allows the selection of specific emotional content. In an embodiment, the App may change the SCCMC during the call. For example, when the first user's emotion changes to a different emotion, the App may be triggered to change the SCCMC during the call. The App may select a new SCCMC for display on the second user's device 104 to reflect the different emotions of the first user. The user may be able to respond to the SCCMC.

[0128] The user may be able to respond to the SCCMC by clicking on the SCCMC. For example, a second user may be able to provide feedback to the first user on a SCCMC sent by the first user to the second user. The feedback sent by the second user may indicate the second user's satisfaction with the SCCMC sent by the first user to the second user. For example, the second user may provide feedback to the first user by clicking on the SCCMC received from the first user. Clicking may generate a heart or other emoticon on the SCCMC displayed on the second user's mobile device 104. Clicking may also generate a heart or other emoticon on the screen of the first user's mobile device 102.

[0129] Figures 5A-5C An embodiment of a user interface feature by which a first user can share SCCMC tones with a second user is illustrated. Figure 5A The diagram illustrates a message that a second user may receive from a first user, prompting the second user to accept delivery of the first user's SCCMC tone and set the first user's SCCMC tone on the second user's mobile device. The message may include text, video, audio, links, and the like. Figure 5B Detailed screenshots of an interface are illustrated by which a first user may select which friends the first user would like to have receive that particular SCCMC tone. Figure 5C Illustrated is a confirmation that may be displayed to the first user when the SCCMC has been successfully shared with the first user's selected friends.

[0130] In various embodiments, content delivery may be accomplished via a client-server architecture (including a cloud architecture), a peer-to-peer architecture, or other content delivery architecture.

[0131] In the client-server approach, the App may periodically query the application server or content server 112 for newly published or updated content. Content updates may also be triggered via push messages from the server 112. Content may be filtered based on privacy settings set by the content creator. The App may automatically download newly published or updated content and may synchronize local data storage on the device 102 with other content, such as content stored in the cloud.

[0132] In the peer-to-peer approach, users (such as the first and second users in the above example) who are in close proximity to each other or connected via any peer-to-peer network connection can discover each other through the App. For example, a peer application can broadcast its presence to a listening port on another peer device to establish its presence and seek to exchange credentials. Once discovered, the peers are displayed to the user in the App. The user can then selectively send SCCMC tones to one or more discovered peers.

[0133] Figure 6Illustrated is a detailed screenshot of a user interface through which a user can record audio for inclusion in or with an SCCMC tone.

[0134] Figure 7 Provides detailed screenshots of the interface through which the user can see snapshots of various SCCMC tones, such as tones received from other users or tones created to be shared with other users. When the App is installed on the user's phone, the user's phone number can be verified. The verification can be done in the background, such as when the phone is running the Android operating system. The App can then use the phone number to automatically set any SCCMC tones shared with the user by any of the user's friends listed in the contacts on the user's phone. The SCCMC tones can be automatically set by seamlessly matching the phone number associated with the SCCMC tones with the phone number of the contacts in the address book on the user's phone. Once the App is installed on the user's phone, any new SCCMC tones sent to the user can be automatically set, and a notification can be displayed in the notification list of the phone. The user's acceptance of the SCCMC tones can be automatic, can be active and require input from the user, and so on. The user can control the SCCMC tone acceptance in the settings of the phone or the App.

[0135] In an embodiment, when an application on a mobile device 102, 104 is launched, the App may register with the operating system of the mobile device, including registering as a listener for incoming voice calls and incoming text messages.

[0136] When an app receives an incoming call notification, it can receive the incoming call number from the operating system. The app can query its own internal database for the SCCMC tone that is linked to the incoming call number. If a match is found, the app launches a new activity to play the SCCMC tone, such as using the default ringtone application on the phone.

[0137] When an app receives an incoming text message (SMS) notification, it can receive the incoming call number from the operating system. The app can query its own internal database, such as the audio to play that is linked to the incoming call number. If a match is found, the app can play the audio file, such as instead of playing the default sound that is usually used to indicate an incoming text message.

[0138] The App may further allow content to be pushed and synced to the contacts section of a friend / contact's phone (such as the recipient mobile phone 104), where the first user can control the video / audio presentation when the recipient mobile phone 104 rings on the first user's call. In an embodiment, a user may find, manipulate and / or create video and / or audio and push it to the contacts feature on another person's phone (such as the recipient mobile device 104).

[0139] Figure 8 and Fig. 9 Illustrated is an example of displaying SCCMC tones on a user's mobile phone 104 when the user receives a call from a person who created and sent the SCCMC tones to the user.

[0140] Fig.10 A screenshot of a "Home - Feed" interface is illustrated. The interface allows a user to see what he or she has downloaded and what he or she has shared with another user. The interface may allow a user to "like" or comment on a SCCMC tone. The interface shows what is waiting in the user's news feed. The interface may also show updates in other users' SCCMC tones.

[0141] Fig.11A and Fig. 11B 1 illustrates a screen shot for displaying notifications. A user may receive notifications via one or more push messages sent through the mobile operating system of the user's phone. Figures 11A-11B As illustrated in the illustrated embodiment, a user may access notifications via a notification icon 1102 displayed in the App. The App may enable a communication channel between a first user and his or her close contacts including a second user, so that the first user and his or her friends may have an automatically updated channel. For example, Fig.11A As shown, Joe may have updated his SCCMC tone and shared the update with the second user through the communication channel. As an example, the first user can change the audio and video associated with the SCCMC tone that the first user shared with the second user. The App can reflect the change in the user profile of the first user on the second user's phone and notify the second user of the change in the notification interface of the second user's phone. The change can reflect a changed text (e.g., audio) tone, a changed SCCMC tone, etc.

[0142] exist Fig.11A In the illustrated embodiment, the first user not only creates a video ringtone, but also a text ringtone. The App will change the text tone accordingly.

[0143] Embodiments may include user linking. For example, a first user and a second user may link their accounts so that shared content is available between the users. The first user may send a text message with an identification token to the second user via the App. The App on the recipient's phone 104 may listen to the incoming text message and automatically send the identification token to the server 112. The two accounts are then linked to the server side so that content can be shared between the first user and the second user.

[0144] User discovery may also be enabled. The App may also display a feed of publicly shared tones. Users may discover and link their accounts to other users in the public feed via private request / accept messages. Users may "like" tones, add to their library (re-Vyng), comment on tones, report tones, and more. A record button may be pinned to the bottom of the feed screen. Videos may play muted as the user scrolls through the feed. Users may maximize a video by tapping on it, and when the video is maximized, the accompanying audio may play.

[0145] Fig.12 The figure shows a screenshot of the Menu screen. From the Menu, the user can access the Home screen, the Tone List (in Fig.12 ), a user's friends list, and other settings.

[0146] Fig.13 A settings screen shot is illustrated. Various settings options may be provided to allow the second user to set limits for his or her contacts, such as a preview tone before they are automatically updated on the recipient mobile device 104.

[0147] Figures 14A-14I Various screenshots of a process for enabling contact editing between a first user and a second user using SCCMC tones are illustrated. A channel is provided to enable contact editing between a first user and a second user. Edits made by the first user to contacts provided to the second user may be done automatically, or may be done under the control of the first user, the second user, or both the first and second users. The user may allow the SCCMC to be saved, synchronized, or automatically updated.

[0148] The first user can go into the payload that the first user sends to the second user. The first user can put content in the payload in the form of a profile picture, name, and email. If the first user changes any of this content, the second user receives an update (e.g., the first user changes his or her profile picture, email, etc.), which can automatically update the content on the recipient's phone 104, or the recipient can have a setting that controls updates to the recipient's phone, such as by allowing the payload to be updated under control or automatically, based on the settings selected by the recipient.

[0149] The SCCMC tone created for the first user may be shared with the second user, the world, or others selected by the first user. The first user may decide not to share the SCCMC tone.

[0150] The first user may select an existing video or audio to add to the SCCMC audio track (eg, Fig.14E and 14F ), newly captured audio (such as Figure 14G and 14H ), and music (as shown Fig.14I Newly captured audio can be automatically captured and published.

[0151] In an example, the SCCMC tone can exist on the recipient's phone 104 at the level of the contacts function, rather than in an application that the second user must locate, access, and use. The contact card can create all the features of the SCCMC tone using content obtained from the first user's social network. If the first user calls or texts the second user, the SCCMC will display the first user's contacts whenever the second user uses the normal functions of the phone 102 to interact with the call or text from the first user. Thus, the SCCMC tone is played within the contacts function of the recipient's phone without the recipient having to interact with the application.

[0152] Fig.15A , 15B and 15C illustrate screen shots that provide a process for transferring a contact with a SCCMC tone that is then updated on another user's phone contacts automatically or under user control. Fig.15A The screen shot that provides interface is illustrated, and this interface can allow the first user to select who the first user wishes to share SCCMC tone with.The first user can choose to share SCCMC tone publicly with the world, with all friends or with selected friends, for example. Fig. 15B Illustrated are screenshots providing an interface that may allow a second user who receives the SCCMC from a first user to enable or disable the SCCMC. Fig. 15CIllustrated are screen shots providing an interface that may allow a second user to edit a SCCMC received from a first user and control the behavior of the SCCMC on the second user's phone.

[0153] Various content control functions may be enabled. When the App receives new or updated SCCMC tone information, the App may update the local storage of metadata in the application database and update the local contact information of the registered user using the fields stored in the data record of the SCCMC tone. The App may also convert the ringtone audio portion of the SCCMC tone into an acceptable format for a ringtone or text tone on the local device, save the ringtone to a shared data storage device in the phone's operating system, and register the ringtone with the operating system. Using the operating system's application programming interface (API), the App may assign the ringtone to the sender on the recipient's phone 104.

[0154] Fig.16 Depicted are steps of a method for enabling the transfer of a SCCMC tone from a first user (User A) to a second user (User B) according to an embodiment. First, in step 1602, a first user, referred to herein as User A, may save audio and video captured during the creation of the SCCMC tone, which may include audio and video captured by the user's phone's camera, audio and / or video retrieved from a saved file, and / or video and / or audio retrieved from other resources (such as found in a search, retrieved from a storage device (including in the cloud), etc.). Next, in step 1604, the audio and video may be separated into two separate files, one for audio and one for video.

[0155] Next, in step 1606, the file may be associated with a database record that may store information about the SCCMC tone, such as identification information of user A, identification information and attribute information of the audio file and the video file, and metadata, such as additional information about user A and information about how user A wishes to share the SCCMC tone. At step 1608, the SCCMC tone may be uploaded from user A's mobile phone 102 (such as a smart phone) to the content server 112, and at step 1610, the content server 112 may publish the SCCMC tone with certain privacy controls based on user A's preferences, such as whether the SCCMC tone is intended for user A only, whether it is intended to be shared with another party (referred to herein as user B or the second user) or multiple parties, or whether it is intended to be public. Such privacy controls for a given SCCMC tone may be captured in the application that user A uses on user A's device to create and upload the SCCMC tone.

[0156] Next, at step 1612, user B may open the App on user B's mobile phone 104, which queries the content server 112 to evaluate the content available to user B's mobile phone 104 in the feed associated with user B, which may include user B's own creations, content shared with user B by others (such as user A), and public content published by various users. The content server 112 and user B's mobile phone 104 may undergo a synchronization process at step 1614, wherein differences between the available feed on the content server 112 and the content already on user B's phone are determined, after which user B's mobile phone 104 downloads any content not already on user B's phone from the content server 112 at step 1616. Once the updated SCCMC tone data has been downloaded, the SCCMC App on user B's mobile phone 104 may convert the audio file portion of the SCCMC tone into a ringtone at step 1618, which is a special class or format that is typically used on mobile operating systems deployed on certain smart phones (such as Android). TM or iOS TM The SCCMC App may register the audio file associated with the SCCMC tone as a ringtone in the ringtone registry at step 1620. Next, at step 1622, the SCCMC tone may use the operating system's feature to set the listener, which enables listening to incoming calls (on devices running Android TM In the case of devices with mobile operating systems, this feature is implemented by the SDK for the operating system). SCCMC App can access devices running Android TM 、iOS TM Or other native APIs exposed by the operating system to listen to the call. At step 1624, if an incoming call from a number having a tone associated therewith (e.g., user A's number) is detected on user B's mobile phone 104, the SCCMC App retrieves the audio and / or video file portions of the downloaded SCCMC tones, and the audio file portions and the video file portions can be played separately or together.

[0157] In some closed operating system environments, it is not possible to detect incoming calls or set a user's ringtone for a user. In such cases, the content server 112 may simply update the contact information on user B's mobile phone 104, such as allowing user B to save the SCCMC tones for calls from user A, but in such cases, actions by user B that would not otherwise be required may be required. In an embodiment, the content server 112 may also push notifications to the user so that user B can view the content of user A that was shared for user B.

[0158] In embodiments, such as involving a closed operating system, the SCCMC content tones may be downloaded and stored on user B's mobile phone 104 for later access, such as when the operating system makes the ability to listen to the call available, or when the user chooses to "jailbreak" or otherwise open the phone.

[0159] In embodiments, other ways of sharing content may involve P2P delivery of SCCMC tones, optionally with separate audio and video components, directly from the first user's phone to the second user, without the need for a content server 112 or a client-server arrangement, or with a content server 112 providing a supplementary role but not required for delivery of the SCCMC. This may include using iOS TM Using Airdrop in a phone with the same operating system TM function, or on other operating systems such as Android TM ) uses a similar P2P delivery function.

[0160] Fig.17A -D illustrates the growth of user communities through viral sharing effects. Fig.17A A first user is illustrated who has created a SCCMC tone. Fig. 17B 1 illustrates a first user sharing a SCCMC tone with a first group of recipients. Fig. 17C As illustrated in , after receiving the SCCMC tone from the first user, the first recipient creates and shares its own SCCMC tone with the second group of recipients, and the first group of recipients become users of the SCCMC tone in the process. Fig.17D The second group of recipients is illustrated as then sharing their own SCCMC tone with the third group of recipients as the second group of recipients becomes users of the SCCMC tone in the process, thereby continuing the growth cycle.

[0161] Fig.18A , 18B and 18C illustrate examples of monetization. Fig.18A Illustrated is a built-in advertising opportunity made available by a receiver of a SCCMC tone, which asks a sender of a SCCMC: how the sender can create / use / share a SCCMC that the sender controls. Fig.18B Illustrated is a premium sender controlled SCCMC tone published on a content feed. Premium SCCMC tones can support content that content creators acquire from the marketplace. The content creator's marketplace can include SCCMC tone creation plug-ins and brand promotional content. Brand promotional content can be uploaded directly to the marketplace by brands and can include brand songs.

[0162] Creators of SCCMC can select content from the market to be included in the SCCMC that users are creating. Creators of SCCMC can select content from a content library provided to creators of SCCMC by the market. The market can provide SCCMC content plug-ins, which can include voice manipulation functions, photo filters, video filters, etc. Premium SCCMC may also include promotional content for brands. Affiliate codes can be linked to specific content components included in the SCCMC or SCCMC, such as audio or video files. Content can be ranked, rated, shared, liked, or subscribed to by content creators, content senders, or content recipients.

[0163] Fig.18C An example of advertising in a sender-controlled SCCMC is illustrated, where an advertisement is displayed to a second user when a first user calls or sends a text message to the second user. The advertisement may include an image component, an audio component, a video component, and the like.

[0164] In another example, the sender of the SCCMC tone can associate a song with the SCCMC tone and select to have the song play on the recipient's mobile device when the sender calls the recipient or sends a text message to the recipient. The song can be a song performed by a favorite artist, another favorite song, etc. In yet another example, when the sender of the SCCMC tone calls the recipient or sends a text message to the recipient, a pre-recorded message from a celebrity personality can play on the recipient's phone with the SCCMC tone. The pre-recorded message can be selected based on how the recipient chooses to interact with the call or text message by accepting, declining, or replying to the call or message.

[0165] Fig.19 1900 is illustrated, in which a SCCMC may be triggered to play by sending the SCCMC from a user to a friend or group of friends. Fig.19In the example illustrated in , as illustrated in the first step 1902, the first user can send the SCCMC to a group of friends, shown in this example as the second user, the third user, and the fourth user. In the second step 1904, the second user, the third user, and the fourth user can receive the SCCMC from the first user. In the third step 1906, when the SCCMC is received on the second user's mobile phone, the SCCMC can be immediately played on the screen of the second user's mobile phone; when the SCCMC is received on the third user's mobile phone, the SCCMC can be immediately played on the screen of the third user's mobile phone, and when the SCCMC is received on the fourth user's mobile phone, the SCCMC can be immediately played on the screen of the fourth user's mobile phone. In some cases, the SCCMC, which can be triggered to play by sending the SCCMC from a user to a friend or a group of friends, can be referred to as a "Vyng Bomb". It can be noted that, as combined with Figure 2B and 2C As described, the SCCMC may be delivered over the transport layer of the voice and data networks, such that user 2 may receive the SCCMC over the data network and play it when a VOIP call is initiated, and user 3 may receive the SCCMC over the voice network, such that when a call is received, the SCCMC tone is played on the phone screen of user 3. In an embodiment, the trigger for playing the SCCMC may include a push notification over the data network without involving a phone call. In such an embodiment, the SCCMC may play and immediately disappear without the presence of a phone call or a VoIP call.

[0166] Fig. 20 The SCCMC is illustrated as being able to select from a collection of video content. In a first step 2002, a first user may register to receive content from one or more video channels 2012. The video channels 2002 may be selected from a social graph video source 2014 or other video sources 2018, such as libraries organized by editors or libraries organized automatically by one or more algorithms that identify and aggregate content based on, for example, defined topics of interest. The social graph video source 2014 may be created in the SCCMC App, or may be found in other Apps from the user's social graph, such as from Facebook Video. TM , Twitter TM LinkedIn TM SnapChat TM , Periscope TMThe video of the first user's favorite team, such as "cats". The top music video source may include clips of popular music videos. Once the user registers, in the second step 2004, the first user can receive a call. In the third step 2008, a video channel can be selected after the first user receives the call. In the fourth step 2010, the video from the video channel selected in step 2008 can be played on the screen of the first user's mobile phone. In an embodiment, in addition to or as a substitute for the user's social graph, there may be one or more private or closed network channels 2019 that the user can create, and the user can add their own videos therein. These private channels 2019 can be provided by invitation (and optionally closed to uninvited users). For example, there may be a #UCLA channel in which only students attending the university receive permission to view and engage with video content within the channel, such as in an embodiment via some combination of invitation, password, and location (such as by using a geo-fence related to the location of the university).

[0167] In various embodiments described throughout the present disclosure, SCCMC may also be a live face video. When a first user sends an SCCMC or otherwise initiates a call to a second user, the live face video may allow the second user to receive a live face at the first user. When receiving an SCCMC from the first user or a notification that the first user is calling the second user, the live face may allow the second user to immediately see the face of the first user. The live face video may be played on the screen of the second user's device, such as a lock screen, a home screen, or other screens. The live face video may be generated by initiating a live video stream. A live video stream generator may be used to initiate a live video stream. The live stream generator may be an application, such as Skype, Facetime, Periscope, and the like. The live video stream may also be recorded and sent as a series of small video clips. The live face video may be viewed in a window opened on the user's mobile phone screen through the SCCMC app.

[0168] Sending the SCCMC may include automatically recording the user's response to the SCCMC, such as in Fig.21. In a first step 2102, the first user may send the SCCMC to the second user. In a second step 2104, the second user may receive the SCCMC from the first user. The second user's mobile phone may record the second user's reaction in a third step 2018, such as by causing the SCCMC to include an instruction to turn on the camera of the recipient's phone facing the direction of the recipient. At a fourth step 2110, the recipient's phone may turn on the camera and record the recipient's reaction. In an embodiment, the second user may give the second user's mobile phone permission to record the second user's reaction. Before receiving the SCCMC, when the SCCMC is received, before the SCCMC starts playing, and so on, the second user may give the second user's mobile phone permission to record the second user's reaction. In step 2112, the recipient user may be provided with options for handling the reaction, such as being given the option to save, upload, or share the second user's reaction.

[0169] Fig. 22 An alternative embodiment of a process for obtaining reactions to a SCCMC is shown. After the sender sends the SCCMC in step 2202, the recipient receives the SCCMC at step 2204, and at step 2208, the recipient's phone records the recipient's reaction, and the recipient can be given the option to save, upload, or share the reaction at step 2210, such as with the sender, with the recipient's social circle, or otherwise, including the option to store the reaction as a SCCMC, such as in a SCCMC content library.

[0170] Fig.23A process is illustrated in which the playback of the SCCMC may be triggered by a triggering event, such as the recipient's mobile device entering a location such as defined by a geo-fence. At step 2302, the location of the recipient device may be detected at a location such as defined by a geo-fence (or otherwise detected at a location according to various location systems known to those skilled in the art of mobile telecommunications). As an example, the geo-fence may surround a point of interest, such as a location within a museum, park, or exhibition hall, a store or other location related to the promotion or sale of a product or service, an educational location within a school, college, university, or any other location for which there may be relevant content that may be presented via the SCCMC. Upon detection at step 2302, the SCCMC may be retrieved at step 2304, such as from a library of the SCCMC. This may include retrieving content associated with the location (such as content about a nearby museum exhibit or nearby products), and may include retrieving content relevant to the recipient (such as content relevant to the preferences of the recipient, as indicated by selection of a channel, by reaction to particular content, by the preferences of the recipient as indicated by other data sources (such as purchases, interactions with content, or the recipient's social graph), etc.). The SCCMC may be retrieved based in part on whether the recipient has visited the location before (this information may be stored by the host system, or may be stored locally at the geo-fenced computing infrastructure), so that the SCCMC may be varied for repeat visitors. The content server may retrieve the appropriate SCCMC and, at step 2308, send it to the recipient's mobile device. The SCCMC may then be played on the device 2310 as described in conjunction with the various embodiments described throughout the present disclosure.

[0171] Fig.24 A computer system is illustrated, for example, a computer system that can be used in conjunction with the devices, systems, and methods described herein. In general, computer system 2400 may include: a computing device 2410 connected to network 2402, for example, via external device 2404. Computing device 2410 may be or may include any type of client or other network endpoint as described herein, for example, a mobile device as described throughout this disclosure. Computing device 2410 may also or alternatively be any device suitable for interacting with other devices via network 2402, such as a laptop computer, desktop computer, personal digital assistant, tablet device, mobile phone, television, set-top box, wearable computer, etc. Computing device 2410 may also or alternatively include a server, such as any server described herein.

[0172] In some aspects, the computing device 2410 may be implemented using hardware (e.g., in a desktop computer), software (e.g., in a virtual machine, etc.), or a combination of software and hardware. The computing device 2410 may be a standalone device, a device integrated into another entity or device, a platform distributed across multiple entities, or a virtualized device executing in a virtualized environment.

[0173] The network 2402 may include any of the networks described above, for example, data network(s) or internetwork(s) suitable for transmitting data and control information among participants in the computer system 2400. This may include public networks (such as the Internet), private networks, and telecommunication networks (such as the public switched telephone network or cellular networks that use third generation cellular technology (e.g., 3G TM or IMT-2000 TM ), fourth generation cellular technology (e.g., 4G TM , LTE TM 、MT-Advanced TM 、E-UTRA TM etc.) or WiMAX-Advanced TM (IEEE 802.16m) and / or other technologies), as well as any of a variety of corporate regional, metropolitan, campus, or other local area networks or enterprise networks, along with any switches, routers, hubs, gateways, etc. that can be used to carry data among the participants in computer system 2400. Network 2402 can also include a combination of data networks and is not necessarily limited to strictly public or private networks.

[0174] External device 2404 may be any computer or other remote resource connected to computing device 2410 via network 2402. This may include a gateway or other network device containing content requested by computing device 2410, a remote server, etc., a network storage device or resource, or any other resource or device that may be connected to computing device 2410 via network 2402.

[0175] Generally speaking, computing device 2410 may include a processor 2412, memory 2414, network interface 2416, data storage 2418, and one or more input / output interfaces 2420. Computing device 2410 may further include peripherals 2422 and other external input / output devices that may be connected to or in communication with input / output interface 2420.

[0176] Processor 2412 may be any processor or other processing circuitry capable of processing instructions for execution within computing device 2410 or computer system 2400. Processor 2412 may include a single-threaded processor, a multi-threaded processor, a multi-core processor, etc. Processor 2412 may be capable of processing instructions stored in memory 2414 or data storage 2418.

[0177] Memory 2414 can store information within computing device 2410. Memory 2414 can include any volatile or non-volatile memory or other computer-readable medium, including but not limited to random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable PROM (EPROM), registers, etc. Memory 2414 can store program instructions, program data, executable files, and other software and data that are useful for controlling the operation of computing device 2410 and configuring computing device 2410 to perform functions for a user. Memory 2414 can include multiple different stages and types of memory for different aspects of the operation of computing device 2410. For example, a processor may include on-board memory and / or cache for faster access to certain data or instructions, and may include separate main memory, etc. to expand memory capacity as needed. As contemplated herein, all such memory types may be part of memory 2414.

[0178] In general, the memory 2414 may include a non-volatile computer-readable medium containing computer code that, when executed by the computing device 2410, creates an execution environment for the computer program in question, for example, code constituting processor firmware, a protocol stack, a database management system, an operating system, or a combination thereof, and / or code that implements some or all of the steps set forth in the various flow charts and other algorithm descriptions set forth herein. Although a single memory 2414 is depicted, it will be understood that any number of memories may be effectively incorporated into the computing device 2410. For example, even when the computing device 2410 is powered off, a first memory may provide a non-volatile storage device such as a disk drive for permanent or long-term storage of files and code. A second memory such as a random access memory may provide a volatile (but higher speed) memory for storing instructions and data for execution. A third memory may be used to improve performance by providing a higher speed memory physically adjacent to the processor 2412 for registers, caching, and the like.

[0179] The network interface 2416 may include any hardware and / or software for connecting the computing device 2410 in a communication relationship with other resources via the network 2402. This may include remote resources accessible via the Internet, as well as local resources available using short-range communication protocols, such as using physical connections (e.g., Ethernet), radio frequency communications (e.g., WiFi), optical communications (e.g., fiber optics, infrared, etc.), ultrasonic communications, or any combination of these or other media that may be used to carry data between the computing device 2410 and other devices. The network interface 2416 may include, for example, a router, a modem, a network card, an infrared transceiver, a radio frequency (RF) transceiver, a near field communication interface, a radio frequency identification (RFID) tag reader, or any other data reading and writing resource, etc.

[0180] More generally, the network interface 2416 may include any combination of hardware and software suitable for coupling components of the computing device 2410 to other computing or communication resources. By way of example and not limitation, this may include: electronic devices for wired or wireless Ethernet connections operating in accordance with the IEEE 802.11 standard (or any variation thereof), or any other short-range or long-range wireless networking components, etc. This may include: hardware for short-range data communications such as Bluetooth or infrared transceivers, which may be used to couple to other local devices, or to connect to a local area network, etc., which in turn couples to a data network 2402, such as the Internet. This may also or instead include hardware / software for WiMAX connections or cellular network connections (using, for example, CDMA, GSM, LTE, or any other suitable protocol or combination of protocols). The network interface 2416 may be included as part of the input / output device 2420, or vice versa.

[0181] The data storage device 2418 may be an internal memory storage device that provides a computer-readable medium, such as a disk drive, an optical drive, a magnetic drive, a flash drive, or other device capable of providing mass storage for the computing device 2410. The data storage device 2418 may store computer-readable instructions, data structures, program modules, and other data for the computing device 2410 or computer system 2400 in a non-volatile form for subsequent retrieval and use. For example, the data storage device 2418 may store, but is not limited to, one or more of an operating system, application programs, program data, databases, files, and other program modules or other software objects, and the like.

[0182] The input / output interface 2420 can support input from output to other devices that can be coupled to the computing device 2410. For example, this can include a serial port (e.g., an RS-232 port), a universal serial bus (USB) port, an optical port, an Ethernet port, a telephone port, an audio jack, an audio / video component input, an HDMI port, and the like, any of which can be used to form a wired connection with other local devices. This can also or instead include an infrared interface, an RF interface, a magnetic card reader, or other input / output system for coupling with other local devices in a communication relationship. It will be understood that although the network interface 2416 for network communication is described separately from the input / output interface 2420 for local device communication, the two interfaces can be the same, or can share functionality, such as in the case where a USB port is used to attach to a WiFi accessory, or in the case where an Ethernet connection is used to couple to a local network-attached storage device.

[0183] Peripheral devices 2422 may include any device used to provide information to or receive information from computing device 2400. This may include: human input / output (I / O) devices such as keyboards, mice, mouse pads, trackballs, joysticks, microphones, foot pedals, cameras, touch screens, scanners, or other devices that user 2430 may use to provide input to computing device 2410. This may also or instead include displays, speakers, printers, projectors, headsets, or any other audiovisual device for presenting information to a user. Peripheral devices 2422 may also or instead include digital signal processing devices, actuators, or other devices to support control or communication with other devices or components. Other I / O devices suitable for use as peripheral devices 2422 include tactile devices, three-dimensional rendering systems, augmented reality displays, magnetic card readers, and the like. On the one hand, peripheral device 2422 can be used as network interface 2416, such as with a USB device configured to provide communication via a short-range (e.g., Bluetooth, WiFi, infrared, RF, etc.) or long-range (e.g., cellular data or WiMAX) communication protocol. On the other hand, peripheral device 2422 can provide a device for enhancing the operation of computing device 2410, such as a global positioning system (GPS) device, a security dongle, etc. On the other hand, the peripheral device can be a storage device such as a flash memory card, a USB drive or other solid-state device, or an optical drive, a magnetic drive, a disk drive, or other devices or combinations of devices suitable for mass storage. More generally, any device or combination of devices suitable for use with computing device 2400 can be used as peripheral device 2422 contemplated herein.

[0184] Other hardware 2426 may be incorporated into the computing device 2400, such as a coprocessor, a digital signal processing system, a math coprocessor, a graphics engine, a video driver, etc. Other hardware 2426 may also or instead include expanded input / output ports, additional memory, additional drives (e.g., a DVD drive or other accessories), etc.

[0185] The bus 2432 or a combination of buses may serve as an electromechanical platform for interconnecting components of the computing device 2400, such as the processor 2412, memory 2414, network interface 2416, other hardware 2426, data storage 2418, and input / output interfaces. As shown, each of the components of the computing device 2410 may be interconnected using a system bus 2432 or other communication mechanism for transferring information.

[0186] The methods and systems described herein may be implemented using the processor 2412 of the computer system 2400 to execute one or more instruction sequences contained in the memory 2414 to perform predetermined tasks. In an embodiment, the computing device 2400 may be deployed as multiple parallel processors that are synchronized to execute code together to improve performance, or the computing device 2400 may be implemented in a virtualized environment in which software on a hypervisor or other virtualization management facility emulates components of the computing device 2400 as appropriate to reproduce some or all of the functionality of the hardware instantiation of the computing device 2400.

[0187] In one aspect, computing device 2400 may be a mobile device such as described below.

[0188] Fig.25 A mobile device is illustrated. Mobile device 2500 can be configured to create, send, or receive SCCMC as described herein. Mobile device 2500 can be, for example, any of the endpoints described above or other computing devices. Mobile device 2500 can be, for example, a smart phone or a tablet device. In general, mobile device 2500 can include a display 2502 (e.g., a screen), a communication interface 2504 configured to couple mobile device 2500 in a communication relationship with a network, a processor 2506, a memory 2508, a camera 2510, a microphone 2512, a speaker 2514, and a location tracking device 2516.

[0189] The communication interface 2504 may include hardware and software for communicating using a variety of protocols, such as short-range communication protocols (e.g., Bluetooth), wireless communication protocols (e.g., Wi-Fi, cellular networks, etc.), etc. The communication interface 2504 may couple the mobile device 2500 to a cellular network, etc., through a mobile operator with which the user has a billing account. The communication interface 2504 may also or instead couple the mobile device 2500 to a data network, through which the mobile device 2500 may access an online application store, etc. The communication interface 2504 may also or instead couple the mobile device 2500 to other mobile devices through a data network, such as any of the networks described herein.

[0190] In one aspect, the memory 2508 carries computer code that, when executed on the processor 2506, performs steps for creating, sending, receiving, editing, etc. SCCMC data structures, etc. The memory 2508 may also or instead carry computer code that forms an application for the mobile device 2500, e.g., executed by the processor 2506 on the mobile device 2500.

[0191] Display 2502 may include a screen such as a touch screen or other user interface for receiving user input as contemplated herein, for example, for authoring or editing a SCCMC.Display 2502 may also or instead display the SCCMC to a user.

[0192] Camera 2510 may be used to capture images or video for use with the SCCMC.

[0193] Microphone 2512 may be used to receive voice or audio signals as described herein.

[0194] Speaker 2514 can be used to play audio from the SCCMC for the user. Speaker 2514 can instead include an audio output on the mobile device 2500 such as a headphone jack or the like.

[0195] The location tracking device 2516 can be used to establish a location for the mobile device 2500, for example, for use in proximity services contemplated herein. The location tracking device 2516 can include a GPS receiver or the like.

[0196] In some embodiments, mobile device 2500 can be used to facilitate a user (such as a recipient) to establish a Fig.26Personal channels as shown. Personal channels can enable a user to select media content such as video and image content from the user's mobile device 2500 and designate it for display on the user's mobile device 2500 when an incoming trigger occurs. An incoming trigger can be an incoming call from another mobile device, a text message, etc. A user can import multiple files and organize them for playback as a personal channel in a predefined order or in a random order.

[0197] Fig.26 A screenshot of a user interface 2600 for enabling a user to establish such a personal channel is illustrated. The user interface 2600 may be configured to allow a user to slide or otherwise select features for establishing a personal channel by accessing the SCCMC App installed on the mobile device 2500. For example, the SCCMC App may enable a menu feature labeled "My Channel" 2602 to be selected from the user interface 2600. Upon selecting the menu feature "My Channel" 2602, the user may be able to pick media content that the user may want to use as part of a personal channel. In some embodiments, the media content may include: SCCMC that may be received from any other user, or content produced or captured by the user (such as using the camera of the user's mobile phone), or videos obtained from a social networking server (such as a social network server hosted with YouTube). TM , Snapchat TM , Instagram TM or similar services on user-generated video platforms), or from any other source without limitation. Media content selected by the user can be accessed through the use of Fig.26 The user interface 2600 depicted in FIG. 26 shows an example of an action button labeled “Personal” 2604 that is added to the user's personal channel.

[0198] In some embodiments, the personal channels created by the user can not only be played on the user's device, but also can be configured to be shared with other users. In many examples, the user can organize videos or other media content into channels, and then share the channel with other users, including controlling the playback of the channel as a series of SCCMC media items on the phones of the various recipients of the shared channel. These channels can also be customized or configured for specific users or user groups by allowing users to create channels suitable for specific friends or groups of friends, and containing user-generated video content showing common friends or common topics of interest or showing video content that the user thinks friends will be particularly interested in.

[0199] In an embodiment, a channel shared with friends or other users may include a wide range of media content, including SCCMC received from other users, content produced or captured by the user (such as using the user's mobile phone's camera), content stored locally by other applications (such as social media applications), content downloaded from various social sources (such as YouTube), and content from other social sources (such as Facebook, Facebook, and other social media sources). TM or other user-generated video platforms), or from any other source. The ability to configure, customize, and share one or more channels may allow the SCCMC App to be used as a self-generated media creation and content distribution platform where users can form and orchestrate a wide range of experiences for friends and other users that play on the lock screens or other contact-initiated elements of friends' and other users' devices.

[0200] In many examples, media content of a user's personal channel may be enabled based on the identity of another user. For example, a user may associate a video or any other media item with the name of a "friend" or a specific contact in the user's mobile device.

[0201] Fig. 27 and 28 An example of a screen shot of a user interface 2700 for enabling media content based on a user's identity is illustrated. The user interface 2700 may be enabled by the SCCMC App installed on the user's mobile device. The user may open a contact list on the user's mobile phone, and a menu 2702 displayed on the user interface 2700 may provide options such as "Ask a friend," "Upload," "Customize for a contact," and other options that may enable media content from a user's personal channel to be set for a particular contact based on the contact's identity. For example, a user may set a custom video or channel for a single contact or friend by making the appropriate selection from the user interface 2700.

[0202] In various embodiments, various media content may be set in the name of a friend, including: SCCMC received from other users, which is produced or captured by the user, such as by using the camera of the user's mobile phone; TM or other user-generated video platforms, or from any other source. In some embodiments, such as in a user with an app such as Snapchat installed on the user's device TM , Instagram TMIn the case of other social networking applications such as Facebook, Facebook, and others, the SCCMC App can enable a user to integrate a mobile device with a user-generated social media application or platform used by the user's mobile device. These social networking applications can enable users to save content to a local device so that when content is created, it is saved in a local folder associated with each application. The SCCMC App can include a local storage monitoring component that can monitor local folders and make the contents of the local folders available for selection as SCCMC media items that can be played in a user's channel, set as content associated with a friend, and the like. This can be achieved by importing the content into one or more locations accessed by the SCCMC App, or by marking local folders so that content can be imported or played from the local folders when needed.

[0203] Once this setup is initiated, the selected or tagged content can be configured to play on the user's phone's lock screen when a friend calls or upon other trigger conditions (such as when a text or email is received from a friend), independent of channels the user may have selected or other triggers arriving from other parties.

[0204] In an embodiment, the user may select an option to "Ask a friend" 2704 (or multiple friends), where the SCCMC App may enable a feature that allows the user to select a friend to set a media item, such as a video to play when a friend calls, so that the friend has control over the media content item. In many examples, this feature may be enabled, such as when custom content is set by a friend, as discussed in some previous examples, which allows the user to set a video to play when a friend contacts the user. After initiating this "Ask a friend" 2704 feature, a message (such as a text message) may be initiated to the friend through the App, automatically embedding a custom link to a website where the friend can upload a media item, such as a video, etc.

[0205] In an embodiment, the user may also be allowed to select more than one contact from the user interface 2700 for selecting the "Ask Friends" 2704 feature. The user interface 2700 may present a list of contacts or "friends" to the user, and the user may select a top friend by selecting an action button (such as button 2704) for implementing the "Ask Friends" 2704 option. In many examples, the top friend may include the contact that the user contacts most often, such as by calling or messaging. In other examples, other parameters may be used to classify the contact list as the user's top friend. When the user clicks button 2704, the SCCMC App may enable initiation of sending a message, such as a text message, to each top friend through the App. Initiating a message may include automatically embedding a custom link into a website where each top friend can upload a media item, such as a video.

[0206] Fig.29 A screenshot of an exemplary user interface 2900 is illustrated that may enable a user (such as a sender's friend contact) to create a video that may be played when the friend contact calls the user or initiates any contact with the user. The friend-created video may be uploaded to a website by the friend, and a link thereof may be sent to the friend, such as when the user selects a contact for a friend from the "Ask a Friend" 2704 feature of the SCCMC App. A video may be one type of media item that may be used to set a specific media type for a specific contact.

[0207] In an embodiment, media items may include: SCCMC received from other users, content produced or captured by a friend or user (such as using the friend's or user's mobile phone's camera), TM ) or other user-generated video platforms, or from any other channel. In an embodiment, the upload webpage through which friends can set items to be played on the user's device can be configured to work with mobile and desktop browsers. When a friend uploads a media item, a push notification can be automatically directed to the user's device indicating that the content has been uploaded. When the friend calls the user or optionally in other triggering contacts (such as text, chat or email contacts), the media content items uploaded by the friend can be played on the user's phone, such as on the phone's lock screen.

[0208] In an embodiment, a user may use the "Ask Friends" 2704 feature to ask various friends to upload or select media content, so that a variety of different media content may be played based on different contacts, thereby augmenting the items that the user has already set up or selected for friends, or that the user has typically selected for unknown inbound contacts.

[0209] Various embodiments as discussed herein can facilitate the creation, editing, deployment, and management of media content that is available in a mobile phone or received from other devices or network sources and is to be used as an SCCMC to build a personalized experience or to be shared among groups of friends and contacts to personalize the experience during any triggering event (such as text messaging, ringtones, etc.) using various menu options that are displayed through a user interface (such as in conjunction with a mobile phone). Figure 26-29 those shown in the figure) enable this to be achieved.

[0210] In many examples, the SCCMC App may enable the setting of media content including the SCCMC. The media content may include: video content, which may include contextual video messages. Fig.30 A flow chart of a method 3000 for providing settings for contextual video messages included in a SCCMC for interactions between users is illustrated.

[0211] like Fig.30 As illustrated in FIG. 30, at 3002, user 1 (such as a user of sender mobile phone 102) may access a messaging facility. Sender mobile phone 102 ( Figure 1 ) can participate in a conversation with another user, User 2 (such as recipient mobile phone 104 ( Figure 1 The messaging interaction may use any available messaging facility, such as SMS, chat interaction, or any of the many messaging applications available on the two users' respective mobile devices (such as WhatsApp). TM 、iMessage TM , Snapchat TM , Instagram TM , Messenger, etc.). In an embodiment, the app can host the entire communication exchange without relying on third-party applications. At 3004, user 1 can enter some text on the input interface of the messaging facility of user 1's mobile device and send the text or other message to user 2. The input interface of the messaging facility can be configured to receive at least text input. In one example, the input from user 1 is limited to what can be typed or specified in the text. The input interface of the messaging facility can also be configured to receive text, graphics, audio, video, and combinations thereof, including emoticons and other graphics that can be inserted into the input interface. The SCCMC App installed on the mobile device of user 1 may have access to input entered on the input interface of the messaging facility. Similarly, the SCCMC App installed on the mobile device of user 2 may be configured to access input received on the input interface of the messaging facility on the mobile device of user 2.

[0212] At 3008, input sent by user 1 may be received at a messaging facility of user 2's mobile device. At 3010, the SCCMC App installed on user 2's mobile device may be configured to obtain input received on an input interface of the messaging facility of user 2's mobile device, and process the content of the input to determine the meaning of the text and understand the context of the input. In an embodiment, the SCCMC App installed on user 2's mobile device may also be configured to obtain input received on an input interface of the messaging facility of user 2's mobile device, and process the content of the input to determine the meaning of audio, video, graphics, emoticons, etc. to understand the context of the input. The processing of the input may be enabled by using an advanced processing facility including a natural language processing (NLP) facility, which may include one or more algorithms or methods for determining the meaning of a body of text, such as text received at an input interface of the messaging facility of user 2's mobile device. In view of the present disclosure, it will be appreciated that in some examples, one or more algorithms of the natural language processing (NLP) facility may be shown to be executed faster when the input is limited to text and other limited characters.

[0213] In an embodiment, the natural language processing (NLP) facility may also include: additional algorithms or methods for determining the meaning of a conversation in an audio or video clip. In many examples, the NLP facility may include: an NLP processing module that may be configured to execute a series of instructions, such as identifying the meaning and / or context of an input according to one or more algorithms and methods. By way of these examples, the NLP facility may be configured to extract words, phrases, emoticons, geographic location information, and sensor derived data from text received at a messaging facility of a mobile device of user 2. In an embodiment, the information extracted from the text may then be matched with a database of content, such as a database stored in a memory unit 2414 of the computer system 2400 discussed previously. In many examples, the NLP processing module may be part of the processor 2412 of the computer system 2400 discussed previously. In another example, the NLP processing module may be part of other hardware 2426 of the computer system 2400.

[0214] At 3012, the meaning identified by the NLP processing module can be used by the SCCMCApp installed on the mobile device of user 2 to assign media content to the input received at the input of the messaging facility of the mobile device of user 2. The media content data types discussed previously (such as video clips or video channels) can be appropriately linked to the input message. In many examples, the media content can be selected from an organizational library such as the SCCMC content types discussed previously. At 3014, in many embodiments, the assigned media content can be displayed on the lock screen of the mobile device of user 2. At 3018, user 2 can then click on his or her lock screen to send a reply to user 1. In many examples, user 2 can send a reply directly from the lock screen of the mobile device of user 2 without opening the input interface of the messaging facility on his or her mobile device.

[0215] In many examples, when a reply is sent to User 1, User 2's camera may be activated by default, and a picture or photo of User 2 may be associated with the reply sent from User 2 to User 1. In other examples, when a reply is sent to User 1, User 2's microphone may be activated by default, and an audio clip of User 2 may be associated with the reply sent from User 2 to User 1.

[0216] In many examples, when user 2 types text for a reply message, certain content may be automatically pulled from a database of content to match the text keywords entered by user 2. In an embodiment, user 2 may use the content automatically pulled from the content database, or may instead access a messaging facility to complete the text and then send the message. During the messaging interaction, when a message sent from user 2 is received at user 1's messaging facility, the message sent from user 2 may also be processed in the same manner.

[0217] Figures 31A-31E The diagram shows a method based on a plurality of embodiments. Fig.30 An example of a messaging interaction between User 1 and User 2 of the contextual video messaging stream discussed in . Fig.31A 31 illustrates a screen shot of a display interface 3100A of a mobile device of user 2 when user 2 receives a message from user 1 having a text phrase "Hungry? Let's meet for lunch!" The text phrase may be displayed on the mobile device of user 2 along with associated and matching video or other content. Matching of the video or other content to the text phrase may be performed based on the content of the text phrase, such as in Fig.30 For example, Fig.31AThe video in shows a hungry Homer Simpson cartoon character with a bloated and quivering belly to indicate the emotion of hunger. In the example, the video can originate from an organized library of SCCMC content types of the SCCMC App. Similarly, Fig.31B The illustration shows a screenshot of the display screen 3100B of user 2's mobile device when a text message phrase "I'm so hungry!!!" is received from user 1 at user 2's mobile device. In an embodiment, the text message phrase may be displayed with a matching video / photo of the sender, such as a photo of user 1 taken from the camera of their mobile device while sending the message and expressing hunger. In another example, the display screen screenshot of user 2's mobile device may receive an audio message from user 1 saying "I'm hungry!", which includes a matching video / photo. In an embodiment, a matching video / photo of the sender, such as a photo of user 1 taken from the camera of user 1's mobile device, may be sent when the message is sent.

[0218] Fig.31C Illustrated is a screenshot of the display screen 3100C of user 1 when a reply message is sent from user 2 to user 1, such as on the lock screen of user 1, and wherein the reply message contains the text phrase "at 8 pm at Benny's!". The text phrase is associated with matching content, such as interactive information about the restaurant name mentioned in the text (i.e., "Benny's"). In an embodiment, such information may be available even if the lock screen is in place on the mobile device. In an embodiment, the interactive information may include, for example, a map of the restaurant location, a rating of the restaurant, a brochure of the restaurant, directions, tips, reviews, and other contextual advertising information associated with the restaurant name. In an example, the reply message may also or instead include information about the type of food, and the matching content may be displayed accordingly. In many examples, a message containing the phrase "Want to eat tortillas?" may be expanded by a video or animation showing tortillas. The meaning assigned to the phrase mentioned in the message may be based on the context and the content of other messages. By the above example, the initial reference to eating tortillas can be used to help infer that the subsequent message of the proposed name and time is a reference to the restaurant location where tortillas are available. In addition, the context of the information can be identified, such as by using the user's interaction history with other users, the user's or other users' history of using the SCCMC App, the user's geographic location, user contact data (such as stored contacts), the profile of User 1 or User 2 (or both), and many other information and combinations thereof. Thus, the matching content for the text in the message can be SCCMC media content, which can be automatically associated with the media content item and can be related to both the content of the message and the context of the message.

[0219] In many examples, the association of media content with text entered in an input message can be implemented by applying one or more machine learning algorithms. In many examples, machine learning algorithms and other methods can be used to interpret the text body in the input message and identify what type of media can be associated with what type of text input. In an example, the interpretation can be based on feedback information input into the machine learning algorithm, such as feedback information collected directly from users about the satisfaction with the content recommended by the SCCMC App. In addition to the feedback information collected directly from users, the feedback information can also or instead be derived based on some feedback metrics associated with the SCCMC App. In many examples, the feedback information can be derived based on metrics associated with the use of the recommended content, such as whether the user saves the content, shares the content, interacts with the content, and so on. In response to the text received in the input message, the feedback information can be used to provide more relevant and relatively effective media content.

[0220] For example, Fig.31D The illustration shows that text entered by the user on their lock screen 3100D can be automatically completed and matched with related media content. Fig.31E The completed content is illustrated in the screenshot of display 3100E of . In an embodiment, the matched and completed content on display 3100E can be used directly by the user from the lock screen, or alternatively, the user can overlay the content. In both scenarios, user feedback can be used to derive feedback information for the machine learning algorithm.

[0221] In an embodiment, the methods and systems described herein include SCCMC-based structures (such as SCCMC apps and SCCMC media content) that can facilitate user access to or can be used as any of a variety of mobile communication capabilities or applications on a user's mobile device, such as text messaging, video calls, audio calls, etc., including the use of various network and transmission protocols, such as IP-based protocols, cellular network protocols, or other protocols. In an embodiment, the communication capabilities or applications can be accessed directly from the user's lock screen without the user unlocking the phone or opening the communication application. As such, the user does not need to unlock the mobile device to access the SCCMC app and its media content. In an embodiment, the methods and systems can be docked with one or more communication port features of the mobile device (such as features for handling incoming IP-based messages, features for handling notifications, features for handling incoming cellular messages, etc.) to effectively redirect information transmitted to the mobile device to an application with a lock screen function that is used for other communication capabilities or applications. In an embodiment, an application with a lock screen function can run independently of the locked or unlocked state of a phone or other mobile device, such as when the device is locked, the screen is blank, it displays its lock screen, and the like. Apps with lock screen functionality can block messages sent to or from messaging apps (e.g., Snapchat). TM , Facebook Messenger TM 、iMessage TM , WhatsApp TMThe receiver's mobile device may include an SCCMC data structure that can implement contact media adaptation functions before being sent from the sender's mobile device, including providing a user interface that runs when the phone is in lock screen mode. The user interface may include an application with a lock screen function so that the user of the mobile device can receive and respond to communications associated with the intercepted information before being passed to the communication application, communications associated with the intercepted information directly processed by the user interface, and the like. In an embodiment, the application with a lock screen function can display a wide range of content in response to the intercepted information, such as video content, animation, or other types of content throughout the wide range described in this disclosure. In an embodiment, the content may consist of a video. In an embodiment, the video may be selected by the sender of the information. In an embodiment, a video may be selected from a channel, where the video and / or channel may be selected based on the content of the intercepted information and / or the context of the information (such as based on the time of day, the sender's location, the recipient's location, the state of the receiver's device, or other factors). In an embodiment, an application with a lock screen function may also interface with a target / source communication capability or application to facilitate end-to-end communication between the relevant communication capability or application on the sender's mobile device and the recipient's mobile device without the need for a third-party communication application.

[0222] Fig.32A diagram of a unified messaging platform 3200 as described herein is depicted. In an embodiment, a sender 3202 (such as a mobile phone, etc.) can communicate with a recipient 3204 (such as a recipient mobile phone, etc.) over an electronic medium (such as an IP-based network, a cellular network, etc.) using the methods described herein. In an embodiment, communication among one or more senders 3202 and one or more recipients 3204 can include a platform server 3206. Communication among one or more senders 3202 and one or more recipients 3204 can include direct use of messaging services, such as native messaging services 3210, 3212, messaging applications 3214, 3218, sender-controlled contact media content applications 3220, 3222, etc. Communication among one or more senders 3202 and one or more recipients 3204 can also include use of content association services 3224, 3228. The sender may use a sender mobile phone 3202 or similar computing device configured with one or more capabilities and / or applications, such as a native messaging service 3210, which may facilitate communication with other users, such as recipients, servers, and the like. In an embodiment, the sender mobile phone 3202 may be further configured with a messaging application 3214, such as those mentioned and described herein, for facilitating communication of text, images, videos, and other electronic content between the sender and the recipient. In an embodiment, the sender mobile phone 3202 may also be configured with one or more unified messaging applications or capabilities, such as a sender-controlled contact media content application 3220, which may work in conjunction with (or in part or in whole replace) the native messaging service and / or messaging application of the mobile device to facilitate the sender-controlled contact media content control functions and capabilities described herein.

[0223] In an embodiment, these SCCMC features include: performing natural language processing (NLP) 3230, 3232 on messages prepared in any messaging application including the SCCMC applications 3220, 3222, and on content associated with outgoing messages as described herein. In an embodiment, other features of the sender 3202 may include: one or sender emotion sensing 3240, sender user state determination 3242, etc., which can provide information to the SCCMC application 3220, and provide information from other SCCMC applications. In an embodiment, the features, capabilities, etc. of the sender 3202 can be embodied as a separate application, a local service available to the SCCMC application 3220, a remote service accessed through the emotion sensing and / or user state determination interface capabilities of the sender mobile device, etc. In an embodiment, the emotion sensing 3240 can also include emotion determination capabilities by interfacing with the emotion determination service 3250 of the unified messaging platform 3200, or can directly or partially perform the emotion determination function using the processing and storage capabilities of the sender 3202 mobile phone, etc.

[0224] In an embodiment, Fig.32 The platform server 3206 of the unified messaging platform 3200 depicted in the figure can be connected to at least the sender 3202 or the recipient 3204 through a network connection, etc. The unified messaging platform 3200 server 3206 can further provide services, applications and / or capabilities for unified messaging as described herein, including emotion determination 3250 for determining the emotions of users of the unified messaging platform 3200 including the sender 3202 or the recipient 3204. In an embodiment, the platform server 3206 can also provide natural language processing 3252 services for processing messages composed on the sender device 3202, which can be sent to the recipient device 3204 or received on the recipient device 3204. In many examples, NLP services 3230, 3232 can be provided to any participant in the platform via the natural language processing capabilities 3252 executed on the platform server 3206. The platform server 3206 or other aspects of the platform 3200 can also provide capabilities such as machine learning via machine learning capabilities 3260. The platform server may further provide content association services 3262 via content association capabilities that are performed at least in part using computing and / or storage features of or accessible to the platform server 3206 .

[0225] In an embodiment, the unified messaging platform 3200 may further include: a media content library 3270, which may include or be associated with a sender-controlled contact media content structure 3272. Access to the content library 3270 may be provided to participants on the platform 3200, one or more senders 3202, via one or more networks and, where applicable, via a platform server 3206. Similarly, one or more recipients 3204 of the platform 3200 may access the media content library 3200 via a network, via a server 3206, and the like. Features, functions, and services of the media content library are further described herein, including the association and / or integration of one or more SCCMC structures.

[0226] In an embodiment, a unified messaging platform such as Fig.32 ) includes a recipient 3204 that may be embodied as a mobile device, etc. In many examples, the recipient 3204 of the platform 3200 may include unified messaging capabilities similar to the capabilities of the sender 3202, such as content association services via a content association module 3228, local messaging services via a local messaging module 3212, messaging services via a messaging application or service 3218, NLP services through a natural language processing module 3232, etc. In embodiments, the recipient 3204 may include or have access to other capabilities, such as a recipient SCCMC application 3280, which may include functionality similar to the sender SCCMC application 3220, but may also be adapted to work in conjunction with a local lock screen service 3282, an application with lock screen functionality 3284, etc., which may utilize the recipient 3204 computing device to embody and / or use processing and / or storage resources of the recipient 3204 computing device. The capabilities of the recipient SCCMC application 3280 are further described herein, including its cooperative operation with the application with lock screen functionality 3284 and / or its optional integration with the application with lock screen functionality 3282.

[0227] In an embodiment, the media content library 3270 may provide storage for media content and optionally for SCCMC items, or may be associated with a SCCMC structure as described herein. Fig.33 An embodiment of a media content library 3300 is depicted that facilitates association of content items with media types, SCCMC references (if any), one or more channels (eg, topics of interest), and the like. Fig.33 Embodiments may also include an index for accessing items in the media content library 3270, SCCMC structure 3272, and the like.

[0228] Fig.34 An embodiment of a media content library 3402 is depicted that can combine media content items 3404 and SCCMC content items and / or structures 3406. As described herein for many embodiments, the media items and / or SCCMC content items or structures 3406 can be arranged into channels, which can be based on topics of interest. In an embodiment, an exemplary channel indicator 3408 for a collection of SCCMC structures for sender 1 (SCCMC 1-x) can provide a mechanism by which the sender's SCCMC structures can be accessed as channels, as described herein. Fig.34 In an embodiment of the invention, the composite media content library 3402 may also provide access to media content items as a media channel 3410.

[0229] In an embodiment, Fig.35 Depicted is an alternative configuration of a media content library 3502 that can operate in conjunction with an SCCMC structure library 3508 to provide the services and capabilities of a unified messaging platform, etc., as described herein. Fig.35 In an embodiment of the present invention, the media content library 3502 and the SCCMC library 3508 can be accessed by various functional modules of the platform, such as the sender SCCMC application 3220, the content association capability 3220, and the receiver SCCMC application 3222. In an embodiment, each application or capability can determine the appropriate library to access based on the type of media selected and / or the function implemented. As used herein, channels can be configured for the SCCMC library 3508 and the media content library 3502 so that items in each library can be accessed via channel references without directly referencing each specific item.

[0230] Fig.36 The embodiment of the invention depicts a sender device component 3600 of a unified messaging platform and the information flow among the sender component 3600. Fig.32The sender elements are described except for the communication interface 3606 and the unified messaging user interface 3604. In an embodiment, the information flow among the elements may include one or more messaging applications 3214 or services 3210 that interface with the sender SCCMC application 3220. The messaging application may send information such as text or other electronic content to the SCCMC application 3220, and the sender may use, for example, the functionality of the messaging application to compose or select the information with or without using the user interface of the messaging application. The sender may access the functionality of the messaging application through the unified messaging user interface 3604 via the SCCMC application 3220. In an embodiment, the SCCMC application 3220 may access the user state determination facility 3218 to determine the state of the user of the mobile phone, and access the emotion sensing facility 3212 to determine the emotion of the user. The SCCMC application 3220, in conjunction with the natural language processing capability 3214, can use this user state and / or sentiment information to operate a content association function 3224 to associate content items from the media content library 3270 and / or the SCCMC structure library 3272 with the content of a message from a messaging service or application. In an embodiment, the SCCMC application 3220 can reconfigure the message with the content items and provide it to the communication interface 3606 of the sender device 3602, where it can be transmitted to the recipient computing device via an IP network, etc. In many examples, the unified messaging user interface 3604 can include features that facilitate the sender to select among multiple content items that can be selected through the content association operations described herein.

[0231] Fig.37 Describe the embodiment of the gallery 3700 of the content item, the gallery 3700 of the content item can be suitable for use with one or more messages being sent and received.In an embodiment, a user (e.g., a sender) can operate the unified messaging user interface 3604 to manually select one or more content items.In another embodiment, the content item can be automatically selected, and the message can be configured with the selected content item for passing to the recipient.No matter manually, automatically or a combination thereof, the content item 3700 is the candidate media content that can be accessed manually and / or automatically.

[0232] Fig.38Depicted are exemplary operations of a method and system 3800 for modifying content of an incoming message via interception by an application with a lock screen function, and processing it using the methods and systems described herein to append, adapt, and / or replace content items associated with the intercepted message. In an embodiment, a message that may be received by a recipient 3802 may be intercepted by an application with a lock screen function 3284, which may include or have access to functionality such as natural language processing 3232 and content association services 3228. In an embodiment, a message that may be received by a recipient 3802 and intercepted by an application with a lock screen function 3284 may also be interfaced with a recipient SCCMC application 3280 to determine content items that may be automatically or manually selected from a media library 3270, a SCCMC structure library 3272, and the like. In an embodiment, the determined content items may be configured with the intercepted message and processed (in part or in whole) by the recipient SCCMC application 3280, or forwarded to a corresponding messaging service or application. In an embodiment, the modified message can be presented in a unified messaging user interface 3604, which can be accessed by an application 3284 with a lock screen function without changing the lock state of the phone or connected device. When the phone or connected device remains in another locked state, the recipient 3802 can respond to the modified message using the unified messaging user interface 3604. Fig.39 An exemplary embodiment of a unified messaging user interface 3604 including a responsive modified message 3900 from a recipient 3802 to a sender 3602 and / or others is depicted in accordance with many embodiments of the present disclosure.

[0233] In an embodiment, methods and systems related to automatic gif / image / video recognition based on the content of contact events (such as text messages, etc.) can be beneficially applied to a wide range of languages. In particular, languages ​​including a large number of characters (such as certain oriental languages ​​that may have a thousand or more characters) can benefit from the methods and systems described herein by automatically annotating the typed content using at least one or more natural language processing methods described herein. In addition to facilitating the association of image-related content with such messages based on NLP of their textual content, audio-based messages can also benefit from NLP-based image selection by adding a speech-to-text function that prepares textual content items for NLP. Images obtained from NLP processing of generated text can be associated with audio. In an embodiment, images can be tagged with audio content so that transmitting an image to a recipient messaging application can result in displaying the image while the audio is being played back. In an embodiment, more than one image can be associated with an audio message, such as NLP based on multiple parts of an audio message. More than one image can be configured in a slide show coordinated with an audio file based on NLP output, etc., so that each of the more than one image can be played back in a sequence that tracks the associated audio. In an embodiment, the NLP may determine that a first portion of an audio transcribed message may embody a first theme or emotion, and that a second portion of the first portion may embody a second theme or emotion. A first image may be recommended and / or selected for the first portion, and the first image may be configured to be displayed (e.g., by associating playback metadata with the image, etc.) for a duration consistent with the first portion of the audio transcribed message. Similarly, a second image may be recommended and / or selected for the second portion, and configured to be displayed for a duration consistent with the second portion of the audio transcribed message. In an embodiment, audio-related image(s) may be recommended, selected, and exchanged between at least two users in the form of audio messaging (such as VoIP, audio file capture and playback, etc.) as described herein.

[0234] Fig.40 Depicts the addition of speech-to-text functionality to enable the use of natural language processing Fig.32Embodiments of the platform of, such as for recommending candidate images to be associated with an audio message. Among other things, the sender 4002 may be configured with the natural language processing 3214 described herein. A user of the sender system 4002 may record an audio message. The audio message may be processed at the sender 4002 using a speech-to-text facility 4008, which may generate a text transcription of the audio recording. The NLP facility 3214 may process the transcription to determine the subject, intent, emotion, and other understanding of the audio message. As described elsewhere herein, based on the subject, intent, emotion, or other understanding determined by the NLP, the sender SCCMC application 3208 may present one or more recommended image content to be associated with the audio message to the sender user. The SCCMC application may transmit one or more of the image content selected by the audio message and / or the sender user, and / or the transcription to one or more of the platform server 4006 and the recipient 4004. The processing of the transcription may be implemented by the natural language processing facility 3214' of the platform server 4006. Likewise, speech-to-text processing of audio messages may be performed by speech-to-text facility 4008′ of platform server 4006. Platform server 4006 may perform speech-to-text and NLP, and recommend one or more image content to the sender user, etc. Platform server 4006 may combine the image content selected by the sender user with the original audio file so that the image content is presented on the electronic interface of the recipient device 4004 when the audio message is played, such as on the lock screen of the recipient device 4004.

[0235] refer to Fig.41 , depicts the use of e.g. Fig.40 Flowchart of audio message processing for a platform of . The sender computer 4101 can generate an audio content item 4111, which can be provided to a platform 4102 that can receive audio content. A speech-to-text algorithm 4104 can be applied to the audio to generate a text transcription of the audio. The transcription can be processed by a natural language processing facility 4106. The output of the natural language processing facility 4106 can be processed by an image content review and recommendation facility 4108, which can use the NLP output to select from a plurality of available content images 4115 to recommend at least one content image to a user (such as a user of the sender 4101). The user can select and / or approve the recommended image in step 4110 image content selection / approval. At step 4112, the selected image can be linked with the received audio file, and the audio content can be further linked to the selected image for coordinated playback on the recipient computer 4103.

[0236] In order to support a user interface that facilitates the selection of one or more images to be associated with a message, a user preference metric for one or more images may be used when recommending an image. Measuring user preferences may be similar to those used in a social media environment, such as providing a "thumbs-up / thumbs-down" function, providing a "like" function, detecting the amount of time a user stays on an image, and the like may be used to help determine the likelihood that an image (such as a GIF, photo, video, etc.) is popular. These image popularity metrics may be combined with other metrics associated with the image, such as utilization of the image in a message, reuse of the image, social media activity (sharing, re-sharing, forwarding, retweeting, etc.) of a message with a given image to form an additional degree of image popularity likelihood. These measures may be combined with natural language processing of the message text including the image to better determine whether the resulting output of the NLP is consistent with the social media perception of the combined text and image. As an example, the NLP of the text may suggest a particular image. Social media-based perception of the combination of text and a particular image may support the suggestion (e.g., a social media user may indicate a preference for the combination, such as by liking a message or taking another action, such as forwarding the message), or may indicate that the suggested image may not be an appropriate image for the text (e.g., a social media user may indicate dislike for the combined image / text, such as by liking the message or showing a lack of action, such as not forwarding the message). Thus, the priority of an image may be represented as a function of user responses to the image in a user interface, social media responses to the image and / or to a message including the image, such as image priority = f(user interface response) + f(social media response). Additionally, in an embodiment, image priority may be used in a platform user interface to determine the order, arrangement, image quality, image size, etc. for presenting candidate images in the user interface. Images with lower priority scores may be presented as smaller images, may be presented further down in an image gallery, and so on.

[0237] Social media perception of an image and / or a message including the image may also be used as an input, such as a seed for natural language processing of the text content of the image. Social media perception that may indicate the popularity of a message may facilitate determining the temporal relevance of the message, which may indicate the importance of sentiment or other understanding of the message, which may be derived from natural language processing of the message and, therefore, from the temporal relevance of certain words in the message. The temporal relevance may be a short-term or long-term relevance, and may be linked to one or more words in the message, such that those words appearing in other messages may indicate a similar relevance, or at least may indicate a relevance of sentiment, etc., conveyed by the message. This information may be useful for validating and guiding improvements to natural language processing of the message.

[0238] The image source for the method and system for automatic image recommendation and / or selection to be included in the message may include a public image source, such as a publicly available social media photo gallery. In an embodiment, a public gallery such as social media of the participants of the message (such as the sender and the receiver) may be a resource for candidate images. By using tools such as facial recognition, images can be organized based on detectable faces in the photos. Further, by using NLP of the text portion of the message transmitted from the sender to the receiver, the context of the message can be determined. The context of the message can be encoded so that it can be used to facilitate the selection of one or more candidate images from the sender's social media accessible photos, the receiver's photos, or a combination thereof. In an example, NLP can be used to analyze the message text content to generate a context or understanding of the message. In an embodiment, such an understanding can suggest that the sender is inviting the receiver to dinner. The understanding can be encoded as keywords, such as dinner, invitation, etc., which may be useful for indexing into the sender and / or receiver's photo gallery. Images similar to the NLP understanding of the message indicated by the keywords can be suggested for use with the message.

[0239] In an embodiment, the methods, systems, and algorithms described herein for facilitating recommendation and / or selection of image-like content for use with text and / or audio messages may be based on natural language processing of transcriptions of text and / or audio, and in addition to image-like content, these methods, systems, and algorithms may also be applied to other types of content. In an embodiment, messages with text may be tagged, such as with hash tags, etc. (e.g., tweets, etc.). Hash tags may be applied across multiple messages, typically as an organizational tool to facilitate associating different messages that may share a common theme. Natural language processing that may generate information about a message (such as its theme) may be applied to message themes, sentiments, etc. derived from NLP, recommending one or more hash tags from a pool of hash tags. Hash tags may also be associated with descriptive metadata, which may be generated using NLP applied to hash tags. This descriptive metadata is useful in matching and / or recommending hash tags to content. In an embodiment, the NLP applied to the message text content can produce an understanding (e.g., topic, sentiment, etc.) that can be encoded to facilitate matching candidate hash tags from a hash tag pool based on, for example, the hash tag itself and / or metadata associated with the hash tag. Compared to the user interface described herein for manually selecting image content to be combined with the message text content, candidates with tags can be presented in a user interface. User interactions with the presented hash tags can be measured, and the user interactions can be used to further determine which hash tags are preferred for different types of text message content. In an embodiment, a portion of the hash tag pool can be processed using natural language processing to generate metadata describing each hash tag, such as a topic, sentiment, intent, or understanding. The metadata generated by the natural language processing can be used for the selection of candidate hash tags, such as by determining the similarity of the metadata generated by this natural language processing with the output of the natural language processing of the message. In an embodiment, the content matching platform can transmit the output of the natural language processing of the message to the messaging platform via an application programming interface.

[0240] In an embodiment, an interface such as an API can facilitate the interaction between a content matching platform that employs the methods and systems described herein and a messaging platform such as a messaging platform that uses hash tags. An API or similar computer interface can facilitate the content matching platform to programmatically receive the message content that it can process, such as using NLP to develop an understanding of the message. The content matching platform can use NLP to select among multiple hash tags to provide to the messaging platform through the API for possible inclusion in the message. In an embodiment, the content matching platform can provide information derived from the understanding to the messaging platform that can implement a certain degree of hash tag matching. In an embodiment, the API can facilitate users of the messaging platform to view candidate hash tags provided by the content matching platform. Users can view candidate hash tags in the native interface of the messaging platform or in an interface shared with the content matching platform, such as by providing a link to a web page containing the interface to share.

[0241] In an embodiment, an interface such as an API may facilitate communication between messaging platforms (e.g., TWITTER TM ) and a content matching platform. The interface can facilitate the exchange of message-based content and image-based content (e.g., gifs, etc.) between the messaging platform and the content matching platform. The interface can facilitate the messaging platform to provide message text content, etc. to the content matching platform, and the message matching platform can apply natural language processing, among other things, to develop an understanding of the message intent, such as themes, emotions, etc. The results of this understanding or other processing performed on the message text content can be used to identify one or more candidate image type content items. The API can facilitate the transfer of these one or more candidate image type content items from the content matching platform to the messaging platform for inclusion in the text portion of the message. In an embodiment, the messaging platform can receive any of the following: one or more candidate image items, an encoding of the understanding, a link to one or more candidate image items, a complete message including text content provided from the messaging platform, and one or more candidate image items, etc.

[0242] refer to Fig.42, depicts a flow diagram for using natural language processing to facilitate recommendation and / or automatic selection of hash tags for a hash tag-based messaging platform via a messaging platform interface such as an API. The text of a message, etc., may flow from a messaging platform 4201 to a content matching platform message receiving interface 4202 via an inter-platform communication API 4203. The message (such as a text portion of the message) may be processed by a natural language processing facility 4206 to determine intent, sentiment, theme, or other understanding of the message text as described herein. The output of the NLP facility 4206 may be used to review and recommend at least one hash tag from available hash tags 4115 for possible use with a message from the messaging platform 4201. One or more recommended hash tags may be presented in a user interface 4210 that facilitates a user to select and / or approve one or more recommended hash tags. (One or more) hash tags can be aligned with the message in step 4212, which can produce a message with the selected hash tag, which can be transmitted to the messaging platform 4201 via the messaging system interface API 4203, in which the message can be processed according to the messaging platform 4201 protocol.

[0243] In an embodiment, natural language processing of text content (such as text portions of contact events, messages, etc.) can produce an understanding of the text, which can indicate the subject, emotion, intent, etc. of the text, which can also be applied to messages containing processed text content. The results of NLP applied to the text content of the message can be associated with the message in the form of descriptive metadata. The results of NLP can also be used to classify messages into categories of subject, emotion, intent, etc. In an embodiment, a messaging platform that can use hash tags to facilitate access to messages, etc. can benefit from automatic tagging of messages with hash tags indicating categories. As an example, NLP can be used to process messages in a messaging platform centered on hash tags to determine that the subject of the message is armed personnel and the emotion of the message is respect. Hash tags indicating such emotion-based themes can be recommended and / or automatically added to messages. NLP-based classified hash tags can be made directly visible to users of the messaging platform, or can be hidden from users, while enabling the platform to better organize and manage messages exchanged on the messaging platform. The NLP described herein can facilitate the creation of hash tag types-emotion, intent, relevance, popularity, and the like.

[0244] In an embodiment, in addition to facilitating the sender of a message, a contact event, etc. to control how the message is presented to the receiver on the receiver's computing device user interface (e.g., which image is displayed, which ringtone is played back, which video is played back, etc.), a platform for providing comparable functionality can be adapted to collaborate between the sender and the receiver on how to present a contact event and / or a message on the receiver's computing device user interface. In a simplified two-user example, two users can share control of how to present messages and / or content events on each other's user interfaces. Either user can control at least a portion of the content presented as a sender and as a receiver in the user interface. This shared control can be achieved by the user interface features described herein, which facilitate the sender to determine what images, ringtones, etc. are activated on the receiver's computer interface. However, the receiver may have an impact, including the content that the sender may have selected to be played back on the receiver's computer. A user in the role of a receiver can configure his computing device to receive the sender's preferences and messages from the sender, while ensuring that the receiver is set to define a portion of the message that the receiver can control, or if the receiver has not set any preferences for the portion or has not yet selected to control the portion, the default sender's settings. In this example, a user in the sender role can indicate a first image to be presented to the recipient; however, a user in the receiver role can indicate a different image to be presented to the recipient in response to the sender's message. In addition in this example, a user can assume the receiver role and provide data to another user's computing device that can limit the images that the other user can choose from. In this example, the platform can identify a dozen candidate images based on processing of the text portion of the message written by the sender. However, when the candidate images are presented to the sender for manual selection, only a subset that meets the criteria established by the receiver is visible and / or selectable. In addition in this example, a user in the receiver role can provide a specific image list, image set, link to a list of links to specific images to the user in the sender role, and the sender's content matching platform can select from them when presenting candidate images to the sender user for selection.

[0245] In the example of a recipient mobile device using the methods and systems described herein, an application with a lock screen function on the recipient mobile phone, computing device, etc. can capture messages from a wide range of messaging clients (including but not limited to WhatsApp). TM , Facebook Messenger TM, default SMS / MMS messaging app, etc.), and uses processing capabilities including natural language processing (NLP) capabilities to analyze the content of the incoming message and pair it with a relevant content item for display on the lock screen of the recipient's mobile device, such as a graphics interchange format item ("GIF"), a video, or any other type of content proposed throughout the present disclosure. The paired content can then be played on the lock screen as part of or in response to a message notification to the user. Continuing with this example, the lock screen-enabled application can further allow a user to respond to messages they receive directly on the lock screen without having to unlock the phone or open a separate messaging application. The user interface of the lock screen-enabled application on the lock screen can include a "click to view" button that takes the user to an interface panel where the user can begin creating a response. In an embodiment, the phone can remain locked for all other purposes so that exiting the interface panel returns the phone display to its lock screen.

[0246] By these examples, when the user enters text in the interface panel, one or more of at least three response modules can be activated. The plain text response module can be activated so that the user can enter a text response, and when applicable, the message transmission client that the incoming message wants is docked. In an embodiment, the GIF reply module can use natural language processing to automatically match the relevant GIF (or other such image files) with the user's response while typing the user's response, and the matching is such as based on a series of GIFs (or other) and the association of keywords, themes, etc. in a table, database, library, etc., which can be accessed to retrieve appropriate GIFs (or other images) based on keywords or themes derived from natural language processing. The GIF module can also update the interface panel with a GIF (or other image) that may be relevant (such as in a part of the panel), while enabling the user to use the keyboard and also to see what he or she is typing. Interface elements (such as, for example, sliding left or right) can allow the user to select in a GIF that may be relevant, and / or propose other automatically matched GIFs (or other images). Alternatively, a GIF gallery (or other images) may be presented in a portion of the interface panel to facilitate easier viewing and selection from a series of automatically matched GIFs (or other images). For example, swiping left or right may bring additional GIFs, etc., into view. A video / photo module may be activated to facilitate the user to take photos or videos using a camera (e.g., a camera integrated into a phone, computing device, etc.) that the user can access through his or her mobile device, including a networked camera, etc. The video / photo module may also facilitate the user to access his or her photo / video library, third-party photo / video libraries, the complete SCCMC structure, SCCMC channels or image / photo / video content therein, etc.

[0247] Independent of which module(s) are activated to facilitate a user's response to an intercepted message, a response may be sent back to the sender's point of origin, optionally via the same messaging client, SCCMC app, etc., to which the originating message was directed. Thus, an application with lock screen functionality may include the ability to adapt outgoing communications so that they are suitable for various communication channels. An application with lock screen functionality may also include the ability to interface with and / or activate one or more communication capabilities or applications of a computing device, mobile phone, tablet device, etc., without having to abandon the locked state of the device. In the example, if a Snapchat message is sent from a sender's mobile device, the sender may receive a response from the sender's mobile device. TM The message sent by the client is intercepted by the lock screen application, which can then communicate with the Snapchat application resident on the mobile device once the user indicates that the response message (with or without audio, GIF, photo, video, etc.) is ready to be sent.TM Apps to communicate, or Snapchat can be generated within apps with lock screen functionality TM formatted response message to deliver the response back to the corresponding Snapchat on the sender's mobile phone, computing device, etc. TM As proposed, in many embodiments, an application with a lock screen function can simulate Snapchat TM device-to-device communication, and sent directly back to the sender’s Snapchat TM After the user completes the input and indicates that the prepared response should be sent, the user's display on his or her computing device, mobile device, tablet device, etc. can return to the lock screen.

[0248] In an embodiment, the methods and systems described herein may include configuring a library of audio, video, and image content for use in processing messages, etc., as described herein. The library may include items, such as images, which may be configured with attributes that facilitate associating items of the library with messages, such as text messages. In an embodiment, configuring attributes may be based on an understanding of messages that can be processed by an application with a lock screen function, such as text messages processed using natural language processing to develop an understanding of topics, keywords, emotions, etc. In an embodiment, an attribute may represent an aspect of a message related to the intent of the message, such as the emotion being conveyed, etc. Natural language processing of text and other messages may result in attribute values ​​and types that can be applied to items in the library. In some examples, applying these attributes and attribute types may be done by a person reviewing the image. In many embodiments, they may be automatically applied by a computer executing an algorithm or other method that may determine various aspects of an image, such as facial expressions associated with emotions, etc. In an embodiment, machine learning may be used to develop automatic determinations, such as by training a machine learning system using a training message set to which a person has assigned attributes. In an example, an image of a user smiling may be associated with an attribute of a happy emotion. The sentiment attributes of items of such a library may be populated with one or more data elements representing the output of natural language processing of a text string, the output indicating happy sentiment meaning in the text.

[0249] In an embodiment, a method and system for creating, using and maintaining a media content structure includes a sender-controlled contact media content (SCCMC) structure and its application, and may further include adapting, configuring, etc. one or more media content structures that can be accessed by a receiver computing device, computer, mobile device, etc. during a contact event interaction between a sender computing device and a receiver device, including those stored on the receiver's computing device or provided by a server, etc. The media content structure (such as an SCCMC structure) that can be accessed by a receiver computing device in response to receiving a message (such as a text message, etc.) from a sender computing device can include or reference one or more data elements, such as images, videos, audio, text content, etc. In an embodiment, before presenting a media content structure (such as content associated with the sender of the message) to the receiver, one or more data elements can be adapted, configured, or replaced by another data element based on information obtained from the sender's computing device. Data elements such as images can be adapted or replaced by another image to reflect the sender's mood or emotion. In an example, the SCCMC for the sender can include an image of the sender with a pleasant smile. However, in other examples, at the time the sender sends the message to the recipient, the sender may not be feeling well, for example due to a common cold, etc. Based on information indicating the sender's status, the pleasant smiling image may be changed to an image reflecting the recipient's status. The replacement image may come from a portion of the SCCMC structure, from another SCCMC structure of the sender, from a third-party source, from the sender's computing device, and the like. Information about the sender's current mood, emotional state, health state, and the like may be sent from the sender's computing device to the recipient's computing device along with the message. In an embodiment, an application with a lock screen function on the recipient's computing device may include a SCCMC application on the recipient's computing device, which may process the message and accompanying information to determine which aspects of the SCCMC to adapt before presenting SCCMC items corresponding to one or more attributes of the sender.

[0250] In an embodiment, a sender may use a SCCMC messaging application to create a message that may include information about his or her current mood, etc. The SCCMC application may provide a user interface with features that enable a user to specify his or her mood. Alternatively, an emotion determination system running on a sender computing device, a recipient computing device, or in a remote location (such as in the cloud) may rely on a technology that facilitates detecting the sender's emotions, such as facial recognition, speech recognition, the sender's distraction level, etc. An emotion determination system running on a sender computing device or in a remote location (such as in the cloud) may rely on a technology that facilitates detecting the sender's emotions simultaneously with the sender sending the message from the sender's computing device to the recipient's computing device. The emotion determination system may interact with or run as part of an SCCMC application, an application with a lock screen function, or a combination thereof to provide emotion information, which in turn may be used as an attribute, such as for selecting an appropriate media content structure, such as an SCCMC media structure or other media structure, such as selecting a GIF, or other images or videos that reflect the sender's emotions or are consistent with the sender's emotions.

[0251] In an embodiment, the application with the lock screen function is an application based on SCCMC, and can be part of a platform that can intercept messages from the sender's computing device before delivering the message to the recipient's computing device, including messages prepared in other messaging applications as described herein. In an embodiment, the platform can process the intercepted message, including processing the message while the sender is inputting the message on the sender's computing device, independently of the messaging application that the sender is using (or not using it at all), to provide the sender with the services of the platform before facilitating the delivery of the message to one or more recipients. In an embodiment, the services provided may include the inclusion of images, GIFs, videos, audios, alternative texts, etc. The services provided may be customized for the sender based on information about the sender known by the platform, such as the activity level of the sender with the computing device, the sender's emotions, the sender's preferences, etc. The services provided may be further customized by natural language processing of the text portion of the intercepted message, and the services provided may include suggestions for gifs, images, videos, audios, texts, etc. to be included in or integrated into messages, etc. The platform may automatically or in response to the sender's acceptance of one or more offered services, prepare and / or update a content media structure (such as a sender's SCCMC structure) for transmission to one or more recipients as part of a contact event interaction. As an example, an updated SCCMC structure or a portion thereof may be sent to a corresponding SCCMC application on one or more recipient computing devices prior to a message so that the recipient's computing device can use the updated SCCMC structure when processing the accompanying message.

[0252] In another embodiment of the platform, a state determination system running on the sender's computing device (such as operating with or as part of a SCCMC application, etc.) may be provided with or may detect the sender's current mood, emotional state, health state, etc., and responsively prepare or update a media content structure (such as an SCCMC structure) that may be provided to possible recipients (such as currently active users in a group of which the sender is a member). Alternatively, the updated media content structure may be provided to a platform server, which may store the media content structure, such as storing it as an SCCMC, such as for use in facilitating messaging during a contact event interaction. The methods and systems described herein for updating the SCCMC structure on a recipient computing device may be employed to update a variety of recipient devices.

[0253] In an embodiment, a contact event processing platform is provided that intercepts messages originating on a sender computing device based on a messaging capability or application operable on the sender's computing device, the contact event processing platform utilizes natural language processing to process the intercepted message, and based thereon, forwards the message with an appropriate media content structure (such as an SCCMC structure), based on the results of the processing and / or natural language processing. In an embodiment, the contact event processing platform includes: an application with a lock screen function that provides a unified messaging user interface for a recipient computing device, such as when the recipient computing mobile device is in a lock screen mode, while also delivering adaptations to the intercepted message, such as based on processing that is compatible with other messaging applications that are targeted.

[0254] In an embodiment, the contact event processing platform can intercept messages on a recipient computing device that are destined for other messaging applications that are operable on the recipient computing device. The platform can utilize natural language processing to process the intercepted message, and based on that it can forward the message and / or media content structure (such as an SCCMC structure) in a unified messaging user interface of the recipient computing device, based on the results of the processing and / or natural language processing to be processed, such as when the recipient computing device is in a lock screen mode, while also delivering an adaptation to the intercepted message, based on processing that is compatible with the other messaging application targeted on the recipient computing device.

[0255] In an embodiment, an application with a lock screen function can facilitate interaction with a user via a user's computing device display when the user's computing device is locked (e.g., conventionally when the user's device is locked, only the lock screen can be presented on the mobile device display). However, an application with a lock screen function can facilitate interaction with a user through the user's lock screen, or facilitate interaction with a user on a screen other than the user's mobile device screen (such as a nearby screen on a TV, a screen specified by the user, a default screen, etc.). In an embodiment, a user can interact with an application with a lock screen function via another screen using at least a user interaction capability associated with another screen (such as a touch screen, a touchpad, voice activation, a keyboard, etc.). In this way, regardless of whether the user's computing device can be in a lock screen mode, an application with a lock screen function can provide a way for the user to view and respond to messages being processed by the application with a lock screen function.

[0256] In an embodiment, an application with a lock screen function as described herein may be connected to, associated with, or integrated with an augmented reality and / or virtual reality (AR / VR) user interface. An application with a lock screen function may provide information (such as images, audio, video, text, etc.) to the AR / VR user interface so that the information can be presented in the interface. The information may be based on or include at least a portion of the SCCMC controlled by the sender of the message. As an example, under the control of the AR / VR user interface, a SCCMC application running on or communicating with a computing device may receive information processed by it, such as by intercepting a message for another messaging platform as mentioned herein, to access the corresponding SCCMC structure. In an embodiment, the information in the structure may include media content, such as images, audio, video, etc., that may be provided to the AR / VR user interface for integration with the AR / VR user interface presentation. In an embodiment, in response to receiving message information from the SCCMC application, the AR / VR user interface may incorporate active elements such as avatars that may be deployed in the AR / VR user interface.

[0257] In an embodiment, the methods and systems of sender-controlled media (SCM) and the like described herein can facilitate the use of other forms of content including audio, visual, animation, and the like to enhance existing caller ID functions. The use of SCM platform functions that facilitate the rendering of SCM data structure content and the like through an electronic display can be integrated or coupled with the existing call functions of a smart phone to deliver a visual caller ID as a new form of caller ID. Caller IDs typically include a caller's number or other alphanumeric value (e.g., name), such as that found in a conventional phone book, and the like. On a smart phone device including a two-dimensional electronic display graphics screen, this information is typically rendered in an incoming call display interface. Such an incoming call interface typically provides options to the user, such as answering the call, sending the call to voicemail, rejecting the call, and the like. The integration of SCM content rendering capabilities with incoming call interface capabilities can facilitate the presentation of a range of content to the call recipient, including a real-time image of the caller (e.g., by using the camera function of the caller's phone, and the like). In an embodiment, a visual caller ID can provide a rich form of caller identification while optionally retaining the incoming caller option.

[0258] As described and depicted herein, a caller can use an SCM content creation interface to create and manage SCM content and data structures. This interface can be automatically activated in response to a user initiating a call from a phone equipped with an SCM platform capability (such as an SCM content creation application, etc.). Such an interface can be adapted to generate a call-specific SCM data structure that can be shared with a recipient when initiating a call. Such a call-specific SCM data structure can be temporary, existing only for the initiated call. However, a call-specific SCM data structure can be created and stored, such as on an SCM platform server, etc., so that, for example, when a caller calls the same recipient, they can be accessed. In this way, the creation of a call-specific SCM data structure can actually generate a recipient-specific SCM data structure.

[0259] In an embodiment, the SCM platform can use the methods and systems of sender-controlled media (SCM) described herein and depicted in the accompanying drawings, for example, to replace traditional caller ID information or to cooperate with it to send video or other forms of media to the caller. Therefore, the platform can overcome the obvious lack of information that is transmitted in traditional caller ID messages accompanying phone calls. In an example, the video can be a profile video. The video can be a customized video based on an advertiser or retail product. In an example, the SCM is associated with the caller ID and is linked to a profile in which a filter can be set to send a predetermined SCM data structure to multiple users, for example, for coordinated call activities for retail, affiliated groups (school / work / church calls), etc. In an example, the SCM data structure can be grouped with a pre-arranged set of contacts so that it can be sent to those specific phone numbers, such as when the call is configured to be sent to those specific phone numbers in a call queue, etc. In other examples, the SCM data structure may include and / or reference an identity photo or video to facilitate soft authentication or otherwise establish a level of reliability in establishing a connection with a caller (e.g., to increase the recipient's comfort level when answering the call). In an example, the SCM data structure may include or reference a registration logo. In an example, the SCM may be a live broadcast of the face of the user initiating the call. In an example, the SCM data structure may include an inside joke or shared memory.

[0260] refer to Fig.43, presenting an interface screen for creating, activating, and presenting a visual caller ID using the methods and systems for visual caller ID described herein. In an embodiment, creation screen 4302 depicts a visual caller ID creation screen, in which an image currently captured by the user's phone camera is shown to the user. The screen prompts the user to keep and record a caller ID video. The video is captured and optionally associated with an SCM data structure that is to be used for a call to be placed by the user. In an embodiment, screen 4302 can be activated as a result of the user initiating a call to a recipient; however, screen 4302 can be activated separately from initiating a call. Screen 4304 presents the user with an opportunity to review the captured visual caller ID, which is a video in this example, but can be an image, graphic, or other content. The captured video can be saved so that the user can reference it when generating a visual caller ID for other users. The captured visual caller ID can be set as the default visual caller ID for calls placed by the user. This can be initiated by creating an SCM data structure for the user, which can be presented to the user when initiating an outgoing call so that the user can choose to continue with the presented visual caller ID or create a new visual caller ID, as enabled by screen 4302. Screen 4306 depicts an incoming call screen for the recipient of the call from the user. In screen 4306, the visual caller ID is presented / rendered / played back depending on the type of visual caller ID and the recipient's preferences. Screen 4306 also shows a traditional caller ID 4308, indicating the possibility of controlling the incoming call screen by a traditional caller ID display function (for presenting caller ID 4308) and an SCM-based display function for presenting a visual caller ID associated with the caller for at least this call instance.

[0261] In embodiments, the visual caller generation interface, an embodiment of which is depicted in screen 4302, may present options to the user based at least in part on the call context, such as whether a call has already been initiated to the recipient when the interface is activated. Options during this situation may enable the user to choose not to send the visual caller ID, to adjust aspects of the visual caller ID, to authorize the SCM data structure platform to adjust aspects of the visual caller ID (e.g., based on aspects of the recipient, etc.). These and other options are described elsewhere herein.

[0262] In an embodiment, an SCM data structure for a call to a caller group may be created and accessed. The SCM data structure, etc. may be associated with a recipient parameter, which may be populated with a value indicating a specific SCM data structure that should be used when calling the recipient. In an embodiment, the recipient's corresponding parameter may indicate that the member is part of a call group, such as a member of a team, an employee of a company, etc.

[0263] In an embodiment, the SCM data structure content can be used in conjunction with a conventional caller ID, or can replace a conventional caller ID. The sender-controlled media (SCM) data structure may include content that can be used in a caller's global caller ID profile, such as a video or image, etc., so that instead of conventional caller ID content, a video or image can be rendered on the electronic display of the receiver's phone. In an embodiment, the global caller ID profile may be stored in the cloud or stored to other network facilities, to a central node in a network of a smart phone or a call originating device, etc. By these examples, the global caller ID profile may be stored in the cloud to a central node, which in some cases may be a cloud hub for incoming and outgoing calls. The network of a smart phone may provide central node capabilities, but other cloud network systems and providers may also provide central node capabilities. In an embodiment, existing caller ID functions and communication channels may be adapted to facilitate access to SCM data structure content, etc. In an embodiment, an example of adapting an existing caller ID function to accommodate SCM data structure content access may include providing a URL or other link as a conventional caller ID value. In an embodiment, the SCM data structure can be directly passed to one or more recipients of such a call or message in a peer-to-peer manner by the call or message initiator. By adapting the incoming call function to identify such a link, the content of the link can be rendered as a visual caller ID. In an embodiment, the incoming call function can be adapted to identify such a link, and the control of the display screen of the phone (for example, one or two for rendering and receiving user input) is transferred to the SCM data structure processing application that facilitates access to the content of the link and makes it rendered for the recipient. Accessing the content of the link can include referencing the SCM data structure, which has been transferred to the recipient device by the SCM data structure communication method and system described herein. In an embodiment, initiating a call from a caller to a recipient can include: sending the caller SCM data structure information specific to the call to the SCM data structure processing application executed on the recipient's phone. Alternatively, initiation of the call can include sending call-specific caller SCM data structure information to an intermediate server, such as a web server that maintains the call-specific caller SCM data structure awaiting an access request, such as an access request from a recipient phone that has detected a link to the call-specific caller SCM data structure in the caller ID for the initiated call.

[0264] In an embodiment, an exemplary system may provide a visual caller ID function, which may include a call processing function executed on a personal mobile device, which may present the content of an SCM data structure referenced in the caller ID data associated with the call. In such an exemplary system, caller ID data may be provided to the call processing function of the personal mobile device using existing caller ID technology, etc. The exemplary system may further include: an SCM selection interface presented to the initiator of the incoming call on an electronic interface of a call originating device operated by the initiator. The selection interface or other visual caller ID interface may facilitate the initiator of the incoming call to select the SCM data structure simultaneously with the initiation of the call. In an embodiment, the selection interface may be activated in response to the user initiating a call. In order to facilitate linking the SCM data structure selected by the call initiator with the caller ID information sent to the recipient, an SCM linking function executed on a networked server receives a selection of the initiator's SCM data structure selection from the SCM selection interface (or other visual caller ID interface), the SCM linking function may form a digital link based on an indication, and may utilize, for example, a digital link to update the caller ID data of the incoming call.

[0265] In an embodiment, an alternative exemplary visual caller ID system can facilitate generating visual caller identification data. A visual caller ID selection interface can be responsive to a user of a call originating device initiating a call and cause options to be generated for content of the visual caller ID to be presented in the visual caller ID selection interface. In addition to the visual caller ID selection interface, for example, in response to receiving an indication of a visual caller ID from an SCM selection interface (or other visual caller ID interface), an SCM linking function, for example, executed on the call originating device can form a digital link to an SCM data structure based on the indication. Finally, a caller ID field for the caller ID of the initiated call can be updated with the digital link so that the visual content referenced in the SCM data structure can be presented to the recipient on an incoming call user interface; thereby providing visual caller ID content and options for handling the call to the intended recipient.

[0266] In an embodiment, a visual caller ID function may be provided by a method that may include presenting at least one content item in an SCM selection function of an electronic display of a caller's calling device or other mobile device. In response to the caller selecting a content item in the SCM selection function, which may include capturing an image or video using a camera interface of the phone, the caller's device may initiate a call to the recipient. Also in response to the caller selecting the content item, a link to an SCM data structure may be generated through which the selected content item is accessible. Next, the generated link may be used to update the caller ID data for the call so that the link is provided as a caller ID data value to the recipient's phone. Upon receiving the updated caller ID data value, content from the SCM data structure referenced by the caller ID data value may be rendered in an electronic user interface of the call recipient's call receiving device.

[0267] In an embodiment, a visual caller ID function may be provided by a method that may include, in response to receiving an indication of a request to initiate a call from a caller, presenting at least one of an image and a video in a visual caller ID selection user interface of an outgoing call function of a call originating device. The visual caller ID selection user interface may facilitate generating an indication of a user selection of one of the image, the video, and no selection. Next, an SCM data structure may be configured to reference the selection. Next, the caller ID data for the call may be updated to include a link to the configured SCM data structure. At a call receiving device (e.g., a phone) of a recipient, and in response to receiving a call, the contents of the configured SCM data structure may be presented on a display screen of the call receiving device by following a link in the caller identification information.

[0268] In an embodiment, the method and system of the visual caller ID function described herein can be enhanced in various ways, and no examples thereof are described. In an embodiment, the visual caller ID interface of the call originating device can facilitate the use of the camera function of the call originating device to capture content to be used as visual caller ID content. The captured content can be any of a still image and a video. The captured content can be further presented as at least one of the optional items of visual content in the visual caller ID interface. In an embodiment, the caller ID field of the initiated call can include a link to an SCM data structure. Such as an SCM data structure can indicate an image presented on an electronic display of a call receiving device when a call is received. As mentioned, the content item to be selected for use in the visual caller ID can be an image captured by the camera function of the call initiating device during the activation of the visual caller ID selection interface. In an embodiment, the SCM data structure can indicate a video presented on an electronic display of a call receiving device when a call is received. Moreover, the SCM data structure can indicate a multimedia object presented on an electronic display of a call receiving device when a call is received. The visual caller ID includes a profile of the caller. The visual caller ID contains promotional content. In an embodiment, the linked SCM data structure may include images and / or video captured simultaneously with the initiation of the call. Moreover, in an embodiment, the visual caller ID selection interface may further present the call origination phone numbers and, based on the user's selection of one of them, update the visual caller ID SCM data structure link. Such functionality may be useful for users who operate multiple disjoint businesses to facilitate the user's selection of a business by selecting a business-specific phone number.

[0269] In embodiments, the visual caller ID method and system may be affected by aspects of the recipient of a call or contact instance, such as the recipient's context, the recipient's name, the recipient's group affiliation, characteristics that distinguish the recipient from other recipients in the recipient's group, and the like. In embodiments, the visual caller ID media content object affected by the recipient may be a selectable combination of images, text, audio, and video that a user, such as a participant of an SCM platform described herein, selects to, for example, form a template that may be adapted based on the recipient context and rendered when the user calls a recipient (e.g., another participant in the SCM platform environment). Elements of such a template may include: a field for the recipient's name, a field for the recipient's group, a field for a group to which the user has assigned the recipient, a portion of a visual content item (e.g., where a facial image of the recipient may be discarded), and the like.

[0270] In an embodiment, configuring and / or using a visual caller ID affected by a recipient may include a computing system, such as a server, a caller's mobile phone, etc., which receives an indication of initiating a call from a sender to a recipient. In an embodiment, initiation may be caused by a user of a call originating device selecting a contact in a call origination interface. Additional steps for configuring a visual caller ID affected by a recipient may include: determining an SCM data structure for use in configuring a visual caller ID for a call based on a selection of a visual content item by the initiator of the call. In an embodiment, the selected content item may be cross-referenced to a library of SCM data structures. An SCM data structure that is found to include and / or reference the selected content item may be selected for use with the call. Alternatively, an SCM data structure generation function may collect information about the caller, optionally including information about the call (e.g., time of day, location of the caller, etc.), and the selected content item to generate an SCM data structure for use with the call. In order to utilize recipient-related aspects to influence the visual caller ID of a call, an SCM platform participant database may be referenced, such as using an identifier of the recipient of the call. Such an identifier may be a telephone number of the recipient. Alternatively, the caller may access the SCM platform participant database and select the recipient therein, effectively unambiguously identifying the recipient entry in the SCM platform participant database. Using the SCM data structure for the call and the SCM platform participant information for the recipient, an adapted SCM data structure may be generated by adapting a portion of the SCM data structure based on information items referenced in the SCM participant database using the recipient identifier. To complete the use of a recipient-influenced visual caller ID for the call, the caller ID of the call may be configured with a link to the adapted SCM data structure.

[0271] In an embodiment, selection of a visual content item may occur in a visual caller ID selection interface executed on a call originating device. Such a visual caller ID selection interface may cause options for content for the visual caller ID to be presented in the visual caller ID selection interface so that the caller may complete a selection of content items for the call. In an embodiment, the options for content presented in the visual caller ID selection interface may include an image provided by a camera of the call originating device. To facilitate generation of the image provided by the camera of the call originating device, the visual caller ID selection interface accesses the camera and presents content imaged by the camera as one of the options for content.

[0272] In an embodiment, adapting a portion of an SCM data structure to affect a recipient's visual caller ID may include adapting a recipient name element of the data structure to correspond to the recipient's name. Adapting a portion of the SCM data structure may include adapting a recipient group affiliation element of the data structure to correspond to the recipient's group affiliation. Adapting a portion of the SCM data structure may be based on the recipient's name. Alternatively, adapting a portion of the SCM data structure may be based on the recipient's group affiliation. Multiple recipients may be called, some of which may be part of a group. For multiple recipients that are not part of the group, it may be preferred to use a different visual caller ID. Therefore, the portion of the SCM data structure may be adapted based on the correspondence of the recipient's name to a list of exception names.

[0273] In an embodiment, calls from business owners to clients, cold leads, warm leads, referred leads, etc., can benefit from the use of visual caller ID by customizing portions of the visual caller ID based on the client status of each recipient. As an example, an information item for a recipient referenced in an SCM participant database can indicate a client status. Based on this indication, a client message portion of the SCM data structure can be adapted. Exemplary client status and resulting client message portion adaptations can include: the recipient is an existing client, so messages sent to this recipient will refer to the existing relationship between the caller and the recipient; the recipient may be a referral from an existing client, so messages sent to the recipient may reflect this referral, the recipient may be a lead without a referral, so messages sent to this recipient may be non-referral lead messages.

[0274] In embodiments, visual caller ID content may be adapted based on call context, such as time of call, date of call, proximity to a holiday or other event, temporal proximity to an upcoming meeting between the caller and recipient, location of the caller, location of the recipient, intent of the call, and the like. Dynamically adapting caller ID content such as visual caller ID content (e.g., at the time of the call) may be based on call-specific context (e.g., different from recipient context, etc.). In embodiments, caller ID content may be automatically selected based on call-specific context. Adjusting a portion of the content (such as an image, an object in an image, etc.) may be based on call-specific context, such as time of day, time remaining before an offer ends (e.g., a promotion), and the like.

[0275] In an embodiment, methods and systems for call context-based visual caller ID content may involve, upon receiving an indication of an initiated call from a sender to a recipient, performing an action related to the visual caller ID (e.g., selecting, generating, adapting, etc.). Alternatively, performing the action may be associated with configuring a call queue for the recipient, such as for automatic calling, etc. For time-based call context, an amount of time remaining in an offer related to the call, such as an offer from a sender to a recipient, an offer from a third party to a sender and / or a recipient, a time when an offer is available to a sender, a recipient, or both but not specifically for the sender, the recipient, or both, etc. Other relevant time-based aspects of the call context may be the time until an upcoming event, or the time elapsed since a previous event, such as a public event, a private event, an event detected on a calendar of at least one of the sender and the recipient. Still other time-based aspects of the call that may affect the visual caller ID may include the day of the call, date, time of day, etc. Based at least in part on the time-dependent context of a call, methods and systems for call context-based visual caller ID content may involve generating an SCM data structure that may affect the visual caller ID of the call.

[0276] In an embodiment, an SCM data structure may be generated based on relevant information and profiles for one of the call participants (such as a sender and a receiver, etc.). The SCM data structure may also be configured based on call context information, such as the location of the sender and / or receiver, the time / day / date of the call, the expected call duration, the product or service associated with the call, the message of the call, etc. In an example of adapting a visual caller ID based on the time of day, the image may be adapted to reflect the current time of day by adjusting the overall hue of the image to correspond to the amount of daylight. For daytime calls, the hue may be lighter; for nighttime or evening calls, the hue may be milder. Generating an SCM data structure may occur simultaneously with the caller initiating the call, such as by the caller using a camera function of the call originating device to capture an image and / or video of the caller, which may be used at least in part for an SCM data structure, such as a display content portion of the SCM data structure. The SCM data structure may also be generated by adapting a portion of an existing SCM data structure based on the amount of time remaining. In order to implement the use of SCM data structure content, etc. as visual caller ID content, a link to the generated SCM data structure can be used to configure the caller ID data element for the call. The use of the SCM data structure by the recipient call receiving device (e.g., smart phone, etc.) can be implemented by an application executed on the recipient device, which accesses the SCM data structure through the received caller ID information, so that the SCM data structure indicates an image, graphic, video, rendering, handwritten note, etc., which is presented on the electronic display of the call receiving device when the call is received. The SCM data structure may include a multimedia object (e.g., an animated cartoon with a caller's voice, etc.), which is presented on the recipient's call receiving device when the call is received. In an embodiment, the image can be an image captured by a camera function of the call initiating device, which is an image captured simultaneously with the initiation of the call. The SCM data structure adapted for call time may include a sender's profile (e.g., a call initiator, a party represented by the sender, etc.), promotional content (e.g., promotional content provided to the recipient by the caller (optionally acting as a representative)), etc.

[0277] The above examples of visual caller ID content generation, delivery, and use may include functionality such as sending an SCM data structure and / or its link along with and / or in place of traditional caller ID information. The present disclosure includes descriptions of a range of forms of SCM data structure content; any such form may be included in these visual caller ID methods and systems. In an embodiment, the visual caller ID methods and systems may utilize some existing channels, such as caller ID channels that are traditionally controlled by a third party (such as a telephone call service provider), etc.; however, the visual caller ID may be controlled by the caller / sender rather than a third party through these methods and systems. This capability supports caller delivery information (such as profiles, advertisements, retailer content, etc.) as a form of caller ID. As described herein, an SCM data structure used to provide a visual caller ID to a recipient of a call may be linked to the caller's existing caller ID so that the caller ID may be used, for example, as a link to an SCM data structure database, through which the corresponding SCM data structure and / or visual caller ID content may be accessed for presentation to the recipient of the incoming call. Caller ID-based access to the visual caller ID SCM data structure content can support a caller in selecting an originating telephone number for a call that may be different from the native telephone number associated with the call originating device of the caller. In this way, for example, a caller can initiate an automatic call to one or more recipients using visual caller ID content that is specific to different call originating numbers. As an example, an insurance service broker can select an originating telephone number for a first insurance provider for a call from the broker to a customer of the first insurance provider, and select an originating telephone number for a second insurance provider for a second call, for example, to a potential customer of the second insurance provider. Moreover, as described in the above examples, the visual caller ID content may be affected by and / or combined with recipient related information. One such example of a visual caller ID affected by recipient related information occurs when the recipient is part of a group (e.g., a member of a church, work, school, business, professional group, etc.). The recipient's group affiliation may be used to adapt and / or otherwise affect the generation of the SCM data structure for the call, such as changing a logo, etc., presented as a visual caller ID to reflect the group affiliation, etc. For the retail example, the visual caller ID may be automatically adapted based on the relationship between the caller and the recipient. For example, the message associated with the caller may be changed based on the recipient being an existing customer of the caller, the recipient being a referral of the caller, the call going to a new potential customer (e.g., a cold call), etc. In an embodiment, the caller may be associated with a profile and SCM data structure that is used to provide the visual caller ID to the recipient of the call.By way of these examples, a caller may be associated with a profile that may be from a single phone or call originating device, but that phone is associated with at least two (or more) subscriber identification modules, SIM cards, or other electronic devices onboard or pluggable that may control identification of the phone. In these examples, a caller may be associated with a profile that is specifically associated with multiple subscriber identification modules, etc., so even though the caller may appear to have multiple phones or call originating devices, the caller with its multiple subscriber identification modules, etc. may appear to be from a single source when associated with one or more specific SCM data structures used to provide visual caller ID.

[0278] In embodiments, methods and systems for visual caller ID (such as methods and systems based on SCM data structures) may support a user setting content (such as video, audio, text, images, multimedia, etc.) as part of a temporary visual caller ID that expires, such as after a certain amount of time, in connection with an event, etc. In embodiments, this allows a user to have a visual caller ID that expresses a feeling or emotion for a certain amount of time, after which it returns to the content that was previously set as their profile visual caller ID. This temporary visual caller ID may be accessible to specific contacts, groups, or all recipients.

[0279] In an embodiment, a method and system for the generation and use of temporary visual caller ID content may include: configuring temporary content for a visual caller ID for a call initiated by a user of a call originating device, such as using a visual caller ID interface executed on a call originating device. An SCM data structure with temporary content may be generated from an existing SCM data structure by copying the existing SCM data structure and including a temporary visual content item that replaces the existing visual content. The existing SCM data structure may be configured with a control data field that may facilitate marking the time / duration of a link to the temporary visual caller ID content while retaining a link to the content to be accessed and once the temporary content expires, will be used for the visual caller ID. A function executed on, for example, a call originating phone of a caller or a server that routes a call from a caller may access a (temporary) SCM data structure for the caller after receiving an indication that the caller initiated a call, and present the temporary visual content item or the existing visual content item as a visual caller ID to the recipient of the call. The function can determine which visual caller ID content to present based on the relationship between the current time (such as the time of the call) and the time-limited use parameter of the temporary visual content item. In an embodiment, configuring the visual caller ID includes configuring an SCM data structure having a link to the temporary visual content item and at least one of the temporary visual content items. In an example, the relationship between the current time and the time-limited use parameter of the temporary visual content item can include the remaining time of using the temporary visual content item. The relationship can be the use status of the temporary visual content item. In an example, the time-limited use parameter of the temporary visual content item can include an expiration date for using the temporary visual content item; the parameter can include a use start date and a use end date for using the temporary visual content item. Other forms of the time-limited use parameter can include: a calendar date, a pre-date duration of use before the calendar date, and a post-date duration of use of the temporary visual content item after the calendar date, a link to an entry in the digital calendar of the user of the call originating device, and the like. For embodiments in which SCM data structure content may be distributed among various servers and edge devices to facilitate faster access to content for incoming calls, methods and systems for use of temporary visual caller ID content may include, for example, utilizing expiration criteria to adequately update the caller ID functionality system.

[0280] In an example, a temporary content item for visual caller ID use may be time-limited so that the content may be good for a duration from the time of first use, good until an event, and the like. An event may be a specific time in the future, such as a pre-arranged meeting, and the like. An event may also be based on an asynchronous trigger, such as when a user is not working for the day. An event may also be conditionally described, such as based on a candidate being elected to office. Conditional events may also be used to activate temporary content, such as changing a caller's salutation based on the results of an election, and the like. In further examples, conditional events may include the caller's location, a change to the caller's location, one or more recurrences of the caller in a certain location, and the like. In an embodiment, conditional events may include: a caller's relationship status, a change to a caller's relationship status, the time between changes to a caller's relationship status, a change in relationship status on the receiving and sending ends of a call, and the like. In an embodiment, conditional events may include a caller's birthday, the proximity of a caller's birthday to a specific age range or minimum age, a birthday used for age verification, proximity to other users with similar or identical birthdays, and the like. In an embodiment, conditional events may include holidays relevant to the caller or any caller, proximity of the caller's birthday to certain holidays, holidays related to the caller's identified religious, social, or family affiliations, proximity to other users with similar interests on holidays relevant to the caller, and the like.

[0281] In embodiments such as multi-call / call group based procedures, the visual caller ID may be adapted to select a recipient. The caller ID and related content may be configured, selected and / or adapted, such as by editing the SCM with one of several options when contacting a group, etc., to effectively configure the SCM for each recipient in the group based on, for example, the recipient's relationship to the group. In an exemplary automatic calling procedure, the caller ID content (such as the visual caller ID content, etc.) may be adapted to select a recipient. If the recipient is an SCM platform user, the caller ID may be adapted based on, for example, the context of the recipient's membership in the call group.

[0282] In an embodiment, a method and system for recipient-specific caller ID (e.g., SCM data structure) adaptation may include, among other things, detecting the initiation of a first call from a first caller to a first recipient of a recipient call group. Using at least an existing caller ID delivery mechanism, the call may make visual caller ID content configured based on the characteristics of the recipient call group accessible to the call receiving device of the first recipient simultaneously with the first call received to the call receiving device. In this way, visual caller ID content specific to the call group (such as content described in a sender-controlled media data structure associated with the group, caller, or a combination thereof) may be provided to at least one recipient in the call group. As calls to recipients in the group proceed, information about the caller, recipient, and group may be processed for the purpose of generating recipient-specific caller ID information. In response to detecting the initiation of a second call from the first caller to a second recipient of the recipient call group, and based on a comparison of the characteristics of the recipient group with the corresponding characteristics of the second recipient, the visual caller ID content may be different from the group visual caller ID content, and the visual caller ID content may be used for the call to the second recipient. Such an action may be accomplished by indicating a first SCM data structure for the group and a second SCM data structure for a recipient for whom, while a member of the group, the group SCM data structure is not appropriate. An example may include a group containing adults and children. Certain messages that may be appropriate for adults may not be appropriate for children in the group. Rather than sending messages appropriate for children to all members of the group, the recipient-differentiated caller ID content method and system enables the sending of recipient-differentiated caller ID content to a group of recipients.

[0283] In an embodiment, the visual caller ID content can be accessed by the call receiving device of the recipient as a link to an SCM data structure including at least one of the visual caller ID content and its link (e.g., a URL to a network-based location). For such an embodiment, multiple SCM data structures can be configured at a network-based location so that the only distinguishing information segment of the phone that is passed to the recipient in the call is a link to the SCM data structure specific to the recipient. For all recipients, the processing of the SCM data structure can be the same, wherein the content in the SCM data structure specific to the recipient is rendered for the recipient. Therefore, the link to the SCM data structure can be provided to the call receiving device as caller ID information. In order to facilitate the timing of presenting the visual caller ID content to the recipient so that the recipient can view the caller ID content when requesting a call connection, the visual caller ID content can be accessed by the call receiving device as a link in the SCM data structure, and the visual caller ID content is provided to the call receiving device simultaneously with receiving the call. As described herein, the characteristic that can distinguish the recipient from other recipients in the group can be a group subordinate identifier. To facilitate distinguishing members of the group, the corresponding characteristics of the second receiver can be the group affiliation status between the receiver and the receiver group, such as member, long-term member, temporary member, trial member, new member, junior member, free member, paid member, etc.

[0284] In an embodiment, configuring the visual caller ID content for the second recipient may include adjusting at least one element of the visual caller ID content based on a characteristic of the recipient group and further based on the result of the comparison. As an example, the element of the visual caller ID content that may be adjusted may be a logo of the general group caller ID content, and the message in the visual caller ID content may be adjusted for the recipient as an exception (optionally along with the logo) to receive the group visual caller ID content. As another example, the corresponding characteristic may distinguish the second recipient from at least one other recipient in the call group. Other examples include: the characteristic of the recipient group indicates an enterprise contact, and the recipient-specific distinction indicates whether the recipient is an internal or external enterprise contact, so that confidential messages of the internal contact are not delivered to the external enterprise contact as visual caller ID content.

[0285] In addition to recipient-to-recipient differences, the corresponding characteristic used to distinguish the second recipient may also be the status of the second recipient relative to the object of the second call; for example, if the call is to recruit members to a group, former members of the group may receive different messages than recipients who are not former members of the group.

[0286] Generating group-specific visual caller ID content for recipients for whom group caller ID content is suitable may include: configuring the visual caller ID content based on the characteristics of the recipient's call group by selecting the first caller visual caller ID content and adjusting the first caller visual caller ID content to reflect the characteristics of the recipient group. In an embodiment, this may be exemplified by an image of a recent group meeting that may be changed for calling a group member in the group meeting. Further continuing, configuring the visual caller ID content for the second call may be further based on an identifier of the user who activated the initiation of the first call, the purpose of the call, the time of the call, and / or the aspects of the recipient's call receiving device. In an embodiment, configuring the visual caller ID content may be based on the characteristics of the call receiving device. Such characteristics may be selected from a list of call receiving device characteristics, including but not limited to device storage capacity, device screen size, device operating mode, and the availability of a sender-controlled media (SCM) application on the call receiving device.

[0287] In an embodiment, in addition to a conventional caller ID delivery channel, the visual caller ID content can be made accessible to the call receiving device by transmitting at least one of the visual caller ID content and a link thereto to the call receiving device over a short messaging service connection. Similarly, the visual caller ID content or a link thereto can be transmitted via an iMessage channel linking the first caller to the call receiving device.

[0288] In an embodiment, the SCM data structure may be adapted for recipient-specific exceptions at the recipient's call receiving device. The SCM data structure processing application may process the SCM data structure identified, for example, in the caller ID data, and adapt it based on, for example, certain fields in the SCM data structure, using recipient-specific content such as from the recipient's profile, marking certain fields as optional and / or marking them as conditional based on the recipient's relationship to a group, etc.

[0289] In an embodiment, a group SCM data structure including visual caller ID content items, etc., may be automatically created on a computing device, such as a server, that has access to group-specific information, such as a member list of group members, conditions that determine when members in the member list should not receive group visual caller ID content, etc. The group SCM data structure may be configured, selected, adapted, etc., at a server, at a sender's call originating device, at a recipient's call receiving device, etc.

[0290] Other factors that may affect the content of the Visual Caller ID may be related to the network over which the call is placed, such as network speed, the location of the SCM data structure relative to the recipient, user device characteristics (such as storage capacity, screen size, operating mode, parental controls present on the device, etc.

[0291] In an embodiment, a call origination interceptor function may allow a pre-arranged single profile or contact group (e.g., group email, group text, etc.) established for a group contact to be overwritten, etc. Such an interceptor function may facilitate the classic problem of avoiding unexpected "reply all" situations. When originating a group contact (or even a single contact) to which an existing SCM data structure is assigned, or when there is a pre-arranged SCM data structure, the originator may be directed to a specific screen that interrupts the sending of the pre-arranged SCM. The screen may be configured to require confirmation of whether the SCM should be used for this call to the recipient or to all recipients in the case of multiple recipients (e.g., group text, etc.). In an example, a skip button may be presented to the user, which may, for example, remove the recipient from the group contact, cancel the message when it is to a single user, organize the list to which the message is to be sent, etc. In another example, the contact initiator may be reminded of (i) the specific SCM to be used for this contact / to be sent to this group, (ii) the size of the group, and (iii) the option of skipping the recipient, the use of the caller ID, the use of the specific SCM, etc.

[0292] In embodiments, the instant generation of visual content (such as visual caller ID content for calls placed from smartphones, computers, etc.) and SCM data structure content (such as messages, texts, video calls, etc.) for presentation to recipients of digital contacts can facilitate contact time differentiation of contact-originated content for one or more recipients, such as when contacting a group of recipients, some of which the caller may prefer to exclude from receiving generalized group-based contact-originated content, such as group text messages, etc. In embodiments, the instant generation of visual contact content can be facilitated by an initiated contact interruption function that presents options for one or more recipients to a contact initiator (e.g., contact sender, caller, etc.) before the one or more recipients are notified of an impending contact / call. In a call-based example, such a contact interruption process for operation on a call originating device of a caller can be activated in response to detecting the initiation of a call that results in visual caller ID content being accessible to a visual caller ID rendering function executing on a call receiving device. The interruption process may include pausing the delivery of the call until a call delivery signal is detected, such as a signal from the caller, to continue the call using the visual caller ID content selected for the call. The interruption process may present a visual caller ID configuration user interface to the caller while pausing the call delivery. The visual caller ID configuration user interface may be configured to enable selection of visual caller ID content for the call from, for example, multiple visual caller ID content options. By these examples, the visual caller ID content options may include: existing content, at least one replacement content, and content generation via the camera and / or microphone functions of the call originating device. In an embodiment, the visual caller ID content option may be an emoticon. In an embodiment, the visual caller ID content option may be a gif including a gif with an embedded action. In an embodiment, the visual caller ID content option may be a jpeg or other image content. In an embodiment, the visual caller ID content option may be a sticker, a label, or the like. By interrupting the call before the call reaches the recipient (optionally before the call originating device initiates the call), the caller may be able to affect the visual caller ID of the call by selecting content (or excluding content). The selection may result in a call delivery signal being generated and the selected content item being transmitted to the recipient. In an embodiment, the selected content item may be an emoticon. In an embodiment, the selected content item may be a gif. In an embodiment, the selected content item may be a sticker or a label. The selected content item may be empty. The selected content item may be one of an emoticon, a gif, and a sticker. The interrupt function may activate a visual caller ID configuration user interface that may configure the visual caller ID content using the audio / video recording capabilities of the caller's device.The visual caller ID configuration user interface may include an option for specifying that the selected visual caller ID is to be used for the call made to the recipient. The interface may also present an indication of the visual caller ID content in the SCM data structure as existing content. Existing content may include recipient-specific content. Existing content may be instantly adapted to the recipient via the interrupt function, and optionally via the visual caller ID configuration user interface. In an embodiment, existing visual caller ID content may be specified for a call by a caller to a member of a group of which the recipient is a member. In an example, a call may include multiple recipients, wherein existing content is specified for a caller group including multiple recipients. Other examples include a single recipient call. The interrupt function facilitates the user-selected content option to be used as the visual caller ID content for a subset of recipients during a call pause. For example, during a call pause, a caller may select a recipient of this subset of recipients.

[0293] In an embodiment, the delivery of the call may be suspended based on a characteristic of the recipient, such as a relationship between the recipient and a group of recipients to which the call is directed. In an embodiment, the interruption function may include detecting the initiation of the call on the caller's device; the interruption function further causes the delivery to be suspended.

[0294] refer to Fig.44 , depicting an interrupter screen for methods and systems for supporting instant visual caller ID content adaptation, generation, and control of outgoing calls, which may include calls to groups of callers, some of whom existing visual caller ID is not suitable. Fig.44 The embodiment depicts a user calling a recipient, and the method and system for interrupting the delivery of outgoing SCM data structure content can be applied to any type of electronic contact, including but not limited to email, text messaging, video broadcasting, video calling, etc. Screen 4402 depicts an option for interrupting the delivery of an SCM data structure to a recipient. In this option, the user has initiated a call to the recipient. This creation screen 4402 can be due to (i) no SCM data structure available for the caller, (ii) the user has indicated in a previous screen of the interrupter method and system to generate a new, and optionally temporary, visual caller ID, (iii) the user indicates or accepts a suggestion from the SCM data structure platform to not use the existing visual caller ID for this call to this recipient. Screen 4402 provides the user with multiple options for generating a visual caller ID, such as typing a message, selecting an emoticon, recording a video, and the like. Screen 4404 depicts the recipient's incoming call screen, where the visual caller ID generated and / or selected in screen 4402 is presented to the recipient along with the option to accept the call 4406 and the regular caller ID 4408.

[0295] In an embodiment, a visual caller ID method and system may include generating and using visual content associated with live recorded audio / text as a visual caller ID SCM data structure in real time (e.g., during the initiation of a call), such as by determining the intent of the recorded audio / text and selecting visual content based on that understanding. As an example of a visual caller ID based on call content, a method may include detecting the initiation of a call from a caller to a recipient, and pausing the delivery of the call until a call delivery signal is detected. While the delivery of the call is suspended, the method may include generating visual caller ID content through a series of steps, including: (i) capturing live content including at least one of audio and text input by a caller to a call originating device; (ii) processing the captured live content to determine the intent of the live content; (iii) automatically selecting a visual element based on the intent, combining the selected visual element with the live content, thereby generating visual caller ID content; and (iv) and generating a call delivery signal for delivering the call. Processing the captured live content may include audio-to-text conversion of the captured live content. The processing may further include natural language processing (NLP) of at least one of the captured text and the text converted from the captured audio to determine at least one of the intent of the call, the subject of the call, and the action of the call. In an embodiment, automatic selection of content may be implemented by a visual content selection algorithm that, among other techniques, implements the best fit of visual content items to the determined intent, action, or subject. The intent in such content may be based on the recipient of the call, the relationship between the recipient and a group of recipients associated with the call (such as a call recipient group), and the like. As part of a method and system for generating context based on call content, pausing a call may be based on a range of aspects related to the call, such as characteristics of the recipient, which may be a relationship between the recipient and a group of recipients to which the call is directed.

[0296] In embodiments, the narrative features of the methods and systems described herein may allow users of an SCM platform who are engaged in a conversation to receive suggestions for SCMs to be delivered along with the conversations exchanged by the users. Various methods and systems for understanding content (such as machine learning, natural language processing, etc.) may be applied to sense characteristics of the exchange, which may be used as narrative information for suggesting SCM content, such as in the manner of a panel-style storyboard for addition to the conversation, etc.

[0297] In an embodiment, a message such as a text message that may be part of a text exchange between users may be intercepted as the message is being sent from a first user in the exchange to a second user in the exchange. The interception facilitates utilizing natural language processing to develop an understanding of the message, which is further used to facilitate suggesting a video / image / audio of a SCCMC associated with the message.

[0298] Suggesting videos / images / audio, etc. based on an understanding of the message exchange may facilitate the narrative. Other resources for suggestions and evaluation of suggested content may include social media. In an embodiment, social media-based perception of the combination of text and a particular image may support the suggestion (e.g., a social media user may indicate a preference for the combination, such as by liking a message or taking another action, such as forwarding a message), or may indicate that a suggested image may not be an appropriate image for the text (e.g., a social media user may indicate a dislike for the combined image / text, such as by liking the message or showing a lack of action, such as not forwarding the message).

[0299] Visual content suggestions such as videos and images and audio content, etc. may be the result of natural language processing of exchanges among users. In an example, NLP can be used to analyze the message text content to generate an understanding or context of the message. In an embodiment, such an understanding can suggest that the sender is inviting the recipient to participate in a sporting event. The understanding can be encoded as keywords, such as team, invitation, etc., which may be useful for indexing into a gallery of photos of the sender and / or recipient. Images similar to the NPL understanding of the message indicated by the keywords can be suggested for use with the message. As an example, for the purpose of enhancing the narrative, images of the above-mentioned teams can be suggested as well as a schedule showing the event date and the players participating in the game.

[0300] In an embodiment, the context of one or more participants in an electronic communication between users of the SCM data structure platform can be used to enhance the narrative, such as by suggesting images to be included in the communication, etc. One such context is the current emotion of one or more participants in the communication. As described herein, a sender / receiver state detection function can facilitate the detection of the emotions of participants in the electronic communication. The methods and systems described herein may include a method that may include: adapting the SCM data structure in response to receiving an indication of the current state of the sender or receiver in the digital exchange. The SCM data structure can be used to facilitate the narrative by adapting images suitable for a set, such as the emotions of the sender and / or receiver, etc. as described herein. Technologies such as facial recognition of the sender or receiver and processing the content of the communication to develop understanding can adjust content matching parameters so that instead of selecting images in which the sender and receiver are smiling, images in which the participants are unhappy and not smiling are selected.

[0301] In an embodiment, a platform of a user such as a participant in an SCM platform can generate information about a caller such as an unknown caller, which may be useful to other users of the platform when deciding how to respond to a request to connect to an unknown caller. Information collected about calls from callers to other recipients can be stored in a caller identification and evaluation database, which can be updated and used by participants in a platform or group (such as a user group of the SCM platform). This database can be indexed by caller ID information about the caller, for example, provided by a call connection network. The database may include public number search information, additional search information from the results of the public number search, such as criminal records, other phone numbers, names, addresses, etc. of possible callers, etc. The public number search results may be transmitted to participants of the platform, etc., for confirmation and / or determination of relevance to the caller's evaluation. Additional information that may be collected from platform participants may include call tracking information for previous calls from the caller. Platform participants may generate caller evaluation data, such as in response to an evaluation request for an unknown caller, etc. The caller evaluation data may include approval / disapproval of usefulness, suitability, spammer status, etc. In an embodiment, enhancing information about an unknown caller may include generating a profile, such as an SCM data structure, for the unknown caller based on information about the caller collected from internal and external data sources. Such a profile may be generated in response to receiving an indication that a caller who is not a participant of the SCM platform is calling a participant of the platform. The automatically generated profile may be embodied as part of an SCM data structure associated with a caller ID of the unknown caller. Upon subsequent calls from the unknown caller, the recipient SCM data structure processing software (e.g., an app on the recipient's phone) may process the SCM data structure of the caller for the purpose of presenting the visual caller ID content of the caller to the recipient; thereby providing a recipient context about the unknown caller that would otherwise not be readily available when a call from the unknown caller is incoming.

[0302] In an embodiment, when a user of the SCM data structure processing platform is receiving a phone call (text message, email, etc.) from someone who is not a user of the platform, the platform can provide a lot of available information and aggregate it into information that is passed to the caller as the caller's SCM data structure. The information in the automatically generated caller-specific SCM data structure can be used to better identify unknown callers, and can include content that is easily available on the Internet (including social media), any information that can be obtained from a sales system point associated with the platform, any information that can be obtained from other callers / user interactions on the platform, etc. In an example, specific interactions with other people on the platform can supplement information to identify or at least better identify unknown callers. In an example, this feature enables platform users to avoid spam calls. For example, a phone call from a restaurant to a user of the platform can adapt the restaurant's caller ID through an Internet search of the restaurant to include photos or videos from the restaurant's website or social media presence as its visual caller ID. In an example, an incoming call from a ride-sharing service may cause the visual caller ID content to be adapted based on an active request to the ride-sharing service that is associated with the recipient's phone number to show information from the request, such as a live map and picture of the driver and vehicle, as part of an SCM data structure referenced by the caller ID data, and so on.

[0303] In an embodiment, an unknown caller information improvement method can be activated by receiving the following indication: a caller who is not a participant of a sender-controlled media (SCM) platform has initiated a call to a receiver who is a participant of a sender-controlled media (SCM) platform. The information improvement method may include: using the caller ID information provided with the call to collect content related to the caller from multiple sources, including information available in the data storage device of the platform, public number search information, etc. The collected information can be used to generate a sender-controlled media (SCM) data structure for the caller. This automatically generated SCM data structure can be passed to the call receiving device of the receiver, where the visual content of the SCM data structure can be presented to the receiver, such as through a visual caller ID interface on the call receiving device. By encouraging platform participants to provide their views and / or observations of each caller to the participants of the platform, the information available in the data storage device of the platform may include: information collected about calls issued from the caller to other participants in the SCM platform. This information can be optionally used with unverified information about the caller provided by platform members and others to inform the caller's SCM data structure generation process.

[0304] In an example, the information available in the data storage device of the platform includes: the score of the caller made by the participants of the SCM platform. Once the SCM data structure is created for the unknown caller, it can be associated with the caller so that future calls from the caller can facilitate access to the generated SCM data structure. In an embodiment, instead of sending the existing caller ID information to the recipient device, the generated SCM data structure can be provided to the call receiving device using one or more visual caller ID delivery methods such as described herein. This can be achieved by replacing the existing caller ID information (if any) with a link to the generated SCM data structure. As a method for enabling the recipient of a call from an unknown caller to utilize the information collected and the evaluations performed by other platform participants, the SCM data structure generated for the caller can be provided to the SCM platform application executed on the call receiving device simultaneously with the incoming call.

[0305] In an embodiment, utilizing the SCM data structure platform to handle unknown callers can facilitate taking action on the caller, such as suggesting that the caller join the platform, such as by providing an audio message while attempting to complete the call, or by sending a text message to the caller simultaneously with the call (e.g., while attempting to complete the call, while waiting for the recipient to make a decision to accept the call, while an accepted call is in progress, while completing an accepted call, at another time).

[0306] In an embodiment, a method and system for handling an unknown or unknown caller may include generating SCM data structure content, such as an audio message presented to the unknown caller concurrently with the call placed by the caller to the recipient. The content may be relayed to the call originating device of the caller at various times associated with the call, such as before the call is presented to the recipient in an incoming call handling user interface of the recipient's call receiving device (e.g., the recipient's smart phone), while the call is presented to the recipient for acceptance or rejection of the call, during the call (e.g., after the recipient has indicated acceptance of the call), due to the recipient rejecting the call, at a time indicated by the recipient during or immediately after the call ends (e.g., the recipient indicates in an unknown caller handling interface screen that one of multiple types of messages is sent to the caller), at the end of the call, and at another time, such as the next time the platform handles a call from the caller.

[0307] In an embodiment, a method for handling an unknown or unnamed caller may be initiated by receiving an indication that a caller who is not a participant of a sender controlled media (SCM) platform has initiated a call to a recipient who is a participant of the sender controlled media (SCM) platform. The call may include caller ID information, which may be used to facilitate the collection of content related to the caller from multiple sources, including information available in a data storage device of the platform, and public number search information. The collected information may be used as a basis for the caller to automatically generate a sender controlled media (SCM) data structure, which may include visual and / or audio content, a portion of which may be delivered to the caller's call origination device. The portion delivered to the call origination device may be delivered as a ringback tone, as an SMS message, as an MMS message and iMessage, as a traditional voicemail, as a visual voicemail, and the like. In an embodiment, the content delivered to the caller's call origination device may include an offer and / or instruction to join the SCM data structure platform, such as to facilitate easier access to platform participants, and the like.

[0308] In an embodiment, a method and system for handling a call from an unknown or unnamed caller to a platform recipient may include generating caller ID-specific SCM data structure content to be used as a visual caller ID and caller-directed content for the recipient, which may be structured as part of a caller ID-specific SCM data structure or a caller ID-directed SCM data structure so that the caller-directed content is delivered to the caller, such as delivered to the caller's call origination device. The exemplary method may be initiated by receiving an indication that a caller who is not a participant of a sender-controlled media (SCM) platform has initiated a call to a recipient who is a participant of a sender-controlled media (SCM) platform. Any caller ID information associated with the call may be used to facilitate the collection of content related to the caller from multiple sources, including information available in a data storage device of the platform and public number search information. A first SCM data structure for sending to a recipient may be automatically generated based on the collected content. A second SCM data structure for sending to a caller may be automatically generated based on the collected content, and both the first and second SCM data structures or portions of their content may be delivered to the recipient and the caller, respectively.

[0309] The automatically generated first SCM data structure can be delivered to the recipient's call receiving device, where the visual content of the first SCM data structure can be presented to the recipient, such as through a visual caller ID interface on the call receiving device. By encouraging platform participants to provide their views and / or observations of each caller to the participants of the platform, the information available in the data storage device of the platform may include: information collected about calls issued from the caller to other participants in the SCM platform. This information can be optionally used with unverified information about the caller provided by platform members and others to inform the caller's SCM data structure generation process. In the example, the information available in the data storage device of the platform includes: the score of the caller made by the participants of the SCM platform. Once the SCM data structure is created for an unknown caller, it can be associated with the caller so that future calls from the caller can facilitate access to the generated SCM data structure. In an embodiment, instead of sending the existing caller ID information to the recipient device, the generated first SCM data structure can be provided to the call receiving device using one or more visual caller ID delivery methods such as described herein. This can be achieved by replacing the existing caller ID information (if any) with a link to the generated first SCM data structure. As a method of enabling the recipient of a call from an unknown caller to take advantage of information collected and assessments made by other platform participants, an SCM data structure generated for the caller may be provided to an SCM platform application executing on the call receiving device concurrently with the incoming call.

[0310] The portion of the second SCM data structure delivered to the call originating device may be delivered as a ringback tone, as an SMS message, as an MMS message and iMessage, as a traditional voicemail, as a visual voicemail, etc. In an embodiment, the content delivered to the caller's call originating device may include an offer and / or instructions to join the SCM data structure platform, such as to facilitate easier access to platform participants, etc.

[0311] refer to Fig.45, which is a flow chart of a method for unknown caller ID visual caller ID generation and feedback aggregation. In an embodiment, a visual caller ID can be generated for an unknown caller using process 4500, optionally embodied as part of an SCM data structure. At step 4502, a call indication from an unknown caller is detected and / or received. The indication is provided to a content collection function at step 4506, which uses caller ID 4504 to make a call to find and collect information about the caller from the outside (e.g., the Internet) and the inside (e.g., internal SCM platform resources). Next, in step 4508, the collected information can be used to generate an SCM data structure for caller 4510. The SCM data structure for caller 4510 can be provided to an SCM platform communication function, which can implement a step 4512 of passing the SCM data structure to a recipient device, such as a visual caller ID item that the recipient can accept, reject, or provide feedback on, such as whether the visual caller ID is helpful, etc. The SCM data structure or a portion thereof (such as a visual content item) can be optionally provided to the caller. The content provided to the caller may be adapted to inform the caller of the presence of an SCM data structure platform operating on behalf of the recipient to filter / enhance caller ID information, an invitation to join the platform, etc. At step 4514, feedback from the caller, recipient, or both may be aggregated and saved in a data structure 4516, which may be accessed by the platform as an internal platform resource, etc.

[0312] In embodiments, a post-call experience for a call recipient may include the platform taking actions to communicate with the recipient after the caller receives or completes a phone call. For calls with businesses, the platform may coordinate the presentation of content to call participants (typically consumers), such as content from third-party advertisers, retailers, and other partners to deliver coupons, text alerts, reviews, and other retail-based inducements to further maintain an initiated relationship or continue a previous relationship during the call. Other examples of post-call experiences may include platforms that pair with partners who want to learn more from call participants, such as follow-up comments, reviews, reporting work, etc. The methods and systems for post-call experiences described herein may facilitate a range of content delivery and interaction both directly and in real time (e.g., by presenting interactive content in a user interface of a call participant's phone) and indirectly (e.g., texting, emailing, etc.).

[0313] In an embodiment, the method and system of the post-call experience may include using the call in progress screen to present automatically generated interactive content, such as by using the SCM data structure method and system described herein, optionally docking the phone call in / out of the application or function with the SCM data structure platform receiver / caller application. The post-call experience method and system docked with the existing phone call application can provide, for example, a post-call (and / or active call) experienced by configuring the sender controlled media (SCM) application to receive the SCM data structure of the sender participant from the SCM platform. In the example, the sender participant can be any user of the platform, but can preferably be a party interested in collecting additional inputs, and / or establishing / continuing the relationship between the sender / caller and the receiver established or referenced in the call. In the case of the SCM data structure application configured, the post-call experience can be activated by docking the SCM application with the phone screen control portion of the phone application, so that the content in the received SCM data structure is provided to the phone application as the content to be presented on the electronic display of the phone while the phone call is in progress. Essentially, the effect of this on user content (e.g., images, text, video, moving content, etc.) can be presented in an ongoing call screen on a call participant's smartphone, etc. Feedback and / or input from the call participant can be captured and displayed by monitoring (such as using an SCM application) user actions associated with the electronic display (e.g., user clicks, swipes, scrolls, ignore actions, or no actions at all) while presenting the content during or after the call, and the content is displayed on the call participant's screen. To the extent that the user action may affect the presented content, the SCM data structure may need to be updated to affect the electronic display in response to the monitored user action (such as changing to a "thank you" screen, etc.). The docking of the SCM data structure application with the phone application can be directed to the phone screen control portion of the phone application. In an embodiment, the SCM data structure application or at least a portion useful for post-call call participant interaction can have the phone screen control portion of the phone application integrated therein. In an embodiment, the SCM data structure application or at least a portion useful for post-call call participant interaction can be integrated into the phone application. Such a portion may include functionality for receiving and processing SCM data structures. Other forms of interfacing the SCM data structure application with the phone application may include, but are not limited to, configuring a real-time communication channel between the SCM application and the phone application, providing SCM content to the phone application through the channel, and providing user actions on the phone screen to the SCM application.

[0314] refer to Fig.46, presenting a user screen for a post-call experience method and system. An incoming call may include caller identification features, such as those exemplarily presented in screen 4602, where a vendor credit service company (CSC) is placing a call to a recipient. Caller identification features may be presented in association with the methods and systems of visual caller ID presented herein, and in association with the methods and systems of source-controlled media data structure operations described herein. After indicating that a call between, for example, the CSC and the recipient has ended, but optionally, before the phone returns to its pre-call operating mode, a post-call experience screen 4604 (among others) may be presented to collect feedback about the call, such as a call to the recipient with a CSC representative. In an embodiment, context from the caller identification screen (such as the name of the representative) may be used in the post-call experience screen to more effectively personalize the post-call experience. A portion of the post-call experience screen 4604 may include a rating option 4606 for the call, an option 4608 to provide more feedback, and an option to end the post-call experience 4610, such as not providing feedback.

[0315] In an embodiment, the post-call experience may include determining and / or using the intent or understanding of content during the call (such as a conversation between call participants) using, for example, audio natural language processing, audio to text and text natural language processing, machine learning, artificial intelligence, etc. to generate content (optionally, content in an SCM data structure) that is contextually relevant to the call content, and presenting the generated (optionally SCM data structure) content to at least one of the call participants, such as the initiator of the call (e.g., the caller) and the call recipient (e.g., the recipient). The post-call experience based on call intent may be affected by a process that includes collecting call context for an active call between a caller and a recipient, and developing an understanding of the call (e.g., during or after the call is completed), which may include: at least one of the call intent, call participants, and actions related to the call (e.g., actions of or associated with the call participants) based on the content communicated between the caller and the recipient and optionally based on the context of the call (e.g., the time of the call, etc.). This understanding can be exploited by a content suggestion algorithm that determines candidate content items (e.g., videos, images, photos from a gallery, etc.) related to at least one participant of the call based on the suitability of the content items to the understanding. The content suggestion algorithm, method, and system applied in the example can be described elsewhere in this article. With the suggested content, the next step can be to present at least one candidate content item to at least one of the caller and the receiver during (or after) the call in the electronic interface of the device used during the call. In the example, the discussion during the call may be about the product. The understanding developed from the call may be that the first participant used the product and the second participant did not use it. The suggested content may include: an image posted by the first participant social media from the first participant who used the product. This image may be presented to the first participant for approval to share it with the second participant. Based on feedback from the first participant, the content may be configured as an SCM data structure of the SCM data structure application that is delivered to the second participant's phone. The delivered SCM data structure can activate the SCM application on the second participant's phone to optionally present the content item approved by the first participant through the screen control portion of the phone application. Thus, in an embodiment, the electronic interface of the device is adapted to facilitate selection by a first call participant of one of the presented content items for sharing with a second call participant.

[0316] In an embodiment, the understanding of the call content may be based on natural language processing of the call audio and / or a transcription of the call audio (such as a real-time transcription stream, etc.).

[0317] In an embodiment, call context may optionally be used to enrich the in-call and / or post-call experience. Call context may also be a form of machine-based understanding, such as understanding of at least one previous call between the caller and the recipient, understanding of at least one other call made by the caller, and understanding of at least one other call received by the recipient. Other ways in which other calls of the participants may affect the post-call experience may include using call context collected from at least one previous call between the caller and the recipient. The call context may be related to the use of the SCM data structure platform by the call participants, such as the context may include sender-controlled media transmitted to the recipient simultaneously with the call, a sender-controlled media platform profile of the caller, a sender-controlled media platform profile of the recipient, and the like.

[0318] In an embodiment, providing an in-call or post-call experience may include configuring a smartphone application that takes over control of the call in progress screen. Control of the screen may be taken during call connection, upon detection of termination of the call, before releasing the phone function of the phone, and the like. Such in-call or post-call processing may include monitoring the call in progress state of a call participant device, such as a smartphone. Upon detecting a call in progress state indicating, for example, that an ongoing call has terminated, activating a post-call application of the device, the activation causing the screen of the device to be updated. In an embodiment, the screen update may include any content, but may preferably include: a call participant's call to an action, or informational content related to the call. The call in progress state may instead indicate permission to control the phone screen, such as after an initial time period for the recipient to accept the call. An example of a call to an action may include: at least one selectable element presented in the screen, the at least one selectable element, when selected by the call participant, causes an update of a database associated with the call to the action based on the selected at least one selectable element. As an example, the call to the action may be to agree to or disagree with the terms of business between the call participants. Based on the call participant's acceptance or rejection of the terms, the database will be updated differently. In an embodiment, activating the post-call application may cause the call functionality of the phone active during the call to remain active (substantially maintaining the phone call functionality so as not to allow another call to be initiated or received) until either the call participant confirms the screen update (e.g., by the call participant selecting an element in the updated screen to confirm), the screen update timer expires, and the user dismisses the updated screen. In an embodiment, the call participant may confirm the screen update when the call participant selects a selectable element presented in the screen update, or the call participant dismisses the screen update. The post-call screen update application may continue to present the updated screen until the call participant confirms the screen update or the screen update timer expires.

[0319] In an embodiment, a method and system for implementing a post-call experience may include: upon detecting an indication that a call has ended, signaling a post-call application to take control of a user interface screen of the phone. The indication may be based on the content of the call (e.g., words spoken during the call) or based on a connection state (e.g., a call network signals that the connection through which the call was made has terminated). An exemplary method thereof may include: in response to receiving an indication that an ongoing call is about to end, activating a post-call application of a call participant device, which causes a screen update of the device, the screen update including content related to the caller or the content of the call, and may include a call to action, information, and the like. As described above, the indication received regarding the call in progress state indicating the end is based on the connection state. However, the indication may be provided by a call function of the device, such as when a user of the device touches an "end" function. However, as described above, the indication that the post-call experience may be activated may be based on the content of the call. In an embodiment, as the content of the call is spoken, the content of the call may be processed using near real-time speech analysis of the audio, such as by detecting call end words, such as "goodbye," and the like. Other forms of call content analysis may be employed, including natural language understanding and the like, which may indicate that the call is about to end, or at least that the conversation about the call is winding down.

[0320] In an embodiment, the content provided by the post-call application may include a call to action. Exemplary calls to actions may include: providing an offer of a product or service to a call participant, asking a call participant to comment on the call, soliciting comments about the call, capturing call participant feedback about the spammer status of one of the call participants regarding the other of the call participants, collecting call participant feedback about an SCM data structure or, more likely, its content related to the call (e.g., visual caller ID, etc.), and the like.

[0321] In an embodiment, a post-call application may update a screen to include content from a sender-controlled media (SCM) data structure of the originator of the call that is automatically generated based on the call in progress. The updates and / or calls to actions may facilitate collecting call participant feedback regarding the automatically generated SCM data structure content. In an embodiment, another type of feedback that may be collected through a post-call experience may include feedback regarding the originator of the call, goods or services referenced during the call, and the like. When one or more call participants utilize the SCM data structure methods and systems described herein, the screen update may include visual content provided by one of the call participants, such as from a sender-controlled media (SCM) data structure associated with one of the call participants.

[0322] In an embodiment, methods and systems for caller handling influenced by a call community can facilitate protecting recipients of the call community when receiving phone calls from users outside the platform. A form of call / contact firewall can be configured based on the confidence of other users completing phone calls or other communications on the platform to assess whether unknown callers (unknown message senders) are spammers, "bad companies", other callers to be avoided. In an example, call community members can provide comments and contribute to confidence intervals for "safe" callers, which can be determined based on "successful" calls relative to spam calls based on these comments and indicators.

[0323] refer to Fig.47 , depicting a block diagram of a call community firewall configuration and a flow diagram of a process for handling calls by the firewall. The call community firewall process may include detecting an incoming call at step 4702. Such an incoming call 4712 may be caused by a call from an external caller 4710, such as a caller that is not part of the community. The community firewall 4714 may act as a proxy or filter for such incoming calls 4712 for at least a portion of the members of the community 4718. At step 4704, caller ID information of the caller 4710 may be used to identify one or more entries in a community firewall database 4716. Based on information contained in the database 4716, such as input from community members 4720 regarding the caller and / or the member's preferences regarding firewall operations on behalf of the member(s), at least one call impacting action may be determined at step 4706. The call impact actions performed at step 4708 may include, for example, allowing the call to be transferred to the recipient, enhancing the call with visual caller ID information, providing the caller's SCM data structure to the recipient's phone, terminating the call, providing content items back to the caller (e.g., voice response), and many other such call impact actions.

[0324] In an embodiment, a call / message / contact firewall may activate (one or more) firewall actions for calls received at a recipient phone for or to a participant of the firewall. The (one or more) firewall actions may be based on caller ID-based criteria derived from firewall participant experience / feedback with the caller. In an embodiment, a call community contact firewall may include a caller ID capture function that retrieves caller ID data (e.g., provided by a call network) for calls to recipients who are members of a call community (e.g., placed through a call network). The firewall may include a caller ID lookup function that retrieves a firewall state from a firewall state database linked by the retrieved caller ID data (e.g., the caller's phone number). The firewall state database links the caller ID data with a call community score for the caller received by a call community member (e.g., a call community score for a previous call associated with the caller ID data). The firewall action function then implements at least one call impacting action based on the retrieved firewall state. In an embodiment, the call impacting action may include: terminating the call, connecting the call to the recipient, redirecting the call to voicemail, redirecting the call to an automated response system.

[0325] In an embodiment, the call impact action may include providing a call action recommendation to the recipient and taking at least one other call impact action based on the recipient's response thereto. The action that may be taken by activating the firewall action function or as a result thereof may include generating a multimedia profile based on the caller ID data and delivering it to the recipient. The multimedia profile may be configured as a sender-controlled media (SCM) data structure. In an embodiment, delivering this form of the probe may include communicating about the profile with the SCM interface function of the participant's call receiving device. In an embodiment, the multimedia form of the profile may be delivered via a multimedia messaging service (MMS). In this scenario, the caller may also be contacted. As an example, a multimedia profile based on caller ID data may be generated and delivered as a multimedia messaging service (MMS) message to a telephone number indicated in the caller ID of the call. Another type of content that may be provided to the caller may include an audio message that may be played for the initiator of the call in lieu of placing a call to the recipient. The call action may further include redirecting the call to an isolation module, redirecting the call to a law enforcement agency, and the like. In an embodiment, a multimedia profile may be configured as a sender controlled media (SCM) data structure, but may be implemented with a third party web application (in whole or in part) through an SCM, which may be created and uploaded through a webapp, application programming interface, other links, and the like. By way of these examples, an SCM data structure may be created in a third party portal or system external to the platform, and the SCM data structure may be passed from therein, but then imported into the platform for use as an SCM data structure. In an embodiment, an enterprise having many locations and even locations that still have landlines may establish an SCM data structure to extend the associated multimedia profile during a call or message to a user on the platform. In doing so, a single number may be used as a caller ID number for outreach, while more geographically specific call-back numbers may be provided during a call or post-call experience and follow-up. In an example, a barbershop having many locations may establish content for an SCM data structure to extend a multimedia profile related to the barbershop during a call or message to a user on the platform. In an embodiment, the in-call or in-message experience may include linking to a local barbershop and providing coupons and / or messages from the barbershop to barbershop locations near each caller or message recipient in the group call. In an embodiment, the post-call or post-message experience may also be used to link to local barbershops and other entities to provide coupons, feedback, and various post-call or post-message products.

[0326] In an embodiment, a contact firewall action set for callers and / or for protecting one or more recipients can be established based on the call community (e.g., SCM platform participant) experience with the caller, which can be a data structure. Information provided by a third party and / or publicly accessible about the sender / caller can also be used to build a call community firewall. When a participant receives a call, or when a sender indicating that the sender is not a participant in the media (SCM) platform has initiated contact with a receiver who is a participant of the media (SCM) platform controlled by the sender, a contact firewall for the participant of the media (SCM) platform controlled by the sender may be generated. The sender information associated with the contact instance (e.g., caller ID, source IP address, recorded name, etc.) can be used to poll the participants of the platform about the spam sender status value of the sender, essentially asking the participants whether they think the caller is worth passing to the recipient. In an embodiment, information and polling results can be aggregated across many calls of platform members to the caller. The aggregated information can be processed (e.g., summed, averaged, or otherwise analyzed), and a spam sender value that can be compared with a spam sender threshold can be generated. If a threshold is exceeded or the aggregation of polling results otherwise indicates that the caller should be blocked, the platform may reject future contacts from the sender, such that the sender is blocked from contacting participants of the platform. Such a firewall may be used for phone calls, emails, text messages, and any other form of electronic contact. In an embodiment, such a firewall may be applied to a service that receives and scans paper mail, which is then delivered as an electronic contact (e.g., as a daily or weekly email, etc.).

[0327] In an embodiment, when a member receives a call, or when an indication that a sender who is not a member has initiated contact with a recipient who is a member, a contact firewall for protecting participants in a call group can be generated. The sender information associated with the contact instance (e.g., caller ID, source IP address, recorded name, etc.) can be used to poll members about the sender's spam sender status value, essentially asking members whether they think the caller is worthy of being passed to the recipient. Information and polling results can be aggregated across many calls of members by callers. Aggregated information can be processed (e.g., summed, averaged, or otherwise analyzed), and a spam sender value that can be compared with a spam sender threshold can be generated. If the aggregation of the threshold or polling results indicates that the caller should be blocked in other ways, future contacts from the sender can be rejected, so that the sender is blocked from contacting members. Such a firewall can be used for telephone calls, emails, text messages, and any other form of electronic contact. In an embodiment, such a firewall may be applied to a service that receives and scans paper mail, which is then delivered to members of a call group as an electronic contact (eg, as daily or weekly emails, etc.).

[0328] As described herein, the contact firewall can take actions that affect the contact, including terminating the contact, forwarding the contact to a participant, providing the participant with the option to accept the contact, redirecting the contact to an automated response system, redirecting the contact to an offline messaging service (such as voicemail), or facilitating the sender to leave a text, image, or video message, etc. The firewall can provide additional control to the recipient by providing call action suggestions to the recipient and taking call affecting actions based on the recipient's response to the call.

[0329] In an embodiment, actions that may be taken by or as a result of activating a firewall action function may include generating a multimedia profile based on caller ID data and delivering it to a recipient. The multimedia profile may be configured as a sender controlled media (SCM) data structure. Delivering this form of a probe may include communicating about the profile with an SCM interface function of a participant's call receiving device. In an embodiment, the multimedia form of the profile may be delivered via a multimedia messaging service (MMS). In this scenario, the caller may also be contacted. As an example, a multimedia profile based on caller ID data may be generated and delivered as a multimedia messaging service (MMS) message to a telephone number indicated in the caller ID of the call. In an embodiment, another type of content that may be provided to the caller may include an audio message that may be played for the initiator of the call in lieu of placing a call to the recipient. The call action may further include redirecting the call to an isolation module, redirecting the call to a law enforcement agency, and the like. In an embodiment, the action taken by the firewall may include delivering an audio message to the initiator of the call. Such an action may be implemented in lieu of placing a call to the recipient.

[0330] In an embodiment, the call community firewall method and system may include collecting call originator feedback from community members for each call originator that originates a call to a community member. The call community firewall method may further include storing the feedback from the members in a contact firewall data structure indexed by call originator identification information (such as caller ID, etc.). Moreover, a contact firewall function may be configured that intercepts calls to group members while determining whether the feedback for the call originator in the contact firewall data structure indicates that an action affecting the call should be taken.

[0331] In an embodiment, the firewall can operate on calls routed through the VoIP channel, and the firewall can be processed on the cloud server. The cloud server can act as a call receiving agent for the recipient.

[0332] In an embodiment, a sender controlled media (SCM) platform method and system can facilitate the authentication of participants in a contact instance (e.g., a phone call, a customer service text communication, etc.). In an embodiment, a retailer or other third party can rely on the platform to communicate with and / or authenticate its users. In an example, a credit card provider such as American Express can use the platform method and system to develop its own SCM data structure, which can be transmitted to customers who engage with electronic communication / contact instances to facilitate trusting all parties. In an example, a customer using an SCM data structure application or the like on a digital device such as a mobile phone can interact with the SCM data structure content presented as a technology for establishing a form of authentication. In an example, the SCM data structure can include information about the participant, including information that may only be known to the participant, such as an authentication key, a picture, a facial picture, a test to establish a guarantee, a selection of difficult names, and other media, with which the interaction can be used to authenticate users, retailers, and / or communication channels. In an example, when an agent of a retailer / credit card provider works with the client (e.g., via phone, text, etc.), the agent can verify identity, name, address, or other information by visually displaying content on the client's screen, such as on the lock screen of the client device, optionally using the SCM data structure rendering functionality described herein. The agent can use this interface to show relevant tutorials, video photos, or other media to enhance the conversation or customer support experience.

[0333] refer to Fig.48 , depicts a call participant authentication screen using SCM data structure methods and systems to provide content to a recipient of a call concurrently with the call, as well as other such methods described herein. In an embodiment, an incoming call from an enterprise may include: a call participant authentication screen 4802, which may further include an interactive item 4804 of content to facilitate at least partial authentication of the recipient of the call. Upon selecting a valid response in the interactive item 4804 of content, a second caller identification screen 4806 may be presented to the recipient with an option to talk to the caller 4808.

[0334] In an embodiment, the authentication system can use the content of the SCM data structure generated by the user to authenticate the user for current or future actions. Such an authentication system can affect user authentication by using a processor to access an instance of a sender-controlled media (SCM) data structure, which optionally uses encoding and other techniques to protect the first contact participant in the digital contact / communication between the first contact participant and the second contact participant. In an example, a user who is in contact with a supplier may have created an SCM data structure that contains information useful for the supplier to authenticate the user, such as a password that the person using the user's phone may not know, etc. The communication may require the user to provide a response to the SCM data structure content presented in the user interface portion of the digital communication. Based on the response, the real-time content captured by the interface can be used to update the context of the digital communication (e.g., a real-time captured image of the face of the phone user). In an embodiment, the algorithm can attempt to process the updated digital communication context, for example, by comparing the real-time content with another part of the secure SCM data structure to verify the first contact participant. In an example, the real-time captured image of the face of the phone user can be matched with the stored image of the verified user face. Thus, based on the result of the verification, a digital signal is sent to the second contact participant that the first contact participant is or is not an authenticated participant. In an example, content communication may occur via an SCM data structure application or function running on one or both of the user's phone (or computer, laptop, tablet device, etc.) and other participant computing systems. Optionally, an algorithm for securely managing participant access to SCM data structure content may be executed on a third server that controls access to the SCM data structure.

[0335] In an embodiment, the real-time content is an image captured through a camera interface of a digital device; the image may be the face of a user near the digital device. In an embodiment, the real-time content may include: audio captured through a microphone interface of a digital device, which may produce an audio recording of a user near the digital device. Additionally, the audio may be a voice recording of a word spoken into the microphone of the digital device. Verification may be based on a comparison of a spoken word with a copy of the spoken word identified in an SCM data structure. Verification may be based on comparing audio files, transcriptions of audio files, and combinations thereof. Alternatives to the content captured in real time may include: biometric indicators captured through a biometric interface of a digital device, such as fingerprints, retinal scans, multi-biometric factor measurements, and the like.

[0336] In an embodiment, after not receiving a response to a text message or phone call, a user can generate and submit to the platform for delivery of a video follow-up, such as a video message. A video message can be delivered, not just a voicemail or text message. In an example, the follow-up can be configured as content in an SCM data structure, and can be a video "voicemail", follow-up details (e.g., notes of the call, a transcription of the call, a URL, a document link, etc.), location information, call history, etc. Delivery of follow-up content can be implemented through the SCM platform methods and systems described herein or through a third-party communication channel.

[0337] refer to Fig.49 , depicting a screen for generating and viewing a visual / video voicemail. In response to the recipient not answering the call, a visual voicemail screen 4902 may be presented to the caller on the screen of the call originating device of the caller. Screen 4902 may include options for recording a voicemail, recording a video, capturing a new image, selecting an existing image, typing a text message, and combinations thereof, and ending the call without leaving any message. In an embodiment, the recorded video may be transmitted to the voicemail processing function of the recipient, such as a voicemail system that supports playback of an audio recording left by the caller. In an embodiment, the recorded video may be accessed via an SCM data structure created or modified based on the recorded video. In addition to or in lieu of an audio message, a link to the corresponding SCM data structure may be forwarded to the recipient. When the recipient's call message processing function encounters an indication of an SCM data structure left by the caller, a portion of the SCM data structure processing application may be executed on the recipient's phone, thereby causing the caller's visual / video message to be played back on the recipient's phone. Such playback is depicted in video voicemail playback screen 4904.

[0338] In an embodiment, upon receiving a no-answer indication from a recipient of a telephone call placed by a caller who may be a participant in the SCM data structure platform, a portion of the platform (such as a call interface of the caller's phone) may activate an SCM data structure content selection / generation interface on the caller's phone, through which the caller may cause generated content to be used in a follow-up message to the recipient. The generated content may then be delivered to the recipient's phone and / or the recipient's message storage queue (e.g., a voicemail queue). Upon retrieval from the phone message storage queue, the SCM data structure platform interface executed on the recipient's phone (optionally in conjunction with a companion function executed on a server) presents / plays back the content through a user interface of the electronic device, through which the recipient accesses the phone message storage queue.

[0339] A no-answer method and system for generating a video message may include receiving an indication that a recipient of a call placed from a caller to a recipient has no answer. In response to the indication, a sender controlled media (SCM) content selection / generation interface may be activated on the caller's phone so that the caller can generate content items such as video recordings, which can then be delivered to the recipient. In an embodiment, the SCM content selection / generation interface facilitates capturing content using a camera function of the caller's phone. The content may include at least one of a still image and a video. In an embodiment, the generated content item is an image captured by the camera function of the caller's phone during use of the SCM content selection / generation interface.

[0340] In an embodiment, generating video content to a caller in response to the caller receiving an indication of no answer from the recipient of the call may include: activating a sender controlled media (SCM) content selection / generation interface on the caller's phone, through which the SCM data structure to be generated includes the content item selected by the caller. Delivering the generated content item may include: delivering a link to the SCM data structure and at least one of the SCM data structures of the call message stor...

Claims

1. A method comprising: receiving an indication of initiation of a call from a sender to a recipient; determining a call context including at least one of a day, date, and time of day for the call; generating an SCM data structure by adapting a portion of the SCM data structure based on the determined call context; as well as A link to the generated SCM data structure is used to configure the caller ID for the call.

2. The method according to claim 1, wherein: The SCM data structure indicates the image that is presented on the electronic display of the call receiving device when a call is received.

3. The method according to claim 2, wherein: The image is an image captured by a camera function of the call initiating device, which is captured simultaneously with the initiation of the call.

4. The method according to claim 1, wherein: The SCM data structure indicates the video presented on the electronic display of the call receiving device when a call is received.

5. The method according to claim 1, wherein: The SCM data structure indicates the multimedia objects that are presented on the electronic display of the call receiving device when a call is received.

6. The method according to claim 1, wherein: The SCM data structure includes the caller's profile.

7. The method according to claim 1, wherein: The SCM data structure includes promotional content.

8. The method according to claim 1, wherein: The linked SCM data structure includes at least one of an image and a video captured concurrently with the initiation of the call.

9. A method comprising: configuring, using a visual caller ID interface executed on a call originating device, a visual caller ID for a call initiated by a user of the call originating device, the visual caller ID including the temporary visual content item; as well as Upon receiving an indication of a user initiating a call, at least one of the temporary visual content item and the permanent visual content item is provided as a visual caller ID to the recipient of the call based on a relationship between the current time of the temporary visual content item and the time-limited use parameter.

10. The method according to claim 9, wherein: Configuring the visual caller ID includes configuring an SCM data structure having at least one of a link to the temporary visual content item and the temporary visual content item.

11. The method according to claim 9, wherein: The relationship includes a time remaining for use of the temporary visual content item.

12. The method according to claim 9, wherein: The relationship includes: a usage status of the temporary visual content item.

13. The method according to claim 9, wherein: The time-limited use parameters include: an expiration date for the use of the temporary visual content item.

14. The method according to claim 9, wherein: The time-limited usage parameters include: a usage start date and a usage end date for usage of the temporary visual content item.

15. The method according to claim 9, wherein: The time-limited use parameters include: a calendar date, a pre-date use duration before the calendar date, and a post-date use duration after the calendar date for using the temporary visual content item.

16. The method according to claim 9, wherein: The time-limited-use parameters include: a link to an entry in the digital calendar of the user of the call originating device.

Images