Identifying users or user groups based on personality profiles

Abstract

This invention provides a method for providing a personality profile for a user or user group based on one or more media items associated with a single user or user group. [Solution] A personality profiling engine 10 determines the characteristics of a media item by generating a personality profile 61 or sentiment profile corresponding to the media item 21 being analyzed, such as a song. This enables a variety of new applications that allow for the classification, searching, recommendation, and targeting of media items or media users. For example, personality profiling or sentiment profiling can be used to recommend similar media items or to display advertisements that media users are likely to be interested in.

Description

Background Art

[0001] This application relates to the analysis of media content for determining a media profile and a personality profile from generated semantic descriptors of media items. The media profile and the personality profile can be used in many use cases, for example, for determining media users having a personality profile matching a target profile. The use cases can include a media recommendation engine, virtual reality, smart assistants, advertisements (target marketing), and computer games.

Summary of the Invention

[0002] In a broad aspect, the disclosure relates to the generation of a personality profile for a user or user group from one or more media items associated with a single user or user group. The media item can be any type of media content, particularly an audio or video clip. The audio media item preferably includes a piece of music or a music portion, preferably a piece of music. Photos, a series of photos, videos, slides, graphic representations are further examples of media items. The generated profile characterizes the personality or emotional situation of the consumer of the media item, i.e., the user who consumed the media item.

[0003] A method for providing a personality profile includes obtaining identification information for a group of media items, which includes one or more media items associated with a user or a user group. Media items may be identified, for example, by a list that references the storage location of the media item (e.g., a user or user group playlist, or a user's streaming history), by an enumeration of the media item's name or title (e.g., artist, album, song), or by a unique identifier (e.g., ISRC, MD5 sum, audio identification fingerprint, etc.). The storage location of the corresponding audio / video file may be determined by a table lookup or search procedure.

[0004] Next, a set of media content descriptors is obtained for each of the one or more identified media items in the group. A set of media content descriptors for a media item (also referred to as the media profile of the media item, or the song profile in the case of a song media item) includes many media content descriptors (also referred to as characteristics) that characterize the media item in terms of different aspects. The set of media content descriptors includes, optionally, other descriptors, a semantic descriptor of the media item. A semantic descriptor describes the content of the media item at a high level, such as the genre to which the media item belongs. In this sense, it is possible to classify a media item into one of many semantic classes and indicate with high probability which semantic class the media item belongs to. For example, a semantic descriptor may be represented as a binary value (0 or 1) indicating the class affiliation of the media item, or as a real number indicating the probability that the media belongs to a semantic class. A semantic descriptor may also be an emotional descriptor indicating that the media item corresponds to an emotional aspect such as atmosphere. A sentiment descriptor can classify a media item into one or more sentiment classes, indicating with high probability which sentiment class the media item belongs to. A sentiment descriptor can be represented as a binary value (0 or 1) indicating the class the media item belongs to, or as a real number indicating the probability that the media belongs to a sentiment class.

[0005] Media content descriptors may be calculated from identified media items, or they may be read from a database containing pre-analyzed media content descriptors for multiple media items. Thus, the step of obtaining a set of media content descriptors for each of the one or more identified media items may include reading a set of media content descriptors for the media items from the database. Some media content descriptors have a numerical value that quantifies the degree of each semantic descriptor and / or sentiment descriptor present for the media item. For example, a numerical media content descriptor may be normalized to have a value between 0 and 1 or between 0% and 100%.

[0006] Based on each media content descriptor of an individual media item, a set of aggregated media content descriptors is determined for the group of identified one or more media items. The aggregated media content descriptors characterize the semantic descriptors and / or sentiment descriptors of the media items in the group. A set of aggregated media content descriptors that include mood and are associated with a user or user group may also be referred to as the sentiment profile of the user or user group. Aggregated media content descriptors can be calculated, in particular for media content descriptors with numerical values, by averaging the values ​​of the individual media content descriptors of the media items. Note that methods other than a simple average of the values ​​of individual media content descriptors are possible. For example, root mean square (RMS) or other methods (e.g., "log-mean-exponential averaging") that emphasize larger values ​​in the set may be applied. Therefore, the step of determining a set of aggregated media content descriptors may include calculating an aggregated numerical content descriptor from each of the numerical content descriptors of the identified media items in the group.

[0007] A set of aggregated media content descriptors (i.e., the user's sentiment profile) is mapped to the personality profile of a user or user group associated with a group of media items. The personality profile has multiple personality scores for its elements. These personality scores are calculated from the aggregated properties of a set of aggregated media content descriptors (e.g., the user's or user group's sentiment profile). Typically, a personality profile is based on a personality scheme (value pairs representing personality traits) that defines many profile elements, including attributes. The values ​​of the profile elements are also referred to as profile scores. Examples of personality schemes include the Myers-Briggs personality type index (MBTI), the Ego Equilibrium, the Big Five personality traits (Openness, Conscientiousness, Extraversion, Agreeableness, Neuroticism (OCEAN)), or the Enneagram. Other schemes are possible for defining personality profile elements.

[0008] The identified media items are relevant to the user's emotional / psychological context and may enable the determination of the user's personality profile. If the identification information for one or more media items includes the user's short-term media consumption history (e.g., recently listened to music), the generated personality profile will characterize the user's current or recent mood. If the identification information for one or more media items includes playlists that identify the user's long-term media item usage history, the generated personality profile will characterize the user's long-term personality profile. In some embodiments, particularly in advertising and branding use cases, it is also possible to consider a blend of the long-term and short-term personality profiles (based on the mood of recently listened-to music) as the user's relevant personality profile.

[0009] The generated personality profile may be classified into one of several personality types corresponding to a personality scheme, for example. This classification may be based on a profile score compared to a threshold. Other classification schemes may be used, such as determining the highest score. Depending on the result of the comparison, a personality type may be assigned to the profile and, consequently, to the user. For example, a personality profile (e.g., MBTI) has multiple numerical values ​​(scores) that represent a personality type as a whole. To make a decision, it is possible to determine a "single personality type" by determining the "maximum personality attribute" from such a profile. Both methods allow for the psychological characterization of the user, but the former is more granular, while the latter determines a specific personality type.

[0010] This method further includes comparing the personality profiles of multiple users or user groups to a target personality profile and determining at least one user or user group with the best-matching personality profile with respect to the target profile. The identified best-matching user or user group is then selected for further activities related to the target profile, such as receiving information associated with the target profile. If the target personality profile corresponds to a brand or product, it becomes possible to select a user or user group that best matches the product or brand in terms of their personality or emotions.

[0011] The results of the decision step may be displayed on a computer device or transmitted to a database server. For example, the identification information of at least one determined user or user group is transmitted to the database server. The identification information of at least one determined user or user group may be used in many use cases, such as determining media users with personality profiles that match a target profile (e.g., media recommendation engines, smart assistants, smart homes, advertising, product targeting, marketing, virtual reality, and games).

[0012] A set of media content descriptors for a media item may further include one or more acoustic descriptors for the media item. The acoustic descriptors (also referred to as acoustic attributes) of a media item may be determined based on an acoustic digital audio analysis of the media item's content. For example, the acoustic analysis may be based on a spectrogram derived for the audio content of the media item. Various techniques can be employed to obtain acoustic descriptors from an audio signal. Examples of acoustic descriptors include tempo (beats per minute), duration, key, mode, presence or absence of rhythm, and energy (of the spectrum).

[0013] A set of media content descriptors for a media item may be determined at least in part on one or more artificial intelligence models that determine one or more emotional descriptors and / or one or more semantic descriptors for the media item. One or more semantic descriptors may include at least one of the following: genre, or vocal attributes such as the presence or absence of vocals, and the gender of the vocals (low-pitched or high-pitched, respectively). Examples of emotional descriptors include the mood of the music and the mood of the rhythm. The artificial intelligence models may be based on machine learning techniques such as deep learning (deep neural networks). For example, an artificial neural network may be used to determine the emotional and semantic descriptors for a media item. The neural network may be trained on a broad set of data provided by music experts and data science experts. It is also possible to use an artificial intelligence model or machine learning technique (e.g., a neural network) to determine the acoustic descriptor (such as bpm or key) of a media item.

[0014] The media item may be segmented, and based on the results of the analysis of individual segments, a set of media content descriptors for the media item may be determined. For example, a media item may be segmented into media item parts, acoustic analysis and / or artificial intelligence techniques may be applied to the individual parts, and the acoustic descriptors and / or semantic descriptors generated for these parts may then be aggregated to form an acoustic descriptor and / or semantic descriptor for the complete media item, similar to how the media content descriptors for a media item are aggregated for the entire group of media items.

[0015] The personality score of a personality profile (i.e., attribute values ​​(value pairs of profile elements)) may be determined based on mapping rules that define how to calculate a personality score from a set of aggregated media content descriptors. The mapping rules may define one of the set of aggregated media content descriptors that contribute to the personality score and how it contributes. For example, the personality score of a personality profile may be determined based on a weighted aggregated numerical media content descriptor for an identified media item. Based on the weighting, different content descriptors may contribute to the score to different degrees. Furthermore, the personality score of a personality profile may be determined based on the presence or absence of aggregated media content descriptors for an identified media item. In other words, if aggregated media content descriptors exist, their contribution to the score may be made, for example, by weighting normalized numerical aggregated content descriptors. Alternatively, the contribution to the score when a collective media content descriptor is not expected to exist can be represented by weighting the difference between 1 and a normalized numerical collective content descriptor value (which has a value between 0 and 1).

[0016] Mapping rules may be learned using machine learning techniques. For example, the weights that aggregated numerical content descriptors contribute to a score may be determined by machine learning using a number of target profiles (real-world user profiles) and a suitable machine learning technique capable of determining rules and / or weights for mapping content descriptors to personality profiles and vice versa. Such machine learning techniques may also determine which content descriptors can contribute to a profile score and select each content descriptor, and vice versa.

[0017] In one embodiment, the user's (long-term) personality profile is determined from a playlist that identifies the user's (longer) media item usage history. In another embodiment, the user's (short-term) mood profile is determined from the user's short-term media consumption history.

[0018] A separate personality profile is provided for each of the multiple users or user groups. Therefore, each user or user group is characterized in terms of emotions and personality by its respective personality profile. Furthermore, the target personality profile may correspond to the user's target group or product or brand profile. Thus, the target user group or product or brand is also characterized in terms of emotions and personality by its personality profile.

[0019] For example, a target personality profile may be generated from a product or brand profile by mapping elements of the product or brand profile to the personality score of the target profile. The score of the target personality profile may be determined based on mapping rules that define how to calculate the score from the elements of the product or brand profile. The mapping rules may be learned by machine learning techniques similar to those described above with respect to aggregated media content descriptors that contribute to the personality score and the mapping rules that define how they appear.

[0020] The search for one or more best-matching personality profiles may be based on comparing the personality profile of a user or user group with a target personality profile. For example, comparing profiles may be based on matching profile elements and selecting personality profiles of media items that have identical or similar elements to the target personality profile. Furthermore, comparing profiles may be based on similarity search, where corresponding scores of profile elements are compared and a matching score value indicating the similarity of each pair of profiles is calculated. The matching score for a pair of profiles may be based on the individual matching scores of the corresponding attribute values ​​(scores) of the profile elements. For example, the difference between corresponding values ​​(scores) of profile elements may be calculated (e.g., Euclidean distance, Manhattan distance, cosine distance, etc.), from which a matching score for the profile pair being compared can be calculated. Multiple best-matching personality profiles may be determined, and the personality profiles of users or user groups may be ranked according to their matching scores. This makes it possible to determine the best-matching user or user group, the second-best match, etc.

[0021] Comparing profiles may further depend on the context or environment of each user or user group. Examples of context or environment include the user's location, date and time, weather, and other people nearby. Similar contexts or examples may be adopted for user groups.

[0022] Further media items corresponding to the target personality profile may be selected for presentation to at least one determined user or user group. These media items may be audio or video clips related to the target personality profile, particularly to the corresponding product or brand. The media items may also provide information about the product or brand, for example, in the form of advertisements for the product or brand.

[0023] An electronic message may be automatically generated for at least one designated user or user group, and the generated message may be electronically transmitted to the user or user group. The electronic message may include information about a product or brand associated with the target personality profile, and for example, the electronic message may include additional media items selected in accordance with the target profile.

[0024] The one or more identified media items used to determine the user's personality profile may correspond to the user's most recently consumed media items, and the user's personality profile may characterize the user's current mood. Comparing the user's personality profile to a target personality profile may be performed in real time, based on the most recently determined user personality profile.

[0025] Comparing a user's or user group's personality profile to a target personality profile and determining at least one user or user group with the best-matching personality profile may be performed repeatedly, for example, after a determined period or after many media items have been presented to the user (group), and this comparison may be based on the most recently determined user personality profile. In this way, the determination of a user's personality profile and the best-matching user (group) can be updated periodically (for example, in real time after media items have been presented to the user). This enables an adaptive media presentation service in which new media items corresponding to the target profile are presented to the user based on the media items the user has consumed in the past.

[0026] Personality profiles may be generated on the server platform. This method may further include transmitting identification information of one or more identified media items associated with the user from the user device associated with the user to the server platform. Therefore, the server can receive information about the user's media consumption (e.g., playlists) and determine the user's personality profile from this information. As mentioned above, this may be repeated. The user device may be any user device, such as a personal computer, tablet computer, mobile computer, smartphone, wearable device, smart speaker, smart home environment, car radio, etc., or any combination thereof. After determining the best-matching user by comparing the user's personality profile with a target personality profile, the server can send selected media items corresponding to the target profile to the user device. This information is received and presented to the user, or the selected media items are played.

[0027] The identification information of one or more media items (e.g., playlists) for a user may be stored in a server platform, and a user's personality profile is generated on the server platform. After determining the best-matching user (group) by comparing the personality profile of the user (group) with a target personality profile, the server can send the representation of the identified media items corresponding to the target profile to the user device associated with the user. This information is either received and presented to the user or the identified media items are played.

[0028] In another aspect of the present disclosure, a computer device for executing any of the above methods is proposed. This computer device may be a server computer having a memory for storing instructions and a processor for executing the instructions. The computer device may further include a network interface for communicating with the user device. The computer device may receive information regarding media items consumed by the user from the user device. The computer device may be configured to generate a personality profile as described in the above disclosure. Depending on the use case, the personality profile may be used for recommending similar media items and determining media users having a personality profile matching the target profile. Information regarding further identified media items corresponding to the target profile may be sent to the user device.

[0029] Embodiments of the disclosed device may include, but are not limited to, the use of one or more processors, one or more application specific integrated circuits (ASICs), and / or one or more field programmable gate arrays (FPGAs). Embodiments of the device may also include the use of other conventional hardware and / or customized hardware, such as a software programmable processor, such as a graphics processing unit (GPU) processor.

[0030] Another aspect of the present disclosure may relate to computer software, a computer program product, or any medium or data embodying computer software instructions for causing at least one processor to execute any of the method steps described in the present disclosure by executing on a programmable computer or dedicated hardware having at least one processor.

[0031] Referring particularly to the above applications, several exemplary embodiments are described herein, but it will be understood that the present disclosure is not limited to such fields of use and is applicable in a broader context.

[0032] In particular, the method according to the present disclosure relates to a method of operating a device according to the above exemplary embodiments and variations thereof, and each description of the device similarly applies to the corresponding method, and vice versa, and it should be understood that similar descriptions may be omitted for the sake of simplicity. Also, the above aspects can be combined in many ways even if not explicitly disclosed. Those skilled in the art will understand that these aspects and combinations of features / steps are possible as long as no contradiction is explicitly excluded.

[0033] In the course of the following discussion, other and further exemplary embodiments of the present disclosure will become apparent by reference to the accompanying drawings. [[ID=十七]]

Brief Description of the Drawings

[0034] The following describes exemplary embodiments of this disclosure with reference to the attached drawings, but these are only a few examples. [Figure 1] This figure schematically illustrates the operation of one embodiment of the present disclosure. [Figure 2a] This diagram illustrates the generation of semantic descriptors from audio files. [Figure 2b] This figure illustrates the generation of semantic descriptors by the audio content analysis unit. [Figure 3a] This diagram shows the mapping of atmosphere content descriptors to the EI (Extroversion-Introversion) personality score in the MBTI personality scheme. [Figure 3b] This figure shows the mapping of atmosphere content descriptors to the openness personality score in the OCEAN personality scheme. [Figure 4a] This figure shows an example of a graphical representation of a personality profile in the MBTI Personality Scheme. [Figure 4b] This figure shows an example of a graphic representation of a personality profile in the OCEAN Personality Scheme. [Figure 5] This figure illustrates one embodiment of a method for determining a user or user group. [Figure 6] This diagram shows the mapping from product profile attributes to atmosphere content descriptors. [Modes for carrying out the invention]

[0035] In one broad aspect of this disclosure, a personality profiling engine generates personality or sentiment profiles corresponding to media items being analyzed, such as music, thereby determining the characteristics of a media item. This enables a variety of new applications (also referred to in this disclosure as “Use Cases”) that allow for the classification, searching, recommendation, and targeting of media items or media users. For example, personality profiling or sentiment profiling may be used to recommend similar media items or to display advertisements that media users may be interested in.

[0036] For example, if the input to a personality profiling engine is a user's short-term music listening history, it can determine a personality profile that characterizes the listener's mood based on the songs the user has recently played. If the input is a long-term music listening history, it can determine the listener's general personality profile. It is also possible to calculate the difference between the user's long-term personality profile and their current mood to determine if the user is in an exceptional situation.

[0037] Personality profiles generated by a personality profiling engine can detect emotional signatures of music listeners, focusing on the atmosphere, mood, and values ​​that define a person's multifaceted personality. This allows for addressing questions such as whether a listener is self-conscious or haughty, or whether they enjoy exercise or travel.

[0038] In the case of audio, for example, it is possible to find similar-sounding music tracks based on the emotional descriptor and / or semantic descriptor of an audio file. A media similarity engine using the generated emotional profile may utilize machine learning or artificial intelligence (AI) to match and find musically and / or emotionally similar tracks. Such a media similarity engine can listen to and understand music like a human, then search millions of music tracks for specific sonic or emotional patterns, finding the required music in seconds by matching requirements. Based on the generated profile, it is possible to search for, for example, only instrumental or vocal tracks, or according to other semantic criteria such as genre, tempo, mood, or bass vs. treble.

[0039] The basis of the proposed technology is a personality profiling engine that performs tagging of media items using media content descriptors based on audio analysis and / or artificial intelligence (e.g., deep learning algorithms, neural networks, etc.). The personality profiling engine may tag media tracks by weighted mood, emotion, and musical attributes such as genre, key, and tempo (beats per minute (bpm)) by enriching metadata using AI. The personality profiling engine may analyze the mood, genre, acoustic attributes, and contextual situation of a media item (e.g., a musical track (song)) and obtain weighted values ​​for different "tags" within these categories. The personality profiling engine may analyze a media catalog and tag each media item in the catalog with corresponding metadata. Media items are, for example, • Acoustic attributes (bpm, key, energy, etc.) • Atmosphere / rhythm atmosphere, Genre, • Vocal attributes (instrumental, high notes, low notes), Contextual situation, It may also be tagged with a media content descriptor related to it.

[0040] In the mood category for tagging music from an "emotional" perspective, the personality profiling engine may output up to 35 "complex mood" values ​​that are taxonomically classifiable within, for example, 18 mood subfamilies organized as 6 main families. All 6 main families and 18 subfamilies include human emotions. The level of detail in the mood taxonomy can be arbitrarily narrowed down; that is, the 35 "complex moods" can be further subdivided as needed, or additional "complex moods" can be added.

[0041] Figure 1 schematically illustrates the operation of one embodiment of the present disclosure, which generates personality profiles and determines the similarity of profiles to provide various recommendations for similar media items, or to match users or user groups. The personality profiling engine 10 receives one or more media files 21 from the media database 20. Media files are identified in the media list 30 provided to the personality profiling engine 10 in order to read media items from the database 20. The media list 30 may be the user's playlist read from a playlist database that stores the most recent media items played by the user and user-defined playlists representing the user's media preferences.

[0042] Analysis of the media file 21 determines media content descriptors 43, including the acoustic descriptor, semantic descriptor, and / or emotional descriptor of the audio content. An audio content analysis unit 40, equipped with an acoustic analysis unit 41 that analyzes the acoustic features of the audio content, determines several media content descriptors 43 by analyzing the time-frequency plane in a way that generates a frequency-domain representation, such as a spectrogram, of the audio content and calculates acoustic features such as tempo (bpm) or key. The spectrogram may be transformed according to perspective and / or logarithmic scales, for example, in the form of a Log-Mel spectrogram. The media content descriptors may be stored in a media content descriptor database 44.

[0043] The audio content analysis unit 40 of the personality profiling engine 10 further comprises an artificial intelligence unit 42 that uses an artificial intelligence model to determine media content descriptors 43, such as emotion descriptors and / or semantic descriptors, of the audio content. The artificial intelligence unit 42 may operate on any appropriate representation of the audio content, such as a time-domain representation generated by the acoustic analysis unit 41, a frequency-domain representation of the audio content (e.g., a Log-Mel spectrogram as described above), or intermediate characteristics derived from the audio waveform and / or frequency-domain representation. The artificial intelligence unit 42 may generate, for example, an atmosphere descriptor of the audio content that characterizes the mood of the music and / or rhythm of the audio content. These AI models may be trained on their own large-scale expert data.

[0044] Figure 2a shows an example of semantic descriptor generation from an audio file by the audio content analysis unit. In this embodiment, audio file samples are optionally segmented into audio chunks and converted into frequency representations such as Log-Mel spectrograms. The audio content analysis unit 40 then extracts low-level, mid-level, and / or high-level semantic descriptors from the spectrograms by applying various audio analysis techniques.

[0045] Figure 2b further illustrates an example of semantic descriptor generation by the audio content analysis unit 40. While Figure 2a shows direct audio content analysis using conventional signal processing methods, Figure 2b shows audio content analysis using a neural network, which first requires learning from "ground truth" data ("prior knowledge"). The audio file is converted into a spectrogram, and media content descriptors 43, such as the atmosphere, genre, and context of the audio file, are generated by applying one or more neural networks. The neural networks are trained for this task based on large-scale expert data (large-scale and detailed "ground truth" media annotations for supervised neural network training). In an example of semantic descriptor generation by the artificial intelligence unit 42, spectrogram data of the audio file is fed to the neural network as input, which generates semantic descriptors as output. In embodiments, one or more convolutional neural networks are used to generate descriptors for, for example, genre, rhythm atmosphere, and speech family. Other network configurations and combinations of networks can be used similarly.

[0046] The mapping unit 50 maps media content descriptors 43 for audio files to media personality profiles 61 by applying mapping rules 51 received from the mapping rule database 52. The mapping rules 51 may define the media content descriptors used in calculating the profile score (i.e., the values ​​of the profile attributes) and the weights applied to the media content descriptors. The mapping rules 51 may also be represented as a matrix that links media content descriptors and profile attributes to assign weights to the media content descriptors. The generated personality profiles 61 may be provided to the media similarity engine 70 to determine similar profiles, or they may be stored in the profile database 60 for later use.

[0047] When a personality profile is generated for a media item group, media content descriptors 43 are generated for each media item in that group (or read from the media content descriptor database 44), and a collective media content descriptor is generated for the entire group of media items. The collective nature of the numerical media content descriptors can be achieved by calculating the average value of each media content descriptor for the group of media items. Other collective nature algorithms, such as the root mean square (RMS), can also be used. The mapping unit 50 then acts on the collective media content descriptor (e.g., sentiment profile) to generate a personality profile for the entire group of media items.

[0048] The media similarity engine 70 can receive profiles directly from the personality profiling engine 10, or from the profile database 60, as shown in Figure 1. The media similarity engine 70 determines profile similarity based on profile element matching or similarity search, as disclosed below, through profile comparison. If a profile 71 similar to the target profile is determined, corresponding media items or users may be determined, and recommendations may be made. For example, one or more media items matching the user's playlist may be determined and automatically played on the user's terminal device. Other use cases are described in this disclosure.

[0049] As mentioned above, the personality profiling engine can use machine learning or deep learning techniques to determine the emotional and semantic descriptors of media items. Training may be based on a database consisting of many data points to learn relationships and analyze an individual's music preferences and listening habits. This algorithm can read the user's psychological and emotional portrait and complement existing demographic and behavioral statistics to generate a complete and evolutionary user profile. The output of the personality profiling engine is a profile of the psychologically motivated user ("personality profile") derived from an analysis of the user's music (playlists or listening history).

[0050] A personality profiling engine can derive a user's personality profile from fewer media items or from more media items. For example, based on the most recent 10 or more music items played by the user on a streaming service, the engine can calculate the user's short-term ("instantaneous") profile (reflecting the "current mood of the music listener"). If (more) music items represent the user's long-term listening history or preferred playlists, the engine can calculate the user's intrinsic personality profile.

[0051] The personality profiling engine uses advanced machine learning and deep learning techniques to understand the meaningful content of music from audio signals, achieving a human-like level of comparison beyond simple text language and labels. By extracting essential information about music from audio signals, the algorithm can learn to understand the rhythm, beat, style, genre, and mood of the music. The generated profiles can be applied to music or video streaming services, digital or linear radio, advertising, product targeting, computer games, labels, libraries, publishers, in-store music providers or synchronization agencies, voice assistants / smart assistants, smart homes, and more.

[0052] The personality profiling engine can achieve a human-like level of comparison by applying advanced deep learning techniques to understand the meaningful content of music from audio. This algorithm can analyze and predict relevant mood, genre, context, and other key attributes and assign a weighted relevance score (%).

[0053] The media similarity engine is applicable to recommendation, music targeting, and audio branding tasks and can be used by music or video streaming services, digital or linear radio, fast-moving consumer goods (FMCG) (also known as consumer goods products (CPG)), advertisers, creative agencies, dating companies, in-store music providers, or e-commerce.

[0054] The personality engine may be configured to generate a personality profile based on a group of media items associated with a user by performing the following steps: In the first step, a group list is obtained containing identification information for one or more media items, for example, in the form of a user-defined playlist. Next, a set of media content descriptors is generated for each of the one or more identified media items in the group, or read from a database of previously analyzed media items. The set of media content descriptors includes at least one of the following for each media item: an acoustic descriptor, a semantic descriptor, and an emotional descriptor. This method then includes determining a set of aggregated media content descriptors (i.e., the user's emotional profile) for the entire group of the one or more identified media items, based on each media content descriptor of the individual media items. Finally, the aggregated set of media content descriptors is mapped to the personality profile for the group of media items. The score of the profile elements is calculated from the aggregate properties of the set of aggregated media content descriptors.

[0055] In an exemplary embodiment, the personality profiling engine is applied to determining the mood of a media user. For example, the mood of a music listener is determined based on the input "short-term music listening history." Alternatively, the general personality profile of a music listener is determined from the input "long-term music listening history." In further use cases, an individual's personality profile may be associated with the personality profiles of others to determine individuals with similar profiles at a particular moment (e.g., matching, recommendations for products with similar profiles (e.g., e-commerce), or suggestions for connecting with others (friends, dating, social networks, etc.)).

[0056] Furthermore, by using a personality profiling engine, media items such as music (e.g., current playlists and / or suggestions, or other forms of entertainment (movies, etc.), or environments such as smart homes) may be adapted to a) an individual's current mood, and / or b) an intention to change the individual's mood (intentions explicitly expressed by the individual, implicit intentions of change brought about by the system, e.g., product recommendations or optimization (increase) of user retention on the platform).

[0057] A personality profiling engine can be used to determine the difference between a user's current mood and their general personality by calculating the difference between a user's long-term personality profile and their current (mood) profile. This is useful, for example, for adapting recommendations in the case of a short-term "deviation" from the user's general personality profile that leans towards a particular musical direction (depending on a specific listening context, time of day, user mood, etc.), and for deciding whether to display ads that would normally fit the user's personality profile but are momentarily inappropriate because the current mood profile deviates from the current listening situation. In either case, the placement of recommendations or ads may be adapted to the user's personal situation at that moment.

[0058] The basis of these embodiments is a personality profiling engine that analyzes groups of media items identified by a provided list. For example, audio tracks in a group of songs (from digital audio files) are analyzed. This analysis may be performed by, for example, audio content analysis and / or the application of machine learning (e.g., deep learning) methods. The personality profiling engine is Algorithms may be applied to extract low-level, mid-level, and high-level characteristics from audio. Examples of low-level characteristics are audio waveform / spectrogram-related characteristics (or "descriptors"), examples of mid-level characteristics (or "descriptors") are "fluctuations," "energy," etc., and examples of high-level characteristics are semantic descriptors and emotional descriptors such as genre, mood, or key. Acoustic waveform and spectrogram analysis may be applied to analyze acoustic attributes such as tempo (beats per minute), key, mode, duration, spectral energy, and presence or absence of rhythm. Neural network / deep learning-based models may be applied to analyze high-level descriptors from audio input (e.g., via Log-Mel frequency spectrograms extracted from various segments of an audio track), such as genre, mood, rhythmic mood, presence of sound (instrumental or vocal), and vocal attributes (e.g., bass or treble). The neural network / deep learning models may be trained on a large training dataset containing thousands (hundreds) of annotated examples of the aforementioned categories tagged by music experts. For example, a deep learning convolutional neural network may be used, but alternatively, other types of neural networks (e.g., recurrent neural networks), other machine learning techniques, or any combination thereof may be used. In embodiments, one model is trained for each category group of mood, genre, rhythmic mood, and presence of sound / vocal attributes. In an alternative example, one common model may be trained collectively, or one model may be trained together for multiple moods and rhythmic moods, or one model may be trained for each individual mood or genre itself.

[0059] Audio analysis may be performed on multiple time points within the audio file (for example, three 15-second sections at the beginning, middle, and end of a song), or it may be performed on the entire audio file.

[0060] The output may be stored at the segment level or at the audio track (song) level (e.g., aggregated from segments). Subsequent steps may also be applied at the segment level (e.g., obtaining a list of moods (or mood scores) for each segment (e.g., applicable to long audio recordings such as classical, DJ mixes, or podcasts, or to audio tracks where genre or mood changes)). The personality profiling engine may store all derived song content descriptors, along with their predicted or percentage values, in one or more databases for further use (see below).

[0061] The output of the audio content analysis is: Tempo: For example, 135 bpm, • Key and mode: For example, F# minor, • Spectral energy: For example, 67% (100% is determined by the maximum value listed in the truck's catalog), • Presence of rhythm: For example, 55% (100% is determined by the maximum value in the track catalog), • Genre: As a list of categories (each independent of others with a percentage value from 0 to 100), for example, Pop 80%, New Wave 60%, Electropop 33%, Dance Pop 25%, • Atmosphere: A list of atmospheres included in the song (each with a percentage value from 0 to 100, independent of others), for example, dreamy 70%, intellectual 60%, inspiring 40%, bitter 16%, • Rhythmic atmosphere: A list of atmospheres included in the song (each with a percentage value from 0 to 100, independent of others), for example, elegant 67%, lyrical 53%, • Vocal attribute: Instrument (0 or 100%) or any combination of low and / or high frequencies (50-100%) These are media (e.g., music) content descriptors derived from input audio (also referred to as audio characteristics or music characteristics).

[0062] In one embodiment, audio content analysis is performed. From the audio characteristics extraction, 14 mid-to-high level characteristics + 52 low-level (spectral) characteristics were obtained, From the deep learning model, 67 genres, 35 moods (+24 (by grouping into subfamilies and families) (see below)), 5 rhythmic moods, and 3 vocal attributes were derived. Output each of these.

[0063] Optionally, subsequent post-processing of the values ​​is performed by applying so-called adjustment coefficients (e.g., assigning high or low weights to genre, mood, or some other category). The adjustment coefficients adapt the machine predictions to more closely resemble human perception. The adjustment coefficients may be determined by experts (e.g., music experts) or learned by machine learning. The adjustment coefficients may be defined by one coefficient per semantic descriptor or sentiment descriptor, or by a non-linear mapping from various machine predictions to adjusted output values.

[0064] Furthermore, optionally, the aggregation of music content descriptors may generate values ​​for groups or "families" of music content descriptors that typically follow a taxonomic framework. In one example, 35 atmospheres predicted by a deep learning model are aggregated into 18 parent "subfamilies" and 6 "main families," forming a total of 59 atmospheres (following a taxonomic framework for atmospheres).

[0065] The analysis may be performed at the song level for a set of songs and delivered in audio form (various digital formats, compressed or uncompressed). For the generation of a personality profile, for a group of songs (commonly referred to as a "playlist"), the song content descriptors and their values ​​for multiple songs may be aggregated.

[0066] In some embodiments (use cases), the listener's current mood is determined. In other use cases, the personality profiling engine determines the listener's long-term personality profile. In both cases, the input is a list of songs, and the output is the user's personality profile (according to one or more personality profiling schemes). To determine the listener's mood, the input consists of the most recently listened songs. These songs provide insight into the user's current mood profile. To determine the listener's general (long-term) personality profile, the input consists of songs (usually a larger set) representing the user's (long-term) history.

[0067] The generation of a personality profile may be based on the characteristics of the music the user listens to, including (not limited to) mood, genre, vocal presence, vocal attributes, key, BPM, energy, and other acoustic attributes (= "music content descriptor," "audio characteristics," or "music characteristics"). This may be determined for each characteristic of the song's musical content.

[0068] In one embodiment, the musical content descriptors of n songs are aggregated to generate an aggregated content descriptor, that is, the user's emotional profile is generated, for example, as the average of the numerical values ​​(%) of each song in the aggregate (playlist). Alternatively, more complex aggregation procedures such as median, geometric mean, RMS (root mean square), or various forms of weighted average may be applied.

[0069] In one embodiment, preliminary analysis of songs in the user's playlist or listening history may extract song content descriptors that may contain numerical values ​​(for example, ranging from 0 to 100% for each value). For each content descriptor (for example, mood "sensibility"), the root mean square (RMS) of all "sensibility" values ​​for individual songs may be calculated and stored. The output of this aggregation is a set of song content descriptors having the same number of descriptors (attributes) as each song. This aggregated song content descriptor (sensibility profile) is used in the second stage of the personality profiling engine to determine the user's personality profile.

[0070] When the aggregated value of each song content descriptor is calculated, a personality profile is generated. For example, a mapping is performed from elements in an emotional profile (representing aggregated song content descriptors for n songs) to one or more personality profiles. In the mapping, mood, genre, style, etc., are converted into psychological and emotional user characteristics (personality traits). The mapping is performed from the song content descriptors to a score of a personality profile (including personality traits / human characteristics). Rules for mapping song content descriptors and their values ​​to one or more types of personality profiles defined by the personality profile scheme may be defined.

[0071] The output of the personality profiling engine is a range of numerical output parameters, referred to as personality profile attributes and scores, which represent the user's personality profile.

[0072] Personality profiles are, MBTI (Myers-Briggs Index), ·Ego Equilibrium, • OCEAN (also known as one of the Big Five personality traits) • Enneagram, These may be defined according to various personality profiling schemes.

[0073] Each of these personality profile schemes is composed of personality attributes (for example, "extraversion" or "openness" and assigned scores (values) such as 51% or 88% (specific examples are shown below)).

[0074] For all of these schemes, a mapping from musical content descriptors to profile scores may be used, and vice versa. Figure 3a shows the mapping of atmosphere content descriptors to EI personality scores in the MBTI personality scheme. The mapping may apply a matrix such as the example shown in Figure 3a. The calculation of scores (values) within the personality profile scheme may involve the presence (%) or absence (100%) of (atmosphere or other musical content descriptors).

[0075] Each scheme may have many "scores" to calculate (for example, the MBTI scheme calculates four scores: EI, SN, TF, and JP). One or more mapping rules may be defined for each score, which affect how the score is calculated from the aggregated music content descriptor. For example, the score is equal to the sum of the values ​​calculated by the matrix divided by the number of values ​​to consider (i.e., the usual averaging mechanism).

[0076] For example, the "regressive" atmosphere (included in the music content descriptor) is used in the EI calculation as part of the MBTI scheme. Figure 3a shows an example of a rule matrix applied to the EI calculation from the atmosphere portion of the music content descriptor. The rule matrix shows how the presence or absence of atmosphere can be used in the calculation of the EI score. Other music content descriptors can be similarly included in the calculation.

[0077] In one embodiment, the EI calculation includes 17 rules incorporating 17 values ​​from the music content descriptor. These rules follow a psychological recipe. For example, the rules in the "metal" group psychologically define "closed shoulder," while the rules in the "tree" group define "open shoulder."

[0078] Similar calculations can be performed on other profiling matrices such as OCEAN.

[0079] As mentioned above, the MBTI personality profile has scores for EI, TF, JP, and SN. The following is an example of how the MBTI personality profile and its scores are expressed. “mbti”:{“name”:“INTJ”,“sources”:{ “EI”:33.66403316629877, “SN”:42.419498057065084, “TF”:57.82423612828757, "JP":61.02633025243475}}

[0080] A basic score classification may be performed based on the score value. This classification may be based on a comparison of the score value with a specific threshold. For example, the EI score in the MBTI scheme represents the balance between a user's extraversion (E) and introversion (I). An EI below 50% indicates introversion, while an EI above 50% indicates extraversion. Therefore, if EI < 50%, the user may be assigned to the I (introverted) class, and otherwise, to the E (extroverted) class. Other MBTI scores can be classified similarly.

[0081] The scores are defined as opposites on each axis (EI, SN, TF, JP). For each character pair, the value that determines which trait a person possesses is either <50% or >50%. To exclude characters from the above example, typically, the right-hand character of the character pair is taken if the value is <50%, and the left-hand character is taken if the value is ≥50%.

[0082] The scores for the generated profiles may be further classified into general personality types, for example, based on the basic classification results of the profile scores. For example, the following general personality types may be derived from the basic score classification results. ·ESTJ: Extraversion (E), Sensitivity (S), Thinking (T), Judgment (J) INFP: Introversion (I), Intuition (N), Mood (F), Perception (P)

[0083] The profile in the example above is classified as an INTJ personality type. By classifying the four-dimensional space of profile scores (EI, TF, JP, SN) into personality types, it becomes possible to arrange personality traits in two dimensions within a square that has significant representation.

[0084] Figure 4a shows a schematic representation of a personality profile related to the MBTI scheme, and the classification result (INTJ) for the user's profile can be shown in color. This figure intuitively represents the user's profile along various psychological dimensions. An individual classified as "INTJ" is interpreted as a "planner, scientist." Additional personality traits associated with this MBTI type may be output on the user interface.

[0085] In the OCEAN Personality Profile Scheme, scores are assigned to the "Big Five" mindset, including openness, conscientiousness, extraversion, agreeableness, and neuroticism. Figure 3b shows the mapping of the Atmosphere Content Descriptor to the Openness Personality Score in the OCEAN Personality Scheme. The following is an example of how the OCEAN Personality Profile and its scores are represented. “ocean”:{ “agreeableness”:51.10149671582637, “conscientiousness”:73.42223321884429, “extraversion”:33.66403316629877, “neuroticism”:50.21693055551433, “openness”:39.72017677623826}

[0086] Figure 4b shows a schematic representation of the personality profile related to the OCEAN scheme. This figure intuitively represents the user's profile along various psychological dimensions.

[0087] In some embodiments, the personality profile can optionally be enriched with or associated with additional person-related parameters characterized by additional information (e.g., age, gender, and / or biosignals of the human body via body sensors (smartwatches, sports tracking devices, emotion sensors, etc.)). Alternatively, the personality profile can also optionally be enriched with or associated with additional parameters characterizing the individual's context and environment (location, date and time, weather, other people nearby).

[0088] In this embodiment, the personality profiling engine and the media similarity engine are configured to determine a user or group of users for a specific target personality profile and to select the user (user group) that best matches the target profile. The personality profiling engine may analyze one or more media items associated with the user or user group for their content in terms of acoustic attributes, genre, style, atmosphere, etc. It then generates a description of the user or user group (in the form of a personality profile). After determining a personality profile for each of the multiple users or user groups, the personality profiling engine compares the personality profiles of the multiple users or user groups with the target personality profile and determines at least one user or user group that has the best matching personality profile for the target profile. The target profile may be specified by a personality profile following a personality profile scheme such as MBTI, OCEAN, Enneagram, or Ego-Equilibrium, similar to the definition of a user profile. The profile may optionally be enriched with person-related parameters (e.g., age, gender, etc.).

[0089] More specifically, a song content descriptor, including a semantic descriptor and / or emotional descriptor, is derived through an analysis of the audio in a set of songs associated with the user. Optionally, the descriptors are aggregated (using various methods) for many tracks (which may represent an album or artist), and the user's emotional profile is determined, for example, by calculating the average of the mood and / or other descriptors of multiple songs (possibly using an average, RMS, or weighted average). Then, as described above, a mapping from the song content descriptor to the personality profile is performed. The system then outputs and stores profiles for multiple users, defined by one of several different personality profile schemes. The profiles may be provided in numerical format (e.g., floating-point numbers for different profile scores within the aforementioned schemes).

[0090] The personality profile of a user or user group is generated as disclosed above and identified by one or more personality profiles following schemes such as MBTI, OCEAN, Enneagram, or Ego-Equilibrium, as described above. Demographic parameters for the user may also be added.

[0091] In the personality profile space between the target profile and the personality profiles of individual users (or groups of users), a search (e.g., similarity search or exact score matching) can be performed. The personality profile that best matches the target personality profile is then identified. In this regard, personality profile scores for different personality profile schemes may be pre-calculated for each user. The best match for the target profile is then found by a similarity search between the defined target profile score and each user's profile score. Next, different options for similarity searches are described.

[0092] The term "similarity search" encompasses a broad range of mechanisms for searching a vast space of objects (in this case, profiles) based on the similarity between any pair of objects (e.g., profiles). Nearest neighbor search and range queries are examples of similarity search. Similarity search can rely on mathematical concepts of metric spaces that allow for the construction of efficient index structures to achieve scalability in the search domain. Alternatively, non-metric spaces such as the Kullback-Leibler divergence or embeddings learned by neural networks may be used for similarity search. Nearest neighbor search is a form of proximity search and can be expressed as an optimization problem to find the point in a given set that is closest to (or most similar to) a given point. Closeness is usually represented by a dissimilarity function, where the lower the similarity of the objects, the larger the value of the dissimilarity function. In this case, the (dis)similarity of the profiles is the criterion for the search.

[0093] The search for the best-matching user of a target profile may be performed in the personality profile space by comparing the target profile with the user's personality profile, or in the content descriptor set space by comparing the target content descriptor set with the media content descriptor set corresponding to the user. In the latter case, the target content descriptor set may be derived from the target profile or a product or brand profile.

[0094] For profile comparison, this search will • Matching of profile elements (depending on the presence or absence of profile elements), • Matching (numerical search) of profile attribute (score) values, • Searching for a range of values ​​like this (for example, the score "respect" is between 75% and 100%), • Vector-based matching and similarity calculations (calculating how "close" (similar in terms of numerical distance) the values ​​of the target profile and personality profile are by comparing the elements of the numerical profile (e.g., using distance measures such as Euclidean distance, Manhattan distance, cosine distance, or other methods such as Kullback-Leibler divergence)) • Learning similarity based on machine learning (a machine learning or deep learning algorithm learns a similarity function based on examples given to the algorithm (in one embodiment, this learned similarity function can then be used permanently)), It may be performed by [this method].

[0095] In an embodiment, the media similarity engine may use one or more of the user's personality profile, the user's current situation or context, and the user's current mood to search for a user that best matches the target profile. For example, as described above, the personality profiling engine analyzes the user's listening history. In this way, the user's personality profile and / or the music listener's emotional profile (including the listener's mood) are determined. The media similarity engine may then be configured to determine and find the user best suited to the target profile based on the individual's (long-term) personal music listening history, personality profile, (short-term) mood profile and / or personality profile, a weighted mixture of short-term and long-term personality profiles, and, optionally, the user's context and environment information. The individual's context and environment can be determined by other numerical factors measured from a mobile device or other personal user device from which location data, weather data, movement data, biosignal data, etc. can be derived. This may be performed instantaneously while the user is listening in a listening session. Therefore, as explained above, the user's personality profile (generated by mapping from a set of media content descriptors) is compared to the target profile. For example, a similarity search is performed between the user's personality profile and the target profile to determine the best-matching user profile (and corresponding user) (and, in some cases, rank them according to their matching scores).

[0096] Figure 5 shows one embodiment of method 100 for determining a user or user group that matches a target personality profile. This method begins in step 110, in which identification information for a group of media items, including one or more media items, is obtained for the user or user group. The identification information for the group of media items may be the user's or user group's playlist or media consumption history.

[0097] For example, the identification information for one or more media items includes the user's (or user group's) short-term media consumption history, and the personality profile characterizes the user's (or user group's) current mood. In step 120, a set of media content descriptors is obtained for each of the identified one or more media items. The media content descriptors include characteristics that define the acoustic descriptor, semantic descriptor, and / or sentiment descriptor of each media item, and may be calculated directly from the media item or read from a database. Details regarding the generation of media content descriptors are as described above.

[0098] In step 130, a set of aggregated media content descriptors is determined for the entire group of one or more identified media items, based on each media content descriptor of the individual media items. For example, if one or more identified media items correspond to a playlist, a set of aggregated media content descriptors is determined for this playlist. If only one media item is identified, a set of aggregated media content descriptors may be determined from the segment of the media item. Then, in step 140, the set of aggregated media content descriptors (e.g., the user's sentiment profile) is mapped to the personality profile defined according to the personality scheme described above. This mapping may be based on mapping rules. In step 150, the generated personality profiles for the user's (or user group's) media items are provided to the media similarity engine. The above process is repeated for multiple users or user groups, generating personality profiles for each further user or user group. In this way, multiple personality profiles are generated, each associated with a corresponding user or user group, and each profile characterizes the user or user group with respect to the applied personality scheme.

[0099] In step 160, the personality profile of a user or user group is compared to the target personality profile to determine at least one user or user group with the best-matching personality profile. The target personality profile corresponds to a target user group or product or brand profile. In step 170, at least one user or user group with the best-matching personality profile is selected. In step 180, a new media item corresponding to the target personality profile is selected for presentation to at least one determined user or user group. For example, an electronic message containing the new media item is automatically generated for at least one determined user or user group, and the generated message is electronically sent to the user or user group. The electronic message (i.e., the new media item) may contain information about the product or brand associated with the target personality profile.

[0100] In one embodiment, the media similarity engine is configured to select individuals by matching the individual's (user's) personality profile with the product or personality profile of a target group. In this embodiment, an advertising customer or brand using the disclosure system first provides a target "personality profile" by defining the target group by setting score values ​​within a specific personality profile scheme (such as MBTI, OCEAN, Enneagram, or Ego-Equilibrium), or by defining a brand / product profile that describes the attributes of the brand or product psychologically, emotionally, or marketing-wise.

[0101] This system may have already profiled some users by (preliminarily) analyzing their music preferences (listening history, favorite tracks / albums / artists). The media similarity engine then finds individuals that match a given product profile or target group profile. The output is a list of users suitable for the brand / product profile or specified target group (e.g., by user ID). Identified individuals can then be targeted by specific advertisements.

[0102] In this embodiment, selection based on matching an individual's (user's) personality profile with the target group's personality profile is based on the definition of the target group. The target group may be defined by setting the target profile values ​​within a specific personality profile scheme (such as MBTI, OCEAN, Enneagram, Ego-Equilibrium, etc.).

[0103] Mapping individuals within the target group may be based on a similarity search of the (defined) personality profiles of the target group against a set of user personality profiles (pre-calculated and determined, for example, based on their viewing habits). The comparison of profiles and generation of matching scores via similarity search are as described above. Individuals corresponding to personality profiles may be ranked based on the matching scores of those profiles. A matching score threshold may be applied to select the best-matching group for each individual.

[0104] In other embodiments, selection based on matching an individual's (user's) personality profile with a product profile may be based on the definitions of the product profile. The advertising customer or brand customer defines the type of emotion they felt towards each product in each advertisement.

[0105] In one example of generating a product profile, a marketing professional defines product attributes and values ​​in a similar way to how target groups are identified by attributes in a personality profile (e.g., MBTI attributes and percentage values ​​("scores")). Therefore, a product profile may include attributes and scores defined by percentage values, similar to a personality profile. These product attributes may be divided into different groups. For example, in one embodiment, such three groups (also referred to as "evaluations") are "brand evocation," "product symbolism," and "product use." Each of the three groups may have the same elements (attributes) or different elements, and the product profile is defined by setting percentage values ​​for these attributes.

[0106] In one exemplary embodiment, each of the three groups (evaluations) can be defined by one of many terms (for example, the "25 positive emotions" commonly used in marketing: empathy, kindness, respect, love, admiration, dreaminess, aspiration, desire, adoration, ecstasy, joy, entertainment, hope, expectation, surprise, vitality, courage, pride, confidence, inspiration, attraction, fascination, reassurance, relaxation, satisfaction), and a percentage value can be assigned to each.

[0107] In another embodiment, only attribute terms associated with the product are defined, and the associated values ​​are not defined. By selecting one word in each of the three evaluation groups that make up the product profile, it becomes possible to define the corresponding musical content descriptor (in one embodiment, primarily atmosphere) necessary for fitting the product to the target group. For example, searching for an individual to promote a new Harley-Davidson motorcycle: • Brand evokes respect • Product symbol: Entertainment • Product usage: Satisfied This can be done by defining these three attributes (one for each evaluation group).

[0108] There are two methods for mapping product profiles to individual personality profiles. a) Applying mapping rules from attributes and scores defining the product profile to a personality profile (e.g., MBTI). This allows for the derivation of a target personality profile that is compared to an individual's (pre-calculated) personality profile using similarity search as described above. The mapping rules from the product profile to personality profile elements may be defined manually or learned by a machine learning algorithm similar to the mapping rules from content descriptors to personality profile elements as disclosed above. b) Mapping from a defined product profile to a corresponding music content descriptor by specifying a set of mapping rules. The sentiment profile of a user (individual) or user group is calculated by aggregating music content descriptors, as described above. Based on the sentiment profile (ad-hoc or preliminary calculation) represented by values ​​in the music content descriptor space, a similarity search is performed in the music content descriptor space to find the best-matching user or user group.

[0109] Figure 6 shows the mapping from product profile attributes to atmosphere content descriptors. It illustrates the mapping rules for the product attribute "sympathy" to atmospheres such as "sentimental," "cool," and "friendly." In this example, the attribute "sympathy" requires nearly 50% of the atmospheres "sentimental" and "innocent," while requiring nearly 100% of the atmospheres "cool," "friendly," and "gentle." Similar but different rules exist for other evaluations (groups of product attributes). Similarly, it shows the relationship between the product attributes "kindness" and "respect" and atmosphere content descriptors.

[0110] Therefore, only the emotional profiles of users (or user groups) that strictly meet the above atmosphere criteria are considered candidates for matching. Depending on the selected similarity method, the closer the numerical values, the higher the relevance score in the output of the matched user (or user group).

[0111] Similar to the mapping from content descriptors to personality profiles described above, mapping rules define how (collected) media content descriptors can be used to search for matching users. Mapping rules define the product profile attributes that contribute to the media content descriptor, their values, and how they manifest. Again, mapping rules may be learned using machine learning techniques.

[0112] By using the personality profiling engine described above, the system can build a database of users searchable by emotional profiles or personality profiles (both derived from music content descriptors, e.g., mood). Once the system determines a product profile, it can then use the mapping or similarity search described above to find a list of users whose personality profiles match the product profile.

[0113] If the best-matching users are identified, ads may be pushed first to the users who best match the product profile or the brand's target group. The system may also output a list of identified users to the target audience based on, for example, a user identifier, as well as a matching score value indicating the degree to which the user is a good fit for the brand or product.

[0114] In one embodiment, the media similarity engine is configured to select individuals in real time by matching their current mood with the product or personality profile of a target group. In this case, the brand defines the target group by its target personality profile. The target group may be defined by a personality profile scheme such as MBTI, OCEAN, Enneagram, or Ego-Equilibrium. The system finds individuals who have an instantaneous short-term personality profile (commonly known as an "instantaneous user mood profile") that fits a given target profile. The brand can then target these individuals with specific advertisements.

[0115] In this embodiment, the system analyzes the user's current media consumption (e.g., the last 10 music tracks) in real time to instantaneously profile the user and assign them to a target group. The definition of the brand's target group or product profile, as well as the mapping between music and individuals (listeners), is carried out in the same manner as described above. Advertisements are pushed to users who match the brand / product's target group. While listening to music, individuals who should receive individually targeted advertisements that best match their current short-term personality profile (commonly known as the "instant user mood profile"), or who should be targeted to specific branding or e-commerce campaigns, are selected.

[0116] In this scenario, user profiles are calculated in "real time," meaning that only the most recent tracks (e.g., 10) viewed by the user are used. By using the personality profiling engine described above, the system calculates user profiles for the most recent short timeframe. By performing this calculation for all users, the system periodically stores all "real-time" user profiles in the database (e.g., every 10 tracks viewed). Once the system has grasped the product or brand profile (as described above), it can find groups of users whose profiles align with the product / brand profile, as described above. Based on the atmosphere alignment between the brand or product and the users, the system outputs a list of users that the brand should target at this time.

[0117] The characteristics of the above-described apparatus (devices, systems) correspond to the characteristics of each method, but it should be noted that for the sake of simplification, they may not be explicitly described. It is understood that the disclosures in this specification also extend to such characteristics of methods. In particular, it should be understood that this disclosure relates to methods for operating the above-described devices and / or to the provision and / or arrangement of the individual elements of these devices.

[0118] Furthermore, it should be noted that the exemplary embodiments of the disclosure can be realized in many ways by using hardware and / or software configurations. For example, the embodiments of the disclosure can be realized by using hardware capable of running dedicated hardware and / or software. The components and / or elements in the figures are illustrative only and do not limit the scope of use or functionality of any hardware, software combined with hardware, firmware, embedded logic components, or combinations of two or more such components that realize a particular embodiment of the disclosure.

[0119] Furthermore, it should be noted that this specification and drawings merely illustrate the principles of the disclosure. Those skilled in the art will be able to realize various configurations that embody the principles of the disclosure and that fall within its conception and scope, although these are not explicitly described or illustrated herein. Moreover, all examples and embodiments outlined in this disclosure are explicitly intended solely for illustrative purposes to assist the reader in understanding the principles of the proposed method. Furthermore, all descriptions in this specification that provide the principles, aspects, and embodiments of the disclosure, as well as their specific examples, are intended to encompass their respective equivalents. Glossary

[0120] The following technical terms are used throughout this specification.

[0121] media Media includes all kinds of media items that can be presented to the user, such as audio (especially music) and video (including embedded audio tracks). Furthermore, photographs, series of photographs, slides, and graphic representations are examples of media items.

[0122] Media content descriptor Media content descriptors (commonly known as "characteristics") are calculated by analyzing the content of a media item. Music content descriptors (commonly known as "music characteristics") are calculated by analyzing digital audio (segments (excerpts) or the entire song). These consist of a set of music content descriptors that include atmosphere, genre, context, acoustic attributes (key, tempo, energy, etc.), vocal attributes (presence of vocals, vocal family, vocal gender (low or high)), etc. Each of these includes the range of a descriptor or characteristic. Characteristics are defined by a name and a floating-point or percentage value (e.g., bpm: 128.0, energy: 100%).

[0123] Song A musical piece is an example of a media item that consists of single lines (melody) or double lines (harmony) and represents audio data containing timbres or sounds produced by one or more voices or instruments, or both. A media content descriptor for a musical piece is also called a musical content descriptor or musical profile.

[0124] Emotional Profile An emotional profile can be determined for many media items, and it includes one or more sets of media or musical content descriptors associated with a mood or emotion. In this case, it is a collection of content descriptors for individual media items. These are typically derived by aggregating media / musical content descriptors from a set of media items associated with a person or individual (for example, consumed by that person or individual). These contain the same elements as the media / musical content descriptors, and their values ​​are determined by aggregating the individual content descriptors (depending on the aggregation method used).

[0125] Person (User): Emotional profile and personality profile A person (also referred to as a user or individual) is characterized by an emotional profile or personality profile. An emotional profile is characterized by elements of a media content descriptor (see above). A personality profile, on the other hand, includes many different elements with percentage values. The elements of a personality profile are weighted elements within a personality profile scheme (defined by a name or attribute and a percentage value, e.g., MBTI: "EI: 51%"). Personality profiles are defined by personality profile schemes such as MBTI, OCEAN, and Enneagram. - User mood (instantaneous, short-term) (i.e., a personality profile interpreted as the user's short-term emotional status (also called the user mood profile)), or - User personality type (long term) (i.e., a personality profile derived from long-term observation of the user's media consumption behavior), This may be related.

[0126] Target group The target group represents a group of individuals. This is identified as one or a combination of "personality profiles." Optionally, it may be enriched with individual-related parameters (e.g., age, gender, etc.).

[0127] product A product profile includes product attributes expressed in a psychological, emotional, or marketing manner. These attributes may be associated with a percentage value of importance.

[0128] brand Product profiles may be related to a brand. Brand profiles include brand attributes expressed in psychological, emotional, or marketing terms. Attributes may be associated with percentage values ​​of importance.

[0129] mapping Mapping is an algorithmically implemented set of rules that translate profiles from one entity (e.g., media item, song) to another entity (e.g., person, product, or brand) (or vice versa). For example, mapping can be applied between a set of content descriptors (emotional profiles) and personality profiles from a personality profiling scheme.

[0130] Similarity search Similarity search is an algorithmic procedure that calculates the similarity, proximity, or distance between two or more "profiles" of any kind (e.g., emotion profiles, personality profiles, product profiles). The output is a ranked list of profile items with matching scores (values ​​indicating the degree of matching between profiles).

Claims

1. A method for determining a user or user group, - Obtain identification information for one or more media items for a user or user group, - Obtaining a set of media content descriptors for each of the identified one or more media items, wherein the set of media content descriptors includes characteristics including a semantic descriptor for each of the media items, and the semantic descriptor includes at least one emotion descriptor for each of the media items. - Based on each of the media content descriptors of the individual media items, determine a set of aggregated media content descriptors for the entirety of the identified one or more media items, - Mapping the set of aggregated media content descriptors to the personality profile of the user or user group, wherein the personality profile includes multiple personality scores for the elements of the profile, and the personality scores are calculated from the aggregated characteristics of the set of aggregated media content descriptors. Includes, A personality profile is determined for each of multiple users or user groups, and the method described above is - Compare the personality profiles of the multiple users or user groups with the target personality profile and determine at least one user or user group that has the best-matching personality profile. Methods that further include the above.

2. The method according to claim 1, wherein the media item includes a musical portion and is preferably a musical piece presented to a user or a group of users.

3. The method according to claim 1 or 2, wherein the identification information of one or more media items includes a playlist of the user or user group.

4. The method according to claim 1 or 2, wherein the identification information of one or more media items includes the user's short-term media consumption history, and the personality profile characterizes the user's current mood.

5. The method according to any one of claims 1 to 4, wherein the set of media content descriptors for a media item includes one or more acoustic descriptors for the media item determined based on an acoustic analysis of the media item.

6. The method according to any one of claims 1 to 5, wherein the set of media content descriptors for a media item is determined based on an artificial intelligence model that determines one or more semantic descriptors and / or emotion descriptors for the media item.

7. The method according to claim 6, wherein the one or more semantic descriptors include at least one of genre, presence of voice, gender of voice, vocal pitch, mood of song, and rhythmic mood.

8. The method according to any one of claims 1 to 7, wherein a segment of a media item is analyzed, and the set of media content descriptors for the media item is determined based on the results of the analysis of the segment.

9. The method according to any one of claims 1 to 8, wherein the step of obtaining a set of media content descriptors for each of the identified media items includes reading the set of media content descriptors for a media item from a database.

10. The method according to any one of claims 1 to 9, wherein the step of determining a set of aggregated media content descriptors includes calculating an aggregated numerical property from each of the numerical properties of the identified media items.

11. The method according to any one of claims 1 to 10, wherein the personality profile is based on a personality scheme that defines many personality scores for profile elements representing personality traits.

12. The method according to any one of claims 1 to 11, wherein the personality score of the personality profile is determined based on a mapping rule that defines a method for calculating the personality score from a set of aggregated media content descriptors.

13. The method according to claim 12, wherein the mapping rule is learned by machine learning technology.

14. The method according to any one of claims 1 to 13, wherein the personality score of the personality profile is determined based on the weighted aggregate numerical characteristics of the identified media items.

15. The method according to any one of claims 1 to 14, wherein the personality score of the personality profile is determined based on the presence or absence of the aggregated characteristics of the identified media items.

16. The method according to any one of claims 1 to 15, wherein comparing profiles is based on matching profile elements and selecting a personality profile of a user or user group having the same or similar elements as the target personality profile.

17. The method according to any one of claims 1 to 16, wherein comparing profiles is based on a similarity search in which corresponding scores of profiles are compared and a matching score value indicating the similarity between each pair of profiles is calculated.

18. The method according to claim 17, further comprising ranking the user's personality profile according to their matching scores.

19. The method according to any one of claims 1 to 18, wherein comparing profiles depends on the context or environment of each user or user group.

20. The method according to any one of claims 1 to 19, wherein the target personality profile corresponds to a target user group or a product or brand profile.

21. The method according to claim 20, wherein the target personality profile is generated from the product or brand profile by mapping elements of the product or brand profile to personality scores of the personality profile.

22. The method according to claim 21, wherein the personality score of the target personality profile is determined based on a mapping rule that defines a method for calculating the personality score from the elements of the product or brand profile.

23. The method according to claim 22, wherein the mapping rule is learned by machine learning technology.

24. The method according to any one of claims 1 to 23, wherein the media item corresponding to the target personality profile is selected for presentation to the at least one determined user or user group.

25. The method according to any one of claims 1 to 24, wherein an electronic message is automatically generated for at least one determined user or user group, and the generated message is electronically transmitted to the user or user group.

26. The method according to claim 25, wherein the electronic message includes information about a product or brand associated with the target personality profile.

27. The method according to any one of claims 1 to 26, wherein the identification information of at least one determined user or user group is transmitted to the database server.

28. The method according to any one of claims 1 to 27, wherein the identified one or more media items correspond to recently consumed media items, the user's personality profile characterizes the user's current mood, and the comparison of the user's personality profile with a target personality profile is performed in real time.

29. The method according to any one of claims 1 to 28, wherein determining the personality profile and comparing it with the target profile is performed repeatedly, particularly after many media items have been provided to the user or user group.

30. A computer device having a memory and a processor configured to perform the method described in any one of claims 1 to 29.

Images