Generating a Personality Profile

Abstract

Provides a personality profile. [Solution] In the personality profiling engine 10, the audio content analysis unit 40, equipped with an acoustic analysis unit 41, determines several media content descriptors 43 by analyzing the time-frequency plane in a way that generates a frequency domain representation such as a spectrogram of the audio content and calculates acoustic characterization items such as tempo (bpm) or key. The mapping unit 50 maps the media content descriptors for the audio files to a media personality profile 61 by applying mapping rules 51 received from the mapping rule database 52. The media similarity engine 70 determines a profile 71 similar to the target profile by comparing the profiles.

Description

Background Art

[0001] This application relates to the analysis of media content to determine media profiles and personality profiles from generated semantic descriptors of media items. Media profiles and personality profiles can be used in many use cases, such as for the determination of media users having recommendation and matching profiles of similar media items. Use cases can include media recommendation engines, virtual reality, smart assistants, advertising (target marketing), and computer games.

Summary of the Invention

[0002] In a broad aspect, the present disclosure relates to the generation of a personality profile from one or more media items. The media item can be any kind of media content, particularly an audio or video clip. The audio media item preferably includes a piece of music or a portion of a piece of music, preferably a piece of music. Photos, a series of photos, videos, slides, graphic representations are further examples of media items. The generated media and personality profiles characterize the personality or emotional situation of the consumer of the media item, i.e., the user who consumed the media item.

[0003] A method for providing a personality profile includes obtaining identification information for a group of media items containing one or more media items. Media items may be identified, for example, by a list that references the storage location of the media item (e.g., a user or user group playlist, or a user's streaming history), by an enumeration of the media item's name or title (e.g., artist, album, song), or by a unique identifier (e.g., ISRC, MD5 sum, audio identification fingerprint, etc.). For example, one or more identified media items may correspond to an album or an artist. The storage location of the corresponding audio / video file may be determined by a table lookup or search procedure.

[0004] Next, a set of media content descriptors is obtained for each of the one or more identified media items in the group. A set of media content descriptors for a media item (also referred to as the media profile of the media item, or the song profile in the case of a song media item) includes many media content descriptors (also referred to as characteristics) that characterize the media item in terms of different aspects. The set of media content descriptors includes, optionally, other descriptors, a semantic descriptor of the media item. A semantic descriptor describes the content of the media item at a high level, such as the genre to which the media item belongs. In this sense, it is possible to classify a media item into one of many semantic classes and indicate with high probability which semantic class the media item belongs to. For example, a semantic descriptor may be represented as a binary value (0 or 1) indicating the class affiliation of the media item, or as a real number indicating the probability that the media belongs to a semantic class. A semantic descriptor may also be an emotional descriptor indicating that the media item corresponds to an emotional aspect such as atmosphere. A sentiment descriptor can classify a media item into one or more sentiment classes, indicating with high probability which sentiment class the media item belongs to. A sentiment descriptor can be represented as a binary value (0 or 1) indicating the class the media item belongs to, or as a real number indicating the probability that the media belongs to a sentiment class.

[0005] Media content descriptors may be calculated from identified media items, or they may be read from a database containing pre-analyzed media content descriptors for multiple media items. Thus, the step of obtaining a set of media content descriptors for each of the one or more identified media items may include reading a set of media content descriptors for the media items from the database. Some media content descriptors have a numerical value that quantifies the degree of each semantic descriptor and / or sentiment descriptor present for the media item. For example, a numerical media content descriptor may be normalized to have a value between 0 and 1 or between 0% and 100%.

[0006] Based on each media content descriptor of an individual media item, a set of aggregated media content descriptors is determined for the entire group of identified media items. The aggregated media content descriptors characterize the semantic descriptors and / or sentiment descriptors of the media items in the group. A set of aggregated media content descriptors that include mood and are associated with a user or user group may also be referred to as the sentiment profile of the user or user group. Aggregated media content descriptors can be calculated, in particular for media content descriptors with numerical values, by averaging the values ​​of the individual media content descriptors of the media items. Note that other methods besides a simple average of the values ​​of individual media content descriptors are possible. For example, root mean square (RMS) or other aggregation methods (e.g., "log-mean-exponential averaging") that emphasize larger values ​​in the aggregate may be applied. For this reason, the step of determining a set of aggregated media content descriptors may include calculating an aggregated numerical content descriptor from each of the numerical content descriptors of the identified media items in the group.

[0007] A set of aggregated media content descriptors (i.e., the user's emotional profile) is mapped to a personality profile for a group of media items. The personality profile has multiple personality scores for its elements. These personality scores are calculated from the aggregated properties of a set of aggregated media content descriptors (e.g., the emotional profile of a user or a group of users). Typically, a personality profile is based on a personality scheme (value pairs representing personality traits) that defines many profile elements, including attributes. The values ​​of the profile elements are also referred to as profile scores. Examples of personality schemes include the Myers-Briggs personality type index (MBTI), the Ego Equilibrium, the Big Five personality traits (Openness, Conscientiousness, Extraversion, Agreeableness, Neuroticism (OCEAN)), or the Enneagram. Other schemes are possible for defining personality profile elements.

[0008] The identified media items are relevant to the user's emotional / psychological context and may enable the determination of the user's personality profile. If the identification information for one or more media items includes the user's short-term media consumption history (e.g., recently listened to music), the generated personality profile will characterize the user's current or recent mood. If the identification information for one or more media items includes playlists that identify the user's long-term media item usage history, the generated personality profile will characterize the user's long-term personality profile. In some embodiments, particularly in advertising and branding use cases, it is also possible to consider a blend of the long-term and short-term personality profiles (based on the mood of recently listened-to music) as the user's relevant personality profile.

[0009] The generated personality profile may be classified into one of several personality types corresponding to a personality scheme, for example. This classification may be based on a profile score compared to a threshold. Other classification schemes may be used, such as determining the highest score. Depending on the result of the comparison, a personality type may be assigned to the profile and, consequently, to the user. For example, a personality profile (e.g., MBTI) has multiple numerical values ​​(scores) that represent a personality type as a whole. To make a decision, it is possible to determine a "single personality type" by determining the "maximum personality attribute" from such a profile. Both methods allow for the psychological characterization of the user, but the former is more granular, while the latter determines a specific personality type.

[0010] The classification results and / or the generated personality profiles or graphic representations of the determined personality types may be displayed on a computer device or transmitted to a database server. Personality profiles corresponding to one or more media items may be used in many use cases, such as recommending similar media items (in media recommendation engines, smart assistants, smart homes, advertising, product targeting, marketing, virtual reality, and games, etc.) or determining media users whose personality profiles match the profiles of analyzed music. Conversely, media items that match a user's personality profile may be selected. In some embodiments, a target group of users for a particular media item is determined from the media item profiles, or the most suitable music is selected for a given target user group.

[0011] A set of media content descriptors for a media item may further include one or more acoustic descriptors for the media item. The acoustic descriptors (also referred to as acoustic attributes) of a media item may be determined based on an acoustic digital audio analysis of the media item's content. For example, the acoustic analysis may be based on a spectrogram derived for the audio content of the media item. Various techniques may be employed to obtain acoustic descriptors from an audio signal, for example, based on an analysis of the audio waveform signal. Examples of acoustic descriptors include tempo (beats per minute), duration, key, mode, presence or absence of rhythm, and energy (of the spectrum).

[0012] A set of media content descriptors for a media item may be determined at least in part on one or more artificial intelligence models that determine one or more emotional descriptors and / or one or more semantic descriptors for the media item. One or more semantic descriptors may include at least one of the following: genre, or vocal attributes such as the presence or absence of vocals, and the gender of the vocals (low-pitched or high-pitched, respectively). Examples of emotional descriptors include the mood of the music and the mood of the rhythm. The artificial intelligence models may be based on machine learning techniques such as deep learning (deep neural networks). For example, an artificial neural network may be used to determine the emotional and semantic descriptors for a media item. The neural network may be trained on a broad set of data provided by music experts and data science experts. It is also possible to use an artificial intelligence model or machine learning technique (e.g., a neural network) to determine the acoustic descriptor (such as bpm or key) of a media item.

[0013] The media item may be segmented, and based on the results of the analysis of individual segments, a set of media content descriptors for the media item may be determined. For example, a media item may be segmented into media item parts, acoustic analysis and / or artificial intelligence techniques may be applied to the individual parts, and the acoustic descriptors and / or semantic descriptors generated for these parts may then be aggregated to form an acoustic descriptor and / or semantic descriptor for the complete media item, similar to how the media content descriptors for a media item are aggregated for the entire group of media items.

[0014] Furthermore, the personality score of a personality profile (i.e., attribute values ​​(value pairs of profile elements)) may be determined based on mapping rules that define how to calculate the personality score from a set of aggregated media content descriptors. The mapping rules may define one of the set of aggregated media content descriptors that contribute to the personality score and how it is defined. For example, the personality score of a personality profile may be determined based on a weighted aggregated numerical content descriptor of an identified media item. Based on the weighting, different content descriptors may contribute to the score to different degrees. In addition, the personality score of a personality profile may be determined based on the presence or absence of aggregated content descriptors of an identified media item. In other words, if aggregated content descriptors exist, their contribution to the score may be made, for example, by weighting normalized numerical aggregated content descriptors. Alternatively, the contribution to the score when a collective content descriptor is not expected to exist can be represented by weighting the difference between 1 and a normalized numerical collective content descriptor value (which has a value between 0 and 1).

[0015] Mapping rules may be learned using machine learning techniques. For example, the weights that aggregated numerical content descriptors contribute to the score may be determined by machine learning using a number of target profiles (real-world user profiles) and a suitable machine learning technique that can determine rules and / or weights regarding how content descriptors are mapped to personality profiles. Such machine learning techniques may also determine which content descriptors can contribute to the profile score and select each content descriptor.

[0016] A user's (long-term) personality profile may be determined from a playlist that identifies the user's (longer) media item usage history, and a user's (short-term) mood profile may be determined from the user's short-term media consumption history. This method may further include calculating the difference between the user's long-term personality profile and the short-term mood profile. Based on this difference, the degree of difference between the user's current mood and the user's general personality can be determined. This can be useful for recommending specific musical directions based on short-term "deviations" from the user's general personality profile.

[0017] In embodiments addressing the selection of suitable media items for a user or user group, a separate personality profile is provided for each of several media items. Thus, each media item is characterized in terms of emotion and personality. A target personality profile may also be defined for a group of users or individual users. Thus, each user or user group is also characterized in terms of emotion and personality by their respective personality profile. This method may further include comparing the personality profiles of media items with the target personality profile and determining at least one media item that has the best-matching personality profile with respect to the target personality profile. This allows for the selection of the best-matching music for a user if the target personality profile corresponds to an individual. Furthermore, if the target personality profile corresponds to a target group of users, this method provides the optimal music selection for the target user group.

[0018] The search for one or more best-matching personality profiles may be based on comparing the personality profiles of media items with the target personality profile. For example, comparing profiles may be based on matching profile elements and selecting personality profiles of media items that have identical or similar elements to the target personality profile. Furthermore, comparing profiles may be based on similarity search, where corresponding scores of profile elements are compared and a matching score value indicating the similarity of each pair of profiles is calculated. The matching score for a pair of profiles may be based on the individual matching scores of the corresponding attribute values ​​(scores) of the profile elements. For example, the difference between corresponding values ​​(scores) of profile elements may be calculated (e.g., Euclidean distance, Manhattan distance, cosine distance, etc.), from which a matching score for the profile pair being compared can be calculated. In an embodiment, multiple best-matching personality profiles are determined, and the personality profiles of media items are ranked according to their matching scores. This makes it possible to determine the best-matching media item, the second-best match, and so on.

[0019] Comparing profiles may further depend on the context or environment of each user or user group. Examples of context or environment include the user's location, date and time, weather, and other people nearby. Similar contexts or examples may be applied to user groups.

[0020] Furthermore, the above disclosure method may determine a target personality profile for the user, in addition to the personality profile of the media item, based on the identification information of one or more suitable media items for the user. In this way, the target profile characterizes the user's personality, and media items that match the user's personality can be found. If one or more of the identified suitable media items are the last media items consumed by the user, the target personality profile represents the user's current mood. The identified media items then match the user's current mood.

[0021] At least one of the determined media items may be selected for playback or recommendation to the user. The selected media item or information associated with the selected media item (e.g., a reference to media item storage or a media database) may be provided to the user or a user device associated with the user so that the media item can be recommended, retrieved, or played to the user.

[0022] In some embodiments, instead of simply presenting the user with songs of the same mood, it may be desirable to actively change the user's mood by presenting songs of different moods. For example, if the user is determined to be sad, a song characterized by a happy mood is selected and played for the user. For this reason, the target profile for the search for the best-matching personality profile may be a profile that complements the user's current mood profile. In this case, the search for the best-matching media item may be based on a comparison between the personality profile of the media item and the target personality profile, and a media item that complements the user's current mood is determined.

[0023] Comparing the personality profile of a media item to a target personality profile and determining at least one media item with the best-matching personality profile may be performed repeatedly, for example, after a determined period or after many media items have been presented to the user. This comparison may be based on the most recently determined user profile as the target profile. In this way, the user's personality profile and the selection of recommendations or playback for the user can be updated periodically (for example, in real time after media items are presented to the user). This enables an adaptive music recommendation service in which new music is played to the user based on music that has been played in the past.

[0024] Personality profiles may be generated on the server platform. This method may further include transmitting identification information of one or more suitable media items for the user from the user device associated with the user to the server platform. For this purpose, the server can receive information about the user's media consumption (e.g., playlists) and determine the user's personality profile from this information. As mentioned above, this may be repeated. User devices may be any user equipment, such as personal computers, tablet computers, mobile computers, smartphones, wearable devices, smart speakers, smart home environments, car radios, etc., or any combination thereof. After determining the best-matching media item by comparing the personality profile of the media item with the user's target personality profile, the server may transmit a representation of at least one determined best-matching media item to the user device. This information is received and presented to the user, or the determined best-matching media item is played.

[0025] Identification information for one or more suitable media items (e.g., playlists) for a user may be stored on the server platform, and the user's personality profile and media items are generated on the server platform. After determining the best-matching media item by comparing the media item's personality profile with the user's target personality profile, the server can send a representation of at least one of the determined media items to the user's associated user device. This information is received and presented to the user, or the determined best-matching media item is played.

[0026] In another embodiment of this disclosure, a computer device is proposed for performing any of the above methods. This computer device may be a server computer comprising memory for storing instructions and a processor for executing instructions. The computer device may further comprise a network interface for communicating with a user device. The computer device may receive from the user device information about media items consumed by the user. The computer device may be configured to generate personality profiles as described above. Depending on the use case, the personality profiles may be used to recommend similar media items or to determine media users whose personality profiles match the profiles of analyzed music. Information about recommended media items may be transmitted to the user device. In embodiments, a target group of users for a particular media item is determined, or the most suitable music is selected for a given target user group.

[0027] Embodiments of the disclosed device may include, but are not limited to, the use of one or more processors, one or more application specific integrated circuits (ASICs), and / or one or more field programmable gate arrays (FPGAs). Embodiments of the device may also include the use of other conventional hardware such as software programmable processors such as graphics processing unit (GPU) processors and / or customized hardware.

[0028] Another aspect of the present disclosure may relate to computer software, a computer program product, or any medium or data embodying computer software instructions for causing at least one processor to execute any of the method steps described in the present disclosure by executing on a programmable computer or dedicated hardware having at least one processor.

[0029] Referring particularly to the above uses, several exemplary embodiments are described herein, but it will be understood that the present disclosure is not limited to such fields of use and is applicable in a broader context.

[0030] Particularly, the method according to the present disclosure relates to a method of operating the devices according to the above exemplary embodiments and variations thereof, and each description regarding the device also applies to the corresponding method, and vice versa, and it should be understood that similar descriptions may be omitted for simplicity. Also, the above aspects can be combined in many ways even if not explicitly disclosed. Those skilled in the art will understand that these aspects and combinations of features / steps are possible as long as no contradiction is explicitly excluded.

[0031] In the course of the following discussion, other and further exemplary embodiments of the present disclosure will become apparent by referring to the accompanying drawings.

Brief Description of the Drawings

[0032] The following describes exemplary embodiments of this disclosure with reference to the attached drawings, but these are only a few examples. [Figure 1] This figure schematically illustrates the operation of one embodiment of the present disclosure. [Figure 2a] This diagram illustrates the generation of semantic descriptors from audio files. [Figure 2b] This figure illustrates the generation of semantic descriptors by the audio content analysis unit. [Figure 3a] This diagram shows the mapping of atmosphere content descriptors to the EI (Extroversion-Introversion) personality score in the MBTI personality scheme. [Figure 3b] This figure shows the mapping of atmosphere content descriptors to the openness personality score in the OCEAN personality scheme. [Figure 4a] This figure shows an example of a graphical representation of a personality profile in the MBTI Personality Scheme. [Figure 4b] This figure shows an example of a graphic representation of a personality profile in the OCEAN Personality Scheme. [Figure 5] This figure illustrates one embodiment of a method for selecting the optimal song for a given target user group. [Modes for carrying out the invention]

[0033] According to one broad aspect of this disclosure, a personality profiling engine for generating personality or sentiment profiles corresponding to media items being analyzed, such as music, determines the characteristics of a media item. This enables a variety of new applications (also referred to in this disclosure as “use cases”) that allow for the classification, searching, recommendation, and targeting of media items or media users. For example, personality or sentiment profiles may be used to recommend media items that a user is likely to be interested in.

[0034] For example, if the input to a personality profiling engine is a user's short-term music listening history, it can determine a personality profile that characterizes the listener's mood based on the songs the user has recently played. If the input is a long-term music listening history, it can determine the listener's general personality profile. It is also possible to calculate the difference between the user's long-term personality profile and their current mood to determine if the user is in an exceptional situation.

[0035] Personality profiles generated by a personality profiling engine can detect emotional signatures of music listeners, focusing on the atmosphere, mood, and values ​​that define a person's multifaceted personality. This allows for addressing questions such as whether a listener is self-conscious or haughty, or whether they enjoy exercise or travel.

[0036] In the case of audio, for example, it is possible to find similar-sounding music tracks based on the emotional descriptor and / or semantic descriptor of an audio file. A media similarity engine using the generated emotional profile may utilize machine learning or artificial intelligence (AI) to match and find musically and / or emotionally similar tracks. Such a media similarity engine can listen to and understand music like a human, then search millions of music tracks for specific sonic or emotional patterns, finding the required music in seconds by matching requirements. Based on the generated profile, it is possible to search for, for example, only instrumental or vocal tracks, or according to other semantic criteria such as genre, tempo, mood, or bass vs. treble.

[0037] The basis of the proposed technology is a personality profiling engine that performs tagging of media items using media content descriptors based on audio analysis and / or artificial intelligence (e.g., deep learning algorithms, neural networks, etc.). The personality profiling engine may tag media tracks by weighted mood, emotion, and musical attributes such as genre, key, and tempo (beats per minute (bpm)) by enriching metadata using AI. The personality profiling engine may analyze the mood, genre, acoustic attributes, and contextual situation of a media item (e.g., a musical track (song)) and obtain weighted values ​​for different "tags" within these categories. The personality profiling engine may analyze a media catalog and tag each media item in the catalog with corresponding metadata. Media items are, for example, • Acoustic attributes (bpm, key, energy, ...) • Atmosphere / rhythm atmosphere, Genre, • Vocal attributes (instrumental, high notes, low notes), Contextual situation, It may also be tagged with a media content descriptor related to it.

[0038] In the mood category for tagging music from an "emotional" perspective, the personality profiling engine may output up to 35 "complex mood" values ​​that are taxonomically classifiable within, for example, 18 mood subfamilies organized as 6 main families. All 6 main families and 18 subfamilies include human emotions. The level of detail in the mood taxonomy can be arbitrarily narrowed down; that is, the 35 "complex moods" can be further subdivided as needed, or additional "complex moods" can be added.

[0039] Figure 1 schematically illustrates the operation of one embodiment of the present disclosure, which generates personality profiles and determines the similarity of profiles to provide various recommendations for similar media items, or to match users or user groups. The personality profiling engine 10 receives one or more media files 21 from the media database 20. Media files are identified in the media list 30 provided to the personality profiling engine 10 in order to read media items from the database 20. The media list 30 may be the user's playlist read from a playlist database that stores the most recent media items played by the user and user-defined playlists representing the user's media preferences.

[0040] Analysis of the media file 21 determines media content descriptors 43, including the acoustic descriptor, semantic descriptor, and / or emotional descriptor of the audio content. An audio content analysis unit 40, equipped with an acoustic analysis unit 41 that analyzes the acoustic features of the audio content, determines several media content descriptors 43 by analyzing the time-frequency plane in a way that generates a frequency-domain representation, such as a spectrogram, of the audio content and calculates acoustic features such as tempo (bpm) or key. The spectrogram may be transformed according to perspective and / or logarithmic scales, for example, in the form of a Log-Mel spectrogram. The media content descriptors may be stored in a media content descriptor database 44.

[0041] The audio content analysis unit 40 of the personality profiling engine 10 further comprises an artificial intelligence unit 42 that uses an artificial intelligence model to determine media content descriptors 43, such as emotion descriptors and / or semantic descriptors, of the audio content. The artificial intelligence unit 42 may operate on any appropriate representation of the audio content, such as a time-domain representation generated by the acoustic analysis unit 41, a frequency-domain representation of the audio content (e.g., a Log-Mel spectrogram as described above), or intermediate characteristics derived from the audio waveform and / or frequency-domain representation. The artificial intelligence unit 42 may generate, for example, an atmosphere descriptor of the audio content that characterizes the mood of the music and / or rhythm of the audio content. These AI models may be trained on their own large-scale expert data.

[0042] Figure 2a shows an example of semantic descriptor generation from an audio file by the audio content analysis unit. In this embodiment, audio file samples are optionally segmented into audio chunks and converted into frequency representations such as Log-Mel spectrograms. The audio content analysis unit 40 then extracts low-level, mid-level, and / or high-level semantic descriptors from the spectrograms by applying various audio analysis techniques.

[0043] Figure 2b further illustrates an example of semantic descriptor generation by the audio content analysis unit 40. While Figure 2a shows direct audio content analysis using conventional signal processing methods, Figure 2b shows audio content analysis using a neural network, which first requires learning from "ground truth" data ("prior knowledge"). The audio file is converted into a spectrogram, and media content descriptors 43, such as the atmosphere, genre, and context of the audio file, are generated by applying one or more neural networks. The neural networks are trained for this task based on large-scale expert data (large-scale and detailed "ground truth" media annotations for supervised neural network training). In an example of semantic descriptor generation by the artificial intelligence unit 42, spectrogram data of the audio file is fed to the neural network as input, which generates semantic descriptors as output. In embodiments, one or more convolutional neural networks are used to generate descriptors for, for example, genre, rhythm atmosphere, and speech family. Other network configurations and combinations of networks can be used similarly.

[0044] The mapping unit 50 maps media content descriptors 43 for audio files to media personality profiles 61 by applying mapping rules 51 received from the mapping rule database 52. The mapping rules 51 may define the media content descriptors used in calculating the profile score (i.e., the values ​​of the profile attributes) and the weights applied to the media content descriptors. The mapping rules 51 may also be represented as a matrix that links media content descriptors and profile attributes to assign weights to the media content descriptors. The generated personality profiles 61 may be provided to the media similarity engine 70 to determine similar profiles, or they may be stored in the profile database 60 for later use.

[0045] When a personality profile is generated for a media item group, media content descriptors 43 are generated for each media item in that group (or read from the media content descriptor database 44), and a collective media content descriptor is generated for the entire group of media items. The collective nature of the numerical media content descriptors can be achieved by calculating the average value of each media content descriptor for the group of media items. Other collective nature algorithms, such as the root mean square (RMS), can also be used. The mapping unit 50 then acts on the collective media content descriptor (e.g., sentiment profile) to generate a personality profile for the entire group of media items.

[0046] The media similarity engine 70 can receive profiles directly from the personality profiling engine 10, or from the profile database 60, as shown in Figure 1. The media similarity engine 70 determines profile similarity based on profile element matching or similarity search, as disclosed below, through profile comparison. If a profile 71 similar to the target profile is determined, corresponding media items or users may be determined, and recommendations may be made. For example, one or more media items matching the user's playlist may be determined and automatically played on the user's terminal device. Other use cases are described in this disclosure.

[0047] As mentioned above, the personality profiling engine can use machine learning or deep learning techniques to determine the emotional and semantic descriptors of media items. Training may be based on a database consisting of many data points to learn relationships and analyze an individual's music preferences and listening habits. This algorithm can read the user's psychological and emotional portrait and complement existing demographic and behavioral statistics to generate a complete and evolutionary user profile. The output of the personality profiling engine is a profile of the psychologically motivated user ("personality profile") derived from an analysis of the user's music (playlists or listening history).

[0048] A personality profiling engine can derive a user's personality profile from fewer media items or from more media items. For example, based on the most recent 10 or more music items played by the user on a streaming service, the engine can calculate the user's short-term ("instantaneous") profile (reflecting the "current mood of the music listener"). If (more) music items represent the user's long-term listening history or preferred playlists, the engine can calculate the user's intrinsic personality profile.

[0049] The personality profiling engine uses advanced machine learning and deep learning techniques to understand the meaningful content of music from audio signals, achieving a human-like level of comparison beyond simple text language and labels. By extracting essential information about music from audio signals, the algorithm can learn to understand the rhythm, beat, style, genre, and mood of the music. The generated profiles can be applied to music or video streaming services, digital or linear radio, advertising, product targeting, computer games, labels, libraries, publishers, in-store music providers or synchronization agencies, voice assistants / smart assistants, smart homes, and more.

[0050] The personality profiling engine can achieve a human-like level of comparison by applying advanced deep learning techniques to understand the meaningful content of music from audio. This algorithm can analyze and predict relevant mood, genre, context, and other key attributes and assign a weighted relevance score (%).

[0051] The media similarity engine is applicable to recommendation, music targeting, and audio branding tasks and can be used by music or video streaming services, digital or linear radio, fast-moving consumer goods (FMCG) (also known as consumer goods products (CPG)), advertisers, creative agencies, dating companies, in-store music providers, or e-commerce. Personality profiling engine

[0052] The personality engine may be configured to generate a personality profile based on a group of media items by performing the following steps: In the first step, a group list is obtained that contains the identification information of one or more media items, for example, in the form of a user-defined playlist. Next, a set of media content descriptors is generated for each of the one or more identified media items in the group, or read from a database of previously analyzed media items. The set of media content descriptors includes at least one of the following for each media item: an acoustic descriptor, a semantic descriptor, and an emotional descriptor. This method then includes determining a set of aggregated media content descriptors (i.e., the user's emotional profile) for the entire group of the one or more identified media items, based on each media content descriptor of the individual media items. Finally, the aggregated set of media content descriptors is mapped to the personality profile for the group of media items. The score of the profile elements is calculated from the aggregate properties of the set of aggregated media content descriptors.

[0053] In an exemplary embodiment, the personality profiling engine is applied to determining the mood of a media user. For example, the mood of a music listener is determined based on the input "short-term music listening history." Alternatively, the general personality profile of a music listener is determined from the input "long-term music listening history." In further use cases, an individual's personality profile may be associated with the personality profiles of others to determine individuals with similar profiles at a particular moment (e.g., matching, recommendations for products with similar profiles (e.g., e-commerce), or suggestions for connecting with others (friends, dating, social networks, etc.)).

[0054] Furthermore, by using a personality profiling engine, media items such as music (e.g., current playlists and / or suggestions, or other forms of entertainment (movies, etc.), or environments such as smart homes) may be adapted to a) an individual's current mood, and / or b) an intention to change the individual's mood (intentions explicitly expressed by the individual, implicit intentions of change brought about by the system, e.g., product recommendations or optimization (increase) of user retention on the platform).

[0055] A personality profiling engine can be used to determine the difference between a user's current mood and their general personality by calculating the difference between a user's long-term personality profile and their current (mood) profile. This is useful, for example, for adapting recommendations in the case of a short-term "deviation" from the user's general personality profile that leans towards a particular musical direction (depending on a specific listening context, time of day, user mood, etc.), and for deciding whether to display ads that would normally fit the user's personality profile but are momentarily inappropriate because the current mood profile deviates from the current listening situation. In either case, the placement of recommendations or ads may be adapted to the user's personal situation at that moment.

[0056] The basis of these embodiments is a personality profiling engine that analyzes groups of media items identified by a provided list. For example, audio tracks in a group of songs (from digital audio files) are analyzed. This analysis may be performed by, for example, audio content analysis and / or the application of machine learning (e.g., deep learning) methods. The personality profiling engine is Algorithms may be applied to extract low-level, mid-level, and high-level characteristics from audio. Examples of low-level characteristics are audio waveform / spectrogram-related characteristics (or "descriptors"), examples of mid-level characteristics (or "descriptors") are "fluctuations," "energy," etc., and examples of high-level characteristics are semantic descriptors and emotional descriptors such as genre, mood, or key. Acoustic attributes such as tempo (beats per minute), key, mode, duration, spectral energy, and presence or absence of rhythm may be analyzed by applying acoustic waveform and spectrogram analysis. Neural network / deep learning-based models may be applied to analyze high-level descriptors from audio input (e.g., via Log-Mel frequency spectrograms extracted from various segments of an audio track), such as genre, mood, rhythmic mood, presence of sound (instrumental or vocal), and vocal attributes (e.g., bass or treble). The neural network / deep learning models may be trained on a large training dataset containing thousands (hundreds) of annotated examples of the aforementioned categories tagged by music experts. For example, a deep learning convolutional neural network may be used, but alternatively, other types of neural networks (e.g., recurrent neural networks), other machine learning techniques, or any combination thereof may be used. In embodiments, one model is trained for each category group of mood, genre, rhythmic mood, and presence of sound / vocal attributes. In an alternative example, one common model may be trained collectively, or one model may be trained together for multiple moods and rhythmic moods, or one model may be trained for each individual mood or genre itself.

[0057] Audio analysis may be performed on multiple time points within the audio file (for example, three 15-second sections at the beginning, middle, and end of a song), or it may be performed on the entire audio file.

[0058] The output may be stored at the segment level or at the audio track (song) level (e.g., aggregated from segments). Subsequent steps may also be applied at the segment level (e.g., obtaining a list of moods (or mood scores) for each segment (e.g., applicable to long audio recordings such as classical, DJ mixes, or podcasts, or to audio tracks where genre or mood changes)). The personality profiling engine may store all derived song content descriptors, along with their predicted or percentage values, in one or more databases for further use (see below).

[0059] The output of the audio content analysis is: • Tempo: For example, 135 bpm, • Key and mode: For example, F# minor, • Spectral energy: For example, 67% (100% is determined by the maximum value listed in the truck's catalog), • Presence of rhythm: For example, 55% (100% is determined by the maximum value in the track catalog), • Genre: As a list of categories (each independent of others with a percentage value from 0 to 100), for example, Pop 80%, New Wave 60%, Electropop 33%, Dance Pop 25%, • Atmosphere: A list of atmospheres included in the song (each with a percentage value from 0 to 100, independent of others), for example, dreamy 70%, intellectual 60%, inspiring 40%, bitter 16%, • Rhythmic atmosphere: A list of atmospheres included in the song (each with a percentage value from 0 to 100, independent of others), for example, elegant 67%, lyrical 53%, • Vocal Attributes: Instrument (0 or 100%) or any combination of "male" (low-pitched) and / or "female" (high-pitched) voices (50-100%). These are media (e.g., music) content descriptors derived from input audio (also referred to as audio characteristics or music characteristics).

[0060] In one embodiment, audio content analysis is performed. From the audio characteristics extraction, 14 mid-to-high level characteristics + 52 low-level (spectral) characteristics were obtained, From the deep learning model, 67 genres, 35 moods (+24 (by grouping into subfamilies and families) (see below)), 5 rhythmic moods, and 3 vocal attributes were derived. Output each of these.

[0061] Optionally, subsequent post-processing of the values ​​is performed by applying so-called adjustment coefficients (e.g., assigning high or low weights to genre, mood, or some other category). The adjustment coefficients adapt the machine predictions to more closely resemble human perception. The adjustment coefficients may be determined by experts (e.g., music experts) or learned by machine learning. The adjustment coefficients may be defined by one coefficient per semantic descriptor or sentiment descriptor, or by a non-linear mapping from various machine predictions to adjusted output values.

[0062] Furthermore, optionally, the aggregation of music content descriptors may generate values ​​for groups or "families" of music content descriptors that typically follow a taxonomic framework. In one example, 35 atmospheres predicted by a deep learning model are aggregated into 18 parent "subfamilies" and 6 "main families," forming a total of 59 atmospheres (following a taxonomic framework for atmospheres).

[0063] The analysis may be performed at the song level for a set of songs and delivered in audio form (various digital formats, compressed or uncompressed). For the generation of a personality profile, for a group of songs (commonly referred to as a "playlist"), the song content descriptors and their values ​​for multiple songs may be aggregated.

[0064] In some embodiments (use cases), the listener's current mood is determined. In other use cases, the personality profiling engine determines the listener's long-term personality profile. In both cases, the input is a list of songs, and the output is the user's personality profile (according to one or more personality profiling schemes). To determine the listener's mood, the input consists of the most recently listened songs. These songs provide insight into the user's current mood profile. To determine the listener's general (long-term) personality profile, the input consists of songs (usually a larger set) representing the user's (long-term) history.

[0065] The generation of a personality profile may be based on the characteristics of the music the user listens to, including (not limited to) mood, genre, vocal presence, vocal attributes, key, BPM, energy, and other acoustic attributes (= "music content descriptor," "audio characteristics," or "music characteristics"). This may be determined for each characteristic of the song's musical content.

[0066] In one embodiment, the musical content descriptors of n songs are aggregated to generate an aggregated content descriptor, that is, the user's emotional profile is generated, for example, as the average of the numerical values ​​(%) of each song in the aggregate (playlist). Alternatively, more complex aggregation procedures such as median, geometric mean, RMS (root mean square), or various forms of weighted average may be applied.

[0067] In one embodiment, preliminary analysis of songs in the user's playlist or listening history may extract song content descriptors that may contain numerical values ​​(for example, ranging from 0 to 100% for each value). For each content descriptor (for example, mood "sensibility"), the root mean square (RMS) of all "sensibility" values ​​for individual songs may be calculated and stored. The output of this aggregation is a set of song content descriptors having the same number of descriptors (attributes) as each song. This aggregated song content descriptor (sensibility profile) is used in the second stage of the personality profiling engine to determine the user's personality profile.

[0068] In some embodiments, instead of a user's playlist, an album or artist's discography (all tracks by an artist) can be used as input for aggregation. Similarly, aggregation of the musical content descriptors can be performed for many tracks (which may represent albums, artists, or playlists) (by using various methods of disclosure).

[0069] When the aggregated value of each song content descriptor is calculated, a personality profile is generated. For example, a mapping is performed from elements in an emotional profile (representing aggregated song content descriptors for n songs) to one or more personality profiles. In the mapping, mood, genre, style, etc., are converted into psychological and emotional user characteristics (personality traits). The mapping is performed from the song content descriptors to a score of a personality profile (including personality traits / human characteristics). Rules for mapping song content descriptors and their values ​​to one or more types of personality profiles defined by the personality profile scheme may be defined.

[0070] The output of the personality profiling engine is a range of numerical output parameters, referred to as personality profile attributes and scores, which represent the user's personality profile.

[0071] Personality profiles are, • MBTI (Myers-Briggs Index) ·Ego Equilibrium, • OCEAN (also known as one of the Big Five personality traits) • Enneagram, These may be defined according to various personality profiling schemes.

[0072] Each of these personality profile schemes is composed of personality attributes (for example, "extraversion" or "openness" and assigned scores (values) such as 51% or 88% (specific examples are shown below)).

[0073] For all of these schemes, a mapping from musical content descriptors to profile scores may be used, and vice versa. Figure 3a shows the mapping of atmosphere content descriptors to EI personality scores in the MBTI personality scheme. The mapping may apply a matrix such as the example shown in Figure 3a. The calculation of scores (values) within the personality profile scheme may involve the presence (%) or absence (100%) of (atmosphere or other musical content descriptors).

[0074] Each scheme may have many "scores" to calculate (for example, the MBTI scheme calculates four scores: EI, SN, TF, and JP). One or more mapping rules may be defined for each score, which affect how the score is calculated from the aggregated music content descriptor. For example, the score is equal to the sum of the values ​​calculated by the matrix divided by the number of values ​​to consider (i.e., the usual averaging mechanism).

[0075] For example, the "regressive" atmosphere (included in the music content descriptor) is used in the EI calculation as part of the MBTI scheme. Figure 3a shows an example of a rule matrix applied to the EI calculation from the atmosphere portion of the music content descriptor. The rule matrix shows how the presence or absence of atmosphere can be used in the calculation of the EI score. Other music content descriptors can be similarly included in the calculation.

[0076] In one embodiment, the EI calculation includes 17 rules incorporating 17 values ​​from the music content descriptor. These rules follow a psychological recipe. For example, the rules in the "metal" group psychologically define "closed shoulder," while the rules in the "tree" group define "open shoulder."

[0077] Similar calculations can be performed on other profiling matrices such as OCEAN.

[0078] As mentioned above, the MBTI personality profile has scores for EI, TF, JP, and SN. The following is an example of how the MBTI personality profile and its scores are expressed. “mbti”:{“name”:“INTJ”,“sources”:{ “EI”:33.66403316629877, “SN”:42.419498057065084, “TF”:57.82423612828757, "JP":61.02633025243475}}

[0079] A basic score classification may be performed based on the score value. This classification may be based on a comparison of the score value with a specific threshold. For example, the EI score in the MBTI scheme represents the balance between a user's extraversion (E) and introversion (I). An EI below 50% indicates introversion, while an EI above 50% indicates extraversion. Therefore, if EI < 50%, the user may be assigned to the I (introverted) class, and otherwise, to the E (extroverted) class. Other MBTI scores can be classified similarly.

[0080] The scores are defined as opposites on each axis (EI, SN, TF, JP). For each character pair, the value that determines which trait a person possesses is either <50% or >50%. To exclude characters from the above example, typically, the right-hand character of the character pair is taken if the value is <50%, and the left-hand character is taken if the value is ≥50%.

[0081] The scores for the generated profiles may be further classified into general personality types, for example, based on the basic classification results of the profile scores. For example, the following general personality types may be derived from the basic score classification results. ·ESTJ: Extraversion (E), Sensitivity (S), Thinking (T), Judgment (J) INFP: Introversion (I), Intuition (N), Mood (F), Perception (P)

[0082] The profile in the example above is classified as an INTJ personality type. By classifying the four-dimensional space of profile scores (EI, TF, JP, SN) into personality types, it becomes possible to arrange personality traits in a two-dimensional space within a square that has significant representation.

[0083] Figure 4a shows a schematic representation of a personality profile related to the MBTI scheme, and the classification result (INTJ) for the user's profile can be shown in color. This figure intuitively represents the user's profile along various psychological dimensions. An individual classified as "INTJ" is interpreted as a "planner, scientist." Additional personality traits associated with this MBTI type may be output on the user interface.

[0084] In the OCEAN Personality Profile Scheme, scores are assigned to the "Big Five" mindset, including openness, conscientiousness, extraversion, agreeableness, and neuroticism. Figure 3b shows the mapping of the Atmosphere Content Descriptor to the Openness Personality Score in the OCEAN Personality Scheme. The following is an example of how the OCEAN Personality Profile and its scores are represented. “ocean”:{ “agreeableness”:51.10149671582637, “conscientiousness”:73.42223321884429, “extraversion”:33.66403316629877, “neuroticism”:50.21693055551433, “openness”:39.72017677623826}

[0085] Figure 4b shows a schematic representation of the personality profile related to the OCEAN scheme. This figure intuitively represents the user's profile along various psychological dimensions.

[0086] In some embodiments, the personality profile can optionally be enriched with or associated with additional person-related parameters characterized by additional information (e.g., age, gender, and / or biosignals of the human body via body sensors (smartwatches, sports tracking devices, emotion sensors, etc.)). Alternatively, the personality profile can also optionally be enriched with or associated with additional parameters characterizing the individual's context and environment (location, date and time, weather, other people nearby).

[0087] In one embodiment, the personality profiling engine is configured to determine a target group of users for a specific media item, such as a song or video clip. The personality profiling engine may analyze the content of one or more media items (e.g., a song, album, or song collection by an artist) in terms of acoustic attributes, genre, style, atmosphere, etc. It then generates a description of the target group (in the form of a personality profile) for the newly released media item, such as a song, album, or artist, and provides this description to, for example, a music label, artist, music marketing, or sound branding agency.

[0088] The personality profiling engine can not only find profiles for a target group for one or more songs, but can also operate in "reverse mode" to find matching songs for a target group of people. While at least 10 tracks are typically required to calculate a profile, only a single track may be needed to recommend profiles of people who would most resonate with (emotionally respond to) that particular track. When used in "reverse mode," the personality profiling engine can recommend a list of tracks suitable for the selected profile. This allows for the generation of playlists for brands targeting this profile. Furthermore, when used by radio stations, it can calculate the emotional "moment" in a radio program immediately before an ad break and align this moment with the brand and the target audience the brand wishes to emotionally address / generate.

[0089] In one embodiment, the input to the personality profiling engine is a song (or, for example, a set of songs belonging to an album or artist), and the output is a description of a target group (e.g., newly released songs, albums, or artists) for that song. The target group is defined by one or more personality profiles that follow one or more personality profiling schemes such as MBTI, OCEAN, Enneagram, or Ego-Equilibrium. The profiles may optionally be enriched with person-related parameters (e.g., age, gender, etc.).

[0090] More specifically, an analysis of the audio within a set of songs derives a song content descriptor, including a semantic descriptor and / or an emotional descriptor. Optionally, the descriptors are aggregated (in various ways) across many tracks (which may represent an album or artist), and the user's emotional profile is determined, for example, by calculating the average of the mood and / or other descriptors of multiple songs (possibly using an average, RMS, or weighted average). Then, as described above, a mapping from the song content descriptor to a personality profile is performed. The system then outputs profiles for one or more relevant person target groups defined by one of several different personality profile schemes. The target group profiles may be provided in numerical format (e.g., floating-point numbers for different profile scores within the aforementioned schemes). Media Similarity Engine

[0091] In this embodiment, the media similarity engine is configured to select the most suitable music for a given target user group. In this embodiment, the target group is defined, and the media similarity engine selects matching music for, for example, broadcast. This makes it possible to suggest music for, for example, an advertising campaign for a brand defined by that target consumer group. Further possible use cases include in-store music, advertisements, etc.

[0092] In these embodiments, as described above, a target group of people is identified by one or more personality profiles following schemes such as MBTI, OCEAN, Enneagram, or Ego-Equilibrium (intended to find music suitable for the target group in the case of music consumption, in-store music, advertising campaigns, and other use cases). Demographic parameters for the target group may also be added.

[0093] In the personality profile space between the target group profile and the "song personality profiles" of individual songs (i.e., a set of content descriptors for songs mapped to personality profiles according to a personality scheme), a search (e.g., similarity search or exact score matching) can be performed. The "song personality profile" from the songs that best match the target group personality profile is then identified. In this regard, personality profile scores for various personality profile schemes may be pre-calculated for the candidate songs. The best match for the target group of people is then found by a similarity search between the profile score of the defined target group and the personality profile score of each song. Next, different options for similarity searches are described.

[0094] The term "similarity search" encompasses a broad range of mechanisms for searching a vast space of objects (in this case, profiles) based on the similarity between any pair of objects (e.g., profiles). Nearest neighbor search and range queries are examples of similarity search. Similarity search can rely on mathematical concepts of metric spaces that allow for the construction of efficient index structures to achieve scalability in the search domain. Alternatively, non-metric spaces such as the Kullback-Leibler divergence or embeddings learned by neural networks may be used for similarity search. Nearest neighbor search is a form of proximity search and can be expressed as an optimization problem to find the point in a given set that is closest to (or most similar to) a given point. Closeness is usually represented by a dissimilarity function, where the lower the similarity of the objects, the larger the value of the dissimilarity function. In this case, the (dis)similarity of the profiles is the criterion for the search.

[0095] The search for the best-matching media items for a target group may be performed in the personality profile space by comparing the target profile with the personality profile of the media item (for example, the song personality profile for individual songs). This search is performed by • Matching of profile elements (depending on the presence or absence of profile elements), • Matching (numerical search) of profile attribute (score) values, • Searching for a range of values ​​like this (for example, the score "respect" is between 75% and 100%), • Vector-based matching and similarity calculations (calculating how "close" (similar in terms of numerical distance) the values ​​of the target profile and personality profile are by comparing the elements of the numerical profile (e.g., using distance measures such as Euclidean distance, Manhattan distance, cosine distance, or other methods such as Kullback-Leibler divergence)) • Learning similarity based on machine learning (a machine learning or deep learning algorithm learns a similarity function based on examples given to the algorithm (in one embodiment, this learned similarity function can then be used permanently)), It may be performed by [this method].

[0096] Alternatively, a media similarity engine can find songs relevant to a target group of people by mapping personality profile schemes to song content descriptors. Therefore, a mapping from target group personality profiles to song content descriptors ("reverse mapping") may be performed, and a search for songs matching the target profiles may be conducted within the song content descriptor space. In this case, the reverse mapping from target group personality profiles to song content descriptors is performed first, and then songs that best match these content descriptors are selected.

[0097] In either case, the output is a list of media items (e.g., music tracks) that match the specified target group.

[0098] In this embodiment, the media similarity engine uses one or more of the user's personality profile, the user's current situation or context, and the user's current mood, • Real-time recommendations of songs on online streaming platforms. • Suggesting songs on mobile device applications, and / or • Automatic playback of music based on profile (lean-back radio) You may choose to do so.

[0099] For example, as mentioned above, a personality profiling engine analyzes a user's listening history. In this way, the user's personality profile and / or the listener's emotional profile (including the listener's mood) is determined. Next, similar to determining the target group for a particular song, the media similarity engine may be configured to determine and find the song best suited to the individual (user) based on the individual's (long-term) personal music listening history, personality profile, (short-term) mood profile and / or personality profile, a weighted blend of short-term and long-term personality profiles, and, optionally, the user's context and environment information. The individual's context and environment can be determined by other numerical factors measured from a mobile device or other personal user device from which location data, weather data, movement data, biosignal data, etc. can be derived. This may be performed instantaneously while the user is listening in a listening session. For example, based on a preliminary analysis of songs the user has previously listened to, as well as song content descriptors and their mapping to one or more personality profiles, a song that best matches the user's personality profile is selected. Therefore, as explained above, the user's personality profile is compared to a personality profile generated by mapping from a set of media content descriptors. For example, a similarity search is performed between the user's target profile and the song's personality profile to determine the best-matching profile (and corresponding song item) (which may be ranked according to their matching score). The output is a list of songs suggested for listening, which can be updated in real time based on new inputs such as updated listening history.

[0100] Similarly, optionally, a set of songs (e.g., an album, a playlist, or a set of songs by the same artist) may be aggregated (as described above) prior to mapping to a personality profile, thereby recommending artists, albums, or playlists to the listener instead of individual songs.

[0101] Figure 5 shows one embodiment of method 100 for selecting the most suitable music for a given target user group. This method begins in step 110, where identification information for a group of media items, including one or more media items, is obtained. In step 120, a set of media content descriptors is obtained for each of the identified one or more media items in the group. The media content descriptors include characteristics that define the acoustic descriptor, semantic descriptor, and / or emotional descriptor of each media item, and may be calculated directly from the media items or read from a database. Details regarding the generation of media content descriptors are as described above.

[0102] In step 130, a set of aggregated media content descriptors is determined for the entire group of one or more identified media items, based on each media content descriptor of the individual media items. For example, if one or more identified media items correspond to an album or artist, a set of aggregated media content descriptors is determined for this album or artist. If only one media item is identified, a set of aggregated media content descriptors may be determined from the segment of the media item. Then, in step 140, the set of aggregated media content descriptors (e.g., the user's sentiment profile) is mapped to the personality profile defined according to the personality scheme as described above. This mapping may be based on mapping rules. In step 150, the generated personality profile for the group of media items is provided to the media similarity engine. The above process is repeated for another group of media items, and a different personality profile is generated for the other group of media items. In this way, each media item is associated with a corresponding group of media items, and multiple personality profiles are generated that characterize each media item with respect to the applied personality scheme.

[0103] In step 160, the personality profile of the media item group is compared with the target personality profile to determine at least one media item with the best matching personality profile. The target personality profile corresponds to a target group of users, including one or more users, and can be determined from the user's media consumption history as described above. In step 170, at least one media item group with the best matching personality profile is selected for playback or recommendation to the user or user group. Finally, in step 180, the system outputs a list of tracks, artists, or albums that are matched with the personality profile of the target user group, along with a matching score (a value indicating the degree of matching for each output item). The calculation of the matching score may be performed by similarity search as described above.

[0104] The characteristics of the above-described apparatus (devices, systems) correspond to the characteristics of each method, but it should be noted that for the sake of simplification, they may not be explicitly described. It is understood that the disclosures in this specification also extend to such characteristics of methods. In particular, it should be understood that this disclosure relates to methods for operating the above-described devices and / or to the provision and / or arrangement of the individual elements of these devices.

[0105] Furthermore, it should be noted that the exemplary embodiments of the disclosure can be realized in many ways by using hardware and / or software configurations. For example, the embodiments of the disclosure can be realized by using hardware capable of running dedicated hardware and / or software. The components and / or elements in the figures are illustrative only and do not limit the scope of use or functionality of any hardware, software combined with hardware, firmware, embedded logic components, or combinations of two or more such components that realize a particular embodiment of the disclosure.

[0106] Furthermore, it should be noted that this specification and drawings merely illustrate the principles of the disclosure. Those skilled in the art will be able to realize various configurations that embody the principles of the disclosure and that fall within its conception and scope, although these are not explicitly described or illustrated herein. Moreover, all examples and embodiments outlined in this disclosure are explicitly intended solely for illustrative purposes to assist the reader in understanding the principles of the proposed method. Furthermore, all descriptions in this specification that provide the principles, aspects, and embodiments of the disclosure, as well as their specific examples, are intended to encompass their respective equivalents. Glossary

[0107] The following technical terms are used throughout this specification.

[0108] media Media includes all kinds of media items that can be presented to the user, such as audio (especially music) and video (including embedded audio tracks). Furthermore, photographs, series of photographs, slides, and graphic representations are examples of media items.

[0109] Media content descriptor Media content descriptors (commonly known as "characteristics") are calculated by analyzing the content of a media item. Music content descriptors (commonly known as "music characteristics") are calculated by analyzing digital audio (segments (excerpts) or the entire song). These consist of a set of music content descriptors that include atmosphere, genre, context, acoustic attributes (key, tempo, energy, etc.), vocal attributes (presence of vocals, vocal family, vocal gender (low or high)), etc. Each of these includes the range of a descriptor or characteristic. Characteristics are defined by a name and a floating-point or percentage value (e.g., bpm: 128.0, energy: 100%).

[0110] Song A musical piece is an example of a media item that consists of single lines (melody) or double lines (harmony) and represents audio data containing timbres or sounds produced by one or more voices or instruments, or both. A media content descriptor for a musical piece is also called a musical content descriptor or musical profile.

[0111] Emotional Profile An emotional profile can be determined for many media items, and it includes one or more sets of media or musical content descriptors associated with a mood or emotion. In this case, it is a collection of content descriptors for individual media items. These are typically derived by aggregating media / musical content descriptors from a set of media items associated with a person or individual (for example, consumed by that person or individual). These contain the same elements as the media / musical content descriptors, and their values ​​are determined by aggregating the individual content descriptors (depending on the aggregation method used).

[0112] Person (User): Emotional profile and personality profile A person (also referred to as a user or individual) is characterized by an emotional profile or personality profile. An emotional profile is characterized by elements of a media content descriptor (see above). A personality profile, on the other hand, includes many different elements with percentage values. The elements of a personality profile are weighted elements within a personality profile scheme (defined by a name or attribute and a percentage value, e.g., MBTI: "EI: 51%"). Personality profiles are defined by personality profile schemes such as MBTI, OCEAN, and Enneagram. - User mood (instantaneous, short-term) (i.e., a personality profile interpreted as the user's short-term emotional status (also called the user mood profile)), or - User personality type (long term) (i.e., a personality profile derived from long-term observation of the user's media consumption behavior), This may be related.

[0113] Target group The target group represents a group of individuals. This is identified as one or a combination of "personality profiles." Optionally, it may be enriched with individual-related parameters (e.g., age, gender, etc.).

[0114] product A product profile includes product attributes expressed in a psychological, emotional, or marketing manner. These attributes may be associated with a percentage value of importance.

[0115] brand Product profiles may be related to a brand. Brand profiles include brand attributes expressed in psychological, emotional, or marketing terms. Attributes may be associated with percentage values ​​of importance.

[0116] mapping Mapping is an algorithmically implemented set of rules that translate profiles from one entity (e.g., media item, song) to another entity (e.g., person, product, or brand) (or vice versa). For example, mapping can be applied between a set of content descriptors (emotional profiles) and personality profiles from a personality profiling scheme.

[0117] Similarity search Similarity search is an algorithmic procedure that calculates the similarity, proximity, or distance between two or more "profiles" of any kind (e.g., emotion profiles, personality profiles, product profiles). The output is a ranked list of profile items with matching scores (values ​​indicating the degree of matching between profiles).

Claims

1. A method for providing a personality profile, - Obtain identification information for one or more media items, - Obtaining a set of media content descriptors for each of the one or more identified media items, wherein the set of media content descriptors includes characteristics including a semantic descriptor for each of the media items, and the semantic descriptor includes at least one emotion descriptor for each of the media items. - Based on each of the media content descriptors of the individual media items, determine a set of aggregated media content descriptors for the identified one or more media items as a whole, - The method involves mapping the set of aggregated media content descriptors to the personality profile, wherein the personality profile includes a plurality of personality scores for the elements of the profile, and the personality scores are calculated from the aggregated characteristics of the set of aggregated media content descriptors. - To provide the personality profile corresponding to one or more media items, Methods that include...

2. The method according to claim 1, wherein the media item includes a musical portion, and is preferably a musical piece.

3. The method according to claim 1 or 2, wherein the identification information of one or more media items includes a playlist of a user or user group.

4. The method according to claim 1 or 2, wherein the identification information of one or more media items includes the user's short-term media consumption history, and the personality profile characterizes the user's current mood.

5. The method according to claim 1 or 2, wherein the one or more identified media items correspond to an album or artist.

6. The method according to any one of claims 1 to 5, wherein the set of media content descriptors for a media item includes one or more acoustic descriptors for the media item determined based on an acoustic analysis of the media item.

7. The method according to any one of claims 1 to 6, wherein the set of media content descriptors for a media item is determined based on an artificial intelligence model that determines one or more semantic descriptors and / or emotion descriptors for the media item.

8. The method according to claim 7, wherein the one or more semantic descriptors include at least one of genre, presence of voice, gender of voice, vocal pitch, mood of song, and rhythmic mood.

9. The method according to any one of claims 1 to 8, wherein a segment of a media item is analyzed, and the set of media content descriptors for the media item is determined based on the results of the segment analysis.

10. The method according to any one of claims 1 to 9, wherein the step of obtaining a set of media content descriptors for each of the identified media items includes reading the set of media content descriptors for a media item from a database.

11. The method according to any one of claims 1 to 10, wherein the step of determining a set of aggregated media content descriptors includes calculating an aggregated numerical characteristic from each of the numerical characteristics of the identified media items.

12. The method according to any one of claims 1 to 11, wherein the personality profile is based on a personality scheme that defines many personality scores for profile elements representing personality traits.

13. The method according to any one of claims 1 to 12, wherein the personality score of the personality profile is determined based on a mapping rule that defines a method for calculating the personality score from a set of aggregated media content descriptors.

14. The method according to claim 13, wherein the mapping rule is learned by machine learning technology.

15. The method according to any one of claims 1 to 14, wherein the personality score of the personality profile is determined based on the weighted aggregate numerical characteristics of the identified media items.

16. The method according to any one of claims 1 to 15, wherein the personality score of the personality profile is determined based on the presence or absence of the aggregated characteristics of the identified media items.

17. The method according to any one of claims 1 to 16, wherein providing the personality profile includes displaying a graphic representation of the personality profile or transmitting the personality profile to a database server.

18. The method according to any one of claims 1 to 17, wherein the personality profile is classified into one of a plurality of personality types.

19. The method according to any one of claims 1 to 18, wherein a user's personality profile is determined from a playlist identifying the user's long-term media item usage history, and a user's mood profile is determined from the user's short-term media consumption history, and the method further comprises calculating the difference between the user's personality profile and mood profile.

20. The method according to any one of claims 1 to 19, wherein a separate personality profile is provided for each of a plurality of media items, and the method further comprises comparing the personality profile of the media items with a target personality profile to determine at least one media item having the best-matching personality profile.

21. The method according to claim 20, wherein comparing profiles is based on matching profile elements and selecting a personality profile of a media item having the same or similar elements as the target personality profile.

22. The method according to claim 20, wherein comparing profiles is based on a similarity search in which corresponding scores of profiles are compared and a matching score indicating the similarity between each pair of profiles is calculated.

23. The method according to claim 22, further comprising ranking the personality profiles of the media items according to their matching scores.

24. The method according to any one of claims 20 to 23, wherein the target personality profile corresponds to a group of users or individual users.

25. The method according to claim 24, wherein the user's target personality profile is determined by the method according to any one of claims 1 to 18, based on identification information of one or more suitable media items for the user.

26. The method according to claim 25, wherein one of the identified preferred media items is the media item that was last consumed by the user.

27. The method according to any one of claims 20 to 26, wherein at least one of the determined media items is selected for playback or recommendation to the user.

28. The method according to any one of claims 20 to 27, wherein information associated with at least one of the determined media items is provided to the user or a user device associated with the user.

29. The method according to any one of claims 20 to 28, wherein the personality profile of the media item is compared with a target personality profile and the determination of at least one media item having the best matching personality profile is performed repeatedly.

30. The personality profile is generated on the server platform, and the method is Transmitting the identification information of one or more suitable media items for the user from the user device associated with the user to the server platform, The user device receives a representation of at least one determined media item, The method according to any one of claims 25 to 29, further comprising:

31. The method according to any one of claims 25 to 30, wherein the identification information of one or more preferred media items for the user is stored on a server platform, the personality profile is generated on the server platform, and the method further comprises transmitting a representation of at least one determined media item to a user device associated with the user.

32. A computer device having a memory and a processor configured to perform the method described in any one of claims 1 to 31.

Images