Music recommendation system and control method for music recommendation system

Abstract

A music recommendation system according to one embodiment of the present invention includes: a text receiving unit for receiving diary text input to a user terminal; an emotion extraction unit for extracting a diary emotion vector for the diary text using an artificial intelligence model on the basis of the diary text; and a music recommendation unit for determining a piece of recommended music information corresponding to the diary text among pieces of music information in a database on the basis of the diary emotion vector.

Description

Music recommendation system and control method of music recommendation system

[0001] The present invention relates to a music recommendation system that recommends music based on diary contents.

[0002] The present invention was derived from research conducted as part of the Ministry of Science and ICT's Information and Communication Broadcasting Innovation Talent Training Project (Project Unique Number: 1711179353, Project Number: 00156360, Research Project Name: Regional Intelligence Innovation Talent Training, Project Management Agency: Information and Communication Planning Evaluation Institute, Project Execution Agency: Soongsil University Industry-Academic Cooperation Foundation, Research Period: January 1, 2023 - December 31, 2023). Meanwhile, the Korean government has no property interest in any aspect of the present invention.

[0003] Recently, text analysis algorithms or AI models have become widely used. For example, ChatGPT, a language model based on the GPT-3.5 architecture, is an AI model pretrained on large-scale data for natural language processing and text understanding. These text analysis models demonstrate high performance in many natural language processing tasks and can perform a variety of text analysis-related tasks.

[0004] Text analysis involves understanding text data and extracting meaning. This can involve a variety of tasks, including text classification, sentiment analysis, summarization, translation, named entity recognition, and relationship extraction.

[0005] The present invention provides a music recommendation system that can provide satisfaction to users by analyzing accurate emotions based on emotional diaries and recommending music appropriate for the emotions.

[0006] In addition, the present invention provides a music recommendation system that is convenient for users compared to the conventional method of analyzing emotions by having users directly input their emotions, and that can solve the problem that users may not be honest in expressing their emotions.

[0007] In addition, the present invention provides a music recommendation system that does not cause the problem of not being able to perform accurate sentiment analysis because the interpretation of color is different in the conventional method of analyzing the psychological health state of a user by focusing on the color of a picture.

[0008] In addition, the present invention provides a music recommendation system that allows users to reduce stress by having time to look back on and organize their day while writing a diary and receiving music recommendations.

[0009] In addition, the present invention provides a music recommendation system that can increase accessibility by utilizing text rather than pictures and eliminate entry barriers through the medium of music, thereby enabling expansion of service areas in various content fields.

[0010] In addition, the present invention provides a music recommendation system capable of recommending music that takes into account both the user's musical taste and emotions.

[0011] In addition, the present invention provides a music recommendation system that can improve performance, such as increasing data processing speed or reducing required memory capacity, compared to conventional systems.

[0012] A music recommendation system according to one aspect of the disclosed invention may include a text receiving unit that receives a diary text input into a user terminal; an emotion extraction unit that extracts a diary emotion vector for the diary text using an artificial intelligence model based on the diary text; and a music recommendation unit that determines recommended music information corresponding to the diary text from among music information in a database based on the diary emotion vector.

[0013] In addition, the present invention may further include a display control unit that controls the display of the user terminal to display the recommended music information.

[0014] In addition, the music recommendation unit determines, as the recommended music information, music information in which a music emotion vector of a value corresponding to the diary emotion vector is set among music information in which a music emotion vector is preset one by one in the database, and the diary emotion vector of the diary text may include vector values ​​for each of a plurality of emotions to represent the emotion of the writer who entered the diary text, and the music emotion vector may include vector values ​​for each of a plurality of emotions to represent the emotion of music.

[0015] In addition, the method may further include a machine learning unit that learns the artificial intelligence model through a machine learning method by setting the learning diary text as an input variable and the vector values ​​included in the diary sentiment vector for the learning diary text as output variables.

[0016] In addition, among the music information of the database, the method may further include a vector unset music information acquisition unit that acquires music information of vector unset music for which the music emotion vector is not set; and a music emotion vector labeling unit that sets the music emotion vector of the vector unset music based on a diary emotion vector extracted from a diary text input through the user terminal while the vector unset music is played through the user terminal.

[0017] In addition, the method further includes a user-played music information acquisition unit that receives a music list capture image including a list of music titles or a list of singer names, and the music recommendation unit determines the recommended music information based on the emotional vector and the user-played music information included in the music list capture image, and the music list capture image may be a screen image captured in a state where at least one of music titles and singer names of music played by the user terminal is displayed on the display of the user terminal.

[0018] In addition, the user playback music information acquisition unit can acquire the user playback music information by extracting the text of the music information from the music list capture image.

[0019] In addition, the music recommendation unit: sets a first music score representing a degree to which a user of the user terminal is predicted to prefer, based on the user-played music information, to correspond to each piece of music information in the database; and based on the diary sentiment vector and the first music score set for each piece of music information in the database, the music information corresponding to the diary text can be determined from among the music information in the database.

[0020] In addition, the music recommendation unit: sets a second music score indicating the degree to which the music emotion vector of the music information is similar to the diary emotion vector so that each second music score corresponds to each piece of music information in the database; and based on a value obtained by multiplying the first music score set for each piece of music information by a first weight and a value obtained by multiplying the second music score set for each piece of music information by a second weight, the music recommendation unit can determine recommended music information corresponding to the diary text from among the pieces of music information in the database.

[0021] A method for controlling a music recommendation system according to an aspect of the disclosed invention comprises the steps of: receiving a diary text input to a user terminal by a text receiving unit; extracting a diary sentiment vector for the diary text using an artificial intelligence model by a sentiment extraction unit based on the diary text; receiving a music list capture image including a list of music titles or a list of singers by a user playback music information obtaining unit; determining recommended music information corresponding to the diary text from music information in a database based on the diary sentiment vector by a music recommendation unit; and controlling a display of the user terminal to display the recommended music information by a display control unit, wherein the step of determining the recommended music information comprises: determining the recommended music information by the music recommendation unit based on the sentiment vector and user playback music information included in the music list capture image; setting, by the music recommendation unit, a first music score representing a degree to which a user of the user terminal is predicted to prefer, based on the user playback music information, so that each first music score corresponds to each piece of music information in the database; A step of setting a second music score, which indicates the degree to which the music emotion vector of the music information is similar to the weather emotion vector, by the music recommendation unit, to correspond to each piece of music information in the database;And, by the music recommendation unit, a step of determining recommended music information corresponding to the diary text from among the music information of the database based on a value obtained by multiplying the first music score set for each of the music information by a first weight and a value obtained by multiplying the second music score set for each of the music information by a second weight, wherein the diary emotion vector of the diary text includes vector values ​​for each of a plurality of emotions to represent the emotion of the writer who entered the diary text, and the music emotion vector includes vector values ​​for each of a plurality of emotions to represent the emotion of music, and the music list capture image may be a screen image captured in a state where at least one piece of music information among music titles and singer names of music played by the user terminal is displayed on the display of the user terminal.;

[0022] A computer program according to one aspect of the disclosed invention can be stored in a computer-readable, non-transitory recording medium to execute a control method of the music recommendation system.

[0023] According to one aspect of the disclosed invention, it is possible to provide satisfaction to users by analyzing accurate emotions based on emotional diaries and recommending music appropriate for the emotions.

[0024] In addition, according to an embodiment of the present invention, it is convenient for users compared to the conventional method of analyzing emotions by having users directly input their emotions, and it can solve the problem that users may not be honest in expressing their emotions.

[0025] In addition, according to an embodiment of the present invention, in the conventional method of analyzing a user's psychological health state by focusing on the color of a picture, the problem of not being able to perform accurate emotional analysis because the interpretation of color is different may not occur.

[0026] Additionally, according to an embodiment of the present invention, a user can reduce stress by having time to look back on and organize the day while writing a diary and receiving music recommendations.

[0027] In addition, according to an embodiment of the present invention, accessibility can be increased by utilizing text rather than pictures, and entry barriers can be eliminated through the medium of music, so expansion of service areas in various content fields is expected.

[0028] Additionally, according to an embodiment of the present invention, music recommendations that take into account both the user's music taste and emotions may be possible.

[0029] In addition, according to an embodiment of the present invention, performance such as an increase in data processing speed or a decrease in required memory capacity can be improved compared to a conventional system.

[0030] FIG. 1 is a diagram illustrating a music recommendation system according to one embodiment.

[0031] FIG. 2 is a diagram illustrating a weather text according to one embodiment.

[0032] FIG. 3 is a table showing the types of vector values ​​included in a weather sentiment vector according to one embodiment.

[0033] Figure 4 is a diagram for explaining a process of using an artificial intelligence model according to one embodiment.

[0034] FIG. 5 is a diagram illustrating learning data used to learn an artificial intelligence model according to one embodiment.

[0035] FIG. 6 is a diagram for explaining a process of setting a music emotion vector for music according to one embodiment.

[0036] FIG. 7 is a diagram illustrating examples of screens on which recommended music information is output according to one embodiment.

[0037] FIG. 8 is a drawing illustrating a music list capture image according to one embodiment.

[0038] FIG. 9 is a diagram illustrating a method of using optical character recognition technology according to one embodiment.

[0039] FIG. 10 is a diagram illustrating a method for determining recommended music information according to one embodiment.

[0040] Fig. 11 is a flowchart of a control method of a music recommendation system according to one embodiment.

[0041] Like reference numerals refer to like elements throughout the specification. This specification does not describe all elements of the embodiments, and general information within the technical field to which the disclosed invention pertains or information that overlaps between the embodiments is omitted.

[0042] Additionally, when a part is said to "include" a component, this does not mean that it excludes other components, but rather that it may include other components, unless otherwise specifically stated.

[0043] As used herein, the term "~unit" refers to a unit that processes at least one function or operation, and may refer to, for example, software, an FPGA, or a hardware component. The function provided by the "~unit" may be performed separately by multiple components, or may be integrated with other additional components. The "~unit" of this specification is not necessarily limited to software or hardware, and may be configured to be located in an addressable storage medium, or may be configured to play one or more processors. According to embodiments, multiple "~units" may be implemented as a single component, or a single "~unit" may include multiple components.

[0044] The terms first, second, etc. are used to distinguish one component from another, and the components are not limited by the aforementioned terms.

[0045] Singular expressions include plural expressions unless the context clearly indicates otherwise.

[0046] The identification codes for each step are used for convenience of explanation and do not describe the order of each step. Each step may be performed in a different order than specified unless the context clearly indicates a specific order.

[0047] The operating principle and embodiments of the disclosed invention will be described with reference to the attached drawings below.

[0048] FIG. 1 is a diagram illustrating a music recommendation system according to one embodiment.

[0049] Referring to FIG. 1, the music recommendation system (100) may be a system configured to recommend music that matches the user's emotions based on diary contents entered by the user.

[0050] The music recommendation system (100) is provided on a server and can communicate with at least one user terminal (200) via wired or wireless means. The user terminal (200) may be a terminal used by a user who wishes to obtain information on music that matches the user's emotions or the contents of a diary written by the user by using the music recommendation system (100). The user terminal (200) may be a terminal used by the user to directly write his / her own diary, but is not limited thereto. The user terminal (200) used to write the diary and the user terminal (200) used to check the recommended music information (500) may be different terminals, and the contents of the diary text (300) written by the user may not be received from the user terminal (200), but may receive the contents of the diary text that is pre-stored on the server.

[0051] Music information can be information about any type of music. For example, music information about a piece of music can include the title, the name of the singer, the name of the composer, etc.

[0052] Recommended music information (500) may be music information of music to be recommended to the user.

[0053] Meanwhile, a music recommendation system (100) according to one embodiment may be provided on a server, receive a diary text (300) from a user terminal (200) through communication with the user terminal (200), and transmit recommended music information (500) to the user terminal (200). However, the method by which the music recommendation system (100) provides recommended music information (500) is not limited thereto. For example, when the user terminal (200) downloads the music recommendation system (100) by downloading an application from a server, the music recommendation system (100) may be provided on the user terminal (200).

[0054] That is, as long as the user can obtain recommended music information (500) through the user terminal (200), it does not matter which terminal or server the music recommendation system (100) is installed on or how it is operated.

[0055] A music recommendation system (100) may include a text receiving unit (110), an emotion extraction unit (120), a music recommendation unit (130), a display control unit (140), a vector non-setting music information acquisition unit (150), a memory (160), an artificial intelligence model (161), a user-played music information acquisition unit (170), a music emotion vector labeling unit (180), and a machine learning unit (190).

[0056] FIG. 2 is a diagram illustrating a weather text according to one embodiment.

[0057] Referring to FIGS. 1 and 2, the text receiving unit (110) may be configured to receive a diary text (300) input into a user terminal (200).

[0058] At this time, the text receiving unit (110) may receive the diary text (300) by receiving data in text format directly input into the user terminal (200), or may receive data in text format obtained by extracting text from an image using OCR (optical character recognition) technology from a handwritten diary captured by the camera of the user terminal (200).

[0059] If the music recommendation system (100) is provided in the user terminal (200), the text receiving unit (110) can receive the diary text (300) written in hand and captured by the camera of the user terminal (200) or directly from the user terminal memory. If the music recommendation system (100) is provided in the server, the text receiving unit (110) can also receive the diary text (300) from the user terminal (200) or the server memory. In other words, it does not matter how the text receiving unit (110) receives the diary text (300).

[0060] The text receiving unit (110) can transmit the diary text (300) to the emotion extraction unit (120).

[0061] FIG. 3 is a table illustrating the types of vector values ​​included in a weather sentiment vector according to one embodiment, and FIG. 4 is a diagram for explaining a process of using an artificial intelligence model according to one embodiment.

[0062] Referring to FIGS. 1, 3, and 4, the emotion extraction unit (120) can extract a diary emotion vector for the diary text (300) using an artificial intelligence model (161) based on the diary text (300). The emotion extraction unit (120) can transmit the diary emotion vectors for each diary text (300) to the music recommendation unit (130).

[0063] The diary sentiment vector may be a bundle of vector values ​​corresponding to each diary text (300). The diary sentiment vector of a single diary text (300) may include vector values ​​for each of a plurality of emotions to represent the emotions of the writer who entered the diary text (300). For example, the diary sentiment vector may include vector values ​​for each of six types of emotions, such as joy, anxiety, anger, embarrassment, sadness, and hurt, but the types and number of vectors included in the diary sentiment vector are not limited thereto.

[0064] The music recommendation system (100) can analyze the user's sentiment from the diary entered by the user through an artificial intelligence model (161). At this time, a refinement process for the diary data needs to be performed first. The music recommendation system (100) can refine the input diary data to only accept Korean characters using regular expressions, but is not limited thereto, and foreign languages ​​can also be input. The music recommendation system (100) can perform a spell check on the input data using the Naver spell check API to refine spelling errors, but the spelling refinement method is not limited thereto. The refined data can be used as input to an artificial intelligence model to analyze the sentiment of the diary. At this time, prior to analyzing the sentiment of the diary, a pre-learning model specialized in Korean (e.g., KoELECTRA) can be improved by performing transfer learning so that the model is specialized in sentiment analysis.

[0065] The database (400) can store information about music that may be recommended to users. The database (400) may be located on a server or within the music recommendation system (100). However, it may also be located separately from the music recommendation system (100) and transmit and receive information with the music recommendation system (100).

[0066] The music recommendation unit (130) can determine recommended music information (500) corresponding to the diary text (300) from among the music information in the database (400) based on the diary sentiment vector.

[0067] Specifically, the music recommendation unit (130) can determine, as recommended music information (500), music information in which a music emotion vector corresponding to a weather emotion vector is set among music information in which each music emotion vector is preset in the database (400).

[0068] A music emotion vector may include vector values ​​for each of multiple emotions to represent the emotional nature of the music. For example, a music emotion vector may include vector values ​​for each of six types of emotions: joy, anxiety, anger, embarrassment, sadness, and hurt. However, the types and number of vectors included in a music emotion vector are not limited to this.

[0069] FIG. 5 is a diagram illustrating learning data used to learn an artificial intelligence model according to one embodiment.

[0070] Referring to FIG. 5, examples of some sentences of a learning diary text (300) that can be used to learn an artificial intelligence model (161) and the vector types of diary sentiment vectors set to correspond thereto can be confirmed.

[0071] Referring to FIGS. 1 and 5, the machine learning unit (190) can learn an artificial intelligence model (161) through a machine learning method by using a learning diary text as an input variable and setting vector values ​​included in a diary sentiment vector for the learning diary text as output variables.

[0072] Machine learning utilizes models composed of multiple parameters and can mean optimizing those parameters based on given data. Depending on the type of learning problem, machine learning can include supervised learning, unsupervised learning, and reinforcement learning. Supervised learning learns mappings between inputs and outputs and is applicable when input-output pairs are given as data. Unsupervised learning is applicable when there are only inputs and no outputs, and can identify patterns between inputs, etc.

[0073] The machine learning unit (190) can generate an artificial intelligence model (161) in various ways. For example, the machine learning unit (190) can learn features extracted from learning diary text using a deep learning-based learning method. At this time, a CNN (Convolutional Neural Networks) structure that stacks multiple stages of convolution layers can be utilized to learn a method of extracting features from learning diary text. However, the learning method of the machine learning unit (190) is not necessarily limited to a method utilizing the CNN structure. For example, the learning method of the machine learning unit (190) can be a method through a machine learning algorithm including an artificial neural network (ANN) or a recurrent neural network (RNN).

[0074] The artificial intelligence model (161) may be, but is not limited to, a Transformer-based pre-training model. A Transformer-based pre-training model performs natural language processing and machine translation tasks, and can be a model with higher learning speed and efficiency than models based on a Recurrent Neural Network (RNN), as it is faster and allows for parallel processing. The Transformer-based pre-training model is based on an attention mechanism, which allows it to understand the overall context of the input sequence and focus on important information. While existing recurrent models, such as RNN and LSTM (Long Short-Term Memory), tend to lose information as sentences become longer, the Transformer-based pre-training model can solve this problem and achieve higher performance.

[0075] An AI model (161) according to one embodiment may be the KoELECTRA model. The KoELECTRA model may be a pre-trained model using a large amount of unlabeled Korean data based on contrastive learning. KoELECTRA is a Korean version of the ELECTRA (Efficiently Learning an Encoder that Classifies Token Replacements Accurately) model, and may be a model that learns more efficiently and demonstrates higher performance than existing language models. ELECTRA is a model that applies the concept of a Generative Adversarial Network (GAN) to natural language processing and may consist of two parts: a Generator and a Discriminator. The Generator may perform learning by masking input tokens to restore them, and the Discriminator may perform learning by discriminating tokens generated by the Generator. This learning method can effectively utilize unlabeled data to obtain highly expressive latent representations. KoELECTRA has led to various performance improvements in Korean natural language processing tasks and is being utilized in various tasks such as natural language understanding, text classification, and sentiment analysis.

[0076] The artificial intelligence model (161) can be stored in the memory (160) of the music recommendation system (100).

[0077] FIG. 6 is a diagram for explaining a process of setting a music emotion vector for music according to one embodiment.

[0078] In order to recommend music that is close to the emotion desired by the user, musical emotion information for each song may be required.

[0079] Referring to Figure 6, it can be confirmed that the emotions (anger, sadness, anxiety, hurt, embarrassment, joy) corresponding to each song are stored through 'emotion-specific tags' that distinguish the playlist that the user wants when the user has a specific emotion for the music, and that the music embedding vector is generated based on this to correspond to each song.

[0080] For each song, music embedding vectors can be preset by the user or the singer. However, if a music embedding vector has not been preset for a particular song, the question arises as to which music embedding vector to set for that song and how to set it.

[0081] The vector non-set music information acquisition unit (150) can acquire music information of vector non-set music for which a music emotion vector has not been set among the music information of the database (400). That is, the vector non-set music information acquisition unit (150) can determine music information for which a music emotion vector has not been set among the music information of the database (400) as vector non-set music information. The vector non-set music information acquisition unit (150) can transmit the vector non-set music information to the emotion extraction unit (120) or the emotion vector labeling unit.

[0082] The emotion extraction unit (120) can set a music emotion vector for vector-unset music using an artificial intelligence model (161) based on the lyrics of vector-unset music.

[0083] Meanwhile, the method of setting the musical emotion vector for music with no vector setting is not limited to this.

[0084] The music emotion vector labeling unit (180) can set the music emotion vector of the vector-unset music based on the diary emotion vector extracted from the diary text (300) input through the user terminal (200) while the vector-unset music is played through the user terminal (200).

[0085] FIG. 7 is a diagram illustrating examples of screens on which recommended music information is output according to one embodiment.

[0086] Referring to FIG. 1 and FIG. 7, the display control unit (140) can control the display of the user terminal (200) to display recommended music information (500).

[0087] The display control unit (140) can control the display so that the contents of the diary text (300) and the recommended music information (500) corresponding to the diary text (300) are simultaneously displayed on the screen. At this time, the display can display the contents of the diary, the music title of the recommended music, the name of the singer, etc.

[0088] Meanwhile, it may be desirable to decide on the music recommended to the user not simply by referring to the contents of the diary, but by considering the music that the user frequently listens to.

[0089] FIG. 8 is a drawing illustrating a music list capture image according to one embodiment.

[0090] Referring to FIG. 8, a music playlist can be extracted from a screenshot of a music playlist provided by a user. At this time, text can be extracted using the Naver Clova OCR API, and the music title and singer name can be preprocessed using regular expressions, and then the corresponding information can be stored in a database (400), but the present invention is not limited thereto.

[0091] Referring to FIG. 1 and FIG. 8, the user playback music information acquisition unit (170) can receive a music list capture image (600) including a list of music titles or a list of singer names.

[0092] The music list capture image (600) may be a screen image captured while at least one piece of music information among the music title and singer name of music played by the user terminal (200) is displayed on the display of the user terminal (200).

[0093] The user-played music information may be music information of music played through the user terminal (200).

[0094] The user playback music information acquisition unit (170) can transmit a music list capture image (600) to the music recommendation unit (130).

[0095] The music recommendation unit (130) can determine recommended music information (500) based on the user-played music information included in the emotional vector and music list capture image (600).

[0096] FIG. 9 is a diagram illustrating a method of using optical character recognition technology according to one embodiment.

[0097] Referring to FIG. 9, the music recommendation system (100) can recognize a character area and receive text input through optical character recognition (OCR).

[0098] Optical character recognition may be a technology that recognizes a character area in an image and converts the image of the recognized area into characters. In other words, optical character recognition may be a technology that extracts character information from an image. In one embodiment, a text receiving unit (110) that performs diary retrieval and a user playback music information obtaining unit (170) that performs user music playlist retrieval may utilize OCR to identify character content in an image. In one embodiment, optical character recognition may utilize NAVER Clova ai's NAVER OCR API, but is not limited thereto.

[0099] The user playback music information acquisition unit (170) can acquire user playback music information by extracting the text of music information from the music list capture image (600).

[0100] FIG. 10 is a diagram illustrating a method for determining recommended music information according to one embodiment.

[0101] Referring to FIG. 10, the music recommendation system (100) can perform music recommendation through two methods.

[0102] The music recommendation system (100) may identify music that a user may be interested in through a collaborative filtering model. This model can infer a value (first music score) that the user is likely to be interested in for numerous music pieces. According to one embodiment, the collaborative filtering model used may be RecVAE [7], but is not limited thereto. RecVAE demonstrates performance close to state-of-the-art for various recommendation-related datasets, and may be a model that demonstrates particularly good performance even for datasets with high data sparsity. RecVAE may be largely composed of an encoder and a decoder. The encoder may convert input music consumption information into a latent vector, and the decoder may convert the latent vector into a vector containing scores (first music scores) for each song. In the vector containing the converted scores (first music scores), scores for songs already consumed by the user are converted to the minimum value, thereby preventing recommendations for songs already consumed. Model training can be performed by dividing 20% ​​of the dataset into a validation set and 10% into a test set.

[0103] The music recommendation system (100) can find music with similar emotions by using a music embedding vector created using the collected music emotion data. Specifically, the music recommendation system (100) calculates the similarity between the embedding vectors created through multi-hot encoding between the daily emotion vector from the artificial intelligence model (161), which is a emotion analysis model, and the music emotion vector corresponding to the emotional tag of the music, and uses the calculated similarity as a score for each music (second music score).

[0104] As a method for calculating the similarity between vectors, the commonly used cosine similarity, as in [Equation 1], can be used, but is not limited to this.

[0105]

[0106] [Equation 1]

[0107]

[0108]

[0109] The music recommendation system (100) can recommend the top five songs by multiplying the scores for each song (first music score and second music score) from the two methods described above by a weight and sorting the results in descending order.

[0110] Referring to FIG. 1 and FIG. 10, the music recommendation unit (130) can set a first music score, which indicates the degree to which the user of the user terminal (200) is predicted to prefer, to correspond to each piece of music information in the database (400), based on the user-played music information.

[0111] The music recommendation unit (130) can determine recommended music information (500) corresponding to the diary text (300) from among the music information in the database (400) based on the first music score set for each piece of music information in the diary sentiment vector and the database (400).

[0112] The music recommendation unit (130) can set a second music score that indicates the degree to which the music emotion vector of the music information is similar to the weather emotion vector so that each music information in the database (400) corresponds to it.

[0113] The music recommendation unit (130) can determine recommended music information (500) corresponding to the diary text (300) from among the music information in the database (400) based on a value obtained by multiplying the first music score set for each piece of music information by a first weight and a value obtained by multiplying the second music score set for each piece of music information by a second weight. At this time, the first weight and the second weight can be preset by an administrator or user of the music recommendation system (100).

[0114] The music recommendation unit (130) sets the calculated value by multiplying the first music score set for each piece of music information by the first weight and adding the value obtained by multiplying the second music score set for each piece of music information by the second weight as the final score value for each piece of music, and for all pieces of music in the database (400), the final score values ​​of each piece of music are sorted in descending order, and the music information of the pieces of music with a preset number of top scores can be determined as recommended music information (500) so as to recommend the music to the user.

[0115] At least one component may be added or deleted in response to the performance of the components described above. Furthermore, those skilled in the art will readily understand that the relative positions of the components may be altered in response to the performance or structure of the system.

[0116] Figure 11 is a flowchart of a control method for a music recommendation system according to one embodiment. This is merely a preferred embodiment for achieving the purpose of the present invention, and it is understood that certain components may be added or deleted as needed.

[0117] Referring to FIG. 11, the text receiving unit (110) can receive a diary text (300) entered into the user terminal (200) (1001).

[0118] The emotion extraction unit (120) can extract a diary emotion vector for the diary text (300) using an artificial intelligence model (161) based on the diary text (300) (1002).

[0119] The user playback music information acquisition unit (170) can receive a music list capture image (600) including a list of music titles or a list of singer names (1003).

[0120] The music recommendation unit (130) can set a first music score, which indicates the degree to which the user of the user terminal (200) is expected to prefer, to correspond to each piece of music information in the database (400) based on the user-played music information (1004).

[0121] The music recommendation unit (130) can set a second music score indicating the degree to which the music emotion vector of the music information is similar to the weather emotion vector so that each piece of music information in the database (400) corresponds to it (1005).

[0122] The music recommendation unit (130) can determine recommended music information (500) corresponding to the diary text (300) among the music information in the database (400) based on a value obtained by multiplying the first music score set for each piece of music information by a first weight and a value obtained by multiplying the second music score set for each piece of music information by a second weight (1006).

[0123] The display control unit (140) can control the display of the user terminal (200) to display recommended music information (500) (1007).

[0124] The text receiving unit (110), the emotion extraction unit (120), the music recommendation unit (130), the display control unit (140), the vector non-setting music information acquisition unit (150), the memory (160), the artificial intelligence model (161), the user-played music information acquisition unit (170), the music emotion vector labeling unit (180), and the machine learning unit (190) may include any one of a plurality of processors included in the music recommendation system (100). In addition, the social network posting classification method according to the embodiments of the present invention described so far and the embodiments to be described in the future may be implemented in the form of a program that can be driven by a processor.

[0125] Here, the program may include program commands, data files, and data structures, either singly or in combination. The program may be designed and produced using machine language code or high-level language code. The program may be specifically designed to implement the control method of the music recommendation system (100) described above, or may be implemented using various functions or definitions that are known and available to those skilled in the art of computer software. The program for implementing the control method of the music recommendation system (100) described above may be recorded on a recording medium readable by a processor. In this case, the recording medium may be a memory (160).

[0126] The memory (160) can store a program that performs the operations described above and the operations described below, and the memory (160) can execute the stored program. In the case where there are multiple processors and memories (160), they can be integrated into a single chip or provided in physically separate locations. The memory (160) can include volatile memory such as Static Random Access Memory (S-RAM) and Dynamic Random Access Memory (D-RAM) for temporarily storing data. In addition, the memory (160) can include nonvolatile memory such as Read Only Memory (ROM), Erasable Programmable Read Only Memory (EPROM), and Electrically Erasable Programmable Read Only Memory (EEPROM) for long-term storage of control programs and control data.

[0127] The processor may include various logic circuits and operation circuits, process data according to a program provided from memory (160), and generate control signals according to the processing results. The processor and memory (160) may be provided in a server.

[0128] The disclosed embodiments have been described with reference to the attached drawings as described above. Those skilled in the art will understand that the present invention can be implemented in forms other than the disclosed embodiments without altering the technical spirit or essential features of the present invention. The disclosed embodiments are illustrative and should not be construed as limiting.

Claims

1. A text receiving unit that receives a diary text entered into a user terminal; A sentiment extraction unit that extracts a sentiment vector for the diary text based on the above diary text using an artificial intelligence model; and A music recommendation system including a music recommendation unit that determines recommended music information corresponding to the diary text from among music information in a database based on the above diary sentiment vector.

2. In paragraph 1, A music recommendation system further comprising a display control unit that controls a display of the user terminal to display the recommended music information.

3. In paragraph 1, The above music recommendation section is, Among the music information in which each music emotion vector is preset in the above database, the music information of the music in which the music emotion vector of the value corresponding to the weather emotion vector is set is determined as the recommended music information, The diary sentiment vector of the above diary text is, Including vector values ​​for each of multiple emotions to represent the emotions of the writer who entered the above diary text, The above music emotion vector is, A music recommendation system comprising vector values ​​for each of a plurality of emotions to represent the emotion of music.

4. In paragraph 3, A music recommendation system further comprising a machine learning unit that learns the artificial intelligence model through a machine learning method by setting the learning diary text as an input variable and the vector values ​​included in the diary sentiment vector for the learning diary text as output variables.

5. In paragraph 3, A music recommendation system further comprising a vector unset music information acquisition unit that acquires music information of vector unset music for which the music emotion vector is not set among the music information of the above database.

6. In paragraph 5, The above emotional extraction unit, A music recommendation system that sets a music sentiment vector for the vector-unset music using the artificial intelligence model based on the lyrics of the vector-unset music.

7. In paragraph 5, A music recommendation system further comprising a music emotion vector labeling unit that sets a music emotion vector of the vector-unset music based on a diary emotion vector extracted from a diary text input through the user terminal while the vector-unset music is played through the user terminal.

8. In paragraph 3, Further comprising a user playback music information acquisition unit that receives a music list capture image including a list of music titles or a list of singer names, The above music recommendation section is, Based on the user-played music information included in the above emotional vector and the music list capture image, the recommended music information is determined, The above music list capture image is, A music recommendation system, wherein at least one piece of music information among the music title and singer name of the music played by the user terminal is a captured screen image displayed on the display of the user terminal.

9. In paragraph 8, The user playback music information acquisition unit is A music recommendation system that extracts text of music information from the captured image of the music list above and obtains the user-played music information.

10. In paragraph 8, The above music recommendation section: Based on the user playback music information, a first music score indicating the degree to which the user of the user terminal is predicted to prefer is set to correspond to each piece of music information in the database; and A music recommendation system that determines recommended music information corresponding to the diary text among the music information of the database based on the first music score set for each of the diary sentiment vectors and music information of the database.

11. In paragraph 10, The above music recommendation section: The second music score indicating the degree to which the music emotion vector of the above music information is similar to the weather emotion vector is set to correspond to each music information of the above database; and A music recommendation system that determines recommended music information corresponding to the diary text among the music information in the database based on a value obtained by multiplying the first music score set for each of the music information by a first weight and a value obtained by multiplying the second music score set for each of the music information by a second weight.

12. As a control method for a music recommendation system, A step of receiving a diary text entered into a user terminal by a text receiving unit; A step of extracting a diary sentiment vector for the diary text using an artificial intelligence model based on the diary text by a sentiment extraction unit; A step of receiving a music list capture image including a list of music titles or a list of singer names by a user playback music information acquisition unit; A step of determining recommended music information corresponding to the diary text from among music information in a database based on the diary sentiment vector by a music recommendation unit; and A step of controlling the display of the user terminal to display the recommended music information by the display control unit, The steps to determine the above recommended music information are: A step of determining the recommended music information based on the emotional vector and the user played music information included in the music list capture image by the music recommendation unit; A step of setting a first music score, which indicates the degree to which the user of the user terminal is predicted to prefer, based on the user-played music information, by the music recommendation unit, to correspond to each piece of music information in the database; A step of setting a second music score, which indicates the degree to which the music emotion vector of the music information is similar to the weather emotion vector, by the music recommendation unit so that each music information in the database corresponds to the second music score; and A step of determining recommended music information corresponding to the diary text from among the music information in the database based on a value obtained by multiplying the first music score set for each of the music information by a first weight and a value obtained by multiplying the second music score set for each of the music information by a second weight, by the music recommendation unit, The diary sentiment vector of the above diary text is, Including vector values ​​for each of multiple emotions to represent the emotions of the writer who entered the above diary text, The above music emotion vector is, Contains vector values ​​for each of multiple emotions to represent the emotion of the music, The above music list capture image is, A control method for a music recommendation system, wherein the screen image is captured while at least one of the music titles and singer names of the music played by the user terminal is displayed on the display of the user terminal.

13. A non-transitory recording medium storing a computer-readable computer program for executing the control method of the music recommendation system of Article 12.

Images