Method and system for music curation based on user voice characteristics

Abstract

According to an embodiment of the present invention, a method for curating music based on user voice characteristics is provided, comprising the steps of: receiving a user's voice file from a user terminal and extracting user voice information from the voice file; extracting timbre characteristic information from the extracted voice information; comparing the timbre characteristic information with the timbre characteristic information of previously stored artists and extracting a predetermined number of artists in order of high similarity; extracting user's vocal range information from the voice information; and extracting music that matches the user's vocal range information among the extracted artists' music.

Description

Technology Field

[0001] The present invention relates to a method and system for music curation based on user voice characteristics, and more specifically, to a method and system for analyzing a user's timbre and vocal range to search for similar artists and recommending songs suitable for the user from among the hit songs of said artists. Background Technology

[0002] With the recent popularization of online mobile devices such as smartphones, users have gained easier access to numerous digital media, and consequently, the need for effective information search and recommendation methods is increasing.

[0003] In particular, regarding music recommendation, there is an increasing demand for methods that can recommend music suitable for a user even if the user has no prior knowledge of the music.

[0004] Music recommendation techniques are algorithms that go beyond simple music information retrieval to provide recommendations that reflect the user's tastes and environment. Generally, to analyze user preferences and environments, music recommendation techniques receive user input or utilize listening patterns, such as lists of music the user has played in the past.

[0005] However, these general recommendation methods fail to reflect the user's vocal characteristics and have limitations, particularly in finding songs that sound good when sung by the user themselves. Recently, advancements in voice embedding and deep learning technologies have made it possible to quantify a user's vocal tone, leading to the emergence of technologies that compare similar speakers or artists. However, there are no instances of implementing these technologies into personalized music recommendation systems by combining them with actual popularity data.

[0006] Therefore, technology is needed to recommend music that suits the user when they sing it themselves. The problem to be solved

[0007] The purpose of the present invention is to solve the problems of the aforementioned prior art.

[0008] The objective of the present invention is to quantitatively analyze a user's vocal tone and range, search for similar artists based on this analysis, and recommend songs from among the artists' popular hits that are vocally suitable for the user, thereby enabling the user to satisfy the instinctive desire to make a positive impression on others, as well as the desire for self-satisfaction and attractiveness.

[0009] The objectives of the present invention are not limited to those mentioned above, and other unmentioned objectives will be clearly understood from the description below. means of solving the problem

[0010] According to one embodiment of the present invention for achieving the above-described purpose, a method for curating music based on user voice characteristics is provided, comprising: a service server receiving a user's voice file from a user terminal and extracting user voice information from the voice file; a step of extracting timbre feature information from the extracted voice information; a step of comparing the timbre feature information with previously stored timbre feature information of artists and extracting a predetermined number of artists in order of high similarity; a step of extracting user's vocal range information from the voice information; and a step of extracting music that matches the user's vocal range information among the extracted artists' music.

[0011] It may further include a step of extracting music that matches the user's vocal range information to filter recommendation candidates, assigning scores to the filtered candidates, and extracting a preset number of music in order of highest score.

[0012] The above score may be assigned a higher value the more similar the music of an artist is to the user's timbre characteristic information, and the higher the popularity ranking during a preset period. Effects of the invention

[0013] According to an embodiment of the present invention, by quantitatively analyzing the user's vocal tone and range, searching for similar artists based on this, and recommending songs that are vocally suitable for the user among the artists' popular hit songs, the user can satisfy the instinctive desire to make a positive impression on others, as well as the desire for self-satisfaction and attractiveness. Brief explanation of the drawing

[0014] FIG. 1 is a diagram schematically illustrating the configuration of a user voice characteristic-based music curation system according to one embodiment of the present invention. FIG. 2 is a diagram illustrating the detailed operation and internal configuration of a service server according to an embodiment of the present invention. FIG. 3 is a diagram showing information provided to a user terminal according to an embodiment of the present invention. Specific details for implementing the invention

[0015] The following detailed description of the invention refers to the accompanying drawings, which illustrate specific embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. It should be understood that various embodiments of the invention are different but need not be mutually exclusive. For example, specific shapes, structures, and characteristics described herein with respect to one embodiment may be implemented in other embodiments without departing from the spirit and scope of the invention. It should also be understood that the location or arrangement of individual components within each disclosed embodiment may be changed without departing from the spirit and scope of the invention. Accordingly, the following detailed description is not intended to be limiting, and the scope of the invention is limited only by the appended claims, including all equivalents to those claimed therein, provided appropriately described. Similar reference numerals in the drawings refer to the same or similar functions across various aspects.

[0016] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the attached drawings in order to enable a person skilled in the art to easily practice the present invention.

[0017] FIG. 1 is a diagram schematically illustrating the configuration of a user voice characteristic-based music curation system according to one embodiment of the present invention.

[0018] Referring to FIG. 1, a system according to an embodiment of the present invention is composed of a user terminal (100), a service server (200), and an external server (300).

[0019] The user terminal (100), service server (200), and external server (300) can communicate with each other through a communication network, for example, a mobile communication network, a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), the World Wide Web (WWW), and a wireless communication network (WiFi).

[0020] The user terminal (100) may be implemented in any form as long as it is a device capable of communicating with the outside and has a computing function. For example, it may be implemented as a smartphone, tablet PC, desktop, laptop, PDA, etc., but is not limited thereto.

[0021] The user terminal (100) performs the function of connecting to the service server (200) to transmit the user's voice file, and performs the function of receiving recommended music information provided by the service server (200) and displaying it so that the user can check it. To this end, the user terminal (100) may be equipped with a voice input means such as a microphone. In addition, the user terminal (100) and the microphone may be connected and linked.

[0022] The service server (200) receives a voice file from the user terminal (100), analyzes it, and performs the function of recommending music suitable for the user.

[0023] The external server (300) performs the function of providing information about multiple songs to the service server (200).

[0024] FIG. 2 is a diagram illustrating the detailed operation and internal configuration of a service server according to an embodiment of the present invention.

[0025] Referring to FIG. 2, a service server (200) according to one embodiment may be configured to include a voice information receiving unit (210), a timbre analysis unit (220), a frequency range analysis unit (230), and a recommendation information generating unit (240).

[0026] The voice information receiving unit (210), the timbre analysis unit (220), the vocal range analysis unit (230), and the recommendation information generation unit (240) may be program modules or hardware capable of communicating with an external device. Such program modules or hardware may be included in the service server (200) or other device capable of communicating therewith in the form of an operating system, an application program module, and other program modules, and may be physically stored on various known storage devices. Meanwhile, such program modules or hardware include, but are not limited to, routines, subroutines, programs, objects, components, data structures, etc., that perform specific tasks or execute specific abstract data types as described below according to the present invention.

[0027] The voice information receiving unit (210) receives a voice file from the user terminal (100). The voice file may be a file containing voice information of the user's voice or a song sung by the user. After extracting the voice information, the voice information receiving unit (210) may perform pre-processing such as noise removal and amplification.

[0028] The timbre analysis unit (220) extracts timbre characteristics from the information contained in the voice file received by the voice information receiving unit (210).

[0029] To extract timbre features, a method can be utilized in which the speech information contained in a speech file is divided into frames of a certain length, and embeddings are extracted from each frame. For example, a method can be used to extract embeddings by generating log-mel spectrograms using the ecapa tdnn library from speechbrain. Additionally, feature vectors such as MFCC (Mel-frequency cepstral coefficients), LPC (Linear predictive coding), and LPCC (Linear predictive cepstral coefficients) can be extracted. However, the method for extracting timbre features is not limited to these.

[0030] Recently, there are algorithms (e.g., ECAPA-TDNN, etc.) that extract speech features even when only short speech information (e.g., within a few seconds) is available. For example, the length of the speech information must be at least 0.5 seconds to reliably and accurately extract speech features, and if it is 1 second or longer, superior results can be obtained.

[0031] Therefore, by using these methods, the user's timbre feature vector can be extracted.

[0032] The timbre analysis unit (220) finds the artist most similar to the user's timbre feature vector. To this end, the feature vector of each artist may be stored in a database. This can be achieved by classifying each music information received from an external server (300) by artist, extracting a timbre feature vector for each artist in the same manner as described above, and then comparing it with the user's timbre feature vector.

[0033] For example, by using cosine similarity values ​​or Euclidean similarity values, which are indicators for measuring similarity between vectors, it is possible to extract artists whose timbre feature vectors are most similar to the user's timbre feature vector.

[0034] According to one embodiment, a preset number of artists can be extracted in order of highest similarity values ​​between timbre feature vectors.

[0035] The vocal range analysis unit (230) analyzes the vocal range based on the user's voice file. The vocal range can be determined through frequency analysis of the user's voice information. According to one embodiment, the user's vocal range (frequency range) can be identified by extracting the lowest frequency and highest frequency values. Additionally, it can be identified by recognizing pitch based on a tuner or a deep learning algorithm.

[0036] The recommendation information generation unit (240) performs the function of recommending music that matches the user's vocal range among the hit songs of artists extracted by the timbre analysis unit (220).

[0037] Specifically, by filtering the hit songs of the extracted artists, songs suitable for the user can be extracted based on popularity. Hit song information can be identified based on information collected from an external server (300). For example, it can be received via API based on Spotify Popularity.

[0038] First, music that matches the user's vocal range is filtered. Specifically, the vocal range of each piece of music (the range of the lowest and highest frequencies of the voice contained in the music) is compared with the user's vocal range (the range of the lowest and highest frequencies of the user's voice), and music outside the user's vocal range is excluded from recommendation candidates. In other words, filtering is performed.

[0039] After that, the recommendation information generation unit (240) can calculate the score (P1) of each piece of music through the following mathematical formula and generate a pre-set number of pieces of music as recommended music information in order of highest score (P1).

[0040]

[0041] In the above mathematical formula, C is the timbre similarity value between the artist of the music and the user's voice, and P is the popularity of the music.

[0042] The popularity of music can be defined as a value that increases as the average popularity ranking of the music during a previously set period increases. The popularity ranking is defined as being higher the closer it is to 1st place.

[0043] Meanwhile, according to another embodiment, the recommendation information generation unit (240) can calculate the score (P2) of each piece of music through the following mathematical formula and generate a preset number of pieces of music as recommended music information in order of highest score (P2).

[0044]

[0045] In the above mathematical formula, C and P are the same as described above, and A is a value inversely proportional to the difference between the user's age group (this information may be received together during the process of receiving the user's voice file) and the age group of the artist of the corresponding music.

[0046] In other words, it is possible to extract a pre-set number of songs by artists with a vocal tone similar to the user's, focusing on recent popular hit songs that correspond to the user's vocal range. Additionally, the user's age group can also be taken into account to extract songs by artists of a similar age range.

[0047] The recommendation information generation unit (240) provides the information to the user terminal (100) as recommendation information, using a preset number of music information.

[0048] FIG. 3 is a diagram showing information provided to a user terminal according to an embodiment of the present invention.

[0049] Referring to Fig. 3, the user can record and transmit their voice through the “Upload or Record Your Voice” menu, and referring to the bottom part of Fig. 3, the service server can provide recommended music to the user terminal.

[0050] According to an embodiment of the present invention, high user satisfaction can be achieved through personalized music recommendations that reflect the user's timbre and vocal range, and can provide practical utility, especially to general users and vocal trainees who wish to sing songs that suit them.

[0051] The foregoing description of the present invention is for illustrative purposes only, and those skilled in the art will understand that other specific forms can be easily modified without altering the technical spirit or essential features of the present invention. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. For example, each component described as a single unit may be implemented in a distributed manner, and components described as distributed may likewise be implemented in a combined form.

[0052] The scope of the present invention is defined by the claims set forth below, and all modifications or variations derived from the meaning and scope of the claims and equivalent concepts thereof should be interpreted as being included within the scope of the present invention. Explanation of the symbols

[0053] 100: User terminal 200: Service Server 300: External Server 210: Voice information receiver 220: Timbre Analysis Department 230: Vocal Range Analysis Section 240: Recommendation Information Generation Section

Claims

Claim 1 A service server, as a method for curating music based on user voice characteristics, comprises the steps of: receiving a user's voice file from a user terminal and extracting the user's voice information from the voice file; extracting timbre feature information from the extracted voice information; comparing the timbre feature information with the timbre feature information of previously stored artists and extracting a predetermined number of artists in order of high similarity; extracting the user's vocal range information from the voice information; extracting music that matches the user's vocal range information among the extracted artists' music; and filtering a recommendation candidate group by extracting music that matches the user's vocal range information, assigning a score (P2) calculated according to the following mathematical formula to the filtered candidate group, and extracting a predetermined number of music in order of high score (P2). A user voice characteristic-based music curation method, wherein in the above mathematical formula, C is a timbre similarity value between the music artist and the user's voice, P is the popularity of the music, and A is a value inversely proportional to the difference between the user's age group and the music artist's age group. Claim 2 delete Claim 3 delete

Images