System

Abstract

To efficiently manage music business by creating new music and predicting its popularity.SOLUTION: A system includes the means for: generating music which may become popular in future by making good use of generative AI on the basis of past popular music; classifying the generated music by meshes including genres, the sexuality, ages, phases of life, and areas; specifying user's preferences with genres, ages, and areas; filtering the classified music through filters that the user sets; recommending music based upon the user's preference and taste out of pieces of filtered music; and providing the recommended music for the user through distribution service.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In the music industry, it is not easy to find inspiration and ideas for creating new songs. It is also difficult to predict whether a song will become popular, which makes it difficult to efficiently operate music production and distribution services. [Means for solving the problem]

[0005] This invention utilizes generative AI to generate lyrics and music that are likely to become popular in the future, based on past popular songs. Furthermore, the generated lyrics and music are classified into meshes (genre, gender, age, social climate, region, etc.), and the classified lyrics and music are provided as music or through distribution services. This makes it possible to create new music and predict its popularity, thereby realizing efficient operation of the music business. [Brief explanation of the drawings]

[0006] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 2 is a sequence diagram showing a flow of processing in the data processing system according to the first embodiment of the first form example. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Embodiment 1. [Figure 13] FIG. 10 is a sequence diagram showing a processing flow of a data processing system in a second embodiment of the second form example. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Embodiment Example 2. [Figure 15] FIG. 10 is a sequence diagram showing the flow of processing in a data processing system according to a third embodiment of the third embodiment. [Figure 16] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Embodiment 3. [Figure 17] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. [Figure 18] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Form Example 1 when an emotion engine is combined. [Figure 19] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the second embodiment of the second form example when an emotion engine is combined. [Figure 20] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Form Example 2 when an emotion engine is combined. [Figure 21] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the third embodiment of the third form example when an emotion engine is combined. [Figure 22] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Form Example 3 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0007] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0008] First, the terms used in the following description will be explained.

[0009] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, the processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

[0010] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0011] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0012] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0014] [First embodiment]

[0015] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0016] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0018] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0019] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0021] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0022] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0023] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0024] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0025] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0026] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0027] "Example 1"

[0028] In one embodiment of the present invention, a database of past popular songs is constructed. This database includes information such as song melody, rhythm, lyrics, genre, gender, era, social climate, and region. The generation AI learns from this database and extracts characteristics of past songs.

[0029] "Example 2"

[0030] Generative AI generates new lyrics and music based on what it has learned. For example, an AI that has learned the characteristics of 1980s pop music can generate new 1980s-style pop songs based on those characteristics.

[0031] "Example 3"

[0032] The generated lyrics and music are classified by mesh (genre, gender, era, social climate, region, etc.). This classification is used to provide music and distribution services. For example, a 1980s-style pop song would be classified into the "1980s" and "Pop" categories and recommended to users who like these categories.

[0033] "Example 4"

[0034] Users can also customize Mesh's classification results to suit their preferences. For example, users can set specific preferences, such as preferring the "1980s" category but not the "Pop" category. This allows users to efficiently find music that matches their tastes.

[0035] The processing flow of each embodiment will be described below.

[0036] "Example 1"

[0037] Step 1: Build a database of past popular songs. This database includes information such as the melody, rhythm, lyrics, genre, gender, era, social climate, and region of the song.

[0038] Step 2: The generative AI learns from this database and extracts the characteristics of past songs.

[0039] "Example 2"

[0040] Step 1: The generative AI generates new lyrics and music based on the results of its learning.

[0041] Step 2: For example, an AI that has learned the characteristics of 1980s pop music can use those characteristics to generate new 1980s-style pop songs.

[0042] "Example 3"

[0043] Step 1: The generated lyrics and music are classified into meshes (genre, gender, era, social climate, region, etc.).

[0044] Step 2: This classification is used to provide music and streaming services. For example, a 1980s-style pop song would be classified into the "1980s" and "Pop" categories and recommended to users who like those categories.

[0045] "Example 4"

[0046] Step 1: Users can customize the mesh classification results to suit their preferences and tastes.

[0047] Step 2: Users can set specific preferences, such as preferring the "1980s" category but not the "Pop" category, allowing them to efficiently find music that matches their tastes.

[0048] Example 1

[0049] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0050] In conventional music generation systems, it is difficult to effectively utilize past music data to generate new music, and it is also difficult to provide generated music that matches the user's tastes and preferences. Furthermore, there is a lack of means to analyze the characteristics of generated music in detail and provide appropriate information to the user.

[0051] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0052] In this invention, the server includes: means for generating lyrics and music that are likely to become popular in the future using a generative AI based on music that was popular in the past; means for classifying the generated lyrics and music by genre, gender, age, social climate, region, etc.; means for providing the classified lyrics and music through music or distribution services, etc.; means for building a database of music that was popular in the past; means for preprocessing music data stored in the database; means for training a generative AI model using the preprocessed data; means for extracting music characteristics from the trained data; and means for a user to input a prompt sentence and for the generative AI model to generate a response. This makes it possible to effectively utilize past music data to generate new music and provide it according to the user's tastes and preferences.

[0053] "Past popular songs" are musical works that were widely popular during a certain period in the past.

[0054] "Generative AI" is a system that uses artificial intelligence technology to generate new data and content.

[0055] "Lyrics and music" refers to the lyrics of a song and musical elements such as melody and rhythm.

[0056] A "genre" is a category that classifies styles or types of music.

[0057] "Gender" refers to the gender of the song's artist or singer.

[0058] "Decade" refers to the time or era in which a song was released.

[0059] "Social climate" refers to the social situation and trends at the time a song was released.

[0060] "Region" refers to the geographical location or cultural sphere in which the song was released.

[0061] A "database" is a system for systematically organizing and storing specific information.

[0062] "Preprocessing" is the initial stage of processing to convert data into a format suitable for analysis and learning.

[0063] A "generative AI model" is a specific implementation of generative AI: a machine learning model trained to perform a specific task.

[0064] A "prompt" is text that contains instructions or questions that are input to a generative AI model.

[0065] A "response" is the answer or output that a generative AI model generates based on a prompt.

[0066] This invention is a system that creates new music by building a database of popular songs from the past and training a generation AI model based on that data. Specific embodiments of this system are described below.

[0067] Building a database

[0068] The server builds a database of past popular songs. This database includes information such as the song's melody, rhythm, lyrics, genre, gender, era, social climate, and region. Specifically, the data is stored using a relational database management system (RDBMS) such as MySQL (registered trademark).

[0069] Data Preprocessing

[0070] The server preprocesses the music data stored in the database. This includes data cleaning, normalization, and feature extraction. For example, it detects missing values ​​and outliers and handles them appropriately. It also converts melodic and rhythmic patterns into numerical data and normalizes the data.

[0071] Training generative AI models

[0072] The server uses the preprocessed data to train a generative AI model. This training process uses machine learning frameworks such as TENSORFLOW (registered trademark) and PyTorch. The preprocessed data is input to the model, and the model is trained for each epoch. Once the model is trained, it is saved.

[0073] Feature extraction

[0074] The generative AI model extracts the characteristics of past songs from the learned data. For example, it can identify melodic patterns that were popular in a particular decade or rhythmic characteristics common to a particular genre. The extracted features are stored in a database.

[0075] Entering prompts and generating responses

[0076] The user inputs a prompt into the generative AI model, which then generates a response based on the input prompt. For example, if a user inputs "Please tell me the characteristics of Japanese pop songs from the 1990s," the generative AI model generates an answer based on the data it has learned and displays it to the user via the server.

[0077] Specific examples

[0078] Building a database

[0079] The server stores the following data in the database:

[0080] Song A: Melody = "CDEF", Rhythm = "4 / 4", Lyrics = "Love Song", Genre = "Pop", Gender = "Male", Era = "1990s", Social Climate = "Bubble Economy", Region = "Japan"

[0081] Song B: Melody = "GABC", Rhythm = "3 / 4", Lyrics = "Freedom Song", Genre = "Rock", Gender = "Female", Era = "2000s", Social Mood = "DotNet Bubble", Region = "America"

[0082] Prompt Sentence Examples

[0083] The user inputs the following prompt into the generative AI model:

[0084] "What are the characteristics of Japanese pop songs from the 1990s?"

[0085] "What are the rhythm patterns of American rock songs from the 2000s?"

[0086] This system makes it possible to effectively utilize past music data to generate new music and provide it according to the user's tastes and preferences.

[0087] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0088] Step 1: Building the database

[0089] The server builds a database of songs that were popular in the past. As input, it uses a dataset containing information such as song melody, rhythm, lyrics, genre, gender, era, social climate, and region. Specifically, the server starts a relational database management system (RDBMS) such as MySQL and creates a table to store song information. Next, it imports song data obtained from a CSV file or API into the table. As output, it obtains a database containing song information.

[0090] Step 2: Preprocessing the data

[0091] The server preprocesses the music data stored in the database. It uses the music data in the database as input. Specifically, the server detects missing values ​​and outliers and handles them appropriately. It also converts melodic and rhythmic patterns into numerical data and normalizes the data. The preprocessed data is obtained as output.

[0092] Step 3: Training the generative AI model

[0093] The server uses the preprocessed data to train a generative AI model. The preprocessed data is used as input. Specifically, the server installs a machine learning framework such as TensorFlow or PyTorch and runs a training script. The preprocessed data is input into the model and the model is trained for each epoch. The output is a generative AI model that has completed training.

[0094] Step 4: Feature extraction

[0095] The generative AI model extracts the characteristics of past songs from the trained data. The trained generative AI model and training data are used as input. Specifically, the generative AI model extracts features based on the training data, finding patterns of melodies that were popular during a particular era, or rhythmic characteristics common to a particular genre. The extracted features are obtained as output.

[0096] Step 5: Entering a prompt and generating a response

[0097] The user inputs a prompt sentence into the generative AI model. The prompt sentence entered by the user is used as input. Specifically, the user inputs the prompt sentence from their device, and the server sends the input prompt sentence to the generative AI model. The generative AI model generates a response based on the prompt sentence and returns it to the server. The server displays the generated response on the user's device. The response generated by the generative AI model is obtained as output.

[0098] (Application example 1)

[0099] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0100] Although there is technology to generate new music by utilizing the characteristics of past popular songs, a system for generating and providing music tailored to users' tastes and preferences has not been fully established. Furthermore, there is a lack of a means to efficiently classify generated music and provide it to users. This poses a challenge in quickly and accurately providing users with the music they want.

[0101] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0102] In this invention, the server includes means for generating lyrics and music that are likely to become popular in the future using generative AI based on past popular music, means for classifying the generated lyrics and music by mesh (genre, gender, age, social climate, region, etc.), means for providing the classified lyrics and music as music or through a distribution service, etc., means for creating prompts for music generation based on filters set by the user, and means for providing the generated music to the user. This makes it possible to generate and provide music that matches the user's tastes and preferences.

[0103] "Past popular songs" are songs that were widely popular and listened to by many people during a specific period in the past.

[0104] "Generative AI" refers to systems and algorithms that use artificial intelligence technology to generate new data and content.

[0105] "Lyrics and music" refers to the lyrics and melody of a song, and are the basic elements that make up a musical work.

[0106] "Mesh" is a framework for classifying music based on multiple criteria such as genre, gender, era, social climate, and region.

[0107] A "distribution service" is a service that provides users with content such as music and video via the Internet.

[0108] A "filter" is a function that allows the user to narrow down the characteristics of the music to be generated by setting specific conditions or criteria.

[0109] A "prompt" is input data or text that provides specific conditions or instructions to the generating AI.

[0110] "User" means any individual or entity that uses the System or Services.

[0111] The system for implementing this invention utilizes AI to generate lyrics and music that are likely to become popular in the future based on popular music of the past, and provides them to users according to their tastes and preferences. Specific embodiments of this system are described below.

[0112] System Configuration

[0113] The system mainly consists of the following components:

[0114] 1. Server: Hosts the generative AI model and performs music generation and classification.

[0115] 2. User terminal: A device used by a user to access the system, such as a smartphone or computer.

[0116] 3. Database: Stores data on popular songs from the past.

[0117] Hardware and software used

[0118] Hardware: Servers, smartphones, PCs

[0119] Software: TensorFlow, Keras, database management system (e.g., MySQL)

[0120] Data processing and calculation

[0121] 1. Loading the database: The server loads a database of past popular songs and obtains information such as the song's melody, rhythm, lyrics, genre, gender, era, social climate, and region.

[0122] 2. Loading the generative AI model: The server loads the pre-trained generative AI model, which is built using TensorFlow and Keras.

[0123] 3. Prompt Creation: Based on the input from the user device, the server creates a prompt for song generation, including the filters set by the user (such as era, genre, region, etc.).

[0124] 4. Music Generation: The server inputs prompts into the generative AI model to generate a new song.

[0125] 5. Song Classification: The generated songs are classified based on criteria such as genre, gender, era, social climate, and region.

[0126] 6. Providing songs: The classified songs are provided to the user's device.

[0127] Specific examples

[0128] If a user wants to generate American pop songs from the 1980s, they can set a filter like this:

[0129] Prompt Sentence Examples

[0130] Generate a song with the following characteristics: Era: 1980s, Genre: Pop, Region: USA

[0131] By inputting this prompt into a generative AI model, a new song with characteristics of 1980s American pop songs is generated, tailored to the user's tastes and preferences.

[0132] In this way, users can enjoy new songs that utilize the characteristics of past popular songs.

[0133] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0134] Step 1:

[0135] The server loads a database of past popular songs, which includes information such as the song's melody, rhythm, lyrics, genre, gender, era, social climate, and region. This allows the server to obtain characteristic data for the song.

[0136] Step 2:

[0137] The server loads a pre-trained generative AI model. This model is built using TensorFlow and Keras and learns the characteristics of popular songs from the past. This prepares the model for music generation.

[0138] Step 3:

[0139] The user sets filters (such as era, genre, and region) for music generation using the device, and the user's preferences and tastes are acquired as input data.

[0140] Step 4:

[0141] The server creates prompts based on the filters set by the user, including information such as the era, genre, and region specified by the user, which then generates prompt sentences to be fed into the generative AI model.

[0142] Step 5:

[0143] The server inputs prompts into the generative AI model to generate new music. The generative AI model generates new melodies and lyrics based on the characteristics of popular songs from the past. This results in new music tailored to the user's preferences.

[0144] Step 6:

[0145] The server classifies the generated songs by criteria such as genre, gender, age, social climate, and region, allowing the generated songs to be organized based on meshes.

[0146] Step 7:

[0147] The server provides the classified songs to the user's terminal, and the user can listen to the generated songs through the terminal, thereby allowing the user to enjoy new songs.

[0148] Example 2

[0149] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0150] Conventional music generation systems have made it difficult for users to generate new lyrics and music that reflect the characteristics of a particular musical style or era. Furthermore, there is a lack of means to customize the generated music to suit the user's tastes and preferences. Furthermore, there is also a lack of means to efficiently classify and provide the generated music.

[0151] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0152] In this invention, the server includes means for a user to input a prompt text, means for a terminal to send the prompt text to the server, means for the server to receive the prompt text and pass it to a generative AI model, means for the generative AI model to generate new lyrics and music, means for the server to receive the generated lyrics and music and send it to the terminal, means for the terminal to display the generated lyrics and music to the user, means for generating lyrics and music that are likely to become popular in the future using generative AI based on music that was popular in the past, means for classifying the generated lyrics and music by mesh (genre, gender, age, social climate, region, etc.), and means for providing the classified lyrics and music as music or a distribution service, etc. This allows a user to generate new lyrics and music by specifying a specific musical style and efficiently classify and provide them.

[0153] "User" means an individual or organization that uses the system to input prompts and generate new lyrics and music.

[0154] A "terminal" is a device that allows a user to input a prompt and communicate with a server. Examples include a PC or smartphone.

[0155] The "server" is a computer system that receives prompts, passes them to the generative AI model, and sends the generated lyrics and music to the terminal.

[0156] A "prompt sentence" is text that a user enters to specify characteristics of a particular musical style or era.

[0157] A "generative AI model" is an artificial intelligence model that learns the characteristics of popular songs from the past and generates new lyrics and music based on the results of that learning.

[0158] "Lyrics and Music" refers to the lyrics and melody generated by the generative AI model.

[0159] "Mesh" is a method of classifying generated lyrics and music based on criteria such as genre, gender, age, social climate, and region.

[0160] "Music and distribution services" refers to platforms and services that provide users with created lyrics and music. Examples include music streaming services and download services.

[0161] "Deep learning" is a machine learning technique that generative AI models use to learn the characteristics of popular songs from the past.

[0162] "Customizable means" refers to a function that allows users to adjust the mesh classification results to suit their preferences and tastes.

[0163] MODE FOR CARRYING OUT THE INVENTION

[0164] This invention is a system that allows users to create new lyrics and music that reflect the characteristics of a particular musical style or era. Specific embodiments of this system are described below.

[0165] System configuration

[0166] This system consists of a user, a terminal, a server, and a generative AI model. The user inputs a prompt using the terminal, and the server generates new lyrics and music using the generative AI model.

[0167] Hardware and software used

[0168] Device: a device such as a computer, smartphone, or tablet.

[0169] Server: A high-performance computer system

[0170] Generative AI model: Artificial intelligence model using deep learning

[0171] Data processing and calculation

[0172] When a user inputs a prompt text through their device, the device sends it to the server. The server receives the prompt text and passes it to the generative AI model. The generative AI model learns the characteristics of popular songs from the past and generates new lyrics and music based on the learning results. The generated lyrics and music are sent from the server to the device and displayed to the user.

[0173] Specific examples

[0174] For example, if a user inputs a prompt such as "Please create a new song in the style of a 1980s pop song," the following specific actions will be taken:

[0175] 1. The user types into a text box on the device, "Please create a new song in the style of a 1980s pop song."

[0176] 2. The terminal sends this prompt text to the server as an HTTP request.

[0177] 3. The server receives the prompt and passes the instruction to the generative AI model: "Create a new song in the style of a 1980s pop song."

[0178] 4. The generative AI model generates new songs based on the characteristics of 1980s pop music, such as songs with synthesizer sounds and catchy melody lines.

[0179] 5. The server receives the generated song and sends it to the device as an HTTP response.

[0180] 6. The device displays the generated song to the user, who can then play and enjoy the song.

[0181] In this way, users can specify a specific musical style and use the generative AI model to generate new lyrics and music. The system can also efficiently classify the generated lyrics and music and customize them to suit the user's tastes and preferences.

[0182] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0183] Step 1:

[0184] The user enters a prompt statement.

[0185] Specifically, the user types into a text box on the device, "Please create a new song in the style of a 1980s pop song."

[0186] Input: The prompt text entered by the user

[0187] Output: The prompt displayed on the terminal

[0188] Step 2:

[0189] The terminal sends the prompt to the server.

[0190] As a specific operation, the terminal generates an HTTP request and transmits data including a prompt sentence to the server.

[0191] Input: The prompt text entered by the user

[0192] Output: HTTP request sent to the server

[0193] Step 3:

[0194] The server receives the prompt sentence and passes it to the generative AI model.

[0195] Specifically, the server receives an HTTP request, analyzes the prompt text, and converts it into a data format to be passed to the generative AI model.

[0196] Input: The HTTP request sent to the server

[0197] Output: The prompt text converted into a data format to be passed to the generative AI model.

[0198] Step 4:

[0199] Generative AI models generate new lyrics and music.

[0200] Specifically, the generative AI model generates a new song that reflects the characteristics of 1980s pop music based on a prompt.

[0201] Input: The prompt passed to the generative AI model

[0202] Output: The new lyrics and music generated

[0203] Step 5:

[0204] The server receives the generated lyrics and music and sends them to the device.

[0205] Specifically, the server receives the lyrics and music generated by the generative AI model and sends them to the terminal as an HTTP response.

[0206] Input: New lyrics and music received from a generative AI model

[0207] Output: HTTP response sent to the device

[0208] Step 6:

[0209] The terminal displays the generated lyrics and music to the user.

[0210] Specifically, the device analyzes the HTTP response received from the server and displays the generated lyrics and music to the user, who can then play and enjoy the music.

[0211] Input: HTTP response received from the server

[0212] Output: New lyrics and music displayed to the user

[0213] (Application example 2)

[0214] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0215] While conventional music generation systems could generate new songs based on past songs, they lacked the ability to generate songs based on a generative AI model by inputting specific prompts from the user. Furthermore, the means for streaming, downloading, and sharing the generated songs were limited, making it difficult to meet the diverse needs of users. Furthermore, the ability to customize the generated songs to suit the user's tastes and preferences was also insufficient.

[0216] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0217] In this invention, the server includes: means for generating lyrics and music that are likely to become popular in the future using generative AI based on past popular music; means for classifying the generated lyrics and music by mesh (genre, gender, age, social climate, region, etc.); means for providing the classified lyrics and music through music distribution services, etc.; means for a user to input a prompt and generate new music based on the generative AI model; and means for streaming, downloading, and sharing the generated music. This allows users to input specific prompts and generate music based on the generative AI model, and the generated music can be provided in a variety of ways. It can also be customized to suit the user's preferences and tastes.

[0218] "Generative AI" is artificial intelligence that learns from past data and generates new data based on the results of that learning.

[0219] A "prompt" is an instruction that the user inputs to the generation AI, and is used to specify the content and characteristics of the data to be generated.

[0220] "Mesh" is a method of categorizing data based on different criteria such as genre, gender, age, social climate, and region.

[0221] "Streaming playback" is a method of playing data in real time over the Internet.

[0222] "Download" is a method of saving data to a user's device via the internet.

[0223] "Sharing" refers to the method of sharing the generated data with other users via the Internet.

[0224] "Customization" is the practice of tailoring data and services to a user's tastes and preferences.

[0225] A system for implementing the present invention is configured as follows.

[0226] System Configuration

[0227] 1. Server:

[0228] Generative AI: Artificial intelligence that generates new lyrics and music based on popular songs from the past. It uses deep learning to learn the characteristics of past songs and generates new music based on the results of that learning.

[0229] Classification method: The generated lyrics and music are classified by criteria such as genre, gender, age, social climate, and region.

[0230] Provision method: Classified lyrics and songs are provided as music or through distribution services.

[0231] Generation method: A new song is generated using a generative AI model based on a prompt entered by the user.

[0232] Playback and sharing options: Stream, download, and share the generated songs.

[0233] 2. Terminal:

[0234] User interface: An interface for users to enter prompt text. It works on devices such as smartphones and PCs.

[0235] Playback and sharing features: Features for playing, downloading, and sharing the generated songs.

[0236] Program processing explanation

[0237] The server uses generative AI to learn the characteristics of past popular songs and generates new lyrics and music based on the learning results. When the user inputs a prompt through their device, the server uses a generative AI model based on the prompt to generate a new song. The generated songs are categorized by criteria such as genre, gender, age, social climate, and region, and are customized to suit the user's tastes and preferences. The generated songs can be streamed, downloaded, and shared.

[0238] Hardware and software used

[0239] Hardware: Servers, smartphones, PCs

[0240] Software: Python, OpenAI® API, user interface application

[0241] Specific examples

[0242] A user opens the application on their smartphone and types, "Generate lyrics for a 1980s-style pop song with a love theme." The server receives this prompt and uses a generative AI model to generate new lyrics for a 1980s-style pop song. The generated lyrics are customized to the user's tastes and preferences and can be streamed, downloaded, or shared on their smartphone.

[0243] Prompt Sentence Examples

[0244] "Generate lyrics for a 1980s-style pop song about love."

[0245] "Generate lyrics for a 1980s-style pop song about friendship."

[0246] "Generate lyrics for a 1980s-style pop song with an adventure theme."

[0247] In this way, users can input specific prompts to generate new music based on the generative AI model and enjoy it in a variety of ways.

[0248] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0249] Step 1:

[0250] The user enters a prompt sentence at the terminal.

[0251] Input: The user enters a prompt such as "Generate lyrics for a 1980s-style pop song about love."

[0252] What it does: The terminal's user interface receives the prompt and sends it to the server.

[0253] Output: The prompt text is sent to the server.

[0254] Step 2:

[0255] The server receives the prompt and sends a request to the generative AI model.

[0256] Input: The prompt text sent from the terminal.

[0257] How it works: The server parses the prompt and sends a request to the generative AI model to generate a new song.

[0258] Output: A request is sent to the generative AI model.

[0259] Step 3:

[0260] A generative AI model generates new music based on a prompt.

[0261] Input: The prompt text sent by the server.

[0262] How it works: The generative AI model uses deep learning based on past music data to generate new music that matches the prompt.

[0263] Output: The generated music data.

[0264] Step 4:

[0265] The server receives the generated music data and classifies it into meshes.

[0266] Input: Music data sent from the generative AI model.

[0267] How it works: The server classifies music data by criteria such as genre, gender, age, social climate, and region.

[0268] Output: Classified song data.

[0269] Step 5:

[0270] The server customizes the classified music data to suit the user's tastes and preferences.

[0271] Input: Categorized song data and information about user tastes and preferences.

[0272] How it works: The server customizes music data based on the user's tastes and preferences.

[0273] Output: Customized song data.

[0274] Step 6:

[0275] The server transmits the customized music data to the terminal.

[0276] Input: Customized song data.

[0277] How it works: The server sends customized music data to the device.

[0278] Output: Song data sent to the device.

[0279] Step 7:

[0280] The terminal provides the received music data to the user.

[0281] Input: Song data sent from the server.

[0282] What it does: Provides an interface for devices to stream, download, and share music data.

[0283] Output: Your song is ready for users to play, download, and share.

[0284] In this way, users can input specific prompts to generate new music based on the generative AI model and enjoy it in a variety of ways.

[0285] Example 3

[0286] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0287] Conventional music generation systems have had problems with recommending music that meets a user's specific tastes and preferences, and they are unable to adequately categorize or customize the music they generate. Furthermore, they lack the technology to learn the characteristics of past popular music and generate new music based on that.

[0288] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0289] In this invention, the server includes a means for a user to input a prompt sentence, a means for generating lyrics and music using a generative AI model, a means for classifying the generated lyrics and music by genre, gender, age, social climate, region, etc., a means for recommending the classified lyrics and music to the user, and a means for the user to customize the recommended music. This enables music to be recommended according to the user's specific tastes and preferences, and makes it easy to classify and customize the generated music. It can also learn the characteristics of past popular music and generate new music based on that.

[0290] A "prompt sentence" is specific text that a user inputs to give instructions to a generative AI model for generating music.

[0291] A "generative AI model" is an artificial intelligence model that uses deep learning to learn the characteristics of popular songs from the past and generates new lyrics and music based on the results of that learning.

[0292] "Genre" is a classification item that indicates the style or type of music, and includes, for example, rock, pop, jazz, etc.

[0293] "Gender" is a classification based on the target audience of the song or the gender of the artist.

[0294] "Era and Social Condition" is a classification category based on the historical background and social situation reflected in the music.

[0295] "Region" is a classification that indicates that a song is associated with a particular geographic area or culture.

[0296] "Recommendation" refers to the act of presenting the most suitable music to the user based on the user's past preference data and classification results.

[0297] "Customization" refers to the act of a user adjusting the classification results of recommended songs to suit their own tastes and preferences.

[0298] MODE FOR CARRYING OUT THE INVENTION

[0299] This invention is a system in which a user inputs a prompt sentence, a generative AI model is used to generate new lyrics and music, and the system classifies them by genre, gender, age, social climate, region, etc., recommends them to the user, and the user can further customize the recommended music.

[0300] 1. Program Generation

[0301] The server generates lyrics and music using a generative AI model. This generative AI model uses a general artificial intelligence model that uses deep learning technology, for example. Specifically, Python libraries such as TensorFlow and PyTorch are often used.

[0302] 2. Data Classification

[0303] The server classifies the generated lyrics and music by genre, gender, age, social climate, region, etc. This classification is performed using natural language processing technology and machine learning algorithms. Specifically, Python's NLTK and Scikit-learn are used.

[0304] 3. Category Recommendation

[0305] The server recommends the classified songs to the user. This recommendation system uses collaborative filtering or content-based filtering. Specifically, Apache (registered trademark) Mahout or TensorFlow may be used. The server recommends the most suitable songs based on the user's past preference data.

[0306] 4. Customization features

[0307] Users can customize Mesh's classification results to suit their preferences and tastes. For example, a user can set specific preferences, such as preferring the "1980s" category but not the "Pop" category. This customization is provided through a user interface using front-end frameworks such as React and Vue.js.

[0308] Specific examples

[0309] User A likes 1980s rock music but not pop music. The server uses a generative AI model to generate a new song and classify it into the categories "1980s" and "rock." User A is recommended this song based on his preferences.

[0310] Prompt Sentence Examples

[0311] "Generate a 1980s-style rock song. The lyrics should have themes of youth and freedom."

[0312] In this way, a system is realized that allows users to efficiently search for music that suits their preferences. The flow of the specification process in the third embodiment will be described with reference to FIG.

[0313] Step 1:

[0314] The user enters a prompt statement.

[0315] The user opens a web browser, accesses the system interface, enters "Generate a 1980s-style rock song. The lyrics should include themes of youth and freedom" in the input field, and clicks the submit button.

[0316] Input: The prompt text entered by the user

[0317] Output: The prompt is sent to the server.

[0318] Step 2:

[0319] The server generates lyrics and music using a generative AI model.

[0320] The server sends the prompt received from the user to the generative AI model. The generative AI model generates new lyrics and music based on the prompt and returns the results to the server. The generative AI model uses a general artificial intelligence model that uses deep learning technology. Specifically, it uses the Python libraries TensorFlow and PyTorch.

[0321] Input: prompt statement

[0322] Output: Generated lyrics and music

[0323] Step 3:

[0324] The server classifies the generated lyrics and music by genre, gender, age, social climate, region, etc.

[0325] The server receives the generated lyrics and music and classifies them into categories such as "1980s" or "rock" using Python's NLTK and Scikit-learn. The classification results are stored in a database.

[0326] Input: Generated lyrics and music

[0327] Output: Classified lyrics and songs

[0328] Step 4:

[0329] The server recommends songs to users based on the classification results.

[0330] The server refers to the user's past preference data and recommends the most suitable songs. This recommendation system uses collaborative filtering and content-based filtering. Specifically, Apache Mahout and TensorFlow are used. If a user has previously liked "1980s" and "Rock," new songs in the same category will be recommended.

[0331] Input: Classified lyrics and songs, user preference data

[0332] Output: Recommended songs

[0333] Step 5:

[0334] Users customize recommended songs.

[0335] Users can review the list of recommended songs and filter out categories that don't suit their tastes. For example, they can leave the "1980s" category and filter out the "Pop" category. This customization feature is provided through a user interface using front-end frameworks such as React and Vue.js.

[0336] Input: Recommended songs, user customization settings

[0337] Output: Customized song list

[0338] (Application example 3)

[0339] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0340] Conventional music distribution systems make it difficult for users to efficiently find music that matches their preferences. Furthermore, music recommendations often fail to accommodate users' specific tastes, making it difficult to discover new music. Furthermore, there is a lack of technology to generate new music based on past trends, making it impossible to meet the diverse needs of users.

[0341] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[0342] In this invention, the server includes means for generating lyrics and music that are likely to become popular in the future using generative AI based on past popular music, means for classifying the generated lyrics and music by mesh (genre, gender, age, social climate, region, etc.), means for providing the classified lyrics and music as music or through a distribution service, etc., means for recommending music based on user preferences, means for specifying user preferences by genre, age, and region, and means for filtering and recommending music that matches the user's preferences. This enables users to efficiently find music that suits their preferences and makes it easier to discover new music.

[0343] "Past popular songs" are songs that were popular with many people and widely listened to during a certain period in the past.

[0344] "Generative AI" is a system that uses artificial intelligence technology to generate new data and content.

[0345] "Lyrics and music" refers to the lyrics and melody of a musical work.

[0346] "Mesh" is a method of classifying data based on multiple categories such as genre, gender, age, social climate, and region.

[0347] A "genre" is a category that classifies musical styles and forms.

[0348] "Gender" refers to the gender of the song's target listeners.

[0349] "Era and social conditions" refers to a specific era and the social situation of that era.

[0350] "Region" refers to a specific geographical location or cultural context.

[0351] "Music and distribution services" refers to platforms and services that provide musical works to users.

[0352] "User preferences" refers to musical characteristics such as genres, eras, and regions that individual users particularly prefer.

[0353] "Filtering" is the process of sorting data based on specific criteria.

[0354] "Recommendation" is the act of suggesting appropriate music based on the user's preferences.

[0355] "Deep learning" is a machine learning technique that uses multi-layer neural networks to learn the characteristics of data.

[0356] The following system configuration will be described as an embodiment of the present invention.

[0357] The server has a means to generate lyrics and music that are likely to become popular in the future using generative AI based on past popular songs. This generative AI uses deep learning to learn the characteristics of past popular songs and generates new lyrics and music based on the learning results. The generated lyrics and music are categorized by meshes such as genre, gender, age, social climate, and region.

[0358] The classified lyrics and music are then provided as music or through music distribution services. Users access the distribution service using a device such as a smartphone and are recommended songs based on their preferences. Users can specify their preferences by genre, age, and region, which allows the system to filter and recommend songs that match their preferences.

[0359] For example, if a user likes "1980s pop songs," the server uses generative AI to learn the characteristics of 1980s pop songs and generate new songs based on those characteristics. The generated songs are then categorized into "1980s" and "Pop" and recommended based on the user's preferences.

[0360] The following hardware and software are used to realize this system. The server has high-performance computing resources to run the generative AI model. Specifically, a server equipped with a GPU is suitable. A deep learning framework (e.g., TensorFlow or PyTorch) is used as software. An application for accessing the music distribution service is installed on the user's device. This application provides an interface for inputting the user's preferences and displays songs recommended by the server.

[0361] Examples of prompts to input to a generative AI model include:

[0362] Write a Python program that recommends songs based on user preferences. The song database contains information about title, genre, era, and region. User preferences are specified by genre, era, and region. Write a function that filters and recommends songs that match the user's preferences.

[0363] In this way, users can efficiently search for music that suits their tastes and easily discover new music.

[0364] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0365] Step 1:

[0366] The server acquires a database of past popular songs.

[0367] Input: Database of popular songs from the past

[0368] Output: Get the music database

[0369] Specific operation: The server accesses a database of past popular songs and obtains information such as title, genre, era, and region.

[0370] Step 2:

[0371] The server uses a generative AI model to learn the characteristics of popular songs from the past.

[0372] Input: Song database

[0373] Output: Trained generative AI model

[0374] How it works: The server uses a deep learning framework (e.g., TensorFlow or PyTorch) to learn the features of the music database, allowing the generative AI model to understand the characteristics of past popular songs.

[0375] Step 3:

[0376] The server generates new lyrics and music using a trained generative AI model.

[0377] Input: trained generative AI model

[0378] Output: Generated lyrics and music

[0379] How it works: The server inputs prompts into the trained generative AI model to generate new lyrics and music. The generated lyrics and music reflect the characteristics of popular songs from the past.

[0380] Step 4:

[0381] The server classifies the generated lyrics and music into meshes (genre, gender, age, social climate, region, etc.).

[0382] Input: Generated lyrics and music

[0383] Output: Classified lyrics and songs

[0384] Specific operation: The server classifies the generated lyrics and music based on categories such as genre, gender, era, social climate, and region, making it clear that the music belongs to a specific category.

[0385] Step 5:

[0386] The server provides the classified lyrics and music as music or distribution services.

[0387] Input: Classified lyrics and songs

[0388] Output: Songs provided to the distribution service

[0389] Specific operation: The server uploads the classified lyrics and music to a music distribution service, making them accessible to users.

[0390] Step 6:

[0391] A user uses a terminal to access a distribution service and input their preferences.

[0392] Input: User preferences (genre, era, region)

[0393] Output: User preference settings

[0394] Specific operation: The user accesses the distribution service using a device such as a smartphone and enters preferences such as genre, era, and region.

[0395] Step 7:

[0396] The server filters and recommends songs based on the user's preferences.

[0397] Input: User preferences, categorized lyrics and songs

[0398] Output: Recommended songs

[0399] Specific operation: The server filters appropriate songs from the classified lyrics and music based on the user's preferences and recommends them to the user, allowing the user to efficiently find songs that suit their tastes.

[0400] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0401] "Example 1"

[0402] One embodiment of the present invention is a system that combines an emotion engine that recognizes user emotions. This system uses generative AI to generate lyrics and music that are likely to become popular in the future based on past popular songs, classifies the generated lyrics and music by mesh (genre, gender, age, social climate, region, etc.), and provides the classified lyrics and music as music or through a distribution service.

[0403] "Example 2"

[0404] The emotion engine recognizes the user's emotions and adjusts the generation and classification of lyrics and music based on those emotions. For example, if the user expresses joy, the emotion engine communicates that information to the generation AI, which then generates lyrics and music with a joyful theme. Conversely, if the user expresses sadness, the emotion engine communicates that information to the generation AI, which then generates lyrics and music with a sadness theme.

[0405] "Example 3"

[0406] Furthermore, the emotion engine recognizes the user's emotions and adjusts the delivery of lyrics and music based on those emotions. For example, if the user expresses a happy emotion, the emotion engine communicates that information to the delivery service, and the delivery service provides the user with lyrics and music with a happy theme. Conversely, if the user expresses a sad emotion, the emotion engine communicates that information to the delivery service, and the delivery service provides the user with lyrics and music with a sad theme.

[0407] The processing flow of each embodiment will be described below.

[0408] "Example 1"

[0409] Step 1: Recognize the user's emotions with the emotion engine.

[0410] Step 2: Based on the recognized emotions, the generative AI generates lyrics and music.

[0411] Step 3: Classify the generated lyrics and music into meshes (genre, gender, era, social climate, region, etc.).

[0412] Step 4: Provide the classified lyrics and music as music or through a distribution service.

[0413] "Example 2"

[0414] Step 1: Recognize the user's emotions with the emotion engine.

[0415] Step 2: Based on the recognized emotions, the generative AI generates lyrics and music.

[0416] Step 3: Classify the generated lyrics and music into meshes (genre, gender, era, social climate, region, etc.).

[0417] Step 4: Provide the classified lyrics and music as music or through a distribution service.

[0418] "Example 3"

[0419] Step 1: Recognize the user's emotions with the emotion engine.

[0420] Step 2: Based on the recognized emotions, the generative AI generates lyrics and music.

[0421] Step 3: Classify the generated lyrics and music into meshes (genre, gender, era, social climate, region, etc.).

[0422] Step 4: Provide the classified lyrics and music as music or through a distribution service.

[0423] Example 1

[0424] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0425] Conventional music generation systems have difficulty generating new music that reflects the characteristics of past popular songs, and are unable to provide music that matches the user's emotions. Furthermore, there is a lack of means to customize the generated music to suit the user's tastes and preferences. This makes it difficult to increase user satisfaction.

[0426] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0427] In this invention, the server includes means for generating lyrics and music that are likely to become popular in the future using a generative AI based on music that has been popular in the past, means for classifying the generated lyrics and music by genre, gender, age, social climate, region, etc., means for providing the classified lyrics and music through a music distribution service, etc., means for acquiring emotion data using an emotion engine that recognizes user emotions, and means for generating new music using a generative AI model based on the acquired emotion data. This makes it possible to generate and provide music that matches the user's emotions, thereby increasing user satisfaction.

[0428] "Generative AI" is an artificial intelligence technology that learns from past data and generates new data based on the results of that learning.

[0429] An "emotion engine" is a technology that analyzes emotions from a user's facial expressions, voice, etc. and acquires that data.

[0430] A "music distribution service" is a service that provides music to users via the Internet.

[0431] "Genre" is a category for classifying the style or type of music.

[0432] "Gender" is information indicating the gender of the user targeted by the song.

[0433] "Decade" is information that indicates the time when the song was popular and the age group of the target users.

[0434] "Social trends" refers to information that indicates the social background and cultural situation at the time a song became popular.

[0435] "Region" is information indicating the geographical location where a song became popular.

[0436] "Emotion data" is information about a user's emotions analyzed by the emotion engine.

[0437] "Machine learning" is a technology in which a computer learns patterns from data and makes predictions and judgments based on the results of that learning.

[0438] "Mesh" is a method of classifying music into multiple categories such as genre, gender, era, social climate, and region.

[0439] MODE FOR CARRYING OUT THE INVENTION

[0440] This invention is a system that builds a database of past popular songs and uses generative AI to generate new songs based on that database. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it is possible to provide songs that correspond to the user's emotions.

[0441] Building a database

[0442] The server builds a database of songs that were popular in the past. This database includes information such as the song's melody, rhythm, lyrics, genre, gender, era, social context, and region. Specifically, the data is stored using a database management system such as MySQL. For example, the server inserts data by executing the SQL query "INSERT INTO songs (melody, rhythm, lyrics, genre, gender, era, social_context, region) VALUES (...)".

[0443] Generative AI training

[0444] The server uses the constructed database to train a generative AI model. This generative AI model uses, for example, OpenAI's GPT-4 (registered trademark). During the training process, the features of past songs are extracted and patterns are recognized. Specifically, the server calls a function called "train_model(database)" and supplies the data in the database to the model.

[0445] Emotion engine emotion recognition

[0446] The device is equipped with an emotion engine for recognizing the user's emotions. This emotion engine uses, for example, Microsoft's Azure Cognitive Services. It analyzes emotions from the user's facial expressions and voice and sends the data to the server. Specifically, the device calls the function "emotion_data = recognize_emotion(user_input)" and sends the obtained emotion data to the server.

[0447] Music generation

[0448] The server uses a generative AI model to generate lyrics and music for a new song based on the user's emotional data sent from the emotion engine. The generated song reflects the characteristics of past popular songs. Specifically, the server calls the function "new_song = generate_song(emotion_data)" to generate a new song.

[0449] Song classification

[0450] The server classifies the generated songs into meshes such as genre, gender, era, social climate, and region. This classification is performed using a search engine such as ElasticSearch (registered trademark). Specifically, the server calls a function called "classify_song(new_song)" to classify the songs into appropriate categories.

[0451] Music provided by

[0452] Users can receive songs generated through their devices through music distribution services, such as Spotify and Apple Music. For example, users can play songs using the "stream_song(new_song)" function.

[0453] Examples and prompts

[0454] Specific examples

[0455] If the emotion engine determines that the user is in a sad mood, the device sends the data "emotion_data = {'emotion': 'sad'}" to the server. The server receives this data and calls the function "new_song = generate_song({'emotion': 'sad'})" to generate a new sad song. The generated song is classified as a "breakup song" and distributed on Spotify through the function "stream_song(new_song)".

[0456] Prompt Sentence Examples

[0457] "If the emotion engine recognizes that the user is in a sad mood, generate a new sad song using a generative AI model that has learned the characteristics of past sad songs."

[0458] In this way, the system can generate music that responds to the user's emotions and provide it on the appropriate platform.

[0459] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0460] Program processing flow

[0461] Step 1: Building the database

[0462] The server builds a database of songs that were popular in the past. As input, it receives information such as the song's melody, rhythm, lyrics, genre, gender, era, social context, and region. Specifically, the server uses a database management system such as MySQL to insert data by executing the SQL query "INSERT INTO songs (melody, rhythm, lyrics, genre, gender, era, social_context, region) VALUES (...)". As output, the song data is saved in the database.

[0463] Step 2: Training the generative AI

[0464] The server uses the constructed database to train the generative AI model. It receives the music data in the database as input. Specifically, the server calls a function called "train_model(database)" and supplies the data in the database to the model. The output is a generative AI model that has learned the characteristics of past music.

[0465] Step 3: Emotion Recognition in the Emotion Engine

[0466] The device is equipped with an emotion engine to recognize the user's emotions. It receives the user's facial expressions and voice data as input. Specifically, the device calls the function "emotion_data = recognize_emotion(user_input)" and sends the obtained emotion data to the server. The user's emotion data is obtained as output.

[0467] Step 4: Create a song

[0468] The server uses a generative AI model to generate lyrics and music for a new song based on the user's emotional data sent from the emotion engine. It receives the emotional data and generative AI model as input. Specifically, the server calls the function "new_song = generate_song(emotion_data)" to generate a new song. The generated song is obtained as output.

[0469] Step 5: Classify your songs

[0470] The server classifies the generated songs into meshes such as genre, gender, era, social climate, and region. It receives the generated song data as input. Specifically, the server calls a function called "classify_song(new_song)" to classify the song into the appropriate category. The output is the classified song data.

[0471] Step 6: Submit your music

[0472] The user can receive the generated music through a music distribution service or other means via their device. The input is the classified music data. Specifically, the user plays the music using the function "stream_song(new_song)". The output is the user listening to the music.

[0473] In this way, the system can generate music that responds to the user's emotions and provide it on the appropriate platform.

[0474] (Application example 1)

[0475] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0476] Conventional music generation systems were able to generate new music based on popular songs from the past, but it was difficult to generate and distribute music that responded to the user's emotions in real time. Furthermore, there was a lack of means to customize the generated music to suit the user's tastes and preferences. This made it impossible to provide a personalized music experience that responded to the user's emotions and preferences.

[0477] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0478] In this invention, the server includes: means for generating lyrics and music that are likely to become popular in the future using generative AI based on music that has been popular in the past; means for classifying the generated lyrics and music by mesh (genre, gender, age, social climate, region, etc.); means for providing the classified lyrics and music as music or a distribution service, etc.; means for generating music that corresponds to the user's emotions using an emotion engine that recognizes the user's emotions; and means for streaming the generated music in real time. This makes it possible to generate and distribute music that corresponds to the user's emotions in real time, and further customize it to suit the user's tastes and preferences.

[0479] "Past popular songs" are musical works that were popular with many people and widely listened to during a certain period in the past.

[0480] "Generative AI" is a system that uses artificial intelligence technology to automatically generate new music and lyrics.

[0481] "Mesh" is a method of classifying data based on different categories such as genre, gender, age, social climate, and region.

[0482] The "emotion engine" is a system that recognizes emotions from the user's facial expressions, voice, etc. and analyzes those emotions.

[0483] "Real-time streaming" is a technology that instantly distributes created music to users via the Internet.

[0484] "Deep learning" is a machine learning technique that uses multi-layered neural networks to learn features from large amounts of data and make predictions and generate results.

[0485] "Customizable" refers to the ability to change the system settings and output according to the user's preferences and tastes.

[0486] A system for implementing the present invention has the following configuration.

[0487] System configuration

[0488] 1. Emotion Recognition Module

[0489] The server uses the smartphone camera and head-mounted display sensors to recognize emotions from the user's facial expressions and voice, using software such as OpenCV and Keras.

[0490] 2. Music Generation Module

[0491] The server uses a generative AI to generate new music based on a database of past popular songs. The generative AI uses deep learning to learn the characteristics of past songs and generate music that matches the user's emotions. The music is generated using a proprietary music generation AI (Music Generator).

[0492] 3. Song Classification Module

[0493] The server categorizes the generated songs based on categories such as genre, gender, age, social climate, and region, and the categorized songs can be customized to suit the user's tastes and preferences.

[0494] 4. Music distribution module

[0495] The server then streams the generated music in real time, using technology that provides music to users over the Internet.

[0496] Processing flow

[0497] 1. Emotion recognition

[0498] When a user uses a smartphone camera or a head-mounted display, the server captures the user's facial expressions and voice, detects the face using OpenCV, and inputs the face into a Keras emotion recognition model to predict the user's emotions.

[0499] 2. Music Generation

[0500] The server generates new music based on the user's emotions using a generative AI that has learned from a database of popular songs from the past. The generated music is optimized to match the user's emotions.

[0501] 3. Song Classification

[0502] The server categorizes the generated songs based on genre, gender, age, social climate, region, etc. The categorized songs are then customized to suit the user's tastes and preferences.

[0503] 4. Music distribution

[0504] The server then streams the generated music in real time, allowing users to instantly listen to it via their smartphones or head-mounted displays.

[0505] Specific examples

[0506] For example, if a user has a sad expression, the server uses the emotion recognition module to detect "sadness." The music generation module generates a new song with a melody and lyrics that correspond to "sadness," and the song classification module classifies it appropriately. Finally, the music distribution module streams the generated song to the user in real time.

[0507] Prompt Sentence Examples

[0508] "You will develop an application that recognizes the user's emotions and generates new music based on those emotions. The emotion recognition is performed using a smartphone camera, and the generated music is based on a database of past popular songs."

[0509] In this way, a system can be realized that generates and distributes music in real time according to the user's emotions.

[0510] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0511] Step 1:

[0512] The server captures the user's facial expressions and voice using the smartphone's camera and head-mounted display's sensors. The input is the user's real-time video and audio, and the output is the captured image and audio data. Specifically, it activates the camera and microphone and records the user's face and voice.

[0513] Step 2:

[0514] The server performs face detection on the captured image data using OpenCV. The input is the image data obtained in step 1, and the output is the area information of the detected face. Specifically, the image data is converted to grayscale and the face detection algorithm is applied to identify the position of the face.

[0515] Step 3:

[0516] The server inputs the detected face region into the Keras emotion recognition model to predict the user's emotion. The input is the face region information obtained in step 2, and the output is the predicted emotion label. Specifically, the server resizes the face region to an appropriate size and inputs it into the emotion recognition model to classify the emotion.

[0517] Step 4:

[0518] The server uses a generative AI to generate new music based on the predicted emotions. The input is the emotion label obtained in step 3, and the output is the generated music data. Specifically, the emotion label is input to the generative AI as a prompt, and a new music piece is generated based on a database of popular music from the past.

[0519] Step 5:

[0520] The server classifies the generated songs based on categories such as genre, gender, era, social climate, and region. The input is the song data obtained in step 4, and the output is the classified song data. Specifically, it analyzes the song metadata and classifies them into appropriate categories.

[0521] Step 6:

[0522] The server customizes the classified songs to suit the user's tastes and preferences. The input is the classified song data obtained in step 5 and the user's tastes and preferences, and the output is customized song data. Specifically, the server refers to the user's profile information and adjusts the song playlist.

[0523] Step 7:

[0524] The server streams the customized music in real time. The input is the customized music data obtained in step 6, and the output is the music played on the user's device. Specifically, the music data is sent to the user's device via the Internet and playback begins.

[0525] In this way, a system can be realized that generates and distributes music in real time according to the user's emotions.

[0526] Example 2

[0527] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0528] Conventional music generation systems have difficulty customizing music to suit the user's emotions, making it impossible to provide music that matches the user's emotions. In addition, the classification and provision methods for generated music are limited, making it difficult to meet the diverse needs of users.

[0529] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for generating lyrics and music that are likely to become popular in the future using a generation AI based on music that was popular in the past; means for classifying the generated lyrics and music by mesh (genre, gender, age, social climate, region, etc.); means for providing the classified lyrics and music through music or distribution services; means for recognizing a user's emotions and adjusting the generation and classification of lyrics and music based on the emotions; means for transmitting the user's emotion data to the server; means for generating an appropriate prompt for the generation AI based on the emotion data; and means for providing the generated lyrics and music to the user's terminal. This makes it possible to generate and provide music customized to the user's emotions.

[0530] "Generative AI" is an artificial intelligence that learns the characteristics of popular songs from the past and generates new lyrics and music based on the results of that learning.

[0531] "Mesh" is a method of classifying generated lyrics and music based on criteria such as genre, gender, age, social climate, and region.

[0532] The "Emotion Engine" is a system that recognizes the user's emotions and adjusts the generation and classification of lyrics and music based on those emotions.

[0533] A "prompt sentence" is a text sentence that the emotion engine generates to give instructions to the generation AI.

[0534] "User emotion data" refers to data that includes emotion information recognized from the user's facial expressions and behavior.

[0535] A "server" is a computer system that manages the entire system, including the generative AI and emotion engine, and processes and communicates data.

[0536] "Terminal" refers to the device used by the user to perform emotion recognition and listen to the generated lyrics and music.

[0537] The "classification means" refers to a method or device for classifying the generated lyrics and music based on meshes.

[0538] "Providing means" means a method or device for providing classified lyrics and music to users.

[0539] The present invention is a system for generating and providing music customized according to the emotions of a user. A specific embodiment of this system is described below.

[0540] Hardware and software used

[0541] Server: A server equipped with a high-performance GPU (e.g., a computer equipped with a high-performance graphics processor) is used to train and infer generative AI models.

[0542] Device: The device used by the user (e.g., a smartphone or personal computer) has emotion recognition software (e.g., an emotion recognition software development kit) installed.

[0543] Data processing and calculation

[0544] User emotion recognition

[0545] Users input their facial expressions through the device's camera, for example, by taking a photo of their face using a smartphone camera.

[0546] The device uses emotion recognition software to recognize emotions from the user's facial expressions. For example, if the user smiles, the device will recognize the emotion "happiness."

[0547] The device sends the recognized emotion data (e.g., happiness, sadness) to the server in JSON format.

[0548] Emotional Data Processing

[0549] The server inputs the received emotion data into the emotion engine. For example, the server passes emotion data of "joy" to the emotion engine.

[0550] The emotion engine prepares to generate appropriate prompt sentences based on the user's emotions.

[0551] Generate prompt statement

[0552] The emotion engine generates prompts to be passed to the generative AI based on the user's emotions, such as "The user is expressing the emotion of joy, so please generate a 1980s-style pop song."

[0553] Lyrics and music generation

[0554] The generative AI generates new lyrics and music based on the prompt received from the emotion engine, utilizing the learned musical characteristics. For example, the generative AI could generate a new pop song with a joyful theme based on the characteristics of 1980s pop music.

[0555] Providing generated results

[0556] The server sends the generated lyrics and music to the user's device, where the generated music file is downloaded to the user's device.

[0557] Users can listen to the lyrics and music generated on their device, for example by playing a new pop song on their smartphone's music player app.

[0558] Examples of concrete examples and prompts

[0559] Specific examples

[0560] User Emotion: Delight

[0561] Generated music: 1980s-style pop song

[0562] Prompt Sentence Examples

[0563] "Generate a 1980s-style pop song because the user is expressing joy."

[0564] The system can provide a customized musical experience based on the user's emotions. For example, if the user expresses sadness, the emotion engine will instruct the generative AI to generate lyrics and music with a sadness theme.

[0565] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0566] Step 1:

[0567] The user inputs their facial expressions through the device camera. Specifically, they take a picture of their face using the smartphone camera. The input is the user's facial expression data, and the output is the image data captured by the camera app.

[0568] Step 2:

[0569] The device uses emotion recognition software to recognize emotions from the user's facial expressions. Specifically, emotion recognition software such as the Affectiva SDK is used to analyze captured image data and identify the user's emotion (e.g., joy, sadness). The input is image data, and the output is recognized emotion data.

[0570] Step 3:

[0571] The device sends the recognized emotion data to the server. Specifically, the emotion data (e.g., joy, sadness) is sent to the server in JSON format. The input is the emotion data, and the output is the JSON data sent to the server.

[0572] Step 4:

[0573] The server inputs the received emotion data into the emotion engine. Specifically, the server passes the emotion data "joy" to the emotion engine. The input is emotion data in JSON format, and the output is the emotion data passed to the emotion engine.

[0574] Step 5:

[0575] The emotion engine generates an appropriate prompt based on the user's emotion. Specifically, it creates a prompt such as "The user is expressing the emotion of joy, so please generate a 1980s-style pop song." The input is emotion data, and the output is the generated prompt.

[0576] Step 6:

[0577] The generative AI generates new lyrics and music using the learned musical characteristics based on the prompt received from the emotion engine. Specifically, the generative AI generates a new pop song with a joyful theme based on the characteristics of 1980s pop music. The input is the prompt, and the output is the generated lyrics and music.

[0578] Step 7:

[0579] The server sends the generated lyrics and music to the user's device. Specifically, the generated music file is downloaded to the user's device. The input is the generated lyrics and music, and the output is the music file sent to the user's device.

[0580] Step 8:

[0581] The user listens to the lyrics and music generated on the device. Specifically, they play a new pop song on their smartphone's music player app. The input is a music file stored on the device, and the output is the music being played.

[0582] (Application example 2)

[0583] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0584] Conventional music generation systems have had difficulty generating and delivering music in real time that responds to the user's emotions. There is also a need to provide a more personalized music experience by generating music based on the user's emotions.

[0585] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0586] In this invention, the server includes means for generating lyrics and music that are likely to become popular in the future using generative AI based on past popular music, means for classifying the generated lyrics and music by mesh (genre, gender, age, social climate, region, etc.), means for providing the classified lyrics and music as music or a distribution service, etc., means for recognizing a user's emotions and adjusting the generation of lyrics and music based on those emotions, and means for distributing the generated lyrics and music in real time. This makes it possible to generate music in real time according to the user's emotions and provide a personalized music experience.

[0587] "Generative AI" is artificial intelligence that learns from past data and generates new data based on the results of that learning.

[0588] "Mesh" is a method of classifying data based on different criteria such as genre, gender, age, social climate, and region.

[0589] "Emotion recognition" is a technology that analyzes and recognizes a user's emotions from facial expressions, tone of voice, etc.

[0590] "Real-time delivery" is a technology that provides generated content to users instantly.

[0591] "Deep learning" is a type of machine learning that uses multi-layered neural networks to learn data and perform advanced pattern recognition.

[0592] "Personalization" refers to customizing services and content to suit individual users' preferences and tastes.

[0593] A system for implementing the present invention is configured as follows.

[0594] System Configuration

[0595] 1. Hardware:

[0596] Server: A server with high-performance computing capabilities (e.g., AWS (registered trademark) EC2, Google (registered trademark) Cloud Compute Engine)

[0597] Devices: Smartphones (iPhone(R), ANDROID(R)), smart glasses (Google Glass(R), Vuzix Blade), head-mounted displays (Oculus Quest, HTC Vive)

[0598] 2. Software:

[0599] Emotion recognition libraries: Microsoft Azure Emotion API, Affectiva SDK

[0600] Generative AI models: GPT-3 (registered trademark), GPT-4

[0601] Music generation libraries: Magenta, Jukedeck

[0602] Program processing

[0603] 1. Emotion recognition:

[0604] It captures data in real time from the device's camera and microphone.

[0605] The acquired data is input into an emotion recognition library to analyze the user's emotions.

[0606] The analysis results are sent to the server.

[0607] 2. Generation AI:

[0608] The server inputs prompt sentences into the generative AI model based on the emotional data received from the emotion recognition library.

[0609] A generative AI model generates new music based on a prompt.

[0610] The generated music data is processed using a music generation library to create the final music file.

[0611] 3. Real-time streaming:

[0612] The server streams the created music files to the user's device.

[0613] Specific examples

[0614] Example prompt sentence:

[0615] The user expresses joy. Generate a 1980s-style pop song.

[0616] The user expresses sadness. Generate a calming ballad.

[0617] This system allows users to enjoy music that perfectly matches their emotions in real time. For example, it can automatically generate relaxing music when you're tired from work, or exciting music when you're having fun with friends, which is very convenient.

[0618] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0619] Step 1:

[0620] The device captures data in real time from the camera and microphone. While the user is using the device, the camera captures the user's facial expressions and the microphone records the user's voice. This data is then input into the emotion recognition library.

[0621] Step 2:

[0622] An emotion recognition library is used to analyze the user's emotions from the acquired data. Specifically, the Microsoft Azure Emotion API and Affectiva SDK are used to analyze facial and voice data to identify the user's emotions (happiness, sadness, surprise, etc.). The analysis results are sent to the server as emotion data.

[0623] Step 3:

[0624] The server inputs a prompt sentence into the generative AI model based on the emotion data received from the emotion recognition library. For example, if the user is expressing joy, the server generates a prompt sentence such as, "The user is expressing joy. Please generate a 1980s-style pop song." This prompt sentence is input into the generative AI model.

[0625] Step 4:

[0626] Generative AI models (GPT-3 and GPT-4) generate new music based on the input prompt. The generative AI model generates music that matches the prompt based on the results of learning from past data. The generated music data is sent to a music generation library.

[0627] Step 5:

[0628] Using a music generation library (such as Magenta or Jukedeck), the generated music data is processed to create the final music file. Specifically, the music data is adjusted and effects are added to make it easier for users to listen to. The completed music file is then stored on the server.

[0629] Step 6:

[0630] The server then streams the created music files to the user's device, where the user can listen to the music created in real time, providing a musical experience tailored to the user's emotions.

[0631] Example 3

[0632] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0633] Conventional music generation systems have difficulty generating music based on past trends, and have not adequately provided music that matches the user's preferences and emotions. In addition, classifying and customizing the generated music is time-consuming, making it difficult for users to efficiently find music that matches their preferences.

[0634] The identification process by the identification processing unit 290 of the data processing device 12 in Example 3 is realized by the following means. In this invention, the server includes: means for generating lyrics and music that are likely to become popular in the future using a generation AI model based on past popular music; means for classifying the generated lyrics and music by mesh (genre, gender, age, social climate, region, etc.); means for providing the classified lyrics and music through music or distribution services; means for a user to input a request for music generation; means for customizing the generated lyrics and music to the user's tastes and preferences; and means for recognizing the user's emotions using an emotion engine and adjusting the provision of lyrics and music based on the emotions. This allows users to efficiently find music that suits their tastes and emotions.

[0635] A "generative AI model" is an algorithm or program that uses artificial intelligence technology to generate new lyrics or music.

[0636] "Mesh" is a method of classifying data based on multiple categories such as genre, gender, age, social climate, and region.

[0637] An "emotion engine" is a technology that recognizes a user's emotions and provides appropriate responses and content based on those emotions.

[0638] A "music generation request" is input information that a user uses to instruct a generative AI model to generate new lyrics and music.

[0639] "Customization" is the act of a user tailoring provided data or content to suit their own preferences and tastes.

[0640] A "distribution service" is a service that provides music and other content to users via the Internet.

[0641] This invention is a system that uses a generative AI model to generate new lyrics and music and provides them based on the user's preferences and emotions. Specific embodiments of this system are described below.

[0642] System configuration

[0643] The system consists of three main components: a server, a terminal, and a user. The server has key functions such as the generative AI model, emotion engine, and database. The terminal is a device that allows the user to access and operate the system. The user is the entity that inputs requests for music generation and receives the generated music.

[0644] Hardware and software used

[0645] Generative AI models: Examples of generative AI models include OpenAI's GPT-4 and Google's BERT.

[0646] Emotion engine: For example, Microsoft's Azure Cognitive Services or IBM's Watson (registered trademark) are used as emotion engines.

[0647] Natural language processing tools: Python's NLTK library and SpaCy are used.

[0648] Data processing and calculation

[0649] The server receives a song generation request from the user and inputs a prompt into the generative AI model. The generative AI model generates new lyrics and music based on the prompt. The generated lyrics and music are classified into meshes such as genre, gender, era, social climate, and region. This classification is performed using natural language processing tools.

[0650] Specific examples

[0651] The user inputs a prompt such as "Generate lyrics for 1980s rock music with a joyful theme." The server sends this prompt to the generative AI model and receives the generated lyrics and music. The server then classifies the generated lyrics and music using a mesh and provides the results to the user. The user can customize the provided classification results to suit their own preferences and tastes.

[0652] Prompt Sentence Examples

[0653] Generate lyrics with a joyful theme in 1980s rock.

[0654] Emotion-Based Adjustment

[0655] The server recognizes the user's emotion using an emotion engine. For example, if the user expresses a happy emotion, the server preferentially provides songs with a happy theme. If the user expresses a sad emotion, the server preferentially provides songs with a sad theme.

[0656] In this way, the user can efficiently find music that matches his or her preferences and emotions. The flow of the identification process in the third embodiment will be described with reference to FIG.

[0657] Step 1:

[0658] A user logs in to the system.

[0659] The user accesses the system's login screen using a terminal and enters their username and password. The server checks the entered information against the database, and if authentication is successful, it loads the user's profile information and displays the dashboard.

[0660] Input: Username, Password

[0661] Output: User's dashboard

[0662] Step 2:

[0663] A user inputs a request to create a piece of music.

[0664] The user clicks the "Music Generation" button on the dashboard, enters a prompt into the input form that appears, for example, "Generate lyrics for a 1980s rock song with a theme of joy," and clicks the "Submit" button.

[0665] Input: prompt statement

[0666] Output: Request to the generative AI model

[0667] Step 3:

[0668] The server generates lyrics and music using a generative AI model.

[0669] The server receives a request from the user and sends a prompt to the generative AI model, which then generates new lyrics and music based on the prompt and returns the results to the server.

[0670] Input: prompt statement

[0671] Output: Generated lyrics and music

[0672] Step 4:

[0673] The server classifies the generated lyrics and music into meshes.

[0674] The server receives the generated lyrics and music and uses natural language processing tools such as Python's NLTK library and SpaCy to classify them into meshes such as genre, gender, age, social climate, and region. The classification results are stored in a database.

[0675] Input: Generated lyrics and music

[0676] Output: Lyrics and music classified by mesh

[0677] Step 5:

[0678] The server provides the classification results to the user.

[0679] The server displays the categorized list of lyrics and songs on the user's dashboard, where the user can scroll through the list and click on songs of interest to view more details.

[0680] Input: Lyrics and songs classified by mesh

[0681] Output: Classification results displayed on the user's dashboard

[0682] Step 6:

[0683] Users customize the classification results.

[0684] Users click the "Customize" button on the dashboard and set their preferences on the settings screen that appears. For example, they enter specific preferences such as "I like 1980s rock, but I want to exclude pop," and click the "Save" button.

[0685] Input: User preference settings

[0686] Output: Customized classification results

[0687] Step 7:

[0688] The server uses an emotion engine to recognize the user's emotion.

[0689] The server sends data acquired from the user's webcam and microphone to the emotion engine, which analyzes the user's emotions from their facial expressions and voice and returns the results to the server.

[0690] Input: User's facial expression data, voice data

[0691] Output: Recognized emotion data

[0692] Step 8:

[0693] The server adjusts the lyrics and music presentation based on emotion.

[0694] The server adjusts the lyrics and music it provides based on the user's recognized emotions. For example, if the user expresses joy, the server will prioritize joy-themed music in the list.

[0695] Input: Recognized emotion data

[0696] Output: Adjusted recommended song list

[0697] Step 9:

[0698] The server provides the final recommended songs to the user.

[0699] The server will then select the final recommended songs based on the user's preferences and emotions and display them on the user's dashboard. The user can click on the recommended songs to play and enjoy them.

[0700] Input: Adjusted recommended song list

[0701] Output: Final song recommendations displayed on the user's dashboard

[0702] (Application example 3)

[0703] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0704] Conventional music recommendation systems do not adequately provide music based on the user's emotions and preferences, making it difficult for users to efficiently find music that matches their emotions and preferences. Furthermore, even when music is generated based on past trends, it is difficult to meet the diverse needs of users.

[0705] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means. In this invention, the server includes: means for generating lyrics and music that are likely to become popular in the future using a generation AI based on music that was popular in the past; means for classifying the generated lyrics and music by mesh (genre, gender, age, social climate, region, etc.); means for providing the classified lyrics and music through music or distribution services, etc.; means for recognizing a user's emotions and adjusting the provision of lyrics and music based on those emotions; and means for customizing the mesh classification results to suit the user's preferences. This makes it possible to provide music based on the user's emotions and preferences, allowing the user to efficiently find music that matches their emotions and preferences.

[0706] A "popular song of the past" is a musical work that was widely popular during a particular period in the past.

[0707] "Generative AI" is a system that uses artificial intelligence technology to automatically generate new lyrics and music.

[0708] "Mesh" is a method of classifying data based on multiple categories such as genre, gender, age, social climate, and region.

[0709] A "classifying means" is a method or device for categorizing generated lyrics or music based on specific criteria.

[0710] "Music and distribution services" are platforms and systems for providing musical works to users.

[0711] "Means for recognizing emotions" refers to technology for detecting emotions from a user's facial expressions, voice, etc., and acquiring that information.

[0712] The "means for tailoring the presentation" is a method or device for selecting and presenting lyrics or music appropriate to the user based on the recognized emotion.

[0713] A "preference customization means" is a method or apparatus for tailoring the lyrics and music classification provided to a user according to the user's individual preferences.

[0714] "Deep learning" is a machine learning technique that uses multi-layer neural networks to learn the characteristics of data and make predictions and classifications.

[0715] A "smartphone" is a portable information terminal that, in addition to the functions of a mobile phone, can also connect to the Internet and run applications.

[0716] "Smart glasses" are a wearable device in the form of glasses that can display information and allow interaction.

[0717] A "head-mounted display" is a display device worn on the head, and is a device for providing visual information.

[0718] A "robot" is a mechanical device that operates autonomously or remotely to perform a specific task.

[0719] The following system configuration will be described as an embodiment of the present invention.

[0720] System Configuration

[0721] Hardware

[0722] Server: Manages the music database, runs generative AI models, and processes user emotion data.

[0723] Device: The device used by the user, such as a smartphone, smart glasses, head-mounted display, or robot.

[0724] Emotion recognition devices: Devices that recognize user emotions, such as cameras and microphones on smartphones and sensors on smart glasses.

[0725] software

[0726] Generative AI model: A deep learning model that learns the characteristics of popular songs from the past and generates new lyrics and music.

[0727] Emotion recognition software: Software for recognizing emotions from a user's facial expressions and voice (e.g., the EmotionRecognizer library).

[0728] Music recommendation engine: An algorithm for recommending appropriate songs based on the user's emotions and preferences.

[0729] Processing flow

[0730] 1. Emotion recognition:

[0731] The device's camera and microphone are used to capture the user's facial expressions and voice.

[0732] Emotion recognition software analyzes the captured data and recognizes the user's emotions.

[0733] 2. Music Generation:

[0734] A generative AI model on the server generates new lyrics and music based on data on past popular songs.

[0735] The generated lyrics and music are categorized into meshes such as genre, gender, age, social climate, and region.

[0736] 3. Song Recommendation:

[0737] The server filters suitable songs based on the recognized user emotions and user preference settings.

[0738] From the filtered songs, the system recommends the most suitable songs for the user.

[0739] Specific examples

[0740] For example, if a user smiles through their smartphone camera, the emotion recognition software will recognize the emotion "joy." The server will then select and recommend songs that match the user's tastes from a selection of 1980s pop songs generated using a generative AI model.

[0741] Prompt Sentence Examples

[0742] An example of a prompt for the generative AI model is as follows:

[0743] Create a smartphone application that recognizes a user's emotions and recommends songs that match those emotions. Songs can be categorized by genre, era, gender, social climate, region, etc., and can be customized to suit the user's preferences. The EmotionRecognizer library is used for emotion recognition, and the song database is provided in JSON format.

[0744] In this way, it becomes possible to provide music based on the user's emotions and preferences, and a system can be realized that allows users to efficiently find music that matches their emotions and preferences.

[0745] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0746] Step 1:

[0747] The device captures the user's facial expressions and voice using a camera and microphone. The input is the user's real-time facial expressions and voice, and the output is captured image data and audio data. Specifically, the device's camera captures the user's face, and the microphone records the user's voice.

[0748] Step 2:

[0749] The device uses emotion recognition software (e.g., EmotionRecognizer library) to analyze the captured image data and audio data and recognize the user's emotions. The input is the image data and audio data obtained in step 1, and the output is the recognized emotion (e.g., joy, sadness). Specifically, the emotion recognition software extracts facial features from the image data and analyzes the tone of voice from the audio data.

[0750] Step 3:

[0751] The server uses a generative AI model to generate new lyrics and music based on data on past popular songs. The input is data on past popular songs, and the output is the generated new lyrics and music. Specifically, the generative AI model uses deep learning to learn the characteristics of songs and generates new songs based on the learning results.

[0752] Step 4:

[0753] The server classifies the generated lyrics and music into meshes such as genre, gender, era, social climate, and region. The input is the lyrics and music generated in step 3, and the output is the classified music data. Specifically, the server analyzes the music metadata and classifies the music based on each category.

[0754] Step 5:

[0755] The server filters appropriate songs from the classified music data based on the user's emotions and preferences. The input is the emotions recognized in step 2 and the user's preferences, and the music data classified in step 4, and the output is a filtered music list. Specifically, the server searches for songs that match the user's emotions and preferences and creates a list.

[0756] Step 6:

[0757] The server selects the most suitable song for the user from the filtered song list and recommends it to the device. The input is the song list filtered in step 5, and the output is the recommended song. Specifically, the server selects the most suitable song randomly or based on an algorithm and sends it to the device.

[0758] Step 7:

[0759] The device provides the recommended songs to the user. The input is the song recommended in step 6, and the output is the song played to the user. Specifically, the device streams the song and plays it for the user.

[0760] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0761] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0762] Another example of generative AI is Gemini (registered trademark) (Internet search engine). <url: https: gemini.google.com ?hl="ja">) are mentioned.

[0763] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0764] [Second embodiment]

[0765] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0766] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0767] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0768] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0769] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0770] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0771] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0772] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0773] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0774] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0775] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0776] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0777] "Example 1"

[0778] In one embodiment of the present invention, a database of past popular songs is constructed. This database includes information such as song melody, rhythm, lyrics, genre, gender, era, social climate, and region. The generation AI learns from this database and extracts characteristics of past songs.

[0779] "Example 2"

[0780] Generative AI generates new lyrics and music based on what it has learned. For example, an AI that has learned the characteristics of 1980s pop music can generate new 1980s-style pop songs based on those characteristics.

[0781] "Example 3"

[0782] The generated lyrics and music are classified by mesh (genre, gender, era, social climate, region, etc.). This classification is used to provide music and distribution services. For example, a 1980s-style pop song would be classified into the "1980s" and "Pop" categories and recommended to users who like these categories.

[0783] "Example 4"

[0784] Users can also customize Mesh's classification results to suit their preferences. For example, users can set specific preferences, such as preferring the "1980s" category but not the "Pop" category. This allows users to efficiently find music that matches their tastes.

[0785] The processing flow of each embodiment will be described below.

[0786] "Example 1"

[0787] Step 1: Build a database of past popular songs. This database includes information such as the melody, rhythm, lyrics, genre, gender, era, social climate, and region of the song.

[0788] Step 2: The generative AI learns from this database and extracts the characteristics of past songs.

[0789] "Example 2"

[0790] Step 1: The generative AI generates new lyrics and music based on the results of its learning.

[0791] Step 2: For example, an AI that has learned the characteristics of 1980s pop music can use those characteristics to generate new 1980s-style pop songs.

[0792] "Example 3"

[0793] Step 1: The generated lyrics and music are classified into meshes (genre, gender, era, social climate, region, etc.).

[0794] Step 2: This classification is used to provide music and streaming services. For example, a 1980s-style pop song would be classified into the "1980s" and "Pop" categories and recommended to users who like those categories.

[0795] "Example 4"

[0796] Step 1: Users can customize the mesh classification results to suit their preferences and tastes.

[0797] Step 2: Users can set specific preferences, such as preferring the "1980s" category but not the "Pop" category, allowing them to efficiently find music that matches their tastes.

[0798] Example 1

[0799] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0800] In conventional music generation systems, it is difficult to effectively utilize past music data to generate new music, and it is also difficult to provide generated music that matches the user's tastes and preferences. Furthermore, there is a lack of means to analyze the characteristics of generated music in detail and provide appropriate information to the user.

[0801] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0802] In this invention, the server includes: means for generating lyrics and music that are likely to become popular in the future using a generative AI based on music that was popular in the past; means for classifying the generated lyrics and music by genre, gender, age, social climate, region, etc.; means for providing the classified lyrics and music through music or distribution services, etc.; means for building a database of music that was popular in the past; means for preprocessing music data stored in the database; means for training a generative AI model using the preprocessed data; means for extracting music characteristics from the trained data; and means for a user to input a prompt sentence and for the generative AI model to generate a response. This makes it possible to effectively utilize past music data to generate new music and provide it according to the user's tastes and preferences.

[0803] "Past popular songs" are musical works that were widely popular during a certain period in the past.

[0804] "Generative AI" is a system that uses artificial intelligence technology to generate new data and content.

[0805] "Lyrics and music" refers to the lyrics of a song and musical elements such as melody and rhythm.

[0806] A "genre" is a category that classifies styles or types of music.

[0807] "Gender" refers to the gender of the song's artist or singer.

[0808] "Decade" refers to the time or era in which a song was released.

[0809] "Social climate" refers to the social situation and trends at the time a song was released.

[0810] "Region" refers to the geographical location or cultural sphere in which the song was released.

[0811] A "database" is a system for systematically organizing and storing specific information.

[0812] "Preprocessing" is the initial stage of processing to convert data into a format suitable for analysis and learning.

[0813] A "generative AI model" is a specific implementation of generative AI: a machine learning model trained to perform a specific task.

[0814] A "prompt" is text that contains instructions or questions that are input to a generative AI model.

[0815] A "response" is the answer or output that a generative AI model generates based on a prompt.

[0816] This invention is a system that creates new music by building a database of popular songs from the past and training a generation AI model based on that data. Specific embodiments of this system are described below.

[0817] Building a database

[0818] The server builds a database of past popular songs, including information on the song's melody, rhythm, lyrics, genre, gender, era, social climate, and region. Specifically, the data is stored using a relational database management system (RDBMS) such as MySQL.

[0819] Data Preprocessing

[0820] The server preprocesses the music data stored in the database. This includes data cleaning, normalization, and feature extraction. For example, it detects missing values ​​and outliers and handles them appropriately. It also converts melodic and rhythmic patterns into numerical data and normalizes the data.

[0821] Training generative AI models

[0822] The server uses the preprocessed data to train a generative AI model. This training process uses machine learning frameworks such as TensorFlow and PyTorch. The preprocessed data is input to the model, and the model is trained every epoch. Once the model is trained, it is saved.

[0823] Feature extraction

[0824] The generative AI model extracts the characteristics of past songs from the learned data. For example, it can identify melodic patterns that were popular in a particular decade or rhythmic characteristics common to a particular genre. The extracted features are stored in a database.

[0825] Entering prompts and generating responses

[0826] The user inputs a prompt into the generative AI model, which then generates a response based on the input prompt. For example, if a user inputs "Please tell me the characteristics of Japanese pop songs from the 1990s," the generative AI model generates an answer based on the data it has learned and displays it to the user via the server.

[0827] Specific examples

[0828] Building a database

[0829] The server stores the following data in the database:

[0830] Song A: Melody = "CDEF", Rhythm = "4 / 4", Lyrics = "Love Song", Genre = "Pop", Gender = "Male", Era = "1990s", Social Climate = "Bubble Economy", Region = "Japan"

[0831] Song B: Melody = "GABC", Rhythm = "3 / 4", Lyrics = "Freedom Song", Genre = "Rock", Gender = "Female", Era = "2000s", Social Mood = "DotNet Bubble", Region = "America"

[0832] Prompt Sentence Examples

[0833] The user inputs the following prompt into the generative AI model:

[0834] "What are the characteristics of Japanese pop songs from the 1990s?"

[0835] "What are the rhythm patterns of American rock songs from the 2000s?"

[0836] This system makes it possible to effectively utilize past music data to generate new music and provide it according to the user's tastes and preferences.

[0837] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0838] Step 1: Building the database

[0839] The server builds a database of songs that were popular in the past. As input, it uses a dataset containing information such as song melody, rhythm, lyrics, genre, gender, era, social climate, and region. Specifically, the server starts a relational database management system (RDBMS) such as MySQL and creates a table to store song information. Next, it imports song data obtained from a CSV file or API into the table. As output, it obtains a database containing song information.

[0840] Step 2: Preprocessing the data

[0841] The server preprocesses the music data stored in the database. It uses the music data in the database as input. Specifically, the server detects missing values ​​and outliers and handles them appropriately. It also converts melodic and rhythmic patterns into numerical data and normalizes the data. The preprocessed data is obtained as output.

[0842] Step 3: Training the generative AI model

[0843] The server uses the preprocessed data to train a generative AI model. The preprocessed data is used as input. Specifically, the server installs a machine learning framework such as TensorFlow or PyTorch and runs a training script. The preprocessed data is input into the model and the model is trained for each epoch. The output is a generative AI model that has completed training.

[0844] Step 4: Feature extraction

[0845] The generative AI model extracts the characteristics of past songs from the trained data. The trained generative AI model and training data are used as input. Specifically, the generative AI model extracts features based on the training data, finding patterns of melodies that were popular during a particular era, or rhythmic characteristics common to a particular genre. The extracted features are obtained as output.

[0846] Step 5: Entering a prompt and generating a response

[0847] The user inputs a prompt sentence into the generative AI model. The prompt sentence entered by the user is used as input. Specifically, the user inputs the prompt sentence from their device, and the server sends the input prompt sentence to the generative AI model. The generative AI model generates a response based on the prompt sentence and returns it to the server. The server displays the generated response on the user's device. The response generated by the generative AI model is obtained as output.

[0848] (Application example 1)

[0849] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0850] Although there is technology to generate new music by utilizing the characteristics of past popular songs, a system for generating and providing music tailored to users' tastes and preferences has not been fully established. Furthermore, there is a lack of a means to efficiently classify generated music and provide it to users. This poses a challenge in quickly and accurately providing users with the music they want.

[0851] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0852] In this invention, the server includes means for generating lyrics and music that are likely to become popular in the future using generative AI based on past popular music, means for classifying the generated lyrics and music by mesh (genre, gender, age, social climate, region, etc.), means for providing the classified lyrics and music as music or through a distribution service, etc., means for creating prompts for music generation based on filters set by the user, and means for providing the generated music to the user. This makes it possible to generate and provide music that matches the user's tastes and preferences.

[0853] "Past popular songs" are songs that were widely popular and listened to by many people during a specific period in the past.

[0854] "Generative AI" refers to systems and algorithms that use artificial intelligence technology to generate new data and content.

[0855] "Lyrics and music" refers to the lyrics and melody of a song, and are the basic elements that make up a musical work.

[0856] "Mesh" is a framework for classifying music based on multiple criteria such as genre, gender, era, social climate, and region.

[0857] A "distribution service" is a service that provides users with content such as music and video via the Internet.

[0858] A "filter" is a function that allows the user to narrow down the characteristics of the music to be generated by setting specific conditions or criteria.

[0859] A "prompt" is input data or text that provides specific conditions or instructions to the generating AI.

[0860] "User" means any individual or entity that uses the System or Services.

[0861] The system for implementing this invention utilizes AI to generate lyrics and music that are likely to become popular in the future based on popular music of the past, and provides them to users according to their tastes and preferences. Specific embodiments of this system are described below.

[0862] System Configuration

[0863] The system mainly consists of the following components:

[0864] 1. Server: Hosts the generative AI model and performs music generation and classification.

[0865] 2. User terminal: A device used by a user to access the system, such as a smartphone or computer.

[0866] 3. Database: Stores data on popular songs from the past.

[0867] Hardware and software used

[0868] Hardware: Servers, smartphones, PCs

[0869] Software: TensorFlow, Keras, database management system (e.g., MySQL)

[0870] Data processing and calculation

[0871] 1. Loading the database: The server loads a database of past popular songs and obtains information such as the song's melody, rhythm, lyrics, genre, gender, era, social climate, and region.

[0872] 2. Loading the generative AI model: The server loads the pre-trained generative AI model, which is built using TensorFlow and Keras.

[0873] 3. Prompt Creation: Based on the input from the user device, the server creates a prompt for song generation, including the filters set by the user (such as era, genre, region, etc.).

[0874] 4. Music Generation: The server inputs prompts into the generative AI model to generate a new song.

[0875] 5. Song Classification: The generated songs are classified based on criteria such as genre, gender, era, social climate, and region.

[0876] 6. Providing songs: The classified songs are provided to the user's device.

[0877] Specific examples

[0878] If a user wants to generate American pop songs from the 1980s, they can set a filter like this:

[0879] Prompt Sentence Examples

[0880] Generate a song with the following characteristics: Era: 1980s, Genre: Pop, Region: USA

[0881] By inputting this prompt into a generative AI model, a new song with characteristics of 1980s American pop songs is generated, tailored to the user's tastes and preferences.

[0882] In this way, users can enjoy new songs that utilize the characteristics of past popular songs.

[0883] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0884] Step 1:

[0885] The server loads a database of past popular songs, which includes information such as the song's melody, rhythm, lyrics, genre, gender, era, social climate, and region. This allows the server to obtain characteristic data for the song.

[0886] Step 2:

[0887] The server loads a pre-trained generative AI model. This model is built using TensorFlow and Keras and learns the characteristics of popular songs from the past. This prepares the model for music generation.

[0888] Step 3:

[0889] The user sets filters (such as era, genre, region, etc.) for music generation using the device, and the user's preferences and tastes are acquired as input data.

[0890] Step 4:

[0891] The server creates prompts based on the filters set by the user, including information such as the era, genre, and region specified by the user, which then generates prompt sentences to be fed into the generative AI model.

[0892] Step 5:

[0893] The server inputs prompts into the generative AI model to generate new music. The generative AI model generates new melodies and lyrics based on the characteristics of popular songs from the past. This results in new music tailored to the user's preferences.

[0894] Step 6:

[0895] The server classifies the generated songs by criteria such as genre, gender, age, social climate, and region, allowing the generated songs to be organized based on meshes.

[0896] Step 7:

[0897] The server provides the classified songs to the user's terminal, and the user can listen to the generated songs through the terminal, thereby allowing the user to enjoy new songs.

[0898] Example 2

[0899] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0900] Conventional music generation systems have made it difficult for users to generate new lyrics and music that reflect the characteristics of a particular musical style or era. Furthermore, there is a lack of means to customize the generated music to suit the user's tastes and preferences. Furthermore, there is also a lack of means to efficiently classify and provide the generated music.

[0901] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0902] In this invention, the server includes means for a user to input a prompt text, means for a terminal to send the prompt text to the server, means for the server to receive the prompt text and pass it to a generative AI model, means for the generative AI model to generate new lyrics and music, means for the server to receive the generated lyrics and music and send it to the terminal, means for the terminal to display the generated lyrics and music to the user, means for generating lyrics and music that are likely to become popular in the future using generative AI based on music that was popular in the past, means for classifying the generated lyrics and music by mesh (genre, gender, age, social climate, region, etc.), and means for providing the classified lyrics and music as music or a distribution service, etc. This allows a user to generate new lyrics and music by specifying a specific musical style and efficiently classify and provide them.

[0903] "User" means an individual or organization that uses the system to input prompts and generate new lyrics and music.

[0904] A "terminal" is a device that allows a user to input a prompt and communicate with a server. Examples include a PC or smartphone.

[0905] The "server" is a computer system that receives prompts, passes them to the generative AI model, and sends the generated lyrics and music to the terminal.

[0906] A "prompt sentence" is text that a user enters to specify characteristics of a particular musical style or era.

[0907] A "generative AI model" is an artificial intelligence model that learns the characteristics of popular songs from the past and generates new lyrics and music based on the results of that learning.

[0908] "Lyrics and Music" refers to the lyrics and melody generated by the generative AI model.

[0909] "Mesh" is a method of classifying generated lyrics and music based on criteria such as genre, gender, age, social climate, and region.

[0910] "Music and distribution services" refers to platforms and services that provide users with created lyrics and music. Examples include music streaming services and download services.

[0911] "Deep learning" is a machine learning technique that generative AI models use to learn the characteristics of popular songs from the past.

[0912] "Customizable means" refers to a function that allows users to adjust the mesh classification results to suit their preferences and tastes.

[0913] MODE FOR CARRYING OUT THE INVENTION

[0914] This invention is a system that allows users to create new lyrics and music that reflect the characteristics of a particular musical style or era. Specific embodiments of this system are described below.

[0915] System configuration

[0916] This system consists of a user, a terminal, a server, and a generative AI model. The user inputs a prompt sentence using the terminal, and the server generates new lyrics and music using the generative AI model.

[0917] Hardware and software used

[0918] Device: a device such as a computer, smartphone, or tablet.

[0919] Server: A high-performance computer system

[0920] Generative AI model: Artificial intelligence model using deep learning

[0921] Data processing and calculation

[0922] When a user inputs a prompt text through their device, the device sends it to the server. The server receives the prompt text and passes it to the generative AI model. The generative AI model learns the characteristics of popular songs from the past and generates new lyrics and music based on the learning results. The generated lyrics and music are sent from the server to the device and displayed to the user.

[0923] Specific examples

[0924] For example, if a user inputs a prompt such as "Please create a new song in the style of a 1980s pop song," the following specific actions will be taken:

[0925] 1. The user types into a text box on the device, "Please create a new song in the style of a 1980s pop song."

[0926] 2. The terminal sends this prompt text to the server as an HTTP request.

[0927] 3. The server receives the prompt and passes the instruction to the generative AI model: "Create a new song in the style of a 1980s pop song."

[0928] 4. The generative AI model generates new songs based on the characteristics of 1980s pop music, such as songs with synthesizer sounds and catchy melody lines.

[0929] 5. The server receives the generated song and sends it to the device as an HTTP response.

[0930] 6. The device displays the generated song to the user, who can then play and enjoy the song.

[0931] In this way, users can specify a specific musical style and use the generative AI model to generate new lyrics and music. The system can also efficiently classify the generated lyrics and music and customize them to suit the user's tastes and preferences.

[0932] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0933] Step 1:

[0934] The user enters a prompt statement.

[0935] Specifically, the user types into a text box on the device, "Please create a new song in the style of a 1980s pop song."

[0936] Input: The prompt text entered by the user

[0937] Output: The prompt displayed on the terminal

[0938] Step 2:

[0939] The terminal sends the prompt to the server.

[0940] As a specific operation, the terminal generates an HTTP request and transmits data including a prompt sentence to the server.

[0941] Input: The prompt text entered by the user

[0942] Output: HTTP request sent to the server

[0943] Step 3:

[0944] The server receives the prompt sentence and passes it to the generative AI model.

[0945] Specifically, the server receives an HTTP request, analyzes the prompt text, and converts it into a data format to be passed to the generative AI model.

[0946] Input: The HTTP request sent to the server

[0947] Output: The prompt text converted into a data format to be passed to the generative AI model.

[0948] Step 4:

[0949] Generative AI models generate new lyrics and music.

[0950] Specifically, the generative AI model generates a new song that reflects the characteristics of 1980s pop music based on a prompt.

[0951] Input: The prompt passed to the generative AI model

[0952] Output: The new lyrics and music generated

[0953] Step 5:

[0954] The server receives the generated lyrics and music and sends them to the device.

[0955] Specifically, the server receives the lyrics and music generated by the generative AI model and sends them to the terminal as an HTTP response.

[0956] Input: New lyrics and music received from a generative AI model

[0957] Output: HTTP response sent to the device

[0958] Step 6:

[0959] The terminal displays the generated lyrics and music to the user.

[0960] Specifically, the device analyzes the HTTP response received from the server and displays the generated lyrics and music to the user, who can then play and enjoy the music.

[0961] Input: HTTP response received from the server

[0962] Output: New lyrics and music displayed to the user

[0963] (Application example 2)

[0964] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0965] While conventional music generation systems could generate new songs based on past songs, they lacked the ability to generate songs based on a generative AI model by inputting specific prompts from the user. Furthermore, the means for streaming, downloading, and sharing the generated songs were limited, making it difficult to meet the diverse needs of users. Furthermore, the ability to customize the generated songs to suit the user's tastes and preferences was also insufficient.

[0966] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0967] In this invention, the server includes: means for generating lyrics and music that are likely to become popular in the future using generative AI based on past popular music; means for classifying the generated lyrics and music by mesh (genre, gender, age, social climate, region, etc.); means for providing the classified lyrics and music through music distribution services, etc.; means for a user to input a prompt and generate new music based on the generative AI model; and means for streaming, downloading, and sharing the generated music. This allows users to input specific prompts and generate music based on the generative AI model, and the generated music can be provided in a variety of ways. It can also be customized to suit the user's preferences and tastes.

[0968] "Generative AI" is artificial intelligence that learns from past data and generates new data based on the results of that learning.

[0969] A "prompt" is an instruction that the user inputs to the generation AI, and is used to specify the content and characteristics of the data to be generated.

[0970] "Mesh" is a method of categorizing data based on different criteria such as genre, gender, age, social climate, and region.

[0971] "Streaming playback" is a method of playing data in real time over the Internet.

[0972] "Download" is a method of saving data to a user's device via the internet.

[0973] "Sharing" refers to the method of sharing the generated data with other users via the Internet.

[0974] "Customization" is the practice of tailoring data and services to a user's tastes and preferences.

[0975] A system for implementing the present invention is configured as follows.

[0976] System Configuration

[0977] 1. Server:

[0978] Generative AI: Artificial intelligence that generates new lyrics and music based on popular songs from the past. It uses deep learning to learn the characteristics of past songs and generates new music based on the results of that learning.

[0979] Classification method: The generated lyrics and music are classified by criteria such as genre, gender, age, social climate, and region.

[0980] Provision method: Classified lyrics and songs are provided as music or through distribution services.

[0981] Generation method: A new song is generated using a generative AI model based on a prompt entered by the user.

[0982] Playback and sharing options: Stream, download, and share the generated songs.

[0983] 2. Terminal:

[0984] User interface: An interface for users to enter prompt text. It works on devices such as smartphones and PCs.

[0985] Playback and sharing features: Features for playing, downloading, and sharing the generated songs.

[0986] Program processing explanation

[0987] The server uses generative AI to learn the characteristics of past popular songs and generates new lyrics and music based on the learning results. When the user inputs a prompt through their device, the server uses a generative AI model based on the prompt to generate a new song. The generated songs are categorized by criteria such as genre, gender, age, social climate, and region, and are customized to suit the user's tastes and preferences. The generated songs can be streamed, downloaded, and shared.

[0988] Hardware and software used

[0989] Hardware: Servers, smartphones, PCs

[0990] Software: Python, OpenAI API, user interface application

[0991] Specific examples

[0992] A user opens the application on their smartphone and types, "Generate lyrics for a 1980s-style pop song with a love theme." The server receives this prompt and uses a generative AI model to generate new lyrics for a 1980s-style pop song. The generated lyrics are customized to the user's tastes and preferences and can be streamed, downloaded, or shared on their smartphone.

[0993] Prompt Sentence Examples

[0994] "Generate lyrics for a 1980s-style pop song about love."

[0995] "Generate lyrics for a 1980s-style pop song about friendship."

[0996] "Generate lyrics for a 1980s-style pop song with an adventure theme."

[0997] In this way, users can input specific prompts to generate new music based on the generative AI model and enjoy it in a variety of ways.

[0998] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0999] Step 1:

[1000] The user enters a prompt sentence at the terminal.

[1001] Input: The user enters a prompt such as "Generate lyrics for a 1980s-style pop song about love."

[1002] What it does: The terminal's user interface receives the prompt and sends it to the server.

[1003] Output: The prompt text is sent to the server.

[1004] Step 2:

[1005] The server receives the prompt and sends a request to the generative AI model.

[1006] Input: The prompt text sent from the terminal.

[1007] How it works: The server parses the prompt and sends a request to the generative AI model to generate a new song.

[1008] Output: A request is sent to the generative AI model.

[1009] Step 3:

[1010] A generative AI model generates new music based on a prompt.

[1011] Input: The prompt text sent by the server.

[1012] How it works: The generative AI model uses deep learning based on past music data to generate new music that matches the prompt.

[1013] Output: The generated music data.

[1014] Step 4:

[1015] The server receives the generated music data and classifies it into meshes.

[1016] Input: Music data sent from the generative AI model.

[1017] How it works: The server classifies music data by criteria such as genre, gender, age, social climate, and region.

[1018] Output: Classified song data.

[1019] Step 5:

[1020] The server customizes the classified music data to suit the user's tastes and preferences.

[1021] Input: Categorized song data and information about user tastes and preferences.

[1022] How it works: The server customizes music data based on the user's tastes and preferences.

[1023] Output: Customized song data.

[1024] Step 6:

[1025] The server transmits the customized music data to the terminal.

[1026] Input: Customized song data.

[1027] How it works: The server sends customized music data to the device.

[1028] Output: Song data sent to the device.

[1029] Step 7:

[1030] The terminal provides the received music data to the user.

[1031] Input: Song data sent from the server.

[1032] What it does: Provides an interface for devices to stream, download, and share music data.

[1033] Output: Your song is ready for users to play, download, and share.

[1034] In this way, users can input specific prompts to generate new music based on the generative AI model and enjoy it in a variety of ways.

[1035] Example 3

[1036] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1037] Conventional music generation systems have had problems with recommending music that meets a user's specific tastes and preferences, and they are unable to adequately categorize or customize the music they generate. Furthermore, they lack the technology to learn the characteristics of past popular music and generate new music based on that.

[1038] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1039] In this invention, the server includes a means for a user to input a prompt sentence, a means for generating lyrics and music using a generative AI model, a means for classifying the generated lyrics and music by genre, gender, age, social climate, region, etc., a means for recommending the classified lyrics and music to the user, and a means for the user to customize the recommended music. This enables music to be recommended according to the user's specific tastes and preferences, and makes it easy to classify and customize the generated music. It can also learn the characteristics of past popular music and generate new music based on that.

[1040] A "prompt sentence" is specific text that a user inputs to give instructions to a generative AI model for generating music.

[1041] A "generative AI model" is an artificial intelligence model that uses deep learning to learn the characteristics of popular songs from the past and generates new lyrics and music based on the results of that learning.

[1042] "Genre" is a classification item that indicates the style or type of music, and includes, for example, rock, pop, jazz, etc.

[1043] "Gender" is a classification based on the target audience of the song or the gender of the artist.

[1044] "Era and Social Condition" is a classification category based on the historical background and social situation reflected in the music.

[1045] "Region" is a classification that indicates that a song is associated with a particular geographic area or culture.

[1046] "Recommendation" refers to the act of presenting the most suitable music to the user based on the user's past preference data and classification results.

[1047] "Customization" refers to the act of a user adjusting the classification results of recommended songs to suit their own tastes and preferences.

[1048] MODE FOR CARRYING OUT THE INVENTION

[1049] This invention is a system in which a user inputs a prompt sentence, a generative AI model is used to generate new lyrics and music, and the system classifies them by genre, gender, age, social climate, region, etc., recommends them to the user, and the user can further customize the recommended music.

[1050] 1. Program Generation

[1051] The server generates lyrics and music using a generative AI model. This generative AI model uses a general artificial intelligence model that uses deep learning technology, for example. Specifically, Python libraries such as TensorFlow and PyTorch are often used.

[1052] 2. Data Classification

[1053] The server classifies the generated lyrics and music by genre, gender, age, social climate, region, etc. This classification is performed using natural language processing technology and machine learning algorithms. Specifically, Python's NLTK and Scikit-learn are used.

[1054] 3. Category Recommendation

[1055] The server then recommends the classified songs to the user. This recommendation system uses collaborative filtering and content-based filtering. Specifically, Apache Mahout and TensorFlow are sometimes used. The server recommends the most suitable songs based on the user's past preference data.

[1056] 4. Customization features

[1057] Users can customize Mesh's classification results to suit their preferences and tastes. For example, a user can set specific preferences, such as preferring the "1980s" category but not the "Pop" category. This customization is provided through a user interface using front-end frameworks such as React and Vue.js.

[1058] Specific examples

[1059] User A likes 1980s rock music but not pop music. The server uses a generative AI model to generate a new song and classify it into the categories "1980s" and "rock." User A is recommended this song based on his preferences.

[1060] Prompt Sentence Examples

[1061] "Generate a 1980s-style rock song. The lyrics should have themes of youth and freedom."

[1062] In this way, a system is realized that allows users to efficiently search for music that suits their preferences. The flow of the specification process in the third embodiment will be described with reference to FIG.

[1063] Step 1:

[1064] The user enters a prompt statement.

[1065] The user opens a web browser, accesses the system interface, enters "Generate a 1980s-style rock song. The lyrics should include themes of youth and freedom" in the input field, and clicks the submit button.

[1066] Input: The prompt text entered by the user

[1067] Output: The prompt is sent to the server.

[1068] Step 2:

[1069] The server generates lyrics and music using a generative AI model.

[1070] The server sends the prompt received from the user to the generative AI model. The generative AI model generates new lyrics and music based on the prompt and returns the results to the server. The generative AI model uses a general artificial intelligence model that uses deep learning technology. Specifically, it uses the Python libraries TensorFlow and PyTorch.

[1071] Input: prompt statement

[1072] Output: Generated lyrics and music

[1073] Step 3:

[1074] The server classifies the generated lyrics and music by genre, gender, age, social climate, region, etc.

[1075] The server receives the generated lyrics and music and classifies them into categories such as "1980s" or "rock" using Python's NLTK and Scikit-learn. The classification results are stored in a database.

[1076] Input: Generated lyrics and music

[1077] Output: Classified lyrics and songs

[1078] Step 4:

[1079] The server recommends songs to users based on the classification results.

[1080] The server refers to the user's past preference data and recommends the most suitable songs. This recommendation system uses collaborative filtering and content-based filtering. Specifically, Apache Mahout and TensorFlow are used. If a user has previously liked "1980s" and "Rock," new songs in the same category will be recommended.

[1081] Input: Classified lyrics and songs, user preference data

[1082] Output: Recommended songs

[1083] Step 5:

[1084] Users customize recommended songs.

[1085] Users can review the list of recommended songs and filter out categories that don't suit their tastes. For example, they can leave the "1980s" category and filter out the "Pop" category. This customization feature is provided through a user interface using front-end frameworks such as React and Vue.js.

[1086] Input: Recommended songs, user customization settings

[1087] Output: Customized song list

[1088] (Application example 3)

[1089] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1090] Conventional music distribution systems make it difficult for users to efficiently find music that matches their preferences. Furthermore, music recommendations often fail to accommodate users' specific tastes, making it difficult to discover new music. Furthermore, there is a lack of technology to generate new music based on past trends, making it impossible to meet the diverse needs of users.

[1091] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1092] In this invention, the server includes means for generating lyrics and music that are likely to become popular in the future using generative AI based on past popular music, means for classifying the generated lyrics and music by mesh (genre, gender, age, social climate, region, etc.), means for providing the classified lyrics and music as music or through a distribution service, etc., means for recommending music based on user preferences, means for specifying user preferences by genre, age, and region, and means for filtering and recommending music that matches the user's preferences. This enables users to efficiently find music that suits their preferences and makes it easier to discover new music.

[1093] "Past popular songs" are songs that were popular with many people and widely listened to during a certain period in the past.

[1094] "Generative AI" is a system that uses artificial intelligence technology to generate new data and content.

[1095] "Lyrics and music" refers to the lyrics and melody of a musical work.

[1096] "Mesh" is a method of classifying data based on multiple categories such as genre, gender, age, social climate, and region.

[1097] A "genre" is a category that classifies musical styles and forms.

[1098] "Gender" refers to the gender of the song's target listeners.

[1099] "Era and social conditions" refers to a specific era and the social situation of that era.

[1100] "Region" refers to a specific geographical location or cultural context.

[1101] "Music and distribution services" refers to platforms and services that provide musical works to users.

[1102] "User preferences" refers to musical characteristics such as genres, eras, and regions that individual users particularly prefer.

[1103] "Filtering" is the process of sorting data based on specific criteria.

[1104] "Recommendation" is the act of suggesting appropriate music based on the user's preferences.

[1105] "Deep learning" is a machine learning technique that uses multi-layer neural networks to learn the characteristics of data.

[1106] The following system configuration will be described as an embodiment of the present invention.

[1107] The server has a means to generate lyrics and music that are likely to become popular in the future using generative AI based on past popular songs. This generative AI uses deep learning to learn the characteristics of past popular songs and generates new lyrics and music based on the learning results. The generated lyrics and music are categorized by meshes such as genre, gender, age, social climate, and region.

[1108] The classified lyrics and music are then provided as music or through music distribution services. Users access the distribution service using a device such as a smartphone and are recommended songs based on their preferences. Users can specify their preferences by genre, age, and region, which allows the system to filter and recommend songs that match their preferences.

[1109] For example, if a user likes "1980s pop songs," the server uses generative AI to learn the characteristics of 1980s pop songs and generate new songs based on those characteristics. The generated songs are then categorized into "1980s" and "Pop" and recommended based on the user's preferences.

[1110] The following hardware and software are used to realize this system. The server has high-performance computing resources to run the generative AI model. Specifically, a server equipped with a GPU is suitable. A deep learning framework (e.g., TensorFlow or PyTorch) is used as software. An application for accessing the music distribution service is installed on the user's device. This application provides an interface for inputting the user's preferences and displays songs recommended by the server.

[1111] Examples of prompts to input to a generative AI model include:

[1112] Write a Python program that recommends songs based on user preferences. The song database contains information about title, genre, era, and region. User preferences are specified by genre, era, and region. Write a function that filters and recommends songs that match the user's preferences.

[1113] In this way, users can efficiently search for music that suits their tastes and easily discover new music.

[1114] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1115] Step 1:

[1116] The server acquires a database of past popular songs.

[1117] Input: Database of popular songs from the past

[1118] Output: Get the music database

[1119] Specific operation: The server accesses a database of past popular songs and obtains information such as title, genre, era, and region.

[1120] Step 2:

[1121] The server uses a generative AI model to learn the characteristics of popular songs from the past.

[1122] Input: Song database

[1123] Output: Trained generative AI model

[1124] How it works: The server uses a deep learning framework (e.g., TensorFlow or PyTorch) to learn the features of the music database, allowing the generative AI model to understand the characteristics of past popular songs.

[1125] Step 3:

[1126] The server generates new lyrics and music using a trained generative AI model.

[1127] Input: trained generative AI model

[1128] Output: Generated lyrics and music

[1129] How it works: The server inputs prompts into the trained generative AI model to generate new lyrics and music. The generated lyrics and music reflect the characteristics of popular songs from the past.

[1130] Step 4:

[1131] The server classifies the generated lyrics and music into meshes (genre, gender, age, social climate, region, etc.).

[1132] Input: Generated lyrics and music

[1133] Output: Classified lyrics and songs

[1134] Specific operation: The server classifies the generated lyrics and music based on categories such as genre, gender, era, social climate, and region, making it clear that the music belongs to a specific category.

[1135] Step 5:

[1136] The server provides the classified lyrics and music as music or distribution services.

[1137] Input: Classified lyrics and songs

[1138] Output: Songs provided to the distribution service

[1139] Specific operation: The server uploads the classified lyrics and music to a music distribution service, making them accessible to users.

[1140] Step 6:

[1141] A user uses a terminal to access a distribution service and input their preferences.

[1142] Input: User preferences (genre, era, region)

[1143] Output: User preference settings

[1144] Specific operation: The user accesses the distribution service using a device such as a smartphone and enters preferences such as genre, era, and region.

[1145] Step 7:

[1146] The server filters and recommends songs based on the user's preferences.

[1147] Input: User preferences, categorized lyrics and songs

[1148] Output: Recommended songs

[1149] Specific operation: The server filters appropriate songs from the classified lyrics and music based on the user's preferences and recommends them to the user, allowing the user to efficiently find songs that suit their tastes.

[1150] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1151] "Example 1"

[1152] One embodiment of the present invention is a system that combines an emotion engine that recognizes user emotions. This system uses generative AI to generate lyrics and music that are likely to become popular in the future based on past popular songs, classifies the generated lyrics and music by mesh (genre, gender, age, social climate, region, etc.), and provides the classified lyrics and music as music or through a distribution service.

[1153] "Example 2"

[1154] The emotion engine recognizes the user's emotions and adjusts the generation and classification of lyrics and music based on those emotions. For example, if the user expresses joy, the emotion engine communicates that information to the generation AI, which then generates lyrics and music with a joyful theme. Conversely, if the user expresses sadness, the emotion engine communicates that information to the generation AI, which then generates lyrics and music with a sadness theme.

[1155] "Example 3"

[1156] Furthermore, the emotion engine recognizes the user's emotions and adjusts the delivery of lyrics and music based on those emotions. For example, if the user expresses a happy emotion, the emotion engine communicates that information to the delivery service, and the delivery service provides the user with lyrics and music with a happy theme. Conversely, if the user expresses a sad emotion, the emotion engine communicates that information to the delivery service, and the delivery service provides the user with lyrics and music with a sad theme.

[1157] The processing flow of each embodiment will be described below.

[1158] "Example 1"

[1159] Step 1: Recognize the user's emotions with the emotion engine.

[1160] Step 2: Based on the recognized emotions, the generative AI generates lyrics and music.

[1161] Step 3: Classify the generated lyrics and music into meshes (genre, gender, era, social climate, region, etc.).

[1162] Step 4: Provide the classified lyrics and music as music or through a distribution service.

[1163] "Example 2"

[1164] Step 1: Recognize the user's emotions with the emotion engine.

[1165] Step 2: Based on the recognized emotions, the generative AI generates lyrics and music.

[1166] Step 3: Classify the generated lyrics and music into meshes (genre, gender, era, social climate, region, etc.).

[1167] Step 4: Provide the classified lyrics and music as music or through a distribution service.

[1168] "Example 3"

[1169] Step 1: Recognize the user's emotions with the emotion engine.

[1170] Step 2: Based on the recognized emotions, the generative AI generates lyrics and music.

[1171] Step 3: Classify the generated lyrics and music into meshes (genre, gender, era, social climate, region, etc.).

[1172] Step 4: Provide the classified lyrics and music as music or through a distribution service.

[1173] Example 1

[1174] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1175] Conventional music generation systems have difficulty generating new music that reflects the characteristics of past popular songs, and are unable to provide music that matches the user's emotions. Furthermore, there is a lack of means to customize the generated music to suit the user's tastes and preferences. This makes it difficult to increase user satisfaction.

[1176] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1177] In this invention, the server includes means for generating lyrics and music that are likely to become popular in the future using a generative AI based on music that has been popular in the past, means for classifying the generated lyrics and music by genre, gender, age, social climate, region, etc., means for providing the classified lyrics and music through a music distribution service, etc., means for acquiring emotion data using an emotion engine that recognizes user emotions, and means for generating new music using a generative AI model based on the acquired emotion data. This makes it possible to generate and provide music that matches the user's emotions, thereby increasing user satisfaction.

[1178] "Generative AI" is an artificial intelligence technology that learns from past data and generates new data based on the results of that learning.

[1179] An "emotion engine" is a technology that analyzes emotions from a user's facial expressions, voice, etc. and acquires that data.

[1180] A "music distribution service" is a service that provides music to users via the Internet.

[1181] "Genre" is a category for classifying the style or type of music.

[1182] "Gender" is information indicating the gender of the user targeted by the song.

[1183] "Decade" is information that indicates the time when the song was popular and the age group of the target users.

[1184] "Social trends" refers to information that indicates the social background and cultural situation at the time a song became popular.

[1185] "Region" is information indicating the geographical location where a song became popular.

[1186] "Emotion data" is information about a user's emotions analyzed by the emotion engine.

[1187] "Machine learning" is a technology in which a computer learns patterns from data and makes predictions and judgments based on the results of that learning.

[1188] "Mesh" is a method of classifying music into multiple categories such as genre, gender, era, social climate, and region.

[1189] MODE FOR CARRYING OUT THE INVENTION

[1190] This invention is a system that builds a database of past popular songs and uses generative AI to generate new songs based on that database. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it is possible to provide songs that correspond to the user's emotions.

[1191] Building a database

[1192] The server builds a database of songs that were popular in the past. This database includes information such as the song's melody, rhythm, lyrics, genre, gender, era, social context, and region. Specifically, the data is stored using a database management system such as MySQL. For example, the server inserts data by executing the SQL query "INSERT INTO songs (melody, rhythm, lyrics, genre, gender, era, social_context, region) VALUES (...)".

[1193] Generative AI training

[1194] The server uses the constructed database to train a generative AI model. This generative AI model uses, for example, OpenAI's GPT-4. During the training process, the features of past songs are extracted and patterns are recognized. Specifically, the server calls a function called "train_model(database)" and supplies the data in the database to the model.

[1195] Emotion engine emotion recognition

[1196] The device is equipped with an emotion engine to recognize the user's emotions. This emotion engine uses, for example, Microsoft's Azure Cognitive Services. It analyzes emotions from the user's facial expressions and voice and sends the data to the server. Specifically, the device calls the function "emotion_data = recognize_emotion(user_input)" and sends the obtained emotion data to the server.

[1197] Music generation

[1198] The server uses a generative AI model to generate lyrics and music for a new song based on the user's emotional data sent from the emotion engine. The generated song reflects the characteristics of past popular songs. Specifically, the server calls the function "new_song = generate_song(emotion_data)" to generate a new song.

[1199] Song classification

[1200] The server classifies the generated songs into a mesh of genre, gender, era, social climate, region, etc. This classification is performed using a search engine such as Elasticsearch. Specifically, the server calls a function called "classify_song(new_song)" to classify the songs into the appropriate category.

[1201] Music provided by

[1202] Users can receive songs generated through their devices through music distribution services, such as Spotify and Apple Music. For example, users can play songs using the "stream_song(new_song)" function.

[1203] Examples and prompts

[1204] Specific examples

[1205] If the emotion engine determines that the user is in a sad mood, the device sends the data "emotion_data = {'emotion': 'sad'}" to the server. The server receives this data and calls the function "new_song = generate_song({'emotion': 'sad'})" to generate a new sad song. The generated song is classified as a "breakup song" and distributed on Spotify through the function "stream_song(new_song)".

[1206] Prompt Sentence Examples

[1207] "If the emotion engine recognizes that the user is in a sad mood, generate a new sad song using a generative AI model that has learned the characteristics of past sad songs."

[1208] In this way, the system can generate music that responds to the user's emotions and provide it on the appropriate platform.

[1209] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1210] Program processing flow

[1211] Step 1: Building the database

[1212] The server builds a database of songs that were popular in the past. As input, it receives information such as the song's melody, rhythm, lyrics, genre, gender, era, social context, and region. Specifically, the server uses a database management system such as MySQL to insert data by executing the SQL query "INSERT INTO songs (melody, rhythm, lyrics, genre, gender, era, social_context, region) VALUES (...)". As output, the song data is saved in the database.

[1213] Step 2: Training the generative AI

[1214] The server uses the constructed database to train the generative AI model. It receives the music data in the database as input. Specifically, the server calls a function called "train_model(database)" and supplies the data in the database to the model. The output is a generative AI model that has learned the characteristics of past music.

[1215] Step 3: Emotion Recognition in the Emotion Engine

[1216] The device is equipped with an emotion engine to recognize the user's emotions. It receives the user's facial expressions and voice data as input. Specifically, the device calls the function "emotion_data = recognize_emotion(user_input)" and sends the obtained emotion data to the server. The user's emotion data is obtained as output.

[1217] Step 4: Create a song

[1218] The server uses a generative AI model to generate lyrics and music for a new song based on the user's emotional data sent from the emotion engine. It receives the emotional data and generative AI model as input. Specifically, the server calls the function "new_song = generate_song(emotion_data)" to generate a new song. The generated song is obtained as output.

[1219] Step 5: Classify your songs

[1220] The server classifies the generated songs into meshes such as genre, gender, era, social climate, and region. It receives the generated song data as input. Specifically, the server calls a function called "classify_song(new_song)" to classify the song into the appropriate category. The output is the classified song data.

[1221] Step 6: Submit your music

[1222] The user can receive the generated music through a music distribution service or other means via their device. The input is the classified music data. Specifically, the user plays the music using the function "stream_song(new_song)". The output is the user listening to the music.

[1223] In this way, the system can generate music that responds to the user's emotions and provide it on the appropriate platform.

[1224] (Application example 1)

[1225] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1226] Conventional music generation systems were able to generate new music based on popular songs from the past, but it was difficult to generate and distribute music that responded to the user's emotions in real time. Furthermore, there was a lack of means to customize the generated music to suit the user's tastes and preferences. This made it impossible to provide a personalized music experience that responded to the user's emotions and preferences.

[1227] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1228] In this invention, the server includes: means for generating lyrics and music that are likely to become popular in the future using generative AI based on music that has been popular in the past; means for classifying the generated lyrics and music by mesh (genre, gender, age, social climate, region, etc.); means for providing the classified lyrics and music as music or a distribution service, etc.; means for generating music that corresponds to the user's emotions using an emotion engine that recognizes the user's emotions; and means for streaming the generated music in real time. This makes it possible to generate and distribute music that corresponds to the user's emotions in real time, and further customize it to suit the user's tastes and preferences.

[1229] "Past popular songs" are musical works that were popular with many people and widely listened to during a certain period in the past.

[1230] "Generative AI" is a system that uses artificial intelligence technology to automatically generate new music and lyrics.

[1231] "Mesh" is a method of classifying data based on different categories such as genre, gender, age, social climate, and region.

[1232] The "emotion engine" is a system that recognizes emotions from the user's facial expressions, voice, etc. and analyzes those emotions.

[1233] "Real-time streaming" is a technology that instantly distributes created music to users via the Internet.

[1234] "Deep learning" is a machine learning technique that uses multi-layered neural networks to learn features from large amounts of data and make predictions and generate results.

[1235] "Customizable" refers to the ability to change the system settings and output according to the user's preferences and tastes.

[1236] A system for implementing the present invention has the following configuration.

[1237] System configuration

[1238] 1. Emotion Recognition Module

[1239] The server uses the smartphone camera and head-mounted display sensors to recognize emotions from the user's facial expressions and voice, using software such as OpenCV and Keras.

[1240] 2. Music Generation Module

[1241] The server uses a generative AI to generate new music based on a database of past popular songs. The generative AI uses deep learning to learn the characteristics of past songs and generate music that matches the user's emotions. The music is generated using a proprietary music generation AI (Music Generator).

[1242] 3. Song Classification Module

[1243] The server categorizes the generated songs based on categories such as genre, gender, age, social climate, and region, and the categorized songs can be customized to suit the user's tastes and preferences.

[1244] 4. Music distribution module

[1245] The server then streams the generated music in real time, using technology that provides music to users over the Internet.

[1246] Processing flow

[1247] 1. Emotion recognition

[1248] When a user uses a smartphone camera or a head-mounted display, the server captures the user's facial expressions and voice, detects the face using OpenCV, and inputs the face into a Keras emotion recognition model to predict the user's emotions.

[1249] 2. Music Generation

[1250] The server generates new music based on the user's emotions using a generative AI that has learned from a database of popular songs from the past. The generated music is optimized to match the user's emotions.

[1251] 3. Song Classification

[1252] The server categorizes the generated songs based on genre, gender, age, social climate, region, etc. The categorized songs are then customized to suit the user's tastes and preferences.

[1253] 4. Music distribution

[1254] The server then streams the generated music in real time, allowing users to instantly listen to it via their smartphones or head-mounted displays.

[1255] Specific examples

[1256] For example, if a user has a sad expression, the server uses the emotion recognition module to detect "sadness." The music generation module generates a new song with a melody and lyrics that correspond to "sadness," and the song classification module classifies it appropriately. Finally, the music distribution module streams the generated song to the user in real time.

[1257] Prompt Sentence Examples

[1258] "You will develop an application that recognizes the user's emotions and generates new music based on those emotions. The emotion recognition is performed using a smartphone camera, and the generated music is based on a database of past popular songs."

[1259] In this way, a system can be realized that generates and distributes music in real time according to the user's emotions.

[1260] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1261] Step 1:

[1262] The server captures the user's facial expressions and voice using the smartphone's camera and head-mounted display's sensors. The input is the user's real-time video and audio, and the output is the captured image and audio data. Specifically, it activates the camera and microphone and records the user's face and voice.

[1263] Step 2:

[1264] The server performs face detection on the captured image data using OpenCV. The input is the image data obtained in step 1, and the output is the area information of the detected face. Specifically, the image data is converted to grayscale and the face detection algorithm is applied to identify the position of the face.

[1265] Step 3:

[1266] The server inputs the detected face region into the Keras emotion recognition model to predict the user's emotion. The input is the face region information obtained in step 2, and the output is the predicted emotion label. Specifically, the server resizes the face region to an appropriate size and inputs it into the emotion recognition model to classify the emotion.

[1267] Step 4:

[1268] The server uses a generative AI to generate new music based on the predicted emotions. The input is the emotion label obtained in step 3, and the output is the generated music data. Specifically, the emotion label is input to the generative AI as a prompt, and a new music piece is generated based on a database of popular music from the past.

[1269] Step 5:

[1270] The server classifies the generated songs based on categories such as genre, gender, era, social climate, and region. The input is the song data obtained in step 4, and the output is the classified song data. Specifically, it analyzes the song metadata and classifies them into appropriate categories.

[1271] Step 6:

[1272] The server customizes the classified songs to suit the user's tastes and preferences. The input is the classified song data obtained in step 5 and the user's tastes and preferences, and the output is customized song data. Specifically, the server refers to the user's profile information and adjusts the song playlist.

[1273] Step 7:

[1274] The server streams the customized music in real time. The input is the customized music data obtained in step 6, and the output is the music played on the user's device. Specifically, the music data is sent to the user's device via the Internet and playback begins.

[1275] In this way, a system can be realized that generates and distributes music in real time according to the user's emotions.

[1276] Example 2

[1277] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1278] Conventional music generation systems have difficulty customizing music to suit the user's emotions, making it impossible to provide music that matches the user's emotions. In addition, the classification and provision methods for generated music are limited, making it difficult to meet the diverse needs of users.

[1279] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for generating lyrics and music that are likely to become popular in the future using a generation AI based on music that was popular in the past; means for classifying the generated lyrics and music by mesh (genre, gender, age, social climate, region, etc.); means for providing the classified lyrics and music through music or distribution services; means for recognizing a user's emotions and adjusting the generation and classification of lyrics and music based on the emotions; means for transmitting the user's emotion data to the server; means for generating an appropriate prompt for the generation AI based on the emotion data; and means for providing the generated lyrics and music to the user's terminal. This makes it possible to generate and provide music customized to the user's emotions.

[1280] "Generative AI" is an artificial intelligence that learns the characteristics of popular songs from the past and generates new lyrics and music based on the results of that learning.

[1281] "Mesh" is a method of classifying generated lyrics and music based on criteria such as genre, gender, age, social climate, and region.

[1282] The "Emotion Engine" is a system that recognizes the user's emotions and adjusts the generation and classification of lyrics and music based on those emotions.

[1283] A "prompt sentence" is a text sentence that the emotion engine generates to give instructions to the generation AI.

[1284] "User emotion data" refers to data that includes emotion information recognized from the user's facial expressions and behavior.

[1285] A "server" is a computer system that manages the entire system, including the generative AI and emotion engine, and processes and communicates data.

[1286] "Terminal" refers to the device used by the user to perform emotion recognition and listen to the generated lyrics and music.

[1287] The "classification means" refers to a method or device for classifying the generated lyrics and music based on meshes.

[1288] "Providing means" means a method or device for providing classified lyrics and music to users.

[1289] The present invention is a system for generating and providing music customized according to the emotions of a user. A specific embodiment of this system is described below.

[1290] Hardware and software used

[1291] Server: A server equipped with a high-performance GPU (e.g., a computer equipped with a high-performance graphics processor) is used to train and infer generative AI models.

[1292] Device: The device used by the user (e.g., a smartphone or personal computer) has emotion recognition software (e.g., an emotion recognition software development kit) installed.

[1293] Data processing and calculation

[1294] User emotion recognition

[1295] Users input their facial expressions through the device's camera, for example, by taking a photo of their face using a smartphone camera.

[1296] The device uses emotion recognition software to recognize emotions from the user's facial expressions. For example, if the user smiles, the device will recognize the emotion "happiness."

[1297] The device sends the recognized emotion data (e.g., happiness, sadness) to the server in JSON format.

[1298] Emotional Data Processing

[1299] The server inputs the received emotion data into the emotion engine. For example, the server passes emotion data of "joy" to the emotion engine.

[1300] The emotion engine prepares to generate appropriate prompt sentences based on the user's emotions.

[1301] Generate prompt statement

[1302] The emotion engine generates prompts to be passed to the generative AI based on the user's emotions, such as "The user is expressing the emotion of joy, so please generate a 1980s-style pop song."

[1303] Lyrics and music generation

[1304] The generative AI generates new lyrics and music based on the prompt received from the emotion engine, utilizing the learned musical characteristics. For example, the generative AI could generate a new pop song with a joyful theme based on the characteristics of 1980s pop music.

[1305] Providing generated results

[1306] The server sends the generated lyrics and music to the user's device, where the generated music file is downloaded to the user's device.

[1307] Users can listen to the lyrics and music generated on their device, for example by playing a new pop song on their smartphone's music player app.

[1308] Examples of concrete examples and prompts

[1309] Specific examples

[1310] User Emotion: Delight

[1311] Generated music: 1980s-style pop song

[1312] Prompt Sentence Examples

[1313] "Generate a 1980s-style pop song because the user is expressing joy."

[1314] The system can provide a customized musical experience based on the user's emotions. For example, if the user expresses sadness, the emotion engine will instruct the generative AI to generate lyrics and music with a sadness theme.

[1315] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1316] Step 1:

[1317] The user inputs their facial expressions through the device camera. Specifically, they take a picture of their face using the smartphone camera. The input is the user's facial expression data, and the output is the image data captured by the camera app.

[1318] Step 2:

[1319] The device uses emotion recognition software to recognize emotions from the user's facial expressions. Specifically, emotion recognition software such as the Affectiva SDK is used to analyze captured image data and identify the user's emotion (e.g., joy, sadness). The input is image data, and the output is recognized emotion data.

[1320] Step 3:

[1321] The device sends the recognized emotion data to the server. Specifically, the emotion data (e.g., joy, sadness) is sent to the server in JSON format. The input is the emotion data, and the output is the JSON data sent to the server.

[1322] Step 4:

[1323] The server inputs the received emotion data into the emotion engine. Specifically, the server passes the emotion data "joy" to the emotion engine. The input is emotion data in JSON format, and the output is the emotion data passed to the emotion engine.

[1324] Step 5:

[1325] The emotion engine generates an appropriate prompt based on the user's emotion. Specifically, it creates a prompt such as "The user is expressing the emotion of joy, so please generate a 1980s-style pop song." The input is emotion data, and the output is the generated prompt.

[1326] Step 6:

[1327] The generative AI generates new lyrics and music using the learned musical characteristics based on the prompt received from the emotion engine. Specifically, the generative AI generates a new pop song with a joyful theme based on the characteristics of 1980s pop music. The input is the prompt, and the output is the generated lyrics and music.

[1328] Step 7:

[1329] The server sends the generated lyrics and music to the user's device. Specifically, the generated music file is downloaded to the user's device. The input is the generated lyrics and music, and the output is the music file sent to the user's device.

[1330] Step 8:

[1331] The user listens to the lyrics and music generated on the device. Specifically, they play a new pop song on their smartphone's music player app. The input is a music file stored on the device, and the output is the music being played.

[1332] (Application example 2)

[1333] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1334] Conventional music generation systems have had difficulty generating and delivering music in real time that responds to the user's emotions. There is also a need to provide a more personalized music experience by generating music based on the user's emotions.

[1335] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1336] In this invention, the server includes means for generating lyrics and music that are likely to become popular in the future using generative AI based on past popular music, means for classifying the generated lyrics and music by mesh (genre, gender, age, social climate, region, etc.), means for providing the classified lyrics and music as music or a distribution service, etc., means for recognizing a user's emotions and adjusting the generation of lyrics and music based on those emotions, and means for distributing the generated lyrics and music in real time. This makes it possible to generate music in real time according to the user's emotions and provide a personalized music experience.

[1337] "Generative AI" is artificial intelligence that learns from past data and generates new data based on the results of that learning.

[1338] "Mesh" is a method of classifying data based on different criteria such as genre, gender, age, social climate, and region.

[1339] "Emotion recognition" is a technology that analyzes and recognizes a user's emotions from facial expressions, tone of voice, etc.

[1340] "Real-time delivery" is a technology that provides generated content to users instantly.

[1341] "Deep learning" is a type of machine learning that uses multi-layered neural networks to learn data and perform advanced pattern recognition.

[1342] "Personalization" refers to customizing services and content to suit individual users' preferences and tastes.

[1343] A system for implementing the present invention is configured as follows.

[1344] System Configuration

[1345] 1. Hardware:

[1346] Server: A server with high-performance computing power (e.g., AWS EC2, Google Cloud Compute Engine)

[1347] Devices: Smartphones (iPhone, Android), smart glasses (Google Glass, Vuzix Blade), head-mounted displays (Oculus Quest, HTC Vive)

[1348] 2. Software:

[1349] Emotion recognition libraries: Microsoft Azure Emotion API, Affectiva SDK

[1350] Generative AI models: GPT-3, GPT-4

[1351] Music generation libraries: Magenta, Jukedeck

[1352] Program processing

[1353] 1. Emotion recognition:

[1354] It captures data in real time from the device's camera and microphone.

[1355] The acquired data is input into an emotion recognition library to analyze the user's emotions.

[1356] The analysis results are sent to the server.

[1357] 2. Generation AI:

[1358] The server inputs prompt sentences into the generative AI model based on the emotional data received from the emotion recognition library.

[1359] A generative AI model generates new music based on a prompt.

[1360] The generated music data is processed using a music generation library to create the final music file.

[1361] 3. Real-time streaming:

[1362] The server streams the created music files to the user's device.

[1363] Specific examples

[1364] Example prompt sentence:

[1365] The user expresses joy. Generate a 1980s-style pop song.

[1366] The user expresses sadness. Generate a calming ballad.

[1367] This system allows users to enjoy music that perfectly matches their emotions in real time. For example, it can automatically generate relaxing music when you're tired from work, or exciting music when you're having fun with friends, which is very convenient.

[1368] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1369] Step 1:

[1370] The device captures data in real time from the camera and microphone. While the user is using the device, the camera captures the user's facial expressions and the microphone records the user's voice. This data is then input into the emotion recognition library.

[1371] Step 2:

[1372] An emotion recognition library is used to analyze the user's emotions from the acquired data. Specifically, the Microsoft Azure Emotion API and Affectiva SDK are used to analyze facial and voice data to identify the user's emotions (happiness, sadness, surprise, etc.). The analysis results are sent to the server as emotion data.

[1373] Step 3:

[1374] The server inputs a prompt sentence into the generative AI model based on the emotion data received from the emotion recognition library. For example, if the user is expressing joy, the server generates a prompt sentence such as, "The user is expressing joy. Please generate a 1980s-style pop song." This prompt sentence is input into the generative AI model.

[1375] Step 4:

[1376] Generative AI models (GPT-3 and GPT-4) generate new music based on the input prompt. The generative AI model generates music that matches the prompt based on the results of learning from past data. The generated music data is sent to a music generation library.

[1377] Step 5:

[1378] Using a music generation library (such as Magenta or Jukedeck), the generated music data is processed to create the final music file. Specifically, the music data is adjusted and effects are added to make it easier for users to listen to. The completed music file is then stored on the server.

[1379] Step 6:

[1380] The server then streams the created music files to the user's device, where the user can listen to the music created in real time, providing a musical experience tailored to the user's emotions.

[1381] Example 3

[1382] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1383] Conventional music generation systems have difficulty generating music based on past trends, and have not adequately provided music that matches the user's preferences and emotions. In addition, classifying and customizing the generated music is time-consuming, making it difficult for users to efficiently find music that matches their preferences.

[1384] The identification process by the identification processing unit 290 of the data processing device 12 in Example 3 is realized by the following means. In this invention, the server includes: means for generating lyrics and music that are likely to become popular in the future using a generation AI model based on past popular music; means for classifying the generated lyrics and music by mesh (genre, gender, age, social climate, region, etc.); means for providing the classified lyrics and music through music or distribution services; means for a user to input a request for music generation; means for customizing the generated lyrics and music to the user's tastes and preferences; and means for recognizing the user's emotions using an emotion engine and adjusting the provision of lyrics and music based on the emotions. This allows users to efficiently find music that suits their tastes and emotions.

[1385] A "generative AI model" is an algorithm or program that uses artificial intelligence technology to generate new lyrics or music.

[1386] "Mesh" is a method of classifying data based on multiple categories such as genre, gender, age, social climate, and region.

[1387] An "emotion engine" is a technology that recognizes a user's emotions and provides appropriate responses and content based on those emotions.

[1388] A "music generation request" is input information that a user uses to instruct a generative AI model to generate new lyrics and music.

[1389] "Customization" is the act of a user tailoring provided data or content to suit their own preferences and tastes.

[1390] A "distribution service" is a service that provides music and other content to users via the Internet.

[1391] This invention is a system that uses a generative AI model to generate new lyrics and music and provides them based on the user's preferences and emotions. Specific embodiments of this system are described below.

[1392] System configuration

[1393] The system consists of three main components: a server, a terminal, and a user. The server has key functions such as the generative AI model, emotion engine, and database. The terminal is a device that allows the user to access and operate the system. The user is the entity that inputs requests for music generation and receives the generated music.

[1394] Hardware and software used

[1395] Generative AI models: Examples of generative AI models include OpenAI's GPT-4 and Google's BERT.

[1396] Emotion engine: For example, Microsoft's Azure Cognitive Services or IBM's Watson are used as emotion engines.

[1397] Natural language processing tools: Python's NLTK library and SpaCy are used.

[1398] Data processing and calculation

[1399] The server receives a song generation request from the user and inputs a prompt into the generative AI model. The generative AI model generates new lyrics and music based on the prompt. The generated lyrics and music are classified into meshes such as genre, gender, era, social climate, and region. This classification is performed using natural language processing tools.

[1400] Specific examples

[1401] The user inputs a prompt such as "Generate lyrics for 1980s rock music with a joyful theme." The server sends this prompt to the generative AI model and receives the generated lyrics and music. The server then classifies the generated lyrics and music using a mesh and provides the results to the user. The user can customize the provided classification results to suit their own preferences and tastes.

[1402] Prompt Sentence Examples

[1403] Generate lyrics with a joyful theme in 1980s rock.

[1404] Emotion-Based Adjustment

[1405] The server recognizes the user's emotion using an emotion engine. For example, if the user expresses a happy emotion, the server preferentially provides songs with a happy theme. If the user expresses a sad emotion, the server preferentially provides songs with a sad theme.

[1406] In this way, the user can efficiently find music that matches his or her preferences and emotions. The flow of the identification process in the third embodiment will be described with reference to FIG.

[1407] Step 1:

[1408] A user logs in to the system.

[1409] The user accesses the system's login screen using a terminal and enters their username and password. The server checks the entered information against the database, and if authentication is successful, it loads the user's profile information and displays the dashboard.

[1410] Input: Username, Password

[1411] Output: User's dashboard

[1412] Step 2:

[1413] A user inputs a request to create a piece of music.

[1414] The user clicks the "Music Generation" button on the dashboard, enters a prompt into the input form that appears, for example, "Generate lyrics for a 1980s rock song with a theme of joy," and clicks the "Submit" button.

[1415] Input: prompt statement

[1416] Output: Request to the generative AI model

[1417] Step 3:

[1418] The server generates lyrics and music using a generative AI model.

[1419] The server receives a request from the user and sends a prompt to the generative AI model, which then generates new lyrics and music based on the prompt and returns the results to the server.

[1420] Input: prompt statement

[1421] Output: Generated lyrics and music

[1422] Step 4:

[1423] The server classifies the generated lyrics and music into meshes.

[1424] The server receives the generated lyrics and music and uses natural language processing tools such as Python's NLTK library and SpaCy to classify them into meshes such as genre, gender, age, social climate, and region. The classification results are stored in a database.

[1425] Input: Generated lyrics and music

[1426] Output: Lyrics and music classified by mesh

[1427] Step 5:

[1428] The server provides the classification results to the user.

[1429] The server displays the categorized list of lyrics and songs on the user's dashboard, where the user can scroll through the list and click on songs of interest to view more details.

[1430] Input: Lyrics and songs classified by mesh

[1431] Output: Classification results displayed on the user's dashboard

[1432] Step 6:

[1433] Users customize the classification results.

[1434] Users click the "Customize" button on the dashboard and set their preferences on the settings screen that appears. For example, they enter specific preferences such as "I like 1980s rock, but I want to exclude pop," and click the "Save" button.

[1435] Input: User preference settings

[1436] Output: Customized classification results

[1437] Step 7:

[1438] The server uses an emotion engine to recognize the user's emotion.

[1439] The server sends data acquired from the user's webcam and microphone to the emotion engine, which analyzes the user's emotions from their facial expressions and voice and returns the results to the server.

[1440] Input: User's facial expression data, voice data

[1441] Output: Recognized emotion data

[1442] Step 8:

[1443] The server adjusts the lyrics and music presentation based on emotion.

[1444] The server adjusts the lyrics and music it provides based on the user's recognized emotions. For example, if the user expresses joy, the server will prioritize joy-themed music in the list.

[1445] Input: Recognized emotion data

[1446] Output: Adjusted recommended song list

[1447] Step 9:

[1448] The server provides the final recommended songs to the user.

[1449] The server will then select the final recommended songs based on the user's preferences and emotions and display them on the user's dashboard. The user can click on the recommended songs to play and enjoy them.

[1450] Input: Adjusted recommended song list

[1451] Output: Final song recommendations displayed on the user's dashboard

[1452] (Application example 3)

[1453] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1454] Conventional music recommendation systems do not adequately provide music based on the user's emotions and preferences, making it difficult for users to efficiently find music that matches their emotions and preferences. Furthermore, even when music is generated based on past trends, it is difficult to meet the diverse needs of users.

[1455] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means. In this invention, the server includes: means for generating lyrics and music that are likely to become popular in the future using a generation AI based on music that was popular in the past; means for classifying the generated lyrics and music by mesh (genre, gender, age, social climate, region, etc.); means for providing the classified lyrics and music through music or distribution services, etc.; means for recognizing a user's emotions and adjusting the provision of lyrics and music based on those emotions; and means for customizing the mesh classification results to suit the user's preferences. This makes it possible to provide music based on the user's emotions and preferences, allowing the user to efficiently find music that matches their emotions and preferences.

[1456] A "popular song of the past" is a musical work that was widely popular during a particular period in the past.

[1457] "Generative AI" is a system that uses artificial intelligence technology to automatically generate new lyrics and music.

[1458] "Mesh" is a method of classifying data based on multiple categories such as genre, gender, age, social climate, and region.

[1459] A "classifying means" is a method or device for categorizing generated lyrics or music based on specific criteria.

[1460] "Music and distribution services" are platforms and systems for providing musical works to users.

[1461] "Means for recognizing emotions" refers to technology for detecting emotions from a user's facial expressions, voice, etc., and acquiring that information.

[1462] The "means for tailoring the presentation" is a method or device for selecting and presenting lyrics or music appropriate to the user based on the recognized emotion.

[1463] A "preference customization means" is a method or apparatus for tailoring the lyrics and music classification provided to a user according to the user's individual preferences.

[1464] "Deep learning" is a machine learning technique that uses multi-layer neural networks to learn the characteristics of data and make predictions and classifications.

[1465] A "smartphone" is a portable information terminal that, in addition to the functions of a mobile phone, can also connect to the Internet and run applications.

[1466] "Smart glasses" are a wearable device in the form of glasses that can display information and allow interaction.

[1467] A "head-mounted display" is a display device worn on the head, and is a device for providing visual information.

[1468] A "robot" is a mechanical device that operates autonomously or remotely to perform a specific task.

[1469] The following system configuration will be described as an embodiment of the present invention.

[1470] System Configuration

[1471] Hardware

[1472] Server: Manages the music database, runs generative AI models, and processes user emotion data.

[1473] Device: The device used by the user, such as a smartphone, smart glasses, head-mounted display, or robot.

[1474] Emotion recognition devices: Devices that recognize user emotions, such as cameras and microphones on smartphones and sensors on smart glasses.

[1475] software

[1476] Generative AI model: A deep learning model that learns the characteristics of popular songs from the past and generates new lyrics and music.

[1477] Emotion recognition software: Software for recognizing emotions from a user's facial expressions and voice (e.g., the EmotionRecognizer library).

[1478] Music recommendation engine: An algorithm for recommending appropriate songs based on the user's emotions and preferences.

[1479] Processing flow

[1480] 1. Emotion recognition:

[1481] The device's camera and microphone are used to capture the user's facial expressions and voice.

[1482] Emotion recognition software analyzes the captured data and recognizes the user's emotions.

[1483] 2. Music Generation:

[1484] A generative AI model on the server generates new lyrics and music based on data on past popular songs.

[1485] The generated lyrics and music are categorized into meshes such as genre, gender, age, social climate, and region.

[1486] 3. Song Recommendation:

[1487] The server filters suitable songs based on the recognized user emotions and user preference settings.

[1488] From the filtered songs, the system recommends the most suitable songs for the user.

[1489] Specific examples

[1490] For example, if a user smiles through their smartphone camera, the emotion recognition software will recognize the emotion "joy." The server will then select and recommend songs that match the user's tastes from a selection of 1980s pop songs generated using a generative AI model.

[1491] Prompt Sentence Examples

[1492] An example of a prompt for the generative AI model is as follows:

[1493] Create a smartphone application that recognizes a user's emotions and recommends songs that match those emotions. Songs can be categorized by genre, era, gender, social climate, region, etc., and can be customized to suit the user's preferences. The EmotionRecognizer library is used for emotion recognition, and the song database is provided in JSON format.

[1494] In this way, it becomes possible to provide music based on the user's emotions and preferences, and a system can be realized that allows users to efficiently find music that matches their emotions and preferences.

[1495] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1496] Step 1:

[1497] The device captures the user's facial expressions and voice using a camera and microphone. The input is the user's real-time facial expressions and voice, and the output is captured image data and audio data. Specifically, the device's camera captures the user's face, and the microphone records the user's voice.

[1498] Step 2:

[1499] The device uses emotion recognition software (e.g., EmotionRecognizer library) to analyze the captured image data and audio data and recognize the user's emotions. The input is the image data and audio data obtained in step 1, and the output is the recognized emotion (e.g., joy, sadness). Specifically, the emotion recognition software extracts facial features from the image data and analyzes the tone of voice from the audio data.

[1500] Step 3:

[1501] The server uses a generative AI model to generate new lyrics and music based on data on past popular songs. The input is data on past popular songs, and the output is the generated new lyrics and music. Specifically, the generative AI model uses deep learning to learn the characteristics of songs and generates new songs based on the learning results.

[1502] Step 4:

[1503] The server classifies the generated lyrics and music into meshes such as genre, gender, era, social climate, and region. The input is the lyrics and music generated in step 3, and the output is the classified music data. Specifically, the server analyzes the music metadata and classifies the music based on each category.

[1504] Step 5:

[1505] The server filters appropriate songs from the classified music data based on the user's emotions and preferences. The input is the emotions recognized in step 2 and the user's preferences, and the music data classified in step 4, and the output is a filtered music list. Specifically, the server searches for songs that match the user's emotions and preferences and creates a list.

[1506] Step 6:

[1507] The server selects the most suitable song for the user from the filtered song list and recommends it to the device. The input is the song list filtered in step 5, and the output is the recommended song. Specifically, the server selects the most suitable song randomly or based on an algorithm and sends it to the device.

[1508] Step 7:

[1509] The device provides the recommended songs to the user. The input is the song recommended in step 6, and the output is the song played to the user. Specifically, the device streams the song and plays it for the user.

[1510] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1511] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1512] Another example of generative AI is Gemini (internet search engine). <url: https: gemini.google.com ?hl="ja">) are mentioned.

[1513] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1514] [Third embodiment]

[1515] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1516] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1517] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1518] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1519] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1520] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1521] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1522] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1523] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1524] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1525] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1526] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1527] "Example 1"

[1528] In one embodiment of the present invention, a database of past popular songs is constructed. This database includes information such as song melody, rhythm, lyrics, genre, gender, era, social climate, and region. The generation AI learns from this database and extracts characteristics of past songs.

[1529] "Example 2"

[1530] Generative AI generates new lyrics and music based on what it has learned. For example, an AI that has learned the characteristics of 1980s pop music can generate new 1980s-style pop songs based on those characteristics.

[1531] "Example 3"

[1532] The generated lyrics and music are classified by mesh (genre, gender, era, social climate, region, etc.). This classification is used to provide music and distribution services. For example, a 1980s-style pop song would be classified into the "1980s" and "Pop" categories and recommended to users who like these categories.

[1533] "Example 4"

[1534] Users can also customize Mesh's classification results to suit their preferences. For example, users can set specific preferences, such as preferring the "1980s" category but not the "Pop" category. This allows users to efficiently find music that matches their tastes.

[1535] The processing flow of each embodiment will be described below.

[1536] "Example 1"

[1537] Step 1: Build a database of past popular songs. This database includes information such as the melody, rhythm, lyrics, genre, gender, era, social climate, and region of the song.

[1538] Step 2: The generative AI learns from this database and extracts the characteristics of past songs.

[1539] "Example 2"

[1540] Step 1: The generative AI generates new lyrics and music based on the results of its learning.

[1541] Step 2: For example, an AI that has learned the characteristics of 1980s pop music can use those characteristics to generate new 1980s-style pop songs.

[1542] "Example 3"

[1543] Step 1: The generated lyrics and music are classified into meshes (genre, gender, era, social climate, region, etc.).

[1544] Step 2: This classification is used to provide music and streaming services. For example, a 1980s-style pop song would be classified into the "1980s" and "Pop" categories and recommended to users who like those categories.

[1545] "Example 4"

[1546] Step 1: Users can customize the mesh classification results to suit their preferences and tastes.

[1547] Step 2: Users can set specific preferences, such as preferring the "1980s" category but not the "Pop" category, allowing them to efficiently find music that matches their tastes.

[1548] Example 1

[1549] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1550] In conventional music generation systems, it is difficult to effectively utilize past music data to generate new music, and it is also difficult to provide generated music that matches the user's tastes and preferences. Furthermore, there is a lack of means to analyze the characteristics of generated music in detail and provide appropriate information to the user.

[1551] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1552] In this invention, the server includes: means for generating lyrics and music that are likely to become popular in the future using a generative AI based on music that was popular in the past; means for classifying the generated lyrics and music by genre, gender, age, social climate, region, etc.; means for providing the classified lyrics and music through music or distribution services, etc.; means for building a database of music that was popular in the past; means for preprocessing music data stored in the database; means for training a generative AI model using the preprocessed data; means for extracting music characteristics from the trained data; and means for a user to input a prompt sentence and for the generative AI model to generate a response. This makes it possible to effectively utilize past music data to generate new music and provide it according to the user's tastes and preferences.

[1553] "Past popular songs" are musical works that were widely popular during a certain period in the past.

[1554] "Generative AI" is a system that uses artificial intelligence technology to generate new data and content.

[1555] "Lyrics and music" refers to the lyrics of a song and musical elements such as melody and rhythm.

[1556] A "genre" is a category that classifies styles or types of music.

[1557] "Gender" refers to the gender of the song's artist or singer.

[1558] "Decade" refers to the time or era in which a song was released.

[1559] "Social climate" refers to the social situation and trends at the time a song was released.

[1560] "Region" refers to the geographical location or cultural sphere in which the song was released.

[1561] A "database" is a system for systematically organizing and storing specific information.

[1562] "Preprocessing" is the initial stage of processing to convert data into a format suitable for analysis and learning.

[1563] A "generative AI model" is a specific implementation of generative AI: a machine learning model trained to perform a specific task.

[1564] A "prompt" is text that contains instructions or questions that are input to a generative AI model.

[1565] A "response" is the answer or output that a generative AI model generates based on a prompt.

[1566] This invention is a system that creates new music by building a database of popular songs from the past and training a generation AI model based on that data. Specific embodiments of this system are described below.

[1567] Building a database

[1568] The server builds a database of past popular songs, including information on the song's melody, rhythm, lyrics, genre, gender, era, social climate, and region. Specifically, the data is stored using a relational database management system (RDBMS) such as MySQL.

[1569] Data Preprocessing

[1570] The server preprocesses the music data stored in the database. This includes data cleaning, normalization, and feature extraction. For example, it detects missing values ​​and outliers and handles them appropriately. It also converts melodic and rhythmic patterns into numerical data and normalizes the data.

[1571] Training generative AI models

[1572] The server uses the preprocessed data to train a generative AI model. This training process uses machine learning frameworks such as TensorFlow and PyTorch. The preprocessed data is input to the model, and the model is trained every epoch. Once the model is trained, it is saved.

[1573] Feature extraction

[1574] The generative AI model extracts the characteristics of past songs from the learned data. For example, it can identify melodic patterns that were popular in a particular decade or rhythmic characteristics common to a particular genre. The extracted features are stored in a database.

[1575] Entering prompts and generating responses

[1576] The user inputs a prompt into the generative AI model, which then generates a response based on the input prompt. For example, if a user inputs "Please tell me the characteristics of Japanese pop songs from the 1990s," the generative AI model generates an answer based on the data it has learned and displays it to the user via the server.

[1577] Specific examples

[1578] Building a database

[1579] The server stores the following data in the database:

[1580] Song A: Melody = "CDEF", Rhythm = "4 / 4", Lyrics = "Love Song", Genre = "Pop", Gender = "Male", Era = "1990s", Social Climate = "Bubble Economy", Region = "Japan"

[1581] Song B: Melody = "GABC", Rhythm = "3 / 4", Lyrics = "Freedom Song", Genre = "Rock", Gender = "Female", Era = "2000s", Social Mood = "DotNet Bubble", Region = "America"

[1582] Prompt Sentence Examples

[1583] The user inputs the following prompt into the generative AI model:

[1584] "What are the characteristics of Japanese pop songs from the 1990s?"

[1585] "What are the rhythm patterns of American rock songs from the 2000s?"

[1586] This system makes it possible to effectively utilize past music data to generate new music and provide it according to the user's tastes and preferences.

[1587] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1588] Step 1: Building the database

[1589] The server builds a database of songs that were popular in the past. As input, it uses a dataset containing information such as song melody, rhythm, lyrics, genre, gender, era, social climate, and region. Specifically, the server starts a relational database management system (RDBMS) such as MySQL and creates a table to store song information. Next, it imports song data obtained from a CSV file or API into the table. As output, it obtains a database containing song information.

[1590] Step 2: Preprocessing the data

[1591] The server preprocesses the music data stored in the database. It uses the music data in the database as input. Specifically, the server detects missing values ​​and outliers and handles them appropriately. It also converts melodic and rhythmic patterns into numerical data and normalizes the data. The preprocessed data is obtained as output.

[1592] Step 3: Training the generative AI model

[1593] The server uses the preprocessed data to train a generative AI model. The preprocessed data is used as input. Specifically, the server installs a machine learning framework such as TensorFlow or PyTorch and runs a training script. The preprocessed data is input into the model and the model is trained for each epoch. The output is a generative AI model that has completed training.

[1594] Step 4: Feature extraction

[1595] The generative AI model extracts the characteristics of past songs from the trained data. The trained generative AI model and training data are used as input. Specifically, the generative AI model extracts features based on the training data, finding patterns of melodies that were popular during a particular era, or rhythmic characteristics common to a particular genre. The extracted features are obtained as output.

[1596] Step 5: Entering a prompt and generating a response

[1597] The user inputs a prompt sentence into the generative AI model. The prompt sentence entered by the user is used as input. Specifically, the user inputs the prompt sentence from their device, and the server sends the input prompt sentence to the generative AI model. The generative AI model generates a response based on the prompt sentence and returns it to the server. The server displays the generated response on the user's device. The response generated by the generative AI model is obtained as output.

[1598] (Application example 1)

[1599] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1600] Although there is technology to generate new music by utilizing the characteristics of past popular songs, a system for generating and providing music tailored to users' tastes and preferences has not been fully established. Furthermore, there is a lack of a means to efficiently classify generated music and provide it to users. This poses a challenge in quickly and accurately providing users with the music they want.

[1601] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1602] In this invention, the server includes means for generating lyrics and music that are likely to become popular in the future using generative AI based on past popular music, means for classifying the generated lyrics and music by mesh (genre, gender, age, social climate, region, etc.), means for providing the classified lyrics and music as music or through a distribution service, etc., means for creating prompts for music generation based on filters set by the user, and means for providing the generated music to the user. This makes it possible to generate and provide music that matches the user's tastes and preferences.

[1603] "Past popular songs" are songs that were widely popular and listened to by many people during a specific period in the past.

[1604] "Generative AI" refers to systems and algorithms that use artificial intelligence technology to generate new data and content.

[1605] "Lyrics and music" refers to the lyrics and melody of a song, and are the basic elements that make up a musical work.

[1606] "Mesh" is a framework for classifying music based on multiple criteria such as genre, gender, era, social climate, and region.

[1607] A "distribution service" is a service that provides users with content such as music and video via the Internet.

[1608] A "filter" is a function that allows the user to narrow down the characteristics of the music to be generated by setting specific conditions or criteria.

[1609] A "prompt" is input data or text that provides specific conditions or instructions to the generating AI.

[1610] "User" means any individual or entity that uses the System or Services.

[1611] The system for implementing this invention utilizes AI to generate lyrics and music that are likely to become popular in the future based on popular music of the past, and provides them to users according to their tastes and preferences. Specific embodiments of this system are described below.

[1612] System Configuration

[1613] The system mainly consists of the following components:

[1614] 1. Server: Hosts the generative AI model and performs music generation and classification.

[1615] 2. User terminal: A device used by a user to access the system, such as a smartphone or computer.

[1616] 3. Database: Stores data on popular songs from the past.

[1617] Hardware and software used

[1618] Hardware: Servers, smartphones, PCs

[1619] Software: TensorFlow, Keras, database management system (e.g., MySQL)

[1620] Data processing and calculation

[1621] 1. Loading the database: The server loads a database of past popular songs and obtains information such as the song's melody, rhythm, lyrics, genre, gender, era, social climate, and region.

[1622] 2. Loading the generative AI model: The server loads the pre-trained generative AI model, which is built using TensorFlow and Keras.

[1623] 3. Prompt Creation: Based on the input from the user device, the server creates a prompt for song generation, including the filters set by the user (such as era, genre, region, etc.).

[1624] 4. Music Generation: The server inputs prompts into the generative AI model to generate a new song.

[1625] 5. Song Classification: The generated songs are classified based on criteria such as genre, gender, era, social climate, and region.

[1626] 6. Providing songs: The classified songs are provided to the user's device.

[1627] Specific examples

[1628] If a user wants to generate American pop songs from the 1980s, they can set a filter like this:

[1629] Prompt Sentence Examples

[1630] Generate a song with the following characteristics: Era: 1980s, Genre: Pop, Region: USA

[1631] By inputting this prompt into a generative AI model, a new song with characteristics of 1980s American pop songs is generated, tailored to the user's tastes and preferences.

[1632] In this way, users can enjoy new songs that utilize the characteristics of past popular songs.

[1633] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1634] Step 1:

[1635] The server loads a database of past popular songs, which includes information such as the song's melody, rhythm, lyrics, genre, gender, era, social climate, and region. This allows the server to obtain characteristic data for the song.

[1636] Step 2:

[1637] The server loads a pre-trained generative AI model. This model is built using TensorFlow and Keras and learns the characteristics of popular songs from the past. This prepares the model for music generation.

[1638] Step 3:

[1639] The user sets filters (such as era, genre, and region) for music generation using the device, and the user's preferences and tastes are acquired as input data.

[1640] Step 4:

[1641] The server creates prompts based on the filters set by the user, including information such as the era, genre, and region specified by the user, which then generates prompt sentences to be fed into the generative AI model.

[1642] Step 5:

[1643] The server inputs prompts into the generative AI model to generate new music. The generative AI model generates new melodies and lyrics based on the characteristics of popular songs from the past. This results in new music tailored to the user's preferences.

[1644] Step 6:

[1645] The server classifies the generated songs by criteria such as genre, gender, age, social climate, and region, allowing the generated songs to be organized based on meshes.

[1646] Step 7:

[1647] The server provides the classified songs to the user's terminal, and the user can listen to the generated songs through the terminal, thereby allowing the user to enjoy new songs.

[1648] Example 2

[1649] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1650] Conventional music generation systems have made it difficult for users to generate new lyrics and music that reflect the characteristics of a particular musical style or era. Furthermore, there is a lack of means to customize the generated music to suit the user's tastes and preferences. Furthermore, there is also a lack of means to efficiently classify and provide the generated music.

[1651] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1652] In this invention, the server includes means for a user to input a prompt text, means for a terminal to send the prompt text to the server, means for the server to receive the prompt text and pass it to a generative AI model, means for the generative AI model to generate new lyrics and music, means for the server to receive the generated lyrics and music and send it to the terminal, means for the terminal to display the generated lyrics and music to the user, means for generating lyrics and music that are likely to become popular in the future using generative AI based on music that was popular in the past, means for classifying the generated lyrics and music by mesh (genre, gender, age, social climate, region, etc.), and means for providing the classified lyrics and music as music or a distribution service, etc. This allows a user to generate new lyrics and music by specifying a specific musical style and efficiently classify and provide them.

[1653] "User" means an individual or organization that uses the system to input prompts and generate new lyrics and music.

[1654] A "terminal" is a device that allows a user to input a prompt and communicate with a server. Examples include a PC or smartphone.

[1655] The "server" is a computer system that receives prompts, passes them to the generative AI model, and sends the generated lyrics and music to the terminal.

[1656] A "prompt sentence" is text that a user enters to specify characteristics of a particular musical style or era.

[1657] A "generative AI model" is an artificial intelligence model that learns the characteristics of popular songs from the past and generates new lyrics and music based on the results of that learning.

[1658] "Lyrics and Music" refers to the lyrics and melody generated by the generative AI model.

[1659] "Mesh" is a method of classifying generated lyrics and music based on criteria such as genre, gender, age, social climate, and region.

[1660] "Music and distribution services" refers to platforms and services that provide users with created lyrics and music. Examples include music streaming services and download services.

[1661] "Deep learning" is a machine learning technique that generative AI models use to learn the characteristics of popular songs from the past.

[1662] "Customizable means" refers to a function that allows users to adjust the mesh classification results to suit their preferences and tastes.

[1663] MODE FOR CARRYING OUT THE INVENTION

[1664] This invention is a system that allows users to create new lyrics and music that reflect the characteristics of a particular musical style or era. Specific embodiments of this system are described below.

[1665] System configuration

[1666] This system consists of a user, a terminal, a server, and a generative AI model. The user inputs a prompt sentence using the terminal, and the server generates new lyrics and music using the generative AI model.

[1667] Hardware and software used

[1668] Device: a device such as a computer, smartphone, or tablet.

[1669] Server: A high-performance computer system

[1670] Generative AI model: Artificial intelligence model using deep learning

[1671] Data processing and calculation

[1672] When a user inputs a prompt text through their device, the device sends it to the server. The server receives the prompt text and passes it to the generative AI model. The generative AI model learns the characteristics of popular songs from the past and generates new lyrics and music based on the learning results. The generated lyrics and music are sent from the server to the device and displayed to the user.

[1673] Specific examples

[1674] For example, if a user inputs a prompt such as "Please create a new song in the style of a 1980s pop song," the following specific actions will be taken:

[1675] 1. The user types into a text box on the device, "Please create a new song in the style of a 1980s pop song."

[1676] 2. The terminal sends this prompt text to the server as an HTTP request.

[1677] 3. The server receives the prompt and passes the instruction to the generative AI model: "Create a new song in the style of a 1980s pop song."

[1678] 4. The generative AI model generates new songs based on the characteristics of 1980s pop music, such as songs with synthesizer sounds and catchy melody lines.

[1679] 5. The server receives the generated song and sends it to the device as an HTTP response.

[1680] 6. The device displays the generated song to the user, who can then play and enjoy the song.

[1681] In this way, users can specify a specific musical style and use the generative AI model to generate new lyrics and music. The system can also efficiently classify the generated lyrics and music and customize them to suit the user's tastes and preferences.

[1682] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1683] Step 1:

[1684] The user enters a prompt statement.

[1685] Specifically, the user types into a text box on the device, "Please create a new song in the style of a 1980s pop song."

[1686] Input: The prompt text entered by the user

[1687] Output: The prompt displayed on the terminal

[1688] Step 2:

[1689] The terminal sends the prompt to the server.

[1690] As a specific operation, the terminal generates an HTTP request and transmits data including a prompt sentence to the server.

[1691] Input: The prompt text entered by the user

[1692] Output: HTTP request sent to the server

[1693] Step 3:

[1694] The server receives the prompt sentence and passes it to the generative AI model.

[1695] Specifically, the server receives an HTTP request, analyzes the prompt text, and converts it into a data format to be passed to the generative AI model.

[1696] Input: The HTTP request sent to the server

[1697] Output: The prompt text converted into a data format to be passed to the generative AI model.

[1698] Step 4:

[1699] Generative AI models generate new lyrics and music.

[1700] Specifically, the generative AI model generates a new song that reflects the characteristics of 1980s pop music based on a prompt.

[1701] Input: The prompt passed to the generative AI model

[1702] Output: The new lyrics and music generated

[1703] Step 5:

[1704] The server receives the generated lyrics and music and sends them to the device.

[1705] Specifically, the server receives the lyrics and music generated by the generative AI model and sends them to the terminal as an HTTP response.

[1706] Input: New lyrics and music received from a generative AI model

[1707] Output: HTTP response sent to the device

[1708] Step 6:

[1709] The terminal displays the generated lyrics and music to the user.

[1710] Specifically, the device analyzes the HTTP response received from the server and displays the generated lyrics and music to the user, who can then play and enjoy the music.

[1711] Input: HTTP response received from the server

[1712] Output: New lyrics and music displayed to the user

[1713] (Application example 2)

[1714] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1715] While conventional music generation systems could generate new songs based on past songs, they lacked the ability to generate songs based on a generative AI model by inputting specific prompts from the user. Furthermore, the means for streaming, downloading, and sharing the generated songs were limited, making it difficult to meet the diverse needs of users. Furthermore, the ability to customize the generated songs to suit the user's tastes and preferences was also insufficient.

[1716] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1717] In this invention, the server includes: means for generating lyrics and music that are likely to become popular in the future using generative AI based on past popular music; means for classifying the generated lyrics and music by mesh (genre, gender, age, social climate, region, etc.); means for providing the classified lyrics and music through music distribution services, etc.; means for a user to input a prompt and generate new music based on the generative AI model; and means for streaming, downloading, and sharing the generated music. This allows users to input specific prompts and generate music based on the generative AI model, and the generated music can be provided in a variety of ways. It can also be customized to suit the user's preferences and tastes.

[1718] "Generative AI" is artificial intelligence that learns from past data and generates new data based on the results of that learning.

[1719] A "prompt" is an instruction that the user inputs to the generation AI, and is used to specify the content and characteristics of the data to be generated.

[1720] "Mesh" is a method of categorizing data based on different criteria such as genre, gender, age, social climate, and region.

[1721] "Streaming playback" is a method of playing data in real time over the Internet.

[1722] "Download" is a method of saving data to a user's device via the internet.

[1723] "Sharing" refers to the method of sharing the generated data with other users via the Internet.

[1724] "Customization" is the practice of tailoring data and services to a user's tastes and preferences.

[1725] A system for implementing the present invention is configured as follows.

[1726] System Configuration

[1727] 1. Server:

[1728] Generative AI: Artificial intelligence that generates new lyrics and music based on popular songs from the past. It uses deep learning to learn the characteristics of past songs and generates new music based on the results of that learning.

[1729] Classification method: The generated lyrics and music are classified by criteria such as genre, gender, age, social climate, and region.

[1730] Provision method: Classified lyrics and songs are provided as music or through distribution services.

[1731] Generation method: A new song is generated using a generative AI model based on a prompt entered by the user.

[1732] Playback and sharing options: Stream, download, and share the generated songs.

[1733] 2. Terminal:

[1734] User interface: An interface for users to enter prompt text. It works on devices such as smartphones and PCs.

[1735] Playback and sharing features: Features for playing, downloading, and sharing the generated songs.

[1736] Program processing explanation

[1737] The server uses generative AI to learn the characteristics of past popular songs and generates new lyrics and music based on the learning results. When the user inputs a prompt through their device, the server uses a generative AI model based on the prompt to generate a new song. The generated songs are categorized by criteria such as genre, gender, age, social climate, and region, and are customized to suit the user's tastes and preferences. The generated songs can be streamed, downloaded, and shared.

[1738] Hardware and software used

[1739] Hardware: Servers, smartphones, PCs

[1740] Software: Python, OpenAI API, user interface application

[1741] Specific examples

[1742] A user opens the application on their smartphone and types, "Generate lyrics for a 1980s-style pop song with a love theme." The server receives this prompt and uses a generative AI model to generate new lyrics for a 1980s-style pop song. The generated lyrics are customized to the user's tastes and preferences and can be streamed, downloaded, or shared on their smartphone.

[1743] Prompt Sentence Examples

[1744] "Generate lyrics for a 1980s-style pop song about love."

[1745] "Generate lyrics for a 1980s-style pop song about friendship."

[1746] "Generate lyrics for a 1980s-style pop song with an adventure theme."

[1747] In this way, users can input specific prompts to generate new music based on the generative AI model and enjoy it in a variety of ways.

[1748] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1749] Step 1:

[1750] The user enters a prompt sentence at the terminal.

[1751] Input: The user enters a prompt such as "Generate lyrics for a 1980s-style pop song about love."

[1752] What it does: The terminal's user interface receives the prompt and sends it to the server.

[1753] Output: The prompt text is sent to the server.

[1754] Step 2:

[1755] The server receives the prompt and sends a request to the generative AI model.

[1756] Input: The prompt text sent from the terminal.

[1757] How it works: The server parses the prompt and sends a request to the generative AI model to generate a new song.

[1758] Output: A request is sent to the generative AI model.

[1759] Step 3:

[1760] A generative AI model generates new music based on a prompt.

[1761] Input: The prompt text sent by the server.

[1762] How it works: The generative AI model uses deep learning based on past music data to generate new music that matches the prompt.

[1763] Output: The generated music data.

[1764] Step 4:

[1765] The server receives the generated music data and classifies it into meshes.

[1766] Input: Music data sent from the generative AI model.

[1767] How it works: The server classifies music data by criteria such as genre, gender, age, social climate, and region.

[1768] Output: Classified song data.

[1769] Step 5:

[1770] The server customizes the classified music data to suit the user's tastes and preferences.

[1771] Input: Categorized song data and information about user tastes and preferences.

[1772] How it works: The server customizes music data based on the user's tastes and preferences.

[1773] Output: Customized song data.

[1774] Step 6:

[1775] The server transmits the customized music data to the terminal.

[1776] Input: Customized song data.

[1777] How it works: The server sends customized music data to the device.

[1778] Output: Song data sent to the device.

[1779] Step 7:

[1780] The terminal provides the received music data to the user.

[1781] Input: Song data sent from the server.

[1782] What it does: Provides an interface for devices to stream, download, and share music data.

[1783] Output: Your song is ready for users to play, download, and share.

[1784] In this way, users can input specific prompts to generate new music based on the generative AI model and enjoy it in a variety of ways.

[1785] Example 3

[1786] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1787] Conventional music generation systems have had problems with recommending music that meets a user's specific tastes and preferences, and they are unable to adequately categorize or customize the music they generate. Furthermore, they lack the technology to learn the characteristics of past popular music and generate new music based on that.

[1788] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1789] In this invention, the server includes a means for a user to input a prompt sentence, a means for generating lyrics and music using a generative AI model, a means for classifying the generated lyrics and music by genre, gender, age, social climate, region, etc., a means for recommending the classified lyrics and music to the user, and a means for the user to customize the recommended music. This enables music to be recommended according to the user's specific tastes and preferences, and makes it easy to classify and customize the generated music. It can also learn the characteristics of past popular music and generate new music based on that.

[1790] A "prompt sentence" is specific text that a user inputs to give instructions to a generative AI model for generating music.

[1791] A "generative AI model" is an artificial intelligence model that uses deep learning to learn the characteristics of popular songs from the past and generates new lyrics and music based on the results of that learning.

[1792] "Genre" is a classification item that indicates the style or type of music, and includes, for example, rock, pop, jazz, etc.

[1793] "Gender" is a classification based on the target audience of the song or the gender of the artist.

[1794] "Era and Social Condition" is a classification category based on the historical background and social situation reflected in the music.

[1795] "Region" is a classification that indicates that a song is associated with a particular geographic area or culture.

[1796] "Recommendation" refers to the act of presenting the most suitable music to the user based on the user's past preference data and classification results.

[1797] "Customization" refers to the act of a user adjusting the classification results of recommended songs to suit their own tastes and preferences.

[1798] MODE FOR CARRYING OUT THE INVENTION

[1799] This invention is a system in which a user inputs a prompt sentence, a generative AI model is used to generate new lyrics and music, and the system classifies them by genre, gender, age, social climate, region, etc., recommends them to the user, and the user can further customize the recommended music.

[1800] 1. Program Generation

[1801] The server generates lyrics and music using a generative AI model. This generative AI model uses a general artificial intelligence model that uses deep learning technology, for example. Specifically, Python libraries such as TensorFlow and PyTorch are often used.

[1802] 2. Data Classification

[1803] The server classifies the generated lyrics and music by genre, gender, age, social climate, region, etc. This classification is performed using natural language processing technology and machine learning algorithms. Specifically, Python's NLTK and Scikit-learn are used.

[1804] 3. Category Recommendation

[1805] The server then recommends the classified songs to the user. This recommendation system uses collaborative filtering and content-based filtering. Specifically, Apache Mahout and TensorFlow are sometimes used. The server recommends the most suitable songs based on the user's past preference data.

[1806] 4. Customization features

[1807] Users can customize Mesh's classification results to suit their preferences and tastes. For example, a user can set specific preferences, such as preferring the "1980s" category but not the "Pop" category. This customization is provided through a user interface using front-end frameworks such as React and Vue.js.

[1808] Specific examples

[1809] User A likes 1980s rock music but not pop music. The server uses a generative AI model to generate a new song and classify it into the categories "1980s" and "rock." User A is recommended this song based on his preferences.

[1810] Prompt Sentence Examples

[1811] "Generate a 1980s-style rock song. The lyrics should have themes of youth and freedom."

[1812] In this way, a system is realized that allows users to efficiently search for music that suits their preferences. The flow of the specification process in the third embodiment will be described with reference to FIG.

[1813] Step 1:

[1814] The user enters a prompt statement.

[1815] The user opens a web browser, accesses the system interface, enters "Generate a 1980s-style rock song. The lyrics should include themes of youth and freedom" in the input field, and clicks the submit button.

[1816] Input: The prompt text entered by the user

[1817] Output: The prompt is sent to the server.

[1818] Step 2:

[1819] The server generates lyrics and music using a generative AI model.

[1820] The server sends the prompt received from the user to the generative AI model. The generative AI model generates new lyrics and music based on the prompt and returns the results to the server. The generative AI model uses a general artificial intelligence model that uses deep learning technology. Specifically, it uses the Python libraries TensorFlow and PyTorch.

[1821] Input: prompt statement

[1822] Output: Generated lyrics and music

[1823] Step 3:

[1824] The server classifies the generated lyrics and music by genre, gender, age, social climate, region, etc.

[1825] The server receives the generated lyrics and music and classifies them into categories such as "1980s" or "rock" using Python's NLTK and Scikit-learn. The classification results are stored in a database.

[1826] Input: Generated lyrics and music

[1827] Output: Classified lyrics and songs

[1828] Step 4:

[1829] The server recommends songs to users based on the classification results.

[1830] The server refers to the user's past preference data and recommends the most suitable songs. This recommendation system uses collaborative filtering and content-based filtering. Specifically, Apache Mahout and TensorFlow are used. If a user has previously liked "1980s" and "Rock," new songs in the same category will be recommended.

[1831] Input: Classified lyrics and songs, user preference data

[1832] Output: Recommended songs

[1833] Step 5:

[1834] Users customize recommended songs.

[1835] Users can review the list of recommended songs and filter out categories that don't suit their tastes. For example, they can leave the "1980s" category and filter out the "Pop" category. This customization feature is provided through a user interface using front-end frameworks such as React and Vue.js.

[1836] Input: Recommended songs, user customization settings

[1837] Output: Customized song list

[1838] (Application example 3)

[1839] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1840] Conventional music distribution systems make it difficult for users to efficiently find music that matches their preferences. Furthermore, music recommendations often fail to accommodate users' specific tastes, making it difficult to discover new music. Furthermore, there is a lack of technology to generate new music based on past trends, making it impossible to meet the diverse needs of users.

[1841] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1842] In this invention, the server includes means for generating lyrics and music that are likely to become popular in the future using generative AI based on past popular music, means for classifying the generated lyrics and music by mesh (genre, gender, age, social climate, region, etc.), means for providing the classified lyrics and music as music or through a distribution service, etc., means for recommending music based on user preferences, means for specifying user preferences by genre, age, and region, and means for filtering and recommending music that matches the user's preferences. This enables users to efficiently find music that suits their preferences and makes it easier to discover new music.

[1843] "Past popular songs" are songs that were popular with many people and widely listened to during a certain period in the past.

[1844] "Generative AI" is a system that uses artificial intelligence technology to generate new data and content.

[1845] "Lyrics and music" refers to the lyrics and melody of a musical work.

[1846] "Mesh" is a method of classifying data based on multiple categories such as genre, gender, age, social climate, and region.

[1847] A "genre" is a category that classifies musical styles and forms.

[1848] "Gender" refers to the gender of the song's target listeners.

[1849] "Era and social conditions" refers to a specific era and the social situation of that era.

[1850] "Region" refers to a specific geographical location or cultural context.

[1851] "Music and distribution services" refers to platforms and services that provide musical works to users.

[1852] "User preferences" refers to musical characteristics such as genres, eras, and regions that individual users particularly prefer.

[1853] "Filtering" is the process of sorting data based on specific criteria.

[1854] "Recommendation" is the act of suggesting appropriate music based on the user's preferences.

[1855] "Deep learning" is a machine learning technique that uses multi-layer neural networks to learn the characteristics of data.

[1856] The following system configuration will be described as an embodiment of the present invention.

[1857] The server has a means to generate lyrics and music that are likely to become popular in the future using generative AI based on past popular songs. This generative AI uses deep learning to learn the characteristics of past popular songs and generates new lyrics and music based on the learning results. The generated lyrics and music are categorized by meshes such as genre, gender, age, social climate, and region.

[1858] The classified lyrics and music are then provided as music or through music distribution services. Users access the distribution service using a device such as a smartphone and are recommended songs based on their preferences. Users can specify their preferences by genre, age, and region, which allows the system to filter and recommend songs that match their preferences.

[1859] For example, if a user likes "1980s pop songs," the server uses generative AI to learn the characteristics of 1980s pop songs and generate new songs based on those characteristics. The generated songs are then categorized into "1980s" and "Pop" and recommended based on the user's preferences.

[1860] The following hardware and software are used to realize this system. The server has high-performance computing resources to run the generative AI model. Specifically, a server equipped with a GPU is suitable. A deep learning framework (e.g., TensorFlow or PyTorch) is used as software. An application for accessing the music distribution service is installed on the user's device. This application provides an interface for inputting the user's preferences and displays songs recommended by the server.

[1861] Examples of prompts to input to a generative AI model include:

[1862] Write a Python program that recommends songs based on user preferences. The song database contains information about title, genre, era, and region. User preferences are specified by genre, era, and region. Write a function that filters and recommends songs that match the user's preferences.

[1863] In this way, users can efficiently search for music that suits their tastes and easily discover new music.

[1864] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1865] Step 1:

[1866] The server acquires a database of past popular songs.

[1867] Input: Database of popular songs from the past

[1868] Output: Get the music database

[1869] Specific operation: The server accesses a database of past popular songs and obtains information such as title, genre, era, and region.

[1870] Step 2:

[1871] The server uses a generative AI model to learn the characteristics of popular songs from the past.

[1872] Input: Song database

[1873] Output: Trained generative AI model

[1874] How it works: The server uses a deep learning framework (e.g., TensorFlow or PyTorch) to learn the features of the music database, allowing the generative AI model to understand the characteristics of past popular songs.

[1875] Step 3:

[1876] The server generates new lyrics and music using a trained generative AI model.

[1877] Input: trained generative AI model

[1878] Output: Generated lyrics and music

[1879] How it works: The server inputs prompts into the trained generative AI model to generate new lyrics and music. The generated lyrics and music reflect the characteristics of popular songs from the past.

[1880] Step 4:

[1881] The server classifies the generated lyrics and music into meshes (genre, gender, age, social climate, region, etc.).

[1882] Input: Generated lyrics and music

[1883] Output: Classified lyrics and songs

[1884] Specific operation: The server classifies the generated lyrics and music based on categories such as genre, gender, era, social climate, and region, making it clear that the music belongs to a specific category.

[1885] Step 5:

[1886] The server provides the classified lyrics and music as music or distribution services.

[1887] Input: Classified lyrics and songs

[1888] Output: Songs provided to the distribution service

[1889] Specific operation: The server uploads the classified lyrics and music to a music distribution service, making them accessible to users.

[1890] Step 6:

[1891] A user uses a terminal to access a distribution service and input their preferences.

[1892] Input: User preferences (genre, era, region)

[1893] Output: User preference settings

[1894] Specific operation: The user accesses the distribution service using a device such as a smartphone and enters preferences such as genre, era, and region.

[1895] Step 7:

[1896] The server filters and recommends songs based on the user's preferences.

[1897] Input: User preferences, categorized lyrics and songs

[1898] Output: Recommended songs

[1899] Specific operation: The server filters appropriate songs from the classified lyrics and music based on the user's preferences and recommends them to the user, allowing the user to efficiently find songs that suit their tastes.

[1900] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1901] "Example 1"

[1902] One embodiment of the present invention is a system that combines an emotion engine that recognizes user emotions. This system uses generative AI to generate lyrics and music that are likely to become popular in the future based on past popular songs, classifies the generated lyrics and music by mesh (genre, gender, age, social climate, region, etc.), and provides the classified lyrics and music as music or through a distribution service.

[1903] "Example 2"

[1904] The emotion engine recognizes the user's emotions and adjusts the generation and classification of lyrics and music based on those emotions. For example, if the user expresses joy, the emotion engine communicates that information to the generation AI, which then generates lyrics and music with a joyful theme. Conversely, if the user expresses sadness, the emotion engine communicates that information to the generation AI, which then generates lyrics and music with a sadness theme.

[1905] "Example 3"

[1906] Furthermore, the emotion engine recognizes the user's emotions and adjusts the delivery of lyrics and music based on those emotions. For example, if the user expresses a happy emotion, the emotion engine communicates that information to the delivery service, and the delivery service provides the user with lyrics and music with a happy theme. Conversely, if the user expresses a sad emotion, the emotion engine communicates that information to the delivery service, and the delivery service provides the user with lyrics and music with a sad theme.

[1907] The processing flow of each embodiment will be described below.

[1908] "Example 1"

[1909] Step 1: Recognize the user's emotions with the emotion engine.

[1910] Step 2: Based on the recognized emotions, the generative AI generates lyrics and music.

[1911] Step 3: Classify the generated lyrics and music into meshes (genre, gender, era, social climate, region, etc.).

[1912] Step 4: Provide the classified lyrics and music as music or through a distribution service.

[1913] "Example 2"

[1914] Step 1: Recognize the user's emotions with the emotion engine.

[1915] Step 2: Based on the recognized emotions, the generative AI generates lyrics and music.

[1916] Step 3: Classify the generated lyrics and music into meshes (genre, gender, era, social climate, region, etc.).

[1917] Step 4: Provide the classified lyrics and music as music or through a distribution service.

[1918] "Example 3"

[1919] Step 1: Recognize the user's emotions with the emotion engine.

[1920] Step 2: Based on the recognized emotions, the generative AI generates lyrics and music.

[1921] Step 3: Classify the generated lyrics and music into meshes (genre, gender, era, social climate, region, etc.).

[1922] Step 4: Provide the classified lyrics and music as music or through a distribution service.

[1923] Example 1

[1924] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1925] Conventional music generation systems have difficulty generating new music that reflects the characteristics of past popular songs, and are unable to provide music that matches the user's emotions. Furthermore, there is a lack of means to customize the generated music to suit the user's tastes and preferences. This makes it difficult to increase user satisfaction.

[1926] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1927] In this invention, the server includes means for generating lyrics and music that are likely to become popular in the future using a generative AI based on music that has been popular in the past, means for classifying the generated lyrics and music by genre, gender, age, social climate, region, etc., means for providing the classified lyrics and music through a music distribution service, etc., means for acquiring emotion data using an emotion engine that recognizes user emotions, and means for generating new music using a generative AI model based on the acquired emotion data. This makes it possible to generate and provide music that matches the user's emotions, thereby increasing user satisfaction.

[1928] "Generative AI" is an artificial intelligence technology that learns from past data and generates new data based on the results of that learning.

[1929] An "emotion engine" is a technology that analyzes emotions from a user's facial expressions, voice, etc. and acquires that data.

[1930] A "music distribution service" is a service that provides music to users via the Internet.

[1931] "Genre" is a category for classifying the style or type of music.

[1932] "Gender" is information indicating the gender of the user targeted by the song.

[1933] "Decade" is information that indicates the time when the song was popular and the age group of the target users.

[1934] "Social trends" refers to information that indicates the social background and cultural situation at the time a song became popular.

[1935] "Region" is information indicating the geographical location where a song became popular.

[1936] "Emotion data" is information about a user's emotions analyzed by the emotion engine.

[1937] "Machine learning" is a technology in which a computer learns patterns from data and makes predictions and judgments based on the results of that learning.

[1938] "Mesh" is a method of classifying music into multiple categories such as genre, gender, era, social climate, and region.

[1939] MODE FOR CARRYING OUT THE INVENTION

[1940] This invention is a system that builds a database of past popular songs and uses generative AI to generate new songs based on that database. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it is possible to provide songs that correspond to the user's emotions.

[1941] Building a database

[1942] The server builds a database of songs that were popular in the past. This database includes information such as the song's melody, rhythm, lyrics, genre, gender, era, social context, and region. Specifically, the data is stored using a database management system such as MySQL. For example, the server inserts data by executing the SQL query "INSERT INTO songs (melody, rhythm, lyrics, genre, gender, era, social_context, region) VALUES (...)".

[1943] Generative AI training

[1944] The server uses the constructed database to train a generative AI model. This generative AI model uses, for example, OpenAI's GPT-4. During the training process, the features of past songs are extracted and patterns are recognized. Specifically, the server calls a function called "train_model(database)" and supplies the data in the database to the model.

[1945] Emotion engine emotion recognition

[1946] The device is equipped with an emotion engine to recognize the user's emotions. This emotion engine uses, for example, Microsoft's Azure Cognitive Services. It analyzes emotions from the user's facial expressions and voice and sends the data to the server. Specifically, the device calls the function "emotion_data = recognize_emotion(user_input)" and sends the obtained emotion data to the server.

[1947] Music generation

[1948] The server uses a generative AI model to generate lyrics and music for a new song based on the user's emotional data sent from the emotion engine. The generated song reflects the characteristics of past popular songs. Specifically, the server calls the function "new_song = generate_song(emotion_data)" to generate a new song.

[1949] Song classification

[1950] The server classifies the generated songs into a mesh of genre, gender, era, social climate, region, etc. This classification is performed using a search engine such as Elasticsearch. Specifically, the server calls a function called "classify_song(new_song)" to classify the songs into the appropriate category.

[1951] Music provided by

[1952] Users can receive songs generated through their devices through music distribution services, such as Spotify and Apple Music. For example, users can play songs using the "stream_song(new_song)" function.

[1953] Examples and prompts

[1954] Specific examples

[1955] If the emotion engine determines that the user is in a sad mood, the device sends the data "emotion_data = {'emotion': 'sad'}" to the server. The server receives this data and calls the function "new_song = generate_song({'emotion': 'sad'})" to generate a new sad song. The generated song is classified as a "breakup song" and distributed on Spotify through the function "stream_song(new_song)".

[1956] Prompt Sentence Examples

[1957] "If the emotion engine recognizes that the user is in a sad mood, generate a new sad song using a generative AI model that has learned the characteristics of past sad songs."

[1958] In this way, the system can generate music that responds to the user's emotions and provide it on the appropriate platform.

[1959] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1960] Program processing flow

[1961] Step 1: Building the database

[1962] The server builds a database of songs that were popular in the past. As input, it receives information such as the song's melody, rhythm, lyrics, genre, gender, era, social context, and region. Specifically, the server uses a database management system such as MySQL to insert data by executing the SQL query "INSERT INTO songs (melody, rhythm, lyrics, genre, gender, era, social_context, region) VALUES (...)". As output, the song data is saved in the database.

[1963] Step 2: Training the generative AI

[1964] The server uses the constructed database to train the generative AI model. It receives the music data in the database as input. Specifically, the server calls a function called "train_model(database)" and supplies the data in the database to the model. The output is a generative AI model that has learned the characteristics of past music.

[1965] Step 3: Emotion Recognition in the Emotion Engine

[1966] The device is equipped with an emotion engine to recognize the user's emotions. It receives the user's facial expressions and voice data as input. Specifically, the device calls the function "emotion_data = recognize_emotion(user_input)" and sends the obtained emotion data to the server. The user's emotion data is obtained as output.

[1967] Step 4: Create a song

[1968] The server uses a generative AI model to generate lyrics and music for a new song based on the user's emotional data sent from the emotion engine. It receives the emotional data and generative AI model as input. Specifically, the server calls the function "new_song = generate_song(emotion_data)" to generate a new song. The generated song is obtained as output.

[1969] Step 5: Classify your songs

[1970] The server classifies the generated songs into meshes such as genre, gender, era, social climate, and region. It receives the generated song data as input. Specifically, the server calls a function called "classify_song(new_song)" to classify the song into the appropriate category. The output is the classified song data.

[1971] Step 6: Submit your music

[1972] The user can receive the generated music through a music distribution service or other means via their device. The input is the classified music data. Specifically, the user plays the music using the function "stream_song(new_song)". The output is the user listening to the music.

[1973] In this way, the system can generate music that responds to the user's emotions and provide it on the appropriate platform.

[1974] (Application example 1)

[1975] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1976] Conventional music generation systems were able to generate new music based on popular songs from the past, but it was difficult to generate and distribute music that responded to the user's emotions in real time. Furthermore, there was a lack of means to customize the generated music to suit the user's tastes and preferences. This made it impossible to provide a personalized music experience that responded to the user's emotions and preferences.

[1977] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1978] In this invention, the server includes: means for generating lyrics and music that are likely to become popular in the future using generative AI based on music that has been popular in the past; means for classifying the generated lyrics and music by mesh (genre, gender, age, social climate, region, etc.); means for providing the classified lyrics and music as music or a distribution service, etc.; means for generating music that corresponds to the user's emotions using an emotion engine that recognizes the user's emotions; and means for streaming the generated music in real time. This makes it possible to generate and distribute music that corresponds to the user's emotions in real time, and further customize it to suit the user's tastes and preferences.

[1979] "Past popular songs" are musical works that were popular with many people and widely listened to during a certain period in the past.

[1980] "Generative AI" is a system that uses artificial intelligence technology to automatically generate new music and lyrics.

[1981] "Mesh" is a method of classifying data based on different categories such as genre, gender, age, social climate, and region.

[1982] The "emotion engine" is a system that recognizes emotions from the user's facial expressions, voice, etc. and analyzes those emotions.

[1983] "Real-time streaming" is a technology that instantly distributes created music to users via the Internet.

[1984] "Deep learning" is a machine learning technique that uses multi-layered neural networks to learn features from large amounts of data and make predictions and generate results.

[1985] "Customizable" refers to the ability to change the system settings and output according to the user's preferences and tastes.

[1986] A system for implementing the present invention has the following configuration.

[1987] System configuration

[1988] 1. Emotion Recognition Module

[1989] The server uses the smartphone camera and head-mounted display sensors to recognize emotions from the user's facial expressions and voice, using software such as OpenCV and Keras.

[1990] 2. Music Generation Module

[1991] The server uses a generative AI to generate new music based on a database of past popular songs. The generative AI uses deep learning to learn the characteristics of past songs and generate music that matches the user's emotions. The music is generated using a proprietary music generation AI (Music Generator).

[1992] 3. Song Classification Module

[1993] The server categorizes the generated songs based on categories such as genre, gender, age, social climate, and region, and the categorized songs can be customized to suit the user's tastes and preferences.

[1994] 4. Music distribution module

[1995] The server then streams the generated music in real time, using technology that provides music to users over the Internet.

[1996] Processing flow

[1997] 1. Emotion recognition

[1998] When a user uses a smartphone camera or a head-mounted display, the server captures the user's facial expressions and voice, detects the face using OpenCV, and inputs the face into a Keras emotion recognition model to predict the user's emotions.

[1999] 2. Music Generation

[2000] The server generates new music based on the user's emotions using a generative AI that has learned from a database of popular songs from the past. The generated music is optimized to match the user's emotions.

[2001] 3. Song Classification

[2002] The server categorizes the generated songs based on genre, gender, age, social climate, region, etc. The categorized songs are then customized to suit the user's tastes and preferences.

[2003] 4. Music distribution

[2004] The server then streams the generated music in real time, allowing users to instantly listen to it via their smartphones or head-mounted displays.

[2005] Specific examples

[2006] For example, if a user has a sad expression, the server uses the emotion recognition module to detect "sadness." The music generation module generates a new song with a melody and lyrics that correspond to "sadness," and the song classification module classifies it appropriately. Finally, the music distribution module streams the generated song to the user in real time.

[2007] Prompt Sentence Examples

[2008] "You will develop an application that recognizes the user's emotions and generates new music based on those emotions. The emotion recognition is performed using a smartphone camera, and the generated music is based on a database of past popular songs."

[2009] In this way, a system can be realized that generates and distributes music in real time according to the user's emotions.

[2010] The flow of the specific processing in the application example 1 will be described with refere...

Claims

1. A means for generating songs that are likely to become popular in the future using AI based on songs that have been popular in the past; A means for classifying the generated music pieces into meshes including genre, gender, age, social situation, and region; A means for users to specify their preferences by genre, age, and region, means for filtering the classified songs based on a filter set by the user; means for recommending songs from the filtered songs based on the user's tastes and preferences; and means for providing the recommended songs to the user via a distribution service.

2. A means for generating songs that are likely to become popular in the future using AI based on songs that have been popular in the past; A means for classifying the generated music pieces into meshes including genre, gender, age, social situation, and region; means for customizing the classified songs to suit a user's tastes and preferences; means for recognizing a user's emotion using an emotion engine and tailoring the customized music presentation based on the emotion; and means for providing the adjusted music piece through a distribution service.

3. A means for generating songs that are likely to become popular in the future using AI based on songs that have been popular in the past; A means for classifying the generated music pieces into meshes including genre, gender, age, social situation, and region; means for customizing said mesh classification results to suit a user's preferences and tastes; means for recognizing a user's emotion using an emotion engine and adjusting the customized classification result based on the emotion; a means for recommending music pieces based on the adjusted classification result and the user's tastes and preferences; and means for providing the recommended songs to the user via a distribution service.

Images