system

Abstract

We provide the system. [Solution] An input means that allows the user to input parameters such as the type of song, music genre, and theme, A generation means that generates music, lyrics, sheet music, vocals, and accompaniment based on parameters input via the aforementioned input means, A return means that returns the music data generated by the generation means to the user, A storage means for storing the aforementioned music data, A management means for managing copyrights to music data stored in the aforementioned storage means, A means of buying and selling the aforementioned copyrights between users, A revenue-generating means that uses the profits generated by the aforementioned trading means as revenue for the platform, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of the chatbot's character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In recent years, there has been a demand for an environment where individuals can easily create music. However, conventional systems require technical knowledge and expensive software, and there is a lack of a platform that can consistently perform music generation, editing, storage, and management. In addition, there is no system that can easily manage the copyright of the generated music and conduct transactions between users, making it difficult for individuals to commercially utilize and monetize the music they generate. Therefore, there is an urgent need to provide a system that allows anyone to easily create music, appropriately manage its copyright, and conduct transactions.

Means for Solving the Problems

[0005] The present invention includes an input means that allows users to input parameters such as song type, music genre, and theme, and a generation means that generates a song, lyrics, sheet music, vocals, and accompaniment based on the parameters entered via this input means. Furthermore, it includes a return means that returns the generated song data to the user, a storage means for storing the song data, and a management means for managing the copyright of the stored song data. In addition, it provides a system that includes a trading means for buying and selling copyrights between users, and a revenue means that makes the profits generated from these transactions into platform revenue. The present invention makes it possible for individuals to easily generate songs, manage and buy and sell their copyrights, and monetize them.

[0006] An "input means" is a device or interface that allows a user to input parameters such as the type of music, music genre, and theme.

[0007] "Generation means" refers to a device or software that automatically generates music, lyrics, sheet music, vocals, accompaniment, etc., based on parameters input via input means.

[0008] "Return means" refers to a device or software that returns the music data generated by the generation means to the user.

[0009] "Storage means" refers to a device or software for permanently or temporarily storing generated musical data.

[0010] "Management means" refers to a device or software that manages copyright and access rights to music data stored in storage means.

[0011] "Means of sale" refers to devices or software used for buying and selling copyrights between users.

[0012] A "revenue-generating device" refers to a device or software that generates revenue for the platform from the profits generated by the buying and selling of copyrights between users.

[0013] "Editing means" refers to a device or software that allows the user to arbitrarily edit the generated music data. [Brief explanation of the drawing]

[0014] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14]A sequence diagram showing the processing flow of a data processing system in Application Example 2 when a sentiment engine is combined.

Embodiments for Carrying Out the Invention

[0015] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a labeled processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), etc.

[0018] In the following embodiments, a labeled RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0019] In the following embodiments, a labeled storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0020] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0022] [First Embodiment]

[0023] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0024] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0027] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0030] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0034] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0035] This invention is a system for users to easily generate music and manage and buy / sell its copyrights. This system includes an input means for users to input parameters such as the type of music, music genre, and theme.

[0036] Steps for users to generate music

[0037] Users access the music generation screen from their own devices (PCs or smartphones). Here, they input parameters such as the type of song, music genre, and theme. For example, specific inputs such as "Pop" and "Summer Memories" are possible.

[0038] Server-based music generation

[0039] The server receives a request from the user and instructs the AI ​​model to generate music. Based on the input parameters, the AI ​​model automatically generates the music, lyrics, sheet music, vocals, and accompaniment. The server then receives the generated music data.

[0040] Return and display of music data

[0041] The server sends the generated music data back to the user's device. The user's device displays the received music in a playable state. The user can listen to this music and edit it as needed.

[0042] Saving music data

[0043] If the user is satisfied with the generated music data, they click the save button to save the song. The server receives the save request and saves the music data to the database. This allows the user to access the song again at any time.

[0044] Copyright management and trading

[0045] The server provides tools for managing copyrights to stored music data on a per-user basis. Users enter pricing and sales conditions on the music sales page and send sales requests to the server. The server stores this information and makes it public in the copyright management system.

[0046] Transactions between users

[0047] Other users can view the published song information and select the songs they wish to purchase. Once a purchase request is sent to the server, the server processes the transaction and grants the buyer the right to use the song. Furthermore, the platform deducts its revenue from the transaction amount and distributes the remainder to the seller. This facilitates smooth transactions between users and makes it possible for individuals to commercially use and monetize their created music.

[0048] Specific usage examples

[0049] For example, suppose User A wants to generate a jazz song with the theme "winter love." User A accesses the song generation screen and enters "jazz" and "winter love." The server sends this information to the AI ​​model, which generates an appropriate song. This song is displayed on User A's device and is playable. User A edits and saves the song, then sets a sales price and creates a copyright listing.

[0050] If another user B likes this song and wants to buy it, they click the purchase button on the sales page. The server processes the transaction, grants user B the right to use the song, and distributes the remaining amount to user A after deducting the platform's revenue from the transaction amount.

[0051] As described above, this system allows users to easily create music and manage and buy / sell its copyrights. Furthermore, since the system can generate revenue as a platform, it contributes to the creation of a new music market.

[0052] The following describes the processing flow.

[0053] User registration and authentication

[0054] Step 1:

[0055] The user enters their email address, password, and profile information into the registration form on their device and clicks the registration button.

[0056] Step 2:

[0057] The server receives registration information sent by the user and stores it in the database.

[0058] Step 3:

[0059] The server sends an authentication email to the user. This email contains an authentication link.

[0060] Step 4:

[0061] The user opens the verification email and clicks the verification link.

[0062] Step 5:

[0063] The server receives the request from the authentication link and updates the user's authentication status.

[0064] Song generation request

[0065] Step 1:

[0066] The user enters parameters such as the song type, genre, and theme on the device's music generation screen and clicks the generate button.

[0067] Step 2:

[0068] The terminal sends the entered parameters to the server.

[0069] Step 3:

[0070] The server sends the request data received from the user to the AI ​​model.

[0071] Step 4:

[0072] The AI ​​model generates music, lyrics, sheet music, vocals, and accompaniment based on the parameters.

[0073] Step 5:

[0074] The server receives the generated results from the AI ​​model and sends them back to the user's device.

[0075] Step 6:

[0076] The device displays the generated music data to the user.

[0077] Editing and saving music

[0078] Step 1:

[0079] The user edits the generated music data on the device's editing screen. For example, they can change the melody, revise the lyrics, and adjust the accompaniment.

[0080] Step 2:

[0081] The user clicks the save button to save the edited music data.

[0082] Step 3:

[0083] The server receives a save request from the user and saves the edited music data to the database.

[0084] Copyright establishment and sales

[0085] Step 1:

[0086] Users access the music sales screen from their devices and set the price and sales conditions for the songs.

[0087] Step 2:

[0088] The user submits a sales request to the server.

[0089] Step 3:

[0090] The server stores sales information in a database and publishes copyright listings.

[0091] Transactions between users

[0092] Step 1:

[0093] Other users select a song from the sales page and click the purchase button.

[0094] Step 2:

[0095] The device sends a purchase request to the server.

[0096] Step 3:

[0097] The server processes the purchase request and grants the purchaser the right to use the music.

[0098] Step 4:

[0099] The server deducts the platform's revenue from the transaction amount and distributes the remainder to the music sellers.

[0100] The above outlines the specific processing steps in the patent system.

[0101] (Example 1)

[0102] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0103] While systems exist today for easily generating music and managing and trading its copyrights, they do not fully meet user needs for smooth music generation, management, and trading procedures. Furthermore, improving the efficiency of revenue sharing associated with the editing and trading of generated music remains a challenge. This invention aims to solve these problems and provide an efficient music generation, management, and trading system that is easily accessible to users.

[0104] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0105] In this invention, the server includes an input means on which a user can input parameters such as the type of music, music genre, and theme; a generation means that generates music, lyrics, sheet music, vocals, and accompaniment based on the parameters input via the input means; a return means that sends the music data generated by the generation means back to the user; a storage means that stores the music data; a management means that manages the copyright to the music data stored in the storage means; a trading means that buys and sells the copyright between users; a revenue means that makes the profit generated by the trading means into platform revenue; a transaction means that checks the published music information of other users and processes purchase requests when they wish to purchase it; and a distribution means that deducts a commission from the transaction amount and distributes the remaining amount to the seller. This makes it possible for a user to smoothly perform a series of operations, from inputting parameters to generate music, and managing, editing, and buying and selling that music.

[0106] An "input method" is an interface for users to input parameters such as the type of music, music genre, and theme.

[0107] The "generation means" is a system for generating music, lyrics, sheet music, vocals, and accompaniment based on parameters input via the input means.

[0108] The "return means" is a mechanism for sending the music data generated by the generation means to the user.

[0109] A "storage method" refers to a system for saving generated music data to a database or storage device.

[0110] "Management means" refers to a tool for managing the copyright of music data stored in the aforementioned storage means.

[0111] A "trading method" refers to a system for users to buy and sell copyrighted music that has been preserved.

[0112] "Revenue-generating means" refers to a mechanism for generating revenue for the platform from the profits generated by the aforementioned buying and selling means.

[0113] A "transaction method" is a system for other users to view publicly available song information and process purchase requests if they wish to buy it.

[0114] A "distribution mechanism" is a system for distributing the remaining amount to the seller after deducting fees from the transaction amount.

[0115] "Editing means" refers to an interface that allows the user to edit the aforementioned music data.

[0116] "Authentication means" refers to a system used for user registration and authentication.

[0117] This invention relates to a system for users to easily generate music and manage and buy / sell its copyrights. The system includes an input means for users to input parameters such as the type of music, music genre, and theme; a generation means for generating music, lyrics, sheet music, vocals, and accompaniment based on the input parameters; a return means for sending the generated music data back to the user; a storage means for storing the music data; a management means for managing the copyrights to the stored music data; a buying and selling means for users to buy and sell copyrights; a revenue means for making the profits generated by the buying and selling means into platform revenue; a transaction means for other users to check the published music information and process purchase requests if they wish to purchase it; a distribution means for deducting a commission from the transaction amount and distributing the remaining amount to the seller; and an editing means for editing music data and an authentication means for user registration and authentication.

[0118] Hardware and software configuration

[0119] This system consists of user terminals (PCs, smartphones) and a server based on a high-performance computer. Users can access the server from their terminals via an internet browser or a dedicated application.

[0120] For the AI ​​model that functions as a generation tool, for example, OpenAI's GPT-3® or a neural network model specifically for music generation can be used. For storage and management, a cloud-based database system (e.g., Amazon Web Services, Google Cloud Platform) can be applied.

[0121] System operation

[0122] 1. When a user submits a song generation request

[0123] The user accesses the music generation screen from their device, enters parameters such as "Genre: Pop" and "Theme: Summer Memories," and presses the generate button. The following is an example of the prompt text.

[0124] Genre: Pop

[0125] Theme: Summer Memories

[0126] 2. When the server receives a music generation request and sends it to the AI ​​model

[0127] The server receives the user's request, analyzes the input parameters, and sends them to the AI ​​model. Based on these parameters, the AI ​​model generates music, lyrics, sheet music, vocals, and accompaniment.

[0128] 3. When the generated music data is returned to the user and saved.

[0129] The AI ​​model generates music data, which is received by the server and sent back to the user's device. The user plays the music on their device and edits it as needed. If the user is satisfied with the music, they click the save button to save the music data to the database.

[0130] 4. When managing and buying / selling copyrights of musical works

[0131] Copyright for saved music data is managed on the server, and users can set prices and sales conditions on the sales page. Based on the information set by the user, other users can view the music information and submit purchase requests. The server processes the transaction and grants the buyer the right to use the music. A commission is deducted from the transaction amount, and the remainder is distributed to the seller.

[0132] Thus, the system of the present invention allows users to easily generate music and smoothly manage and buy / sell that music. Furthermore, the system can increase platform revenue and contribute to the creation of a new music market.

[0133] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0134] Step 1: User submits a song generation request

[0135] The user accesses the music generation screen from their device, enters parameters such as "Genre: Pop" and "Theme: Summer Memories," and presses the generate button. The input data (song type, music genre, theme, etc.) is sent to the server.

[0136] Specific actions:

[0137] The user opens a web browser and accesses the music generation page.

[0138] The user enters detailed information about the song into the form.

[0139] The user clicks the "Generate" button, and the input data is sent to the server (Input: song parameters, Output: HTTP request to the server).

[0140] Step 2: The server receives the music generation request and sends it to the AI ​​model.

[0141] The server receives a request from the user, analyzes the input parameters, and sends them to the generated AI model. The AI ​​model receives a prompt message along with the analysis results.

[0142] Specific actions:

[0143] The server receives an HTTP request and parses the input data.

[0144] The server calls the API of the generated AI model and passes the analysis results as prompt messages (input: user request, output: prompt messages).

[0145] Step 3: Music generation using an AI model

[0146] The AI ​​model generates music, lyrics, sheet music, vocals, and accompaniment based on the received prompt text. This generation process is performed using an advanced neural network. The generated music data is saved as a temporary file.

[0147] Specific actions:

[0148] The AI ​​model executes a music generation algorithm based on the prompt text.

[0149] The generated music data is saved as a temporary file (input: prompt text, output: music data).

[0150] Step 4: Receiving the generated music data from the server and returning it to the user terminal.

[0151] The server receives the music data generated by the AI ​​model and sends it back to the user's device. The music data is sent to the user as an HTTP response.

[0152] Specific actions:

[0153] The server receives music data generated from the AI ​​model.

[0154] The server creates a response to send music data to the user's terminal (input: music data, output: HTTP response to the user).

[0155] Step 5: User plays and edits music.

[0156] The user's terminal receives the generated music data and displays it in a playable state. The user can listen to the music and edit it as needed. After editing, they click the save button to send the edits to the server.

[0157] Specific actions:

[0158] The user opens a music playback application or browser.

[0159] Play and edit received music files.

[0160] The user's edits are sent to the server (input: edited music data, output: save request to the server).

[0161] Step 6: User submits a request to save music.

[0162] If the user is satisfied with the generated music data, they click the save button, and a save request is sent to the server. The music data is then sent back to the server.

[0163] Specific actions:

[0164] The user clicks the "Save" button.

[0165] The save request and music data are sent to the server (input: music data, output: HTTP request to the server).

[0166] Step 7: The server receives the music save request and saves it to the database.

[0167] The server receives the save request and saves the music data to the database. A message confirming the save is sent back to the user.

[0168] Specific actions:

[0169] The server receives the save request and connects to the database.

[0170] This process saves music data to a database and sends a save completion notification to the user (Input: Music data, Output: Save completion message).

[0171] Step 8: Server-based copyright management and publication of sales information for music.

[0172] The server provides tools for managing copyrights to stored music data, and users set prices and conditions on the sales page. The set sales information is made public.

[0173] Specific actions:

[0174] The server registers the music data with the copyright management system.

[0175] Users enter sales information and publish it (input: sales information, output: published sales page).

[0176] Step 9: Another user submits a request to purchase music.

[0177] Other users view the published song information and click the purchase button. A purchase request is sent to the server.

[0178] Specific actions:

[0179] Those interested in purchasing the song access the public page to view it.

[0180] Click the purchase button to send a purchase request to the server (Input: Purchase request, Output: HTTP request to the server).

[0181] Step 10: Server receives music purchase request and processes transaction.

[0182] The server receives the purchase request and grants the buyer the right to use the music. It deducts a commission from the transaction amount and distributes the remaining amount to the seller.

[0183] Specific actions:

[0184] The server processes the purchase request and grants the purchaser the right to use the music.

[0185] The transaction amount is reduced by a commission, and the remaining amount is distributed to the seller (Input: Purchase Request, Output: Transaction Completion Message).

[0186] (Application Example 1)

[0187] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0188] In today's world, there is a need for a system that allows users to easily create music, distribute it to other users, rate it, and sell it. However, current systems fragment the process from music creation to distribution, rating, and selling, resulting in a poor user experience. Furthermore, copyright management and monetization of created music are often handled separately, requiring users to go through cumbersome procedures. In response to this, a system that can handle everything from music creation to distribution, rating, and selling in an integrated manner is needed.

[0189] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0190] In this invention, the server includes an input means on which a user can input parameters such as the type of music, music genre, and theme; a generation means that generates music, lyrics, sheet music, vocals, and accompaniment based on the parameters input via the input means; a return means that returns the music data generated by the generation means to the user; a storage means that stores the music data; a management means that manages the copyright to the music data stored in the storage means; a trading means that buys and sells the copyright between users; a revenue means that makes the profit generated by the trading means into platform revenue; an evaluation means that evaluates the music data generated by the user and creates a ranking; and a distribution means that distributes the generated music data to other users. As a result, users can enjoy a consistent experience from music generation to distribution, evaluation, and buying and selling, and copyright management and monetization can be handled centrally.

[0191] An "input method" is an interface for users to input parameters such as the type of music, music genre, and theme.

[0192] The "generation means" is a function for generating music, lyrics, sheet music, vocals, and accompaniment based on parameters input via the aforementioned input means.

[0193] The "return means" is a function for returning the music data generated by the generation means to the user.

[0194] "Storage means" refers to storage means for storing the aforementioned music data.

[0195] "Management means" refers to functions for managing copyrights to stored music data.

[0196] "Means of buying and selling" refers to a function for buying and selling the aforementioned copyrights among users.

[0197] "Revenue-generating means" refers to a function that generates revenue for the platform from the profits generated by the aforementioned buying and selling means.

[0198] The "evaluation method" is an interface for users to evaluate generated music data and create rankings.

[0199] "Distribution method" refers to a function for distributing generated music data to other users.

[0200] This invention provides a system that allows users to easily create music, manage and sell its copyrights, and enable other users to rate and distribute that music. This system is implemented by the following means.

[0201] First, the user inputs song parameters (song type, music genre, theme, etc.) via an input device such as a smartphone. The entered parameters are then sent to the server.

[0202] The server receives these parameters and uses generation methods to generate music, lyrics, sheet music, vocals, and accompaniment. OpenAI's GPT-4 (registered trademark) and music generation-specific models are used as generation methods. The generated music data is sent back from the server to the user's terminal.

[0203] The user's device displays the returned music data in a playable state. The user can listen to the music and edit it as needed using the editing tools. The edited music data is saved via the storage tools.

[0204] Copyright for stored music data is managed on a per-user basis using management tools. Users can then set the selling price of their music and sell it to other users using the trading tools. When a sale is completed, the profits generated are processed as revenue for the platform through revenue-generating mechanisms.

[0205] Furthermore, the evaluation system allows users to rate music data generated by other users and create rankings. Users can then improve their music based on these ratings. Finally, highly-rated music and newly generated music are distributed to other users via the distribution system.

[0206] This system allows users to enjoy a consistent experience from music creation and editing to storage, distribution, and buying / selling. Furthermore, copyright management and monetization can be handled centrally. This is expected to contribute to the creation of a new music market.

[0207] For example, if a user wants to generate a jazz song with the theme "Summer Memories," it would look like this:

[0208] The user inputs the parameters "jazz" and "summer memories" into an input device. The server uses this information to generate a song using an AI model and sends the generated song data back to the user's terminal. The user listens to the song, edits it, saves it, and distributes it to other users.

[0209] Example of a prompt:

[0210] Song type: Pop

[0211] Music genre: Jazz

[0212] Theme: Summer Memories

[0213] In this way, the invention effectively meets user needs and provides a new musical experience.

[0214] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0215] Step 1:

[0216] The user launches the music generation application and accesses the input fields. The user enters the type of song (e.g., pop), music genre (e.g., jazz), and theme (e.g., summer memories). This prepares the user to send the parameters of the song they want to generate to the server.

[0217] input:

[0218] Type of music (e.g., pop)

[0219] Music genres (jazz, etc.)

[0220] Theme (e.g., summer memories)

[0221] output:

[0222] Music generation parameters (sent to the server in JSON format)

[0223] Step 2:

[0224] The server receives music generation parameters (JSON data) from the user. The server analyzes this data and sends a music generation request to the AI ​​model. The AI ​​model generates music, lyrics, sheet music, vocals, and accompaniment based on the specified parameters.

[0225] input:

[0226] Music generation parameters (JSON format)

[0227] Data processing:

[0228] Convert music generation parameters into prompt statements for the AI ​​model.

[0229] output:

[0230] Generated music data (music, lyrics, sheet music, vocals, accompaniment)

[0231] Step 3:

[0232] The server analyzes the music data received from the generating AI model and sends that data back to the user's device. The user's device receives the music data and displays it in a playable state. The user can listen to the generated music by clicking the play button.

[0233] input:

[0234] Generated music data

[0235] output:

[0236] Music data playable on the user's device

[0237] Step 4:

[0238] Users listen to music and edit the music data using editing tools as needed. The edited music data is temporarily saved on the device.

[0239] input:

[0240] Played music data

[0241] User-edited parameters

[0242] Data processing:

[0243] Update music data based on user-edited parameters.

[0244] output:

[0245] Edited music data

[0246] Step 5:

[0247] After the user finishes editing, they click the save button. The user's device sends the edited music data to the server, which then persistently stores the data using a storage method. Copyright management information is also added to the saved data.

[0248] input:

[0249] Edited music data

[0250] output:

[0251] Persistent music data

[0252] Step 6:

[0253] Users set a selling price for their saved music data and send a buy / sell request to the server. The server makes the music data available to other users using the buy / sell system and facilitates the transaction. Any profits generated are processed using the revenue system.

[0254] input:

[0255] Selling price setting

[0256] Buy / Sell Request

[0257] output:

[0258] Released song information

[0259] Revenue sharing information

[0260] Step 7:

[0261] Other users rate the generated songs and send their ratings to the server. The server uses the rating system to collect the rating data and create rankings. Highly rated songs are recommended to other users using the distribution system.

[0262] input:

[0263] Evaluation data from other users

[0264] output:

[0265] Rating Ranking

[0266] Recommended Song List

[0267] In this way, specific inputs and outputs, as well as data processing, are performed at each step, allowing users to obtain a consistent music generation and distribution experience.

[0268] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0269] This invention is a system for users to easily generate music and manage and sell its copyrights. This system not only has an input means for users to input parameters such as the type of music, music genre, and theme, but also incorporates an emotion engine that recognizes the user's emotions.

[0270] Steps for users to generate music

[0271] Users access the music generation screen from their own devices (PCs or smartphones). In addition to inputting parameters such as song type, music genre, and theme, the emotion engine recognizes the user's facial expressions and voice tone to analyze their current emotions.

[0272] The role of the emotional engine

[0273] The emotion engine analyzes data acquired through the user's camera and microphone to identify the user's emotional state (e.g., joy, sadness, surprise, anger). This emotional data is also used as a parameter for music generation. For example, if the emotion engine recognizes that the user is in a state of "joy," it will suggest a song with an upbeat tempo that matches that feeling of joy.

[0274] Server-based music generation

[0275] The server receives requests from users and results from the emotion engine, and requests the AI ​​model to generate music. The AI ​​model automatically generates music, lyrics, sheet music, vocals, and accompaniment based on the input parameters and the user's emotion data. The server then receives the generated music data.

[0276] Return and display of music data

[0277] The server sends the generated music data back to the user's device. The user's device displays the received music in a playable state. The user can listen to this music and edit it as needed.

[0278] Saving Music Data

[0279] If the user is satisfied with the music data generated, click the save button to save the music. The server receives the save request and saves the music data in the database. As a result, the user can re-access the music at any time.

[0280] Copyright Management and Trading

[0281] The server provides a tool for managing the copyright of the saved music data for each user. The user enters the price settings and sales conditions on the music sales page and sends a sales request to the server. The server saves this information and publishes it in the copyright management system.

[0282] Transactions between Users

[0283] Other users can view the published music information and select the music they wish to purchase. When a purchase request is sent to the server, the server processes the transaction and grants the purchaser the right to use the music. Additionally, the platform's revenue is deducted from the transaction amount, and the remainder is distributed to the seller. As a result, smooth transactions are conducted between users, and it is possible to commercially utilize and monetize the music generated by individuals.

[0284] Specific Usage Examples

[0285] For example, let's assume that User A generates a jazz piece themed "Love in Winter," and the emotion engine recognizes User A's emotion as "joy." User A accesses the music generation screen and provides "jazz," "Love in Winter," and the joy emotion data provided by the emotion engine as input. The server sends this information to the AI model, and the model generates an appropriate piece of music. This music is displayed in a playable state on User A's terminal. User A edits and saves the music, and then sets the selling price and creates a copyright listing.

[0286] If another user B likes this piece of music and wants to purchase it, they click the purchase button on the sales page. The server processes the transaction, grants user B the right to use the music, and distributes the remainder after deducting the platform's revenue from the transaction amount to user A.

[0287] As described above, by using this system, users can easily generate music and manage and trade its copyright. The introduction of the emotion engine enables the generation of music tailored to the emotions of users, providing a more personalized music experience. In addition, since the system can obtain revenue as a platform, it contributes to the creation of a new music market.

[0288] The processing flow will be described below.

[0289] Processing Flow of a Music Generation System Combined with an Emotion Engine

[0290] Step 1:

[0291] The user inputs their email address, password, and profile information into the registration form from the terminal and clicks the registration button.

[0292] Step 2:

[0293] The server receives the registration information sent by the user and saves it in the database.

[0294] Step 3:

[0295] The server sends an authentication email to the user. This email contains an authentication link.

[0296] Step 4:

[0297] The user opens the authentication email and clicks the authentication link.

[0298] Step 5:

[0299] The server receives a request from the authentication link and updates the user's authentication status.

[0300] Music generation request

[0301] Step 1:

[0302] The user inputs parameters such as the type, genre, and theme of the music on the music generation screen of the terminal, turns on the camera and microphone of the terminal, and the emotion engine obtains the user's emotion data.

[0303] Step 2:

[0304] The emotion engine analyzes the data obtained through the camera and microphone to identify the user's emotional state.

[0305] Step 3:

[0306] The server sends the request data received from the user and the results of the emotion engine to the AI model.

[0307] Step 4:

[0308] The AI model generates music, lyrics, sheet music, vocals, and accompaniment based on the parameters and the user's emotion data.

[0309] Step 5:

[0310] The server receives the generation results from the AI model and returns them to the user's terminal.

[0311] Step 6:

[0312] The terminal displays the generated music data to the user and provides it in a playable state. <°

[0313] Music editing and saving

[0314] Step 1:

[0315] The user edits the generated music data on the device's editing screen. For example, they can change the melody, revise the lyrics, and adjust the accompaniment.

[0316] Step 2:

[0317] The user clicks the save button to save the edited music data.

[0318] Step 3:

[0319] The server receives a save request from the user and saves the edited music data to the database.

[0320] Copyright establishment and sales

[0321] Step 1:

[0322] Users access the music sales screen from their devices and set the price and sales conditions for the songs.

[0323] Step 2:

[0324] The user submits a sales request to the server.

[0325] Step 3:

[0326] The server stores sales information in a database and publishes copyright listings.

[0327] Transactions between users

[0328] Step 1:

[0329] Other users select a song from the sales page and click the purchase button.

[0330] Step 2:

[0331] The device sends a purchase request to the server.

[0332] Step 3:

[0333] The server processes the purchase request and grants the purchaser the right to use the music.

[0334] Step 4:

[0335] The server deducts the platform's revenue from the transaction amount and distributes the remainder to the music sellers.

[0336] Specific usage examples

[0337] Step 1:

[0338] Let's say User A generates a jazz song with the theme "winter love," and the emotion engine recognizes User A's emotion as "joy." User A accesses the song generation screen and provides "jazz," "winter love," and the joy emotion data provided by the emotion engine as input.

[0339] Step 2:

[0340] The server sends this information to the AI ​​model, which then generates the appropriate song.

[0341] Step 3:

[0342] This song is displayed in a playable state on User A's device. User A edits and saves the song, then sets a selling price and creates a copyright listing.

[0343] Step 4:

[0344] If another user B likes this song and wants to buy it, they click the purchase button on the sales page.

[0345] Step 5:

[0346] The server processes the transaction, grants user B the right to use the music, and distributes the remainder (transaction amount minus platform revenue) to user A.

[0347] As described above, this system allows users to easily generate music and manage and sell its copyrights. The introduction of an emotion engine enables music generation tailored to the user's emotions, providing a more personalized musical experience. Furthermore, the system generates revenue as a platform, contributing to the creation of a new music market.

[0348] (Example 2)

[0349] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0350] Traditional music generation systems lacked sufficient functionality for users to easily generate music and manage and trade its copyrights. Furthermore, they were unable to generate music that reflected the user's emotions, making it difficult to provide a personalized experience. Additionally, the processes for saving, managing, and trading generated music were inconvenient, highlighting the need for a consistent user experience.

[0351] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0352] In this invention, the server includes an input means on which the user can input parameters such as the type of song, music genre, and theme; an emotion recognition means that recognizes the user's emotions and analyzes the emotion data; a generation means that generates songs, lyrics, sheet music, vocals, and accompaniments based on the parameters and emotion data input via the input means and the emotion recognition means; a return means that returns the song data generated by the generation means to the user; a storage means that stores the song data; a management means that manages the copyright to the song data stored in the storage means; a trading means that buys and sells the copyright between users; and a revenue means that makes the profit generated by the trading means into revenue for the platform. As a result, users can easily and efficiently generate songs, receive personalized songs tailored to their emotions, and enjoy a consistent process of storage, management, and trading.

[0353] An "input method" is a means by which the user inputs parameters such as the type of song, music genre, and theme.

[0354] An "emotion recognition tool" is a means of recognizing a user's emotions and analyzing that emotional data.

[0355] "Generation means" refers to means for generating music, lyrics, sheet music, vocals, and accompaniment based on parameters and emotion data input via input means and emotion recognition means.

[0356] "Return means" refers to the means of returning the music data generated by the generation means to the user.

[0357] "Storage method" refers to the means of saving the generated music data.

[0358] "Management means" refers to means for managing copyrights to music data stored in storage means.

[0359] "Means of buying and selling" refers to the means by which users buy and sell copyrights among themselves.

[0360] "Revenue-generating means" refers to the means by which the platform generates revenue from the profits produced through buying and selling.

[0361] This invention is a system that allows users to easily generate music and manage and sell its copyrights. The system incorporates an emotion recognition engine to enable music generation that reflects the user's emotions. The system consists of various components, including the user's terminal, a data processing server, the emotion recognition engine, and a generation AI model used for music generation. Specific hardware used includes personal computers and smartphones. Software used includes a web browser and an AI model (e.g., OpenAI's GPT-3 or Google's MusicLM).

[0362] User-inputted parameters

[0363] Users access the music generation system's webpage using a browser on their PC or smartphone. There, they input the type of song, music genre, theme, etc. They also grant permission for the camera and microphone to be used for AI-powered emotion recognition. At that point, a form appears on the screen for entering specific parameters such as "jazz" or "winter love."

[0364] emotion recognition

[0365] When a user allows the use of the camera and microphone, the device sends video and audio to an emotion recognition engine. The engine analyzes facial expressions and tone of voice to identify the user's current emotions. For example, the engine outputs emotion data such as "joy," "sadness," or "surprise."

[0366] Data processing and music generation

[0367] The user's input parameters and emotion data obtained from the emotion recognition engine are sent to the server in a single batch. The server processes this data and sends a request for music generation to the generative AI model. The generative AI model generates music, lyrics, sheet music, vocals, accompaniment, etc., based on these inputs. In doing so, the AI ​​model generates music with characteristics that match the user's emotions. For example, if the analysis results indicate the emotion of "joy," a bright and upbeat song will be generated.

[0368] Return and display of music data

[0369] The music data generated by the AI ​​model is sent back to the server, from which it is transmitted to the user's device. The user's device displays the received data in a playable format, allowing the user to listen to the generated music. If the user is satisfied with the music, clicking the "Save" button sends the request to the server, which then saves the music data to its database.

[0370] Copyright management and buying and selling of music

[0371] The server manages the copyrights to the stored music data and allows users to set the sales conditions for the music. If another user wishes to purchase a song, a purchase request is sent to the server, and the server processes the transaction. The remaining amount after deducting the platform's revenue from the transaction amount is distributed to the user who created the music.

[0372] Specific example

[0373] For example, suppose user A inputs parameters such as "jazz" and "winter love," and the emotion recognition engine detects the emotion "joy." The server sends this data to an AI model, which generates a cheerful jazz song tailored to user A's parameters. This song is displayed on user A's device in a playable format, and user A can listen to, edit, and save the song.

[0374] Example of a prompt

[0375] "Please generate a jazz song. The theme is 'winter love,' and the emotion is 'joy.'"

[0376] As described above, this system allows users to easily generate music and seamlessly manage and sell its copyrights. Furthermore, the introduction of emotion recognition enables the generation of personalized music that responds to the user's emotions.

[0377] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0378] Step 1:

[0379] The user accesses the music generation screen.

[0380] Input: Launch a browser on the user's device and access the music generation system's webpage.

[0381] Specific steps: The user uses a browser on their computer or smartphone to enter a specified web address and open the music generation system screen. They then log in following the interface displayed there.

[0382] Step 2:

[0383] The user enters the parameters of the song.

[0384] Input: Parameters such as song type (e.g., ballad), music genre (e.g., jazz), and theme (e.g., "Winter Love").

[0385] Specific operation: The user enters the required information into the input form on the screen. After entering the parameters, they click the "Complete" button.

[0386] Output: The input parameters are saved on the device and sent to the server as data to proceed with the emotion recognition process.

[0387] Step 3:

[0388] The device activates the camera and microphone.

[0389] Input: Request for use of camera and microphone with user permission.

[0390] Specific action: The user grants permission to use the camera and microphone through a dialog box displayed on the screen.

[0391] Output: Once permission is obtained, the camera and microphone will activate and begin collecting data on the user's facial expressions and voice.

[0392] Step 4:

[0393] The emotion recognition engine analyzes the user's emotions.

[0394] Input: Facial expression data and audio data acquired from the camera and microphone.

[0395] Specific operation: The emotion recognition engine analyzes data in real time to identify the user's emotional state (e.g., joy, sadness, surprise, etc.).

[0396] Output: The emotional data obtained as an analysis result is sent to the server.

[0397] Step 5:

[0398] The server receives user input and sentiment data.

[0399] Input: Parameters entered by the user and emotion data sent from the emotion recognition engine.

[0400] Specific operation: The server receives this data and integrates it as a series of data.

[0401] Output: The integrated data is prepared as input for the AI ​​model.

[0402] Step 6:

[0403] The server sends a request to the AI ​​model.

[0404] Input: Integrated song parameters and emotion data.

[0405] Specific operation: The server creates a specific prompt message for the generated AI model and sends a request for music generation.

[0406] Output: Prompt message sent to the AI ​​model: "Generate a jazz song. The theme is 'winter love,' and the emotion is 'joy.'"

[0407] Step 7:

[0408] Music generation using AI models

[0409] Input: The prompt message sent from the server.

[0410] Specific operation: The AI ​​model generates music based on prompt text. It generates data in various forms, including lyrics, sheet music, vocals, and accompaniment.

[0411] Output: The generated music data is sent back to the server.

[0412] Step 8:

[0413] The server sends the generated music data to the user's terminal.

[0414] Input: Generated music data returned from the AI ​​model.

[0415] Specific operation: The server receives the generated data and sends it to the user's terminal.

[0416] Output: Music data sent to the user's device.

[0417] Step 9:

[0418] The device displays music data.

[0419] Input: Music data sent from the server.

[0420] Specific operation: The device converts the data into a playable format and displays it to the user visually and audibly.

[0421] Output: Music displayed in a playable format.

[0422] Step 10:

[0423] Users edit songs

[0424] Input: Songs displayed in playable formats.

[0425] Specific actions: The user edits the music using the music editing tools as needed. Once the modifications are complete, they submit another save request.

[0426] Output: The edited music data is saved to the device.

[0427] Step 11:

[0428] The device sends a save request to the server.

[0429] Input: Edited song data.

[0430] Specific action: The user clicks the save button, and a save request is sent to the server.

[0431] Output: The server receives the music data.

[0432] Step 12:

[0433] The server saves the music data to the database.

[0434] Input: Music data for which a save request was received.

[0435] Specific operation: The server saves the music data to a database, allowing users to access it again.

[0436] Output: Music data stored in the database.

[0437] Step 13:

[0438] The server manages the copyright of the music.

[0439] Input: Saved music data.

[0440] Specific operation: The server manages copyright information for each song and assigns copyright information to each song.

[0441] Output: Managed copyright information.

[0442] Step 14:

[0443] The user enters the music sales conditions.

[0444] Input: Sales conditions (price, terms of use, etc.) entered on the music sales page.

[0445] Specific operation: The user enters the sales price and terms of use, and sends a sales request to the server.

[0446] Output: Sales conditions data from users.

[0447] Step 15:

[0448] The server processes the transaction.

[0449] Input: Sales conditions data from users and purchase requests from other users.

[0450] Specific operation: The server processes the transaction and grants the buyer the right to use the music. Then, it distributes the remaining amount to the seller after deducting the platform's revenue from the transaction amount.

[0451] Output: Transaction information and distributed profits.

[0452] (Application Example 2)

[0453] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0454] Traditional music generation systems typically generate music based on user-inputted parameters such as song type, genre, and theme, with little consideration given to the user's emotional state. Furthermore, copyright management and trading of generated music were complex, making platform monetization difficult. Additionally, there was a lack of easy ways for users to save, manage, and commercially utilize the music they generated themselves.

[0455] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0456] In this invention, the server includes an input means on which the user can input parameters such as the type of song, music genre, and theme; an emotion recognition means that recognizes the user's emotional state using an emotion engine and reflects the parameters based on that emotion in song generation; and a generation means that generates songs, lyrics, sheet music, vocals, and accompaniment based on the parameters input via the input means and the emotion recognition means. This enables the automatic generation of personalized songs that reflect the user's emotional state, and allows for efficient monetization through copyright management and sales.

[0457] "Input method" refers to a device or interface for users to input parameters such as the type of music, music genre, and theme.

[0458] "Generation means" refers to a system or algorithm that automatically generates music, lyrics, sheet music, vocals, and accompaniment based on parameters input via the input means and emotion recognition means.

[0459] "Emotion recognition means" refers to technology that uses an emotion engine to recognize the user's emotional state and reflects parameters based on that emotion in music generation.

[0460] "Return means" refers to the communication infrastructure and software used to return the music data generated by the generation means to the user.

[0461] "Storage means" refers to a storage device or database system for storing the music data generated by the generation means.

[0462] "Management means" refers to tools or software for managing the copyright of music data stored in the aforementioned storage means.

[0463] "Means of buying and selling" refers to platforms or trading systems for buying and selling the aforementioned copyrights among users.

[0464] "Revenue-generating means" refers to a mechanism or management system for generating revenue for the platform from the profits generated by the aforementioned buying and selling means.

[0465] "Editing means" refers to software or interfaces that allow users to edit generated music data.

[0466] "Authentication means" refers to the systems and protocols used for user registration and authentication.

[0467] A "server" is a computer or network system that runs a music generation system and processes requests from users.

[0468] System Overview

[0469] This system allows users to input parameters such as song type, music genre, and theme, collect emotional data using emotion recognition means, and generate and manage songs based on that data. Specific components include input means, generation means, emotion recognition means, return means, storage means, management means, buying and selling means, and revenue means.

[0470] Program Processing Description

[0471] 1. User interface (input means):

[0472] Users access the music generation screen through a smartphone application. They input parameters such as the type of song, music genre, and theme. For example, they can select options such as "rock" or "summer adventure."

[0473] 2. Emotion recognition means:

[0474] The system analyzes facial images and audio data acquired through the user's camera and microphone. Emotion recognition, in particular, utilizes the "EmotionRecognizer" module to identify the user's emotional state (joy, sadness, surprise, anger, etc.). For example, if the user smiles at the camera, the system determines that they are in a state of "joy."

[0475] 3. Music generation (generation methods):

[0476] The server receives data from the input means and emotion recognition means and generates music using the "MusicGenerator" module. During generation, the original parameters and emotion data are reflected. For example, if rock is selected and the user's emotional state is "joy," an upbeat song with positive elements will be generated.

[0477] 4. Return of generated music (return method):

[0478] The server sends the generated music data back to the user's smartphone. The user can then play and check the received music within the app.

[0479] 5. Preservation means:

[0480] If the user is satisfied with the generated music data, they can click the save button to send a save request to the server. The server receives this request and saves the music data to its database.

[0481] 6. Music management and trading (management methods and trading methods):

[0482] The server manages the copyright of the stored music data. Users enter pricing and sales conditions on the music sales page and submit sales requests. Other users can view and purchase the music published on the sales page.

[0483] 7. Monetization (means of revenue):

[0484] A system is in place where the platform's revenue is deducted from the profits generated by the buying and selling methods, and the remainder is distributed to the sellers.

[0485] Specific example

[0486] For example, suppose a user wants to generate a song with the themes of "rock" and "summer adventure." If the emotion recognition system detects "joy" from the user's facial expression, a song will be generated based on "rock," "summer adventure," and "joy."

[0487] Examples of prompts for generative AI models

[0488] The user wants to generate a song with the themes of "rock" and "summer adventure." The emotion engine detected "joy" from the user's facial expression. Please generate a song based on this.

[0489] This allows users to easily create, save, manage, and sell personalized music. Furthermore, it is expected that generating music based on emotional data will further enhance user satisfaction.

[0490] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0491] Step 1:

[0492] The user accesses the music generation screen through a smartphone application. Here, the user inputs parameters such as the type of music, music genre, and theme (input: user input, output: parameter data). This data is necessary for subsequent processing and serves to specify the user's wishes.

[0493] Step 2:

[0494] The user's facial expressions and audio data are collected through the smartphone's camera and microphone (input: camera video, audio data; output: emotion data). This data is analyzed by an emotion recognition system to identify the user's emotional state (e.g., joy, sadness, surprise, anger). The "EmotionRecognizer" module is used for this analysis.

[0495] Step 3:

[0496] The terminal sends the collected emotion data and the parameter data of the initially entered song to the server (input: emotion data, parameter data; output: integrated data). The server receives this data, integrates it, and creates a request for song generation.

[0497] Step 4:

[0498] The server uses the "MusicGenerator" module to generate music, lyrics, sheet music, vocals, and accompaniment based on integrated data (input: integrated data, output: generated music data). The data processing performed here involves generating the optimal music by considering emotional data, song type, music genre, and theme.

[0499] Step 5:

[0500] The generated music data is sent back from the server to the user's smartphone (input: generated music data, output: data sent to the user's device). The user's smartphone receives this data and displays it on the music playback screen.

[0501] Step 6:

[0502] When a user plays and reviews a generated song and wishes to save it, they click the save button to send a save request to the server (input: user's save request, output: song save confirmation). The server receives this request and saves the song data to the database.

[0503] Step 7:

[0504] A management system operates to manage copyrights for saved music data (input: saved music data, output: copyright management data). Users can enter pricing and sales conditions on the music sales page and submit sales requests.

[0505] Step 8:

[0506] When another user purchases a publicly available song, the buying and selling mechanism is activated and the transaction takes place (input: purchase request, output: song usage rights). The server processes the transaction, deducts the platform's revenue from the generated profit, and distributes it to the seller (input: transaction data, output: revenue data).

[0507] Step 9:

[0508] The platform calculates revenue from the profits generated through buying and selling and distributes it to users (input: revenue data, output: revenue distribution data). This enables smooth transactions between users and allows individuals to commercially use and monetize music they have created.

[0509] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0510] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0511] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0512] [Second Embodiment]

[0513] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0514] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0515] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0516] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0517] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0518] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0519] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0520] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0521] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0522] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0523] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0524] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0525] This invention is a system for users to easily generate music and manage and buy / sell its copyrights. This system includes an input means for users to input parameters such as the type of music, music genre, and theme.

[0526] Steps for users to generate music

[0527] Users access the music generation screen from their own devices (PCs or smartphones). Here, they input parameters such as the type of song, music genre, and theme. For example, specific inputs such as "Pop" and "Summer Memories" are possible.

[0528] Server-based music generation

[0529] The server receives a request from the user and instructs the AI ​​model to generate music. Based on the input parameters, the AI ​​model automatically generates the music, lyrics, sheet music, vocals, and accompaniment. The server then receives the generated music data.

[0530] Return and display of music data

[0531] The server sends the generated music data back to the user's device. The user's device displays the received music in a playable state. The user can listen to this music and edit it as needed.

[0532] Saving music data

[0533] If the user is satisfied with the generated music data, they click the save button to save the song. The server receives the save request and saves the music data to the database. This allows the user to access the song again at any time.

[0534] Copyright management and trading

[0535] The server provides tools for managing copyrights to stored music data on a per-user basis. Users enter pricing and sales conditions on the music sales page and send sales requests to the server. The server stores this information and makes it public in the copyright management system.

[0536] Transactions between users

[0537] Other users can view the published song information and select the songs they wish to purchase. Once a purchase request is sent to the server, the server processes the transaction and grants the buyer the right to use the song. Furthermore, the platform deducts its revenue from the transaction amount and distributes the remainder to the seller. This facilitates smooth transactions between users and makes it possible for individuals to commercially use and monetize their created music.

[0538] Specific usage examples

[0539] For example, suppose User A wants to generate a jazz song with the theme "winter love." User A accesses the song generation screen and enters "jazz" and "winter love." The server sends this information to the AI ​​model, which generates an appropriate song. This song is displayed on User A's device and is playable. User A edits and saves the song, then sets a sales price and creates a copyright listing.

[0540] If another user B likes this song and wants to buy it, they click the purchase button on the sales page. The server processes the transaction, grants user B the right to use the song, and distributes the remaining amount to user A after deducting the platform's revenue from the transaction amount.

[0541] As described above, this system allows users to easily create music and manage and buy / sell its copyrights. Furthermore, since the system can generate revenue as a platform, it contributes to the creation of a new music market.

[0542] The following describes the processing flow.

[0543] User registration and authentication

[0544] Step 1:

[0545] The user enters their email address, password, and profile information into the registration form on their device and clicks the registration button.

[0546] Step 2:

[0547] The server receives registration information sent by the user and stores it in the database.

[0548] Step 3:

[0549] The server sends an authentication email to the user. This email contains an authentication link.

[0550] Step 4:

[0551] The user opens the verification email and clicks the verification link.

[0552] Step 5:

[0553] The server receives the request from the authentication link and updates the user's authentication status.

[0554] Song generation request

[0555] Step 1:

[0556] The user enters parameters such as the song type, genre, and theme on the device's music generation screen and clicks the generate button.

[0557] Step 2:

[0558] The terminal sends the entered parameters to the server.

[0559] Step 3:

[0560] The server sends the request data received from the user to the AI ​​model.

[0561] Step 4:

[0562] The AI ​​model generates music, lyrics, sheet music, vocals, and accompaniment based on the parameters.

[0563] Step 5:

[0564] The server receives the generated results from the AI ​​model and sends them back to the user's device.

[0565] Step 6:

[0566] The device displays the generated music data to the user.

[0567] Editing and saving music

[0568] Step 1:

[0569] The user edits the generated music data on the device's editing screen. For example, they can change the melody, revise the lyrics, and adjust the accompaniment.

[0570] Step 2:

[0571] The user clicks the save button to save the edited music data.

[0572] Step 3:

[0573] The server receives a save request from the user and saves the edited music data to the database.

[0574] Copyright establishment and sales

[0575] Step 1:

[0576] Users access the music sales screen from their devices and set the price and sales conditions for the songs.

[0577] Step 2:

[0578] The user submits a sales request to the server.

[0579] Step 3:

[0580] The server stores sales information in a database and publishes copyright listings.

[0581] Transactions between users

[0582] Step 1:

[0583] Other users select a song from the sales page and click the purchase button.

[0584] Step 2:

[0585] The device sends a purchase request to the server.

[0586] Step 3:

[0587] The server processes the purchase request and grants the purchaser the right to use the music.

[0588] Step 4:

[0589] The server deducts the platform's revenue from the transaction amount and distributes the remainder to the music sellers.

[0590] The above outlines the specific processing steps in the patent system.

[0591] (Example 1)

[0592] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0593] While systems exist today for easily generating music and managing and trading its copyrights, they do not fully meet user needs for smooth music generation, management, and trading procedures. Furthermore, improving the efficiency of revenue sharing associated with the editing and trading of generated music remains a challenge. This invention aims to solve these problems and provide an efficient music generation, management, and trading system that is easily accessible to users.

[0594] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0595] In this invention, the server includes an input means on which a user can input parameters such as the type of music, music genre, and theme; a generation means that generates music, lyrics, sheet music, vocals, and accompaniment based on the parameters input via the input means; a return means that sends the music data generated by the generation means back to the user; a storage means that stores the music data; a management means that manages the copyright to the music data stored in the storage means; a trading means that buys and sells the copyright between users; a revenue means that makes the profit generated by the trading means into platform revenue; a transaction means that checks the published music information of other users and processes purchase requests when they wish to purchase it; and a distribution means that deducts a commission from the transaction amount and distributes the remaining amount to the seller. This makes it possible for a user to smoothly perform a series of operations, from inputting parameters to generate music, and managing, editing, and buying and selling that music.

[0596] An "input method" is an interface for users to input parameters such as the type of music, music genre, and theme.

[0597] The "generation means" is a system for generating music, lyrics, sheet music, vocals, and accompaniment based on parameters input via the input means.

[0598] The "return means" is a mechanism for sending the music data generated by the generation means to the user.

[0599] A "storage method" refers to a system for saving generated music data to a database or storage device.

[0600] "Management means" refers to a tool for managing the copyright of music data stored in the aforementioned storage means.

[0601] A "trading method" refers to a system for users to buy and sell copyrighted music that has been preserved.

[0602] "Revenue-generating means" refers to a mechanism for generating revenue for the platform from the profits generated by the aforementioned buying and selling means.

[0603] A "transaction method" is a system for other users to view publicly available song information and process purchase requests if they wish to buy it.

[0604] A "distribution mechanism" is a system for distributing the remaining amount to the seller after deducting fees from the transaction amount.

[0605] "Editing means" refers to an interface that allows the user to edit the aforementioned music data.

[0606] "Authentication means" refers to a system used for user registration and authentication.

[0607] This invention relates to a system for users to easily generate music and manage and buy / sell its copyrights. The system includes an input means for users to input parameters such as the type of music, music genre, and theme; a generation means for generating music, lyrics, sheet music, vocals, and accompaniment based on the input parameters; a return means for sending the generated music data back to the user; a storage means for storing the music data; a management means for managing the copyrights to the stored music data; a buying and selling means for users to buy and sell copyrights; a revenue means for making the profits generated by the buying and selling means into platform revenue; a transaction means for other users to check the published music information and process purchase requests if they wish to purchase it; a distribution means for deducting a commission from the transaction amount and distributing the remaining amount to the seller; and an editing means for editing music data and an authentication means for user registration and authentication.

[0608] Hardware and software configuration

[0609] This system consists of user terminals (PCs, smartphones) and a server based on a high-performance computer. Users can access the server from their terminals via an internet browser or a dedicated application.

[0610] For the AI ​​model that functions as a generation tool, for example, OpenAI's GPT-3 or a neural network model specifically designed for music generation can be used. For storage and management, cloud-based database systems (e.g., Amazon Web Services, Google Cloud Platform) can be applied.

[0611] System operation

[0612] 1. When a user submits a song generation request

[0613] The user accesses the music generation screen from their device, enters parameters such as "Genre: Pop" and "Theme: Summer Memories," and presses the generate button. The following is an example of the prompt text.

[0614] Genre: Pop

[0615] Theme: Summer Memories

[0616] 2. When the server receives a music generation request and sends it to the AI ​​model

[0617] The server receives the user's request, analyzes the input parameters, and sends them to the AI ​​model. Based on these parameters, the AI ​​model generates music, lyrics, sheet music, vocals, and accompaniment.

[0618] 3. When the generated music data is returned to the user and saved.

[0619] The AI ​​model generates music data, which is received by the server and sent back to the user's device. The user plays the music on their device and edits it as needed. If the user is satisfied with the music, they click the save button to save the music data to the database.

[0620] 4. When managing and buying / selling copyrights of musical works

[0621] Copyright for saved music data is managed on the server, and users can set prices and sales conditions on the sales page. Based on the information set by the user, other users can view the music information and submit purchase requests. The server processes the transaction and grants the buyer the right to use the music. A commission is deducted from the transaction amount, and the remainder is distributed to the seller.

[0622] Thus, the system of the present invention allows users to easily generate music and smoothly manage and buy / sell that music. Furthermore, the system can increase platform revenue and contribute to the creation of a new music market.

[0623] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0624] Step 1: User submits a song generation request

[0625] The user accesses the music generation screen from their device, enters parameters such as "Genre: Pop" and "Theme: Summer Memories," and presses the generate button. The input data (song type, music genre, theme, etc.) is sent to the server.

[0626] Specific actions:

[0627] The user opens a web browser and accesses the music generation page.

[0628] The user enters detailed information about the song into the form.

[0629] The user clicks the "Generate" button, and the input data is sent to the server (Input: song parameters, Output: HTTP request to the server).

[0630] Step 2: The server receives the music generation request and sends it to the AI ​​model.

[0631] The server receives a request from the user, analyzes the input parameters, and sends them to the generated AI model. The AI ​​model receives a prompt message along with the analysis results.

[0632] Specific actions:

[0633] The server receives an HTTP request and parses the input data.

[0634] The server calls the API of the generated AI model and passes the analysis results as prompt messages (input: user request, output: prompt messages).

[0635] Step 3: Music generation using an AI model

[0636] The AI ​​model generates music, lyrics, sheet music, vocals, and accompaniment based on the received prompt text. This generation process is performed using an advanced neural network. The generated music data is saved as a temporary file.

[0637] Specific actions:

[0638] The AI ​​model executes a music generation algorithm based on the prompt text.

[0639] The generated music data is saved as a temporary file (input: prompt text, output: music data).

[0640] Step 4: Receiving the generated music data from the server and returning it to the user terminal.

[0641] The server receives the music data generated by the AI ​​model and sends it back to the user's device. The music data is sent to the user as an HTTP response.

[0642] Specific actions:

[0643] The server receives music data generated from the AI ​​model.

[0644] The server creates a response to send music data to the user's terminal (input: music data, output: HTTP response to the user).

[0645] Step 5: User plays and edits music.

[0646] The user's terminal receives the generated music data and displays it in a playable state. The user can listen to the music and edit it as needed. After editing, they click the save button to send the edits to the server.

[0647] Specific actions:

[0648] The user opens a music playback application or browser.

[0649] Play and edit received music files.

[0650] The user's edits are sent to the server (input: edited music data, output: save request to the server).

[0651] Step 6: User submits a request to save music.

[0652] If the user is satisfied with the generated music data, they click the save button, and a save request is sent to the server. The music data is then sent back to the server.

[0653] Specific actions:

[0654] The user clicks the "Save" button.

[0655] The save request and music data are sent to the server (input: music data, output: HTTP request to the server).

[0656] Step 7: The server receives the music save request and saves it to the database.

[0657] The server receives the save request and saves the music data to the database. A message confirming the save is sent back to the user.

[0658] Specific actions:

[0659] The server receives the save request and connects to the database.

[0660] This process saves music data to a database and sends a save completion notification to the user (Input: Music data, Output: Save completion message).

[0661] Step 8: Server-based copyright management and publication of sales information for music.

[0662] The server provides tools for managing copyrights to stored music data, and users set prices and conditions on the sales page. The set sales information is made public.

[0663] Specific actions:

[0664] The server registers the music data with the copyright management system.

[0665] Users enter sales information and publish it (input: sales information, output: published sales page).

[0666] Step 9: Another user submits a request to purchase music.

[0667] Other users view the published song information and click the purchase button. A purchase request is sent to the server.

[0668] Specific actions:

[0669] Those interested in purchasing the song access the public page to view it.

[0670] Click the purchase button to send a purchase request to the server (Input: Purchase request, Output: HTTP request to the server).

[0671] Step 10: Server receives music purchase request and processes transaction.

[0672] The server receives the purchase request and grants the buyer the right to use the music. It deducts a commission from the transaction amount and distributes the remaining amount to the seller.

[0673] Specific actions:

[0674] The server processes the purchase request and grants the purchaser the right to use the music.

[0675] The transaction amount is reduced by a commission, and the remaining amount is distributed to the seller (Input: Purchase Request, Output: Transaction Completion Message).

[0676] (Application Example 1)

[0677] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0678] In today's world, there is a need for a system that allows users to easily create music, distribute it to other users, rate it, and sell it. However, current systems fragment the process from music creation to distribution, rating, and selling, resulting in a poor user experience. Furthermore, copyright management and monetization of created music are often handled separately, requiring users to go through cumbersome procedures. In response to this, a system that can handle everything from music creation to distribution, rating, and selling in an integrated manner is needed.

[0679] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0680] In this invention, the server includes an input means on which a user can input parameters such as the type of music, music genre, and theme; a generation means that generates music, lyrics, sheet music, vocals, and accompaniment based on the parameters input via the input means; a return means that returns the music data generated by the generation means to the user; a storage means that stores the music data; a management means that manages the copyright to the music data stored in the storage means; a trading means that buys and sells the copyright between users; a revenue means that makes the profit generated by the trading means into platform revenue; an evaluation means that evaluates the music data generated by the user and creates a ranking; and a distribution means that distributes the generated music data to other users. As a result, users can enjoy a consistent experience from music generation to distribution, evaluation, and buying and selling, and copyright management and monetization can be handled centrally.

[0681] An "input method" is an interface for users to input parameters such as the type of music, music genre, and theme.

[0682] The "generation means" is a function for generating music, lyrics, sheet music, vocals, and accompaniment based on parameters input via the aforementioned input means.

[0683] The "return means" is a function for returning the music data generated by the generation means to the user.

[0684] "Storage means" refers to storage means for storing the aforementioned music data.

[0685] "Management means" refers to functions for managing copyrights to stored music data.

[0686] "Means of buying and selling" refers to a function for buying and selling the aforementioned copyrights among users.

[0687] "Revenue-generating means" refers to a function that generates revenue for the platform from the profits generated by the aforementioned buying and selling means.

[0688] The "evaluation method" is an interface for users to evaluate generated music data and create rankings.

[0689] "Distribution method" refers to a function for distributing generated music data to other users.

[0690] This invention provides a system that allows users to easily create music, manage and sell its copyrights, and enable other users to rate and distribute that music. This system is implemented by the following means.

[0691] First, the user inputs song parameters (song type, music genre, theme, etc.) via an input device such as a smartphone. The entered parameters are then sent to the server.

[0692] The server receives these parameters and uses generation tools to generate music, lyrics, sheet music, vocals, and accompaniment. OpenAI's GPT-4 and music generation-specific models are used as generation tools. The generated music data is then sent back from the server to the user's terminal.

[0693] The user's device displays the returned music data in a playable state. The user can listen to the music and edit it as needed using the editing tools. The edited music data is saved via the storage tools.

[0694] Copyright for stored music data is managed on a per-user basis using management tools. Users can then set the selling price of their music and sell it to other users using the trading tools. When a sale is completed, the profits generated are processed as revenue for the platform through revenue-generating mechanisms.

[0695] Furthermore, the evaluation system allows users to rate music data generated by other users and create rankings. Users can then improve their music based on these ratings. Finally, highly-rated music and newly generated music are distributed to other users via the distribution system.

[0696] This system allows users to enjoy a consistent experience from music creation and editing to storage, distribution, and buying / selling. Furthermore, copyright management and monetization can be handled centrally. This is expected to contribute to the creation of a new music market.

[0697] For example, if a user wants to generate a jazz song with the theme "Summer Memories," it would look like this:

[0698] The user inputs the parameters "jazz" and "summer memories" into an input device. The server uses this information to generate a song using an AI model and sends the generated song data back to the user's terminal. The user listens to the song, edits it, saves it, and distributes it to other users.

[0699] Example of a prompt:

[0700] Song type: Pop

[0701] Music genre: Jazz

[0702] Theme: Summer Memories

[0703] In this way, the invention effectively meets user needs and provides a new musical experience.

[0704] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0705] Step 1:

[0706] The user launches the music generation application and accesses the input fields. The user enters the type of song (e.g., pop), music genre (e.g., jazz), and theme (e.g., summer memories). This prepares the user to send the parameters of the song they want to generate to the server.

[0707] input:

[0708] Type of music (e.g., pop)

[0709] Music genres (jazz, etc.)

[0710] Theme (e.g., summer memories)

[0711] output:

[0712] Music generation parameters (sent to the server in JSON format)

[0713] Step 2:

[0714] The server receives music generation parameters (JSON data) from the user. The server analyzes this data and sends a music generation request to the AI ​​model. The AI ​​model generates music, lyrics, sheet music, vocals, and accompaniment based on the specified parameters.

[0715] input:

[0716] Music generation parameters (JSON format)

[0717] Data processing:

[0718] Convert music generation parameters into prompt statements for the AI ​​model.

[0719] output:

[0720] Generated music data (music, lyrics, sheet music, vocals, accompaniment)

[0721] Step 3:

[0722] The server analyzes the music data received from the generating AI model and sends that data back to the user's device. The user's device receives the music data and displays it in a playable state. The user can listen to the generated music by clicking the play button.

[0723] input:

[0724] Generated music data

[0725] output:

[0726] Music data playable on the user's device

[0727] Step 4:

[0728] Users listen to music and edit the music data using editing tools as needed. The edited music data is temporarily saved on the device.

[0729] input:

[0730] Played music data

[0731] User-edited parameters

[0732] Data processing:

[0733] Update music data based on user-edited parameters.

[0734] output:

[0735] Edited music data

[0736] Step 5:

[0737] After the user finishes editing, they click the save button. The user's device sends the edited music data to the server, which then persistently stores the data using a storage method. Copyright management information is also added to the saved data.

[0738] input:

[0739] Edited music data

[0740] output:

[0741] Persistent music data

[0742] Step 6:

[0743] Users set a selling price for their saved music data and send a buy / sell request to the server. The server makes the music data available to other users using the buy / sell system and facilitates the transaction. Any profits generated are processed using the revenue system.

[0744] input:

[0745] Selling price setting

[0746] Buy / Sell Request

[0747] output:

[0748] Released song information

[0749] Revenue sharing information

[0750] Step 7:

[0751] Other users rate the generated songs and send their ratings to the server. The server uses the rating system to collect the rating data and create rankings. Highly rated songs are recommended to other users using the distribution system.

[0752] input:

[0753] Evaluation data from other users

[0754] output:

[0755] Rating Ranking

[0756] Recommended Song List

[0757] In this way, specific inputs and outputs, as well as data processing, are performed at each step, allowing users to obtain a consistent music generation and distribution experience.

[0758] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0759] This invention is a system for users to easily generate music and manage and sell its copyrights. This system not only has an input means for users to input parameters such as the type of music, music genre, and theme, but also incorporates an emotion engine that recognizes the user's emotions.

[0760] Steps for users to generate music

[0761] Users access the music generation screen from their own devices (PCs or smartphones). In addition to inputting parameters such as song type, music genre, and theme, the emotion engine recognizes the user's facial expressions and voice tone to analyze their current emotions.

[0762] The role of the emotional engine

[0763] The emotion engine analyzes data acquired through the user's camera and microphone to identify the user's emotional state (e.g., joy, sadness, surprise, anger). This emotional data is also used as a parameter for music generation. For example, if the emotion engine recognizes that the user is in a state of "joy," it will suggest a song with an upbeat tempo that matches that feeling of joy.

[0764] Server-based music generation

[0765] The server receives requests from users and results from the emotion engine, and requests the AI ​​model to generate music. The AI ​​model automatically generates music, lyrics, sheet music, vocals, and accompaniment based on the input parameters and the user's emotion data. The server then receives the generated music data.

[0766] Return and display of music data

[0767] The server sends the generated music data back to the user's device. The user's device displays the received music in a playable state. The user can listen to this music and edit it as needed.

[0768] Saving music data

[0769] If the user is satisfied with the generated music data, they click the save button to save the song. The server receives the save request and saves the music data to the database. This allows the user to access the song again at any time.

[0770] Copyright management and trading

[0771] The server provides tools for managing copyrights to stored music data on a per-user basis. Users enter pricing and sales conditions on the music sales page and send sales requests to the server. The server stores this information and makes it public in the copyright management system.

[0772] Transactions between users

[0773] Other users can view the published song information and select the songs they wish to purchase. Once a purchase request is sent to the server, the server processes the transaction and grants the buyer the right to use the song. Furthermore, the platform deducts its revenue from the transaction amount and distributes the remainder to the seller. This facilitates smooth transactions between users and makes it possible for individuals to commercially use and monetize their created music.

[0774] Specific usage examples

[0775] For example, suppose User A generates a jazz song with the theme "winter love," and the emotion engine recognizes User A's emotion as "joy." User A accesses the song generation screen and provides "jazz," "winter love," and the joy emotion data provided by the emotion engine as input. The server sends this information to the AI ​​model, which generates an appropriate song. This song is displayed on User A's device in a playable state. User A edits and saves the song, then sets a sales price and creates a copyright listing.

[0776] If another user B likes this song and wants to buy it, they click the purchase button on the sales page. The server processes the transaction, grants user B the right to use the song, and distributes the remaining amount to user A after deducting the platform's revenue from the transaction amount.

[0777] As described above, this system allows users to easily generate music and manage and sell its copyrights. The introduction of an emotion engine enables music generation tailored to the user's emotions, providing a more personalized musical experience. Furthermore, the system generates revenue as a platform, contributing to the creation of a new music market.

[0778] The following describes the processing flow.

[0779] Processing flow of a music generation system that combines an emotion engine

[0780] Step 1:

[0781] The user enters their email address, password, and profile information into the registration form on their device and clicks the registration button.

[0782] Step 2:

[0783] The server receives registration information sent by the user and stores it in the database.

[0784] Step 3:

[0785] The server sends an authentication email to the user. This email contains an authentication link.

[0786] Step 4:

[0787] The user opens the verification email and clicks the verification link.

[0788] Step 5:

[0789] The server receives the request from the authentication link and updates the user's authentication status.

[0790] Song generation request

[0791] Step 1:

[0792] The user enters parameters such as song type, genre, and theme on the device's music generation screen, and also turns on the device's camera and microphone so that the emotion engine can acquire the user's emotional data.

[0793] Step 2:

[0794] The emotion engine analyzes data acquired through the camera and microphone to identify the user's emotional state.

[0795] Step 3:

[0796] The server sends the request data received from the user and the results of the emotion engine to the AI ​​model.

[0797] Step 4:

[0798] The AI ​​model generates songs, lyrics, sheet music, vocals, and accompaniments based on parameters and user emotion data.

[0799] Step 5:

[0800] The server receives the generated results from the AI ​​model and sends them back to the user's device.

[0801] Step 6:

[0802] The device displays the generated music data to the user and provides it in a playable state.

[0803] Editing and saving music

[0804] Step 1:

[0805] The user edits the generated music data on the device's editing screen. For example, they can change the melody, revise the lyrics, and adjust the accompaniment.

[0806] Step 2:

[0807] The user clicks the save button to save the edited music data.

[0808] Step 3:

[0809] The server receives a save request from the user and saves the edited music data to the database.

[0810] Copyright establishment and sales

[0811] Step 1:

[0812] Users access the music sales screen from their devices and set the price and sales conditions for the songs.

[0813] Step 2:

[0814] The user submits a sales request to the server.

[0815] Step 3:

[0816] The server stores sales information in a database and publishes copyright listings.

[0817] Transactions between users

[0818] Step 1:

[0819] Other users select a song from the sales page and click the purchase button.

[0820] Step 2:

[0821] The device sends a purchase request to the server.

[0822] Step 3:

[0823] The server processes the purchase request and grants the purchaser the right to use the music.

[0824] Step 4:

[0825] The server deducts the platform's revenue from the transaction amount and distributes the remainder to the music sellers.

[0826] Specific usage examples

[0827] Step 1:

[0828] Let's say User A generates a jazz song with the theme "winter love," and the emotion engine recognizes User A's emotion as "joy." User A accesses the song generation screen and provides "jazz," "winter love," and the joy emotion data provided by the emotion engine as input.

[0829] Step 2:

[0830] The server sends this information to the AI ​​model, which then generates the appropriate song.

[0831] Step 3:

[0832] This song is displayed in a playable state on User A's device. User A edits and saves the song, then sets a selling price and creates a copyright listing.

[0833] Step 4:

[0834] If another user B likes this song and wants to buy it, they click the purchase button on the sales page.

[0835] Step 5:

[0836] The server processes the transaction, grants user B the right to use the music, and distributes the remainder (transaction amount minus platform revenue) to user A.

[0837] As described above, this system allows users to easily generate music and manage and sell its copyrights. The introduction of an emotion engine enables music generation tailored to the user's emotions, providing a more personalized musical experience. Furthermore, the system generates revenue as a platform, contributing to the creation of a new music market.

[0838] (Example 2)

[0839] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0840] Traditional music generation systems lacked sufficient functionality for users to easily generate music and manage and trade its copyrights. Furthermore, they were unable to generate music that reflected the user's emotions, making it difficult to provide a personalized experience. Additionally, the processes for saving, managing, and trading generated music were inconvenient, highlighting the need for a consistent user experience.

[0841] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0842] In this invention, the server includes an input means on which the user can input parameters such as the type of song, music genre, and theme; an emotion recognition means that recognizes the user's emotions and analyzes the emotion data; a generation means that generates songs, lyrics, sheet music, vocals, and accompaniments based on the parameters and emotion data input via the input means and the emotion recognition means; a return means that returns the song data generated by the generation means to the user; a storage means that stores the song data; a management means that manages the copyright to the song data stored in the storage means; a trading means that buys and sells the copyright between users; and a revenue means that makes the profit generated by the trading means into revenue for the platform. As a result, users can easily and efficiently generate songs, receive personalized songs tailored to their emotions, and enjoy a consistent process of storage, management, and trading.

[0843] An "input method" is a means by which the user inputs parameters such as the type of song, music genre, and theme.

[0844] An "emotion recognition tool" is a means of recognizing a user's emotions and analyzing that emotional data.

[0845] "Generation means" refers to means for generating music, lyrics, sheet music, vocals, and accompaniment based on parameters and emotion data input via input means and emotion recognition means.

[0846] "Return means" refers to the means of returning the music data generated by the generation means to the user.

[0847] "Storage method" refers to the means of saving the generated music data.

[0848] "Management means" refers to means for managing copyrights to music data stored in storage means.

[0849] "Means of buying and selling" refers to the means by which users buy and sell copyrights among themselves.

[0850] "Revenue-generating means" refers to the means by which the platform generates revenue from the profits produced through buying and selling.

[0851] This invention is a system that allows users to easily generate music and manage and sell its copyrights. The system incorporates an emotion recognition engine to enable music generation that reflects the user's emotions. The system consists of various components, including the user's terminal, a data processing server, the emotion recognition engine, and a generation AI model used for music generation. Specific hardware used includes personal computers and smartphones. Software used includes a web browser and an AI model (e.g., OpenAI's GPT-3 or Google's MusicLM).

[0852] User-inputted parameters

[0853] Users access the music generation system's webpage using a browser on their PC or smartphone. There, they input the type of song, music genre, theme, etc. They also grant permission for the camera and microphone to be used for AI-powered emotion recognition. At that point, a form appears on the screen for entering specific parameters such as "jazz" or "winter love."

[0854] emotion recognition

[0855] When a user allows the use of the camera and microphone, the device sends video and audio to an emotion recognition engine. The engine analyzes facial expressions and tone of voice to identify the user's current emotions. For example, the engine outputs emotion data such as "joy," "sadness," or "surprise."

[0856] Data processing and music generation

[0857] The user's input parameters and emotion data obtained from the emotion recognition engine are sent to the server in a single batch. The server processes this data and sends a request for music generation to the generative AI model. The generative AI model generates music, lyrics, sheet music, vocals, accompaniment, etc., based on these inputs. In doing so, the AI ​​model generates music with characteristics that match the user's emotions. For example, if the analysis results indicate the emotion of "joy," a bright and upbeat song will be generated.

[0858] Return and display of music data

[0859] The music data generated by the AI ​​model is sent back to the server, from which it is transmitted to the user's device. The user's device displays the received data in a playable format, allowing the user to listen to the generated music. If the user is satisfied with the music, clicking the "Save" button sends the request to the server, which then saves the music data to its database.

[0860] Copyright management and buying and selling of music

[0861] The server manages the copyrights to the stored music data and allows users to set the sales conditions for the music. If another user wishes to purchase a song, a purchase request is sent to the server, and the server processes the transaction. The remaining amount after deducting the platform's revenue from the transaction amount is distributed to the user who created the music.

[0862] Specific example

[0863] For example, suppose user A inputs parameters such as "jazz" and "winter love," and the emotion recognition engine detects the emotion "joy." The server sends this data to an AI model, which generates a cheerful jazz song tailored to user A's parameters. This song is displayed on user A's device in a playable format, and user A can listen to, edit, and save the song.

[0864] Example of a prompt

[0865] "Please generate a jazz song. The theme is 'winter love,' and the emotion is 'joy.'"

[0866] As described above, this system allows users to easily generate music and seamlessly manage and sell its copyrights. Furthermore, the introduction of emotion recognition enables the generation of personalized music that responds to the user's emotions.

[0867] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0868] Step 1:

[0869] The user accesses the music generation screen.

[0870] Input: Launch a browser on the user's device and access the music generation system's webpage.

[0871] Specific steps: The user uses a browser on their computer or smartphone to enter a specified web address and open the music generation system screen. They then log in following the interface displayed there.

[0872] Step 2:

[0873] The user enters the parameters of the song.

[0874] Input: Parameters such as song type (e.g., ballad), music genre (e.g., jazz), and theme (e.g., "Winter Love").

[0875] Specific operation: The user enters the required information into the input form on the screen. After entering the parameters, they click the "Complete" button.

[0876] Output: The input parameters are saved on the device and sent to the server as data to proceed with the emotion recognition process.

[0877] Step 3:

[0878] The device activates the camera and microphone.

[0879] Input: Request for use of camera and microphone with user permission.

[0880] Specific action: The user grants permission to use the camera and microphone through a dialog box displayed on the screen.

[0881] Output: Once permission is obtained, the camera and microphone will activate and begin collecting data on the user's facial expressions and voice.

[0882] Step 4:

[0883] The emotion recognition engine analyzes the user's emotions.

[0884] Input: Facial expression data and audio data acquired from the camera and microphone.

[0885] Specific operation: The emotion recognition engine analyzes data in real time to identify the user's emotional state (e.g., joy, sadness, surprise, etc.).

[0886] Output: The emotional data obtained as an analysis result is sent to the server.

[0887] Step 5:

[0888] The server receives user input and sentiment data.

[0889] Input: Parameters entered by the user and emotion data sent from the emotion recognition engine.

[0890] Specific operation: The server receives this data and integrates it as a series of data.

[0891] Output: The integrated data is prepared as input for the AI ​​model.

[0892] Step 6:

[0893] The server sends a request to the AI ​​model.

[0894] Input: Integrated song parameters and emotion data.

[0895] Specific operation: The server creates a specific prompt message for the generated AI model and sends a request for music generation.

[0896] Output: Prompt message sent to the AI ​​model: "Generate a jazz song. The theme is 'winter love,' and the emotion is 'joy.'"

[0897] Step 7:

[0898] Music generation using AI models

[0899] Input: The prompt message sent from the server.

[0900] Specific operation: The AI ​​model generates music based on prompt text. It generates data in various forms, including lyrics, sheet music, vocals, and accompaniment.

[0901] Output: The generated music data is sent back to the server.

[0902] Step 8:

[0903] The server sends the generated music data to the user's terminal.

[0904] Input: Generated music data returned from the AI ​​model.

[0905] Specific operation: The server receives the generated data and sends it to the user's terminal.

[0906] Output: Music data sent to the user's device.

[0907] Step 9:

[0908] The device displays music data.

[0909] Input: Music data sent from the server.

[0910] Specific operation: The device converts the data into a playable format and displays it to the user visually and audibly.

[0911] Output: Music displayed in a playable format.

[0912] Step 10:

[0913] Users edit songs

[0914] Input: Songs displayed in playable formats.

[0915] Specific actions: The user edits the music using the music editing tools as needed. Once the modifications are complete, they submit another save request.

[0916] Output: The edited music data is saved to the device.

[0917] Step 11:

[0918] The device sends a save request to the server.

[0919] Input: Edited song data.

[0920] Specific action: The user clicks the save button, and a save request is sent to the server.

[0921] Output: The server receives the music data.

[0922] Step 12:

[0923] The server saves the music data to the database.

[0924] Input: Music data for which a save request was received.

[0925] Specific operation: The server saves the music data to a database, allowing users to access it again.

[0926] Output: Music data stored in the database.

[0927] Step 13:

[0928] The server manages the copyright of the music.

[0929] Input: Saved music data.

[0930] Specific operation: The server manages copyright information for each song and assigns copyright information to each song.

[0931] Output: Managed copyright information.

[0932] Step 14:

[0933] The user enters the music sales conditions.

[0934] Input: Sales conditions (price, terms of use, etc.) entered on the music sales page.

[0935] Specific operation: The user enters the sales price and terms of use, and sends a sales request to the server.

[0936] Output: Sales conditions data from users.

[0937] Step 15:

[0938] The server processes the transaction.

[0939] Input: Sales conditions data from users and purchase requests from other users.

[0940] Specific operation: The server processes the transaction and grants the buyer the right to use the music. Then, it distributes the remaining amount to the seller after deducting the platform's revenue from the transaction amount.

[0941] Output: Transaction information and distributed profits.

[0942] (Application Example 2)

[0943] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0944] Traditional music generation systems typically generate music based on user-inputted parameters such as song type, genre, and theme, with little consideration given to the user's emotional state. Furthermore, copyright management and trading of generated music were complex, making platform monetization difficult. Additionally, there was a lack of easy ways for users to save, manage, and commercially utilize the music they generated themselves.

[0945] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0946] In this invention, the server includes an input means on which the user can input parameters such as the type of song, music genre, and theme; an emotion recognition means that recognizes the user's emotional state using an emotion engine and reflects the parameters based on that emotion in song generation; and a generation means that generates songs, lyrics, sheet music, vocals, and accompaniment based on the parameters input via the input means and the emotion recognition means. This enables the automatic generation of personalized songs that reflect the user's emotional state, and allows for efficient monetization through copyright management and sales.

[0947] "Input method" refers to a device or interface for users to input parameters such as the type of music, music genre, and theme.

[0948] "Generation means" refers to a system or algorithm that automatically generates music, lyrics, sheet music, vocals, and accompaniment based on parameters input via the input means and emotion recognition means.

[0949] "Emotion recognition means" refers to technology that uses an emotion engine to recognize the user's emotional state and reflects parameters based on that emotion in music generation.

[0950] "Return means" refers to the communication infrastructure and software used to return the music data generated by the generation means to the user.

[0951] "Storage means" refers to a storage device or database system for storing the music data generated by the generation means.

[0952] "Management means" refers to tools or software for managing the copyright of music data stored in the aforementioned storage means.

[0953] "Means of buying and selling" refers to platforms or trading systems for buying and selling the aforementioned copyrights among users.

[0954] "Revenue-generating means" refers to a mechanism or management system for generating revenue for the platform from the profits generated by the aforementioned buying and selling means.

[0955] "Editing means" refers to software or interfaces that allow users to edit generated music data.

[0956] "Authentication means" refers to the systems and protocols used for user registration and authentication.

[0957] A "server" is a computer or network system that runs a music generation system and processes requests from users.

[0958] System Overview

[0959] This system allows users to input parameters such as song type, music genre, and theme, collect emotional data using emotion recognition means, and generate and manage songs based on that data. Specific components include input means, generation means, emotion recognition means, return means, storage means, management means, buying and selling means, and revenue means.

[0960] Program Processing Description

[0961] 1. User interface (input means):

[0962] Users access the music generation screen through a smartphone application. They input parameters such as the type of song, music genre, and theme. For example, they can select options such as "rock" or "summer adventure."

[0963] 2. Emotion recognition means:

[0964] The system analyzes facial images and audio data acquired through the user's camera and microphone. Emotion recognition, in particular, utilizes the "EmotionRecognizer" module to identify the user's emotional state (joy, sadness, surprise, anger, etc.). For example, if the user smiles at the camera, the system determines that they are in a state of "joy."

[0965] 3. Music generation (generation methods):

[0966] The server receives data from the input means and emotion recognition means and generates music using the "MusicGenerator" module. During generation, the original parameters and emotion data are reflected. For example, if rock is selected and the user's emotional state is "joy," an upbeat song with positive elements will be generated.

[0967] 4. Return of generated music (return method):

[0968] The server sends the generated music data back to the user's smartphone. The user can then play and check the received music within the app.

[0969] 5. Preservation means:

[0970] If the user is satisfied with the generated music data, they can click the save button to send a save request to the server. The server receives this request and saves the music data to its database.

[0971] 6. Music management and trading (management methods and trading methods):

[0972] The server manages the copyright of the stored music data. Users enter pricing and sales conditions on the music sales page and submit sales requests. Other users can view and purchase the music published on the sales page.

[0973] 7. Monetization (means of revenue):

[0974] A system is in place where the platform's revenue is deducted from the profits generated by the buying and selling methods, and the remainder is distributed to the sellers.

[0975] Specific example

[0976] For example, suppose a user wants to generate a song with the themes of "rock" and "summer adventure." If the emotion recognition system detects "joy" from the user's facial expression, a song will be generated based on "rock," "summer adventure," and "joy."

[0977] Examples of prompts for generative AI models

[0978] The user wants to generate a song with the themes of "rock" and "summer adventure." The emotion engine detected "joy" from the user's facial expression. Please generate a song based on this.

[0979] This allows users to easily create, save, manage, and sell personalized music. Furthermore, it is expected that generating music based on emotional data will further enhance user satisfaction.

[0980] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0981] Step 1:

[0982] The user accesses the music generation screen through a smartphone application. Here, the user inputs parameters such as the type of music, music genre, and theme (input: user input, output: parameter data). This data is necessary for subsequent processing and serves to specify the user's wishes.

[0983] Step 2:

[0984] The user's facial expressions and audio data are collected through the smartphone's camera and microphone (input: camera video, audio data; output: emotion data). This data is analyzed by an emotion recognition system to identify the user's emotional state (e.g., joy, sadness, surprise, anger). The "EmotionRecognizer" module is used for this analysis.

[0985] Step 3:

[0986] The terminal sends the collected emotion data and the parameter data of the initially entered song to the server (input: emotion data, parameter data; output: integrated data). The server receives this data, integrates it, and creates a request for song generation.

[0987] Step 4:

[0988] The server uses the "MusicGenerator" module to generate music, lyrics, sheet music, vocals, and accompaniment based on integrated data (input: integrated data, output: generated music data). The data processing performed here involves generating the optimal music by considering emotional data, song type, music genre, and theme.

[0989] Step 5:

[0990] The generated music data is sent back from the server to the user's smartphone (input: generated music data, output: data sent to the user's device). The user's smartphone receives this data and displays it on the music playback screen.

[0991] Step 6:

[0992] When a user plays and reviews a generated song and wishes to save it, they click the save button to send a save request to the server (input: user's save request, output: song save confirmation). The server receives this request and saves the song data to the database.

[0993] Step 7:

[0994] A management system operates to manage copyrights for saved music data (input: saved music data, output: copyright management data). Users can enter pricing and sales conditions on the music sales page and submit sales requests.

[0995] Step 8:

[0996] When another user purchases a publicly available song, the buying and selling mechanism is activated and the transaction takes place (input: purchase request, output: song usage rights). The server processes the transaction, deducts the platform's revenue from the generated profit, and distributes it to the seller (input: transaction data, output: revenue data).

[0997] Step 9:

[0998] The platform calculates revenue from the profits generated through buying and selling and distributes it to users (input: revenue data, output: revenue distribution data). This enables smooth transactions between users and allows individuals to commercially use and monetize music they have created.

[0999] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1000] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1001] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[1002] [Third Embodiment]

[1003] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[1004] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1005] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1006] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[1007] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1008] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1009] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1010] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1011] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1012] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1013] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1014] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[1015] This invention is a system for users to easily generate music and manage and buy / sell its copyrights. This system includes an input means for users to input parameters such as the type of music, music genre, and theme.

[1016] Steps for users to generate music

[1017] Users access the music generation screen from their own devices (PCs or smartphones). Here, they input parameters such as the type of song, music genre, and theme. For example, specific inputs such as "Pop" and "Summer Memories" are possible.

[1018] Server-based music generation

[1019] The server receives a request from the user and instructs the AI ​​model to generate music. Based on the input parameters, the AI ​​model automatically generates the music, lyrics, sheet music, vocals, and accompaniment. The server then receives the generated music data.

[1020] Return and display of music data

[1021] The server sends the generated music data back to the user's device. The user's device displays the received music in a playable state. The user can listen to this music and edit it as needed.

[1022] Saving music data

[1023] If the user is satisfied with the generated music data, they click the save button to save the song. The server receives the save request and saves the music data to the database. This allows the user to access the song again at any time.

[1024] Copyright management and trading

[1025] The server provides tools for managing copyrights to stored music data on a per-user basis. Users enter pricing and sales conditions on the music sales page and send sales requests to the server. The server stores this information and makes it public in the copyright management system.

[1026] Transactions between users

[1027] Other users can view the published song information and select the songs they wish to purchase. Once a purchase request is sent to the server, the server processes the transaction and grants the buyer the right to use the song. Furthermore, the platform deducts its revenue from the transaction amount and distributes the remainder to the seller. This facilitates smooth transactions between users and makes it possible for individuals to commercially use and monetize their created music.

[1028] Specific usage examples

[1029] For example, suppose User A wants to generate a jazz song with the theme "winter love." User A accesses the song generation screen and enters "jazz" and "winter love." The server sends this information to the AI ​​model, which generates an appropriate song. This song is displayed on User A's device and is playable. User A edits and saves the song, then sets a sales price and creates a copyright listing.

[1030] If another user B likes this song and wants to buy it, they click the purchase button on the sales page. The server processes the transaction, grants user B the right to use the song, and distributes the remaining amount to user A after deducting the platform's revenue from the transaction amount.

[1031] As described above, this system allows users to easily create music and manage and buy / sell its copyrights. Furthermore, since the system can generate revenue as a platform, it contributes to the creation of a new music market.

[1032] The following describes the processing flow.

[1033] User registration and authentication

[1034] Step 1:

[1035] The user enters their email address, password, and profile information into the registration form on their device and clicks the registration button.

[1036] Step 2:

[1037] The server receives registration information sent by the user and stores it in the database.

[1038] Step 3:

[1039] The server sends an authentication email to the user. This email contains an authentication link.

[1040] Step 4:

[1041] The user opens the verification email and clicks the verification link.

[1042] Step 5:

[1043] The server receives the request from the authentication link and updates the user's authentication status.

[1044] Song generation request

[1045] Step 1:

[1046] The user enters parameters such as the song type, genre, and theme on the device's music generation screen and clicks the generate button.

[1047] Step 2:

[1048] The terminal sends the entered parameters to the server.

[1049] Step 3:

[1050] The server sends the request data received from the user to the AI ​​model.

[1051] Step 4:

[1052] The AI ​​model generates music, lyrics, sheet music, vocals, and accompaniment based on the parameters.

[1053] Step 5:

[1054] The server receives the generated results from the AI ​​model and sends them back to the user's device.

[1055] Step 6:

[1056] The device displays the generated music data to the user.

[1057] Editing and saving music

[1058] Step 1:

[1059] The user edits the generated music data on the device's editing screen. For example, they can change the melody, revise the lyrics, and adjust the accompaniment.

[1060] Step 2:

[1061] The user clicks the save button to save the edited music data.

[1062] Step 3:

[1063] The server receives a save request from the user and saves the edited music data to the database.

[1064] Copyright establishment and sales

[1065] Step 1:

[1066] Users access the music sales screen from their devices and set the price and sales conditions for the songs.

[1067] Step 2:

[1068] The user submits a sales request to the server.

[1069] Step 3:

[1070] The server stores sales information in a database and publishes copyright listings.

[1071] Transactions between users

[1072] Step 1:

[1073] Other users select a song from the sales page and click the purchase button.

[1074] Step 2:

[1075] The device sends a purchase request to the server.

[1076] Step 3:

[1077] The server processes the purchase request and grants the purchaser the right to use the music.

[1078] Step 4:

[1079] The server deducts the platform's revenue from the transaction amount and distributes the remainder to the music sellers.

[1080] The above outlines the specific processing steps in the patent system.

[1081] (Example 1)

[1082] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1083] While systems exist today for easily generating music and managing and trading its copyrights, they do not fully meet user needs for smooth music generation, management, and trading procedures. Furthermore, improving the efficiency of revenue sharing associated with the editing and trading of generated music remains a challenge. This invention aims to solve these problems and provide an efficient music generation, management, and trading system that is easily accessible to users.

[1084] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1085] In this invention, the server includes an input means on which a user can input parameters such as the type of music, music genre, and theme; a generation means that generates music, lyrics, sheet music, vocals, and accompaniment based on the parameters input via the input means; a return means that sends the music data generated by the generation means back to the user; a storage means that stores the music data; a management means that manages the copyright to the music data stored in the storage means; a trading means that buys and sells the copyright between users; a revenue means that makes the profit generated by the trading means into platform revenue; a transaction means that checks the published music information of other users and processes purchase requests when they wish to purchase it; and a distribution means that deducts a commission from the transaction amount and distributes the remaining amount to the seller. This makes it possible for a user to smoothly perform a series of operations, from inputting parameters to generate music, and managing, editing, and buying and selling that music.

[1086] An "input method" is an interface for users to input parameters such as the type of music, music genre, and theme.

[1087] The "generation means" is a system for generating music, lyrics, sheet music, vocals, and accompaniment based on parameters input via the input means.

[1088] The "return means" is a mechanism for sending the music data generated by the generation means to the user.

[1089] A "storage method" refers to a system for saving generated music data to a database or storage device.

[1090] "Management means" refers to a tool for managing the copyright of music data stored in the aforementioned storage means.

[1091] A "trading method" refers to a system for users to buy and sell copyrighted music that has been preserved.

[1092] "Revenue-generating means" refers to a mechanism for generating revenue for the platform from the profits generated by the aforementioned buying and selling means.

[1093] A "transaction method" is a system for other users to view publicly available song information and process purchase requests if they wish to buy it.

[1094] A "distribution mechanism" is a system for distributing the remaining amount to the seller after deducting fees from the transaction amount.

[1095] "Editing means" refers to an interface that allows the user to edit the aforementioned music data.

[1096] "Authentication means" refers to a system used for user registration and authentication.

[1097] This invention relates to a system for users to easily generate music and manage and buy / sell its copyrights. The system includes an input means for users to input parameters such as the type of music, music genre, and theme; a generation means for generating music, lyrics, sheet music, vocals, and accompaniment based on the input parameters; a return means for sending the generated music data back to the user; a storage means for storing the music data; a management means for managing the copyrights to the stored music data; a buying and selling means for users to buy and sell copyrights; a revenue means for making the profits generated by the buying and selling means into platform revenue; a transaction means for other users to check the published music information and process purchase requests if they wish to purchase it; a distribution means for deducting a commission from the transaction amount and distributing the remaining amount to the seller; and an editing means for editing music data and an authentication means for user registration and authentication.

[1098] Hardware and software configuration

[1099] This system consists of user terminals (PCs, smartphones) and a server based on a high-performance computer. Users can access the server from their terminals via an internet browser or a dedicated application.

[1100] For the AI ​​model that functions as a generation tool, for example, OpenAI's GPT-3 or a neural network model specifically designed for music generation can be used. For storage and management, cloud-based database systems (e.g., Amazon Web Services, Google Cloud Platform) can be applied.

[1101] System operation

[1102] 1. When a user submits a song generation request

[1103] The user accesses the music generation screen from their device, enters parameters such as "Genre: Pop" and "Theme: Summer Memories," and presses the generate button. The following is an example of the prompt text.

[1104] Genre: Pop

[1105] Theme: Summer Memories

[1106] 2. When the server receives a music generation request and sends it to the AI ​​model

[1107] The server receives the user's request, analyzes the input parameters, and sends them to the AI ​​model. Based on these parameters, the AI ​​model generates music, lyrics, sheet music, vocals, and accompaniment.

[1108] 3. When the generated music data is returned to the user and saved.

[1109] The AI ​​model generates music data, which is received by the server and sent back to the user's device. The user plays the music on their device and edits it as needed. If the user is satisfied with the music, they click the save button to save the music data to the database.

[1110] 4. When managing and buying / selling copyrights of musical works

[1111] Copyright for saved music data is managed on the server, and users can set prices and sales conditions on the sales page. Based on the information set by the user, other users can view the music information and submit purchase requests. The server processes the transaction and grants the buyer the right to use the music. A commission is deducted from the transaction amount, and the remainder is distributed to the seller.

[1112] Thus, the system of the present invention allows users to easily generate music and smoothly manage and buy / sell that music. Furthermore, the system can increase platform revenue and contribute to the creation of a new music market.

[1113] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1114] Step 1: User submits a song generation request

[1115] The user accesses the music generation screen from their device, enters parameters such as "Genre: Pop" and "Theme: Summer Memories," and presses the generate button. The input data (song type, music genre, theme, etc.) is sent to the server.

[1116] Specific actions:

[1117] The user opens a web browser and accesses the music generation page.

[1118] The user enters detailed information about the song into the form.

[1119] The user clicks the "Generate" button, and the input data is sent to the server (Input: song parameters, Output: HTTP request to the server).

[1120] Step 2: The server receives the music generation request and sends it to the AI ​​model.

[1121] The server receives a request from the user, analyzes the input parameters, and sends them to the generated AI model. The AI ​​model receives a prompt message along with the analysis results.

[1122] Specific actions:

[1123] The server receives an HTTP request and parses the input data.

[1124] The server calls the API of the generated AI model and passes the analysis results as prompt messages (input: user request, output: prompt messages).

[1125] Step 3: Music generation using an AI model

[1126] The AI ​​model generates music, lyrics, sheet music, vocals, and accompaniment based on the received prompt text. This generation process is performed using an advanced neural network. The generated music data is saved as a temporary file.

[1127] Specific actions:

[1128] The AI ​​model executes a music generation algorithm based on the prompt text.

[1129] The generated music data is saved as a temporary file (input: prompt text, output: music data).

[1130] Step 4: Receiving the generated music data from the server and returning it to the user terminal.

[1131] The server receives the music data generated by the AI ​​model and sends it back to the user's device. The music data is sent to the user as an HTTP response.

[1132] Specific actions:

[1133] The server receives music data generated from the AI ​​model.

[1134] The server creates a response to send music data to the user's terminal (input: music data, output: HTTP response to the user).

[1135] Step 5: User plays and edits music.

[1136] The user's terminal receives the generated music data and displays it in a playable state. The user can listen to the music and edit it as needed. After editing, they click the save button to send the edits to the server.

[1137] Specific actions:

[1138] The user opens a music playback application or browser.

[1139] Play and edit received music files.

[1140] The user's edits are sent to the server (input: edited music data, output: save request to the server).

[1141] Step 6: User submits a request to save music.

[1142] If the user is satisfied with the generated music data, they click the save button, and a save request is sent to the server. The music data is then sent back to the server.

[1143] Specific actions:

[1144] The user clicks the "Save" button.

[1145] The save request and music data are sent to the server (input: music data, output: HTTP request to the server).

[1146] Step 7: The server receives the music save request and saves it to the database.

[1147] The server receives the save request and saves the music data to the database. A message confirming the save is sent back to the user.

[1148] Specific actions:

[1149] The server receives the save request and connects to the database.

[1150] This process saves music data to a database and sends a save completion notification to the user (Input: Music data, Output: Save completion message).

[1151] Step 8: Server-based copyright management and publication of sales information for music.

[1152] The server provides tools for managing copyrights to stored music data, and users set prices and conditions on the sales page. The set sales information is made public.

[1153] Specific actions:

[1154] The server registers the music data with the copyright management system.

[1155] Users enter sales information and publish it (input: sales information, output: published sales page).

[1156] Step 9: Another user submits a request to purchase music.

[1157] Other users view the published song information and click the purchase button. A purchase request is sent to the server.

[1158] Specific actions:

[1159] Those interested in purchasing the song access the public page to view it.

[1160] Click the purchase button to send a purchase request to the server (Input: Purchase request, Output: HTTP request to the server).

[1161] Step 10: Server receives music purchase request and processes transaction.

[1162] The server receives the purchase request and grants the buyer the right to use the music. It deducts a commission from the transaction amount and distributes the remaining amount to the seller.

[1163] Specific actions:

[1164] The server processes the purchase request and grants the purchaser the right to use the music.

[1165] The transaction amount is reduced by a commission, and the remaining amount is distributed to the seller (Input: Purchase Request, Output: Transaction Completion Message).

[1166] (Application Example 1)

[1167] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1168] In today's world, there is a need for a system that allows users to easily create music, distribute it to other users, rate it, and sell it. However, current systems fragment the process from music creation to distribution, rating, and selling, resulting in a poor user experience. Furthermore, copyright management and monetization of created music are often handled separately, requiring users to go through cumbersome procedures. In response to this, a system that can handle everything from music creation to distribution, rating, and selling in an integrated manner is needed.

[1169] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1170] In this invention, the server includes an input means on which a user can input parameters such as the type of music, music genre, and theme; a generation means that generates music, lyrics, sheet music, vocals, and accompaniment based on the parameters input via the input means; a return means that returns the music data generated by the generation means to the user; a storage means that stores the music data; a management means that manages the copyright to the music data stored in the storage means; a trading means that buys and sells the copyright between users; a revenue means that makes the profit generated by the trading means into platform revenue; an evaluation means that evaluates the music data generated by the user and creates a ranking; and a distribution means that distributes the generated music data to other users. As a result, users can enjoy a consistent experience from music generation to distribution, evaluation, and buying and selling, and copyright management and monetization can be handled centrally.

[1171] An "input method" is an interface for users to input parameters such as the type of music, music genre, and theme.

[1172] The "generation means" is a function for generating music, lyrics, sheet music, vocals, and accompaniment based on parameters input via the aforementioned input means.

[1173] The "return means" is a function for returning the music data generated by the generation means to the user.

[1174] "Storage means" refers to storage means for storing the aforementioned music data.

[1175] "Management means" refers to functions for managing copyrights to stored music data.

[1176] "Means of buying and selling" refers to a function for buying and selling the aforementioned copyrights among users.

[1177] "Revenue-generating means" refers to a function that generates revenue for the platform from the profits generated by the aforementioned buying and selling means.

[1178] The "evaluation method" is an interface for users to evaluate generated music data and create rankings.

[1179] "Distribution method" refers to a function for distributing generated music data to other users.

[1180] This invention provides a system that allows users to easily create music, manage and sell its copyrights, and enable other users to rate and distribute that music. This system is implemented by the following means.

[1181] First, the user inputs song parameters (song type, music genre, theme, etc.) via an input device such as a smartphone. The entered parameters are then sent to the server.

[1182] The server receives these parameters and uses generation tools to generate music, lyrics, sheet music, vocals, and accompaniment. OpenAI's GPT-4 and music generation-specific models are used as generation tools. The generated music data is then sent back from the server to the user's terminal.

[1183] The user's device displays the returned music data in a playable state. The user can listen to the music and edit it as needed using the editing tools. The edited music data is saved via the storage tools.

[1184] Copyright for stored music data is managed on a per-user basis using management tools. Users can then set the selling price of their music and sell it to other users using the trading tools. When a sale is completed, the profits generated are processed as revenue for the platform through revenue-generating mechanisms.

[1185] Furthermore, the evaluation system allows users to rate music data generated by other users and create rankings. Users can then improve their music based on these ratings. Finally, highly-rated music and newly generated music are distributed to other users via the distribution system.

[1186] This system allows users to enjoy a consistent experience from music creation and editing to storage, distribution, and buying / selling. Furthermore, copyright management and monetization can be handled centrally. This is expected to contribute to the creation of a new music market.

[1187] For example, if a user wants to generate a jazz song with the theme "Summer Memories," it would look like this:

[1188] The user inputs the parameters "jazz" and "summer memories" into an input device. The server uses this information to generate a song using an AI model and sends the generated song data back to the user's terminal. The user listens to the song, edits it, saves it, and distributes it to other users.

[1189] Example of a prompt:

[1190] Song type: Pop

[1191] Music genre: Jazz

[1192] Theme: Summer Memories

[1193] In this way, the invention effectively meets user needs and provides a new musical experience.

[1194] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1195] Step 1:

[1196] The user launches the music generation application and accesses the input fields. The user enters the type of song (e.g., pop), music genre (e.g., jazz), and theme (e.g., summer memories). This prepares the user to send the parameters of the song they want to generate to the server.

[1197] input:

[1198] Type of music (e.g., pop)

[1199] Music genres (jazz, etc.)

[1200] Theme (e.g., summer memories)

[1201] output:

[1202] Music generation parameters (sent to the server in JSON format)

[1203] Step 2:

[1204] The server receives music generation parameters (JSON data) from the user. The server analyzes this data and sends a music generation request to the AI ​​model. The AI ​​model generates music, lyrics, sheet music, vocals, and accompaniment based on the specified parameters.

[1205] input:

[1206] Music generation parameters (JSON format)

[1207] Data processing:

[1208] Convert music generation parameters into prompt statements for the AI ​​model.

[1209] output:

[1210] Generated music data (music, lyrics, sheet music, vocals, accompaniment)

[1211] Step 3:

[1212] The server analyzes the music data received from the generating AI model and sends that data back to the user's device. The user's device receives the music data and displays it in a playable state. The user can listen to the generated music by clicking the play button.

[1213] input:

[1214] Generated music data

[1215] output:

[1216] Music data playable on the user's device

[1217] Step 4:

[1218] Users listen to music and edit the music data using editing tools as needed. The edited music data is temporarily saved on the device.

[1219] input:

[1220] Played music data

[1221] User-edited parameters

[1222] Data processing:

[1223] Update music data based on user-edited parameters.

[1224] output:

[1225] Edited music data

[1226] Step 5:

[1227] After the user finishes editing, they click the save button. The user's device sends the edited music data to the server, which then persistently stores the data using a storage method. Copyright management information is also added to the saved data.

[1228] input:

[1229] Edited music data

[1230] output:

[1231] Persistent music data

[1232] Step 6:

[1233] Users set a selling price for their saved music data and send a buy / sell request to the server. The server makes the music data available to other users using the buy / sell system and facilitates the transaction. Any profits generated are processed using the revenue system.

[1234] input:

[1235] Selling price setting

[1236] Buy / Sell Request

[1237] output:

[1238] Released song information

[1239] Revenue sharing information

[1240] Step 7:

[1241] Other users rate the generated songs and send their ratings to the server. The server uses the rating system to collect the rating data and create rankings. Highly rated songs are recommended to other users using the distribution system.

[1242] input:

[1243] Evaluation data from other users

[1244] output:

[1245] Rating Ranking

[1246] Recommended Song List

[1247] In this way, specific inputs and outputs, as well as data processing, are performed at each step, allowing users to obtain a consistent music generation and distribution experience.

[1248] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1249] This invention is a system for users to easily generate music and manage and sell its copyrights. This system not only has an input means for users to input parameters such as the type of music, music genre, and theme, but also incorporates an emotion engine that recognizes the user's emotions.

[1250] Steps for users to generate music

[1251] Users access the music generation screen from their own devices (PCs or smartphones). In addition to inputting parameters such as song type, music genre, and theme, the emotion engine recognizes the user's facial expressions and voice tone to analyze their current emotions.

[1252] The role of the emotional engine

[1253] The emotion engine analyzes data acquired through the user's camera and microphone to identify the user's emotional state (e.g., joy, sadness, surprise, anger). This emotional data is also used as a parameter for music generation. For example, if the emotion engine recognizes that the user is in a state of "joy," it will suggest a song with an upbeat tempo that matches that feeling of joy.

[1254] Server-based music generation

[1255] The server receives requests from users and results from the emotion engine, and requests the AI ​​model to generate music. The AI ​​model automatically generates music, lyrics, sheet music, vocals, and accompaniment based on the input parameters and the user's emotion data. The server then receives the generated music data.

[1256] Return and display of music data

[1257] The server sends the generated music data back to the user's device. The user's device displays the received music in a playable state. The user can listen to this music and edit it as needed.

[1258] Saving music data

[1259] If the user is satisfied with the generated music data, they click the save button to save the song. The server receives the save request and saves the music data to the database. This allows the user to access the song again at any time.

[1260] Copyright management and trading

[1261] The server provides tools for managing copyrights to stored music data on a per-user basis. Users enter pricing and sales conditions on the music sales page and send sales requests to the server. The server stores this information and makes it public in the copyright management system.

[1262] Transactions between users

[1263] Other users can view the published song information and select the songs they wish to purchase. Once a purchase request is sent to the server, the server processes the transaction and grants the buyer the right to use the song. Furthermore, the platform deducts its revenue from the transaction amount and distributes the remainder to the seller. This facilitates smooth transactions between users and makes it possible for individuals to commercially use and monetize their created music.

[1264] Specific usage examples

[1265] For example, suppose User A generates a jazz song with the theme "winter love," and the emotion engine recognizes User A's emotion as "joy." User A accesses the song generation screen and provides "jazz," "winter love," and the joy emotion data provided by the emotion engine as input. The server sends this information to the AI ​​model, which generates an appropriate song. This song is displayed on User A's device in a playable state. User A edits and saves the song, then sets a sales price and creates a copyright listing.

[1266] If another user B likes this song and wants to buy it, they click the purchase button on the sales page. The server processes the transaction, grants user B the right to use the song, and distributes the remaining amount to user A after deducting the platform's revenue from the transaction amount.

[1267] As described above, this system allows users to easily generate music and manage and sell its copyrights. The introduction of an emotion engine enables music generation tailored to the user's emotions, providing a more personalized musical experience. Furthermore, the system generates revenue as a platform, contributing to the creation of a new music market.

[1268] The following describes the processing flow.

[1269] Processing flow of a music generation system that combines an emotion engine

[1270] Step 1:

[1271] The user enters their email address, password, and profile information into the registration form on their device and clicks the registration button.

[1272] Step 2:

[1273] The server receives registration information sent by the user and stores it in the database.

[1274] Step 3:

[1275] The server sends an authentication email to the user. This email contains an authentication link.

[1276] Step 4:

[1277] The user opens the verification email and clicks the verification link.

[1278] Step 5:

[1279] The server receives the request from the authentication link and updates the user's authentication status.

[1280] Song generation request

[1281] Step 1:

[1282] The user enters parameters such as song type, genre, and theme on the device's music generation screen, and also turns on the device's camera and microphone so that the emotion engine can acquire the user's emotional data.

[1283] Step 2:

[1284] The emotion engine analyzes data acquired through the camera and microphone to identify the user's emotional state.

[1285] Step 3:

[1286] The server sends the request data received from the user and the results of the emotion engine to the AI ​​model.

[1287] Step 4:

[1288] The AI ​​model generates songs, lyrics, sheet music, vocals, and accompaniments based on parameters and user emotion data.

[1289] Step 5:

[1290] The server receives the generated results from the AI ​​model and sends them back to the user's device.

[1291] Step 6:

[1292] The device displays the generated music data to the user and provides it in a playable state.

[1293] Editing and saving music

[1294] Step 1:

[1295] The user edits the generated music data on the device's editing screen. For example, they can change the melody, revise the lyrics, and adjust the accompaniment.

[1296] Step 2:

[1297] The user clicks the save button to save the edited music data.

[1298] Step 3:

[1299] The server receives a save request from the user and saves the edited music data to the database.

[1300] Copyright establishment and sales

[1301] Step 1:

[1302] Users access the music sales screen from their devices and set the price and sales conditions for the songs.

[1303] Step 2:

[1304] The user submits a sales request to the server.

[1305] Step 3:

[1306] The server stores sales information in a database and publishes copyright listings.

[1307] Transactions between users

[1308] Step 1:

[1309] Other users select a song from the sales page and click the purchase button.

[1310] Step 2:

[1311] The device sends a purchase request to the server.

[1312] Step 3:

[1313] The server processes the purchase request and grants the purchaser the right to use the music.

[1314] Step 4:

[1315] The server deducts the platform's revenue from the transaction amount and distributes the remainder to the music sellers.

[1316] Specific usage examples

[1317] Step 1:

[1318] Let's say User A generates a jazz song with the theme "winter love," and the emotion engine recognizes User A's emotion as "joy." User A accesses the song generation screen and provides "jazz," "winter love," and the joy emotion data provided by the emotion engine as input.

[1319] Step 2:

[1320] The server sends this information to the AI ​​model, which then generates the appropriate song.

[1321] Step 3:

[1322] This song is displayed in a playable state on User A's device. User A edits and saves the song, then sets a selling price and creates a copyright listing.

[1323] Step 4:

[1324] If another user B likes this song and wants to buy it, they click the purchase button on the sales page.

[1325] Step 5:

[1326] The server processes the transaction, grants user B the right to use the music, and distributes the remainder (transaction amount minus platform revenue) to user A.

[1327] As described above, this system allows users to easily generate music and manage and sell its copyrights. The introduction of an emotion engine enables music generation tailored to the user's emotions, providing a more personalized musical experience. Furthermore, the system generates revenue as a platform, contributing to the creation of a new music market.

[1328] (Example 2)

[1329] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1330] Traditional music generation systems lacked sufficient functionality for users to easily generate music and manage and trade its copyrights. Furthermore, they were unable to generate music that reflected the user's emotions, making it difficult to provide a personalized experience. Additionally, the processes for saving, managing, and trading generated music were inconvenient, highlighting the need for a consistent user experience.

[1331] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1332] In this invention, the server includes an input means on which the user can input parameters such as the type of song, music genre, and theme; an emotion recognition means that recognizes the user's emotions and analyzes the emotion data; a generation means that generates songs, lyrics, sheet music, vocals, and accompaniments based on the parameters and emotion data input via the input means and the emotion recognition means; a return means that returns the song data generated by the generation means to the user; a storage means that stores the song data; a management means that manages the copyright to the song data stored in the storage means; a trading means that buys and sells the copyright between users; and a revenue means that makes the profit generated by the trading means into revenue for the platform. As a result, users can easily and efficiently generate songs, receive personalized songs tailored to their emotions, and enjoy a consistent process of storage, management, and trading.

[1333] An "input method" is a means by which the user inputs parameters such as the type of song, music genre, and theme.

[1334] An "emotion recognition tool" is a means of recognizing a user's emotions and analyzing that emotional data.

[1335] "Generation means" refers to means for generating music, lyrics, sheet music, vocals, and accompaniment based on parameters and emotion data input via input means and emotion recognition means.

[1336] "Return means" refers to the means of returning the music data generated by the generation means to the user.

[1337] "Storage method" refers to the means of saving the generated music data.

[1338] "Management means" refers to means for managing copyrights to music data stored in storage means.

[1339] "Means of buying and selling" refers to the means by which users buy and sell copyrights among themselves.

[1340] "Revenue-generating means" refers to the means by which the platform generates revenue from the profits produced through buying and selling.

[1341] This invention is a system that allows users to easily generate music and manage and sell its copyrights. The system incorporates an emotion recognition engine to enable music generation that reflects the user's emotions. The system consists of various components, including the user's terminal, a data processing server, the emotion recognition engine, and a generation AI model used for music generation. Specific hardware used includes personal computers and smartphones. Software used includes a web browser and an AI model (e.g., OpenAI's GPT-3 or Google's MusicLM).

[1342] User-inputted parameters

[1343] Users access the music generation system's webpage using a browser on their PC or smartphone. There, they input the type of song, music genre, theme, etc. They also grant permission for the camera and microphone to be used for AI-powered emotion recognition. At that point, a form appears on the screen for entering specific parameters such as "jazz" or "winter love."

[1344] emotion recognition

[1345] When a user allows the use of the camera and microphone, the device sends video and audio to an emotion recognition engine. The engine analyzes facial expressions and tone of voice to identify the user's current emotions. For example, the engine outputs emotion data such as "joy," "sadness," or "surprise."

[1346] Data processing and music generation

[1347] The user's input parameters and emotion data obtained from the emotion recognition engine are sent to the server in a single batch. The server processes this data and sends a request for music generation to the generative AI model. The generative AI model generates music, lyrics, sheet music, vocals, accompaniment, etc., based on these inputs. In doing so, the AI ​​model generates music with characteristics that match the user's emotions. For example, if the analysis results indicate the emotion of "joy," a bright and upbeat song will be generated.

[1348] Return and display of music data

[1349] The music data generated by the AI ​​model is sent back to the server, from which it is transmitted to the user's device. The user's device displays the received data in a playable format, allowing the user to listen to the generated music. If the user is satisfied with the music, clicking the "Save" button sends the request to the server, which then saves the music data to its database.

[1350] Copyright management and buying and selling of music

[1351] The server manages the copyrights to the stored music data and allows users to set the sales conditions for the music. If another user wishes to purchase a song, a purchase request is sent to the server, and the server processes the transaction. The remaining amount after deducting the platform's revenue from the transaction amount is distributed to the user who created the music.

[1352] Specific example

[1353] For example, suppose user A inputs parameters such as "jazz" and "winter love," and the emotion recognition engine detects the emotion "joy." The server sends this data to an AI model, which generates a cheerful jazz song tailored to user A's parameters. This song is displayed on user A's device in a playable format, and user A can listen to, edit, and save the song.

[1354] Example of a prompt

[1355] "Please generate a jazz song. The theme is 'winter love,' and the emotion is 'joy.'"

[1356] As described above, this system allows users to easily generate music and seamlessly manage and sell its copyrights. Furthermore, the introduction of emotion recognition enables the generation of personalized music that responds to the user's emotions.

[1357] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1358] Step 1:

[1359] The user accesses the music generation screen.

[1360] Input: Launch a browser on the user's device and access the music generation system's webpage.

[1361] Specific steps: The user uses a browser on their computer or smartphone to enter a specified web address and open the music generation system screen. They then log in following the interface displayed there.

[1362] Step 2:

[1363] The user enters the parameters of the song.

[1364] Input: Parameters such as song type (e.g., ballad), music genre (e.g., jazz), and theme (e.g., "Winter Love").

[1365] Specific operation: The user enters the required information into the input form on the screen. After entering the parameters, they click the "Complete" button.

[1366] Output: The input parameters are saved on the device and sent to the server as data to proceed with the emotion recognition process.

[1367] Step 3:

[1368] The device activates the camera and microphone.

[1369] Input: Request for use of camera and microphone with user permission.

[1370] Specific action: The user grants permission to use the camera and microphone through a dialog box displayed on the screen.

[1371] Output: Once permission is obtained, the camera and microphone will activate and begin collecting data on the user's facial expressions and voice.

[1372] Step 4:

[1373] The emotion recognition engine analyzes the user's emotions.

[1374] Input: Facial expression data and audio data acquired from the camera and microphone.

[1375] Specific operation: The emotion recognition engine analyzes data in real time to identify the user's emotional state (e.g., joy, sadness, surprise, etc.).

[1376] Output: The emotional data obtained as an analysis result is sent to the server.

[1377] Step 5:

[1378] The server receives user input and sentiment data.

[1379] Input: Parameters entered by the user and emotion data sent from the emotion recognition engine.

[1380] Specific operation: The server receives this data and integrates it as a series of data.

[1381] Output: The integrated data is prepared as input for the AI ​​model.

[1382] Step 6:

[1383] The server sends a request to the AI ​​model.

[1384] Input: Integrated song parameters and emotion data.

[1385] Specific operation: The server creates a specific prompt message for the generated AI model and sends a request for music generation.

[1386] Output: Prompt message sent to the AI ​​model: "Generate a jazz song. The theme is 'winter love,' and the emotion is 'joy.'"

[1387] Step 7:

[1388] Music generation using AI models

[1389] Input: The prompt message sent from the server.

[1390] Specific operation: The AI ​​model generates music based on prompt text. It generates data in various forms, including lyrics, sheet music, vocals, and accompaniment.

[1391] Output: The generated music data is sent back to the server.

[1392] Step 8:

[1393] The server sends the generated music data to the user's terminal.

[1394] Input: Generated music data returned from the AI ​​model.

[1395] Specific operation: The server receives the generated data and sends it to the user's terminal.

[1396] Output: Music data sent to the user's device.

[1397] Step 9:

[1398] The device displays music data.

[1399] Input: Music data sent from the server.

[1400] Specific operation: The device converts the data into a playable format and displays it to the user visually and audibly.

[1401] Output: Music displayed in a playable format.

[1402] Step 10:

[1403] Users edit songs

[1404] Input: Songs displayed in playable formats.

[1405] Specific actions: The user edits the music using the music editing tools as needed. Once the modifications are complete, they submit another save request.

[1406] Output: The edited music data is saved to the device.

[1407] Step 11:

[1408] The device sends a save request to the server.

[1409] Input: Edited song data.

[1410] Specific action: The user clicks the save button, and a save request is sent to the server.

[1411] Output: The server receives the music data.

[1412] Step 12:

[1413] The server saves the music data to the database.

[1414] Input: Music data for which a save request was received.

[1415] Specific operation: The server saves the music data to a database, allowing users to access it again.

[1416] Output: Music data stored in the database.

[1417] Step 13:

[1418] The server manages the copyright of the music.

[1419] Input: Saved music data.

[1420] Specific operation: The server manages copyright information for each song and assigns copyright information to each song.

[1421] Output: Managed copyright information.

[1422] Step 14:

[1423] The user enters the music sales conditions.

[1424] Input: Sales conditions (price, terms of use, etc.) entered on the music sales page.

[1425] Specific operation: The user enters the sales price and terms of use, and sends a sales request to the server.

[1426] Output: Sales conditions data from users.

[1427] Step 15:

[1428] The server processes the transaction.

[1429] Input: Sales conditions data from users and purchase requests from other users.

[1430] Specific operation: The server processes the transaction and grants the buyer the right to use the music. Then, it distributes the remaining amount to the seller after deducting the platform's revenue from the transaction amount.

[1431] Output: Transaction information and distributed profits.

[1432] (Application Example 2)

[1433] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1434] Traditional music generation systems typically generate music based on user-inputted parameters such as song type, genre, and theme, with little consideration given to the user's emotional state. Furthermore, copyright management and trading of generated music were complex, making platform monetization difficult. Additionally, there was a lack of easy ways for users to save, manage, and commercially utilize the music they generated themselves.

[1435] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1436] In this invention, the server includes an input means on which the user can input parameters such as the type of song, music genre, and theme; an emotion recognition means that recognizes the user's emotional state using an emotion engine and reflects the parameters based on that emotion in song generation; and a generation means that generates songs, lyrics, sheet music, vocals, and accompaniment based on the parameters input via the input means and the emotion recognition means. This enables the automatic generation of personalized songs that reflect the user's emotional state, and allows for efficient monetization through copyright management and sales.

[1437] "Input method" refers to a device or interface for users to input parameters such as the type of music, music genre, and theme.

[1438] "Generation means" refers to a system or algorithm that automatically generates music, lyrics, sheet music, vocals, and accompaniment based on parameters input via the input means and emotion recognition means.

[1439] "Emotion recognition means" refers to technology that uses an emotion engine to recognize the user's emotional state and reflects parameters based on that emotion in music generation.

[1440] "Return means" refers to the communication infrastructure and software used to return the music data generated by the generation means to the user.

[1441] "Storage means" refers to a storage device or database system for storing the music data generated by the generation means.

[1442] "Management means" refers to tools or software for managing the copyright of music data stored in the aforementioned storage means.

[1443] "Means of buying and selling" refers to platforms or trading systems for buying and selling the aforementioned copyrights among users.

[1444] "Revenue-generating means" refers to a mechanism or management system for generating revenue for the platform from the profits generated by the aforementioned buying and selling means.

[1445] "Editing means" refers to software or interfaces that allow users to edit generated music data.

[1446] "Authentication means" refers to the systems and protocols used for user registration and authentication.

[1447] A "server" is a computer or network system that runs a music generation system and processes requests from users.

[1448] System Overview

[1449] This system allows users to input parameters such as song type, music genre, and theme, collect emotional data using emotion recognition means, and generate and manage songs based on that data. Specific components include input means, generation means, emotion recognition means, return means, storage means, management means, buying and selling means, and revenue means.

[1450] Program Processing Description

[1451] 1. User interface (input means):

[1452] Users access the music generation screen through a smartphone application. They input parameters such as the type of song, music genre, and theme. For example, they can select options such as "rock" or "summer adventure."

[1453] 2. Emotion recognition means:

[1454] The system analyzes facial images and audio data acquired through the user's camera and microphone. Emotion recognition, in particular, utilizes the "EmotionRecognizer" module to identify the user's emotional state (joy, sadness, surprise, anger, etc.). For example, if the user smiles at the camera, the system determines that they are in a state of "joy."

[1455] 3. Music generation (generation methods):

[1456] The server receives data from the input means and emotion recognition means and generates music using the "MusicGenerator" module. During generation, the original parameters and emotion data are reflected. For example, if rock is selected and the user's emotional state is "joy," an upbeat song with positive elements will be generated.

[1457] 4. Return of generated music (return method):

[1458] The server sends the generated music data back to the user's smartphone. The user can then play and check the received music within the app.

[1459] 5. Preservation means:

[1460] If the user is satisfied with the generated music data, they can click the save button to send a save request to the server. The server receives this request and saves the music data to its database.

[1461] 6. Music management and trading (management methods and trading methods):

[1462] The server manages the copyright of the stored music data. Users enter pricing and sales conditions on the music sales page and submit sales requests. Other users can view and purchase the music published on the sales page.

[1463] 7. Monetization (means of revenue):

[1464] A system is in place where the platform's revenue is deducted from the profits generated by the buying and selling methods, and the remainder is distributed to the sellers.

[1465] Specific example

[1466] For example, suppose a user wants to generate a song with the themes of "rock" and "summer adventure." If the emotion recognition system detects "joy" from the user's facial expression, a song will be generated based on "rock," "summer adventure," and "joy."

[1467] Examples of prompts for generative AI models

[1468] The user wants to generate a song with the themes of "rock" and "summer adventure." The emotion engine detected "joy" from the user's facial expression. Please generate a song based on this.

[1469] This allows users to easily create, save, manage, and sell personalized music. Furthermore, it is expected that generating music based on emotional data will further enhance user satisfaction.

[1470] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1471] Step 1:

[1472] The user accesses the music generation screen through a smartphone application. Here, the user inputs parameters such as the type of music, music genre, and theme (input: user input, output: parameter data). This data is necessary for subsequent processing and serves to specify the user's wishes.

[1473] Step 2:

[1474] The user's facial expressions and audio data are collected through the smartphone's camera and microphone (input: camera video, audio data; output: emotion data). This data is analyzed by an emotion recognition system to identify the user's emotional state (e.g., joy, sadness, surprise, anger). The "EmotionRecognizer" module is used for this analysis.

[1475] Step 3:

[1476] The terminal sends the collected emotion data and the parameter data of the initially entered song to the server (input: emotion data, parameter data; output: integrated data). The server receives this data, integrates it, and creates a request for song generation.

[1477] Step 4:

[1478] The server uses the "MusicGenerator" module to generate music, lyrics, sheet music, vocals, and accompaniment based on integrated data (input: integrated data, output: generated music data). The data processing performed here involves generating the optimal music by considering emotional data, song type, music genre, and theme.

[1479] Step 5:

[1480] The generated music data is sent back from the server to the user's smartphone (input: generated music data, output: data sent to the user's device). The user's smartphone receives this data and displays it on the music playback screen.

[1481] Step 6:

[1482] When a user plays and reviews a generated song and wishes to save it, they click the save button to send a save request to the server (input: user's save request, output: song save confirmation). The server receives this request and saves the song data to the database.

[1483] Step 7:

[1484] A management system operates to manage copyrights for saved music data (input: saved music data, output: copyright management data). Users can enter pricing and sales conditions on the music sales page and submit sales requests.

[1485] Step 8:

[1486] When another user purchases a publicly available song, the buying and selling mechanism is activated and the transaction takes place (input: purchase request, output: song usage rights). The server processes the transaction, deducts the platform's revenue from the generated profit, and distributes it to the seller (input: transaction data, output: revenue data).

[1487] Step 9:

[1488] The platform calculates revenue from the profits generated through buying and selling and distributes it to users (input: revenue data, output: revenue distribution data). This enables smooth transactions between users and allows individuals to commercially use and monetize music they have created.

[1489] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1490] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1491] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1492] [Fourth Embodiment]

[1493] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1494] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1495] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1496] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1497] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1498] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1499] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1500] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1501] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1502] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1503] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1504] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1505] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1506] This invention is a system for users to easily generate music and manage and buy / sell its copyrights. This system includes an input means for users to input parameters such as the type of music, music genre, and theme.

[1507] Steps for users to generate music

[1508] Users access the music generation screen from their own devices (PCs or smartphones). Here, they input parameters such as the type of song, music genre, and theme. For example, specific inputs such as "Pop" and "Summer Memories" are possible.

[1509] Server-based music generation

[1510] The server receives a request from the user and instructs the AI ​​model to generate music. Based on the input parameters, the AI ​​model automatically generates the music, lyrics, sheet music, vocals, and accompaniment. The server then receives the generated music data.

[1511] Return and display of music data

[1512] The server sends the generated music data back to the user's device. The user's device displays the received music in a playable state. The user can listen to this music and edit it as needed.

[1513] Saving music data

[1514] If the user is satisfied with the generated music data, they click the save button to save the song. The server receives the save request and saves the music data to the database. This allows the user to access the song again at any time.

[1515] Copyright management and trading

[1516] The server provides tools for managing copyrights to stored music data on a per-user basis. Users enter pricing and sales conditions on the music sales page and send sales requests to the server. The server stores this information and makes it public in the copyright management system.

[1517] Transactions between users

[1518] Other users can view the published song information and select the songs they wish to purchase. Once a purchase request is sent to the server, the server processes the transaction and grants the buyer the right to use the song. Furthermore, the platform deducts its revenue from the transaction amount and distributes the remainder to the seller. This facilitates smooth transactions between users and makes it possible for individuals to commercially use and monetize their created music.

[1519] Specific usage examples

[1520] For example, suppose User A wants to generate a jazz song with the theme "winter love." User A accesses the song generation screen and enters "jazz" and "winter love." The server sends this information to the AI ​​model, which generates an appropriate song. This song is displayed on User A's device and is playable. User A edits and saves the song, then sets a sales price and creates a copyright listing.

[1521] If another user B likes this song and wants to buy it, they click the purchase button on the sales page. The server processes the transaction, grants user B the right to use the song, and distributes the remaining amount to user A after deducting the platform's revenue from the transaction amount.

[1522] As described above, this system allows users to easily create music and manage and buy / sell its copyrights. Furthermore, since the system can generate revenue as a platform, it contributes to the creation of a new music market.

[1523] The following describes the processing flow.

[1524] User registration and authentication

[1525] Step 1:

[1526] The user enters their email address, password, and profile information into the registration form on their device and clicks the registration button.

[1527] Step 2:

[1528] The server receives registration information sent by the user and stores it in the database.

[1529] Step 3:

[1530] The server sends an authentication email to the user. This email contains an authentication link.

[1531] Step 4:

[1532] The user opens the verification email and clicks the verification link.

[1533] Step 5:

[1534] The server receives the request from the authentication link and updates the user's authentication status.

[1535] Song generation request

[1536] Step 1:

[1537] The user enters parameters such as the song type, genre, and theme on the device's music generation screen and clicks the generate button.

[1538] Step 2:

[1539] The terminal sends the entered parameters to the server.

[1540] Step 3:

[1541] The server sends the request data received from the user to the AI ​​model.

[1542] Step 4:

[1543] The AI ​​model generates music, lyrics, sheet music, vocals, and accompaniment based on the parameters.

[1544] Step 5:

[1545] The server receives the generated results from the AI ​​model and sends them back to the user's device.

[1546] Step 6:

[1547] The device displays the generated music data to the user.

[1548] Editing and saving music

[1549] Step 1:

[1550] The user edits the generated music data on the device's editing screen. For example, they can change the melody, revise the lyrics, and adjust the accompaniment.

[1551] Step 2:

[1552] The user clicks the save button to save the edited music data.

[1553] Step 3:

[1554] The server receives a save request from the user and saves the edited music data to the database.

[1555] Copyright establishment and sales

[1556] Step 1:

[1557] Users access the music sales screen from their devices and set the price and sales conditions for the songs.

[1558] Step 2:

[1559] The user submits a sales request to the server.

[1560] Step 3:

[1561] The server stores sales information in a database and publishes copyright listings.

[1562] Transactions between users

[1563] Step 1:

[1564] Other users select a song from the sales page and click the purchase button.

[1565] Step 2:

[1566] The device sends a purchase request to the server.

[1567] Step 3:

[1568] The server processes the purchase request and grants the purchaser the right to use the music.

[1569] Step 4:

[1570] The server deducts the platform's revenue from the transaction amount and distributes the remainder to the music sellers.

[1571] The above outlines the specific processing steps in the patent system.

[1572] (Example 1)

[1573] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1574] While systems exist today for easily generating music and managing and trading its copyrights, they do not fully meet user needs for smooth music generation, management, and trading procedures. Furthermore, improving the efficiency of revenue sharing associated with the editing and trading of generated music remains a challenge. This invention aims to solve these problems and provide an efficient music generation, management, and trading system that is easily accessible to users.

[1575] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1576] In this invention, the server includes an input means on which a user can input parameters such as the type of music, music genre, and theme; a generation means that generates music, lyrics, sheet music, vocals, and accompaniment based on the parameters input via the input means; a return means that sends the music data generated by the generation means back to the user; a storage means that stores the music data; a management means that manages the copyright to the music data stored in the storage means; a trading means that buys and sells the copyright between users; a revenue means that makes the profit generated by the trading means into platform revenue; a transaction means that checks the published music information of other users and processes purchase requests when they wish to purchase it; and a distribution means that deducts a commission from the transaction amount and distributes the remaining amount to the seller. This makes it possible for a user to smoothly perform a series of operations, from inputting parameters to generate music, and managing, editing, and buying and selling that music.

[1577] An "input method" is an interface for users to input parameters such as the type of music, music genre, and theme.

[1578] The "generation means" is a system for generating music, lyrics, sheet music, vocals, and accompaniment based on parameters input via the input means.

[1579] The "return means" is a mechanism for sending the music data generated by the generation means to the user.

[1580] A "storage method" refers to a system for saving generated music data to a database or storage device.

[1581] "Management means" refers to a tool for managing the copyright of music data stored in the aforementioned storage means.

[1582] A "trading method" refers to a system for users to buy and sell copyrighted music that has been preserved.

[1583] "Revenue-generating means" refers to a mechanism for generating revenue for the platform from the profits generated by the aforementioned buying and selling means.

[1584] A "transaction method" is a system for other users to view publicly available song information and process purchase requests if they wish to buy it.

[1585] A "distribution mechanism" is a system for distributing the remaining amount to the seller after deducting fees from the transaction amount.

[1586] "Editing means" refers to an interface that allows the user to edit the aforementioned music data.

[1587] "Authentication means" refers to a system used for user registration and authentication.

[1588] This invention relates to a system for users to easily generate music and manage and buy / sell its copyrights. The system includes an input means for users to input parameters such as the type of music, music genre, and theme; a generation means for generating music, lyrics, sheet music, vocals, and accompaniment based on the input parameters; a return means for sending the generated music data back to the user; a storage means for storing the music data; a management means for managing the copyrights to the stored music data; a buying and selling means for users to buy and sell copyrights; a revenue means for making the profits generated by the buying and selling means into platform revenue; a transaction means for other users to check the published music information and process purchase requests if they wish to purchase it; a distribution means for deducting a commission from the transaction amount and distributing the remaining amount to the seller; and an editing means for editing music data and an authentication means for user registration and authentication.

[1589] Hardware and software configuration

[1590] This system consists of user terminals (PCs, smartphones) and a server based on a high-performance computer. Users can access the server from their terminals via an internet browser or a dedicated application.

[1591] For the AI ​​model that functions as a generation tool, for example, OpenAI's GPT-3 or a neural network model specifically designed for music generation can be used. For storage and management, cloud-based database systems (e.g., Amazon Web Services, Google Cloud Platform) can be applied.

[1592] System operation

[1593] 1. When a user submits a song generation request

[1594] The user accesses the music generation screen from their device, enters parameters such as "Genre: Pop" and "Theme: Summer Memories," and presses the generate button. The following is an example of the prompt text.

[1595] Genre: Pop

[1596] Theme: Summer Memories

[1597] 2. When the server receives a music generation request and sends it to the AI ​​model

[1598] The server receives the user's request, analyzes the input parameters, and sends them to the AI ​​model. Based on these parameters, the AI ​​model generates music, lyrics, sheet music, vocals, and accompaniment.

[1599] 3. When the generated music data is returned to the user and saved.

[1600] The AI ​​model generates music data, which is received by the server and sent back to the user's device. The user plays the music on their device and edits it as needed. If the user is satisfied with the music, they click the save button to save the music data to the database.

[1601] 4. When managing and buying / selling copyrights of musical works

[1602] Copyright for saved music data is managed on the server, and users can set prices and sales conditions on the sales page. Based on the information set by the user, other users can view the music information and submit purchase requests. The server processes the transaction and grants the buyer the right to use the music. A commission is deducted from the transaction amount, and the remainder is distributed to the seller.

[1603] Thus, the system of the present invention allows users to easily generate music and smoothly manage and buy / sell that music. Furthermore, the system can increase platform revenue and contribute to the creation of a new music market.

[1604] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1605] Step 1: User submits a song generation request

[1606] The user accesses the music generation screen from their device, enters parameters such as "Genre: Pop" and "Theme: Summer Memories," and presses the generate button. The input data (song type, music genre, theme, etc.) is sent to the server.

[1607] Specific actions:

[1608] The user opens a web browser and accesses the music generation page.

[1609] The user enters detailed information about the song into the form.

[1610] The user clicks the "Generate" button, and the input data is sent to the server (Input: song parameters, Output: HTTP request to the server).

[1611] Step 2: The server receives the music generation request and sends it to the AI ​​model.

[1612] The server receives a request from the user, analyzes the input parameters, and sends them to the generated AI model. The AI ​​model receives a prompt message along with the analysis results.

[1613] Specific actions:

[1614] The server receives an HTTP request and parses the input data.

[1615] The server calls the API of the generated AI model and passes the analysis results as prompt messages (input: user request, output: prompt messages).

[1616] Step 3: Music generation using an AI model

[1617] The AI ​​model generates music, lyrics, sheet music, vocals, and accompaniment based on the received prompt text. This generation process is performed using an advanced neural network. The generated music data is saved as a temporary file.

[1618] Specific actions:

[1619] The AI ​​model executes a music generation algorithm based on the prompt text.

[1620] The generated music data is saved as a temporary file (input: prompt text, output: music data).

[1621] Step 4: Receiving the generated music data from the server and returning it to the user terminal.

[1622] The server receives the music data generated by the AI ​​model and sends it back to the user's device. The music data is sent to the user as an HTTP response.

[1623] Specific actions:

[1624] The server receives music data generated from the AI ​​model.

[1625] The server creates a response to send music data to the user's terminal (input: music data, output: HTTP response to the user).

[1626] Step 5: User plays and edits music.

[1627] The user's terminal receives the generated music data and displays it in a playable state. The user can listen to the music and edit it as needed. After editing, they click the save button to send the edits to the server.

[1628] Specific actions:

[1629] The user opens a music playback application or browser.

[1630] Play and edit received music files.

[1631] The user's edits are sent to the server (input: edited music data, output: save request to the server).

[1632] Step 6: User submits a request to save music.

[1633] If the user is satisfied with the generated music data, they click the save button, and a save request is sent to the server. The music data is then sent back to the server.

[1634] Specific actions:

[1635] The user clicks the "Save" button.

[1636] The save request and music data are sent to the server (input: music data, output: HTTP request to the server).

[1637] Step 7: The server receives the music save request and saves it to the database.

[1638] The server receives the save request and saves the music data to the database. A message confirming the save is sent back to the user.

[1639] Specific actions:

[1640] The server receives the save request and connects to the database.

[1641] This process saves music data to a database and sends a save completion notification to the user (Input: Music data, Output: Save completion message).

[1642] Step 8: Server-based copyright management and publication of sales information for music.

[1643] The server provides tools for managing copyrights to stored music data, and users set prices and conditions on the sales page. The set sales information is made public.

[1644] Specific actions:

[1645] The server registers the music data with the copyright management system.

[1646] Users enter sales information and publish it (input: sales information, output: published sales page).

[1647] Step 9: Another user submits a request to purchase music.

[1648] Other users view the published song information and click the purchase button. A purchase request is sent to the server.

[1649] Specific actions:

[1650] Those interested in purchasing the song access the public page to view it.

[1651] Click the purchase button to send a purchase request to the server (Input: Purchase request, Output: HTTP request to the server).

[1652] Step 10: Server receives music purchase request and processes transaction.

[1653] The server receives the purchase request and grants the buyer the right to use the music. It deducts a commission from the transaction amount and distributes the remaining amount to the seller.

[1654] Specific actions:

[1655] The server processes the purchase request and grants the purchaser the right to use the music.

[1656] The transaction amount is reduced by a commission, and the remaining amount is distributed to the seller (Input: Purchase Request, Output: Transaction Completion Message).

[1657] (Application Example 1)

[1658] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1659] In today's world, there is a need for a system that allows users to easily create music, distribute it to other users, rate it, and sell it. However, current systems fragment the process from music creation to distribution, rating, and selling, resulting in a poor user experience. Furthermore, copyright management and monetization of created music are often handled separately, requiring users to go through cumbersome procedures. In response to this, a system that can handle everything from music creation to distribution, rating, and selling in an integrated manner is needed.

[1660] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1661] In this invention, the server includes an input means on which a user can input parameters such as the type of music, music genre, and theme; a generation means that generates music, lyrics, sheet music, vocals, and accompaniment based on the parameters input via the input means; a return means that returns the music data generated by the generation means to the user; a storage means that stores the music data; a management means that manages the copyright to the music data stored in the storage means; a trading means that buys and sells the copyright between users; a revenue means that makes the profit generated by the trading means into platform revenue; an evaluation means that evaluates the music data generated by the user and creates a ranking; and a distribution means that distributes the generated music data to other users. As a result, users can enjoy a consistent experience from music generation to distribution, evaluation, and buying and selling, and copyright management and monetization can be handled centrally.

[1662] An "input method" is an interface for users to input parameters such as the type of music, music genre, and theme.

[1663] The "generation means" is a function for generating music, lyrics, sheet music, vocals, and accompaniment based on parameters input via the aforementioned input means.

[1664] The "return means" is a function for returning the music data generated by the generation means to the user.

[1665] "Storage means" refers to storage means for storing the aforementioned music data.

[1666] "Management means" refers to functions for managing copyrights to stored music data.

[1667] "Means of buying and selling" refers to a function for buying and selling the aforementioned copyrights among users.

[1668] "Revenue-generating means" refers to a function that generates revenue for the platform from the profits generated by the aforementioned buying and selling means.

[1669] The "evaluation method" is an interface for users to evaluate generated music data and create rankings.

[1670] "Distribution method" refers to a function for distributing generated music data to other users.

[1671] This invention provides a system that allows users to easily create music, manage and sell its copyrights, and enable other users to rate and distribute that music. This system is implemented by the following means.

[1672] First, the user inputs song parameters (song type, music genre, theme, etc.) via an input device such as a smartphone. The entered parameters are then sent to the server.

[1673] The server receives these parameters and uses generation tools to generate music, lyrics, sheet music, vocals, and accompaniment. OpenAI's GPT-4 and music generation-specific models are used as generation tools. The generated music data is then sent back from the server to the user's terminal.

[1674] The user's device displays the returned music data in a playable state. The user can listen to the music and edit it as needed using the editing tools. The edited music data is saved via the storage tools.

[1675] Copyright for stored music data is managed on a per-user basis using management tools. Users can then set the selling price of their music and sell it to other users using the trading tools. When a sale is completed, the profits generated are processed as revenue for the platform through revenue-generating mechanisms.

[1676] Furthermore, the evaluation system allows users to rate music data generated by other users and create rankings. Users can then improve their music based on these ratings. Finally, highly-rated music and newly generated music are distributed to other users via the distribution system.

[1677] This system allows users to enjoy a consistent experience from music creation and editing to storage, distribution, and buying / selling. Furthermore, copyright management and monetization can be handled centrally. This is expected to contribute to the creation of a new music market.

[1678] For example, if a user wants to generate a jazz song with the theme "Summer Memories," it would look like this:

[1679] The user inputs the parameters "jazz" and "summer memories" into an input device. The server uses this information to generate a song using an AI model and sends the generated song data back to the user's terminal. The user listens to the song, edits it, saves it, and distributes it to other users.

[1680] Example of a prompt:

[1681] Song type: Pop

[1682] Music genre: Jazz

[1683] Theme: Summer Memories

[1684] In this way, the invention effectively meets user needs and provides a new musical experience.

[1685] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1686] Step 1:

[1687] The user launches the music generation application and accesses the input fields. The user enters the type of song (e.g., pop), music genre (e.g., jazz), and theme (e.g., summer memories). This prepares the user to send the parameters of the song they want to generate to the server.

[1688] input:

[1689] Type of music (e.g., pop)

[1690] Music genres (jazz, etc.)

[1691] Theme (e.g., summer memories)

[1692] output:

[1693] Music generation parameters (sent to the server in JSON format)

[1694] Step 2:

[1695] The server receives music generation parameters (JSON data) from the user. The server analyzes this data and sends a music generation request to the AI ​​model. The AI ​​model generates music, lyrics, sheet music, vocals, and accompaniment based on the specified parameters.

[1696] input:

[1697] Music generation parameters (JSON format)

[1698] Data processing:

[1699] Convert music generation parameters into prompt statements for the AI ​​model.

[1700] output:

[1701] Generated music data (music, lyrics, sheet music, vocals, accompaniment)

[1702] Step 3:

[1703] The server analyzes the music data received from the generating AI model and sends that data back to the user's device. The user's device receives the music data and displays it in a playable state. The user can listen to the generated music by clicking the play button.

[1704] input:

[1705] Generated music data

[1706] output:

[1707] Music data playable on the user's device

[1708] Step 4:

[1709] Users listen to music and edit the music data using editing tools as needed. The edited music data is temporarily saved on the device.

[1710] input:

[1711] Played music data

[1712] User-edited parameters

[1713] Data processing:

[1714] Update music data based on user-edited parameters.

[1715] output:

[1716] Edited music data

[1717] Step 5:

[1718] After the user finishes editing, they click the save button. The user's device sends the edited music data to the server, which then persistently stores the data using a storage method. Copyright management information is also added to the saved data.

[1719] input:

[1720] Edited music data

[1721] output:

[1722] Persistent music data

[1723] Step 6:

[1724] Users set a selling price for their saved music data and send a buy / sell request to the server. The server makes the music data available to other users using the buy / sell system and facilitates the transaction. Any profits generated are processed using the revenue system.

[1725] input:

[1726] Selling price setting

[1727] Buy / Sell Request

[1728] output:

[1729] Released song information

[1730] Revenue sharing information

[1731] Step 7:

[1732] Other users rate the generated songs and send their ratings to the server. The server uses the rating system to collect the rating data and create rankings. Highly rated songs are recommended to other users using the distribution system.

[1733] input:

[1734] Evaluation data from other users

[1735] output:

[1736] Rating Ranking

[1737] Recommended Song List

[1738] In this way, specific inputs and outputs, as well as data processing, are performed at each step, allowing users to obtain a consistent music generation and distribution experience.

[1739] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1740] This invention is a system for users to easily generate music and manage and sell its copyrights. This system not only has an input means for users to input parameters such as the type of music, music genre, and theme, but also incorporates an emotion engine that recognizes the user's emotions.

[1741] Steps for users to generate music

[1742] Users access the music generation screen from their own devices (PCs or smartphones). In addition to inputting parameters such as song type, music genre, and theme, the emotion engine recognizes the user's facial expressions and voice tone to analyze their current emotions.

[1743] The role of the emotional engine

[1744] The emotion engine analyzes data acquired through the user's camera and microphone to identify the user's emotional state (e.g., joy, sadness, surprise, anger). This emotional data is also used as a parameter for music generation. For example, if the emotion engine recognizes that the user is in a state of "joy," it will suggest a song with an upbeat tempo that matches that feeling of joy.

[1745] Server-based music generation

[1746] The server receives requests from users and results from the emotion engine, and requests the AI ​​model to generate music. The AI ​​model automatically generates music, lyrics, sheet music, vocals, and accompaniment based on the input parameters and the user's emotion data. The server then receives the generated music data.

[1747] Return and display of music data

[1748] The server sends the generated music data back to the user's device. The user's device displays the received music in a playable state. The user can listen to this music and edit it as needed.

[1749] Saving music data

[1750] If the user is satisfied with the generated music data, they click the save button to save the song. The server receives the save request and saves the music data to the database. This allows the user to access the song again at any time.

[1751] Copyright management and trading

[1752] The server provides tools for managing copyrights to stored music data on a per-user basis. Users enter pricing and sales conditions on the music sales page and send sales requests to the server. The server stores this information and makes it public in the copyright management system.

[1753] Transactions between users

[1754] Other users can view the published song information and select the songs they wish to purchase. Once a purchase request is sent to the server, the server processes the transaction and grants the buyer the right to use the song. Furthermore, the platform deducts its revenue from the transaction amount and distributes the remainder to the seller. This facilitates smooth transactions between users and makes it possible for individuals to commercially use and monetize their created music.

[1755] Specific usage examples

[1756] For example, suppose User A generates a jazz song with the theme "winter love," and the emotion engine recognizes User A's emotion as "joy." User A accesses the song generation screen and provides "jazz," "winter love," and the joy emotion data provided by the emotion engine as input. The server sends this information to the AI ​​model, which generates an appropriate song. This song is displayed on User A's device in a playable state. User A edits and saves the song, then sets a sales price and creates a copyright listing.

[1757] If another user B likes this song and wants to buy it, they click the purchase button on the sales page. The server processes the transaction, grants user B the right to use the song, and distributes the remaining amount to user A after deducting the platform's revenue from the transaction amount.

[1758] As described above, this system allows users to easily generate music and manage and sell its copyrights. The introduction of an emotion engine enables music generation tailored to the user's emotions, providing a more personalized musical experience. Furthermore, the system generates revenue as a platform, contributing to the creation of a new music market.

[1759] The following describes the processing flow.

[1760] Processing flow of a music generation system that combines an emotion engine

[1761] Step 1:

[1762] The user enters their email address, password, and profile information into the registration form on their device and clicks the registration button.

[1763] Step 2:

[1764] The server receives registration information sent by the user and stores it in the database.

[1765] Step 3:

[1766] The server sends an authentication email to the user. This email contains an authentication link.

[1767] Step 4:

[1768] The user opens the verification email and clicks the verification link.

[1769] Step 5:

[1770] The server receives the request from the authentication link and updates the user's authentication status.

[1771] Song generation request

[1772] Step 1:

[1773] The user enters parameters such as song type, genre, and theme on the device's music generation screen, and also turns on the device's camera and microphone so that the emotion engine can acquire the user's emotional data.

[1774] Step 2:

[1775] The emotion engine analyzes data acquired through the camera and microphone to identify the user's emotional state.

[1776] Step 3:

[1777] The server sends the request data received from the user and the results of the emotion engine to the AI ​​model.

[1778] Step 4:

[1779] The AI ​​model generates songs, lyrics, sheet music, vocals, and accompaniments based on parameters and user emotion data.

[1780] Step 5:

[1781] The server receives the generated results from the AI ​​model and sends them back to the user's device.

[1782] Step 6:

[1783] The device displays the generated music data to the user and provides it in a playable state.

[1784] Editing and saving music

[1785] Step 1:

[1786] The user edits the generated music data on the device's editing screen. For example, they can change the melody, revise the lyrics, and adjust the accompaniment.

[1787] Step 2:

[1788] The user clicks the save button to save the edited music data.

[1789] Step 3:

[1790] The server receives a save request from the user and saves the edited music data to the database.

[1791] Copyright establishment and sales

[1792] Step 1:

[1793] Users access the music sales screen from their devices and set the price and sales conditions for the songs.

[1794] Step 2:

[1795] The user submits a sales request to the server.

[1796] Step 3:

[1797] The server stores sales information in a database and publishes copyright listings.

[1798] Transactions between users

[1799] Step 1:

[1800] Other users select a song from the sales page and click the purchase button.

[1801] Step 2:

[1802] The device sends a purchase request to the server.

[1803] Step 3:

[1804] The server processes the purchase request and grants the purchaser the right to use the music.

[1805] Step 4:

[1806] The server deducts the platform's revenue from the transaction amount and distributes the remainder to the music sellers.

[1807] Specific usage examples

[1808] Step 1:

[1809] Let's say User A generates a jazz song with the theme "winter love," and the emotion engine recognizes User A's emotion as "joy." User A accesses the song generation screen and provides "jazz," "winter love," and the joy emotion data provided by the emotion engine as input.

[1810] Step 2:

[1811] The server sends this information to the AI ​​model, which then generates the appropriate song.

[1812] Step 3:

[1813] This song is displayed in a playable state on User A's device. User A edits and saves the song, then sets a selling price and creates a copyright listing.

[1814] Step 4:

[1815] If another user B likes this song and wants to buy it, they click the purchase button on the sales page.

[1816] Step 5:

[1817] The server processes the transaction, grants user B the right to use the music, and distributes the remainder (transaction amount minus platform revenue) to user A.

[1818] As described above, this system allows users to easily generate music and manage and sell its copyrights. The introduction of an emotion engine enables music generation tailored to the user's emotions, providing a more personalized musical experience. Furthermore, the system generates revenue as a platform, contributing to the creation of a new music market.

[1819] (Example 2)

[1820] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1821] Traditional music generation systems lacked sufficient functionality for users to easily generate music and manage and trade its copyrights. Furthermore, they were unable to generate music that reflected the user's emotions, making it difficult to provide a personalized experience. Additionally, the processes for saving, managing, and trading generated music were inconvenient, highlighting the need for a consistent user experience.

[1822] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1823] In this invention, the server includes an input means on which the user can input parameters such as the type of song, music genre, and theme; an emotion recognition means that recognizes the user's emotions and analyzes the emotion data; a generation means that generates songs, lyrics, sheet music, vocals, and accompaniments based on the parameters and emotion data input via the input means and the emotion recognition means; a return means that returns the song data generated by the generation means to the user; a storage means that stores the song data; a management means that manages the copyright to the song data stored in the storage means; a trading means that buys and sells the copyright between users; and a revenue means that makes the profit generated by the trading means into revenue for the platform. As a result, users can easily and efficiently generate songs, receive personalized songs tailored to their emotions, and enjoy a consistent process of storage, management, and trading.

[1824] An "input method" is a means by which the user inputs parameters such as the type of song, music genre, and theme.

[1825] An "emotion recognition tool" is a means of recognizing a user's emotions and analyzing that emotional data.

[1826] "Generation means" refers to means for generating music, lyrics, sheet music, vocals, and accompaniment based on parameters and emotion data input via input means and emotion recognition means.

[1827] "Return means" refers to the means of returning the music data generated by the generation means to the user.

[1828] "Storage method" refers to the means of saving the generated music data.

[1829] "Management means" refers to means for managing copyrights to music data stored in storage means.

[1830] "Means of buying and selling" refers to the means by which users buy and sell copyrights among themselves.

[1831] "Revenue-generating means" refers to the means by which the platform generates revenue from the profits produced through buying and selling.

[1832] This invention is a system that allows users to easily generate music and manage and sell its copyrights. The system incorporates an emotion recognition engine to enable music generation that reflects the user's emotions. The system consists of various components, including the user's terminal, a data processing server, the emotion recognition engine, and a generation AI model used for music generation. Specific hardware used includes personal computers and smartphones. Software used includes a web browser and an AI model (e.g., OpenAI's GPT-3 or Google's MusicLM).

[1833] User-inputted parameters

[1834] Users access the music generation system's webpage using a browser on their PC or smartphone. There, they input the type of song, music genre, theme, etc. They also grant permission for the camera and microphone to be used for AI-powered emotion recognition. At that point, a form appears on the screen for entering specific parameters such as "jazz" or "winter love."

[1835] emotion recognition

[1836] When a user allows the use of the camera and microphone, the device sends video and audio to an emotion recognition engine. The engine analyzes facial expressions and tone of voice to identify the user's current emotions. For example, the engine outputs emotion data such as "joy," "sadness," or "surprise."

[1837] Data processing and music generation

[1838] The user's input parameters and emotion data obtained from the emotion recognition engine are sent to the server in a single batch. The server processes this data and sends a request for music generation to the generative AI model. The generative AI model generates music, lyrics, sheet music, vocals, accompaniment, etc., based on these inputs. In doing so, the AI ​​model generates music with characteristics that match the user's emotions. For example, if the analysis results indicate the emotion of "joy," a bright and upbeat song will be generated.

[1839] Return and display of music data

[1840] The music data generated by the AI ​​model is sent back to the server, from which it is transmitted to the user's device. The user's device displays the received data in a playable format, allowing the user to listen to the generated music. If the user is satisfied with the music, clicking the "Save" button sends the request to the server, which then saves the music data to its database.

[1841] Copyright management and buying and selling of music

[1842] The server manages the copyrights to the stored music data and allows users to set the sales conditions for the music. If another user wishes to purchase a song, a purchase request is sent to the server, and the server processes the transaction. The remaining amount after deducting the platform's revenue from the transaction amount is distributed to the user who created the music.

[1843] Specific example

[1844] For example, suppose user A inputs parameters such as "jazz" and "winter love," and the emotion recognition engine detects the emotion "joy." The server sends this data to an AI model, which generates a cheerful jazz song tailored to user A's parameters. This song is displayed on user A's device in a playable format, and user A can listen to, edit, and save the song.

[1845] Example of a prompt

[1846] "Please generate a jazz song. The theme is 'winter love,' and the emotion is 'joy.'"

[1847] As described above, this system allows users to easily generate music and seamlessly manage and sell its copyrights. Furthermore, the introduction of emotion recognition enables the generation of personalized music that responds to the user's emotions.

[1848] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1849] Step 1:

[1850] The user accesses the music generation screen.

[1851] Input: Launch a browser on the user's device and access the music generation system's webpage.

[1852] Specific steps: The user uses a browser on their computer or smartphone to enter a specified web address and open the music generation system screen. They then log in following the interface displayed there.

[1853] Step 2:

[1854] The user enters the parameters of the song.

[1855] Input: Parameters such as song type (e.g., ballad), music genre (e.g., jazz), and theme (e.g., "Winter Love").

[1856] Specific operation: The user enters the required information into the input form on the screen. After entering the parameters, they click the "Complete" button.

[1857] Output: The input parameters are saved on the device and sent to the server as data to proceed with the emotion recognition process.

[1858] Step 3:

[1859] The device activates the camera and microphone.

[1860] Input: Request for use of camera and microphone with user permission.

[1861] Specific action: The user grants permission to use the camera and microphone through a dialog box displayed on the screen.

[1862] Output: Once permission is obtained, the camera and microphone will activate and begin collecting data on the user's facial expressions and voice.

[1863] Step 4:

[1864] The emotion recognition engine analyzes the user's emotions.

[1865] Input: Facial expression data and audio data acquired from the camera and microphone.

[1866] Specific operation: The emotion recognition engine analyzes data in real time to identify the user's emotional state (e.g., joy, sadness, surprise, etc.).

[1867] Output: The emotional data obtained as an analysis result is sent to the server.

[1868] Step 5:

[1869] The server receives user input and sentiment data.

[1870] Input: Parameters entered by the user and emotion data sent from the emotion recognition engine.

[1871] Specific operation: The server receives this data and integrates it as a series of data.

[1872] Output: The integrated data is prepared as input for the AI ​​model.

[1873] Step 6:

[1874] The server sends a request to the AI ​​model.

[1875] Input: Integrated song parameters and emotion data.

[1876] Specific operation: The server creates a specific prompt message for the generated AI model and sends a request for music generation.

[1877] Output: Prompt message sent to the AI ​​model: "Generate a jazz song. The theme is 'winter love,' and the emotion is 'joy.'"

[1878] Step 7:

[1879] Music generation using AI models

[1880] Input: The prompt message sent from the server.

[1881] Specific operation: The AI ​​model generates music based on prompt text. It generates data in various forms, including lyrics, sheet music, vocals, and accompaniment.

[1882] Output: The generated music data is sent back to the server.

[1883] Step 8:

[1884] The server sends the generated music data to the user's terminal.

[1885] Input: Generated music data returned from the AI ​​model.

[1886] Specific operation: The server receives the generated data and sends it to the user's terminal.

[1887] Output: Music data sent to the user's device.

[1888] Step 9:

[1889] The device displays music data.

[1890] Input: Music data sent from the server.

[1891] Specific operation: The device converts the data into a playable format and displays it to the user visually and audibly.

[1892] Output: Music displayed in a playable format.

[1893] Step 10:

[1894] Users edit songs

[1895] Input: Songs displayed in playable formats.

[1896] Specific actions: The user edits the music using the music editing tools as needed. Once the modifications are complete, they submit another save request.

[1897] Output: The edited music data is saved to the device.

[1898] Step 11:

[1899] The device sends a save request to the server.

[1900] Input: Edited song data.

[1901] Specific action: The user clicks the save button, and a save request is sent to the server.

[1902] Output: The server receives the music data.

[1903] Step 12:

[1904] The server saves the music data to the database.

[1905] Input: Music data for which a save request was received.

[1906] Specific operation: The server saves the music data to a database, allowing users to access it again.

[1907] Output: Music data stored in the database.

[1908] Step 13:

[1909] The server manages the copyright of the music.

[1910] Input: Saved music data.

[1911] Specific operation: The server manages copyright information for each song and assigns copyright information to each song.

[1912] Output: Managed copyright information.

[1913] Step 14:

[1914] The user enters the music sales conditions.

[1915] Input: Sales conditions (price, terms of use, etc.) entered on the music sales page.

[1916] Specific operation: The user enters the sales price and terms of use, and sends a sales request to the server.

[1917] Output: Sales conditions data from users.

[1918] Step 15:

[1919] The server processes the transaction.

[1920] Input: Sales conditions data from users and purchase requests from other users.

[1921] Specific operation: The server processes the transaction and grants the buyer the right to use the music. Then, it distributes the remaining amount to the seller after deducting the platform's revenue from the transaction amount.

[1922] Output: Transaction information and distributed profits.

[1923] (Application Example 2)

[1924] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1925] Traditional music generation systems typically generate music based on user-inputted parameters such as song type, genre, and theme, with little consideration given to the user's emotional state. Furthermore, copyright management and trading of generated music were complex, making platform monetization difficult. Additionally, there was a lack of easy ways for users to save, manage, and commercially utilize the music they generated themselves.

[1926] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1927] In this invention, the server includes an input means on which the user can input parameters such as the type of song, music genre, and theme; an emotion recognition means that recognizes the user's emotional state using an emotion engine and reflects the parameters based on that emotion in song generation; and a generation means that generates songs, lyrics, sheet music, vocals, and accompaniment based on the parameters input via the input means and the emotion recognition means. This enables the automatic generation of personalized songs that reflect the user's emotional state, and allows for efficient monetization through copyright management and sales.

[1928] "Input method" refers to a device or interface for users to input parameters such as the type of music, music genre, and theme.

[1929] "Generation means" refers to a system or algorithm that automatically generates music, lyrics, sheet music, vocals, and accompaniment based on parameters input via the input means and emotion recognition means.

[1930] "Emotion recognition means" refers to technology that uses an emotion engine to recognize the user's emotional state and reflects parameters based on that emotion in music generation.

[1931] "Return means" refers to the communication infrastructure and software used to return the music data generated by the generation means to the user.

[1932] "Storage means" refers to a storage device or database system for storing the music data generated by the generation means.

[1933] "Management means" refers to tools or software for managing the copyright of music data stored in the aforementioned storage means.

[1934] "Means of buying and selling" refers to platforms or trading systems for buying and selling the aforementioned copyrights among users.

[1935] "Revenue-generating means" refers to a mechanism or management system for generating revenue for the platform from the profits generated by the aforementioned buying and selling means.

[1936] "Editing means" refers to software or interfaces that allow users to edit generated music data.

[1937] "Authentication means" refers to the systems and protocols used for user registration and authentication.

[1938] A "server" is a computer or network system that runs a music generation system and processes requests from users.

[1939] System Overview

[1940] This system allows users to input parameters such as song type, music genre, and theme, collect emotional data using emotion recognition means, and generate and manage songs based on that data. Specific components include input means, generation means, emotion recognition means, return means, storage means, management means, buying and selling means, and revenue means.

[1941] Program Processing Description

[1942] 1. User interface (input means):

[1943] Users access the music generation screen through a smartphone application. They input parameters such as the type of song, music genre, and theme. For example, they can select options such as "rock" or "summer adventure."

[1944] 2. Emotion recognition means:

[1945] The system analyzes facial images and audio data acquired through the user's camera and microphone. Emotion recognition, in particular, utilizes the "EmotionRecognizer" module to identify the user's emotional state (joy, sadness, surprise, anger, etc.). For example, if the user smiles at the camera, the system determines that they are in a state of "joy."

[1946] 3. Music generation (generation methods):

[1947] The server receives data from the input means and emotion recognition means and generates music using the "MusicGenerator" module. During generation, the original parameters and emotion data are reflected. For example, if rock is selected and the user's emotional state is "joy," an upbeat song with positive elements will be generated.

[1948] 4. Return of generated music (return method):

[1949] The server sends the generated music data back to the user's smartphone. The user can then play and check the received music within the app.

[1950] 5. Preservation means:

[1951] If the user is satisfied with the generated music data, they can click the save button to send a save request to the server. The server receives this request and saves the music data to its database.

[1952] 6. Music management and trading (management methods and trading methods):

[1953] The server manages the copyright of the stored music data. Users enter pricing and sales conditions on the music sales page and submit sales requests. Other users can view and purchase the music published on the sales page.

[1954] 7. Monetization (means of revenue):

[1955] A system is in place where the platform's revenue is deducted from the profits generated by the buying and selling methods, and the remainder is distributed to the sellers.

[1956] Specific example

[1957] For example, suppose a user wants to generate a song with the themes of "rock" and "summer adventure." If the emotion recognition system detects "joy" from the user's facial expression, a song will be generated based on "rock," "summer adventure," and "joy."

[1958] Examples of prompts for generative AI models

[1959] The user wants to generate a song with the themes of "rock" and "summer adventure." The emotion engine detected "joy" from the user's facial expression. Please generate a song based on this.

[1960] This allows users to easily create, save, manage, and sell personalized music. Furthermore, it is expected that generating music based on emotional data will further enhance user satisfaction.

[1961] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1962] Step 1:

[1963] The user accesses the music generation screen through a smartphone application. Here, the user inputs parameters such as the type of music, music genre, and theme (input: user input, output: parameter data). This data is necessary for subsequent processing and serves to specify the user's wishes.

[1964] Step 2:

[1965] The user's facial expressions and audio data are collected through the smartphone's camera and microphone (input: camera video, audio data; output: emotion data). This data is analyzed by an emotion recognition system to identify the user's emotional state (e.g., joy, sadness, surprise, anger). The "EmotionRecognizer" module is used for this analysis.

[1966] Step 3:

[1967] The terminal sends the collected emotion data and the parameter data of the initially entered song to the server (input: emotion data, parameter data; output: integrated data). The server receives this data, integrates it, and creates a request for song generation.

[1968] Step 4:

[1969] The server uses the "MusicGenerator" module to generate music, lyrics, sheet music, vocals, and accompaniment based on integrated data (input: integrated data, output: generated music data). The data processing performed here involves generating the optimal music by considering emotional data, song type, music genre, and theme.

[1970] Step 5:

[1971] The generated music data is sent back from the server to the user's smartphone (input: generated music data, output: data sent to the user's device). The user's smartphone receives this data and displays it on the music playback screen.

[1972] Step 6:

[1973] When a user plays and reviews a generated song and wishes to save it, they click the save button to send a save request to the server (input: user's save request, output: song save confirmation). The server receives this request and saves the song data to the database.

[1974] Step 7:

[1975] A management system operates to manage copyrights for saved music data (input: saved music data, output: copyright management data). Users can enter pricing and sales conditions on the music sales page and submit sales requests.

[1976] Step 8:

[1977] When another user purchases a publicly available song, the buying and selling mechanism is activated and the transaction takes place (input: purchase request, output: song usage rights). The server processes the transaction, deducts the platform's revenue from the generated profit, and distributes it to the seller (input: transaction data, output: revenue data).

[1978] Step 9:

[1979] The platform calculates revenue from the profits generated through buying and selling and distributes it to users (input: revenue data, output: revenue distribution data). This enables smooth transactions between users and allows individuals to commercially use and monetize music they have created.

[1980] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1981] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1982] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[1983] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1984] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[1985] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[1986] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[1987] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[1988] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[1989] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[1990] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[1991] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[1992] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[1993] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1994] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[1995] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[1996] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[1997] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[1998] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[1999] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[2000] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[2001] The following is further disclosed regarding the embodiments described above.

[2002] (Claim 1)

[2003] An input method that allows the user to input parameters such as song type, music genre, and theme,

[2004] A generation means that generates music, lyrics, sheet music, vocals, and accompaniment based on parameters input via the aforementioned input means,

[2005] A return means that returns the music data generated by the generation means to the user,

[2006] A storage means for storing the aforementioned music data,

[2007] A management means for managing copyrights to music data stored in the aforementioned storage means,

[2008] A means of buying and selling the aforementioned copyrights between users,

[2009] A revenue-generating means that uses the profits generated by the aforementioned trading means as revenue for the platform,

[2010] A system that includes this.

[2011] (Claim 2)

[2012] The system according to claim 1, further comprising editing means capable of editing the aforementioned music data.

[2013] (Claim 3)

[2014] The system according to claim 1, further comprising authentication means for performing the aforementioned user registration and authentication.

[2015] "Example 1"

[2016] (Claim 1)

[2017] An input method that allows the user to input parameters such as song type, music genre, and theme,

[2018] A generation means that generates music, lyrics, sheet music, vocals, and accompaniment based on parameters input via the aforementioned input means,

[2019] A return means for returning the music data generated by the generation means to the user,

[2020] A storage means for storing the aforementioned music data,

[2021] A management means for managing copyrights to music data stored in the aforementioned storage means,

[2022] A means of buying and selling the aforementioned copyrights between users,

[2023] A revenue-generating means that uses the profits generated by the aforementioned trading means as revenue for the platform,

[2024] A transaction method that allows other users to view publicly available song information and process purchase requests if they wish to buy it.

[2025] A distribution method in which the transaction amount is divided into fees and the remaining amount is distributed to the seller,

[2026] A system that includes this.

[2027] (Claim 2)

[2028] The system according to claim 1, further comprising editing means capable of editing the aforementioned music data.

[2029] (Claim 3)

[2030] The system according to claim 1, further comprising authentication means for performing the aforementioned user registration and authentication.

[2031] "Application Example 1"

[2032] (Claim 1)

[2033] An input method that allows the user to input parameters such as song type, music genre, and theme,

[2034] A generation means that generates music, lyrics, sheet music, vocals, and accompaniment based on parameters input via the aforementioned input means,

[2035] A return means that returns the music data generated by the generation means to the user,

[2036] A storage means for storing the aforementioned music data,

[2037] A management means for managing copyrights to music data stored in the aforementioned storage means,

[2038] A means of buying and selling the aforementioned copyrights between users,

[2039] A revenue-generating means that uses the profits generated by the aforementioned trading means as revenue for the platform,

[2040] An evaluation method that allows users to evaluate generated music data and create rankings,

[2041] A distribution method for delivering generated music data to other users,

[2042] A system that includes this.

[2043] (Claim 2)

[2044] The system according to claim 1, further comprising editing means capable of editing the aforementioned music data.

[2045] (Claim 3)

[2046] The system according to claim 1, further comprising authentication means for performing the aforementioned user registration and authentication.

[2047] "Example 2 of combining an emotion engine"

[2048] (Claim 1)

[2049] An input method that allows the user to input parameters such as song type, music genre, and theme,

[2050] An emotion recognition means that recognizes the user's emotions and analyzes that emotion data,

[2051] A generation means that generates music, lyrics, sheet music, vocals, and accompaniment based on parameters and emotion data input via the aforementioned input means and emotion recognition means,

[2052] A return means that returns the music data generated by the generation means to the user,

[2053] A storage means for storing the aforementioned music data,

[2054] A management means for managing copyrights to music data stored in the aforementioned storage means,

[2055] A means of buying and selling the aforementioned copyrights between users,

[2056] A revenue-generating means that uses the profits generated by the aforementioned trading means as revenue for the platform,

[2057] A system that includes this.

[2058] (Claim 2)

[2059] The system according to claim 1, further comprising editing means capable of editing the aforementioned music data.

[2060] (Claim 3)

[2061] The system according to claim 1, further comprising authentication means for performing the aforementioned user registration and authentication.

[2062] "Application example 2 when combining with an emotional engine"

[2063] (Claim 1)

[2064] An input method that allows the user to input parameters such as song type, music genre, and theme,

[2065] A generation means that generates music, lyrics, sheet music, vocals, and accompaniment based on parameters input via the aforementioned input means,

[2066] An emotion recognition means that recognizes the user's emotional state using an emotion engine and reflects parameters based on that emotion in music generation,

[2067] A return means that returns the music data generated by the generation means to the user,

[2068] A storage means for storing the aforementioned music data,

[2069] A management means for managing copyrights to music data stored in the aforementioned storage means,

[2070] A means of buying and selling the aforementioned copyrights between users,

[2071] A revenue-generating means that uses the profits generated by the aforementioned trading means as revenue for the platform,

[2072] A system that includes this.

[2073] (Claim 2)

[2074] The system according to claim 1, further comprising editing means capable of editing the aforementioned music data.

[2075] (Claim 3)

[2076] The system according to claim 1, further comprising authentication means for performing the aforementioned user registration and authentication. [Explanation of symbols]

[2077] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. An input method that allows the user to input parameters such as song type, music genre, and theme, A generation means that generates music, lyrics, sheet music, vocals, and accompaniment based on parameters input via the aforementioned input means, A return means that returns the music data generated by the generation means to the user, A storage means for storing the aforementioned music data, A management means for managing copyrights to music data stored in the aforementioned storage means, A means of buying and selling the aforementioned copyrights between users, A revenue-generating means that uses the profits generated by the aforementioned trading means as revenue for the platform, A system that includes this.

2. The system according to claim 1, further comprising editing means capable of editing the aforementioned music data.

3. The system according to claim 1, further comprising authentication means for performing user registration and authentication.

Images