System

Abstract

A system is provided.SOLUTION: A system comprising: means for receiving language data input by a user; means for analyzing the received language data and extracting main keywords; means for generating music data based on the extracted keywords; means for adding performance data and singing data to the generated music data; means for confirming that the added music data does not infringe the rights of others; means for publishing the confirmed music data to a shared site; and means for adding a digital certificate to the published music data.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In music production, an increasing number of users want to create their own original music and songs, but the need for specialized knowledge and skills makes it difficult to do so. There is also the risk of infringing on the rights of existing music and phrases, making it difficult to create music with peace of mind. Furthermore, there is a lack of ways to easily publish completed music and prove one's own copyright. These issues are in need of solutions. [Means for solving the problem]

[0005] To solve the above problems, the present invention provides the following means: A means for receiving language data entered by a user and a means for analyzing the received language data and extracting key keywords are provided. Next, a means for generating music data based on the extracted keywords is provided, and a means for adding performance data and vocal data to the generated music data is provided. Furthermore, a means for verifying that the added music data does not infringe the rights of others is provided, and a means for publishing the verified music data on a sharing site is provided. Finally, a means for adding a digital certificate to published music data is provided, making it possible to prove one's own copyright. This series of means allows for easy and safe music production and the publication of completed compositions.

[0006] "User" means an individual or organization that uses the System to create music.

[0007] "Language Data" refers to text-based information that a user inputs into the system.

[0008] "Analysis" refers to the process of extracting or understanding the meaning of input linguistic data using natural language processing techniques.

[0009] "Keywords" refer to important words and phrases extracted through analysis.

[0010] "Music data" refers to the basic digital information of a song generated based on keywords.

[0011] "Performance data" refers to data including existing performance information and generated performance elements.

[0012] "Singing data" refers to data including existing singing information and generated singing elements.

[0013] "Rights verification" refers to the process of checking that the generated music data does not infringe on existing copyright rights.

[0014] A "sharing site" refers to a platform for publishing music and videos on the Internet.

[0015] "Digital Certificate" refers to a digital record that certifies that a generated musical composition is the copyrighted work of a particular user.

[0016] "Blockchain technology" refers to technology for registering and managing digital certificates using distributed ledger technology. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] ---

[0039] This invention provides a system that allows users to easily create and publish music. This system generates music data based on language data entered by the user, and adds performance and vocal data. It also verifies that the generated music data does not infringe on the rights of others, publishes it on a sharing site, and certifies the copyright by adding a digital certificate.

[0040] Explaining the music production process

[0041] 1. User Input

[0042] Users access the application through their device and write down phrases or images of instruments they have in mind in the input field.

[0043] Example: A user types, "I want a cheerful guitar melody."

[0044] 2. Input Analysis

[0045] The server receives the user's phrase data sent from the terminal and extracts key keywords using natural language processing (NLP) techniques.

[0046] Example: Extract the keywords "bright," "guitar," and "melody."

[0047] 3. Interactive question-answering

[0048] As a next step, the server generates interactive questions for the user based on the analysis results.

[0049] Example: Displaying questions on the screen such as "What tempo and key do you want to play in?"

[0050] 4. Music Generation

[0051] If the user responds with "uptempo C major," the server inputs this information into a music generation AI model and generates basic music data.

[0052] Example: An AI model automatically generates an up-tempo melody and accompaniment in the key of C major.

[0053] 5. Lyric Generation

[0054] If a user requests "I want lyrics written on the theme of love and youth," the server inputs this theme into the lyrics generation model and generates lyrics.

[0055] Example: Lyrics on the theme of "love and youth" are automatically created by AI.

[0056] 6. Adding performance and singing data

[0057] The server adds performance data and singing data to the generated basic music data, using data from existing professional musicians and singers.

[0058] Example: Guitar parts by a professional guitarist and vocal parts by a singer are integrated.

[0059] 7. Confirmation of Rights

[0060] The server scans a music database to check whether the generated music data is similar to existing copyrighted works.

[0061] Example: Making sure the generated songs are not too similar to other existing songs.

[0062] 8. Publishing to a sharing site

[0063] If the user selects "I want to make it public," the server will automatically upload the song using the sharing site's API.

[0064] Example: The server calls the YouTube API and uploads the generated song to YouTube.

[0065] 9. Adding a Digital Certificate

[0066] The server encrypts the metadata associated with the generated song and registers it as an NFT using blockchain technology.

[0067] Example: NFTs registered on the blockchain digitally prove a user's copyright.

[0068] In this way, the system of the present invention provides a variety of functions that allow users to easily and safely create music, share the music they create, and even verify copyright. The above process creates an environment in which users can create and publish original music and songs without specialized knowledge.

[0069] The processing flow will be explained below.

[0070] Step 1:

[0071] A user accesses the system through a terminal and launches an application. The user enters login information for authentication.

[0072] Step 2:

[0073] The user writes the phrase or instrument image they have in mind into the input field on the device. For example, they might type, "I want a bright guitar melody."

[0074] Step 3:

[0075] The device sends the user's input data to the server, which stores the received data and prepares it for analysis.

[0076] Step 4:

[0077] The server applies natural language processing (NLP) technology to the received language data to extract key keywords. As a result of the analysis, the keywords "cheerful," "guitar," and "melody" are identified.

[0078] Step 5:

[0079] The server generates the next dialogue step based on the extracted keywords and sends it to the terminal in the form of a question, such as "What tempo and key do you want to play in?"

[0080] Step 6:

[0081] The user responds to the question from the terminal by entering "Uptempo C major." The terminal sends the response data to the server.

[0082] Step 7:

[0083] The server receives the user's response data and calls the music generation AI model, which inputs tempo and key information to generate the initial song data.

[0084] Step 8:

[0085] The generated initial song data is saved on the server. Next, the user inputs their desired lyrics. For example, they might input "I want lyrics with themes of love and youth."

[0086] Step 9:

[0087] The device sends the user's lyrics request to the server, which then calls the lyric generation AI model and inputs the specified theme. The AI ​​model generates lyrics based on the theme.

[0088] Step 10:

[0089] The generated lyrics are stored on the server and integrated into the initial music data. The server then adds performance data and vocal data.

[0090] Step 11:

[0091] The server selects appropriate performance data from the existing performance database and integrates it into the song. Singing data is also added in the same way.

[0092] Step 12:

[0093] The completed song data is stored on a server, which scans the database to check whether the song data infringes on the rights of others.

[0094] Step 13:

[0095] The server completes the rights verification and prepares to publish the file on the sharing site according to the user's selection. The user selects "I want to publish" on the terminal.

[0096] Step 14:

[0097] The server automatically uploads the music using the API of the sharing site (e.g. YouTube) and notifies the user when the upload is complete.

[0098] Step 15:

[0099] The server generates metadata about the song and uses blockchain technology to generate a digital certificate (NFT), which is then provided to the user.

[0100] The above are the specific processing steps of the music generation AI system.

[0101] Example 1

[0102] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0103] Conventional music production systems require users to configure music data in detail, often requiring specialized knowledge and skills. Furthermore, the process of verifying that the generated music data does not infringe on the rights of others is cumbersome. As a result, music production has been difficult and time-consuming, creating a demand for an environment where anyone can easily create original music and share it safely.

[0104] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0105] In this invention, the server includes means for receiving language data entered by a user, means for analyzing the received language data and extracting key keywords, means for generating music data based on the extracted keywords, means for adding performance data and vocal data to the generated music data, means for verifying that the added music data does not infringe the rights of others, means for publishing the verified music data on a sharing site, means for adding a digital certificate to the published music data, means for scanning and verifying existing databases for the generated music data, and means for encrypting metadata related to the generated music and registering it as an NFT using blockchain technology. This enables users to easily create original music and share it safely and efficiently, even without specialized knowledge.

[0106] "User" refers to a person who intends to use this system to generate music data.

[0107] "Language data" refers to text data such as phrases and instrument images entered by users.

[0108] "Server" refers to a central processing unit that executes the processing of the entire system and manages data.

[0109] "Analysis" refers to the process by which the server processes the received language data and extracts key keywords.

[0110] "Key keywords" refer to important words or phrases extracted from language data.

[0111] "Music data" refers to musical information such as melodies and accompaniments generated by generative AI models.

[0112] "Performance data" refers to musical instrument performance information provided by professional musicians.

[0113] "Singing data" refers to vocal audio information provided by a singer.

[0114] "Ensuring that it does not infringe on the rights of others" refers to the process of checking whether the generated music data is similar to existing copyrighted works.

[0115] "Sharing site" refers to an online platform for sharing generated music data with other users.

[0116] "Digital certificate" refers to electronic evidence that proves the copyright of generated music data.

[0117] "Scanning the database" refers to the process by which the server searches the existing music database to assess similarity.

[0118] "Metadata" refers to additional information associated with the generated song (e.g., title, artist name).

[0119] "Encryption" refers to the process of using cryptography to protect data from unauthorized access.

[0120] "Blockchain technology" refers to a distributed data management technology that prevents data tampering and ensures traceability.

[0121] "NFT" stands for Non-Fungible Token and refers to a means of proving ownership of digital assets on the blockchain.

[0122] This invention provides a system that allows users to easily create and publish music. This system generates music data based on language data entered by the user and has the function of adding performance data and vocal data. It also verifies that the generated music data does not infringe on the rights of others, publishes it on a sharing site, and certifies the copyright by adding a digital certificate.

[0123] Overall system configuration

[0124] Hardware and Software Configuration

[0125] Server: A central processing unit that analyzes, generates, checks, and shares music data. Specifically, it is installed with software libraries for natural language processing (NLP) techniques (e.g., Spacy, NLTK) and music generation AI models (e.g., OpenAI's GPT-3, Google's Magenta).

[0126] Terminal: A device that a user accesses and inputs data from. Examples include smartphones, tablet PCs, and desktop PCs.

[0127] Sharing site: An online platform for publishing generated music data. Examples include YouTube and SoundCloud.

[0128] Data Processing Description

[0129] 1. User Input

[0130] The user accesses the application through their device and writes down the phrases or instrument images they have in mind in the input field. For example, the user might enter, "I want a bright guitar melody."

[0131] 2. Input analysis and keyword extraction

[0132] The server receives the user's phrase data sent from the device and extracts key keywords using natural language processing technology. Specifically, the server analyzes the input data and extracts keywords such as "cheerful," "guitar," and "melody."

[0133] 3. Interactive question-answering

[0134] The next step is for the server to generate interactive questions for the user based on the analysis results, such as "What tempo and key do you want to play in?", and display them to the user.

[0135] 4. Music Generation

[0136] If the user responds with "uptempo C major," the server inputs this information into the music generation AI model and generates basic music data. As a specific example, the server inputs the prompt "Generate a melody and accompaniment in uptempo C major" into the generation AI model and obtains an automatically generated melody and accompaniment.

[0137] 5. Lyric Generation

[0138] If a user requests "I want lyrics with the theme of love and youth," the server inputs this theme into the lyrics generation model and generates lyrics. As a specific example, the server inputs a prompt sentence with the theme of "love and youth" into the lyrics generation model and obtains the generated lyrics.

[0139] 6. Adding performance and singing data

[0140] The server adds performance data and vocal data to the generated basic music data. This data is created using data from existing professional musicians and singers. For example, the server integrates guitar parts by existing professional musicians and vocal parts by existing professional singers into the basic music data.

[0141] 7. Confirmation of Rights

[0142] The server scans a music database to check whether the generated music data is similar to existing copyrighted works. For example, the server crawls a music database and uses a similarity checking algorithm to evaluate the similarity between the generated music and existing music.

[0143] 8. Publishing to a sharing site

[0144] If the user selects "I want to make it public," the server automatically uploads the music using the sharing site's API. For example, the server calls the sharing site's API (e.g., YouTube API), provides the necessary authentication information, and then uploads the generated music file.

[0145] 9. Adding a Digital Certificate

[0146] The server encrypts metadata related to the generated music and registers it as an NFT using blockchain technology. For example, the server encrypts the music metadata and generates a transaction on a blockchain platform (e.g., Ethereum) to register the NFT and obtain a digital certificate.

[0147] In this way, the system of the present invention provides a variety of functions for users to easily and safely create music, share the music they create, and verify copyright, creating an environment in which users can create and publish original music and songs without specialized knowledge.

[0148] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0149] Specific processing steps of the system

[0150] Step 1:

[0151] The user accesses the application using a terminal and writes down the phrases or instrument images that come to mind in the input field.

[0152] Input data: Language data entered by the user (e.g., "Please include a cheerful guitar melody")

[0153] Output data: Transmitted language data

[0154] Step 2:

[0155] The server receives the user's language data sent from the device, then analyzes the data using natural language processing (NLP) technology to extract key keywords.

[0156] Input data: Transmitted language data

[0157] Data processing: Using NLP technology, key keywords such as "upbeat," "guitar," and "melody" are extracted.

[0158] Output data: extracted main keywords

[0159] Step 3:

[0160] As a next step, the server generates interactive questions for the user based on the extracted keywords, which allow the user to confirm detailed settings for the music data to be generated.

[0161] Input data: extracted main keywords

[0162] Data processing: Question generation (e.g., "What tempo and key do you play in?")

[0163] Output data: Generated question

[0164] Step 4:

[0165] The user inputs a response to a question from the server.

[0166] Input data: User response (e.g., "Uptempo C major")

[0167] Output data: User response data

[0168] Step 5:

[0169] The server uses the user's response data to input prompt sentences into the music generation AI model and generate basic music data.

[0170] Input data: User response data (e.g., "Generate a melody and accompaniment in uptempo C major.")

[0171] Data Computing: Running Music Generation AI Models

[0172] Output data: generated melody and accompaniment

[0173] Step 6:

[0174] If the user requests, the server also generates additional musical parts, such as lyrics. The server inputs the user's request into the lyrics generation model and generates lyrics.

[0175] Input data: User request (e.g., "Please write lyrics with themes of love and youth")

[0176] Data calculation: Running the lyrics generation model

[0177] Output: Generated lyrics

[0178] Step 7:

[0179] The server adds performance data and singing data to the generated basic music data, using data from existing professional musicians and singers.

[0180] Input data: basic music data, performance data of professional musicians, singing data of professional singers

[0181] Data processing: Data integration

[0182] Output data: Finished music data

[0183] Step 8:

[0184] The server scans existing music databases to ensure that the generated music data does not infringe on the rights of others.

[0185] Input data: Completed music data

[0186] Data calculation: scanning music databases and checking similarities

[0187] Output data: Infringement confirmation results

[0188] Step 9:

[0189] If the user selects "I want to make it public," the server will automatically upload the song using the sharing site's API.

[0190] Input data: Completed music data, API credentials for the sharing site

[0191] Data Calculation: Upload data via API

[0192] Output data: Songs published on a sharing site

[0193] Step 10:

[0194] The server encrypts the metadata associated with the generated song and registers it as an NFT using blockchain technology.

[0195] Input data: Song metadata

[0196] Data computation: encryption and blockchain registration

[0197] Output data: Digital certificate (NFT)

[0198] Through these steps, the system provides users with a variety of functions to easily and safely create music, share the music they create, and even prove their copyright.

[0199] (Application example 1)

[0200] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0201] Conventional music production systems lack the means for users to easily create and publish music, and to protect the copyright of that music. This makes it difficult for users without specialized knowledge to create original music, resulting in a high barrier to entry for music production. Furthermore, there is a lack of means to prove the copyright of published music, which means users are constantly exposed to the risk of copyright infringement. There is a need to solve these issues.

[0202] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0203] In this invention, the server includes: means for receiving language data input by a user; means for analyzing the received language data and extracting key keywords; means for generating music data based on the extracted keywords; means for adding performance data and vocal data to the generated music data; means for verifying that the added music data does not infringe the rights of others; means for publishing the verified music data on a distribution platform; means for attaching a digital certificate to the published music data using blockchain technology; means for analyzing the user's input using natural language processing technology and generating a prompt; and means for inputting the prompt into a generative AI model to generate music data and lyrics. This allows users to easily create music and publish the music on a distribution platform, while protecting their copyrights with digital certificates using blockchain technology.

[0204] "Language data entered by the user" refers to text information entered by the user through the application when the user wishes to create music.

[0205] "Major keywords" are words and phrases that are important for music generation and are extracted from the input linguistic data.

[0206] "Music data generation" means that the music generation AI model automatically creates the basic structure of a song based on the extracted main keywords.

[0207] "Performance data" is data that includes information about performances by musical instruments, and is an element that is added to music data.

[0208] "Vocal data" is data that includes vocal performances accompanied by lyrics, and is an element that is added to music data.

[0209] "Confirmation of non-infringement" is the process of checking whether the generated music data is highly similar to existing copyrighted works.

[0210] A "distribution platform" is an online service or system used to make generated music data available to the public.

[0211] "Blockchain technology" is a distributed ledger technology that links each data block to prevent data tampering and increase reliability.

[0212] A "Digital Certificate" is a digital certificate for certifying copyright that contains metadata related to published music data and is generated using blockchain technology.

[0213] "Natural language processing technology" is an artificial intelligence technology for analyzing, understanding, and generating human language, and is used to extract key keywords from input language data.

[0214] A "prompt" is a sentence input into a music generation AI model, and is an instruction statement that defines the characteristics of the music data to be generated.

[0215] A "generative AI model" is an artificial intelligence model used to automatically generate music data and lyrics.

[0216] The present invention relates to a system that allows users to easily create and publish music. Details of this system and an embodiment thereof will be described below.

[0217] Music production system configuration

[0218] The system includes the following main functions:

[0219] 1. Means for receiving user input:

[0220] The user uses their own device (such as a smartphone or head-mounted display) to input phrases or images of musical instruments that come to mind into the application. For example, the user might input, "I want a bright guitar melody." This text data is then sent to the server.

[0221] 2. Linguistic data analysis and keyword extraction:

[0222] The server analyzes the received language data using natural language processing technology (such as OpenAI's API) and extracts key keywords, such as "cheerful," "guitar," and "melody."

[0223] 3. Generate prompt sentence:

[0224] The server generates a prompt based on the analysis results. This prompt is a statement that instructs the music generation AI model. For example, the prompt might say, "The music to be generated will be up-tempo, in the key of C major, and based on a bright guitar melody."

[0225] 4. Music data and lyrics generation:

[0226] The server inputs the generated prompt sentence into the AI ​​model and generates music data and lyrics if necessary. For example, if a user requests "I want lyrics on the theme of love and youth," the lyrics generation AI model will generate lyrics on the theme of "love and youth."

[0227] 5. Adding performance and vocal data:

[0228] The server adds performance data and vocal data to the generated basic music data. This data is created using data from existing professional musicians and singers. For example, guitar parts by a professional guitarist and vocal parts by a singer are integrated.

[0229] 6. Acknowledgement of Rights:

[0230] The server scans an existing music database to determine whether the generated music data is similar to other existing copyrighted works. It verifies that the generated music does not have a high degree of similarity to other existing music.

[0231] 7. Publishing your music:

[0232] If the user selects "I want to make it public," the server will automatically upload the created song using the distribution platform's API (such as YouTube API).

[0233] 8. Adding a digital certificate:

[0234] The server then encrypts the metadata associated with the generated music and registers it as a digital certificate (NFT) using blockchain technology, digitally certifying the user's copyright.

[0235] Specific examples of functions

[0236] For example, if the user types "I want a cheerful guitar melody," the system will generate the following prompt:

[0237] The music it generates is upbeat in tempo, set in the key of C major, and based on bright, guitar-driven melodies.

[0238] Similarly, if the user requests "Please write lyrics on the themes of love and youth," we generate the following prompt:

[0239] Generate lyrics on the themes of love and youth.

[0240] With these functions, the system of the present invention allows users to easily create music, publish it on a distribution platform, and protect copyright with digital certificates using blockchain technology.

[0241] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0242] Step 1:

[0243] The server receives the user's input data from the device. For example, the user might say, "I want a cheerful guitar melody," and the device sends that data to the server. This input data becomes the basis for the next analysis step.

[0244] Step 2:

[0245] The server analyzes the received language data using natural language processing technology. Specifically, it uses OpenAI's API to extract key keywords from the input text. For example, "cheerful," "guitar," and "melody" are extracted. This keyword extraction clarifies the elements of music generation.

[0246] Step 3:

[0247] The server generates a prompt based on the extracted keywords. For example, it creates a prompt such as, "The music to be generated will be up-tempo, in the key of C major, and based on a bright guitar melody." This prompt becomes the instruction input to the music generation AI model.

[0248] Step 4:

[0249] The server inputs the generated prompt into a music generation AI model to generate music data. At this time, the tempo, key, type of instrument, etc. are specified. For example, the AI ​​model automatically generates a melody and accompaniment based on an up-tempo C major key. The generated music data is then passed to the next step.

[0250] Step 5:

[0251] The server inputs the theme into the lyric generation model based on the user's request and generates lyrics. For example, if a user requests "I want lyrics with the theme of love and youth," the lyric generation AI model will generate lyrics with the theme of "love and youth." This lyric data is then integrated with the music data.

[0252] Step 6:

[0253] The server adds performance data and vocal data to the generated music data. Specifically, it integrates data from professional musicians and singers. For example, a guitarist adds a guitar part and a vocalist adds a vocal part. This results in more complete music data.

[0254] Step 7:

[0255] The server scans the existing music database to verify the rights of the generated music data. Here, it checks that the generated song does not have a high similarity to other existing songs. If there is a high similarity, it notifies the user. The verified data proceeds to the next step.

[0256] Step 8:

[0257] The server then publishes the verified music data to the distribution platform. Specifically, the music data is automatically uploaded using the distribution platform's API. For example, the server calls the distribution platform's API and performs the procedure to upload the music.

[0258] Step 9:

[0259] The server then adds a digital certificate to the published music data. Specifically, it encrypts the metadata associated with the generated music and registers it as an NFT using blockchain technology. This digitally certifies the user's copyright.

[0260] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0261] ---

[0262] This invention provides a system that allows users to easily create and publish music. This system generates music data based on language data entered by the user, and adds performance and vocal data. It also verifies that the generated music data does not infringe on the rights of others, publishes it on a sharing site, and certifies the copyright by adding a digital certificate.

[0263] Additionally, the present invention incorporates an emotion engine that recognizes the user's emotions and has the ability to adjust the tone and content of music data and lyrics based on the recognized emotions.

[0264] Explaining the music production process using the emotion engine

[0265] 1. User Input

[0266] A user accesses the system through a terminal and launches the application. After entering their login information and performing authentication, they write a phrase or an image of an instrument in the input field. For example, they might enter, "I want a bright guitar melody."

[0267] 2. Emotional Recognition

[0268] The device sends the user's input data to the server, which stores the data and uses an emotion engine to recognize emotions from the input data. For example, emotions such as "joy" or "liveliness" can be detected from the user's input.

[0269] 3. Input Analysis

[0270] The server applies natural language processing (NLP) technology to the received language data to extract key keywords. The keywords extracted as a result of the analysis are "cheerful," "guitar," and "melody."

[0271] 4. Interactive question-answering

[0272] Based on the extracted keywords and the emotion recognition results, the server generates the next dialogue step and sends it to the terminal in the form of a question, such as "What tempo and key do you want to play in?"

[0273] 5. Music Generation

[0274] If the user responds with "uptempo C major," the server inputs this information into the music generation AI model to generate basic music data, and adjusts the tone and structure of the music data based on the emotion recognition results.

[0275] 6. Lyric Generation

[0276] If a user requests lyrics with themes of love and youth, the server will call the lyric generation AI model and input the specified theme. The AI ​​model will generate lyrics based on the theme and the recognized emotions. For example, it will automatically generate lyrics with a bright tone that conveys joy and vitality.

[0277] 7. Addition of performance and singing data

[0278] The server adds performance data and vocal data to the generated basic music data, using data from existing professional musicians and singers. For example, guitar parts by a professional guitarist and vocal parts by a singer are integrated.

[0279] 8. Confirmation of Rights

[0280] The server scans a music database to ensure that the generated music data is not similar to existing copyrighted works, for example, to ensure that the generated song does not have a high degree of similarity to other existing songs.

[0281] 9. Publishing to a sharing site

[0282] If the user selects "I want to make it public," the server automatically uploads the song using the API of the sharing site. For example, the server calls the YouTube API and uploads the created song to YouTube.

[0283] 10. Adding a Digital Certificate

[0284] The server generates metadata about the song and uses blockchain technology to generate a digital certificate (NFT), which is then provided to the user. For example, the NFT registered on the blockchain digitally certifies the user's copyright.

[0285] The system of this invention allows users to easily and safely create music, share the music they create, and even prove their copyright. The introduction of an emotion engine makes it possible to generate music and lyrics that match the user's emotions, providing a more personalized musical experience.

[0286] The processing flow will be explained below.

[0287] Music production process using emotion engine

[0288] Step 1:

[0289] A user accesses the system through a terminal and launches an application. The user enters login information for authentication.

[0290] Step 2:

[0291] The user writes the phrase or instrument image they have in mind into the input field on the device. For example, they might type, "I want a bright guitar melody."

[0292] Step 3:

[0293] The device sends the user's input data to the server, which stores the received data and uses an emotion engine to recognize emotions from the input data.

[0294] Step 4:

[0295] The server analyzes the received data, and the emotion engine recognizes the user's emotions. For example, if someone says, "I want a bright guitar melody," the engine can detect emotions such as "joy" or "liveliness."

[0296] Step 5:

[0297] The server uses natural language processing (NLP) techniques to extract key keywords from the input data, such as "upbeat," "guitar," and "melody."

[0298] Step 6:

[0299] Based on the keywords and the recognized emotions, the server generates the next dialogue step and sends a question to the terminal, for example, "What tempo and key do you want to play in?"

[0300] Step 7:

[0301] The user responds to the question from the terminal with "Uptempo C major." The terminal then sends this response data to the server.

[0302] Step 8:

[0303] The server inputs the tempo, key, and recognized emotion into the music generation AI model to generate initial song data. For example, it creates an up-tempo melody in C major that reflects "joy."

[0304] Step 9:

[0305] The user requests, "I want you to write lyrics on the theme of love and youth" on the device. The device sends the request data to the server.

[0306] Step 10:

[0307] The server calls the lyric generation AI model and inputs the specified theme and emotion data. For example, it generates lyrics based on the theme of "love and youth" and the emotion of "joy."

[0308] Step 11:

[0309] The generated lyrics are stored on the server and integrated into the initial music data. At this stage, the server adds performance data and vocal data.

[0310] Step 12:

[0311] The server selects appropriate performance data from an existing performance database and integrates it into the song. Similarly, vocal data is added, for example, guitar parts by a professional guitarist or vocal parts by a singer.

[0312] Step 13:

[0313] The completed song data is stored on a server, which checks it against a music database to ensure it does not infringe on the rights of others.

[0314] Step 14:

[0315] If the server completes the rights verification and the user selects publishing, the server automatically uploads the song using the sharing site's API. For example, the server uploads the song to YouTube using the YouTube API.

[0316] Step 15:

[0317] The server generates metadata about the song and uses blockchain technology to create a digital certificate (NFT), which is then provided to the user.

[0318] The above are the specific processing steps of the system that generates and publishes music using an emotion engine.

[0319] Example 2

[0320] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0321] Conventional music production systems make it difficult for users to easily create music and lyrics that match their emotions, and the process of verifying that the generated music data does not infringe on the copyrights of others takes a lot of time and effort.Furthermore, there is a lack of means to publish the created music data and prove its copyright.

[0322] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0323] In this invention, the server includes means for receiving language data entered by a user, means for analyzing the received language data and extracting key keywords, means for generating music data based on the extracted keywords and the recognized emotion, means for adding performance data and vocal data to the generated music data, means for verifying that the added music data does not infringe the rights of others, means for publishing the verified music data on a sharing site, means for adding a digital certificate to the published music data, means for recognizing the user's emotion from the received language data, and means for adjusting the tone and content of the music data based on the recognized emotion. This allows users to easily create music and lyrics that match their emotions, enables the safe generation and publication of music data that does not infringe the rights of others, and furthermore, ensures that copyright is proven.

[0324] "User" refers to a person who uses the system to create and publish music and lyrics.

[0325] "Means for receiving" refers to a method or device by which the system receives language data input by a user.

[0326] "Means for analyzing" refers to a method or device that analyzes received language data using natural language processing (NLP) techniques and extracts key keywords.

[0327] "Key Keywords" refer to words and phrases that extract important information or instructions from the analyzed linguistic data.

[0328] "Emotion recognition" refers to the process of identifying a user's emotion from received linguistic data.

[0329] "Music data" refers to electronic data that includes generated musical components such as melodies, chords, and rhythms.

[0330] "Performance data" refers to specific performance information by musical instruments or other sound sources that is added to music data.

[0331] "Vocal data" refers to information about the singer's singing voice and vocal parts that is added to music data.

[0332] "Rights verification" refers to the process of checking whether the generated music data is similar to existing copyrighted works.

[0333] "Sharing site" refers to a web platform or online service for publishing generated music data.

[0334] A "digital certificate" refers to an electronic certificate generated using blockchain technology or other means to certify the copyright of published music data.

[0335] "Tone" refers to the tone and quality of an instrument used in music.

[0336] "Content" refers to the subject matter, themes, and expressions contained in the music data and lyrics.

[0337] This is a system that allows users to easily create and publish music based on their emotions. This system generates music data and lyrics based on user requests, adds performance data and vocal data to them, and after verifying that the copyrights of others are not being infringed, publishes the music on a sharing site and adds a digital certificate to prove copyright.

[0338] Hardware and software used

[0339] The system consists of a server, a terminal, and a user. The server uses a high-performance data processing device, including an AI model for executing natural language processing (NLP) techniques and a generative AI model for music generation and lyric generation. Specifically, the following software is used:

[0340] Natural Language Processing (NLP) technologies: Python NLP libraries (e.g., NLTK, spaCy)

[0341] Music generation AI model: Deep learning model using TensorFlow or PyTorch

[0342] Lyric generation AI models: OpenAI GPT-3 and GPT-4

[0343] Publishing to sharing sites uses APIs to connect to external services, such as YouTube API for publishing songs and blockchain technology for generating digital certificates.

[0344] Program processing explanation

[0345] User Input

[0346] Users access the system from their terminals, log in, and make specific requests to create music (e.g., "Please add a cheerful guitar melody").

[0347] Emotion recognition

[0348] The device sends the user's input data to the server, which uses an emotion engine to identify the user's emotion from this data. NLP technology is used in this process.

[0349] Data analysis and keyword extraction

[0350] The server uses NLP techniques to analyze the user's request and extract key keywords (e.g., "upbeat," "guitar," "melody").

[0351] Interactive question and answering

[0352] Based on the extracted keywords and the recognized emotions, the server generates the next dialogue step and sends it to the terminal in the form of a question (e.g., "What tempo and key do you want to play in?").

[0353] Basic music data generation

[0354] When a user responds with "uptempo C major," the server inputs this information into a music generation AI model to generate basic music data, and adjusts the tone and structure of the music data based on the emotion recognition results.

[0355] Lyrics generation

[0356] When a user requests lyrics on the theme of love and youth, the server calls a lyrics generation AI model and generates lyrics based on the specified theme and recognized emotions (e.g., lyrics with a bright tone).

[0357] Addition of performance and singing data

[0358] The server adds performance data and vocal data by professional musicians and singers to the generated music data (e.g., guitar parts by a professional guitarist and vocal parts by a singer).

[0359] Confirmation of rights

[0360] The server scans the generated music data to ensure it does not infringe on the rights of others (e.g., by using a music database to check for similarity to existing copyrighted works).

[0361] Publishing to a shared site

[0362] When the user selects "I want to make it public," the server uses the sharing site's API to make the song public (e.g., upload it to YouTube).

[0363] Attaching a Digital Certificate

[0364] The server generates metadata about the song and uses blockchain technology to generate a digital certificate (NFT), which users receive and their copyright is protected.

[0365] Examples and prompts

[0366] For example, a user might request, "I want a cheerful guitar melody," and the server would generate music data based on this request. For the user's next request, the server would use the AI ​​model to generate corresponding lyrics, which would then be published and digitally authenticated.

[0367] Prompt Sentence Examples

[0368] "Please identify the emotion from the user's input, such as 'I want a bright guitar melody,' and input 'joy, up-tempo, C major, bright guitar melody' into the generative AI model to generate a song."

[0369] This allows users to easily create music and lyrics that match their emotions, safely release them, and protect their copyrights.

[0370] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0371] Step 1: User login and request submission

[0372] Users access the system through a terminal and enter their login information for authentication. After successful authentication, they input specific requests for music production (e.g., "Please add a cheerful guitar melody.") The input data is received by the terminal and sent to the server.

[0373] Input: User login information and request (e.g., "Please add a cheerful guitar melody")

[0374] Output: The user's authenticated request data sent to the server.

[0375] Step 2: Recognize emotions

[0376] The server passes the received request data to the emotion engine, which uses NLP technology to identify emotions, extracting emotions such as "joy" and "liveliness" from the word "cheerful."

[0377] Input: User request data

[0378] Output: Identified emotion data (e.g., "joy")

[0379] Step 3: Data analysis and keyword extraction

[0380] After emotion recognition, the server uses NLP technology to analyze the user's request and extract key keywords, such as "cheerful," "guitar," and "melody."

[0381] Input: User request data with identified emotions

[0382] Output: Extracted keyword data (e.g., "cheerful," "guitar," "melody")

[0383] Step 4: Interactive question-answering

[0384] The server generates the next dialogue step based on the extracted keywords and the identified emotions, which is sent to the terminal, where the user is prompted with a question such as, "What tempo and key do you want to play in?"

[0385] Input: extracted keywords and identified emotions

[0386] Output: Questions to ask the user (e.g., "What tempo and key do you want to play in?")

[0387] Step 5: Generate basic music data

[0388] When a user responds with "uptempo C major," the server inputs this response into a music generation AI model to generate basic music data. The server also adjusts the tone and structure of the music data based on the identified emotions.

[0389] Input: User response data (e.g., "Uptempo C major") and identified emotion

[0390] Output: Generated basic music data

[0391] Step 6: Generate lyrics

[0392] When a user requests lyrics on the theme of love and youth, the server calls the lyric generation AI model and generates lyrics based on the specified theme and identified emotions. For example, it generates lyrics with a bright tone that evokes joy.

[0393] Input: User's theme request (e.g., "Love and Youth") and identified emotions

[0394] Output: Generated lyrics data

[0395] Step 7: Adding performance and singing data

[0396] The server adds performance data and singing data by professional musicians and singers to the generated music data. For example, guitar parts by a professional guitarist and vocal parts by a singer are integrated.

[0397] Input: Generated basic music data and lyrics data

[0398] Output: Complete music data with performance and vocal data added

[0399] Step 8: Entitlement Verification

[0400] The server scans the music database to ensure the generated music data does not infringe on the rights of others, and in the process checks to ensure that it is not too similar to existing songs.

[0401] Input: Complete music data

[0402] Output: Secure music data with rights verification

[0403] Step 9: Publish to a sharing site

[0404] When the user selects "I want to publish," the device sends this instruction to the server. The server publishes the song using the sharing site's API. For example, the server calls the YouTube API to upload the song to YouTube.

[0405] Input: User's publishing instructions

[0406] Output: Song data uploaded to a sharing site

[0407] Step 10: Attach a Digital Certificate

[0408] The server generates metadata about the song and uses blockchain technology to generate a digital certificate (NFT), which the user receives, digitally protecting the copyright of the song.

[0409] Input: Song data published on a sharing site

[0410] Output: Generated digital certificate (NFT)

[0411] (Application example 2)

[0412] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0413] Current music production systems lack sufficient means for users to easily generate music and lyrics that respond to their emotions and publish them to content distribution services. Furthermore, there is no established method for digitally verifying the copyright of the generated music data. This creates problems for users to create personalized music and share their creations securely and easily.

[0414] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving language data entered by a user, means for analyzing the received language data and generating music data based on key keywords and recognized emotions, and means for adding performance data and vocal data to the generated music data. This allows users to easily generate music and lyrics that correspond to their emotions and publish them on content distribution services. Furthermore, by verifying that the generated music data does not infringe on the rights of others and attaching a digital certificate, copyright verification can be easily performed.

[0415] "User" means any individual or organization that uses the System to create and share music.

[0416] "Language data" refers to text data such as sentences and phrases that users input into the system.

[0417] "Analysis" refers to the process of analyzing received linguistic data and extracting key keywords and information.

[0418] "Emotion" refers to psychological states such as joy, sadness, and excitement that are recognized from the user's input data.

[0419] "Music data" refers to digital data containing the melody, rhythm, and harmony of the generated music.

[0420] "Performance data" refers to digital data of musical instrument performances that are added to music data.

[0421] "Vocal data" refers to digital vocal data that is added to music data.

[0422] "Ensuring that the rights of others are not infringed" refers to the process of ensuring that the generated music data is not similar to existing copyrighted works.

[0423] "Content distribution service" refers to a platform for publishing and sharing generated music data on the Internet.

[0424] "Digital Certificate" refers to an electronic certificate generated using blockchain technology to protect the copyright of generated music data.

[0425] The system for implementing the present invention allows users to easily create and publish music based on language data entered by the user. A series of steps will be explained in detail below.

[0426] 1. System Configuration

[0427] It consists of three main modules: the server, the terminal, and the user. The server is a central processing unit that actually generates music and processes data. The terminal is a device operated by the user, such as a smartphone. The user is the person who creates music.

[0428] 2. User Input

[0429] A user accesses the system through a terminal and launches the application. After entering their login information for authentication, they write a phrase or an image of an instrument in the input field. For example, they might enter, "I want a bright guitar melody."

[0430] 3. Emotional Recognition

[0431] The device sends the user's input data to the server. The server stores the received data and uses an emotion recognition engine called EmotionRecognition to recognize emotions from the input data. For example, emotions such as "joy" or "liveliness" can be detected from the user's input.

[0432] 4. Input Analysis

[0433] The server applies Natural Language Processing (NLP) technology to the received language data to extract key keywords. The keywords extracted as a result of the analysis are "cheerful," "guitar," and "melody."

[0434] 5. Interactive Question-Answering

[0435] Based on the extracted keywords and the emotion recognition results, the server generates the next dialogue step and sends it to the terminal in the form of a question, such as "What tempo and key do you want to play in?"

[0436] 6. Music Generation

[0437] If the user responds with "uptempo C major," the server inputs this information into a music generation AI model called MusicGeneration to generate basic music data, and adjusts the tone and structure of the music data based on the results of emotion recognition.

[0438] 7. Lyric Generation

[0439] If a user requests lyrics with themes of love and youth, the server calls a lyric generation AI model called LyricsGeneration and inputs the specified theme. The AI ​​model generates lyrics based on the theme and the recognized emotions. For example, it automatically generates lyrics with a bright tone that conveys joy and vitality.

[0440] 8. Addition of performance and singing data

[0441] The server adds performance data and vocal data to the generated basic music data, using data from existing professional musicians and singers. For example, guitar parts by a professional guitarist and vocal parts by a singer are integrated.

[0442] 9. Confirmation of Rights

[0443] The server uses a module called CopyrightChecker to check whether the generated music data is similar to existing copyrighted works, for example, to ensure that the generated music does not have a high similarity to other existing music.

[0444] 10. Publication

[0445] If the user selects "I want to make it public," the server automatically uploads the song using the API of a content distribution service. For example, the server calls the API of a sharing site and uploads the created song to that site.

[0446] 11. Adding a Digital Certificate

[0447] The server generates metadata about the song and uses blockchain technology to generate a digital certificate, which is then provided to the user. For example, the certificate registered on the blockchain digitally certifies the user's copyright.

[0448] Examples of concrete examples and prompts

[0449] Example 1: When a user inputs "I want a bright guitar melody," music with a pop tempo is automatically generated.

[0450] Example 2: If the user selects "Love and Youth" as the theme, cheerful lyrics on the theme of love and youth are automatically generated.

[0451] Example prompt sentence:

[0452] "I want a bright guitar melody."

[0453] "I want you to write lyrics with themes of love and youth"

[0454] "Uptempo C major"

[0455] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0456] Step 1:

[0457] A user accesses the system through a terminal and launches an application. The user enters login information for authentication, then writes a phrase or an image of an instrument in the input field. The input is a prompt such as "Please add a cheerful guitar melody." The terminal sends this input data to the server.

[0458] Step 2:

[0459] The server stores the received user input data. Based on the stored data, an emotion recognition engine called EmotionRecognition is used to recognize emotions from the input data. For example, emotions such as "joy" or "liveliness" are recognized from the input. The input data is text data, and the output is emotional data.

[0460] Step 3:

[0461] The server applies Natural Language Processing (NLP) technology to the received language data to extract key keywords. The keywords extracted as a result of the analysis are "cheerful," "guitar," and "melody." Text analysis is used to extract these keywords, with the input being language data and the output being keyword data.

[0462] Step 4:

[0463] The server generates the next dialogue step based on the extracted keywords and the emotion recognition results and sends it to the terminal in the form of a question. For example, it sends a question such as, "What tempo and key do you want to play at?" The input is keywords and emotion data, and the output is the dialogue step question.

[0464] Step 5:

[0465] The user answers questions from the server interactively through the terminal. For example, the user might answer "uptempo C major." The terminal then sends this answer data back to the server. The input is the user's answer, and the output is text data sent to the server.

[0466] Step 6:

[0467] The server inputs user response data and emotional data into a music generation AI model called MusicGeneration to generate basic music data. It also adjusts the tone and structure of the music data based on the results of emotion recognition. The input data are tempo, key, and emotional data, and the output is music data.

[0468] Step 7:

[0469] If the user again requests "Please create lyrics with the theme of love and youth," the server will call a lyrics generation AI model called LyricsGeneration and input the specified theme. The AI ​​model will generate lyrics based on the theme and the recognized emotion. The input is theme and emotion data, and the output is lyric data.

[0470] Step 8:

[0471] The server adds performance and vocal data to the generated basic music data. This uses data from existing professional musicians and singers. For example, guitar parts by a professional guitarist and vocal parts by a singer are integrated. The input is music data and additional performance and vocal data, and the output is the completed music data.

[0472] Step 9:

[0473] The server checks whether the generated music data is similar to existing copyrighted works through a module called CopyrightChecker. For example, it checks whether the generated music is too similar to other existing music. The input is the completed music data, and the output is the result of copyright check.

[0474] Step 10:

[0475] If the user selects "I want to publish," the server automatically uploads the song using the API of a content distribution service. For example, the server calls the API of a sharing site and uploads the created song to that site. The input is the music data and the publishing request, and the output is the upload result to the sharing site.

[0476] Step 11:

[0477] The server generates metadata about the music and generates a digital certificate using blockchain technology. The generated certificate information is provided to the user. For example, the certificate registered on the blockchain digitally certifies the user's copyright. The input is the metadata of the music data, and the output is a digital certificate.

[0478] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0479] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0480] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0481] [Second embodiment]

[0482] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0483] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0484] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0485] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0486] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0487] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0488] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0489] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0490] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0491] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0492] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0493] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0494] ---

[0495] This invention provides a system that allows users to easily create and publish music. This system generates music data based on language data entered by the user, and adds performance and vocal data. It also verifies that the generated music data does not infringe on the rights of others, publishes it on a sharing site, and certifies the copyright by adding a digital certificate.

[0496] Explaining the music production process

[0497] 1. User Input

[0498] Users access the application through their device and write down phrases or images of instruments they have in mind in the input field.

[0499] Example: A user types, "I want a cheerful guitar melody."

[0500] 2. Input Analysis

[0501] The server receives the user's phrase data sent from the terminal and extracts key keywords using natural language processing (NLP) techniques.

[0502] Example: Extract the keywords "bright," "guitar," and "melody."

[0503] 3. Interactive question-answering

[0504] As a next step, the server generates interactive questions for the user based on the analysis results.

[0505] Example: Displaying questions on the screen such as "What tempo and key do you want to play in?"

[0506] 4. Music Generation

[0507] If the user responds with "uptempo C major," the server inputs this information into a music generation AI model and generates basic music data.

[0508] Example: An AI model automatically generates an up-tempo melody and accompaniment in the key of C major.

[0509] 5. Lyric Generation

[0510] If a user requests "I want lyrics written on the theme of love and youth," the server inputs this theme into the lyrics generation model and generates lyrics.

[0511] Example: Lyrics on the theme of "love and youth" are automatically created by AI.

[0512] 6. Adding performance and singing data

[0513] The server adds performance data and singing data to the generated basic music data, using data from existing professional musicians and singers.

[0514] Example: Guitar parts by a professional guitarist and vocal parts by a singer are integrated.

[0515] 7. Confirmation of Rights

[0516] The server scans a music database to check whether the generated music data is similar to existing copyrighted works.

[0517] Example: Making sure the generated songs are not too similar to other existing songs.

[0518] 8. Publishing to a sharing site

[0519] If the user selects "I want to make it public," the server will automatically upload the song using the sharing site's API.

[0520] Example: The server calls the YouTube API and uploads the generated song to YouTube.

[0521] 9. Adding a Digital Certificate

[0522] The server encrypts the metadata associated with the generated song and registers it as an NFT using blockchain technology.

[0523] Example: NFTs registered on the blockchain digitally prove a user's copyright.

[0524] In this way, the system of the present invention provides a variety of functions that allow users to easily and safely create music, share the music they create, and even verify copyright. The above process creates an environment in which users can create and publish original music and songs without specialized knowledge.

[0525] The processing flow will be explained below.

[0526] Step 1:

[0527] A user accesses the system through a terminal and launches an application. The user enters login information for authentication.

[0528] Step 2:

[0529] The user writes the phrase or instrument image they have in mind into the input field on the device. For example, they might type, "I want a bright guitar melody."

[0530] Step 3:

[0531] The device sends the user's input data to the server, which stores the received data and prepares it for analysis.

[0532] Step 4:

[0533] The server applies natural language processing (NLP) technology to the received language data to extract key keywords. As a result of the analysis, the keywords "cheerful," "guitar," and "melody" are identified.

[0534] Step 5:

[0535] The server generates the next dialogue step based on the extracted keywords and sends it to the terminal in the form of a question, such as "What tempo and key do you want to play in?"

[0536] Step 6:

[0537] The user responds to the question from the terminal by entering "Uptempo C major." The terminal sends the response data to the server.

[0538] Step 7:

[0539] The server receives the user's response data and calls the music generation AI model, which inputs tempo and key information to generate the initial song data.

[0540] Step 8:

[0541] The generated initial song data is saved on the server. Next, the user inputs their desired lyrics. For example, they might input "I want lyrics with themes of love and youth."

[0542] Step 9:

[0543] The device sends the user's lyrics request to the server, which then calls the lyric generation AI model and inputs the specified theme. The AI ​​model generates lyrics based on the theme.

[0544] Step 10:

[0545] The generated lyrics are stored on the server and integrated into the initial music data. The server then adds performance data and vocal data.

[0546] Step 11:

[0547] The server selects appropriate performance data from the existing performance database and integrates it into the song. Singing data is also added in the same way.

[0548] Step 12:

[0549] The completed song data is stored on a server, which scans the database to check whether the song data infringes on the rights of others.

[0550] Step 13:

[0551] The server completes the rights verification and prepares to publish the file on the sharing site according to the user's selection. The user selects "I want to publish" on the terminal.

[0552] Step 14:

[0553] The server automatically uploads the music using the API of the sharing site (e.g. YouTube) and notifies the user when the upload is complete.

[0554] Step 15:

[0555] The server generates metadata about the song and uses blockchain technology to generate a digital certificate (NFT), which is then provided to the user.

[0556] The above are the specific processing steps of the music generation AI system.

[0557] Example 1

[0558] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0559] Conventional music production systems require users to configure music data in detail, often requiring specialized knowledge and skills. Furthermore, the process of verifying that the generated music data does not infringe on the rights of others is cumbersome. As a result, music production has been difficult and time-consuming, creating a demand for an environment where anyone can easily create original music and share it safely.

[0560] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0561] In this invention, the server includes means for receiving language data entered by a user, means for analyzing the received language data and extracting key keywords, means for generating music data based on the extracted keywords, means for adding performance data and vocal data to the generated music data, means for verifying that the added music data does not infringe the rights of others, means for publishing the verified music data on a sharing site, means for adding a digital certificate to the published music data, means for scanning and verifying existing databases for the generated music data, and means for encrypting metadata related to the generated music and registering it as an NFT using blockchain technology. This enables users to easily create original music and share it safely and efficiently, even without specialized knowledge.

[0562] "User" refers to a person who intends to use this system to generate music data.

[0563] "Language data" refers to text data such as phrases and instrument images entered by users.

[0564] "Server" refers to a central processing unit that executes the processing of the entire system and manages data.

[0565] "Analysis" refers to the process by which the server processes the received language data and extracts key keywords.

[0566] "Key keywords" refer to important words or phrases extracted from language data.

[0567] "Music data" refers to musical information such as melodies and accompaniments generated by generative AI models.

[0568] "Performance data" refers to musical instrument performance information provided by professional musicians.

[0569] "Singing data" refers to vocal audio information provided by a singer.

[0570] "Ensuring that it does not infringe on the rights of others" refers to the process of checking whether the generated music data is similar to existing copyrighted works.

[0571] "Sharing site" refers to an online platform for sharing generated music data with other users.

[0572] "Digital certificate" refers to electronic evidence that proves the copyright of generated music data.

[0573] "Scanning the database" refers to the process by which the server searches the existing music database to assess similarity.

[0574] "Metadata" refers to additional information associated with the generated song (e.g., title, artist name).

[0575] "Encryption" refers to the process of using cryptography to protect data from unauthorized access.

[0576] "Blockchain technology" refers to a distributed data management technology that prevents data tampering and ensures traceability.

[0577] "NFT" stands for Non-Fungible Token and refers to a means of proving ownership of digital assets on the blockchain.

[0578] This invention provides a system that allows users to easily create and publish music. This system generates music data based on language data entered by the user and has the function of adding performance data and vocal data. It also verifies that the generated music data does not infringe on the rights of others, publishes it on a sharing site, and certifies the copyright by adding a digital certificate.

[0579] Overall system configuration

[0580] Hardware and Software Configuration

[0581] Server: A central processing unit that analyzes, generates, checks, and shares music data. Specifically, it is installed with software libraries for natural language processing (NLP) techniques (e.g., Spacy, NLTK) and music generation AI models (e.g., OpenAI's GPT-3, Google's Magenta).

[0582] Terminal: A device that a user accesses and inputs data from. Examples include smartphones, tablet PCs, and desktop PCs.

[0583] Sharing site: An online platform for publishing generated music data. Examples include YouTube and SoundCloud.

[0584] Data Processing Description

[0585] 1. User Input

[0586] The user accesses the application through their device and writes down the phrases or instrument images they have in mind in the input field. For example, the user might enter, "I want a bright guitar melody."

[0587] 2. Input analysis and keyword extraction

[0588] The server receives the user's phrase data sent from the device and extracts key keywords using natural language processing technology. Specifically, the server analyzes the input data and extracts keywords such as "cheerful," "guitar," and "melody."

[0589] 3. Interactive question-answering

[0590] The next step is for the server to generate interactive questions for the user based on the analysis results, such as "What tempo and key do you want to play in?", and display them to the user.

[0591] 4. Music Generation

[0592] If the user responds with "uptempo C major," the server inputs this information into the music generation AI model and generates basic music data. As a specific example, the server inputs the prompt "Generate a melody and accompaniment in uptempo C major" into the generation AI model and obtains an automatically generated melody and accompaniment.

[0593] 5. Lyric Generation

[0594] If a user requests "I want lyrics with the theme of love and youth," the server inputs this theme into the lyrics generation model and generates lyrics. As a specific example, the server inputs a prompt sentence with the theme of "love and youth" into the lyrics generation model and obtains the generated lyrics.

[0595] 6. Adding performance and singing data

[0596] The server adds performance data and vocal data to the generated basic music data. This data is created using data from existing professional musicians and singers. For example, the server integrates guitar parts by existing professional musicians and vocal parts by existing professional singers into the basic music data.

[0597] 7. Confirmation of Rights

[0598] The server scans a music database to check whether the generated music data is similar to existing copyrighted works. For example, the server crawls a music database and uses a similarity checking algorithm to evaluate the similarity between the generated music and existing music.

[0599] 8. Publishing to a sharing site

[0600] If the user selects "I want to make it public," the server automatically uploads the music using the sharing site's API. For example, the server calls the sharing site's API (e.g., YouTube API), provides the necessary authentication information, and then uploads the generated music file.

[0601] 9. Adding a Digital Certificate

[0602] The server encrypts metadata related to the generated music and registers it as an NFT using blockchain technology. For example, the server encrypts the music metadata and generates a transaction on a blockchain platform (e.g., Ethereum) to register the NFT and obtain a digital certificate.

[0603] In this way, the system of the present invention provides a variety of functions for users to easily and safely create music, share the music they create, and verify copyright, creating an environment in which users can create and publish original music and songs without specialized knowledge.

[0604] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0605] Specific processing steps of the system

[0606] Step 1:

[0607] The user accesses the application using a terminal and writes down the phrases or instrument images that come to mind in the input field.

[0608] Input data: Language data entered by the user (e.g., "Please include a cheerful guitar melody")

[0609] Output data: Transmitted language data

[0610] Step 2:

[0611] The server receives the user's language data sent from the device, then analyzes the data using natural language processing (NLP) technology to extract key keywords.

[0612] Input data: Transmitted language data

[0613] Data processing: Using NLP technology, key keywords such as "upbeat," "guitar," and "melody" are extracted.

[0614] Output data: extracted main keywords

[0615] Step 3:

[0616] As a next step, the server generates interactive questions for the user based on the extracted keywords, which allow the user to confirm detailed settings for the music data to be generated.

[0617] Input data: extracted main keywords

[0618] Data processing: Question generation (e.g., "What tempo and key do you play in?")

[0619] Output data: Generated question

[0620] Step 4:

[0621] The user inputs a response to a question from the server.

[0622] Input data: User response (e.g., "Uptempo C major")

[0623] Output data: User response data

[0624] Step 5:

[0625] The server uses the user's response data to input prompt sentences into the music generation AI model and generate basic music data.

[0626] Input data: User response data (e.g., "Generate a melody and accompaniment in uptempo C major.")

[0627] Data Computing: Running Music Generation AI Models

[0628] Output data: generated melody and accompaniment

[0629] Step 6:

[0630] If the user requests, the server also generates additional musical parts, such as lyrics. The server inputs the user's request into the lyrics generation model and generates lyrics.

[0631] Input data: User request (e.g., "Please write lyrics with themes of love and youth")

[0632] Data calculation: Running the lyrics generation model

[0633] Output: Generated lyrics

[0634] Step 7:

[0635] The server adds performance data and singing data to the generated basic music data, using data from existing professional musicians and singers.

[0636] Input data: basic music data, performance data of professional musicians, singing data of professional singers

[0637] Data processing: Data integration

[0638] Output data: Finished music data

[0639] Step 8:

[0640] The server scans existing music databases to ensure that the generated music data does not infringe on the rights of others.

[0641] Input data: Completed music data

[0642] Data calculation: scanning music databases and checking similarities

[0643] Output data: Infringement confirmation results

[0644] Step 9:

[0645] If the user selects "I want to make it public," the server will automatically upload the song using the sharing site's API.

[0646] Input data: Completed music data, API credentials for the sharing site

[0647] Data Calculation: Upload data via API

[0648] Output data: Songs published on a sharing site

[0649] Step 10:

[0650] The server encrypts the metadata associated with the generated song and registers it as an NFT using blockchain technology.

[0651] Input data: Song metadata

[0652] Data computation: encryption and blockchain registration

[0653] Output data: Digital certificate (NFT)

[0654] Through these steps, the system provides users with a variety of functions to easily and safely create music, share the music they create, and even prove their copyright.

[0655] (Application example 1)

[0656] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0657] Conventional music production systems lack the means for users to easily create and publish music, and to protect the copyright of that music. This makes it difficult for users without specialized knowledge to create original music, resulting in a high barrier to entry for music production. Furthermore, there is a lack of means to prove the copyright of published music, which means users are constantly exposed to the risk of copyright infringement. There is a need to solve these issues.

[0658] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0659] In this invention, the server includes: means for receiving language data input by a user; means for analyzing the received language data and extracting key keywords; means for generating music data based on the extracted keywords; means for adding performance data and vocal data to the generated music data; means for verifying that the added music data does not infringe the rights of others; means for publishing the verified music data on a distribution platform; means for attaching a digital certificate to the published music data using blockchain technology; means for analyzing the user's input using natural language processing technology and generating a prompt; and means for inputting the prompt into a generative AI model to generate music data and lyrics. This allows users to easily create music and publish the music on a distribution platform, while protecting their copyrights with digital certificates using blockchain technology.

[0660] "Language data entered by the user" refers to text information entered by the user through the application when the user wishes to create music.

[0661] "Major keywords" are words and phrases that are important for music generation and are extracted from the input linguistic data.

[0662] "Music data generation" means that the music generation AI model automatically creates the basic structure of a song based on the extracted main keywords.

[0663] "Performance data" is data that includes information about performances by musical instruments, and is an element that is added to music data.

[0664] "Vocal data" is data that includes vocal performances accompanied by lyrics, and is an element that is added to music data.

[0665] "Confirmation of non-infringement" is the process of checking whether the generated music data is highly similar to existing copyrighted works.

[0666] A "distribution platform" is an online service or system used to make generated music data available to the public.

[0667] "Blockchain technology" is a distributed ledger technology that links each data block to prevent data tampering and increase reliability.

[0668] A "Digital Certificate" is a digital certificate for certifying copyright that contains metadata related to published music data and is generated using blockchain technology.

[0669] "Natural language processing technology" is an artificial intelligence technology for analyzing, understanding, and generating human language, and is used to extract key keywords from input language data.

[0670] A "prompt" is a sentence input into a music generation AI model, and is an instruction statement that defines the characteristics of the music data to be generated.

[0671] A "generative AI model" is an artificial intelligence model used to automatically generate music data and lyrics.

[0672] The present invention relates to a system that allows users to easily create and publish music. Details of this system and an embodiment thereof will be described below.

[0673] Music production system configuration

[0674] The system includes the following main functions:

[0675] 1. Means for receiving user input:

[0676] The user uses their own device (such as a smartphone or head-mounted display) to input phrases or images of musical instruments that come to mind into the application. For example, the user might input, "I want a bright guitar melody." This text data is then sent to the server.

[0677] 2. Linguistic data analysis and keyword extraction:

[0678] The server analyzes the received language data using natural language processing technology (such as OpenAI's API) and extracts key keywords, such as "cheerful," "guitar," and "melody."

[0679] 3. Generate prompt sentence:

[0680] The server generates a prompt based on the analysis results. This prompt is a statement that instructs the music generation AI model. For example, the prompt might say, "The music to be generated will be up-tempo, in the key of C major, and based on a bright guitar melody."

[0681] 4. Music data and lyrics generation:

[0682] The server inputs the generated prompt sentence into the AI ​​model and generates music data and lyrics if necessary. For example, if a user requests "I want lyrics on the theme of love and youth," the lyrics generation AI model will generate lyrics on the theme of "love and youth."

[0683] 5. Adding performance and vocal data:

[0684] The server adds performance data and vocal data to the generated basic music data. This data is created using data from existing professional musicians and singers. For example, guitar parts by a professional guitarist and vocal parts by a singer are integrated.

[0685] 6. Acknowledgement of Rights:

[0686] The server scans an existing music database to determine whether the generated music data is similar to other existing copyrighted works. It verifies that the generated music does not have a high degree of similarity to other existing music.

[0687] 7. Publishing your music:

[0688] If the user selects "I want to make it public," the server will automatically upload the created song using the distribution platform's API (such as YouTube API).

[0689] 8. Adding a digital certificate:

[0690] The server then encrypts the metadata associated with the generated music and registers it as a digital certificate (NFT) using blockchain technology, digitally certifying the user's copyright.

[0691] Specific examples of functions

[0692] For example, if the user types "I want a cheerful guitar melody," the system will generate the following prompt:

[0693] The music it generates is upbeat in tempo, set in the key of C major, and based on bright, guitar-driven melodies.

[0694] Similarly, if the user requests "Please write lyrics on the themes of love and youth," we generate the following prompt:

[0695] Generate lyrics on the themes of love and youth.

[0696] With these functions, the system of the present invention allows users to easily create music, publish it on a distribution platform, and protect copyright with digital certificates using blockchain technology.

[0697] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0698] Step 1:

[0699] The server receives the user's input data from the device. For example, the user might say, "I want a cheerful guitar melody," and the device sends that data to the server. This input data becomes the basis for the next analysis step.

[0700] Step 2:

[0701] The server analyzes the received language data using natural language processing technology. Specifically, it uses OpenAI's API to extract key keywords from the input text. For example, "cheerful," "guitar," and "melody" are extracted. This keyword extraction clarifies the elements of music generation.

[0702] Step 3:

[0703] The server generates a prompt based on the extracted keywords. For example, it creates a prompt such as, "The music to be generated will be up-tempo, in the key of C major, and based on a bright guitar melody." This prompt becomes the instruction input to the music generation AI model.

[0704] Step 4:

[0705] The server inputs the generated prompt into a music generation AI model to generate music data. At this time, the tempo, key, type of instrument, etc. are specified. For example, the AI ​​model automatically generates a melody and accompaniment based on an up-tempo C major key. The generated music data is then passed to the next step.

[0706] Step 5:

[0707] The server inputs the theme into the lyric generation model based on the user's request and generates lyrics. For example, if a user requests "I want lyrics with the theme of love and youth," the lyric generation AI model will generate lyrics with the theme of "love and youth." This lyric data is then integrated with the music data.

[0708] Step 6:

[0709] The server adds performance data and vocal data to the generated music data. Specifically, it integrates data from professional musicians and singers. For example, a guitarist adds a guitar part and a vocalist adds a vocal part. This results in more complete music data.

[0710] Step 7:

[0711] The server scans the existing music database to verify the rights of the generated music data. Here, it checks that the generated song does not have a high similarity to other existing songs. If there is a high similarity, it notifies the user. The verified data proceeds to the next step.

[0712] Step 8:

[0713] The server then publishes the verified music data to the distribution platform. Specifically, the music data is automatically uploaded using the distribution platform's API. For example, the server calls the distribution platform's API and performs the procedure to upload the music.

[0714] Step 9:

[0715] The server then adds a digital certificate to the published music data. Specifically, it encrypts the metadata associated with the generated music and registers it as an NFT using blockchain technology. This digitally certifies the user's copyright.

[0716] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0717] ---

[0718] This invention provides a system that allows users to easily create and publish music. This system generates music data based on language data entered by the user, and adds performance and vocal data. It also verifies that the generated music data does not infringe on the rights of others, publishes it on a sharing site, and certifies the copyright by adding a digital certificate.

[0719] Additionally, the present invention incorporates an emotion engine that recognizes the user's emotions and has the ability to adjust the tone and content of music data and lyrics based on the recognized emotions.

[0720] Explaining the music production process using the emotion engine

[0721] 1. User Input

[0722] A user accesses the system through a terminal and launches the application. After entering their login information and performing authentication, they write a phrase or an image of an instrument in the input field. For example, they might enter, "I want a bright guitar melody."

[0723] 2. Emotional Recognition

[0724] The device sends the user's input data to the server, which stores the data and uses an emotion engine to recognize emotions from the input data. For example, emotions such as "joy" or "liveliness" can be detected from the user's input.

[0725] 3. Input Analysis

[0726] The server applies natural language processing (NLP) technology to the received language data to extract key keywords. The keywords extracted as a result of the analysis are "cheerful," "guitar," and "melody."

[0727] 4. Interactive question-answering

[0728] Based on the extracted keywords and the emotion recognition results, the server generates the next dialogue step and sends it to the terminal in the form of a question, such as "What tempo and key do you want to play in?"

[0729] 5. Music Generation

[0730] If the user responds with "uptempo C major," the server inputs this information into the music generation AI model to generate basic music data, and adjusts the tone and structure of the music data based on the emotion recognition results.

[0731] 6. Lyric Generation

[0732] If a user requests lyrics with themes of love and youth, the server will call the lyric generation AI model and input the specified theme. The AI ​​model will generate lyrics based on the theme and the recognized emotions. For example, it will automatically generate lyrics with a bright tone that conveys joy and vitality.

[0733] 7. Addition of performance and singing data

[0734] The server adds performance data and vocal data to the generated basic music data, using data from existing professional musicians and singers. For example, guitar parts by a professional guitarist and vocal parts by a singer are integrated.

[0735] 8. Confirmation of Rights

[0736] The server scans a music database to ensure that the generated music data is not similar to existing copyrighted works, for example, to ensure that the generated song does not have a high degree of similarity to other existing songs.

[0737] 9. Publishing to a sharing site

[0738] If the user selects "I want to make it public," the server automatically uploads the song using the API of the sharing site. For example, the server calls the YouTube API and uploads the created song to YouTube.

[0739] 10. Adding a Digital Certificate

[0740] The server generates metadata about the song and uses blockchain technology to generate a digital certificate (NFT), which is then provided to the user. For example, the NFT registered on the blockchain digitally certifies the user's copyright.

[0741] The system of this invention allows users to easily and safely create music, share the music they create, and even prove their copyright. The introduction of an emotion engine makes it possible to generate music and lyrics that match the user's emotions, providing a more personalized musical experience.

[0742] The processing flow will be explained below.

[0743] Music production process using emotion engine

[0744] Step 1:

[0745] A user accesses the system through a terminal and launches an application. The user enters login information for authentication.

[0746] Step 2:

[0747] The user writes the phrase or instrument image they have in mind into the input field on the device. For example, they might type, "I want a bright guitar melody."

[0748] Step 3:

[0749] The device sends the user's input data to the server, which stores the received data and uses an emotion engine to recognize emotions from the input data.

[0750] Step 4:

[0751] The server analyzes the received data, and the emotion engine recognizes the user's emotions. For example, if someone says, "I want a bright guitar melody," the engine can detect emotions such as "joy" or "liveliness."

[0752] Step 5:

[0753] The server uses natural language processing (NLP) techniques to extract key keywords from the input data, such as "upbeat," "guitar," and "melody."

[0754] Step 6:

[0755] Based on the keywords and the recognized emotions, the server generates the next dialogue step and sends a question to the terminal, for example, "What tempo and key do you want to play in?"

[0756] Step 7:

[0757] The user responds to the question from the terminal with "Uptempo C major." The terminal then sends this response data to the server.

[0758] Step 8:

[0759] The server inputs the tempo, key, and recognized emotion into the music generation AI model to generate initial song data. For example, it creates an up-tempo melody in C major that reflects "joy."

[0760] Step 9:

[0761] The user requests, "I want you to write lyrics on the theme of love and youth" on the device. The device sends the request data to the server.

[0762] Step 10:

[0763] The server calls the lyric generation AI model and inputs the specified theme and emotion data. For example, it generates lyrics based on the theme of "love and youth" and the emotion of "joy."

[0764] Step 11:

[0765] The generated lyrics are stored on the server and integrated into the initial music data. At this stage, the server adds performance data and vocal data.

[0766] Step 12:

[0767] The server selects appropriate performance data from an existing performance database and integrates it into the song. Similarly, vocal data is added, for example, guitar parts by a professional guitarist or vocal parts by a singer.

[0768] Step 13:

[0769] The completed song data is stored on a server, which checks it against a music database to ensure it does not infringe on the rights of others.

[0770] Step 14:

[0771] If the server completes the rights verification and the user selects publishing, the server automatically uploads the song using the sharing site's API. For example, the server uploads the song to YouTube using the YouTube API.

[0772] Step 15:

[0773] The server generates metadata about the song and uses blockchain technology to create a digital certificate (NFT), which is then provided to the user.

[0774] The above are the specific processing steps of the system that generates and publishes music using an emotion engine.

[0775] Example 2

[0776] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0777] Conventional music production systems make it difficult for users to easily create music and lyrics that match their emotions, and the process of verifying that the generated music data does not infringe on the copyrights of others takes a lot of time and effort.Furthermore, there is a lack of means to publish the created music data and prove its copyright.

[0778] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0779] In this invention, the server includes means for receiving language data entered by a user, means for analyzing the received language data and extracting key keywords, means for generating music data based on the extracted keywords and the recognized emotion, means for adding performance data and vocal data to the generated music data, means for verifying that the added music data does not infringe the rights of others, means for publishing the verified music data on a sharing site, means for adding a digital certificate to the published music data, means for recognizing the user's emotion from the received language data, and means for adjusting the tone and content of the music data based on the recognized emotion. This allows users to easily create music and lyrics that match their emotions, enables the safe generation and publication of music data that does not infringe the rights of others, and furthermore, ensures that copyright is proven.

[0780] "User" refers to a person who uses the system to create and publish music and lyrics.

[0781] "Means for receiving" refers to a method or device by which the system receives language data input by a user.

[0782] "Means for analyzing" refers to a method or device that analyzes received language data using natural language processing (NLP) techniques and extracts key keywords.

[0783] "Key Keywords" refer to words and phrases that extract important information or instructions from the analyzed linguistic data.

[0784] "Emotion recognition" refers to the process of identifying a user's emotion from received linguistic data.

[0785] "Music data" refers to electronic data that includes generated musical components such as melodies, chords, and rhythms.

[0786] "Performance data" refers to specific performance information by musical instruments or other sound sources that is added to music data.

[0787] "Vocal data" refers to information about the singer's singing voice and vocal parts that is added to music data.

[0788] "Rights verification" refers to the process of checking whether the generated music data is similar to existing copyrighted works.

[0789] "Sharing site" refers to a web platform or online service for publishing generated music data.

[0790] A "digital certificate" refers to an electronic certificate generated using blockchain technology or other means to certify the copyright of published music data.

[0791] "Tone" refers to the tone and quality of an instrument used in music.

[0792] "Content" refers to the subject matter, themes, and expressions contained in the music data and lyrics.

[0793] This is a system that allows users to easily create and publish music based on their emotions. This system generates music data and lyrics based on user requests, adds performance data and vocal data to them, and after verifying that the copyrights of others are not being infringed, publishes the music on a sharing site and adds a digital certificate to prove copyright.

[0794] Hardware and software used

[0795] The system consists of a server, a terminal, and a user. The server uses a high-performance data processing device, including an AI model for executing natural language processing (NLP) techniques and a generative AI model for music generation and lyric generation. Specifically, the following software is used:

[0796] Natural Language Processing (NLP) technologies: Python NLP libraries (e.g., NLTK, spaCy)

[0797] Music generation AI model: Deep learning model using TensorFlow or PyTorch

[0798] Lyric generation AI models: OpenAI GPT-3 and GPT-4

[0799] Publishing to sharing sites uses APIs to connect to external services, such as YouTube API for publishing songs and blockchain technology for generating digital certificates.

[0800] Program processing explanation

[0801] User Input

[0802] Users access the system from their terminals, log in, and make specific requests to create music (e.g., "Please add a cheerful guitar melody").

[0803] Emotion recognition

[0804] The device sends the user's input data to the server, which uses an emotion engine to identify the user's emotion from this data. NLP technology is used in this process.

[0805] Data analysis and keyword extraction

[0806] The server uses NLP techniques to analyze the user's request and extract key keywords (e.g., "upbeat," "guitar," "melody").

[0807] Interactive question and answering

[0808] Based on the extracted keywords and the recognized emotions, the server generates the next dialogue step and sends it to the terminal in the form of a question (e.g., "What tempo and key do you want to play in?").

[0809] Basic music data generation

[0810] When a user responds with "uptempo C major," the server inputs this information into a music generation AI model to generate basic music data, and adjusts the tone and structure of the music data based on the emotion recognition results.

[0811] Lyrics generation

[0812] When a user requests lyrics on the theme of love and youth, the server calls a lyrics generation AI model and generates lyrics based on the specified theme and recognized emotions (e.g., lyrics with a bright tone).

[0813] Addition of performance and singing data

[0814] The server adds performance data and vocal data by professional musicians and singers to the generated music data (e.g., guitar parts by a professional guitarist and vocal parts by a singer).

[0815] Confirmation of rights

[0816] The server scans the generated music data to ensure it does not infringe on the rights of others (e.g., by using a music database to check for similarity to existing copyrighted works).

[0817] Publishing to a shared site

[0818] When the user selects "I want to make it public," the server uses the sharing site's API to make the song public (e.g., upload it to YouTube).

[0819] Attaching a Digital Certificate

[0820] The server generates metadata about the song and uses blockchain technology to generate a digital certificate (NFT), which users receive and their copyright is protected.

[0821] Examples and prompts

[0822] For example, a user might request, "I want a cheerful guitar melody," and the server would generate music data based on this request. For the user's next request, the server would use the AI ​​model to generate corresponding lyrics, which would then be published and digitally authenticated.

[0823] Prompt Sentence Examples

[0824] "Please identify the emotion from the user's input, such as 'I want a bright guitar melody,' and input 'joy, up-tempo, C major, bright guitar melody' into the generative AI model to generate a song."

[0825] This allows users to easily create music and lyrics that match their emotions, safely release them, and protect their copyrights.

[0826] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0827] Step 1: User login and request submission

[0828] Users access the system through a terminal and enter their login information for authentication. After successful authentication, they input specific requests for music production (e.g., "Please add a cheerful guitar melody.") The input data is received by the terminal and sent to the server.

[0829] Input: User login information and request (e.g., "Please add a cheerful guitar melody")

[0830] Output: The user's authenticated request data sent to the server.

[0831] Step 2: Recognize emotions

[0832] The server passes the received request data to the emotion engine, which uses NLP technology to identify emotions, extracting emotions such as "joy" and "liveliness" from the word "cheerful."

[0833] Input: User request data

[0834] Output: Identified emotion data (e.g., "joy")

[0835] Step 3: Data analysis and keyword extraction

[0836] After emotion recognition, the server uses NLP technology to analyze the user's request and extract key keywords, such as "cheerful," "guitar," and "melody."

[0837] Input: User request data with identified emotions

[0838] Output: Extracted keyword data (e.g., "cheerful," "guitar," "melody")

[0839] Step 4: Interactive question-answering

[0840] The server generates the next dialogue step based on the extracted keywords and the identified emotions, which is sent to the terminal, where the user is prompted with a question such as, "What tempo and key do you want to play in?"

[0841] Input: extracted keywords and identified emotions

[0842] Output: Questions to ask the user (e.g., "What tempo and key do you want to play in?")

[0843] Step 5: Generate basic music data

[0844] When a user responds with "uptempo C major," the server inputs this response into a music generation AI model to generate basic music data. The server also adjusts the tone and structure of the music data based on the identified emotions.

[0845] Input: User response data (e.g., "Uptempo C major") and identified emotion

[0846] Output: Generated basic music data

[0847] Step 6: Generate lyrics

[0848] When a user requests lyrics on the theme of love and youth, the server calls the lyric generation AI model and generates lyrics based on the specified theme and identified emotions. For example, it generates lyrics with a bright tone that evokes joy.

[0849] Input: User's theme request (e.g., "Love and Youth") and identified emotions

[0850] Output: Generated lyrics data

[0851] Step 7: Adding performance and singing data

[0852] The server adds performance data and singing data by professional musicians and singers to the generated music data. For example, guitar parts by a professional guitarist and vocal parts by a singer are integrated.

[0853] Input: Generated basic music data and lyrics data

[0854] Output: Complete music data with performance and vocal data added

[0855] Step 8: Entitlement Verification

[0856] The server scans the music database to ensure the generated music data does not infringe on the rights of others, and in the process checks to ensure that it is not too similar to existing songs.

[0857] Input: Complete music data

[0858] Output: Secure music data with rights verification

[0859] Step 9: Publish to a sharing site

[0860] When the user selects "I want to publish," the device sends this instruction to the server. The server publishes the song using the sharing site's API. For example, the server calls the YouTube API to upload the song to YouTube.

[0861] Input: User's publishing instructions

[0862] Output: Song data uploaded to a sharing site

[0863] Step 10: Attach a Digital Certificate

[0864] The server generates metadata about the song and uses blockchain technology to generate a digital certificate (NFT), which the user receives, digitally protecting the copyright of the song.

[0865] Input: Song data published on a sharing site

[0866] Output: Generated digital certificate (NFT)

[0867] (Application example 2)

[0868] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0869] Current music production systems lack sufficient means for users to easily generate music and lyrics that respond to their emotions and publish them to content distribution services. Furthermore, there is no established method for digitally verifying the copyright of the generated music data. This creates problems for users to create personalized music and share their creations securely and easily.

[0870] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving language data entered by a user, means for analyzing the received language data and generating music data based on key keywords and recognized emotions, and means for adding performance data and vocal data to the generated music data. This allows users to easily generate music and lyrics that correspond to their emotions and publish them on content distribution services. Furthermore, by verifying that the generated music data does not infringe on the rights of others and attaching a digital certificate, copyright verification can be easily performed.

[0871] "User" means any individual or organization that uses the System to create and share music.

[0872] "Language data" refers to text data such as sentences and phrases that users input into the system.

[0873] "Analysis" refers to the process of analyzing received linguistic data and extracting key keywords and information.

[0874] "Emotion" refers to psychological states such as joy, sadness, and excitement that are recognized from the user's input data.

[0875] "Music data" refers to digital data containing the melody, rhythm, and harmony of the generated music.

[0876] "Performance data" refers to digital data of musical instrument performances that are added to music data.

[0877] "Vocal data" refers to digital vocal data that is added to music data.

[0878] "Ensuring that the rights of others are not infringed" refers to the process of ensuring that the generated music data is not similar to existing copyrighted works.

[0879] "Content distribution service" refers to a platform for publishing and sharing generated music data on the Internet.

[0880] "Digital Certificate" refers to an electronic certificate generated using blockchain technology to protect the copyright of generated music data.

[0881] The system for implementing the present invention allows users to easily create and publish music based on language data entered by the user. A series of steps will be explained in detail below.

[0882] 1. System Configuration

[0883] It consists of three main modules: the server, the terminal, and the user. The server is a central processing unit that actually generates music and processes data. The terminal is a device operated by the user, such as a smartphone. The user is the person who creates music.

[0884] 2. User Input

[0885] A user accesses the system through a terminal and launches the application. After entering their login information for authentication, they write a phrase or an image of an instrument in the input field. For example, they might enter, "I want a bright guitar melody."

[0886] 3. Emotional Recognition

[0887] The device sends the user's input data to the server. The server stores the received data and uses an emotion recognition engine called EmotionRecognition to recognize emotions from the input data. For example, emotions such as "joy" or "liveliness" can be detected from the user's input.

[0888] 4. Input Analysis

[0889] The server applies Natural Language Processing (NLP) technology to the received language data to extract key keywords. The keywords extracted as a result of the analysis are "cheerful," "guitar," and "melody."

[0890] 5. Interactive Question-Answering

[0891] Based on the extracted keywords and the emotion recognition results, the server generates the next dialogue step and sends it to the terminal in the form of a question, such as "What tempo and key do you want to play in?"

[0892] 6. Music Generation

[0893] If the user responds with "uptempo C major," the server inputs this information into a music generation AI model called MusicGeneration to generate basic music data, and adjusts the tone and structure of the music data based on the results of emotion recognition.

[0894] 7. Lyric Generation

[0895] If a user requests lyrics with themes of love and youth, the server calls a lyric generation AI model called LyricsGeneration and inputs the specified theme. The AI ​​model generates lyrics based on the theme and the recognized emotions. For example, it automatically generates lyrics with a bright tone that conveys joy and vitality.

[0896] 8. Addition of performance and singing data

[0897] The server adds performance data and vocal data to the generated basic music data, using data from existing professional musicians and singers. For example, guitar parts by a professional guitarist and vocal parts by a singer are integrated.

[0898] 9. Confirmation of Rights

[0899] The server uses a module called CopyrightChecker to check whether the generated music data is similar to existing copyrighted works, for example, to ensure that the generated music does not have a high similarity to other existing music.

[0900] 10. Publication

[0901] If the user selects "I want to make it public," the server automatically uploads the song using the API of a content distribution service. For example, the server calls the API of a sharing site and uploads the created song to that site.

[0902] 11. Adding a Digital Certificate

[0903] The server generates metadata about the song and uses blockchain technology to generate a digital certificate, which is then provided to the user. For example, the certificate registered on the blockchain digitally certifies the user's copyright.

[0904] Examples of concrete examples and prompts

[0905] Example 1: When a user inputs "I want a bright guitar melody," music with a pop tempo is automatically generated.

[0906] Example 2: If the user selects "Love and Youth" as the theme, cheerful lyrics on the theme of love and youth are automatically generated.

[0907] Example prompt sentence:

[0908] "I want a bright guitar melody."

[0909] "I want you to write lyrics with themes of love and youth"

[0910] "Uptempo C major"

[0911] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0912] Step 1:

[0913] A user accesses the system through a terminal and launches an application. The user enters login information for authentication, then writes a phrase or an image of an instrument in the input field. The input is a prompt such as "Please add a cheerful guitar melody." The terminal sends this input data to the server.

[0914] Step 2:

[0915] The server stores the received user input data. Based on the stored data, an emotion recognition engine called EmotionRecognition is used to recognize emotions from the input data. For example, emotions such as "joy" or "liveliness" are recognized from the input. The input data is text data, and the output is emotional data.

[0916] Step 3:

[0917] The server applies Natural Language Processing (NLP) technology to the received language data to extract key keywords. The keywords extracted as a result of the analysis are "cheerful," "guitar," and "melody." Text analysis is used to extract these keywords, with the input being language data and the output being keyword data.

[0918] Step 4:

[0919] The server generates the next dialogue step based on the extracted keywords and the emotion recognition results and sends it to the terminal in the form of a question. For example, it sends a question such as, "What tempo and key do you want to play at?" The input is keywords and emotion data, and the output is the dialogue step question.

[0920] Step 5:

[0921] The user answers questions from the server interactively through the terminal. For example, the user might answer "uptempo C major." The terminal then sends this answer data back to the server. The input is the user's answer, and the output is text data sent to the server.

[0922] Step 6:

[0923] The server inputs user response data and emotional data into a music generation AI model called MusicGeneration to generate basic music data. It also adjusts the tone and structure of the music data based on the results of emotion recognition. The input data are tempo, key, and emotional data, and the output is music data.

[0924] Step 7:

[0925] If the user again requests "Please create lyrics with the theme of love and youth," the server will call a lyrics generation AI model called LyricsGeneration and input the specified theme. The AI ​​model will generate lyrics based on the theme and the recognized emotion. The input is theme and emotion data, and the output is lyric data.

[0926] Step 8:

[0927] The server adds performance and vocal data to the generated basic music data. This uses data from existing professional musicians and singers. For example, guitar parts by a professional guitarist and vocal parts by a singer are integrated. The input is music data and additional performance and vocal data, and the output is the completed music data.

[0928] Step 9:

[0929] The server checks whether the generated music data is similar to existing copyrighted works through a module called CopyrightChecker. For example, it checks whether the generated music is too similar to other existing music. The input is the completed music data, and the output is the result of copyright check.

[0930] Step 10:

[0931] If the user selects "I want to publish," the server automatically uploads the song using the API of a content distribution service. For example, the server calls the API of a sharing site and uploads the created song to that site. The input is the music data and the publishing request, and the output is the upload result to the sharing site.

[0932] Step 11:

[0933] The server generates metadata about the music and generates a digital certificate using blockchain technology. The generated certificate information is provided to the user. For example, the certificate registered on the blockchain digitally certifies the user's copyright. The input is the metadata of the music data, and the output is a digital certificate.

[0934] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0935] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0936] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0937] [Third embodiment]

[0938] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0939] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0940] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0941] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0942] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0943] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0944] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0945] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0946] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0947] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0948] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0949] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0950] ---

[0951] This invention provides a system that allows users to easily create and publish music. This system generates music data based on language data entered by the user, and adds performance and vocal data. It also verifies that the generated music data does not infringe on the rights of others, publishes it on a sharing site, and certifies the copyright by adding a digital certificate.

[0952] Explaining the music production process

[0953] 1. User Input

[0954] Users access the application through their device and write down phrases or images of instruments they have in mind in the input field.

[0955] Example: A user types, "I want a cheerful guitar melody."

[0956] 2. Input Analysis

[0957] The server receives the user's phrase data sent from the terminal and extracts key keywords using natural language processing (NLP) techniques.

[0958] Example: Extract the keywords "bright," "guitar," and "melody."

[0959] 3. Interactive question-answering

[0960] As a next step, the server generates interactive questions for the user based on the analysis results.

[0961] Example: Displaying questions on the screen such as "What tempo and key do you want to play in?"

[0962] 4. Music Generation

[0963] If the user responds with "uptempo C major," the server inputs this information into a music generation AI model and generates basic music data.

[0964] Example: An AI model automatically generates an up-tempo melody and accompaniment in the key of C major.

[0965] 5. Lyric Generation

[0966] If a user requests "I want lyrics written on the theme of love and youth," the server inputs this theme into the lyrics generation model and generates lyrics.

[0967] Example: Lyrics on the theme of "love and youth" are automatically created by AI.

[0968] 6. Adding performance and singing data

[0969] The server adds performance data and singing data to the generated basic music data, using data from existing professional musicians and singers.

[0970] Example: Guitar parts by a professional guitarist and vocal parts by a singer are integrated.

[0971] 7. Confirmation of Rights

[0972] The server scans a music database to check whether the generated music data is similar to existing copyrighted works.

[0973] Example: Making sure the generated songs are not too similar to other existing songs.

[0974] 8. Publishing to a sharing site

[0975] If the user selects "I want to make it public," the server will automatically upload the song using the sharing site's API.

[0976] Example: The server calls the YouTube API and uploads the generated song to YouTube.

[0977] 9. Adding a Digital Certificate

[0978] The server encrypts the metadata associated with the generated song and registers it as an NFT using blockchain technology.

[0979] Example: NFTs registered on the blockchain digitally prove a user's copyright.

[0980] In this way, the system of the present invention provides a variety of functions that allow users to easily and safely create music, share the music they create, and even verify copyright. The above process creates an environment in which users can create and publish original music and songs without specialized knowledge.

[0981] The processing flow will be explained below.

[0982] Step 1:

[0983] A user accesses the system through a terminal and launches an application. The user enters login information for authentication.

[0984] Step 2:

[0985] The user writes the phrase or instrument image they have in mind into the input field on the device. For example, they might type, "I want a bright guitar melody."

[0986] Step 3:

[0987] The device sends the user's input data to the server, which stores the received data and prepares it for analysis.

[0988] Step 4:

[0989] The server applies natural language processing (NLP) technology to the received language data to extract key keywords. As a result of the analysis, the keywords "cheerful," "guitar," and "melody" are identified.

[0990] Step 5:

[0991] The server generates the next dialogue step based on the extracted keywords and sends it to the terminal in the form of a question, such as "What tempo and key do you want to play in?"

[0992] Step 6:

[0993] The user responds to the question from the terminal by entering "Uptempo C major." The terminal sends the response data to the server.

[0994] Step 7:

[0995] The server receives the user's response data and calls the music generation AI model, which inputs tempo and key information to generate the initial song data.

[0996] Step 8:

[0997] The generated initial song data is saved on the server. Next, the user inputs their desired lyrics. For example, they might input "I want lyrics with themes of love and youth."

[0998] Step 9:

[0999] The device sends the user's lyrics request to the server, which then calls the lyric generation AI model and inputs the specified theme. The AI ​​model generates lyrics based on the theme.

[1000] Step 10:

[1001] The generated lyrics are stored on the server and integrated into the initial music data. The server then adds performance data and vocal data.

[1002] Step 11:

[1003] The server selects appropriate performance data from the existing performance database and integrates it into the song. Singing data is also added in the same way.

[1004] Step 12:

[1005] The completed song data is stored on a server, which scans the database to check whether the song data infringes on the rights of others.

[1006] Step 13:

[1007] The server completes the rights verification and prepares to publish the file on the sharing site according to the user's selection. The user selects "I want to publish" on the terminal.

[1008] Step 14:

[1009] The server automatically uploads the music using the API of the sharing site (e.g. YouTube) and notifies the user when the upload is complete.

[1010] Step 15:

[1011] The server generates metadata about the song and uses blockchain technology to generate a digital certificate (NFT), which is then provided to the user.

[1012] The above are the specific processing steps of the music generation AI system.

[1013] Example 1

[1014] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1015] Conventional music production systems require users to configure music data in detail, often requiring specialized knowledge and skills. Furthermore, the process of verifying that the generated music data does not infringe on the rights of others is cumbersome. As a result, music production has been difficult and time-consuming, creating a demand for an environment where anyone can easily create original music and share it safely.

[1016] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1017] In this invention, the server includes means for receiving language data entered by a user, means for analyzing the received language data and extracting key keywords, means for generating music data based on the extracted keywords, means for adding performance data and vocal data to the generated music data, means for verifying that the added music data does not infringe the rights of others, means for publishing the verified music data on a sharing site, means for adding a digital certificate to the published music data, means for scanning and verifying existing databases for the generated music data, and means for encrypting metadata related to the generated music and registering it as an NFT using blockchain technology. This enables users to easily create original music and share it safely and efficiently, even without specialized knowledge.

[1018] "User" refers to a person who intends to use this system to generate music data.

[1019] "Language data" refers to text data such as phrases and instrument images entered by users.

[1020] "Server" refers to a central processing unit that executes the processing of the entire system and manages data.

[1021] "Analysis" refers to the process by which the server processes the received language data and extracts key keywords.

[1022] "Key keywords" refer to important words or phrases extracted from language data.

[1023] "Music data" refers to musical information such as melodies and accompaniments generated by generative AI models.

[1024] "Performance data" refers to musical instrument performance information provided by professional musicians.

[1025] "Singing data" refers to vocal audio information provided by a singer.

[1026] "Ensuring that it does not infringe on the rights of others" refers to the process of checking whether the generated music data is similar to existing copyrighted works.

[1027] "Sharing site" refers to an online platform for sharing generated music data with other users.

[1028] "Digital certificate" refers to electronic evidence that proves the copyright of generated music data.

[1029] "Scanning the database" refers to the process by which the server searches the existing music database to assess similarity.

[1030] "Metadata" refers to additional information associated with the generated song (e.g., title, artist name).

[1031] "Encryption" refers to the process of using cryptography to protect data from unauthorized access.

[1032] "Blockchain technology" refers to a distributed data management technology that prevents data tampering and ensures traceability.

[1033] "NFT" stands for Non-Fungible Token and refers to a means of proving ownership of digital assets on the blockchain.

[1034] This invention provides a system that allows users to easily create and publish music. This system generates music data based on language data entered by the user and has the function of adding performance data and vocal data. It also verifies that the generated music data does not infringe on the rights of others, publishes it on a sharing site, and certifies the copyright by adding a digital certificate.

[1035] Overall system configuration

[1036] Hardware and Software Configuration

[1037] Server: A central processing unit that analyzes, generates, checks, and shares music data. Specifically, it is installed with software libraries for natural language processing (NLP) techniques (e.g., Spacy, NLTK) and music generation AI models (e.g., OpenAI's GPT-3, Google's Magenta).

[1038] Terminal: A device that a user accesses and inputs data from. Examples include smartphones, tablet PCs, and desktop PCs.

[1039] Sharing site: An online platform for publishing generated music data. Examples include YouTube and SoundCloud.

[1040] Data Processing Description

[1041] 1. User Input

[1042] The user accesses the application through their device and writes down the phrases or instrument images they have in mind in the input field. For example, the user might enter, "I want a bright guitar melody."

[1043] 2. Input analysis and keyword extraction

[1044] The server receives the user's phrase data sent from the device and extracts key keywords using natural language processing technology. Specifically, the server analyzes the input data and extracts keywords such as "cheerful," "guitar," and "melody."

[1045] 3. Interactive question-answering

[1046] The next step is for the server to generate interactive questions for the user based on the analysis results, such as "What tempo and key do you want to play in?", and display them to the user.

[1047] 4. Music Generation

[1048] If the user responds with "uptempo C major," the server inputs this information into the music generation AI model and generates basic music data. As a specific example, the server inputs the prompt "Generate a melody and accompaniment in uptempo C major" into the generation AI model and obtains an automatically generated melody and accompaniment.

[1049] 5. Lyric Generation

[1050] If a user requests "I want lyrics with the theme of love and youth," the server inputs this theme into the lyrics generation model and generates lyrics. As a specific example, the server inputs a prompt sentence with the theme of "love and youth" into the lyrics generation model and obtains the generated lyrics.

[1051] 6. Adding performance and singing data

[1052] The server adds performance data and vocal data to the generated basic music data. This data is created using data from existing professional musicians and singers. For example, the server integrates guitar parts by existing professional musicians and vocal parts by existing professional singers into the basic music data.

[1053] 7. Confirmation of Rights

[1054] The server scans a music database to check whether the generated music data is similar to existing copyrighted works. For example, the server crawls a music database and uses a similarity checking algorithm to evaluate the similarity between the generated music and existing music.

[1055] 8. Publishing to a sharing site

[1056] If the user selects "I want to make it public," the server automatically uploads the music using the sharing site's API. For example, the server calls the sharing site's API (e.g., YouTube API), provides the necessary authentication information, and then uploads the generated music file.

[1057] 9. Adding a Digital Certificate

[1058] The server encrypts metadata related to the generated music and registers it as an NFT using blockchain technology. For example, the server encrypts the music metadata and generates a transaction on a blockchain platform (e.g., Ethereum) to register the NFT and obtain a digital certificate.

[1059] In this way, the system of the present invention provides a variety of functions for users to easily and safely create music, share the music they create, and verify copyright, creating an environment in which users can create and publish original music and songs without specialized knowledge.

[1060] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1061] Specific processing steps of the system

[1062] Step 1:

[1063] The user accesses the application using a terminal and writes down the phrases or instrument images that come to mind in the input field.

[1064] Input data: Language data entered by the user (e.g., "Please include a cheerful guitar melody")

[1065] Output data: Transmitted language data

[1066] Step 2:

[1067] The server receives the user's language data sent from the device, then analyzes the data using natural language processing (NLP) technology to extract key keywords.

[1068] Input data: Transmitted language data

[1069] Data processing: Using NLP technology, key keywords such as "upbeat," "guitar," and "melody" are extracted.

[1070] Output data: extracted main keywords

[1071] Step 3:

[1072] As a next step, the server generates interactive questions for the user based on the extracted keywords, which allow the user to confirm detailed settings for the music data to be generated.

[1073] Input data: extracted main keywords

[1074] Data processing: Question generation (e.g., "What tempo and key do you play in?")

[1075] Output data: Generated question

[1076] Step 4:

[1077] The user inputs a response to a question from the server.

[1078] Input data: User response (e.g., "Uptempo C major")

[1079] Output data: User response data

[1080] Step 5:

[1081] The server uses the user's response data to input prompt sentences into the music generation AI model and generate basic music data.

[1082] Input data: User response data (e.g., "Generate a melody and accompaniment in uptempo C major.")

[1083] Data Computing: Running Music Generation AI Models

[1084] Output data: generated melody and accompaniment

[1085] Step 6:

[1086] If the user requests, the server also generates additional musical parts, such as lyrics. The server inputs the user's request into the lyrics generation model and generates lyrics.

[1087] Input data: User request (e.g., "Please write lyrics with themes of love and youth")

[1088] Data calculation: Running the lyrics generation model

[1089] Output: Generated lyrics

[1090] Step 7:

[1091] The server adds performance data and singing data to the generated basic music data, using data from existing professional musicians and singers.

[1092] Input data: basic music data, performance data of professional musicians, singing data of professional singers

[1093] Data processing: Data integration

[1094] Output data: Finished music data

[1095] Step 8:

[1096] The server scans existing music databases to ensure that the generated music data does not infringe on the rights of others.

[1097] Input data: Completed music data

[1098] Data calculation: scanning music databases and checking similarities

[1099] Output data: Infringement confirmation results

[1100] Step 9:

[1101] If the user selects "I want to make it public," the server will automatically upload the song using the sharing site's API.

[1102] Input data: Completed music data, API credentials for the sharing site

[1103] Data Calculation: Upload data via API

[1104] Output data: Songs published on a sharing site

[1105] Step 10:

[1106] The server encrypts the metadata associated with the generated song and registers it as an NFT using blockchain technology.

[1107] Input data: Song metadata

[1108] Data computation: encryption and blockchain registration

[1109] Output data: Digital certificate (NFT)

[1110] Through these steps, the system provides users with a variety of functions to easily and safely create music, share the music they create, and even prove their copyright.

[1111] (Application example 1)

[1112] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1113] Conventional music production systems lack the means for users to easily create and publish music, and to protect the copyright of that music. This makes it difficult for users without specialized knowledge to create original music, resulting in a high barrier to entry for music production. Furthermore, there is a lack of means to prove the copyright of published music, which means users are constantly exposed to the risk of copyright infringement. There is a need to solve these issues.

[1114] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1115] In this invention, the server includes: means for receiving language data input by a user; means for analyzing the received language data and extracting key keywords; means for generating music data based on the extracted keywords; means for adding performance data and vocal data to the generated music data; means for verifying that the added music data does not infringe the rights of others; means for publishing the verified music data on a distribution platform; means for attaching a digital certificate to the published music data using blockchain technology; means for analyzing the user's input using natural language processing technology and generating a prompt; and means for inputting the prompt into a generative AI model to generate music data and lyrics. This allows users to easily create music and publish the music on a distribution platform, while protecting their copyrights with digital certificates using blockchain technology.

[1116] "Language data entered by the user" refers to text information entered by the user through the application when the user wishes to create music.

[1117] "Major keywords" are words and phrases that are important for music generation and are extracted from the input linguistic data.

[1118] "Music data generation" means that the music generation AI model automatically creates the basic structure of a song based on the extracted main keywords.

[1119] "Performance data" is data that includes information about performances by musical instruments, and is an element that is added to music data.

[1120] "Vocal data" is data that includes vocal performances accompanied by lyrics, and is an element that is added to music data.

[1121] "Confirmation of non-infringement" is the process of checking whether the generated music data is highly similar to existing copyrighted works.

[1122] A "distribution platform" is an online service or system used to make generated music data available to the public.

[1123] "Blockchain technology" is a distributed ledger technology that links each data block to prevent data tampering and increase reliability.

[1124] A "Digital Certificate" is a digital certificate for certifying copyright that contains metadata related to published music data and is generated using blockchain technology.

[1125] "Natural language processing technology" is an artificial intelligence technology for analyzing, understanding, and generating human language, and is used to extract key keywords from input language data.

[1126] A "prompt" is a sentence input into a music generation AI model, and is an instruction statement that defines the characteristics of the music data to be generated.

[1127] A "generative AI model" is an artificial intelligence model used to automatically generate music data and lyrics.

[1128] The present invention relates to a system that allows users to easily create and publish music. Details of this system and an embodiment thereof will be described below.

[1129] Music production system configuration

[1130] The system includes the following main functions:

[1131] 1. Means for receiving user input:

[1132] The user uses their own device (such as a smartphone or head-mounted display) to input phrases or images of musical instruments that come to mind into the application. For example, the user might input, "I want a bright guitar melody." This text data is then sent to the server.

[1133] 2. Linguistic data analysis and keyword extraction:

[1134] The server analyzes the received language data using natural language processing technology (such as OpenAI's API) and extracts key keywords, such as "cheerful," "guitar," and "melody."

[1135] 3. Generate prompt sentence:

[1136] The server generates a prompt based on the analysis results. This prompt is a statement that instructs the music generation AI model. For example, the prompt might say, "The music to be generated will be up-tempo, in the key of C major, and based on a bright guitar melody."

[1137] 4. Music data and lyrics generation:

[1138] The server inputs the generated prompt sentence into the AI ​​model and generates music data and lyrics if necessary. For example, if a user requests "I want lyrics on the theme of love and youth," the lyrics generation AI model will generate lyrics on the theme of "love and youth."

[1139] 5. Adding performance and vocal data:

[1140] The server adds performance data and vocal data to the generated basic music data. This data is created using data from existing professional musicians and singers. For example, guitar parts by a professional guitarist and vocal parts by a singer are integrated.

[1141] 6. Acknowledgement of Rights:

[1142] The server scans an existing music database to determine whether the generated music data is similar to other existing copyrighted works. It verifies that the generated music does not have a high degree of similarity to other existing music.

[1143] 7. Publishing your music:

[1144] If the user selects "I want to make it public," the server will automatically upload the created song using the distribution platform's API (such as YouTube API).

[1145] 8. Adding a digital certificate:

[1146] The server then encrypts the metadata associated with the generated music and registers it as a digital certificate (NFT) using blockchain technology, digitally certifying the user's copyright.

[1147] Specific examples of functions

[1148] For example, if the user types "I want a cheerful guitar melody," the system will generate the following prompt:

[1149] The music it generates is upbeat in tempo, set in the key of C major, and based on bright, guitar-driven melodies.

[1150] Similarly, if the user requests "Please write lyrics on the themes of love and youth," we generate the following prompt:

[1151] Generate lyrics on the themes of love and youth.

[1152] With these functions, the system of the present invention allows users to easily create music, publish it on a distribution platform, and protect copyright with digital certificates using blockchain technology.

[1153] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1154] Step 1:

[1155] The server receives the user's input data from the device. For example, the user might say, "I want a cheerful guitar melody," and the device sends that data to the server. This input data becomes the basis for the next analysis step.

[1156] Step 2:

[1157] The server analyzes the received language data using natural language processing technology. Specifically, it uses OpenAI's API to extract key keywords from the input text. For example, "cheerful," "guitar," and "melody" are extracted. This keyword extraction clarifies the elements of music generation.

[1158] Step 3:

[1159] The server generates a prompt based on the extracted keywords. For example, it creates a prompt such as, "The music to be generated will be up-tempo, in the key of C major, and based on a bright guitar melody." This prompt becomes the instruction input to the music generation AI model.

[1160] Step 4:

[1161] The server inputs the generated prompt into a music generation AI model to generate music data. At this time, the tempo, key, type of instrument, etc. are specified. For example, the AI ​​model automatically generates a melody and accompaniment based on an up-tempo C major key. The generated music data is then passed to the next step.

[1162] Step 5:

[1163] The server inputs the theme into the lyric generation model based on the user's request and generates lyrics. For example, if a user requests "I want lyrics with the theme of love and youth," the lyric generation AI model will generate lyrics with the theme of "love and youth." This lyric data is then integrated with the music data.

[1164] Step 6:

[1165] The server adds performance data and vocal data to the generated music data. Specifically, it integrates data from professional musicians and singers. For example, a guitarist adds a guitar part and a vocalist adds a vocal part. This results in more complete music data.

[1166] Step 7:

[1167] The server scans the existing music database to verify the rights of the generated music data. Here, it checks that the generated song does not have a high similarity to other existing songs. If there is a high similarity, it notifies the user. The verified data proceeds to the next step.

[1168] Step 8:

[1169] The server then publishes the verified music data to the distribution platform. Specifically, the music data is automatically uploaded using the distribution platform's API. For example, the server calls the distribution platform's API and performs the procedure to upload the music.

[1170] Step 9:

[1171] The server then adds a digital certificate to the published music data. Specifically, it encrypts the metadata associated with the generated music and registers it as an NFT using blockchain technology. This digitally certifies the user's copyright.

[1172] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1173] ---

[1174] This invention provides a system that allows users to easily create and publish music. This system generates music data based on language data entered by the user, and adds performance and vocal data. It also verifies that the generated music data does not infringe on the rights of others, publishes it on a sharing site, and certifies the copyright by adding a digital certificate.

[1175] Additionally, the present invention incorporates an emotion engine that recognizes the user's emotions and has the ability to adjust the tone and content of music data and lyrics based on the recognized emotions.

[1176] Explaining the music production process using the emotion engine

[1177] 1. User Input

[1178] A user accesses the system through a terminal and launches the application. After entering their login information and performing authentication, they write a phrase or an image of an instrument in the input field. For example, they might enter, "I want a bright guitar melody."

[1179] 2. Emotional Recognition

[1180] The device sends the user's input data to the server, which stores the data and uses an emotion engine to recognize emotions from the input data. For example, emotions such as "joy" or "liveliness" can be detected from the user's input.

[1181] 3. Input Analysis

[1182] The server applies natural language processing (NLP) technology to the received language data to extract key keywords. The keywords extracted as a result of the analysis are "cheerful," "guitar," and "melody."

[1183] 4. Interactive question-answering

[1184] Based on the extracted keywords and the emotion recognition results, the server generates the next dialogue step and sends it to the terminal in the form of a question, such as "What tempo and key do you want to play in?"

[1185] 5. Music Generation

[1186] If the user responds with "uptempo C major," the server inputs this information into the music generation AI model to generate basic music data, and adjusts the tone and structure of the music data based on the emotion recognition results.

[1187] 6. Lyric Generation

[1188] If a user requests lyrics with themes of love and youth, the server will call the lyric generation AI model and input the specified theme. The AI ​​model will generate lyrics based on the theme and the recognized emotions. For example, it will automatically generate lyrics with a bright tone that conveys joy and vitality.

[1189] 7. Addition of performance and singing data

[1190] The server adds performance data and vocal data to the generated basic music data, using data from existing professional musicians and singers. For example, guitar parts by a professional guitarist and vocal parts by a singer are integrated.

[1191] 8. Confirmation of Rights

[1192] The server scans a music database to ensure that the generated music data is not similar to existing copyrighted works, for example, to ensure that the generated song does not have a high degree of similarity to other existing songs.

[1193] 9. Publishing to a sharing site

[1194] If the user selects "I want to make it public," the server automatically uploads the song using the API of the sharing site. For example, the server calls the YouTube API and uploads the created song to YouTube.

[1195] 10. Adding a Digital Certificate

[1196] The server generates metadata about the song and uses blockchain technology to generate a digital certificate (NFT), which is then provided to the user. For example, the NFT registered on the blockchain digitally certifies the user's copyright.

[1197] The system of this invention allows users to easily and safely create music, share the music they create, and even prove their copyright. The introduction of an emotion engine makes it possible to generate music and lyrics that match the user's emotions, providing a more personalized musical experience.

[1198] The processing flow will be explained below.

[1199] Music production process using emotion engine

[1200] Step 1:

[1201] A user accesses the system through a terminal and launches an application. The user enters login information for authentication.

[1202] Step 2:

[1203] The user writes the phrase or instrument image they have in mind into the input field on the device. For example, they might type, "I want a bright guitar melody."

[1204] Step 3:

[1205] The device sends the user's input data to the server, which stores the received data and uses an emotion engine to recognize emotions from the input data.

[1206] Step 4:

[1207] The server analyzes the received data, and the emotion engine recognizes the user's emotions. For example, if someone says, "I want a bright guitar melody," the engine can detect emotions such as "joy" or "liveliness."

[1208] Step 5:

[1209] The server uses natural language processing (NLP) techniques to extract key keywords from the input data, such as "upbeat," "guitar," and "melody."

[1210] Step 6:

[1211] Based on the keywords and the recognized emotions, the server generates the next dialogue step and sends a question to the terminal, for example, "What tempo and key do you want to play in?"

[1212] Step 7:

[1213] The user responds to the question from the terminal with "Uptempo C major." The terminal then sends this response data to the server.

[1214] Step 8:

[1215] The server inputs the tempo, key, and recognized emotion into the music generation AI model to generate initial song data. For example, it creates an up-tempo melody in C major that reflects "joy."

[1216] Step 9:

[1217] The user requests, "I want you to write lyrics on the theme of love and youth" on the device. The device sends the request data to the server.

[1218] Step 10:

[1219] The server calls the lyric generation AI model and inputs the specified theme and emotion data. For example, it generates lyrics based on the theme of "love and youth" and the emotion of "joy."

[1220] Step 11:

[1221] The generated lyrics are stored on the server and integrated into the initial music data. At this stage, the server adds performance data and vocal data.

[1222] Step 12:

[1223] The server selects appropriate performance data from an existing performance database and integrates it into the song. Similarly, vocal data is added, for example, guitar parts by a professional guitarist or vocal parts by a singer.

[1224] Step 13:

[1225] The completed song data is stored on a server, which checks it against a music database to ensure it does not infringe on the rights of others.

[1226] Step 14:

[1227] If the server completes the rights verification and the user selects publishing, the server automatically uploads the song using the sharing site's API. For example, the server uploads the song to YouTube using the YouTube API.

[1228] Step 15:

[1229] The server generates metadata about the song and uses blockchain technology to create a digital certificate (NFT), which is then provided to the user.

[1230] The above are the specific processing steps of the system that generates and publishes music using an emotion engine.

[1231] Example 2

[1232] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1233] Conventional music production systems make it difficult for users to easily create music and lyrics that match their emotions, and the process of verifying that the generated music data does not infringe on the copyrights of others takes a lot of time and effort.Furthermore, there is a lack of means to publish the created music data and prove its copyright.

[1234] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1235] In this invention, the server includes means for receiving language data entered by a user, means for analyzing the received language data and extracting key keywords, means for generating music data based on the extracted keywords and the recognized emotion, means for adding performance data and vocal data to the generated music data, means for verifying that the added music data does not infringe the rights of others, means for publishing the verified music data on a sharing site, means for adding a digital certificate to the published music data, means for recognizing the user's emotion from the received language data, and means for adjusting the tone and content of the music data based on the recognized emotion. This allows users to easily create music and lyrics that match their emotions, enables the safe generation and publication of music data that does not infringe the rights of others, and furthermore, ensures that copyright is proven.

[1236] "User" refers to a person who uses the system to create and publish music and lyrics.

[1237] "Means for receiving" refers to a method or device by which the system receives language data input by a user.

[1238] "Means for analyzing" refers to a method or device that analyzes received language data using natural language processing (NLP) techniques and extracts key keywords.

[1239] "Key Keywords" refer to words and phrases that extract important information or instructions from the analyzed linguistic data.

[1240] "Emotion recognition" refers to the process of identifying a user's emotion from received linguistic data.

[1241] "Music data" refers to electronic data that includes generated musical components such as melodies, chords, and rhythms.

[1242] "Performance data" refers to specific performance information by musical instruments or other sound sources that is added to music data.

[1243] "Vocal data" refers to information about the singer's singing voice and vocal parts that is added to music data.

[1244] "Rights verification" refers to the process of checking whether the generated music data is similar to existing copyrighted works.

[1245] "Sharing site" refers to a web platform or online service for publishing generated music data.

[1246] A "digital certificate" refers to an electronic certificate generated using blockchain technology or other means to certify the copyright of published music data.

[1247] "Tone" refers to the tone and quality of an instrument used in music.

[1248] "Content" refers to the subject matter, themes, and expressions contained in the music data and lyrics.

[1249] This is a system that allows users to easily create and publish music based on their emotions. This system generates music data and lyrics based on user requests, adds performance data and vocal data to them, and after verifying that the copyrights of others are not being infringed, publishes the music on a sharing site and adds a digital certificate to prove copyright.

[1250] Hardware and software used

[1251] The system consists of a server, a terminal, and a user. The server uses a high-performance data processing device, including an AI model for executing natural language processing (NLP) techniques and a generative AI model for music generation and lyric generation. Specifically, the following software is used:

[1252] Natural Language Processing (NLP) technologies: Python NLP libraries (e.g., NLTK, spaCy)

[1253] Music generation AI model: Deep learning model using TensorFlow or PyTorch

[1254] Lyric generation AI models: OpenAI GPT-3 and GPT-4

[1255] Publishing to sharing sites uses APIs to connect to external services, such as YouTube API for publishing songs and blockchain technology for generating digital certificates.

[1256] Program processing explanation

[1257] User Input

[1258] Users access the system from their terminals, log in, and make specific requests to create music (e.g., "Please add a cheerful guitar melody").

[1259] Emotion recognition

[1260] The device sends the user's input data to the server, which uses an emotion engine to identify the user's emotion from this data. NLP technology is used in this process.

[1261] Data analysis and keyword extraction

[1262] The server uses NLP techniques to analyze the user's request and extract key keywords (e.g., "upbeat," "guitar," "melody").

[1263] Interactive question and answering

[1264] Based on the extracted keywords and the recognized emotions, the server generates the next dialogue step and sends it to the terminal in the form of a question (e.g., "What tempo and key do you want to play in?").

[1265] Basic music data generation

[1266] When a user responds with "uptempo C major," the server inputs this information into a music generation AI model to generate basic music data, and adjusts the tone and structure of the music data based on the emotion recognition results.

[1267] Lyrics generation

[1268] When a user requests lyrics on the theme of love and youth, the server calls a lyrics generation AI model and generates lyrics based on the specified theme and recognized emotions (e.g., lyrics with a bright tone).

[1269] Addition of performance and singing data

[1270] The server adds performance data and vocal data by professional musicians and singers to the generated music data (e.g., guitar parts by a professional guitarist and vocal parts by a singer).

[1271] Confirmation of rights

[1272] The server scans the generated music data to ensure it does not infringe on the rights of others (e.g., by using a music database to check for similarity to existing copyrighted works).

[1273] Publishing to a shared site

[1274] When the user selects "I want to make it public," the server uses the sharing site's API to make the song public (e.g., upload it to YouTube).

[1275] Attaching a Digital Certificate

[1276] The server generates metadata about the song and uses blockchain technology to generate a digital certificate (NFT), which users receive and their copyright is protected.

[1277] Examples and prompts

[1278] For example, a user might request, "I want a cheerful guitar melody," and the server would generate music data based on this request. For the user's next request, the server would use the AI ​​model to generate corresponding lyrics, which would then be published and digitally authenticated.

[1279] Prompt Sentence Examples

[1280] "Please identify the emotion from the user's input, such as 'I want a bright guitar melody,' and input 'joy, up-tempo, C major, bright guitar melody' into the generative AI model to generate a song."

[1281] This allows users to easily create music and lyrics that match their emotions, safely release them, and protect their copyrights.

[1282] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1283] Step 1: User login and request submission

[1284] Users access the system through a terminal and enter their login information for authentication. After successful authentication, they input specific requests for music production (e.g., "Please add a cheerful guitar melody.") The input data is received by the terminal and sent to the server.

[1285] Input: User login information and request (e.g., "Please add a cheerful guitar melody")

[1286] Output: The user's authenticated request data sent to the server.

[1287] Step 2: Recognize emotions

[1288] The server passes the received request data to the emotion engine, which uses NLP technology to identify emotions, extracting emotions such as "joy" and "liveliness" from the word "cheerful."

[1289] Input: User request data

[1290] Output: Identified emotion data (e.g., "joy")

[1291] Step 3: Data analysis and keyword extraction

[1292] After emotion recognition, the server uses NLP technology to analyze the user's request and extract key keywords, such as "cheerful," "guitar," and "melody."

[1293] Input: User request data with identified emotions

[1294] Output: Extracted keyword data (e.g., "cheerful," "guitar," "melody")

[1295] Step 4: Interactive question-answering

[1296] The server generates the next dialogue step based on the extracted keywords and the identified emotions, which is sent to the terminal, where the user is prompted with a question such as, "What tempo and key do you want to play in?"

[1297] Input: extracted keywords and identified emotions

[1298] Output: Questions to ask the user (e.g., "What tempo and key do you want to play in?")

[1299] Step 5: Generate basic music data

[1300] When a user responds with "uptempo C major," the server inputs this response into a music generation AI model to generate basic music data. The server also adjusts the tone and structure of the music data based on the identified emotions.

[1301] Input: User response data (e.g., "Uptempo C major") and identified emotion

[1302] Output: Generated basic music data

[1303] Step 6: Generate lyrics

[1304] When a user requests lyrics on the theme of love and youth, the server calls the lyric generation AI model and generates lyrics based on the specified theme and identified emotions. For example, it generates lyrics with a bright tone that evokes joy.

[1305] Input: User's theme request (e.g., "Love and Youth") and identified emotions

[1306] Output: Generated lyrics data

[1307] Step 7: Adding performance and singing data

[1308] The server adds performance data and singing data by professional musicians and singers to the generated music data. For example, guitar parts by a professional guitarist and vocal parts by a singer are integrated.

[1309] Input: Generated basic music data and lyrics data

[1310] Output: Complete music data with performance and vocal data added

[1311] Step 8: Entitlement Verification

[1312] The server scans the music database to ensure the generated music data does not infringe on the rights of others, and in the process checks to ensure that it is not too similar to existing songs.

[1313] Input: Complete music data

[1314] Output: Secure music data with rights verification

[1315] Step 9: Publish to a sharing site

[1316] When the user selects "I want to publish," the device sends this instruction to the server. The server publishes the song using the sharing site's API. For example, the server calls the YouTube API to upload the song to YouTube.

[1317] Input: User's publishing instructions

[1318] Output: Song data uploaded to a sharing site

[1319] Step 10: Attach a Digital Certificate

[1320] The server generates metadata about the song and uses blockchain technology to generate a digital certificate (NFT), which the user receives, digitally protecting the copyright of the song.

[1321] Input: Song data published on a sharing site

[1322] Output: Generated digital certificate (NFT)

[1323] (Application example 2)

[1324] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1325] Current music production systems lack sufficient means for users to easily generate music and lyrics that respond to their emotions and publish them to content distribution services. Furthermore, there is no established method for digitally verifying the copyright of the generated music data. This creates problems for users to create personalized music and share their creations securely and easily.

[1326] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving language data entered by a user, means for analyzing the received language data and generating music data based on key keywords and recognized emotions, and means for adding performance data and vocal data to the generated music data. This allows users to easily generate music and lyrics that correspond to their emotions and publish them on content distribution services. Furthermore, by verifying that the generated music data does not infringe on the rights of others and attaching a digital certificate, copyright verification can be easily performed.

[1327] "User" means any individual or organization that uses the System to create and share music.

[1328] "Language data" refers to text data such as sentences and phrases that users input into the system.

[1329] "Analysis" refers to the process of analyzing received linguistic data and extracting key keywords and information.

[1330] "Emotion" refers to psychological states such as joy, sadness, and excitement that are recognized from the user's input data.

[1331] "Music data" refers to digital data containing the melody, rhythm, and harmony of the generated music.

[1332] "Performance data" refers to digital data of musical instrument performances that are added to music data.

[1333] "Vocal data" refers to digital vocal data that is added to music data.

[1334] "Ensuring that the rights of others are not infringed" refers to the process of ensuring that the generated music data is not similar to existing copyrighted works.

[1335] "Content distribution service" refers to a platform for publishing and sharing generated music data on the Internet.

[1336] "Digital Certificate" refers to an electronic certificate generated using blockchain technology to protect the copyright of generated music data.

[1337] The system for implementing the present invention allows users to easily create and publish music based on language data entered by the user. A series of steps will be explained in detail below.

[1338] 1. System Configuration

[1339] It consists of three main modules: the server, the terminal, and the user. The server is a central processing unit that actually generates music and processes data. The terminal is a device operated by the user, such as a smartphone. The user is the person who creates music.

[1340] 2. User Input

[1341] A user accesses the system through a terminal and launches the application. After entering their login information for authentication, they write a phrase or an image of an instrument in the input field. For example, they might enter, "I want a bright guitar melody."

[1342] 3. Emotional Recognition

[1343] The device sends the user's input data to the server. The server stores the received data and uses an emotion recognition engine called EmotionRecognition to recognize emotions from the input data. For example, emotions such as "joy" or "liveliness" can be detected from the user's input.

[1344] 4. Input Analysis

[1345] The server applies Natural Language Processing (NLP) technology to the received language data to extract key keywords. The keywords extracted as a result of the analysis are "cheerful," "guitar," and "melody."

[1346] 5. Interactive Question-Answering

[1347] Based on the extracted keywords and the emotion recognition results, the server generates the next dialogue step and sends it to the terminal in the form of a question, such as "What tempo and key do you want to play in?"

[1348] 6. Music Generation

[1349] If the user responds with "uptempo C major," the server inputs this information into a music generation AI model called MusicGeneration to generate basic music data, and adjusts the tone and structure of the music data based on the results of emotion recognition.

[1350] 7. Lyric Generation

[1351] If a user requests lyrics with themes of love and youth, the server calls a lyric generation AI model called LyricsGeneration and inputs the specified theme. The AI ​​model generates lyrics based on the theme and the recognized emotions. For example, it automatically generates lyrics with a bright tone that conveys joy and vitality.

[1352] 8. Addition of performance and singing data

[1353] The server adds performance data and vocal data to the generated basic music data, using data from existing professional musicians and singers. For example, guitar parts by a professional guitarist and vocal parts by a singer are integrated.

[1354] 9. Confirmation of Rights

[1355] The server uses a module called CopyrightChecker to check whether the generated music data is similar to existing copyrighted works, for example, to ensure that the generated music does not have a high similarity to other existing music.

[1356] 10. Publication

[1357] If the user selects "I want to make it public," the server automatically uploads the song using the API of a content distribution service. For example, the server calls the API of a sharing site and uploads the created song to that site.

[1358] 11. Adding a Digital Certificate

[1359] The server generates metadata about the song and uses blockchain technology to generate a digital certificate, which is then provided to the user. For example, the certificate registered on the blockchain digitally certifies the user's copyright.

[1360] Examples of concrete examples and prompts

[1361] Example 1: When a user inputs "I want a bright guitar melody," music with a pop tempo is automatically generated.

[1362] Example 2: If the user selects "Love and Youth" as the theme, cheerful lyrics on the theme of love and youth are automatically generated.

[1363] Example prompt sentence:

[1364] "I want a bright guitar melody."

[1365] "I want you to write lyrics with themes of love and youth"

[1366] "Uptempo C major"

[1367] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1368] Step 1:

[1369] A user accesses the system through a terminal and launches an application. The user enters login information for authentication, then writes a phrase or an image of an instrument in the input field. The input is a prompt such as "Please add a cheerful guitar melody." The terminal sends this input data to the server.

[1370] Step 2:

[1371] The server stores the received user input data. Based on the stored data, an emotion recognition engine called EmotionRecognition is used to recognize emotions from the input data. For example, emotions such as "joy" or "liveliness" are recognized from the input. The input data is text data, and the output is emotional data.

[1372] Step 3:

[1373] The server applies Natural Language Processing (NLP) technology to the received language data to extract key keywords. The keywords extracted as a result of the analysis are "cheerful," "guitar," and "melody." Text analysis is used to extract these keywords, with the input being language data and the output being keyword data.

[1374] Step 4:

[1375] The server generates the next dialogue step based on the extracted keywords and the emotion recognition results and sends it to the terminal in the form of a question. For example, it sends a question such as, "What tempo and key do you want to play at?" The input is keywords and emotion data, and the output is the dialogue step question.

[1376] Step 5:

[1377] The user answers questions from the server interactively through the terminal. For example, the user might answer "uptempo C major." The terminal then sends this answer data back to the server. The input is the user's answer, and the output is text data sent to the server.

[1378] Step 6:

[1379] The server inputs user response data and emotional data into a music generation AI model called MusicGeneration to generate basic music data. It also adjusts the tone and structure of the music data based on the results of emotion recognition. The input data are tempo, key, and emotional data, and the output is music data.

[1380] Step 7:

[1381] If the user again requests "Please create lyrics with the theme of love and youth," the server will call a lyrics generation AI model called LyricsGeneration and input the specified theme. The AI ​​model will generate lyrics based on the theme and the recognized emotion. The input is theme and emotion data, and the output is lyric data.

[1382] Step 8:

[1383] The server adds performance and vocal data to the generated basic music data. This uses data from existing professional musicians and singers. For example, guitar parts by a professional guitarist and vocal parts by a singer are integrated. The input is music data and additional performance and vocal data, and the output is the completed music data.

[1384] Step 9:

[1385] The server checks whether the generated music data is similar to existing copyrighted works through a module called CopyrightChecker. For example, it checks whether the generated music is too similar to other existing music. The input is the completed music data, and the output is the result of copyright check.

[1386] Step 10:

[1387] If the user selects "I want to publish," the server automatically uploads the song using the API of a content distribution service. For example, the server calls the API of a sharing site and uploads the created song to that site. The input is the music data and the publishing request, and the output is the upload result to the sharing site.

[1388] Step 11:

[1389] The server generates metadata about the music and generates a digital certificate using blockchain technology. The generated certificate information is provided to the user. For example, the certificate registered on the blockchain digitally certifies the user's copyright. The input is the metadata of the music data, and the output is a digital certificate.

[1390] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1391] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1392] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1393] [Fourth embodiment]

[1394] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1395] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1396] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1397] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1398] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1399] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1400] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1401] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1402] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1403] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1404] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1405] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1406] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1407] ---

[1408] This invention provides a system that allows users to easily create and publish music. This system generates music data based on language data entered by the user, and adds performance and vocal data. It also verifies that the generated music data does not infringe on the rights of others, publishes it on a sharing site, and certifies the copyright by adding a digital certificate.

[1409] Explaining the music production process

[1410] 1. User Input

[1411] Users access the application through their device and write down phrases or images of instruments they have in mind in the input field.

[1412] Example: A user types, "I want a cheerful guitar melody."

[1413] 2. Input Analysis

[1414] The server receives the user's phrase data sent from the terminal and extracts key keywords using natural language processing (NLP) techniques.

[1415] Example: Extract the keywords "bright," "guitar," and "melody."

[1416] 3. Interactive question-answering

[1417] As a next step, the server generates interactive questions for the user based on the analysis results.

[1418] Example: Displaying questions on the screen such as "What tempo and key do you want to play in?"

[1419] 4. Music Generation

[1420] If the user responds with "uptempo C major," the server inputs this information into a music generation AI model and generates basic music data.

[1421] Example: An AI model automatically generates an up-tempo melody and accompaniment in the key of C major.

[1422] 5. Lyric Generation

[1423] If a user requests "I want lyrics written on the theme of love and youth," the server inputs this theme into the lyrics generation model and generates lyrics.

[1424] Example: Lyrics on the theme of "love and youth" are automatically created by AI.

[1425] 6. Adding performance and singing data

[1426] The server adds performance data and singing data to the generated basic music data, using data from existing professional musicians and singers.

[1427] Example: Guitar parts by a professional guitarist and vocal parts by a singer are integrated.

[1428] 7. Confirmation of Rights

[1429] The server scans a music database to check whether the generated music data is similar to existing copyrighted works.

[1430] Example: Making sure the generated songs are not too similar to other existing songs.

[1431] 8. Publishing to a sharing site

[1432] If the user selects "I want to make it public," the server will automatically upload the song using the sharing site's API.

[1433] Example: The server calls the YouTube API and uploads the generated song to YouTube.

[1434] 9. Adding a Digital Certificate

[1435] The server encrypts the metadata associated with the generated song and registers it as an NFT using blockchain technology.

[1436] Example: NFTs registered on the blockchain digitally prove a user's copyright.

[1437] In this way, the system of the present invention provides a variety of functions that allow users to easily and safely create music, share the music they create, and even verify copyright. The above process creates an environment in which users can create and publish original music and songs without specialized knowledge.

[1438] The processing flow will be explained below.

[1439] Step 1:

[1440] A user accesses the system through a terminal and launches an application. The user enters login information for authentication.

[1441] Step 2:

[1442] The user writes the phrase or instrument image they have in mind into the input field on the device. For example, they might type, "I want a bright guitar melody."

[1443] Step 3:

[1444] The device sends the user's input data to the server, which stores the received data and prepares it for analysis.

[1445] Step 4:

[1446] The server applies natural language processing (NLP) technology to the received language data to extract key keywords. As a result of the analysis, the keywords "cheerful," "guitar," and "melody" are identified.

[1447] Step 5:

[1448] The server generates the next dialogue step based on the extracted keywords and sends it to the terminal in the form of a question, such as "What tempo and key do you want to play in?"

[1449] Step 6:

[1450] The user responds to the question from the terminal by entering "Uptempo C major." The terminal sends the response data to the server.

[1451] Step 7:

[1452] The server receives the user's response data and calls the music generation AI model, which inputs tempo and key information to generate the initial song data.

[1453] Step 8:

[1454] The generated initial song data is saved on the server. Next, the user inputs their desired lyrics. For example, they might input "I want lyrics with themes of love and youth."

[1455] Step 9:

[1456] The device sends the user's lyrics request to the server, which then calls the lyric generation AI model and inputs the specified theme. The AI ​​model generates lyrics based on the theme.

[1457] Step 10:

[1458] The generated lyrics are stored on the server and integrated into the initial music data. The server then adds performance data and vocal data.

[1459] Step 11:

[1460] The server selects appropriate performance data from the existing performance database and integrates it into the song. Singing data is also added in the same way.

[1461] Step 12:

[1462] The completed song data is stored on a server, which scans the database to check whether the song data infringes on the rights of others.

[1463] Step 13:

[1464] The server completes the rights verification and prepares to publish the file on the sharing site according to the user's selection. The user selects "I want to publish" on the terminal.

[1465] Step 14:

[1466] The server automatically uploads the music using the API of the sharing site (e.g. YouTube) and notifies the user when the upload is complete.

[1467] Step 15:

[1468] The server generates metadata about the song and uses blockchain technology to generate a digital certificate (NFT), which is then provided to the user.

[1469] The above are the specific processing steps of the music generation AI system.

[1470] Example 1

[1471] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1472] Conventional music production systems require users to configure music data in detail, often requiring specialized knowledge and skills. Furthermore, the process of verifying that the generated music data does not infringe on the rights of others is cumbersome. As a result, music production has been difficult and time-consuming, creating a demand for an environment where anyone can easily create original music and share it safely.

[1473] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1474] In this invention, the server includes means for receiving language data entered by a user, means for analyzing the received language data and extracting key keywords, means for generating music data based on the extracted keywords, means for adding performance data and vocal data to the generated music data, means for verifying that the added music data does not infringe the rights of others, means for publishing the verified music data on a sharing site, means for adding a digital certificate to the published music data, means for scanning and verifying existing databases for the generated music data, and means for encrypting metadata related to the generated music and registering it as an NFT using blockchain technology. This enables users to easily create original music and share it safely and efficiently, even without specialized knowledge.

[1475] "User" refers to a person who intends to use this system to generate music data.

[1476] "Language data" refers to text data such as phrases and instrument images entered by users.

[1477] "Server" refers to a central processing unit that executes the processing of the entire system and manages data.

[1478] "Analysis" refers to the process by which the server processes the received language data and extracts key keywords.

[1479] "Key keywords" refer to important words or phrases extracted from language data.

[1480] "Music data" refers to musical information such as melodies and accompaniments generated by generative AI models.

[1481] "Performance data" refers to musical instrument performance information provided by professional musicians.

[1482] "Singing data" refers to vocal audio information provided by a singer.

[1483] "Ensuring that it does not infringe on the rights of others" refers to the process of checking whether the generated music data is similar to existing copyrighted works.

[1484] "Sharing site" refers to an online platform for sharing generated music data with other users.

[1485] "Digital certificate" refers to electronic evidence that proves the copyright of generated music data.

[1486] "Scanning the database" refers to the process by which the server searches the existing music database to assess similarity.

[1487] "Metadata" refers to additional information associated with the generated song (e.g., title, artist name).

[1488] "Encryption" refers to the process of using cryptography to protect data from unauthorized access.

[1489] "Blockchain technology" refers to a distributed data management technology that prevents data tampering and ensures traceability.

[1490] "NFT" stands for Non-Fungible Token and refers to a means of proving ownership of digital assets on the blockchain.

[1491] This invention provides a system that allows users to easily create and publish music. This system generates music data based on language data entered by the user and has the function of adding performance data and vocal data. It also verifies that the generated music data does not infringe on the rights of others, publishes it on a sharing site, and certifies the copyright by adding a digital certificate.

[1492] Overall system configuration

[1493] Hardware and Software Configuration

[1494] Server: A central processing unit that analyzes, generates, checks, and shares music data. Specifically, it is installed with software libraries for natural language processing (NLP) techniques (e.g., Spacy, NLTK) and music generation AI models (e.g., OpenAI's GPT-3, Google's Magenta).

[1495] Terminal: A device that a user accesses and inputs data from. Examples include smartphones, tablet PCs, and desktop PCs.

[1496] Sharing site: An online platform for publishing generated music data. Examples include YouTube and SoundCloud.

[1497] Data Processing Description

[1498] 1. User Input

[1499] The user accesses the application through their device and writes down the phrases or instrument images they have in mind in the input field. For example, the user might enter, "I want a bright guitar melody."

[1500] 2. Input analysis and keyword extraction

[1501] The server receives the user's phrase data sent from the device and extracts key keywords using natural language processing technology. Specifically, the server analyzes the input data and extracts keywords such as "cheerful," "guitar," and "melody."

[1502] 3. Interactive question-answering

[1503] The next step is for the server to generate interactive questions for the user based on the analysis results, such as "What tempo and key do you want to play in?", and display them to the user.

[1504] 4. Music Generation

[1505] If the user responds with "uptempo C major," the server inputs this information into the music generation AI model and generates basic music data. As a specific example, the server inputs the prompt "Generate a melody and accompaniment in uptempo C major" into the generation AI model and obtains an automatically generated melody and accompaniment.

[1506] 5. Lyric Generation

[1507] If a user requests "I want lyrics with the theme of love and youth," the server inputs this theme into the lyrics generation model and generates lyrics. As a specific example, the server inputs a prompt sentence with the theme of "love and youth" into the lyrics generation model and obtains the generated lyrics.

[1508] 6. Adding performance and singing data

[1509] The server adds performance data and vocal data to the generated basic music data. This data is created using data from existing professional musicians and singers. For example, the server integrates guitar parts by existing professional musicians and vocal parts by existing professional singers into the basic music data.

[1510] 7. Confirmation of Rights

[1511] The server scans a music database to check whether the generated music data is similar to existing copyrighted works. For example, the server crawls a music database and uses a similarity checking algorithm to evaluate the similarity between the generated music and existing music.

[1512] 8. Publishing to a sharing site

[1513] If the user selects "I want to make it public," the server automatically uploads the music using the sharing site's API. For example, the server calls the sharing site's API (e.g., YouTube API), provides the necessary authentication information, and then uploads the generated music file.

[1514] 9. Adding a Digital Certificate

[1515] The server encrypts metadata related to the generated music and registers it as an NFT using blockchain technology. For example, the server encrypts the music metadata and generates a transaction on a blockchain platform (e.g., Ethereum) to register the NFT and obtain a digital certificate.

[1516] In this way, the system of the present invention provides a variety of functions for users to easily and safely create music, share the music they create, and verify copyright, creating an environment in which users can create and publish original music and songs without specialized knowledge.

[1517] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1518] Specific processing steps of the system

[1519] Step 1:

[1520] The user accesses the application using a terminal and writes down the phrases or instrument images that come to mind in the input field.

[1521] Input data: Language data entered by the user (e.g., "Please include a cheerful guitar melody")

[1522] Output data: Transmitted language data

[1523] Step 2:

[1524] The server receives the user's language data sent from the device, then analyzes the data using natural language processing (NLP) technology to extract key keywords.

[1525] Input data: Transmitted language data

[1526] Data processing: Using NLP technology, key keywords such as "upbeat," "guitar," and "melody" are extracted.

[1527] Output data: extracted main keywords

[1528] Step 3:

[1529] As a next step, the server generates interactive questions for the user based on the extracted keywords, which allow the user to confirm detailed settings for the music data to be generated.

[1530] Input data: extracted main keywords

[1531] Data processing: Question generation (e.g., "What tempo and key do you play in?")

[1532] Output data: Generated question

[1533] Step 4:

[1534] The user inputs a response to a question from the server.

[1535] Input data: User response (e.g., "Uptempo C major")

[1536] Output data: User response data

[1537] Step 5:

[1538] The server uses the user's response data to input prompt sentences into the music generation AI model and generate basic music data.

[1539] Input data: User response data (e.g., "Generate a melody and accompaniment in uptempo C major.")

[1540] Data Computing: Running Music Generation AI Models

[1541] Output data: generated melody and accompaniment

[1542] Step 6:

[1543] If the user requests, the server also generates additional musical parts, such as lyrics. The server inputs the user's request into the lyrics generation model and generates lyrics.

[1544] Input data: User request (e.g., "Please write lyrics with themes of love and youth")

[1545] Data calculation: Running the lyrics generation model

[1546] Output: Generated lyrics

[1547] Step 7:

[1548] The server adds performance data and singing data to the generated basic music data, using data from existing professional musicians and singers.

[1549] Input data: basic music data, performance data of professional musicians, singing data of professional singers

[1550] Data processing: Data integration

[1551] Output data: Finished music data

[1552] Step 8:

[1553] The server scans existing music databases to ensure that the generated music data does not infringe on the rights of others.

[1554] Input data: Completed music data

[1555] Data calculation: scanning music databases and checking similarities

[1556] Output data: Infringement confirmation results

[1557] Step 9:

[1558] If the user selects "I want to make it public," the server will automatically upload the song using the sharing site's API.

[1559] Input data: Completed music data, API credentials for the sharing site

[1560] Data Calculation: Upload data via API

[1561] Output data: Songs published on a sharing site

[1562] Step 10:

[1563] The server encrypts the metadata associated with the generated song and registers it as an NFT using blockchain technology.

[1564] Input data: Song metadata

[1565] Data computation: encryption and blockchain registration

[1566] Output data: Digital certificate (NFT)

[1567] Through these steps, the system provides users with a variety of functions to easily and safely create music, share the music they create, and even prove their copyright.

[1568] (Application example 1)

[1569] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1570] Conventional music production systems lack the means for users to easily create and publish music, and to protect the copyright of that music. This makes it difficult for users without specialized knowledge to create original music, resulting in a high barrier to entry for music production. Furthermore, there is a lack of means to prove the copyright of published music, which means users are constantly exposed to the risk of copyright infringement. There is a need to solve these issues.

[1571] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1572] In this invention, the server includes: means for receiving language data input by a user; means for analyzing the received language data and extracting key keywords; means for generating music data based on the extracted keywords; means for adding performance data and vocal data to the generated music data; means for verifying that the added music data does not infringe the rights of others; means for publishing the verified music data on a distribution platform; means for attaching a digital certificate to the published music data using blockchain technology; means for analyzing the user's input using natural language processing technology and generating a prompt; and means for inputting the prompt into a generative AI model to generate music data and lyrics. This allows users to easily create music and publish the music on a distribution platform, while protecting their copyrights with digital certificates using blockchain technology.

[1573] "Language data entered by the user" refers to text information entered by the user through the application when the user wishes to create music.

[1574] "Major keywords" are words and phrases that are important for music generation and are extracted from the input linguistic data.

[1575] "Music data generation" means that the music generation AI model automatically creates the basic structure of a song based on the extracted main keywords.

[1576] "Performance data" is data that includes information about performances by musical instruments, and is an element that is added to music data.

[1577] "Vocal data" is data that includes vocal performances accompanied by lyrics, and is an element that is added to music data.

[1578] "Confirmation of non-infringement" is the process of checking whether the generated music data is highly similar to existing copyrighted works.

[1579] A "distribution platform" is an online service or system used to make generated music data available to the public.

[1580] "Blockchain technology" is a distributed ledger technology that links each data block to prevent data tampering and increase reliability.

[1581] A "Digital Certificate" is a digital certificate for certifying copyright that contains metadata related to published music data and is generated using blockchain technology.

[1582] "Natural language processing technology" is an artificial intelligence technology for analyzing, understanding, and generating human language, and is used to extract key keywords from input language data.

[1583] A "prompt" is a sentence input into a music generation AI model, and is an instruction statement that defines the characteristics of the music data to be generated.

[1584] A "generative AI model" is an artificial intelligence model used to automatically generate music data and lyrics.

[1585] The present invention relates to a system that allows users to easily create and publish music. Details of this system and an embodiment thereof will be described below.

[1586] Music production system configuration

[1587] The system includes the following main functions:

[1588] 1. Means for receiving user input:

[1589] The user uses their own device (such as a smartphone or head-mounted display) to input phrases or images of musical instruments that come to mind into the application. For example, the user might input, "I want a bright guitar melody." This text data is then sent to the server.

[1590] 2. Linguistic data analysis and keyword extraction:

[1591] The server analyzes the received language data using natural language processing technology (such as OpenAI's API) and extracts key keywords, such as "cheerful," "guitar," and "melody."

[1592] 3. Generate prompt sentence:

[1593] The server generates a prompt based on the analysis results. This prompt is a statement that instructs the music generation AI model. For example, the prompt might say, "The music to be generated will be up-tempo, in the key of C major, and based on a bright guitar melody."

[1594] 4. Music data and lyrics generation:

[1595] The server inputs the generated prompt sentence into the AI ​​model and generates music data and lyrics if necessary. For example, if a user requests "I want lyrics on the theme of love and youth," the lyrics generation AI model will generate lyrics on the theme of "love and youth."

[1596] 5. Adding performance and vocal data:

[1597] The server adds performance data and vocal data to the generated basic music data. This data is created using data from existing professional musicians and singers. For example, guitar parts by a professional guitarist and vocal parts by a singer are integrated.

[1598] 6. Acknowledgement of Rights:

[1599] The server scans an existing music database to determine whether the generated music data is similar to other existing copyrighted works. It verifies that the generated music does not have a high degree of similarity to other existing music.

[1600] 7. Publishing your music:

[1601] If the user selects "I want to make it public," the server will automatically upload the created song using the distribution platform's API (such as YouTube API).

[1602] 8. Adding a digital certificate:

[1603] The server then encrypts the metadata associated with the generated music and registers it as a digital certificate (NFT) using blockchain technology, digitally certifying the user's copyright.

[1604] Specific examples of functions

[1605] For example, if the user types "I want a cheerful guitar melody," the system will generate the following prompt:

[1606] The music it generates is upbeat in tempo, set in the key of C major, and based on bright, guitar-driven melodies.

[1607] Similarly, if the user requests "Please write lyrics on the themes of love and youth," we generate the following prompt:

[1608] Generate lyrics on the themes of love and youth.

[1609] With these functions, the system of the present invention allows users to easily create music, publish it on a distribution platform, and protect copyright with digital certificates using blockchain technology.

[1610] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1611] Step 1:

[1612] The server receives the user's input data from the device. For example, the user might say, "I want a cheerful guitar melody," and the device sends that data to the server. This input data becomes the basis for the next analysis step.

[1613] Step 2:

[1614] The server analyzes the received language data using natural language processing technology. Specifically, it uses OpenAI's API to extract key keywords from the input text. For example, "cheerful," "guitar," and "melody" are extracted. This keyword extraction clarifies the elements of music generation.

[1615] Step 3:

[1616] The server generates a prompt based on the extracted keywords. For example, it creates a prompt such as, "The music to be generated will be up-tempo, in the key of C major, and based on a bright guitar melody." This prompt becomes the instruction input to the music generation AI model.

[1617] Step 4:

[1618] The server inputs the generated prompt into a music generation AI model to generate music data. At this time, the tempo, key, type of instrument, etc. are specified. For example, the AI ​​model automatically generates a melody and accompaniment based on an up-tempo C major key. The generated music data is then passed to the next step.

[1619] Step 5:

[1620] The server inputs the theme into the lyric generation model based on the user's request and generates lyrics. For example, if a user requests "I want lyrics with the theme of love and youth," the lyric generation AI model will generate lyrics with the theme of "love and youth." This lyric data is then integrated with the music data.

[1621] Step 6:

[1622] The server adds performance data and vocal data to the generated music data. Specifically, it integrates data from professional musicians and singers. For example, a guitarist adds a guitar part and a vocalist adds a vocal part. This results in more complete music data.

[1623] Step 7:

[1624] The server scans the existing music database to verify the rights of the generated music data. Here, it checks that the generated song does not have a high similarity to other existing songs. If there is a high similarity, it notifies the user. The verified data proceeds to the next step.

[1625] Step 8:

[1626] The server then publishes the verified music data to the distribution platform. Specifically, the music data is automatically uploaded using the distribution platform's API. For example, the server calls the distribution platform's API and performs the procedure to upload the music.

[1627] Step 9:

[1628] The server then adds a digital certificate to the published music data. Specifically, it encrypts the metadata associated with the generated music and registers it as an NFT using blockchain technology. This digitally certifies the user's copyright.

[1629] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1630] ---

[1631] This invention provides a system that allows users to easily create and publish music. This system generates music data based on language data entered by the user, and adds performance and vocal data. It also verifies that the generated music data does not infringe on the rights of others, publishes it on a sharing site, and certifies the copyright by adding a digital certificate.

[1632] Additionally, the present invention incorporates an emotion engine that recognizes the user's emotions and has the ability to adjust the tone and content of music data and lyrics based on the recognized emotions.

[1633] Explaining the music production process using the emotion engine

[1634] 1. User Input

[1635] A user accesses the system through a terminal and launches the application. After entering their login information and performing authentication, they write a phrase or an image of an instrument in the input field. For example, they might enter, "I want a bright guitar melody."

[1636] 2. Emotional Recognition

[1637] The device sends the user's input data to the server, which stores the data and uses an emotion engine to recognize emotions from the input data. For example, emotions such as "joy" or "liveliness" can be detected from the user's input.

[1638] 3. Input Analysis

[1639] The server applies natural language processing (NLP) technology to the received language data to extract key keywords. The keywords extracted as a result of the analysis are "cheerful," "guitar," and "melody."

[1640] 4. Interactive question-answering

[1641] Based on the extracted keywords and the emotion recognition results, the server generates the next dialogue step and sends it to the terminal in the form of a question, such as "What tempo and key do you want to play in?"

[1642] 5. Music Generation

[1643] If the user responds with "uptempo C major," the server inputs this information into the music generation AI model to generate basic music data, and adjusts the tone and structure of the music data based on the emotion recognition results.

[1644] 6. Lyric Generation

[1645] If a user requests lyrics with themes of love and youth, the server will call the lyric generation AI model and input the specified theme. The AI ​​model will generate lyrics based on the theme and the recognized emotions. For example, it will automatically generate lyrics with a bright tone that conveys joy and vitality.

[1646] 7. Addition of performance and singing data

[1647] The server adds performance data and vocal data to the generated basic music data, using data from existing professional musicians and singers. For example, guitar parts by a professional guitarist and vocal parts by a singer are integrated.

[1648] 8. Confirmation of Rights

[1649] The server scans a music database to ensure that the generated music data is not similar to existing copyrighted works, for example, to ensure that the generated song does not have a high degree of similarity to other existing songs.

[1650] 9. Publishing to a sharing site

[1651] If the user selects "I want to make it public," the server automatically uploads the song using the API of the sharing site. For example, the server calls the YouTube API and uploads the created song to YouTube.

[1652] 10. Adding a Digital Certificate

[1653] The server generates metadata about the song and uses blockchain technology to generate a digital certificate (NFT), which is then provided to the user. For example, the NFT registered on the blockchain digitally certifies the user's copyright.

[1654] The system of this invention allows users to easily and safely create music, share the music they create, and even prove their copyright. The introduction of an emotion engine makes it possible to generate music and lyrics that match the user's emotions, providing a more personalized musical experience.

[1655] The processing flow will be explained below.

[1656] Music production process using emotion engine

[1657] Step 1:

[1658] A user accesses the system through a terminal and launches an application. The user enters login information for authentication.

[1659] Step 2:

[1660] The user writes the phrase or instrument image they have in mind into the input field on the device. For example, they might type, "I want a bright guitar melody."

[1661] Step 3:

[1662] The device sends the user's input data to the server, which stores the received data and uses an emotion engine to recognize emotions from the input data.

[1663] Step 4:

[1664] The server analyzes the received data, and the emotion engine recognizes the user's emotions. For example, if someone says, "I want a bright guitar melody," the engine can detect emotions such as "joy" or "liveliness."

[1665] Step 5:

[1666] The server uses natural language processing (NLP) techniques to extract key keywords from the input data, such as "upbeat," "guitar," and "melody."

[1667] Step 6:

[1668] Based on the keywords and the recognized emotions, the server generates the next dialogue step and sends a question to the terminal, for example, "What tempo and key do you want to play in?"

[1669] Step 7:

[1670] The user responds to the question from the terminal with "Uptempo C major." The terminal then sends this response data to the server.

[1671] Step 8:

[1672] The server inputs the tempo, key, and recognized emotion into the music generation AI model to generate initial song data. For example, it creates an up-tempo melody in C major that reflects "joy."

[1673] Step 9:

[1674] The user requests, "I want you to write lyrics on the theme of love and youth" on the device. The device sends the request data to the server.

[1675] Step 10:

[1676] The server calls the lyric generation AI model and inputs the specified theme and emotion data. For example, it generates lyrics based on the theme of "love and youth" and the emotion of "joy."

[1677] Step 11:

[1678] The generated lyrics are stored on the server and integrated into the initial music data. At this stage, the server adds performance data and vocal data.

[1679] Step 12:

[1680] The server selects appropriate performance data from an existing performance database and integrates it into the song. Similarly, vocal data is added, for example, guitar parts by a professional guitarist or vocal parts by a singer.

[1681] Step 13:

[1682] The completed song data is stored on a server, which checks it against a music database to ensure it does not infringe on the rights of others.

[1683] Step 14:

[1684] If the server completes the rights verification and the user selects publishing, the server automatically uploads the song using the sharing site's API. For example, the server uploads the song to YouTube using the YouTube API.

[1685] Step 15:

[1686] The server generates metadata about the song and uses blockchain technology to create a digital certificate (NFT), which is then provided to the user.

[1687] The above are the specific processing steps of the system that generates and publishes music using an emotion engine.

[1688] Example 2

[1689] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1690] Conventional music production systems make it difficult for users to easily create music and lyrics that match their emotions, and the process of verifying that the generated music data does not infringe on the copyrights of others takes a lot of time and effort.Furthermore, there is a lack of means to publish the created music data and prove its copyright.

[1691] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1692] In this invention, the server includes means for receiving language data entered by a user, means for analyzing the received language data and extracting key keywords, means for generating music data based on the extracted keywords and the recognized emotion, means for adding performance data and vocal data to the generated music data, means for verifying that the added music data does not infringe the rights of others, means for publishing the verified music data on a sharing site, means for adding a digital certificate to the published music data, means for recognizing the user's emotion from the received language data, and means for adjusting the tone and content of the music data based on the recognized emotion. This allows users to easily create music and lyrics that match their emotions, enables the safe generation and publication of music data that does not infringe the rights of others, and furthermore, ensures that copyright is proven.

[1693] "User" refers to a person who uses the system to create and publish music and lyrics.

[1694] "Means for receiving" refers to a method or device by which the system receives language data input by a user.

[1695] "Means for analyzing" refers to a method or device that analyzes received language data using natural language processing (NLP) techniques and extracts key keywords.

[1696] "Key Keywords" refer to words and phrases that extract important information or instructions from the analyzed linguistic data.

[1697] "Emotion recognition" refers to the process of identifying a user's emotion from received linguistic data.

[1698] "Music data" refers to electronic data that includes generated musical components such as melodies, chords, and rhythms.

[1699] "Performance data" refers to specific performance information by musical instruments or other sound sources that is added to music data.

[1700] "Vocal data" refers to information about the singer's singing voice and vocal parts that is added to music data.

[1701] "Rights verification" refers to the process of checking whether the generated music data is similar to existing copyrighted works.

[1702] "Sharing site" refers to a web platform or online service for publishing generated music data.

[1703] A "digital certificate" refers to an electronic certificate generated using blockchain technology or other means to certify the copyright of published music data.

[1704] "Tone" refers to the tone and quality of an instrument used in music.

[1705] "Content" refers to the subject matter, themes, and expressions contained in the music data and lyrics.

[1706] This is a system that allows users to easily create and publish music based on their emotions. This system generates music data and lyrics based on user requests, adds performance data and vocal data to them, and after verifying that the copyrights of others are not being infringed, publishes the music on a sharing site and adds a digital certificate to prove copyright.

[1707] Hardware and software used

[1708] The system consists of a server, a terminal, and a user. The server uses a high-performance data processing device, including an AI model for executing natural language processing (NLP) techniques and a generative AI model for music generation and lyric generation. Specifically, the following software is used:

[1709] Natural Language Processing (NLP) technologies: Python NLP libraries (e.g., NLTK, spaCy)

[1710] Music generation AI model: Deep learning model using TensorFlow or PyTorch

[1711] Lyric generation AI models: OpenAI GPT-3 and GPT-4

[1712] Publishing to sharing sites uses APIs to connect to external services, such as YouTube API for publishing songs and blockchain technology for generating digital certificates.

[1713] Program processing explanation

[1714] User Input

[1715] Users access the system from their terminals, log in, and make specific requests to create music (e.g., "Please add a cheerful guitar melody").

[1716] Emotion recognition

[1717] The device sends the user's input data to the server, which uses an emotion engine to identify the user's emotion from this data. NLP technology is used in this process.

[1718] Data analysis and keyword extraction

[1719] The server uses NLP techniques to analyze the user's request and extract key keywords (e.g., "upbeat," "guitar," "melody").

[1720] Interactive question and answering

[1721] Based on the extracted keywords and the recognized emotions, the server generates the next dialogue step and sends it to the terminal in the form of a question (e.g., "What tempo and key do you want to play in?").

[1722] Basic music data generation

[1723] When a user responds with "uptempo C major," the server inputs this information into a music generation AI model to generate basic music data, and adjusts the tone and structure of the music data based on the emotion recognition results.

[1724] Lyrics generation

[1725] When a user requests lyrics on the theme of love and youth, the server calls a lyrics generation AI model and generates lyrics based on the specified theme and recognized emotions (e.g., lyrics with a bright tone).

[1726] Addition of performance and singing data

[1727] The server adds performance data and vocal data by professional musicians and singers to the generated music data (e.g., guitar parts by a professional guitarist and vocal parts by a singer).

[1728] Confirmation of rights

[1729] The server scans the generated music data to ensure it does not infringe on the rights of others (e.g., by using a music database to check for similarity to existing copyrighted works).

[1730] Publishing to a shared site

[1731] When the user selects "I want to make it public," the server uses the sharing site's API to make the song public (e.g., upload it to YouTube).

[1732] Attaching a Digital Certificate

[1733] The server generates metadata about the song and uses blockchain technology to generate a digital certificate (NFT), which users receive and their copyright is protected.

[1734] Examples and prompts

[1735] For example, a user might request, "I want a cheerful guitar melody," and the server would generate music data based on this request. For the user's next request, the server would use the AI ​​model to generate corresponding lyrics, which would then be published and digitally authenticated.

[1736] Prompt Sentence Examples

[1737] "Please identify the emotion from the user's input, such as 'I want a bright guitar melody,' and input 'joy, up-tempo, C major, bright guitar melody' into the generative AI model to generate a song."

[1738] This allows users to easily create music and lyrics that match their emotions, safely release them, and protect their copyrights.

[1739] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1740] Step 1: User login and request submission

[1741] Users access the system through a terminal and enter their login information for authentication. After successful authentication, they input specific requests for music production (e.g., "Please add a cheerful guitar melody.") The input data is received by the terminal and sent to the server.

[1742] Input: User login information and request (e.g., "Please add a cheerful guitar melody")

[1743] Output: The user's authenticated request data sent to the server.

[1744] Step 2: Recognize emotions

[1745] The server passes the received request data to the emotion engine, which uses NLP technology to identify emotions, extracting emotions such as "joy" and "liveliness" from the word "cheerful."

[1746] Input: User request data

[1747] Output: Identified emotion data (e.g., "joy")

[1748] Step 3: Data analysis and keyword extraction

[1749] After emotion recognition, the server uses NLP technology to analyze the user's request and extract key keywords, such as "cheerful," "guitar," and "melody."

[1750] Input: User request data with identified emotions

[1751] Output: Extracted keyword data (e.g., "cheerful," "guitar," "melody")

[1752] Step 4: Interactive question-answering

[1753] The server generates the next dialogue step based on the extracted keywords and the identified emotions, which is sent to the terminal, where the user is prompted with a question such as, "What tempo and key do you want to play in?"

[1754] Input: extracted keywords and identified emotions

[1755] Output: Questions to ask the user (e.g., "What tempo and key do you want to play in?")

[1756] Step 5: Generate basic music data

[1757] When a user responds with "uptempo C major," the server inputs this response into a music generation AI model to generate basic music data. The server also adjusts the tone and structure of the music data based on the identified emotions.

[1758] Input: User response data (e.g., "Uptempo C major") and identified emotion

[1759] Output: Generated basic music data

[1760] Step 6: Generate lyrics

[1761] When a user requests lyrics on the theme of love and youth, the server calls the lyric generation AI model and generates lyrics based on the specified theme and identified emotions. For example, it generates lyrics with a bright tone that evokes joy.

[1762] Input: User's theme request (e.g., "Love and Youth") and identified emotions

[1763] Output: Generated lyrics data

[1764] Step 7: Adding performance and singing data

[1765] The server adds performance data and singing data by professional musicians and singers to the generated music data. For example, guitar parts by a professional guitarist and vocal parts by a singer are integrated.

[1766] Input: Generated basic music data and lyrics data

[1767] Output: Complete music data with performance and vocal data added

[1768] Step 8: Entitlement Verification

[1769] The server scans the music database to ensure the generated music data does not infringe on the rights of others, and in the process checks to ensure that it is not too similar to existing songs.

[1770] Input: Complete music data

[1771] Output: Secure music data with rights verification

[1772] Step 9: Publish to a sharing site

[1773] When the user selects "I want to publish," the device sends this instruction to the server. The server publishes the song using the sharing site's API. For example, the server calls the YouTube API to upload the song to YouTube.

[1774] Input: User's publishing instructions

[1775] Output: Song data uploaded to a sharing site

[1776] Step 10: Attach a Digital Certificate

[1777] The server generates metadata about the song and uses blockchain technology to generate a digital certificate (NFT), which the user receives, digitally protecting the copyright of the song.

[1778] Input: Song data published on a sharing site

[1779] Output: Generated digital certificate (NFT)

[1780] (Application example 2)

[1781] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1782] Current music production systems lack sufficient means for users to easily generate music and lyrics that respond to their emotions and publish them to content distribution services. Furthermore, there is no established method for digitally verifying the copyright of the generated music data. This creates problems for users to create personalized music and share their creations securely and easily.

[1783] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving language data entered by a user, means for analyzing the received language data and generating music data based on key keywords and recognized emotions, and means for adding performance data and vocal data to the generated music data. This allows users to easily generate music and lyrics that correspond to their emotions and publish them on content distribution services. Furthermore, by verifying that the generated music data does not infringe on the rights of others and attaching a digital certificate, copyright verification can be easily performed.

[1784] "User" means any individual or organization that uses the System to create and share music.

[1785] "Language data" refers to text data such as sentences and phrases that users input into the system.

[1786] "Analysis" refers to the process of analyzing received linguistic data and extracting key keywords and information.

[1787] "Emotion" refers to psychological states such as joy, sadness, and excitement that are recognized from the user's input data.

[1788] "Music data" refers to digital data containing the melody, rhythm, and harmony of the generated music.

[1789] "Performance data" refers to digital data of musical instrument performances that are added to music data.

[1790] "Vocal data" refers to digital vocal data that is added to music data.

[1791] "Ensuring that the rights of others are not infringed" refers to the process of ensuring that the generated music data is not similar to existing copyrighted works.

[1792] "Content distribution service" refers to a platform for publishing and sharing generated music data on the Internet.

[1793] "Digital Certificate" refers to an electronic certificate generated using blockchain technology to protect the copyright of generated music data.

[1794] The system for implementing the present invention allows users to easily create and publish music based on language data entered by the user. A series of steps will be explained in detail below.

[1795] 1. System Configuration

[1796] It consists of three main modules: the server, the terminal, and the user. The server is a central processing unit that actually generates music and processes data. The terminal is a device operated by the user, such as a smartphone. The user is the person who creates music.

[1797] 2. User Input

[1798] A user accesses the system through a terminal and launches the application. After entering their login information for authentication, they write a phrase or an image of an instrument in the input field. For example, they might enter, "I want a bright guitar melody."

[1799] 3. Emotional Recognition

[1800] The device sends the user's input data to the server. The server stores the received data and uses an emotion recognition engine called EmotionRecognition to recognize emotions from the input data. For example, emotions such as "joy" or "liveliness" can be detected from the user's input.

[1801] 4. Input Analysis

[1802] The server applies Natural Language Processing (NLP) technology to the received language data to extract key keywords. The keywords extracted as a result of the analysis are "cheerful," "guitar," and "melody."

[1803] 5. Interactive Question-Answering

[1804] Based on the extracted keywords and the emotion recognition results, the server generates the next dialogue step and sends it to the terminal in the form of a question, such as "What tempo and key do you want to play in?"

[1805] 6. Music Generation

[1806] If the user responds with "uptempo C major," the server inputs this information into a music generation AI model called MusicGeneration to generate basic music data, and adjusts the tone and structure of the music data based on the results of emotion recognition.

[1807] 7. Lyric Generation

[1808] If a user requests lyrics with themes of love and youth, the server calls a lyric generation AI model called LyricsGeneration and inputs the specified theme. The AI ​​model generates lyrics based on the theme and the recognized emotions. For example, it automatically generates lyrics with a bright tone that conveys joy and vitality.

[1809] 8. Addition of performance and singing data

[1810] The server adds performance data and vocal data to the generated basic music data, using data from existing professional musicians and singers. For example, guitar parts by a professional guitarist and vocal parts by a singer are integrated.

[1811] 9. Confirmation of Rights

[1812] The server uses a module called CopyrightChecker to check whether the generated music data is similar to existing copyrighted works, for example, to ensure that the generated music does not have a high similarity to other existing music.

[1813] 10. Publication

[1814] If the user selects "I want to make it public," the server automatically uploads the song using the API of a content distribution service. For example, the server calls the API of a sharing site and uploads the created song to that site.

[1815] 11. Adding a Digital Certificate

[1816] The server generates metadata about the song and uses blockchain technology to generate a digital certificate, which is then provided to the user. For example, the certificate registered on the blockchain digitally certifies the user's copyright.

[1817] Examples of concrete examples and prompts

[1818] Example 1: When a user inputs "I want a bright guitar melody," music with a pop tempo is automatically generated.

[1819] Example 2: If the user selects "Love and Youth" as the theme, cheerful lyrics on the theme of love and youth are automatically generated.

[1820] Example prompt sentence:

[1821] "I want a bright guitar melody."

[1822] "I want you to write lyrics with themes of love and youth"

[1823] "Uptempo C major"

[1824] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1825] Step 1:

[1826] A user accesses the system through a terminal and launches an application. The user enters login information for authentication, then writes a phrase or an image of an instrument in the input field. The input is a prompt such as "Please add a cheerful guitar melody." The terminal sends this input data to the server.

[1827] Step 2:

[1828] The server stores the received user input data. Based on the stored data, an emotion recognition engine called EmotionRecognition is used to recognize emotions from the input data. For example, emotions such as "joy" or "liveliness" are recognized from the input. The input data is text data, and the output is emotional data.

[1829] Step 3:

[1830] The server applies Natural Language Processing (NLP) technology to the received language data to extract key keywords. The keywords extracted as a result of the analysis are "cheerful," "guitar," and "melody." Text analysis is used to extract these keywords, with the input being language data and the output being keyword data.

[1831] Step 4:

[1832] The server generates the next dialogue step based on the extracted keywords and the emotion recognition results and sends it to the terminal in the form of a question. For example, it sends a question such as, "What tempo and key do you want to play at?" The input is keywords and emotion data, and the output is the dialogue step question.

[1833] Step 5:

[1834] The user answers questions from the server interactively through the terminal. For example, the user might answer "uptempo C major." The terminal then sends this answer data back to the server. The input is the user's answer, and the output is text data sent to the server.

[1835] Step 6:

[1836] The server inputs user response data and emotional data into a music generation AI model called MusicGeneration to generate basic music data. It also adjusts the tone and structure of the music data based on the results of emotion recognition. The input data are tempo, key, and emotional data, and the output is music data.

[1837] Step 7:

[1838] If the user again requests "Please create lyrics with the theme of love and youth," the server will call a lyrics generation AI model called LyricsGeneration and input the specified theme. The AI ​​model will generate lyrics based on the theme and the recognized emotion. The input is theme and emotion data, and the output is lyric data.

[1839] Step 8:

[1840] The server adds performance and vocal data to the generated basic music data. This uses data from existing professional musicians and singers. For example, guitar parts by a professional guitarist and vocal parts by a singer are integrated. The input is music data and additional performance and vocal data, and the output is the completed music data.

[1841] Step 9:

[1842] The server checks whether the generated music data is similar to existing copyrighted works through a module called CopyrightChecker. For example, it checks whether the generated music is too similar to other existing music. The input is the completed music data, and the output is the result of copyright check.

[1843] Step 10:

[1844] If the user selects "I want to publish," the server automatically uploads the song using the API of a content distribution service. For example, the server calls the API of a sharing site and uploads the created song to that site. The input is the music data and the publishing request, and the output is the upload result to the sharing site.

[1845] Step 11:

[1846] The server generates metadata about the music and generates a digital certificate using blockchain technology. The generated certificate information is provided to the user. For example, the certificate registered on the blockchain digitally certifies the user's copyright. The input is the metadata of the music data, and the output is a digital certificate.

[1847] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1848] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1849] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1850] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1851] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1852] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1853] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1854] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1855] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1856] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1857] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1858] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1859] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1860] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1861] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1862] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1863] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1864] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1865] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1866] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1867] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1868] The following is further disclosed regarding the above embodiment.

[1869] (Claim 1)

[1870] means for receiving language data input by a user;

[1871] means for analyzing the received language data and extracting key keywords;

[1872] means for generating music data based on the extracted keywords;

[1873] means for adding performance data and singing data to the generated music data;

[1874] means for verifying that the added music data does not infringe the rights of others;

[1875] means for publishing the confirmed music data on a sharing site;

[1876] The system further includes a means for adding a digital certificate to the published music data.

[1877] (Claim 2)

[1878] 2. The system according to claim 1, wherein the language data is input and confirmed interactively with a user.

[1879] (Claim 3)

[1880] The system of claim 1, wherein the digital certificate is generated using blockchain technology.

[1881] "Example 1"

[1882] (Claim 1)

[1883] means for receiving language data input by a user;

[1884] means for analyzing the received language data and extracting key keywords;

[1885] means for generating music data based on the extracted keywords;

[1886] means for adding performance data and singing data to the generated music data;

[1887] means for verifying that the added music data does not infringe the rights of others;

[1888] means for publishing the confirmed music data on a sharing site;

[1889] means for adding a digital certificate to the published music data;

[1890] a means for scanning and verifying existing databases for the generated music data;

[1891] A system that includes a means for encrypting metadata associated with the generated music and registering it as an NFT using blockchain technology.

[1892] (Claim 2)

[1893] 2. The system according to claim 1, wherein the language data is input and confirmed interactively with a user.

[1894] (Claim 3)

[1895] The system of claim 1, wherein the digital certificate is generated using blockchain technology.

[1896] "Application Example 1"

[1897] (Claim 1)

[1898] means for receiving language data input by a user;

[1899] means for analyzing the received language data and extracting key keywords;

[1900] means for generating music data based on the extracted keywords;

[1901] means for adding performance data and singing data to the generated music data;

[1902] means for verifying that the added music data does not infringe the rights of others;

[1903] means for publishing the confirmed music data on a distribution platform;

[1904] a means for adding a digital certificate to the published music data using blockchain technology;

[1905] a means for analyzing a user's input using natural language processing technology and generating a prompt sentence;

[1906] A means for inputting the prompt sentence into a generative AI model to generate music data and lyrics;

[1907] A system including:

[1908] (Claim 2)

[1909] 2. The system according to claim 1, wherein the language data is input and confirmed interactively with a user.

[1910] (Claim 3)

[1911] The system of claim 1, wherein the digital certificate is generated using blockchain technology.

[1912] "Example 2: Combining Emotion Engines"

[1913] (Claim 1)

[1914] means for receiving language data input by a user;

[1915] means for analyzing the received language data and extracting key keywords;

[1916] means for generating music data based on the extracted keywords and the recognized emotions;

[1917] means for adding performance data and singing data to the generated music data;

[1918] means for verifying that the added music data does not infringe the rights of others;

[1919] means for publishing the confirmed music data on a sharing site;

[1920] means for adding a digital certificate to the published music data;

[1921] means for recognizing a user's emotion from the received language data;

[1922] The system includes means for adjusting the tone or content of the musical data based on the recognized emotion.

[1923] (Claim 2)

[1924] 2. The system according to claim 1, wherein the language data is input and confirmed interactively with a user.

[1925] (Claim 3)

[1926] The system of claim 1, wherein the digital certificate is generated using blockchain technology.

[1927] "Application example 2 when combining emotion engines"

[1928] (Claim 1)

[1929] means for receiving language data input by a user;

[1930] means for analyzing the received language data and extracting key keywords;

[1931] means for generating music data based on the extracted keywords and the recognized emotions;

[1932] means for adding performance data and singing data to the generated music data;

[1933] means for verifying that the added music data does not infringe the rights of others;

[1934] means for publishing the confirmed music data to a content distribution service;

[1935] The system further includes a means for adding a digital certificate to the published music data.

[1936] (Claim 2)

[1937] 2. The system according to claim 1, wherein the language data is input and confirmed interactively with a user.

[1938] (Claim 3)

[1939] The system of claim 1, wherein the digital certificate is generated using blockchain technology. [Explanation of symbols]

[1940] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for receiving language data input by a user; means for analyzing the received language data and extracting key keywords; means for generating music data based on the extracted keywords; means for adding performance data and singing data to the generated music data; a means for verifying that the added music data does not infringe the rights of others; means for publishing the confirmed music data on a sharing site; The system further includes a means for adding a digital certificate to the published music data.

2. 2. The system according to claim 1, wherein the language data is input and confirmed interactively with a user.

3. The system of claim 1 , wherein the digital certificate is generated using blockchain technology.

Images