system

Abstract

Provide a system. The solution includes a means for producing and releasing music on a platform for managing the rights to use singers' voices; a means for appropriately distributing revenue based on music plays; A means of managing and checking singer voice policies, monitoring measures to prevent unauthorized use; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In recent years, advances in generative AI technology have made it easier to replicate singers' voices. This has led to an increase in cases of infringement of singers' rights, particularly in music streaming services, where revenue from songs using AI voices is unfairly distributed. Therefore, there is a need for a system that protects singers' voice rights and ensures fair revenue distribution. [Means for solving the problem]

[0005] This system includes a means for producing and releasing music on a platform for managing the rights to use singers' voices, a means for appropriately distributing revenue based on music playback, a means for managing and checking singers' voice policies, and a monitoring means for preventing unauthorized use. This provides a mechanism for appropriately protecting singers' voices and fairly distributing revenue. It also includes a means for storing and managing the generated music data and a means for notifying each participant of the revenue distribution results, thereby improving the management efficiency and ensuring transparency of the entire system.

[0006] A "singer" refers to a person who sings when performing music, and whose voice is recognized as an intangible asset.

[0007] "Voice rights" refers to the right to record, reproduce and use a particular singer's voice, and these rights are intangible assets that must be protected.

[0008] "Platform" refers to the underlying system that enables music creators, singers, and listeners to interact with each other online and create and play music.

[0009] "Revenue sharing" refers to the process of fairly distributing revenue earned from the playback and sale of music to the parties involved, including singers, music producers, and copyright holders.

[0010] "Means for managing and checking policies" refers to the internal system functions for ensuring compliance with and checking the conditions for using a singer's voice (e.g., prohibiting use in a specific genre).

[0011] "Monitoring measures to prevent unauthorized use" refers to mechanisms within the system, including monitoring functions to prevent the singer's voice from being used without permission.

[0012] "Means for storing and managing music data" refers to the function for safely and systematically storing generated music data and managing it in an accessible state.

[0013] "Means for notifying the results of revenue distribution" refers to a function within the system for notifying the parties concerned of the calculated results of revenue distribution. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0015] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0018] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0019] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0020] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0022] [First embodiment]

[0023] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0024] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0027] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0030] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0034] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0035] This invention is a next-generation music streaming system that properly manages the rights to use singers' voices and achieves fair revenue distribution. The system includes a platform for managing voice usage rights, a function for distributing revenue based on music playback, a function for managing and checking singers' voice policies, and a monitoring function for preventing unauthorized use.

[0036] System Overview

[0037] This system consists of three main components: a server and user terminals (singers, creators, and listeners). Each component has a specific function and operates by communicating with each other.

[0038] 1. Server Functions

[0039] The server is responsible for the following main functions: managing the rights to use singers' voices, storing and managing song data, calculating revenue distribution and notifying the results. Specifically, it performs the following processes:

[0040] Accepting user registration: Provide an interface for singers to register information regarding the use of their voice.

[0041] Storing music data: Music data and related metadata created by creators are stored in a database.

[0042] Usage policy checks: When a song is uploaded, we check that it complies with the usage policy set by the artist.

[0043] Revenue sharing calculation: Calculate revenue based on playback data and distribute it to each party (singer, creator, original author).

[0044] Unauthorized use monitoring: Monitor singers' voices for unauthorized use.

[0045] 2. User device (singer) functions

[0046] Singers register their voice usage rights on the platform, set policies, and receive revenue sharing results through the system. Specifically, they perform the following operations:

[0047] Register voice usage rights: Upload sample voices and set usage policies.

[0048] Check the revenue sharing results: Receive notifications from the server and check the revenue sharing results.

[0049] 3. User Device (Creator) Functions

[0050] Creators create and upload songs using singers' voices. When uploading, they are required to ensure that the usage policy is adhered to. Specifically, they perform the following operations:

[0051] Voice selection: Select the singer's voice you want to use on the platform.

[0052] Uploading music: Upload the created music data and related metadata to the server.

[0053] 4. User terminal (listener) functions

[0054] Listeners search for and play songs on the platform by:

[0055] Song search: Enter keywords to search for songs.

[0056] Play Song: Select a song from the search results and stream it.

[0057] Specific examples

[0058] One day, singer A registers the rights to use his or her voice on a platform and sets a policy that it can only be used in the pop genre. In response, creator B creates a pop song using singer A's voice and uploads it to the platform. The server checks whether the uploaded song complies with singer A's policy and, if there are no problems, stores it in the database. Later, listener C searches for the pop song on the platform and plays it. Based on the playback data, the server calculates revenue and distributes it to singer A, creator B, and the author of the original song. Singer A and creator B are notified of the revenue distribution results.

[0059] In this way, the present invention provides functions for protecting the rights of singers' voices and achieving fair distribution of revenue, thereby building a music streaming system that can be used safely by all parties involved.

[0060] The processing flow will be explained below.

[0061] 1. User Registration and Contract Process

[0062] Step 1:

[0063] The singer (user) accesses the registration form.

[0064] The singer accesses the registration form provided on the system's web page.

[0065] Step 2:

[0066] The singer (user) enters their name, profile, voice sample, and desired usage policy.

[0067] Singers fill out a form with their name, profile information, a sample audio file of their voice, and voice usage policies (e.g., allowed genres, etc.).

[0068] Step 3:

[0069] The device sends the input to the server.

[0070] When the submit button on the registration form is clicked, the terminal transmits the information entered by the singer to the server.

[0071] Step 4:

[0072] The server stores the received information in a database.

[0073] The server stores the received registration information in a database and generates a unique ID for the singer.

[0074] Step 5:

[0075] The server will send a confirmation email to the singer.

[0076] The server will send a confirmation email to the singer's email address to notify them of the completion of registration.

[0077] 2. Creators' music production and release

[0078] Step 1:

[0079] The creator (user) logs in and accesses the dashboard.

[0080] Creators can log in by entering their account information and use the dashboard.

[0081] Step 2:

[0082] The creator (user) selects the singer's voice they want to use.

[0083] Creators select the voice they want to use from a list of singers provided on their dashboard.

[0084] Step 3:

[0085] The device sends the selection to the server.

[0086] Once the creator selects a voice, the content is sent to the server.

[0087] Step 4:

[0088] The server checks the policy of the selected singer's voice.

[0089] The server verifies that the selected singer's voice complies with the specified usage policy.

[0090] Step 5:

[0091] The device downloads a voice sample.

[0092] Once the server has completed the usage policy check, a voice sample is downloaded to the creator's device.

[0093] Step 6:

[0094] Creators (users) create music using AI tools.

[0095] Creators use AI tools (e.g., DAWs and AI voice plug-ins) to create music.

[0096] Step 7:

[0097] Creators (users) upload their completed songs to the platform.

[0098] Enter your completed music file and its metadata (e.g. title, genre, usage policy, etc.) into the upload form.

[0099] Step 8:

[0100] The device sends the uploaded content to the server.

[0101] Clicking the upload button sends the music file and metadata to the server.

[0102] Step 9:

[0103] The server checks the metadata and stores it in a database.

[0104] The server checks the metadata of the song to ensure it does not violate usage policies, and if there are no problems, stores the song data in a database.

[0105] 3. Listeners play music

[0106] Step 1:

[0107] The listener (user) logs in and searches for songs.

[0108] Listeners log in by entering their account information and then enter keywords into the search bar.

[0109] Step 2:

[0110] The device sends the search keywords to the server.

[0111] When you click the search button, the search keywords are sent to the server.

[0112] Step 3:

[0113] The server searches the database for the corresponding song and returns the results to the terminal.

[0114] The server searches the database for songs that match the keywords and sends the results to the listener's device.

[0115] Step 4:

[0116] The listener (user) selects the song to play.

[0117] Listeners select the song they want to play from the search results and click the play button.

[0118] Step 5:

[0119] The server begins streaming the selected song.

[0120] The server streams the selected songs and sends them to the listener's device.

[0121] 4. Revenue Sharing

[0122] Step 1:

[0123] The server periodically obtains the playback data.

[0124] The server periodically obtains the playback data from the playback history table.

[0125] Step 2:

[0126] The server performs the revenue calculation.

[0127] The server calculates the total revenue for each song based on the number of plays.

[0128] Step 3:

[0129] The server generates data for distributing revenue to each participant (author, singer, creator).

[0130] The server generates data for distributing the profits to each participant and calculates them based on the distribution ratio.

[0131] Step 4:

[0132] The server stores the revenue sharing results in a database and sends notifications to each participant.

[0133] The server stores the distribution results in a database and sends notifications to participants via email or other means.

[0134] The above is a specific processing flow for implementing the present invention, which realizes the protection of the singer's voice rights and the fair distribution of profits.

[0135] Example 1

[0136] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0137] In today's music streaming services, the rights of artists and other creators are often not properly protected. Furthermore, revenue distribution is often opaque and not fair. Furthermore, unauthorized use of artists' voices is common and often goes undetected. To solve these problems, a system is needed that properly manages the rights to use artists' voices, distributes revenue fairly, and prevents unauthorized use.

[0138] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0139] In this invention, the server includes a means for producing and releasing music on a platform for managing the rights to use singers' voices, a means for appropriately distributing revenue based on music playback, a means for managing and checking policies for singers' voices, a monitoring means for preventing unauthorized use, a server means for providing an interface for registering information regarding the use of users' voices, a database means for storing music sheet data and related metadata, and a server means for automatically performing policy checks, thereby enabling the protection of the rights of singers and creators, the fair distribution of revenue, and the prevention of unauthorized use.

[0140] A "singer" is an individual or group of individuals who provides their voice in a musical composition.

[0141] "Usage rights" are legal authorizations that permit the use of a particular vocal or musical work.

[0142] "Platform" refers to an online service or software that enables users to publish their content and interact with other users.

[0143] "Music" is an art form created by combining sounds and voices, and is a means of expression through the ears.

[0144] "Playback" is the act of listening to music or audio using a device.

[0145] "Revenue" is the monetary benefit derived from the provision of a service or consumption of a work.

[0146] A "policy" is a rule or guideline that prescribes a particular purpose or behavior.

[0147] "Monitoring" is the act of continually checking a particular activity or situation and responding when a problem arises.

[0148] An "interface" is the method or means by which a user interacts with a system or software.

[0149] A "server" is a computer system that provides data over a network.

[0150] A "database" is a system for efficiently storing, managing, and searching large amounts of data.

[0151] "Metadata" refers to data that contains information about specific data, such as the title of a song or the name of an artist.

[0152] A "policy check" is a process that verifies whether certain conditions are met based on a registered policy.

[0153] "Revenue sharing" is the act of distributing earned revenue fairly among multiple rights holders and related parties.

[0154] This is a next-generation music streaming system that properly manages the rights to use singers' voices and ensures fair revenue distribution. The system consists of three main components: a server and user terminals (singers, creators, and listeners).

[0155] Server Roles and Functions

[0156] The server is responsible for the central functions of the system. It performs the following main processes:

[0157] Accepting user registrations:

[0158] The server provides an interface for singers to register information about the use of their voice. This interface is a web-based application that singers can access from their terminals.

[0159] Save song data:

[0160] The music data and related metadata created by creators are stored in a database, which uses a common database management system such as MySQL (registered trademark) or PostgreSQL.

[0161] Usage policy check:

[0162] The server automatically checks whether the uploaded song complies with the artist's usage policy, using a specific algorithm.

[0163] Revenue Share Calculation:

[0164] The server calculates revenue based on song playback data and distributes it to each party (singer, creator, original author), taking into account the number of plays, playback time, advertising revenue, etc.

[0165] Abuse Monitoring:

[0166] The server monitors the singer's voice for fraudulent use, which is achieved by using an AI model to check for voice pattern matches.

[0167] Roles and functions of user devices

[0168] The role of the singer

[0169] Singers will register the rights to use their voice and receive notification of the revenue sharing results. Specifically, they will:

[0170] Voice usage rights registration:

[0171] Singers upload their own sample voices from their devices and set usage policies.

[0172] Check your revenue sharing results:

[0173] The singer receives a notification from the server and checks the revenue sharing results.

[0174] The role of creators

[0175] Creators create and upload songs using singers' voices. Specifically, they do the following:

[0176] Voice Selection:

[0177] Creators choose the singer's voice they want to use on the platform.

[0178] Upload your song:

[0179] Creators upload the music data and related metadata they have created to the server, where they are checked to ensure that the usage policy is being followed.

[0180] The role of the listener

[0181] Listeners search for and play songs on the platform by:

[0182] Song search:

[0183] Listeners can input keywords to search for songs.

[0184] Play music:

[0185] Listeners can select songs from the search results and stream them.

[0186] Specific examples

[0187] One day, singer A registers the rights to use his or her voice on a platform and sets a policy that it can only be used in the pop genre. In response, creator B creates a pop song using singer A's voice and uploads it to the platform. The server checks whether the uploaded song complies with singer A's policy and, if there are no problems, stores it in the database. Later, listener C searches for the pop song on the platform and plays it. Based on the playback data, the server calculates revenue and distributes it to singer A, creator B, and the author of the original song. Singer A and creator B are notified of the revenue distribution results.

[0188] Example prompt:

[0189] "Please explain the user experience when Singer A sets a policy that allows music to be used only in the pop genre."

[0190] In this way, the invention provides a concrete means to protect singers' vocal rights and ensure fair distribution of revenue, creating a music streaming system that can be used safely by all parties involved.

[0191] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0192] Step 1: User registration and rights registration

[0193] Input: The singer uses the device to enter their name, contact information, sample voice (audio file), and usage policy.

[0194] What happens: A singer accesses the registration form, fills in the required information, uploads a sample voice, sets usage policies, and clicks the submit button.

[0195] Data processing / calculation: The server receives the input data, parses the usage policy and saves it in the database. It also saves the sample voice audio file.

[0196] Output: The server confirms that user registration and rights registration have been completed, and sends a registration completion notification to the singer's terminal.

[0197] Step 2: Choose your creator's voice

[0198] Input: Creators use their device to enter search criteria for the voice they want to use (e.g. genre, singer name).

[0199] How it works: Creators log in to the platform and enter criteria such as genre and artist name into the search form.

[0200] Data processing / calculation: The server queries the database based on the search criteria and returns filtered sample voice information of the corresponding singer.

[0201] Output: As a search result, a list of available singers and sample voice information are displayed on the creator's device.

[0202] Step 3: Upload your music

[0203] Input: Enter the audio file and metadata (e.g. title, genre, description) of the song created by the creator.

[0204] What happens: Creator selects a music file, enters relevant metadata into the form, and clicks the upload button.

[0205] Data processing / calculation: The server receives the uploaded audio file and metadata, checks the integrity of the file, and then verifies whether the song is permitted based on the usage policy.

[0206] Output: If the song data complies with the policy, the server saves the song data in the database and sends an upload completion notification to the creator's device. If it does not comply, it returns an error message.

[0207] Step 4: Search and play songs

[0208] Input: A listener uses a device to enter keywords to search for a song.

[0209] Specific actions: Listeners access the platform, enter keywords into the search form, and click the search button.

[0210] Data processing / calculation: The server queries the database based on the search keywords and generates a list of relevant songs.

[0211] Output: As a search result, a list of matching songs is displayed on the listener's device. When the listener selects a song and clicks the play button, the server sends the streaming data and plays the song.

[0212] Step 5: Calculate revenue share

[0213] Input: The server compiles song playback data (e.g., number of plays, playback time, advertising revenue).

[0214] What it does: The server records playback data every time a song is played on the platform. It aggregates the playback data over specific time intervals.

[0215] Data processing / calculation: The server uses a revenue sharing algorithm to calculate revenue, taking into account the number of plays, play time, ad revenue, etc.

[0216] Output: The server stores the revenue sharing results in a database and sends notifications to the singer, creator, and original author.

[0217] Step 6: Monitor for fraud

[0218] Input: Periodically scan all song data in the database.

[0219] Specific operation: The server executes a scheduled task at regular intervals to analyze all songs.

[0220] Data processing / calculation: The server uses AI models to detect unregistered usage patterns and unauthorized copying.

[0221] Output: If any unauthorized use is detected, the server will send a warning notice to the singer and the system administrator.

[0222] (Application example 1)

[0223] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0224] With the recent spread of music streaming services, fairness in the rights to use singers' voices and revenue distribution has become increasingly important. However, existing systems have insufficient management of rights to use singers' voices and revenue distribution, resulting in fraudulent use and unfair revenue distribution. Furthermore, there is a need for systems that make it easy for users to search and play music on their smart devices while also providing appropriate rights management. However, there is a lack of systems that combine an interactive, easy-to-use user interface with advanced voice analysis technology.

[0225] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0226] In this invention, the server includes means for producing and releasing music on a platform for managing the rights to use a singer's voice, means for appropriately distributing revenue based on music playback, means for managing and checking policies for the singer's voice, means for monitoring to prevent unauthorized use, means for using smart contracts to manage users' voice rights, means for providing streaming playback of music, means for notifying users of revenue distribution results, and means for providing a user interface that operates on a smartphone or a head-mounted display, thereby ensuring the appropriate use of a singer's voice and fairness in revenue distribution, and further improving the user experience on smart devices.

[0227] The "platform for managing singers' voice usage rights" is an online system that allows singers to register rights to use their voice and manage the scope and conditions of that use.

[0228] "Means for producing and releasing music" refers to a system that allows music creators to create music and publish that music data on the platform.

[0229] "Means for appropriate distribution of revenue" is a function for calculating revenue based on song playback data and distributing it fairly to the parties involved.

[0230] "Means for managing and checking singers' voice policies" refers to a system for checking whether the conditions and scope of voice usage set by singers are being properly observed.

[0231] "Monitoring measures to prevent unauthorized use" refers to a monitoring function to prevent the singer's voice from being used without permission.

[0232] "Means using smart contracts" refers to a mechanism that uses blockchain technology to automate the management of audio rights and revenue distribution, ensuring transparency.

[0233] "Means for providing streaming music playback" refers to a function that allows users to listen to music in real time over the Internet.

[0234] The "means for notifying the profit distribution results" is a function for promptly notifying the parties concerned of the profit calculation results.

[0235] A "user interface that runs on a smartphone or head-mounted display" is an operation screen or interface that allows users to use the system through a device such as a smartphone or VR device.

[0236] The system embodying this invention comprises the following main components: a server, a singer terminal, a creator terminal, and a listener terminal.

[0237] Server Features

[0238] The server provides the following functionality:

[0239] 1. Accepting user registration: The server provides an interface that allows singers, creators, and listeners to create accounts. In this case, account information is managed using AWS (registered trademark) Cognito.

[0240] 2. Managing singers' voice usage rights: Provides a function for singers to register their voice usage rights and set usage policies. Transparency and security are ensured by using smart contracts and blockchain technology (e.g., Ethereum).

[0241] 3. Storage and management of music data: Music data created by creators is stored in a database on the server (e.g., MySQL, PostgreSQL).

[0242] 4. Usage policy check: When a song is uploaded, the server checks whether it complies with the usage policy set by the artist. It uses voice analysis technology such as Google® Cloud Speech-to-Text to scrutinize the audio data.

[0243] 5. Unauthorized use monitoring: Monitor singers' voices for unauthorized use and issue alerts if necessary.

[0244] 6. Providing a music streaming service: Using the Spotify SDK, we provide a fast, high-quality playback experience for listeners to stream music.

[0245] 7. Revenue share calculation: Based on the song playback data, revenue is calculated and distributed fairly to the parties involved. This process is performed using AWS Lambda and DynamoDB.

[0246] 8. Notification of Revenue Distribution Results: The singer, creator, and original author will be notified of the revenue distribution results via email.

[0247] User device functions

[0248] 1. Singer terminal: Singers can register their voice usage rights, manage the policies they have set, and check the revenue distribution results.

[0249] 2. Creator Terminal: Creators can create and upload music and ensure that usage policies are properly adhered to.

[0250] 3. Listener device: Listeners have the ability to search for songs and play them in streaming.

[0251] Specific examples

[0252] For example:

[0253] One day, a singer registers the rights to use her voice on a platform and sets a policy that it can only be used in the pop genre. In response, a creator creates a pop song using the singer's voice and uploads it to the platform. The server checks whether the uploaded song complies with the singer's policy and, if there are no problems, stores it in the database. A listener then searches for and plays the pop song on the platform. Based on the playback data, the server calculates revenue and distributes it to the singer, creator, and the copyright holder of the original song. The singer and creator are notified of the revenue distribution results.

[0254] The following prompt example is also provided:

[0255] Singer: "I only want my voice to be used in songs in the pop genre."

[0256] Creator: "I've created a new pop song and I want to use the singer's voice."

[0257] Listener: "I want to hear the latest pop songs."

[0258] The system ensures the proper use of singers' voices and fair revenue distribution, while also improving the user experience on smart devices.

[0259] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0260] Step 1:

[0261] A singer uses a terminal to register the rights to use their voice on the platform. At this time, the singer uploads a sample of their voice and sets a policy that it can only be used in the pop genre. The server receives the sample voice and policy data as input and registers it in a smart contract using blockchain technology to ensure transparency and security.

[0262] Step 2:

[0263] Creators use their devices to create songs using singers' voices and upload them to the platform. Creators then send the song data to the server based on the policy. The server stores the song data and metadata in a MySQL or PostgreSQL database and uses Google Cloud Speech-to-Text technology to check whether the voice data complies with the policy.

[0264] Step 3:

[0265] The server checks whether the uploaded song complies with the usage policy set by the artist. If it is confirmed, the song data is officially stored in the platform's database. Conversely, if there is a policy violation, the creator is notified and the upload is stopped.

[0266] Step 4:

[0267] Listeners use their devices to search for and stream songs on the platform. Listeners enter keywords, and the server searches the database for songs and outputs the results. The song data is then streamed to the listener's device using the Spotify SDK.

[0268] Step 5:

[0269] The server collects listeners' song play data, recording the number of plays and play time. This data is later used to calculate revenue distribution. The server processes the data in real time using AWS Lambda and DynamoDB to calculate how revenue should be distributed to each party.

[0270] Step 6:

[0271] The server distributes revenue based on the calculation results. The revenue is distributed to the singer, creator, and original song author based on a predetermined revenue distribution ratio. The distribution results are sent to the relevant parties via email or other notification methods.

[0272] Step 7:

[0273] After the server completes all procedures, it displays the revenue distribution results in real time on the user's dashboard, allowing all parties involved to instantly see how the revenue has been distributed.

[0274] Examples:

[0275] Singer: "I only want my voice to be used in songs in the pop genre."

[0276] Creator: "I've created a new pop song and I want to use the singer's voice."

[0277] Listener: "I want to hear the latest pop songs."

[0278] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0279] This is a next-generation music streaming system that recognizes users' emotions to improve the music experience and distributes revenue fairly, while properly managing the rights to use singers' voices. In addition to the functions of conventional music streaming systems, this system uses an emotion engine to recognize users' emotions and adjusts song recommendations and revenue distribution based on these.

[0280] System Overview

[0281] This system includes three main components: a server, user terminals (singers, creators, and listeners), and an emotion engine. These components have specific functions and operate while communicating with each other.

[0282] 1. Server Functions

[0283] The server manages the rights to use singers' voices, stores and manages song data, calculates revenue distribution and notifies the results, and processes emotion data using an emotion engine. Specifically, it performs the following processes:

[0284] Accepting user registration: Provide an interface for singers to register information regarding the use of their voice.

[0285] Storing music data: Music data and related metadata created by creators are stored in a database.

[0286] Usage policy checks: When a song is uploaded, we check that it complies with the usage policy set by the artist.

[0287] Revenue sharing calculation: Calculate revenue based on playback data and distribute it to each party (singer, creator, original author).

[0288] Unauthorized use monitoring: Monitor singers' voices for unauthorized use.

[0289] Emotional Data Analysis: Emotional data generated by the emotion engine is collected and analyzed to improve user experience and adjust revenue sharing.

[0290] 2. User device (singer) functions

[0291] Singers register their voice usage rights on the platform, set policies, and receive revenue sharing results through the system. Specifically, they perform the following operations:

[0292] Register voice usage rights: Upload sample voices and set usage policies.

[0293] Check the revenue sharing results: Receive notifications from the server and check the revenue sharing results.

[0294] 3. User Device (Creator) Functions

[0295] Creators create and upload songs using singers' voices. When uploading, they are required to ensure that the usage policy is adhered to. Specifically, they perform the following operations:

[0296] Voice selection: Select the singer's voice you want to use on the platform.

[0297] Uploading music: Upload the created music data and related metadata to the server.

[0298] 4. User terminal (listener) functions

[0299] Listeners search for and play songs on the platform, and the system recognizes their emotions and recommends appropriate songs. Specifically, they perform the following actions:

[0300] Song search: Enter keywords to search for songs.

[0301] Play Song: Select a song from the search results and stream it.

[0302] Emotion recognition: The emotion engine analyzes the user's emotions using the device's camera and voice input. Based on the analysis results, it recommends songs that suit the user.

[0303] Specific examples

[0304] One day, Singer A registers the rights to use his or her voice on the platform and sets a policy that it can only be used in the pop genre. In response, Creator B creates a pop song using Singer A's voice and uploads it to the platform. The server checks whether the uploaded song complies with Singer A's policy and, if there are no problems, stores it in the database. Listener C then searches for and plays a pop song on the platform. The emotion engine analyzes Listener C's facial expression and recognizes that he or she is relaxed, so songs in a similar pop genre are recommended.

[0305] Based on the playback data, the server calculates revenue and distributes it to singer A, creator B, and the copyright holder of the original song. The revenue distribution results are notified to singer A and creator B. In addition, revenue adjustments are also performed based on emotional data, realizing optimal revenue distribution according to the user's emotions.

[0306] In this way, the present invention not only protects the rights of singers' voices and ensures fair distribution of revenue, but also builds a system that provides a more personalized music experience through a recommendation function based on user emotions.

[0307] The processing flow will be explained below.

[0308] 1. User Registration and Contract Process

[0309] Step 1:

[0310] The singer (user) accesses the registration form.

[0311] The singer accesses the registration form provided on the system's web page.

[0312] Step 2:

[0313] The singer (user) enters their name, profile, voice sample, and desired usage policy.

[0314] Singers fill out a form with their name, profile information, a sample audio file of their voice, and usage policies (e.g., allowed genres, etc.).

[0315] Step 3:

[0316] The device sends the input to the server.

[0317] When the registration form submit button is clicked, the terminal sends the entered information to the server.

[0318] Step 4:

[0319] The server stores the received information in a database.

[0320] The server stores the received information in a database and generates a unique ID for the singer.

[0321] Step 5:

[0322] The server will send a confirmation email to the singer.

[0323] The server will send a confirmation email to the singer's email address to notify them of the completion of registration.

[0324] 2. Creators' music production and release

[0325] Step 1:

[0326] The creator (user) logs in and accesses the dashboard.

[0327] Creators log in on the system's web page and access their dashboard.

[0328] Step 2:

[0329] The creator (user) selects the singer's voice they want to use.

[0330] Creators select the voice they want to use from a list of singers provided on their dashboard.

[0331] Step 3:

[0332] The device sends the selection to the server.

[0333] Once the creator selects a voice, that information is sent to a server.

[0334] Step 4:

[0335] The server checks the policy of the selected singer's voice.

[0336] The server checks whether the selected singer's voice complies with the established usage policy.

[0337] Step 5:

[0338] The device downloads a voice sample.

[0339] Once the policy check is complete, the server will have the device download a voice sample file.

[0340] Step 6:

[0341] Creators (users) create music using AI tools.

[0342] Creators use AI tools to create and complete songs.

[0343] Step 7:

[0344] Creators (users) upload their completed songs to the platform.

[0345] Enter the completed music file and its metadata through the upload form.

[0346] Step 8:

[0347] The device sends the uploaded content to the server.

[0348] Clicking the upload button sends the music file and metadata to the server.

[0349] Step 9:

[0350] The server checks the metadata and stores it in a database.

[0351] The server checks the uploaded metadata, verifies that it does not violate any policies, and then stores it in the database.

[0352] 3. Listeners play music

[0353] Step 1:

[0354] The listener (user) logs in and searches for songs.

[0355] Listeners log into the system and enter keywords into the search bar.

[0356] Step 2:

[0357] The device sends the search keywords to the server.

[0358] When you click the search button, the entered keywords are sent to the server.

[0359] Step 3:

[0360] The server searches the database for the corresponding song and returns the results to the terminal.

[0361] The server searches the database for songs that match the keywords and sends the results to the listener's device.

[0362] Step 4:

[0363] The listener (user) selects the song to play.

[0364] Listeners select the song they want to play from the search results and click the play button.

[0365] Step 5:

[0366] The server begins streaming the selected song.

[0367] The server streams the selected songs and sends them to the listener's device.

[0368] Step 6:

[0369] The device recognizes the user's emotions using an emotion engine.

[0370] During playback, the device uses an emotion engine to analyze emotions from the user's facial expressions and voice.

[0371] Step 7:

[0372] The server receives the emotional data and recommends appropriate songs.

[0373] It receives data from the emotion engine and recommends songs that suit the user.

[0374] 4. Revenue Sharing

[0375] Step 1:

[0376] The server periodically obtains the playback data.

[0377] The server periodically obtains the playback data from the playback history table.

[0378] Step 2:

[0379] The server performs the revenue calculation.

[0380] The server calculates the total revenue for each song based on the number of plays and sentiment data.

[0381] Step 3:

[0382] The server generates data for distributing revenue to each participant (author, singer, creator).

[0383] The server generates data for distributing the profits to each participant based on the distribution ratio.

[0384] Step 4:

[0385] The server stores the revenue sharing results in a database and sends notifications to each participant.

[0386] The server stores the distribution results in a database and notifies each party of the revenue results via email or other means.

[0387] In this way, the present invention is a system that not only protects the rights of singers' voices and ensures fair distribution of revenue, but also provides a more personalized music experience through a recommendation function based on user emotions.

[0388] Example 2

[0389] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0390] Conventional music streaming systems have problems such as difficulty in properly managing singers' audio rights and fair revenue distribution. Furthermore, they lack a function for recommending songs based on user emotions. Therefore, a system to improve the user experience is needed.

[0391] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for producing and releasing music on a platform for managing singers' audio rights, a means for appropriately distributing revenue based on music playback, a means for managing and checking audio usage policies, a monitoring means for preventing unauthorized use, a means for recognizing user emotions and recommending songs based on the emotions, and a means for adjusting revenue distribution based on emotion data. This enables appropriate management of singers' audio rights, fair revenue distribution, and song recommendations based on user emotions.

[0392] "Singer's audio rights" refers to the right of a singer to commercially exploit his or her voice.

[0393] "Platform" refers to a system that enables users such as singers, creators, and listeners to create, play, and manage music via the Internet.

[0394] "Means for appropriately distributing revenue based on music plays" refers to algorithms or functions that distribute revenue to each party based on playback data.

[0395] "Audio usage policy" refers to the rules and standards that a singer sets for the conditions under which their voice is used and that are managed and checked based on those conditions.

[0396] "Monitoring measures to prevent unauthorized use" refers to monitoring systems and algorithms that check whether a singer's voice is being used fraudulently.

[0397] "Means for recognizing user emotions and recommending songs based on that" refers to a system that analyzes user emotions using technologies such as emotion engines and recommends songs based on the results.

[0398] "Means for adjusting revenue sharing based on emotional data" refers to an algorithm or function that adjusts the revenue sharing rate by taking into account the emotional data of users.

[0399] "Music data" refers to music files and associated metadata, such as title, artist information, genre, and other information.

[0400] "Database" refers to a system for managing stored audio rights information, music data, usage policies, playback data, user emotional data, and the like.

[0401] "Revenue Share Results" means the revenue share results for each party calculated based on playback data and other data.

[0402] MODE FOR CARRYING OUT THE INVENTION

[0403] This invention is a next-generation music streaming system that appropriately manages singers' audio rights, recognizes users' emotions to improve the music experience, and distributes revenue fairly. The main components of this system include a server and user terminals (singers, creators, and listeners), each with specific functions and operating while communicating with each other.

[0404] Server Features

[0405] The server is configured to perform the following main tasks:

[0406] Accepting user registration: Provides an interface for singers to register the rights to use their voice. Singers can upload their voice samples and enter genre settings and usage policies. The server receives this data and stores it in a database.

[0407] Music data storage: Creators upload the music data and related metadata they have created, which is then stored and managed on the server.

[0408] Usage policy check: Checks whether the uploaded song complies with the usage policy set by the artist. If it does not comply with the policy, the upload will be rejected and an error message will be sent to the creator.

[0409] Revenue sharing calculation: Calculate revenue based on playback data. Notify each party (singer, creator, original author) of the revenue sharing results.

[0410] Unauthorized use monitoring: We monitor whether singers' voices are being used illegally. If any unauthorized use is detected, we will send a warning message to the singer and take appropriate measures.

[0411] Emotion data analysis: The server analyzes the emotion data collected by the emotion engine, and uses this data to improve user experience and adjust revenue sharing.

[0412] As specific examples of hardware and software, the server uses an Amazon Web Services (AWS) EC2 instance, the database uses AWS RDS (Relational Database Service), and the emotion engine uses the Emotion API from Azure® Cognitive Services.

[0413] User device (singer) functions

[0414] On the singer's device, do the following:

[0415] Registering vocal usage rights: Singers access the system, upload their vocal samples, set usage policies, and specify specific genres and uses.

[0416] Check the revenue sharing results: Receive notifications from the server and check the revenue sharing results.

[0417] User device (creator) functions

[0418] On the creator's device, do the following:

[0419] Voice Selection: Creators access the system and select the singer's voice they wish to use. They then review the usage policy for the selected voice.

[0420] Uploading music: Upload the created music data and related metadata to the server.

[0421] User terminal (listener) functions

[0422] On the listener's device, do the following:

[0423] Song search: Listeners access the system and enter keywords to search for songs.

[0424] Song Playback: The listener selects a song and streams it. During playback, the emotion engine analyzes the listener's emotions.

[0425] Emotion Recognition: The system collects the listener's emotions from the device's camera and voice input. Based on the analysis results of the emotion engine, the system recommends songs that suit the user.

[0426] Specific examples

[0427] One day, Singer A registers the rights to use his or her voice on the platform, allowing it to be used in the pop genre. Creator B then creates a pop song using that voice and uploads it to the server. The server checks that the song complies with Singer A's usage policy and then saves it.

[0428] Listener C accesses the system, searches for a pop song, and plays it. The emotion engine analyzes Listener C's facial expression and recognizes that he or she is in a relaxed state. The system then recommends a pop song suitable for relaxation.

[0429] Finally, the server calculates revenue based on the playback data of listener C and distributes the revenue to singer A, creator B, and the original author. The revenue distribution results are sent to singer A and creator B via a notification function.

[0430] Prompt sentence for generative AI model

[0431] An example of a prompt to enter is: "When a user searches for a pop song they want to listen to in a relaxing state, what kind of songs should be recommended? How can you recommend them based on the analysis results of the emotion engine?"

[0432] In this way, the present invention realizes a system that provides a more personalized music experience by properly managing singers' audio rights, fair revenue distribution, and recommending songs based on user emotions.

[0433] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0434] The flow of this system's program processing

[0435] Step 1: Accepting user registration

[0436] Input: Singer enters their voice sample, usage policy and genre settings.

[0437] Processing: The server uploads the singer's vocal sample and stores the configured usage policy and genre information in a database.

[0438] Output: The singer's voice rights information is stored in a database.

[0439] Specific behavior:

[0440] Singers access the system using a terminal and upload a sample of their voice into a form.

[0441] The server receives this sample and stores it in a database along with the usage policy.

[0442] A registration completion message is sent to the singer.

[0443] Step 2: Save the song data

[0444] Input: Creators upload the music data and related metadata they have created to the server.

[0445] Processing: The server stores the uploaded song data and metadata in a database.

[0446] Output: Song data and metadata are stored in a database.

[0447] Specific behavior:

[0448] Creators use their devices to upload the completed music data to the system.

[0449] The server receives the song data and related metadata and stores it in a database.

[0450] Notify creator of upload completion message.

[0451] Step 3: Check the usage policy

[0452] Input: Song data uploaded by the creator and usage policy set by the singer.

[0453] Processing: The server checks whether the uploaded music data complies with the singer's usage policy.

[0454] Output: If the song data complies with the policy, it is saved in the database; if it does not, an error message is sent to the creator.

[0455] Specific behavior:

[0456] The server obtains the metadata of the song data and the singer's usage policy.

[0457] If the policy is violated, the song data will be refused upload and an error message will be sent to the creator.

[0458] If the policy is met, the music data is stored in the database.

[0459] Step 4: Music playback and emotion recognition

[0460] Input: Commands for listeners to search for and play songs, as well as camera footage and audio input from listeners.

[0461] Processing: The server streams the music and the emotion engine analyzes the listener's emotions.

[0462] Output: Songs are recommended in real time based on the listener's sentiment analysis.

[0463] Specific behavior:

[0464] Listeners can search for songs by entering keywords and start playing them.

[0465] The server streams the music data to the listener's device.

[0466] The emotion engine analyzes the listener's camera footage and audio input data to quantify their emotional state, such as relaxation or excitement.

[0467] Based on the analysis results, the server recommends the most suitable songs to the listener.

[0468] Step 5: Calculate revenue share

[0469] Input: Song playback and emotion data.

[0470] Processing: The server calculates revenue based on playback data and adjusts it based on sentiment data.

[0471] Output: The revenue distribution results are notified to each party (singer, creator, original author).

[0472] Specific behavior:

[0473] The server collects song playback data and tallys up the number of times and duration of playback.

[0474] A revenue sharing adjustment algorithm based on sentiment data is applied to calculate the final revenue sharing amount.

[0475] Notify each party of the revenue sharing results.

[0476] Step 6: Monitor for fraud

[0477] Input: Audio data registered on the server and newly uploaded audio data.

[0478] Processing: The server compares and monitors the audio data using an abuse monitoring algorithm.

[0479] Output: If any unauthorized use is detected, send a warning message to the relevant parties.

[0480] Specific behavior:

[0481] The server routinely monitors newly uploaded audio data.

[0482] The system compares registered voice data with new data and applies an algorithm to detect fraudulent use.

[0483] If unauthorized use is discovered, a warning message will be sent to the relevant singers and creators, urging them to take action.

[0484] In this way, the system of the present invention processes data at each step based on the user's actions to provide a well-curated and personalized music experience.

[0485] (Application example 2)

[0486] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0487] Conventional music streaming systems focused on managing the rights to use singers' voices and distributing revenue, but this made it difficult to provide personalized music experiences, such as song recommendations based on user emotions. There was also a need for improved transparency and fairness in revenue distribution. In addition, there was a need to streamline the revenue distribution process while preventing unauthorized use.

[0488] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0489] In this invention, the server includes a means for producing and releasing music on a platform for managing the rights to use singers' voices, a means for appropriately distributing revenue based on music playback, a means for managing and checking singers' voice policies, a monitoring means for preventing unauthorized use, a means including an emotion engine for analyzing users' emotions in real time, and a means for recommending songs based on the emotion analysis. This allows users to receive optimal song recommendations based on their current emotions, resulting in a more personalized music experience. It also improves the transparency and fairness of revenue distribution, enabling efficient revenue distribution while preventing unauthorized use.

[0490] "Right to use the singer's voice" means the singer's right to use his or her voice for a specific use or purpose, which is used under a controlled agreement.

[0491] "Means for producing and releasing music on the platform" refers to a set of functions and processes that allow singers and creators to produce music and release it through online services.

[0492] "Appropriate revenue distribution measures" refers to methods and systems for distributing revenue fairly and transparently to the parties involved based on the number of times the music is played or other metrics.

[0493] "Means for managing and checking the singer's voice policy" refers to a method or system for checking and managing whether the terms and rules of use set by the singer for his or her voice are being observed.

[0494] "Monitoring measures to prevent unauthorized use" refers to systems that monitor and detect violations to ensure that an artist's voice or music is not used in unauthorized ways or places.

[0495] An "emotion engine" refers to technology that analyzes a user's facial expressions, tone of voice, etc. to detect their emotional state in real time.

[0496] "Means for recommending music" refers to methods or systems that suggest music that matches the user's mood at the time based on the results of an analysis of the user's emotions, etc.

[0497] "Means for storing and managing generated music data" refers to a system that safely stores music data created by creators in digital format and provides management and access as needed.

[0498] "Means for notifying each participant of the results of revenue distribution" refers to a method or system for promptly notifying the participants of the calculated results of revenue distribution.

[0499] MODE FOR CARRYING OUT THE INVENTION

[0500] This invention provides a music streaming system that analyzes a user's emotions in real time to improve the music experience while properly managing the rights to use a singer's voice. Specific embodiments of the invention will now be described.

[0501] System configuration

[0502] The system includes the following main components:

[0503] 1. Server:

[0504] Singer voice usage rights management system: Provides an interface for singers to set and register their own voice usage policies.

[0505] Music data storage and management system: Music data and related metadata created by creators are stored in a database.

[0506] Revenue sharing system: Revenue is calculated based on music playback data and distributed to each party.

[0507] Emotion engine: Analyzes user emotions in real time and uses that data to recommend songs and influence revenue sharing.

[0508] Unauthorized use monitoring system: Monitors whether a singer's voice is being used illegally.

[0509] 2. User Device:

[0510] Singer's device: Register your voice usage rights on the platform and check the revenue sharing results.

[0511] Creator's device: Create and upload songs using the singer's voice.

[0512] Listener's device: In addition to playing music, the device's camera and microphone are used to collect and analyze emotional data.

[0513] Hardware and software used

[0514] Hardware:

[0515] Smartphone (camera, microphone)

[0516] Servers (high-performance processors, storage)

[0517] software:

[0518] Python

[0519] OpenCV (image analysis)

[0520] DeepFace (emotional analysis)

[0521] librosa (audio analysis)

[0522] requests (API communication)

[0523] Program processing explanation

[0524] The server first provides a platform for managing singers' voice usage rights and allows singers to set their own voice usage policies. Creators use this platform to create and upload songs, and the song data is stored on the server. The system checks whether songs comply with the singer's set policies when uploaded and also monitors them to prevent unauthorized use.

[0525] When a listener plays a song, the user device uses the smartphone's camera and microphone to collect emotional data from the user. The collected data is analyzed in real time using an emotion engine. Based on the results of this analysis, songs that fit the listener's mood at that time are recommended.

[0526] Finally, revenue will be calculated based on playback data and sentiment analysis data and distributed appropriately to singers, creators, and other stakeholders. The revenue distribution results will be notified to each participant.

[0527] Examples of concrete examples and prompts

[0528] As a concrete example, consider a scenario in which a user plays a song using a smartphone. As the user plays the song, the smartphone camera captures the user's facial expression, and the emotion engine determines that the user is "happy." Based on this result, the system recommends songs that are suitable for "happy." Below is an example of an input prompt sentence for the generative AI model.

[0529] Prompt statement:

[0530] "What songs would you recommend to a user who is in a positive mood? Based on the results of the emotion analysis, the user has been determined to be "happy." Please recommend some songs based on that determination."

[0531] In this way, the present invention provides a personalized music experience that responds to the user's emotions and realizes transparency and fairness in revenue sharing.

[0532] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0533] Step 1:

[0534] A user launches a smartphone app and creates an account. The input is the user's basic information (name, email address, etc.), and the output is a notification that the user account has been created. In this step, the user information is registered in the database.

[0535] Step 2:

[0536] A singer registers the rights to use their voice on the platform. The input is the singer's sample voice data and usage policy (e.g., only pop genres can be used). The output is a notification that registration is complete. The server saves the input data in the database and adds it to the policy management table.

[0537] Step 3:

[0538] A creator creates a song and uploads it to the server. The input is the song data and metadata (title, genre, etc.), and the output is a notification that the upload is complete. The server stores the song data in a database and checks the usage policy.

[0539] Step 4:

[0540] A listener uses the app to play a song. The input is the listener's song selection, and the output is the song streaming. During this step, the smartphone's camera and microphone collect the user's emotional data (facial expressions, voice).

[0541] Step 5:

[0542] The device's emotion engine analyzes the collected emotion data. The input is the user's facial expression image and voice data, and the output is the emotion analysis result (e.g., happy, sad). Here, OpenCV and DeepFace are used for image analysis, and librosa is used to analyze voice data.

[0543] Step 6:

[0544] Based on the sentiment analysis results, the server recommends songs. The input is the sentiment analysis results and the listener's playback history, and the output is a list of recommended songs. In this step, the generative AI model uses the collected data to select songs that correspond to the prompt.

[0545] Step 7:

[0546] The listener plays the recommended song. The input is a selection from a list of recommended songs, and the output is a streaming playback of the song. The playback data is sent to the server and used for revenue sharing.

[0547] Step 8:

[0548] The server calculates revenue based on the playback data and emotion data. The inputs are the number of plays, song usage, emotion data, etc., and the output is the revenue sharing result. In this step, the revenue sharing system calculates the revenue for each party and records it in the database.

[0549] Step 9:

[0550] The revenue distribution result is notified to each participant. The input is the revenue distribution result, and the output is a notification message. The server sends this notification to each participant's terminal, realizing fair revenue distribution.

[0551] In this way, the system analyzes users' emotions in real time, recommends songs based on the results, and ensures fair revenue distribution.

[0552] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0553] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0554] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0555] [Second embodiment]

[0556] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0557] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0558] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0559] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0560] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0561] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0562] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0563] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0564] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0565] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0566] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0567] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0568] This invention is a next-generation music streaming system that properly manages the rights to use singers' voices and achieves fair revenue distribution. The system includes a platform for managing voice usage rights, a function for distributing revenue based on music playback, a function for managing and checking singers' voice policies, and a monitoring function for preventing unauthorized use.

[0569] System Overview

[0570] This system consists of three main components: a server and user terminals (singers, creators, and listeners). Each component has a specific function and operates by communicating with each other.

[0571] 1. Server Functions

[0572] The server is responsible for the following main functions: managing the rights to use singers' voices, storing and managing song data, calculating revenue distribution and notifying the results. Specifically, it performs the following processes:

[0573] Accepting user registration: Provide an interface for singers to register information regarding the use of their voice.

[0574] Storing music data: Music data and related metadata created by creators are stored in a database.

[0575] Usage policy checks: When a song is uploaded, we check that it complies with the usage policy set by the artist.

[0576] Revenue sharing calculation: Calculate revenue based on playback data and distribute it to each party (singer, creator, original author).

[0577] Unauthorized use monitoring: Monitor singers' voices for unauthorized use.

[0578] 2. User device (singer) functions

[0579] Singers register their voice usage rights on the platform, set policies, and receive revenue sharing results through the system. Specifically, they perform the following operations:

[0580] Register voice usage rights: Upload sample voices and set usage policies.

[0581] Check the revenue sharing results: Receive notifications from the server and check the revenue sharing results.

[0582] 3. User Device (Creator) Functions

[0583] Creators create and upload songs using singers' voices. When uploading, they are required to ensure that the usage policy is adhered to. Specifically, they perform the following operations:

[0584] Voice selection: Select the singer's voice you want to use on the platform.

[0585] Uploading music: Upload the created music data and related metadata to the server.

[0586] 4. User terminal (listener) functions

[0587] Listeners search for and play songs on the platform by:

[0588] Song search: Enter keywords to search for songs.

[0589] Play Song: Select a song from the search results and stream it.

[0590] Specific examples

[0591] One day, singer A registers the rights to use his or her voice on a platform and sets a policy that it can only be used in the pop genre. In response, creator B creates a pop song using singer A's voice and uploads it to the platform. The server checks whether the uploaded song complies with singer A's policy and, if there are no problems, stores it in the database. Later, listener C searches for the pop song on the platform and plays it. Based on the playback data, the server calculates revenue and distributes it to singer A, creator B, and the author of the original song. Singer A and creator B are notified of the revenue distribution results.

[0592] In this way, the present invention provides functions for protecting the rights of singers' voices and achieving fair distribution of revenue, thereby building a music streaming system that can be used safely by all parties involved.

[0593] The processing flow will be explained below.

[0594] 1. User Registration and Contract Process

[0595] Step 1:

[0596] The singer (user) accesses the registration form.

[0597] The singer accesses the registration form provided on the system's web page.

[0598] Step 2:

[0599] The singer (user) enters their name, profile, voice sample, and desired usage policy.

[0600] Singers fill out a form with their name, profile information, a sample audio file of their voice, and voice usage policies (e.g., allowed genres, etc.).

[0601] Step 3:

[0602] The device sends the input to the server.

[0603] When the submit button on the registration form is clicked, the terminal transmits the information entered by the singer to the server.

[0604] Step 4:

[0605] The server stores the received information in a database.

[0606] The server stores the received registration information in a database and generates a unique ID for the singer.

[0607] Step 5:

[0608] The server will send a confirmation email to the singer.

[0609] The server will send a confirmation email to the singer's email address to notify them of the completion of registration.

[0610] 2. Creators' music production and release

[0611] Step 1:

[0612] The creator (user) logs in and accesses the dashboard.

[0613] Creators can log in by entering their account information and use the dashboard.

[0614] Step 2:

[0615] The creator (user) selects the singer's voice they want to use.

[0616] Creators select the voice they want to use from a list of singers provided on their dashboard.

[0617] Step 3:

[0618] The device sends the selection to the server.

[0619] Once the creator selects a voice, the content is sent to the server.

[0620] Step 4:

[0621] The server checks the policy of the selected singer's voice.

[0622] The server verifies that the selected singer's voice complies with the specified usage policy.

[0623] Step 5:

[0624] The device downloads a voice sample.

[0625] Once the server has completed the usage policy check, a voice sample is downloaded to the creator's device.

[0626] Step 6:

[0627] Creators (users) create music using AI tools.

[0628] Creators use AI tools (e.g., DAWs and AI voice plug-ins) to create music.

[0629] Step 7:

[0630] Creators (users) upload their completed songs to the platform.

[0631] Enter your completed music file and its metadata (e.g. title, genre, usage policy, etc.) into the upload form.

[0632] Step 8:

[0633] The device sends the uploaded content to the server.

[0634] Clicking the upload button sends the music file and metadata to the server.

[0635] Step 9:

[0636] The server checks the metadata and stores it in a database.

[0637] The server checks the metadata of the song to ensure it does not violate usage policies, and if there are no problems, stores the song data in a database.

[0638] 3. Listeners play music

[0639] Step 1:

[0640] The listener (user) logs in and searches for songs.

[0641] Listeners log in by entering their account information and then enter keywords into the search bar.

[0642] Step 2:

[0643] The device sends the search keywords to the server.

[0644] When you click the search button, the search keywords are sent to the server.

[0645] Step 3:

[0646] The server searches the database for the corresponding song and returns the results to the terminal.

[0647] The server searches the database for songs that match the keywords and sends the results to the listener's device.

[0648] Step 4:

[0649] The listener (user) selects the song to play.

[0650] Listeners select the song they want to play from the search results and click the play button.

[0651] Step 5:

[0652] The server begins streaming the selected song.

[0653] The server streams the selected songs and sends them to the listener's device.

[0654] 4. Revenue Sharing

[0655] Step 1:

[0656] The server periodically obtains the playback data.

[0657] The server periodically obtains the playback data from the playback history table.

[0658] Step 2:

[0659] The server performs the revenue calculation.

[0660] The server calculates the total revenue for each song based on the number of plays.

[0661] Step 3:

[0662] The server generates data for distributing revenue to each participant (author, singer, creator).

[0663] The server generates data for distributing the profits to each participant and calculates them based on the distribution ratio.

[0664] Step 4:

[0665] The server stores the revenue sharing results in a database and sends notifications to each participant.

[0666] The server stores the distribution results in a database and sends notifications to participants via email or other means.

[0667] The above is a specific processing flow for implementing the present invention, which realizes the protection of the singer's voice rights and the fair distribution of profits.

[0668] Example 1

[0669] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0670] In today's music streaming services, the rights of artists and other creators are often not properly protected. Furthermore, revenue distribution is often opaque and not fair. Furthermore, unauthorized use of artists' voices is common and often goes undetected. To solve these problems, a system is needed that properly manages the rights to use artists' voices, distributes revenue fairly, and prevents unauthorized use.

[0671] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0672] In this invention, the server includes a means for producing and releasing music on a platform for managing the rights to use singers' voices, a means for appropriately distributing revenue based on music playback, a means for managing and checking policies for singers' voices, a monitoring means for preventing unauthorized use, a server means for providing an interface for registering information regarding the use of users' voices, a database means for storing music sheet data and related metadata, and a server means for automatically performing policy checks, thereby enabling the protection of the rights of singers and creators, the fair distribution of revenue, and the prevention of unauthorized use.

[0673] A "singer" is an individual or group of individuals who provides their voice in a musical composition.

[0674] "Usage rights" are legal authorizations that permit the use of a particular vocal or musical work.

[0675] "Platform" refers to an online service or software that enables users to publish their content and interact with other users.

[0676] "Music" is an art form created by combining sounds and voices, and is a means of expression through the ears.

[0677] "Playback" is the act of listening to music or audio using a device.

[0678] "Revenue" is the monetary benefit derived from the provision of a service or consumption of a work.

[0679] A "policy" is a rule or guideline that prescribes a particular purpose or behavior.

[0680] "Monitoring" is the act of continually checking a particular activity or situation and responding when a problem arises.

[0681] An "interface" is the method or means by which a user interacts with a system or software.

[0682] A "server" is a computer system that provides data over a network.

[0683] A "database" is a system for efficiently storing, managing, and searching large amounts of data.

[0684] "Metadata" refers to data that contains information about specific data, such as the title of a song or the name of an artist.

[0685] A "policy check" is a process that verifies whether certain conditions are met based on a registered policy.

[0686] "Revenue sharing" is the act of distributing earned revenue fairly among multiple rights holders and related parties.

[0687] This is a next-generation music streaming system that properly manages the rights to use singers' voices and ensures fair revenue distribution. The system consists of three main components: a server and user terminals (singers, creators, and listeners).

[0688] Server Roles and Functions

[0689] The server is responsible for the central functions of the system. It performs the following main processes:

[0690] Accepting user registrations:

[0691] The server provides an interface for singers to register information about the use of their voice. This interface is a web-based application that singers can access from their terminals.

[0692] Save song data:

[0693] The music data and related metadata created by creators are stored in a database, which uses a common database management system such as MySQL or PostgreSQL.

[0694] Usage policy check:

[0695] The server automatically checks whether the uploaded song complies with the artist's usage policy, using a specific algorithm.

[0696] Revenue Share Calculation:

[0697] The server calculates revenue based on song playback data and distributes it to each party (singer, creator, original author), taking into account the number of plays, playback time, advertising revenue, etc.

[0698] Abuse Monitoring:

[0699] The server monitors the singer's voice for fraudulent use, which is achieved by using an AI model to check for voice pattern matches.

[0700] Roles and functions of user devices

[0701] The role of the singer

[0702] Singers will register the rights to use their voice and receive notification of the revenue sharing results. Specifically, they will:

[0703] Voice usage rights registration:

[0704] Singers upload their own sample voices from their devices and set usage policies.

[0705] Check your revenue sharing results:

[0706] The singer receives a notification from the server and checks the revenue sharing results.

[0707] The role of creators

[0708] Creators create and upload songs using singers' voices. Specifically, they do the following:

[0709] Voice Selection:

[0710] Creators choose the singer's voice they want to use on the platform.

[0711] Upload your song:

[0712] Creators upload the music data and related metadata they have created to the server, where they are checked to ensure that the usage policy is being followed.

[0713] The role of the listener

[0714] Listeners search for and play songs on the platform by:

[0715] Song search:

[0716] Listeners can input keywords to search for songs.

[0717] Play music:

[0718] Listeners can select songs from the search results and stream them.

[0719] Specific examples

[0720] One day, singer A registers the rights to use his or her voice on a platform and sets a policy that it can only be used in the pop genre. In response, creator B creates a pop song using singer A's voice and uploads it to the platform. The server checks whether the uploaded song complies with singer A's policy and, if there are no problems, stores it in the database. Later, listener C searches for the pop song on the platform and plays it. Based on the playback data, the server calculates revenue and distributes it to singer A, creator B, and the author of the original song. Singer A and creator B are notified of the revenue distribution results.

[0721] Example prompt:

[0722] "Please explain the user experience when Singer A sets a policy that allows music to be used only in the pop genre."

[0723] In this way, the invention provides a concrete means to protect singers' vocal rights and ensure fair distribution of revenue, creating a music streaming system that can be used safely by all parties involved.

[0724] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0725] Step 1: User registration and rights registration

[0726] Input: The singer uses the device to enter their name, contact information, sample voice (audio file), and usage policy.

[0727] What happens: A singer accesses the registration form, fills in the required information, uploads a sample voice, sets usage policies, and clicks the submit button.

[0728] Data processing / calculation: The server receives the input data, parses the usage policy and saves it in the database. It also saves the sample voice audio file.

[0729] Output: The server confirms that user registration and rights registration have been completed, and sends a registration completion notification to the singer's terminal.

[0730] Step 2: Choose your creator's voice

[0731] Input: Creators use their device to enter search criteria for the voice they want to use (e.g. genre, singer name).

[0732] How it works: Creators log in to the platform and enter criteria such as genre and artist name into the search form.

[0733] Data processing / calculation: The server queries the database based on the search criteria and returns filtered sample voice information of the corresponding singer.

[0734] Output: As a search result, a list of available singers and sample voice information are displayed on the creator's device.

[0735] Step 3: Upload your music

[0736] Input: Enter the audio file and metadata (e.g. title, genre, description) of the song created by the creator.

[0737] What happens: Creator selects a music file, enters relevant metadata into the form, and clicks the upload button.

[0738] Data processing / calculation: The server receives the uploaded audio file and metadata, checks the integrity of the file, and then verifies whether the song is permitted based on the usage policy.

[0739] Output: If the song data complies with the policy, the server saves the song data in the database and sends an upload completion notification to the creator's device. If it does not comply, it returns an error message.

[0740] Step 4: Search and play songs

[0741] Input: A listener uses a device to enter keywords to search for a song.

[0742] Specific actions: Listeners access the platform, enter keywords into the search form, and click the search button.

[0743] Data processing / calculation: The server queries the database based on the search keywords and generates a list of relevant songs.

[0744] Output: As a search result, a list of matching songs is displayed on the listener's device. When the listener selects a song and clicks the play button, the server sends the streaming data and plays the song.

[0745] Step 5: Calculate revenue share

[0746] Input: The server compiles song playback data (e.g., number of plays, playback time, advertising revenue).

[0747] What it does: The server records playback data every time a song is played on the platform. It aggregates the playback data over specific time intervals.

[0748] Data processing / calculation: The server uses a revenue sharing algorithm to calculate revenue, taking into account the number of plays, play time, ad revenue, etc.

[0749] Output: The server stores the revenue sharing results in a database and sends notifications to the singer, creator, and original author.

[0750] Step 6: Monitor for fraud

[0751] Input: Periodically scan all song data in the database.

[0752] Specific operation: The server executes a scheduled task at regular intervals to analyze all songs.

[0753] Data processing / calculation: The server uses AI models to detect unregistered usage patterns and unauthorized copying.

[0754] Output: If any unauthorized use is detected, the server will send a warning notice to the singer and the system administrator.

[0755] (Application example 1)

[0756] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0757] With the recent spread of music streaming services, fairness in the rights to use singers' voices and revenue distribution has become increasingly important. However, existing systems have insufficient management of rights to use singers' voices and revenue distribution, resulting in fraudulent use and unfair revenue distribution. Furthermore, there is a need for systems that make it easy for users to search and play music on their smart devices while also providing appropriate rights management. However, there is a lack of systems that combine an interactive, easy-to-use user interface with advanced voice analysis technology.

[0758] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0759] In this invention, the server includes means for producing and releasing music on a platform for managing the rights to use a singer's voice, means for appropriately distributing revenue based on music playback, means for managing and checking policies for the singer's voice, means for monitoring to prevent unauthorized use, means for using smart contracts to manage users' voice rights, means for providing streaming playback of music, means for notifying users of revenue distribution results, and means for providing a user interface that operates on a smartphone or a head-mounted display, thereby ensuring the appropriate use of a singer's voice and fairness in revenue distribution, and further improving the user experience on smart devices.

[0760] The "platform for managing singers' voice usage rights" is an online system that allows singers to register rights to use their voice and manage the scope and conditions of that use.

[0761] "Means for producing and releasing music" refers to a system that allows music creators to create music and publish that music data on the platform.

[0762] "Means for appropriate distribution of revenue" is a function for calculating revenue based on song playback data and distributing it fairly to the parties involved.

[0763] "Means for managing and checking singers' voice policies" refers to a system for checking whether the conditions and scope of voice usage set by singers are being properly observed.

[0764] "Monitoring measures to prevent unauthorized use" refers to a monitoring function to prevent the singer's voice from being used without permission.

[0765] "Means using smart contracts" refers to a mechanism that uses blockchain technology to automate the management of audio rights and revenue distribution, ensuring transparency.

[0766] "Means for providing streaming music playback" refers to a function that allows users to listen to music in real time over the Internet.

[0767] The "means for notifying the profit distribution results" is a function for promptly notifying the parties concerned of the profit calculation results.

[0768] A "user interface that runs on a smartphone or head-mounted display" is an operation screen or interface that allows users to use the system through a device such as a smartphone or VR device.

[0769] The system embodying this invention comprises the following main components: a server, a singer terminal, a creator terminal, and a listener terminal.

[0770] Server Features

[0771] The server provides the following functionality:

[0772] 1. Accepting user registration: The server provides an interface that allows singers, creators, and listeners to create accounts. In this case, AWS Cognito is used to manage account information.

[0773] 2. Managing singers' voice usage rights: Provides a function for singers to register their voice usage rights and set usage policies. Transparency and security are ensured by using smart contracts and blockchain technology (e.g., Ethereum).

[0774] 3. Storage and management of music data: Music data created by creators is stored in a database on the server (e.g., MySQL, PostgreSQL).

[0775] 4. Usage policy check: When a song is uploaded, the server checks whether it complies with the artist's usage policy. It uses voice analysis technology such as Google Cloud Speech-to-Text to scrutinize the audio data.

[0776] 5. Unauthorized use monitoring: Monitor singers' voices for unauthorized use and issue alerts if necessary.

[0777] 6. Providing a music streaming service: Using the Spotify SDK, we provide a fast, high-quality playback experience for listeners to stream music.

[0778] 7. Revenue share calculation: Based on the song playback data, revenue is calculated and distributed fairly to the parties involved. This process is performed using AWS Lambda and DynamoDB.

[0779] 8. Notification of Revenue Distribution Results: The singer, creator, and original author will be notified of the revenue distribution results via email.

[0780] User device functions

[0781] 1. Singer terminal: Singers can register their voice usage rights, manage the policies they have set, and check the revenue distribution results.

[0782] 2. Creator Terminal: Creators can create and upload music and ensure that usage policies are properly adhered to.

[0783] 3. Listener device: Listeners have the ability to search for songs and play them in streaming.

[0784] Specific examples

[0785] For example:

[0786] One day, a singer registers the rights to use her voice on a platform and sets a policy that it can only be used in the pop genre. In response, a creator creates a pop song using the singer's voice and uploads it to the platform. The server checks whether the uploaded song complies with the singer's policy and, if there are no problems, stores it in the database. A listener then searches for and plays the pop song on the platform. Based on the playback data, the server calculates revenue and distributes it to the singer, creator, and the copyright holder of the original song. The singer and creator are notified of the revenue distribution results.

[0787] The following prompt example is also provided:

[0788] Singer: "I only want my voice to be used in songs in the pop genre."

[0789] Creator: "I've created a new pop song and I want to use the singer's voice."

[0790] Listener: "I want to hear the latest pop songs."

[0791] The system ensures the proper use of singers' voices and fair revenue distribution, while also improving the user experience on smart devices.

[0792] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0793] Step 1:

[0794] A singer uses a terminal to register the rights to use their voice on the platform. At this time, the singer uploads a sample of their voice and sets a policy that it can only be used in the pop genre. The server receives the sample voice and policy data as input and registers it in a smart contract using blockchain technology to ensure transparency and security.

[0795] Step 2:

[0796] Creators use their devices to create songs using singers' voices and upload them to the platform. Creators then send the song data to the server based on the policy. The server stores the song data and metadata in a MySQL or PostgreSQL database and uses Google Cloud Speech-to-Text technology to check whether the voice data complies with the policy.

[0797] Step 3:

[0798] The server checks whether the uploaded song complies with the usage policy set by the artist. If it is confirmed, the song data is officially stored in the platform's database. Conversely, if there is a policy violation, the creator is notified and the upload is stopped.

[0799] Step 4:

[0800] Listeners use their devices to search for and stream songs on the platform. Listeners enter keywords, and the server searches the database for songs and outputs the results. The song data is then streamed to the listener's device using the Spotify SDK.

[0801] Step 5:

[0802] The server collects listeners' song play data, recording the number of plays and play time. This data is later used to calculate revenue distribution. The server processes the data in real time using AWS Lambda and DynamoDB to calculate how revenue should be distributed to each party.

[0803] Step 6:

[0804] The server distributes revenue based on the calculation results. The revenue is distributed to the singer, creator, and original song author based on a predetermined revenue distribution ratio. The distribution results are sent to the relevant parties via email or other notification methods.

[0805] Step 7:

[0806] After the server completes all procedures, it displays the revenue distribution results in real time on the user's dashboard, allowing all parties involved to instantly see how the revenue has been distributed.

[0807] Examples:

[0808] Singer: "I only want my voice to be used in songs in the pop genre."

[0809] Creator: "I've created a new pop song and I want to use the singer's voice."

[0810] Listener: "I want to hear the latest pop songs."

[0811] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0812] This is a next-generation music streaming system that recognizes users' emotions to improve the music experience and distributes revenue fairly, while properly managing the rights to use singers' voices. In addition to the functions of conventional music streaming systems, this system uses an emotion engine to recognize users' emotions and adjusts song recommendations and revenue distribution based on these.

[0813] System Overview

[0814] This system includes three main components: a server, user terminals (singers, creators, and listeners), and an emotion engine. These components have specific functions and operate while communicating with each other.

[0815] 1. Server Functions

[0816] The server manages the rights to use singers' voices, stores and manages song data, calculates revenue distribution and notifies the results, and processes emotion data using an emotion engine. Specifically, it performs the following processes:

[0817] Accepting user registration: Provide an interface for singers to register information regarding the use of their voice.

[0818] Storing music data: Music data and related metadata created by creators are stored in a database.

[0819] Usage policy checks: When a song is uploaded, we check that it complies with the usage policy set by the artist.

[0820] Revenue sharing calculation: Calculate revenue based on playback data and distribute it to each party (singer, creator, original author).

[0821] Unauthorized use monitoring: Monitor singers' voices for unauthorized use.

[0822] Emotional Data Analysis: Emotional data generated by the emotion engine is collected and analyzed to improve user experience and adjust revenue sharing.

[0823] 2. User device (singer) functions

[0824] Singers register their voice usage rights on the platform, set policies, and receive revenue sharing results through the system. Specifically, they perform the following operations:

[0825] Register voice usage rights: Upload sample voices and set usage policies.

[0826] Check the revenue sharing results: Receive notifications from the server and check the revenue sharing results.

[0827] 3. User Device (Creator) Functions

[0828] Creators create and upload songs using singers' voices. When uploading, they are required to ensure that the usage policy is adhered to. Specifically, they perform the following operations:

[0829] Voice selection: Select the singer's voice you want to use on the platform.

[0830] Uploading music: Upload the created music data and related metadata to the server.

[0831] 4. User terminal (listener) functions

[0832] Listeners search for and play songs on the platform, and the system recognizes their emotions and recommends appropriate songs. Specifically, they perform the following actions:

[0833] Song search: Enter keywords to search for songs.

[0834] Play Song: Select a song from the search results and stream it.

[0835] Emotion recognition: The emotion engine analyzes the user's emotions using the device's camera and voice input. Based on the analysis results, it recommends songs that suit the user.

[0836] Specific examples

[0837] One day, Singer A registers the rights to use his or her voice on the platform and sets a policy that it can only be used in the pop genre. In response, Creator B creates a pop song using Singer A's voice and uploads it to the platform. The server checks whether the uploaded song complies with Singer A's policy and, if there are no problems, stores it in the database. Listener C then searches for and plays a pop song on the platform. The emotion engine analyzes Listener C's facial expression and recognizes that he or she is relaxed, so songs in a similar pop genre are recommended.

[0838] Based on the playback data, the server calculates revenue and distributes it to singer A, creator B, and the copyright holder of the original song. The revenue distribution results are notified to singer A and creator B. In addition, revenue adjustments are also performed based on emotional data, realizing optimal revenue distribution according to the user's emotions.

[0839] In this way, the present invention not only protects the rights of singers' voices and ensures fair distribution of revenue, but also builds a system that provides a more personalized music experience through a recommendation function based on user emotions.

[0840] The processing flow will be explained below.

[0841] 1. User Registration and Contract Process

[0842] Step 1:

[0843] The singer (user) accesses the registration form.

[0844] The singer accesses the registration form provided on the system's web page.

[0845] Step 2:

[0846] The singer (user) enters their name, profile, voice sample, and desired usage policy.

[0847] Singers fill out a form with their name, profile information, a sample audio file of their voice, and usage policies (e.g., allowed genres, etc.).

[0848] Step 3:

[0849] The device sends the input to the server.

[0850] When the registration form submit button is clicked, the terminal sends the entered information to the server.

[0851] Step 4:

[0852] The server stores the received information in a database.

[0853] The server stores the received information in a database and generates a unique ID for the singer.

[0854] Step 5:

[0855] The server will send a confirmation email to the singer.

[0856] The server will send a confirmation email to the singer's email address to notify them of the completion of registration.

[0857] 2. Creators' music production and release

[0858] Step 1:

[0859] The creator (user) logs in and accesses the dashboard.

[0860] Creators log in on the system's web page and access their dashboard.

[0861] Step 2:

[0862] The creator (user) selects the singer's voice they want to use.

[0863] Creators select the voice they want to use from a list of singers provided on their dashboard.

[0864] Step 3:

[0865] The device sends the selection to the server.

[0866] Once the creator selects a voice, that information is sent to a server.

[0867] Step 4:

[0868] The server checks the policy of the selected singer's voice.

[0869] The server checks whether the selected singer's voice complies with the established usage policy.

[0870] Step 5:

[0871] The device downloads a voice sample.

[0872] Once the policy check is complete, the server will have the device download a voice sample file.

[0873] Step 6:

[0874] Creators (users) create music using AI tools.

[0875] Creators use AI tools to create and complete songs.

[0876] Step 7:

[0877] Creators (users) upload their completed songs to the platform.

[0878] Enter the completed music file and its metadata through the upload form.

[0879] Step 8:

[0880] The device sends the uploaded content to the server.

[0881] Clicking the upload button sends the music file and metadata to the server.

[0882] Step 9:

[0883] The server checks the metadata and stores it in a database.

[0884] The server checks the uploaded metadata, verifies that it does not violate any policies, and then stores it in the database.

[0885] 3. Listeners play music

[0886] Step 1:

[0887] The listener (user) logs in and searches for songs.

[0888] Listeners log into the system and enter keywords into the search bar.

[0889] Step 2:

[0890] The device sends the search keywords to the server.

[0891] When you click the search button, the entered keywords are sent to the server.

[0892] Step 3:

[0893] The server searches the database for the corresponding song and returns the results to the terminal.

[0894] The server searches the database for songs that match the keywords and sends the results to the listener's device.

[0895] Step 4:

[0896] The listener (user) selects the song to play.

[0897] Listeners select the song they want to play from the search results and click the play button.

[0898] Step 5:

[0899] The server begins streaming the selected song.

[0900] The server streams the selected songs and sends them to the listener's device.

[0901] Step 6:

[0902] The device recognizes the user's emotions using an emotion engine.

[0903] During playback, the device uses an emotion engine to analyze emotions from the user's facial expressions and voice.

[0904] Step 7:

[0905] The server receives the emotional data and recommends appropriate songs.

[0906] It receives data from the emotion engine and recommends songs that suit the user.

[0907] 4. Revenue Sharing

[0908] Step 1:

[0909] The server periodically obtains the playback data.

[0910] The server periodically obtains the playback data from the playback history table.

[0911] Step 2:

[0912] The server performs the revenue calculation.

[0913] The server calculates the total revenue for each song based on the number of plays and sentiment data.

[0914] Step 3:

[0915] The server generates data for distributing revenue to each participant (author, singer, creator).

[0916] The server generates data for distributing the profits to each participant based on the distribution ratio.

[0917] Step 4:

[0918] The server stores the revenue sharing results in a database and sends notifications to each participant.

[0919] The server stores the distribution results in a database and notifies each party of the revenue results via email or other means.

[0920] In this way, the present invention is a system that not only protects the rights of singers' voices and ensures fair distribution of revenue, but also provides a more personalized music experience through a recommendation function based on user emotions.

[0921] Example 2

[0922] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0923] Conventional music streaming systems have problems such as difficulty in properly managing singers' audio rights and fair revenue distribution. Furthermore, they lack a function for recommending songs based on user emotions. Therefore, a system to improve the user experience is needed.

[0924] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for producing and releasing music on a platform for managing singers' audio rights, a means for appropriately distributing revenue based on music playback, a means for managing and checking audio usage policies, a monitoring means for preventing unauthorized use, a means for recognizing user emotions and recommending songs based on the emotions, and a means for adjusting revenue distribution based on emotion data. This enables appropriate management of singers' audio rights, fair revenue distribution, and song recommendations based on user emotions.

[0925] "Singer's audio rights" refers to the right of a singer to commercially exploit his or her voice.

[0926] "Platform" refers to a system that enables users such as singers, creators, and listeners to create, play, and manage music via the Internet.

[0927] "Means for appropriately distributing revenue based on music plays" refers to algorithms or functions that distribute revenue to each party based on playback data.

[0928] "Audio usage policy" refers to the rules and standards that a singer sets for the conditions under which their voice is used and that are managed and checked based on those conditions.

[0929] "Monitoring measures to prevent unauthorized use" refers to monitoring systems and algorithms that check whether a singer's voice is being used fraudulently.

[0930] "Means for recognizing user emotions and recommending songs based on that" refers to a system that analyzes user emotions using technologies such as emotion engines and recommends songs based on the results.

[0931] "Means for adjusting revenue sharing based on emotional data" refers to an algorithm or function that adjusts the revenue sharing rate by taking into account the emotional data of users.

[0932] "Music data" refers to music files and associated metadata, such as title, artist information, genre, and other information.

[0933] "Database" refers to a system for managing stored audio rights information, music data, usage policies, playback data, user emotional data, and the like.

[0934] "Revenue Share Results" means the revenue share results for each party calculated based on playback data and other data.

[0935] MODE FOR CARRYING OUT THE INVENTION

[0936] This invention is a next-generation music streaming system that appropriately manages singers' audio rights, recognizes users' emotions to improve the music experience, and distributes revenue fairly. The main components of this system include a server and user terminals (singers, creators, and listeners), each with specific functions and operating while communicating with each other.

[0937] Server Features

[0938] The server is configured to perform the following main tasks:

[0939] Accepting user registration: Provides an interface for singers to register the rights to use their voice. Singers can upload their voice samples and enter genre settings and usage policies. The server receives this data and stores it in a database.

[0940] Music data storage: Creators upload the music data and related metadata they have created, which is then stored and managed on the server.

[0941] Usage policy check: Checks whether the uploaded song complies with the usage policy set by the artist. If it does not comply with the policy, the upload will be rejected and an error message will be sent to the creator.

[0942] Revenue sharing calculation: Calculate revenue based on playback data. Notify each party (singer, creator, original author) of the revenue sharing results.

[0943] Unauthorized use monitoring: We monitor whether singers' voices are being used illegally. If any unauthorized use is detected, we will send a warning message to the singer and take appropriate measures.

[0944] Emotion data analysis: The server analyzes the emotion data collected by the emotion engine, and uses this data to improve user experience and adjust revenue sharing.

[0945] As specific examples of hardware and software, the server uses an Amazon Web Services (AWS) EC2 instance, the database uses AWS RDS (Relational Database Service), and the emotion engine uses the Emotion API from Azure Cognitive Services.

[0946] User device (singer) functions

[0947] On the singer's device, do the following:

[0948] Registering vocal usage rights: Singers access the system, upload their vocal samples, set usage policies, and specify specific genres and uses.

[0949] Check the revenue sharing results: Receive notifications from the server and check the revenue sharing results.

[0950] User device (creator) functions

[0951] On the creator's device, do the following:

[0952] Voice Selection: Creators access the system and select the singer's voice they wish to use. They then review the usage policy for the selected voice.

[0953] Uploading music: Upload the created music data and related metadata to the server.

[0954] User terminal (listener) functions

[0955] On the listener's device, do the following:

[0956] Song search: Listeners access the system and enter keywords to search for songs.

[0957] Song Playback: The listener selects a song and streams it. During playback, the emotion engine analyzes the listener's emotions.

[0958] Emotion Recognition: The system collects the listener's emotions from the device's camera and voice input. Based on the analysis results of the emotion engine, the system recommends songs that suit the user.

[0959] Specific examples

[0960] One day, Singer A registers the rights to use his or her voice on the platform, allowing it to be used in the pop genre. Creator B then creates a pop song using that voice and uploads it to the server. The server checks that the song complies with Singer A's usage policy and then saves it.

[0961] Listener C accesses the system, searches for a pop song, and plays it. The emotion engine analyzes Listener C's facial expression and recognizes that he or she is in a relaxed state. The system then recommends a pop song suitable for relaxation.

[0962] Finally, the server calculates revenue based on the playback data of listener C and distributes the revenue to singer A, creator B, and the original author. The revenue distribution results are sent to singer A and creator B via a notification function.

[0963] Prompt sentence for generative AI model

[0964] An example of a prompt to enter is: "When a user searches for a pop song they want to listen to in a relaxing state, what kind of songs should be recommended? How can you recommend them based on the analysis results of the emotion engine?"

[0965] In this way, the present invention realizes a system that provides a more personalized music experience by properly managing singers' audio rights, fair revenue distribution, and recommending songs based on user emotions.

[0966] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0967] The flow of this system's program processing

[0968] Step 1: Accepting user registration

[0969] Input: Singer enters their voice sample, usage policy and genre settings.

[0970] Processing: The server uploads the singer's vocal sample and stores the configured usage policy and genre information in a database.

[0971] Output: The singer's voice rights information is stored in a database.

[0972] Specific behavior:

[0973] Singers access the system using a terminal and upload a sample of their voice into a form.

[0974] The server receives this sample and stores it in a database along with the usage policy.

[0975] A registration completion message is sent to the singer.

[0976] Step 2: Save the song data

[0977] Input: Creators upload the music data and related metadata they have created to the server.

[0978] Processing: The server stores the uploaded song data and metadata in a database.

[0979] Output: Song data and metadata are stored in a database.

[0980] Specific behavior:

[0981] Creators use their devices to upload the completed music data to the system.

[0982] The server receives the song data and related metadata and stores it in a database.

[0983] Notify creator of upload completion message.

[0984] Step 3: Check the usage policy

[0985] Input: Song data uploaded by the creator and usage policy set by the singer.

[0986] Processing: The server checks whether the uploaded music data complies with the singer's usage policy.

[0987] Output: If the song data complies with the policy, it is saved in the database; if it does not, an error message is sent to the creator.

[0988] Specific behavior:

[0989] The server obtains the metadata of the song data and the singer's usage policy.

[0990] If the policy is violated, the song data will be refused upload and an error message will be sent to the creator.

[0991] If the policy is met, the music data is stored in the database.

[0992] Step 4: Music playback and emotion recognition

[0993] Input: Commands for listeners to search for and play songs, as well as camera footage and audio input from listeners.

[0994] Processing: The server streams the music and the emotion engine analyzes the listener's emotions.

[0995] Output: Songs are recommended in real time based on the listener's sentiment analysis.

[0996] Specific behavior:

[0997] Listeners can search for songs by entering keywords and start playing them.

[0998] The server streams the music data to the listener's device.

[0999] The emotion engine analyzes the listener's camera footage and audio input data to quantify their emotional state, such as relaxation or excitement.

[1000] Based on the analysis results, the server recommends the most suitable songs to the listener.

[1001] Step 5: Calculate revenue share

[1002] Input: Song playback and emotion data.

[1003] Processing: The server calculates revenue based on playback data and adjusts it based on sentiment data.

[1004] Output: The revenue distribution results are notified to each party (singer, creator, original author).

[1005] Specific behavior:

[1006] The server collects song playback data and tallys up the number of times and duration of playback.

[1007] A revenue sharing adjustment algorithm based on sentiment data is applied to calculate the final revenue sharing amount.

[1008] Notify each party of the revenue sharing results.

[1009] Step 6: Monitor for fraud

[1010] Input: Audio data registered on the server and newly uploaded audio data.

[1011] Processing: The server compares and monitors the audio data using an abuse monitoring algorithm.

[1012] Output: If any unauthorized use is detected, send a warning message to the relevant parties.

[1013] Specific behavior:

[1014] The server routinely monitors newly uploaded audio data.

[1015] The system compares registered voice data with new data and applies an algorithm to detect fraudulent use.

[1016] If unauthorized use is discovered, a warning message will be sent to the relevant singers and creators, urging them to take action.

[1017] In this way, the system of the present invention processes data at each step based on the user's actions to provide a well-curated and personalized music experience.

[1018] (Application example 2)

[1019] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1020] Conventional music streaming systems focused on managing the rights to use singers' voices and distributing revenue, but this made it difficult to provide personalized music experiences, such as song recommendations based on user emotions. There was also a need for improved transparency and fairness in revenue distribution. In addition, there was a need to streamline the revenue distribution process while preventing unauthorized use.

[1021] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1022] In this invention, the server includes a means for producing and releasing music on a platform for managing the rights to use singers' voices, a means for appropriately distributing revenue based on music playback, a means for managing and checking singers' voice policies, a monitoring means for preventing unauthorized use, a means including an emotion engine for analyzing users' emotions in real time, and a means for recommending songs based on the emotion analysis. This allows users to receive optimal song recommendations based on their current emotions, resulting in a more personalized music experience. It also improves the transparency and fairness of revenue distribution, enabling efficient revenue distribution while preventing unauthorized use.

[1023] "Right to use the singer's voice" means the singer's right to use his or her voice for a specific use or purpose, which is used under a controlled agreement.

[1024] "Means for producing and releasing music on the platform" refers to a set of functions and processes that allow singers and creators to produce music and release it through online services.

[1025] "Appropriate revenue distribution measures" refers to methods and systems for distributing revenue fairly and transparently to the parties involved based on the number of times the music is played or other metrics.

[1026] "Means for managing and checking the singer's voice policy" refers to a method or system for checking and managing whether the terms and rules of use set by the singer for his or her voice are being observed.

[1027] "Monitoring measures to prevent unauthorized use" refers to systems that monitor and detect violations to ensure that an artist's voice or music is not used in unauthorized ways or places.

[1028] An "emotion engine" refers to technology that analyzes a user's facial expressions, tone of voice, etc. to detect their emotional state in real time.

[1029] "Means for recommending music" refers to methods or systems that suggest music that matches the user's mood at the time based on the results of an analysis of the user's emotions, etc.

[1030] "Means for storing and managing generated music data" refers to a system that safely stores music data created by creators in digital format and provides management and access as needed.

[1031] "Means for notifying each participant of the results of revenue distribution" refers to a method or system for promptly notifying the participants of the calculated results of revenue distribution.

[1032] MODE FOR CARRYING OUT THE INVENTION

[1033] This invention provides a music streaming system that analyzes a user's emotions in real time to improve the music experience while properly managing the rights to use a singer's voice. Specific embodiments of the invention will now be described.

[1034] System configuration

[1035] The system includes the following main components:

[1036] 1. Server:

[1037] Singer voice usage rights management system: Provides an interface for singers to set and register their own voice usage policies.

[1038] Music data storage and management system: Music data and related metadata created by creators are stored in a database.

[1039] Revenue sharing system: Revenue is calculated based on music playback data and distributed to each party.

[1040] Emotion engine: Analyzes user emotions in real time and uses that data to recommend songs and influence revenue sharing.

[1041] Unauthorized use monitoring system: Monitors whether a singer's voice is being used illegally.

[1042] 2. User Device:

[1043] Singer's device: Register your voice usage rights on the platform and check the revenue sharing results.

[1044] Creator's device: Create and upload songs using the singer's voice.

[1045] Listener's device: In addition to playing music, the device's camera and microphone are used to collect and analyze emotional data.

[1046] Hardware and software used

[1047] Hardware:

[1048] Smartphone (camera, microphone)

[1049] Servers (high-performance processors, storage)

[1050] software:

[1051] Python

[1052] OpenCV (image analysis)

[1053] DeepFace (emotional analysis)

[1054] librosa (audio analysis)

[1055] requests (API communication)

[1056] Program processing explanation

[1057] The server first provides a platform for managing singers' voice usage rights and allows singers to set their own voice usage policies. Creators use this platform to create and upload songs, and the song data is stored on the server. The system checks whether songs comply with the singer's set policies when uploaded and also monitors them to prevent unauthorized use.

[1058] When a listener plays a song, the user device uses the smartphone's camera and microphone to collect emotional data from the user. The collected data is analyzed in real time using an emotion engine. Based on the results of this analysis, songs that fit the listener's mood at that time are recommended.

[1059] Finally, revenue will be calculated based on playback data and sentiment analysis data and distributed appropriately to singers, creators, and other stakeholders. The revenue distribution results will be notified to each participant.

[1060] Examples of concrete examples and prompts

[1061] As a concrete example, consider a scenario in which a user plays a song using a smartphone. As the user plays the song, the smartphone camera captures the user's facial expression, and the emotion engine determines that the user is "happy." Based on this result, the system recommends songs that are suitable for "happy." Below is an example of an input prompt sentence for the generative AI model.

[1062] Prompt statement:

[1063] "What songs would you recommend to a user who is in a positive mood? Based on the results of the emotion analysis, the user has been determined to be "happy." Please recommend some songs based on that determination."

[1064] In this way, the present invention provides a personalized music experience that responds to the user's emotions and realizes transparency and fairness in revenue sharing.

[1065] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1066] Step 1:

[1067] A user launches a smartphone app and creates an account. The input is the user's basic information (name, email address, etc.), and the output is a notification that the user account has been created. In this step, the user information is registered in the database.

[1068] Step 2:

[1069] A singer registers the rights to use their voice on the platform. The input is the singer's sample voice data and usage policy (e.g., only pop genres can be used). The output is a notification that registration is complete. The server saves the input data in the database and adds it to the policy management table.

[1070] Step 3:

[1071] A creator creates a song and uploads it to the server. The input is the song data and metadata (title, genre, etc.), and the output is a notification that the upload is complete. The server stores the song data in a database and checks the usage policy.

[1072] Step 4:

[1073] A listener uses the app to play a song. The input is the listener's song selection, and the output is the song streaming. During this step, the smartphone's camera and microphone collect the user's emotional data (facial expressions, voice).

[1074] Step 5:

[1075] The device's emotion engine analyzes the collected emotion data. The input is the user's facial expression image and voice data, and the output is the emotion analysis result (e.g., happy, sad). Here, OpenCV and DeepFace are used for image analysis, and librosa is used to analyze voice data.

[1076] Step 6:

[1077] Based on the sentiment analysis results, the server recommends songs. The input is the sentiment analysis results and the listener's playback history, and the output is a list of recommended songs. In this step, the generative AI model uses the collected data to select songs that correspond to the prompt.

[1078] Step 7:

[1079] The listener plays the recommended song. The input is a selection from a list of recommended songs, and the output is a streaming playback of the song. The playback data is sent to the server and used for revenue sharing.

[1080] Step 8:

[1081] The server calculates revenue based on the playback data and emotion data. The inputs are the number of plays, song usage, emotion data, etc., and the output is the revenue sharing result. In this step, the revenue sharing system calculates the revenue for each party and records it in the database.

[1082] Step 9:

[1083] The revenue distribution result is notified to each participant. The input is the revenue distribution result, and the output is a notification message. The server sends this notification to each participant's terminal, realizing fair revenue distribution.

[1084] In this way, the system analyzes users' emotions in real time, recommends songs based on the results, and ensures fair revenue distribution.

[1085] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1086] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1087] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1088] [Third embodiment]

[1089] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1090] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1091] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1092] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1093] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1094] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1095] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1096] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1097] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1098] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1099] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1100] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1101] This invention is a next-generation music streaming system that properly manages the rights to use singers' voices and achieves fair revenue distribution. The system includes a platform for managing voice usage rights, a function for distributing revenue based on music playback, a function for managing and checking singers' voice policies, and a monitoring function for preventing unauthorized use.

[1102] System Overview

[1103] This system consists of three main components: a server and user terminals (singers, creators, and listeners). Each component has a specific function and operates by communicating with each other.

[1104] 1. Server Functions

[1105] The server is responsible for the following main functions: managing the rights to use singers' voices, storing and managing song data, calculating revenue distribution and notifying the results. Specifically, it performs the following processes:

[1106] Accepting user registration: Provide an interface for singers to register information regarding the use of their voice.

[1107] Storing music data: Music data and related metadata created by creators are stored in a database.

[1108] Usage policy checks: When a song is uploaded, we check that it complies with the usage policy set by the artist.

[1109] Revenue sharing calculation: Calculate revenue based on playback data and distribute it to each party (singer, creator, original author).

[1110] Unauthorized use monitoring: Monitor singers' voices for unauthorized use.

[1111] 2. User device (singer) functions

[1112] Singers register their voice usage rights on the platform, set policies, and receive revenue sharing results through the system. Specifically, they perform the following operations:

[1113] Register voice usage rights: Upload sample voices and set usage policies.

[1114] Check the revenue sharing results: Receive notifications from the server and check the revenue sharing results.

[1115] 3. User Device (Creator) Functions

[1116] Creators create and upload songs using singers' voices. When uploading, they are required to ensure that the usage policy is adhered to. Specifically, they perform the following operations:

[1117] Voice selection: Select the singer's voice you want to use on the platform.

[1118] Uploading music: Upload the created music data and related metadata to the server.

[1119] 4. User terminal (listener) functions

[1120] Listeners search for and play songs on the platform by:

[1121] Song search: Enter keywords to search for songs.

[1122] Play Song: Select a song from the search results and stream it.

[1123] Specific examples

[1124] One day, singer A registers the rights to use his or her voice on a platform and sets a policy that it can only be used in the pop genre. In response, creator B creates a pop song using singer A's voice and uploads it to the platform. The server checks whether the uploaded song complies with singer A's policy and, if there are no problems, stores it in the database. Later, listener C searches for the pop song on the platform and plays it. Based on the playback data, the server calculates revenue and distributes it to singer A, creator B, and the author of the original song. Singer A and creator B are notified of the revenue distribution results.

[1125] In this way, the present invention provides functions for protecting the rights of singers' voices and achieving fair distribution of revenue, thereby building a music streaming system that can be used safely by all parties involved.

[1126] The processing flow will be explained below.

[1127] 1. User Registration and Contract Process

[1128] Step 1:

[1129] The singer (user) accesses the registration form.

[1130] The singer accesses the registration form provided on the system's web page.

[1131] Step 2:

[1132] The singer (user) enters their name, profile, voice sample, and desired usage policy.

[1133] Singers fill out a form with their name, profile information, a sample audio file of their voice, and voice usage policies (e.g., allowed genres, etc.).

[1134] Step 3:

[1135] The device sends the input to the server.

[1136] When the submit button on the registration form is clicked, the terminal transmits the information entered by the singer to the server.

[1137] Step 4:

[1138] The server stores the received information in a database.

[1139] The server stores the received registration information in a database and generates a unique ID for the singer.

[1140] Step 5:

[1141] The server will send a confirmation email to the singer.

[1142] The server will send a confirmation email to the singer's email address to notify them of the completion of registration.

[1143] 2. Creators' music production and release

[1144] Step 1:

[1145] The creator (user) logs in and accesses the dashboard.

[1146] Creators can log in by entering their account information and use the dashboard.

[1147] Step 2:

[1148] The creator (user) selects the singer's voice they want to use.

[1149] Creators select the voice they want to use from a list of singers provided on their dashboard.

[1150] Step 3:

[1151] The device sends the selection to the server.

[1152] Once the creator selects a voice, the content is sent to the server.

[1153] Step 4:

[1154] The server checks the policy of the selected singer's voice.

[1155] The server verifies that the selected singer's voice complies with the specified usage policy.

[1156] Step 5:

[1157] The device downloads a voice sample.

[1158] Once the server has completed the usage policy check, a voice sample is downloaded to the creator's device.

[1159] Step 6:

[1160] Creators (users) create music using AI tools.

[1161] Creators use AI tools (e.g., DAWs and AI voice plug-ins) to create music.

[1162] Step 7:

[1163] Creators (users) upload their completed songs to the platform.

[1164] Enter your completed music file and its metadata (e.g. title, genre, usage policy, etc.) into the upload form.

[1165] Step 8:

[1166] The device sends the uploaded content to the server.

[1167] Clicking the upload button sends the music file and metadata to the server.

[1168] Step 9:

[1169] The server checks the metadata and stores it in a database.

[1170] The server checks the metadata of the song to ensure it does not violate usage policies, and if there are no problems, stores the song data in a database.

[1171] 3. Listeners play music

[1172] Step 1:

[1173] The listener (user) logs in and searches for songs.

[1174] Listeners log in by entering their account information and then enter keywords into the search bar.

[1175] Step 2:

[1176] The device sends the search keywords to the server.

[1177] When you click the search button, the search keywords are sent to the server.

[1178] Step 3:

[1179] The server searches the database for the corresponding song and returns the results to the terminal.

[1180] The server searches the database for songs that match the keywords and sends the results to the listener's device.

[1181] Step 4:

[1182] The listener (user) selects the song to play.

[1183] Listeners select the song they want to play from the search results and click the play button.

[1184] Step 5:

[1185] The server begins streaming the selected song.

[1186] The server streams the selected songs and sends them to the listener's device.

[1187] 4. Revenue Sharing

[1188] Step 1:

[1189] The server periodically obtains the playback data.

[1190] The server periodically obtains the playback data from the playback history table.

[1191] Step 2:

[1192] The server performs the revenue calculation.

[1193] The server calculates the total revenue for each song based on the number of plays.

[1194] Step 3:

[1195] The server generates data for distributing revenue to each participant (author, singer, creator).

[1196] The server generates data for distributing the profits to each participant and calculates them based on the distribution ratio.

[1197] Step 4:

[1198] The server stores the revenue sharing results in a database and sends notifications to each participant.

[1199] The server stores the distribution results in a database and sends notifications to participants via email or other means.

[1200] The above is a specific processing flow for implementing the present invention, which realizes the protection of the singer's voice rights and the fair distribution of profits.

[1201] Example 1

[1202] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1203] In today's music streaming services, the rights of artists and other creators are often not properly protected. Furthermore, revenue distribution is often opaque and not fair. Furthermore, unauthorized use of artists' voices is common and often goes undetected. To solve these problems, a system is needed that properly manages the rights to use artists' voices, distributes revenue fairly, and prevents unauthorized use.

[1204] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1205] In this invention, the server includes a means for producing and releasing music on a platform for managing the rights to use singers' voices, a means for appropriately distributing revenue based on music playback, a means for managing and checking policies for singers' voices, a monitoring means for preventing unauthorized use, a server means for providing an interface for registering information regarding the use of users' voices, a database means for storing music sheet data and related metadata, and a server means for automatically performing policy checks, thereby enabling the protection of the rights of singers and creators, the fair distribution of revenue, and the prevention of unauthorized use.

[1206] A "singer" is an individual or group of individuals who provides their voice in a musical composition.

[1207] "Usage rights" are legal authorizations that permit the use of a particular vocal or musical work.

[1208] "Platform" refers to an online service or software that enables users to publish their content and interact with other users.

[1209] "Music" is an art form created by combining sounds and voices, and is a means of expression through the ears.

[1210] "Playback" is the act of listening to music or audio using a device.

[1211] "Revenue" is the monetary benefit derived from the provision of a service or consumption of a work.

[1212] A "policy" is a rule or guideline that prescribes a particular purpose or behavior.

[1213] "Monitoring" is the act of continually checking a particular activity or situation and responding when a problem arises.

[1214] An "interface" is the method or means by which a user interacts with a system or software.

[1215] A "server" is a computer system that provides data over a network.

[1216] A "database" is a system for efficiently storing, managing, and searching large amounts of data.

[1217] "Metadata" refers to data that contains information about specific data, such as the title of a song or the name of an artist.

[1218] A "policy check" is a process that verifies whether certain conditions are met based on a registered policy.

[1219] "Revenue sharing" is the act of distributing earned revenue fairly among multiple rights holders and related parties.

[1220] This is a next-generation music streaming system that properly manages the rights to use singers' voices and ensures fair revenue distribution. The system consists of three main components: a server and user terminals (singers, creators, and listeners).

[1221] Server Roles and Functions

[1222] The server is responsible for the central functions of the system. It performs the following main processes:

[1223] Accepting user registrations:

[1224] The server provides an interface for singers to register information about the use of their voice. This interface is a web-based application that singers can access from their terminals.

[1225] Save song data:

[1226] The music data and related metadata created by creators are stored in a database, which uses a common database management system such as MySQL or PostgreSQL.

[1227] Usage policy check:

[1228] The server automatically checks whether the uploaded song complies with the artist's usage policy, using a specific algorithm.

[1229] Revenue Share Calculation:

[1230] The server calculates revenue based on song playback data and distributes it to each party (singer, creator, original author), taking into account the number of plays, playback time, advertising revenue, etc.

[1231] Abuse Monitoring:

[1232] The server monitors the singer's voice for fraudulent use, which is achieved by using an AI model to check for voice pattern matches.

[1233] Roles and functions of user devices

[1234] The role of the singer

[1235] Singers will register the rights to use their voice and receive notification of the revenue sharing results. Specifically, they will:

[1236] Voice usage rights registration:

[1237] Singers upload their own sample voices from their devices and set usage policies.

[1238] Check your revenue sharing results:

[1239] The singer receives a notification from the server and checks the revenue sharing results.

[1240] The role of creators

[1241] Creators create and upload songs using singers' voices. Specifically, they do the following:

[1242] Voice Selection:

[1243] Creators choose the singer's voice they want to use on the platform.

[1244] Upload your song:

[1245] Creators upload the music data and related metadata they have created to the server, where they are checked to ensure that the usage policy is being followed.

[1246] The role of the listener

[1247] Listeners search for and play songs on the platform by:

[1248] Song search:

[1249] Listeners can input keywords to search for songs.

[1250] Play music:

[1251] Listeners can select songs from the search results and stream them.

[1252] Specific examples

[1253] One day, singer A registers the rights to use his or her voice on a platform and sets a policy that it can only be used in the pop genre. In response, creator B creates a pop song using singer A's voice and uploads it to the platform. The server checks whether the uploaded song complies with singer A's policy and, if there are no problems, stores it in the database. Later, listener C searches for the pop song on the platform and plays it. Based on the playback data, the server calculates revenue and distributes it to singer A, creator B, and the author of the original song. Singer A and creator B are notified of the revenue distribution results.

[1254] Example prompt:

[1255] "Please explain the user experience when Singer A sets a policy that allows music to be used only in the pop genre."

[1256] In this way, the invention provides a concrete means to protect singers' vocal rights and ensure fair distribution of revenue, creating a music streaming system that can be used safely by all parties involved.

[1257] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1258] Step 1: User registration and rights registration

[1259] Input: The singer uses the device to enter their name, contact information, sample voice (audio file), and usage policy.

[1260] What happens: A singer accesses the registration form, fills in the required information, uploads a sample voice, sets usage policies, and clicks the submit button.

[1261] Data processing / calculation: The server receives the input data, parses the usage policy and saves it in the database. It also saves the sample voice audio file.

[1262] Output: The server confirms that user registration and rights registration have been completed, and sends a registration completion notification to the singer's terminal.

[1263] Step 2: Choose your creator's voice

[1264] Input: Creators use their device to enter search criteria for the voice they want to use (e.g. genre, singer name).

[1265] How it works: Creators log in to the platform and enter criteria such as genre and artist name into the search form.

[1266] Data processing / calculation: The server queries the database based on the search criteria and returns filtered sample voice information of the corresponding singer.

[1267] Output: As a search result, a list of available singers and sample voice information are displayed on the creator's device.

[1268] Step 3: Upload your music

[1269] Input: Enter the audio file and metadata (e.g. title, genre, description) of the song created by the creator.

[1270] What happens: Creator selects a music file, enters relevant metadata into the form, and clicks the upload button.

[1271] Data processing / calculation: The server receives the uploaded audio file and metadata, checks the integrity of the file, and then verifies whether the song is permitted based on the usage policy.

[1272] Output: If the song data complies with the policy, the server saves the song data in the database and sends an upload completion notification to the creator's device. If it does not comply, it returns an error message.

[1273] Step 4: Search and play songs

[1274] Input: A listener uses a device to enter keywords to search for a song.

[1275] Specific actions: Listeners access the platform, enter keywords into the search form, and click the search button.

[1276] Data processing / calculation: The server queries the database based on the search keywords and generates a list of relevant songs.

[1277] Output: As a search result, a list of matching songs is displayed on the listener's device. When the listener selects a song and clicks the play button, the server sends the streaming data and plays the song.

[1278] Step 5: Calculate revenue share

[1279] Input: The server compiles song playback data (e.g., number of plays, playback time, advertising revenue).

[1280] What it does: The server records playback data every time a song is played on the platform. It aggregates the playback data over specific time intervals.

[1281] Data processing / calculation: The server uses a revenue sharing algorithm to calculate revenue, taking into account the number of plays, play time, ad revenue, etc.

[1282] Output: The server stores the revenue sharing results in a database and sends notifications to the singer, creator, and original author.

[1283] Step 6: Monitor for fraud

[1284] Input: Periodically scan all song data in the database.

[1285] Specific operation: The server executes a scheduled task at regular intervals to analyze all songs.

[1286] Data processing / calculation: The server uses AI models to detect unregistered usage patterns and unauthorized copying.

[1287] Output: If any unauthorized use is detected, the server will send a warning notice to the singer and the system administrator.

[1288] (Application example 1)

[1289] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1290] With the recent spread of music streaming services, fairness in the rights to use singers' voices and revenue distribution has become increasingly important. However, existing systems have insufficient management of rights to use singers' voices and revenue distribution, resulting in fraudulent use and unfair revenue distribution. Furthermore, there is a need for systems that make it easy for users to search and play music on their smart devices while also providing appropriate rights management. However, there is a lack of systems that combine an interactive, easy-to-use user interface with advanced voice analysis technology.

[1291] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1292] In this invention, the server includes means for producing and releasing music on a platform for managing the rights to use a singer's voice, means for appropriately distributing revenue based on music playback, means for managing and checking policies for the singer's voice, means for monitoring to prevent unauthorized use, means for using smart contracts to manage users' voice rights, means for providing streaming playback of music, means for notifying users of revenue distribution results, and means for providing a user interface that operates on a smartphone or a head-mounted display, thereby ensuring the appropriate use of a singer's voice and fairness in revenue distribution, and further improving the user experience on smart devices.

[1293] The "platform for managing singers' voice usage rights" is an online system that allows singers to register rights to use their voice and manage the scope and conditions of that use.

[1294] "Means for producing and releasing music" refers to a system that allows music creators to create music and publish that music data on the platform.

[1295] "Means for appropriate distribution of revenue" is a function for calculating revenue based on song playback data and distributing it fairly to the parties involved.

[1296] "Means for managing and checking singers' voice policies" refers to a system for checking whether the conditions and scope of voice usage set by singers are being properly observed.

[1297] "Monitoring measures to prevent unauthorized use" refers to a monitoring function to prevent the singer's voice from being used without permission.

[1298] "Means using smart contracts" refers to a mechanism that uses blockchain technology to automate the management of audio rights and revenue distribution, ensuring transparency.

[1299] "Means for providing streaming music playback" refers to a function that allows users to listen to music in real time over the Internet.

[1300] The "means for notifying the profit distribution results" is a function for promptly notifying the parties concerned of the profit calculation results.

[1301] A "user interface that runs on a smartphone or head-mounted display" is an operation screen or interface that allows users to use the system through a device such as a smartphone or VR device.

[1302] The system embodying this invention comprises the following main components: a server, a singer terminal, a creator terminal, and a listener terminal.

[1303] Server Features

[1304] The server provides the following functionality:

[1305] 1. Accepting user registration: The server provides an interface that allows singers, creators, and listeners to create accounts. In this case, AWS Cognito is used to manage account information.

[1306] 2. Managing singers' voice usage rights: Provides a function for singers to register their voice usage rights and set usage policies. Transparency and security are ensured by using smart contracts and blockchain technology (e.g., Ethereum).

[1307] 3. Storage and management of music data: Music data created by creators is stored in a database on the server (e.g., MySQL, PostgreSQL).

[1308] 4. Usage policy check: When a song is uploaded, the server checks whether it complies with the artist's usage policy. It uses voice analysis technology such as Google Cloud Speech-to-Text to scrutinize the audio data.

[1309] 5. Unauthorized use monitoring: Monitor singers' voices for unauthorized use and issue alerts if necessary.

[1310] 6. Providing a music streaming service: Using the Spotify SDK, we provide a fast, high-quality playback experience for listeners to stream music.

[1311] 7. Revenue share calculation: Based on the song playback data, revenue is calculated and distributed fairly to the parties involved. This process is performed using AWS Lambda and DynamoDB.

[1312] 8. Notification of Revenue Distribution Results: The singer, creator, and original author will be notified of the revenue distribution results via email.

[1313] User device functions

[1314] 1. Singer terminal: Singers can register their voice usage rights, manage the policies they have set, and check the revenue distribution results.

[1315] 2. Creator Terminal: Creators can create and upload music and ensure that usage policies are properly adhered to.

[1316] 3. Listener device: Listeners have the ability to search for songs and play them in streaming.

[1317] Specific examples

[1318] For example:

[1319] One day, a singer registers the rights to use her voice on a platform and sets a policy that it can only be used in the pop genre. In response, a creator creates a pop song using the singer's voice and uploads it to the platform. The server checks whether the uploaded song complies with the singer's policy and, if there are no problems, stores it in the database. A listener then searches for and plays the pop song on the platform. Based on the playback data, the server calculates revenue and distributes it to the singer, creator, and the copyright holder of the original song. The singer and creator are notified of the revenue distribution results.

[1320] The following prompt example is also provided:

[1321] Singer: "I only want my voice to be used in songs in the pop genre."

[1322] Creator: "I've created a new pop song and I want to use the singer's voice."

[1323] Listener: "I want to hear the latest pop songs."

[1324] The system ensures the proper use of singers' voices and fair revenue distribution, while also improving the user experience on smart devices.

[1325] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1326] Step 1:

[1327] A singer uses a terminal to register the rights to use their voice on the platform. At this time, the singer uploads a sample of their voice and sets a policy that it can only be used in the pop genre. The server receives the sample voice and policy data as input and registers it in a smart contract using blockchain technology to ensure transparency and security.

[1328] Step 2:

[1329] Creators use their devices to create songs using singers' voices and upload them to the platform. Creators then send the song data to the server based on the policy. The server stores the song data and metadata in a MySQL or PostgreSQL database and uses Google Cloud Speech-to-Text technology to check whether the voice data complies with the policy.

[1330] Step 3:

[1331] The server checks whether the uploaded song complies with the usage policy set by the artist. If it is confirmed, the song data is officially stored in the platform's database. Conversely, if there is a policy violation, the creator is notified and the upload is stopped.

[1332] Step 4:

[1333] Listeners use their devices to search for and stream songs on the platform. Listeners enter keywords, and the server searches the database for songs and outputs the results. The song data is then streamed to the listener's device using the Spotify SDK.

[1334] Step 5:

[1335] The server collects listeners' song play data, recording the number of plays and play time. This data is later used to calculate revenue distribution. The server processes the data in real time using AWS Lambda and DynamoDB to calculate how revenue should be distributed to each party.

[1336] Step 6:

[1337] The server distributes revenue based on the calculation results. The revenue is distributed to the singer, creator, and original song author based on a predetermined revenue distribution ratio. The distribution results are sent to the relevant parties via email or other notification methods.

[1338] Step 7:

[1339] After the server completes all procedures, it displays the revenue distribution results in real time on the user's dashboard, allowing all parties involved to instantly see how the revenue has been distributed.

[1340] Examples:

[1341] Singer: "I only want my voice to be used in songs in the pop genre."

[1342] Creator: "I've created a new pop song and I want to use the singer's voice."

[1343] Listener: "I want to hear the latest pop songs."

[1344] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1345] This is a next-generation music streaming system that recognizes users' emotions to improve the music experience and distributes revenue fairly, while properly managing the rights to use singers' voices. In addition to the functions of conventional music streaming systems, this system uses an emotion engine to recognize users' emotions and adjusts song recommendations and revenue distribution based on these.

[1346] System Overview

[1347] This system includes three main components: a server, user terminals (singers, creators, and listeners), and an emotion engine. These components have specific functions and operate while communicating with each other.

[1348] 1. Server Functions

[1349] The server manages the rights to use singers' voices, stores and manages song data, calculates revenue distribution and notifies the results, and processes emotion data using an emotion engine. Specifically, it performs the following processes:

[1350] Accepting user registration: Provide an interface for singers to register information regarding the use of their voice.

[1351] Storing music data: Music data and related metadata created by creators are stored in a database.

[1352] Usage policy checks: When a song is uploaded, we check that it complies with the usage policy set by the artist.

[1353] Revenue sharing calculation: Calculate revenue based on playback data and distribute it to each party (singer, creator, original author).

[1354] Unauthorized use monitoring: Monitor singers' voices for unauthorized use.

[1355] Emotional Data Analysis: Emotional data generated by the emotion engine is collected and analyzed to improve user experience and adjust revenue sharing.

[1356] 2. User device (singer) functions

[1357] Singers register their voice usage rights on the platform, set policies, and receive revenue sharing results through the system. Specifically, they perform the following operations:

[1358] Register voice usage rights: Upload sample voices and set usage policies.

[1359] Check the revenue sharing results: Receive notifications from the server and check the revenue sharing results.

[1360] 3. User Device (Creator) Functions

[1361] Creators create and upload songs using singers' voices. When uploading, they are required to ensure that the usage policy is adhered to. Specifically, they perform the following operations:

[1362] Voice selection: Select the singer's voice you want to use on the platform.

[1363] Uploading music: Upload the created music data and related metadata to the server.

[1364] 4. User terminal (listener) functions

[1365] Listeners search for and play songs on the platform, and the system recognizes their emotions and recommends appropriate songs. Specifically, they perform the following actions:

[1366] Song search: Enter keywords to search for songs.

[1367] Play Song: Select a song from the search results and stream it.

[1368] Emotion recognition: The emotion engine analyzes the user's emotions using the device's camera and voice input. Based on the analysis results, it recommends songs that suit the user.

[1369] Specific examples

[1370] One day, Singer A registers the rights to use his or her voice on the platform and sets a policy that it can only be used in the pop genre. In response, Creator B creates a pop song using Singer A's voice and uploads it to the platform. The server checks whether the uploaded song complies with Singer A's policy and, if there are no problems, stores it in the database. Listener C then searches for and plays a pop song on the platform. The emotion engine analyzes Listener C's facial expression and recognizes that he or she is relaxed, so songs in a similar pop genre are recommended.

[1371] Based on the playback data, the server calculates revenue and distributes it to singer A, creator B, and the copyright holder of the original song. The revenue distribution results are notified to singer A and creator B. In addition, revenue adjustments are also performed based on emotional data, realizing optimal revenue distribution according to the user's emotions.

[1372] In this way, the present invention not only protects the rights of singers' voices and ensures fair distribution of revenue, but also builds a system that provides a more personalized music experience through a recommendation function based on user emotions.

[1373] The processing flow will be explained below.

[1374] 1. User Registration and Contract Process

[1375] Step 1:

[1376] The singer (user) accesses the registration form.

[1377] The singer accesses the registration form provided on the system's web page.

[1378] Step 2:

[1379] The singer (user) enters their name, profile, voice sample, and desired usage policy.

[1380] Singers fill out a form with their name, profile information, a sample audio file of their voice, and usage policies (e.g., allowed genres, etc.).

[1381] Step 3:

[1382] The device sends the input to the server.

[1383] When the registration form submit button is clicked, the terminal sends the entered information to the server.

[1384] Step 4:

[1385] The server stores the received information in a database.

[1386] The server stores the received information in a database and generates a unique ID for the singer.

[1387] Step 5:

[1388] The server will send a confirmation email to the singer.

[1389] The server will send a confirmation email to the singer's email address to notify them of the completion of registration.

[1390] 2. Creators' music production and release

[1391] Step 1:

[1392] The creator (user) logs in and accesses the dashboard.

[1393] Creators log in on the system's web page and access their dashboard.

[1394] Step 2:

[1395] The creator (user) selects the singer's voice they want to use.

[1396] Creators select the voice they want to use from a list of singers provided on their dashboard.

[1397] Step 3:

[1398] The device sends the selection to the server.

[1399] Once the creator selects a voice, that information is sent to a server.

[1400] Step 4:

[1401] The server checks the policy of the selected singer's voice.

[1402] The server checks whether the selected singer's voice complies with the established usage policy.

[1403] Step 5:

[1404] The device downloads a voice sample.

[1405] Once the policy check is complete, the server will have the device download a voice sample file.

[1406] Step 6:

[1407] Creators (users) create music using AI tools.

[1408] Creators use AI tools to create and complete songs.

[1409] Step 7:

[1410] Creators (users) upload their completed songs to the platform.

[1411] Enter the completed music file and its metadata through the upload form.

[1412] Step 8:

[1413] The device sends the uploaded content to the server.

[1414] Clicking the upload button sends the music file and metadata to the server.

[1415] Step 9:

[1416] The server checks the metadata and stores it in a database.

[1417] The server checks the uploaded metadata, verifies that it does not violate any policies, and then stores it in the database.

[1418] 3. Listeners play music

[1419] Step 1:

[1420] The listener (user) logs in and searches for songs.

[1421] Listeners log into the system and enter keywords into the search bar.

[1422] Step 2:

[1423] The device sends the search keywords to the server.

[1424] When you click the search button, the entered keywords are sent to the server.

[1425] Step 3:

[1426] The server searches the database for the corresponding song and returns the results to the terminal.

[1427] The server searches the database for songs that match the keywords and sends the results to the listener's device.

[1428] Step 4:

[1429] The listener (user) selects the song to play.

[1430] Listeners select the song they want to play from the search results and click the play button.

[1431] Step 5:

[1432] The server begins streaming the selected song.

[1433] The server streams the selected songs and sends them to the listener's device.

[1434] Step 6:

[1435] The device recognizes the user's emotions using an emotion engine.

[1436] During playback, the device uses an emotion engine to analyze emotions from the user's facial expressions and voice.

[1437] Step 7:

[1438] The server receives the emotional data and recommends appropriate songs.

[1439] It receives data from the emotion engine and recommends songs that suit the user.

[1440] 4. Revenue Sharing

[1441] Step 1:

[1442] The server periodically obtains the playback data.

[1443] The server periodically obtains the playback data from the playback history table.

[1444] Step 2:

[1445] The server performs the revenue calculation.

[1446] The server calculates the total revenue for each song based on the number of plays and sentiment data.

[1447] Step 3:

[1448] The server generates data for distributing revenue to each participant (author, singer, creator).

[1449] The server generates data for distributing the profits to each participant based on the distribution ratio.

[1450] Step 4:

[1451] The server stores the revenue sharing results in a database and sends notifications to each participant.

[1452] The server stores the distribution results in a database and notifies each party of the revenue results via email or other means.

[1453] In this way, the present invention is a system that not only protects the rights of singers' voices and ensures fair distribution of revenue, but also provides a more personalized music experience through a recommendation function based on user emotions.

[1454] Example 2

[1455] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1456] Conventional music streaming systems have problems such as difficulty in properly managing singers' audio rights and fair revenue distribution. Furthermore, they lack a function for recommending songs based on user emotions. Therefore, a system to improve the user experience is needed.

[1457] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for producing and releasing music on a platform for managing singers' audio rights, a means for appropriately distributing revenue based on music playback, a means for managing and checking audio usage policies, a monitoring means for preventing unauthorized use, a means for recognizing user emotions and recommending songs based on the emotions, and a means for adjusting revenue distribution based on emotion data. This enables appropriate management of singers' audio rights, fair revenue distribution, and song recommendations based on user emotions.

[1458] "Singer's audio rights" refers to the right of a singer to commercially exploit his or her voice.

[1459] "Platform" refers to a system that enables users such as singers, creators, and listeners to create, play, and manage music via the Internet.

[1460] "Means for appropriately distributing revenue based on music plays" refers to algorithms or functions that distribute revenue to each party based on playback data.

[1461] "Audio usage policy" refers to the rules and standards that a singer sets for the conditions under which their voice is used and that are managed and checked based on those conditions.

[1462] "Monitoring measures to prevent unauthorized use" refers to monitoring systems and algorithms that check whether a singer's voice is being used fraudulently.

[1463] "Means for recognizing user emotions and recommending songs based on that" refers to a system that analyzes user emotions using technologies such as emotion engines and recommends songs based on the results.

[1464] "Means for adjusting revenue sharing based on emotional data" refers to an algorithm or function that adjusts the revenue sharing rate by taking into account the emotional data of users.

[1465] "Music data" refers to music files and associated metadata, such as title, artist information, genre, and other information.

[1466] "Database" refers to a system for managing stored audio rights information, music data, usage policies, playback data, user emotional data, and the like.

[1467] "Revenue Share Results" means the revenue share results for each party calculated based on playback data and other data.

[1468] MODE FOR CARRYING OUT THE INVENTION

[1469] This invention is a next-generation music streaming system that appropriately manages singers' audio rights, recognizes users' emotions to improve the music experience, and distributes revenue fairly. The main components of this system include a server and user terminals (singers, creators, and listeners), each with specific functions and operating while communicating with each other.

[1470] Server Features

[1471] The server is configured to perform the following main tasks:

[1472] Accepting user registration: Provides an interface for singers to register the rights to use their voice. Singers can upload their voice samples and enter genre settings and usage policies. The server receives this data and stores it in a database.

[1473] Music data storage: Creators upload the music data and related metadata they have created, which is then stored and managed on the server.

[1474] Usage policy check: Checks whether the uploaded song complies with the usage policy set by the artist. If it does not comply with the policy, the upload will be rejected and an error message will be sent to the creator.

[1475] Revenue sharing calculation: Calculate revenue based on playback data. Notify each party (singer, creator, original author) of the revenue sharing results.

[1476] Unauthorized use monitoring: We monitor whether singers' voices are being used illegally. If any unauthorized use is detected, we will send a warning message to the singer and take appropriate measures.

[1477] Emotion data analysis: The server analyzes the emotion data collected by the emotion engine, and uses this data to improve user experience and adjust revenue sharing.

[1478] As specific examples of hardware and software, the server uses an Amazon Web Services (AWS) EC2 instance, the database uses AWS RDS (Relational Database Service), and the emotion engine uses the Emotion API from Azure Cognitive Services.

[1479] User device (singer) functions

[1480] On the singer's device, do the following:

[1481] Registering vocal usage rights: Singers access the system, upload their vocal samples, set usage policies, and specify specific genres and uses.

[1482] Check the revenue sharing results: Receive notifications from the server and check the revenue sharing results.

[1483] User device (creator) functions

[1484] On the creator's device, do the following:

[1485] Voice Selection: Creators access the system and select the singer's voice they wish to use. They then review the usage policy for the selected voice.

[1486] Uploading music: Upload the created music data and related metadata to the server.

[1487] User terminal (listener) functions

[1488] On the listener's device, do the following:

[1489] Song search: Listeners access the system and enter keywords to search for songs.

[1490] Song Playback: The listener selects a song and streams it. During playback, the emotion engine analyzes the listener's emotions.

[1491] Emotion Recognition: The system collects the listener's emotions from the device's camera and voice input. Based on the analysis results of the emotion engine, the system recommends songs that suit the user.

[1492] Specific examples

[1493] One day, Singer A registers the rights to use his or her voice on the platform, allowing it to be used in the pop genre. Creator B then creates a pop song using that voice and uploads it to the server. The server checks that the song complies with Singer A's usage policy and then saves it.

[1494] Listener C accesses the system, searches for a pop song, and plays it. The emotion engine analyzes Listener C's facial expression and recognizes that he or she is in a relaxed state. The system then recommends a pop song suitable for relaxation.

[1495] Finally, the server calculates revenue based on the playback data of listener C and distributes the revenue to singer A, creator B, and the original author. The revenue distribution results are sent to singer A and creator B via a notification function.

[1496] Prompt sentence for generative AI model

[1497] An example of a prompt to enter is: "When a user searches for a pop song they want to listen to in a relaxing state, what kind of songs should be recommended? How can you recommend them based on the analysis results of the emotion engine?"

[1498] In this way, the present invention realizes a system that provides a more personalized music experience by properly managing singers' audio rights, fair revenue distribution, and recommending songs based on user emotions.

[1499] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1500] The flow of this system's program processing

[1501] Step 1: Accepting user registration

[1502] Input: Singer enters their voice sample, usage policy and genre settings.

[1503] Processing: The server uploads the singer's vocal sample and stores the configured usage policy and genre information in a database.

[1504] Output: The singer's voice rights information is stored in a database.

[1505] Specific behavior:

[1506] Singers access the system using a terminal and upload a sample of their voice into a form.

[1507] The server receives this sample and stores it in a database along with the usage policy.

[1508] A registration completion message is sent to the singer.

[1509] Step 2: Save the song data

[1510] Input: Creators upload the music data and related metadata they have created to the server.

[1511] Processing: The server stores the uploaded song data and metadata in a database.

[1512] Output: Song data and metadata are stored in a database.

[1513] Specific behavior:

[1514] Creators use their devices to upload the completed music data to the system.

[1515] The server receives the song data and related metadata and stores it in a database.

[1516] Notify creator of upload completion message.

[1517] Step 3: Check the usage policy

[1518] Input: Song data uploaded by the creator and usage policy set by the singer.

[1519] Processing: The server checks whether the uploaded music data complies with the singer's usage policy.

[1520] Output: If the song data complies with the policy, it is saved in the database; if it does not, an error message is sent to the creator.

[1521] Specific behavior:

[1522] The server obtains the metadata of the song data and the singer's usage policy.

[1523] If the policy is violated, the song data will be refused upload and an error message will be sent to the creator.

[1524] If the policy is met, the music data is stored in the database.

[1525] Step 4: Music playback and emotion recognition

[1526] Input: Commands for listeners to search for and play songs, as well as camera footage and audio input from listeners.

[1527] Processing: The server streams the music and the emotion engine analyzes the listener's emotions.

[1528] Output: Songs are recommended in real time based on the listener's sentiment analysis.

[1529] Specific behavior:

[1530] Listeners can search for songs by entering keywords and start playing them.

[1531] The server streams the music data to the listener's device.

[1532] The emotion engine analyzes the listener's camera footage and audio input data to quantify their emotional state, such as relaxation or excitement.

[1533] Based on the analysis results, the server recommends the most suitable songs to the listener.

[1534] Step 5: Calculate revenue share

[1535] Input: Song playback and emotion data.

[1536] Processing: The server calculates revenue based on playback data and adjusts it based on sentiment data.

[1537] Output: The revenue distribution results are notified to each party (singer, creator, original author).

[1538] Specific behavior:

[1539] The server collects song playback data and tallys up the number of times and duration of playback.

[1540] A revenue sharing adjustment algorithm based on sentiment data is applied to calculate the final revenue sharing amount.

[1541] Notify each party of the revenue sharing results.

[1542] Step 6: Monitor for fraud

[1543] Input: Audio data registered on the server and newly uploaded audio data.

[1544] Processing: The server compares and monitors the audio data using an abuse monitoring algorithm.

[1545] Output: If any unauthorized use is detected, send a warning message to the relevant parties.

[1546] Specific behavior:

[1547] The server routinely monitors newly uploaded audio data.

[1548] The system compares registered voice data with new data and applies an algorithm to detect fraudulent use.

[1549] If unauthorized use is discovered, a warning message will be sent to the relevant singers and creators, urging them to take action.

[1550] In this way, the system of the present invention processes data at each step based on the user's actions to provide a well-curated and personalized music experience.

[1551] (Application example 2)

[1552] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1553] Conventional music streaming systems focused on managing the rights to use singers' voices and distributing revenue, but this made it difficult to provide personalized music experiences, such as song recommendations based on user emotions. There was also a need for improved transparency and fairness in revenue distribution. In addition, there was a need to streamline the revenue distribution process while preventing unauthorized use.

[1554] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1555] In this invention, the server includes a means for producing and releasing music on a platform for managing the rights to use singers' voices, a means for appropriately distributing revenue based on music playback, a means for managing and checking singers' voice policies, a monitoring means for preventing unauthorized use, a means including an emotion engine for analyzing users' emotions in real time, and a means for recommending songs based on the emotion analysis. This allows users to receive optimal song recommendations based on their current emotions, resulting in a more personalized music experience. It also improves the transparency and fairness of revenue distribution, enabling efficient revenue distribution while preventing unauthorized use.

[1556] "Right to use the singer's voice" means the singer's right to use his or her voice for a specific use or purpose, which is used under a controlled agreement.

[1557] "Means for producing and releasing music on the platform" refers to a set of functions and processes that allow singers and creators to produce music and release it through online services.

[1558] "Appropriate revenue distribution measures" refers to methods and systems for distributing revenue fairly and transparently to the parties involved based on the number of times the music is played or other metrics.

[1559] "Means for managing and checking the singer's voice policy" refers to a method or system for checking and managing whether the terms and rules of use set by the singer for his or her voice are being observed.

[1560] "Monitoring measures to prevent unauthorized use" refers to systems that monitor and detect violations to ensure that an artist's voice or music is not used in unauthorized ways or places.

[1561] An "emotion engine" refers to technology that analyzes a user's facial expressions, tone of voice, etc. to detect their emotional state in real time.

[1562] "Means for recommending music" refers to methods or systems that suggest music that matches the user's mood at the time based on the results of an analysis of the user's emotions, etc.

[1563] "Means for storing and managing generated music data" refers to a system that safely stores music data created by creators in digital format and provides management and access as needed.

[1564] "Means for notifying each participant of the results of revenue distribution" refers to a method or system for promptly notifying the participants of the calculated results of revenue distribution.

[1565] MODE FOR CARRYING OUT THE INVENTION

[1566] This invention provides a music streaming system that analyzes a user's emotions in real time to improve the music experience while properly managing the rights to use a singer's voice. Specific embodiments of the invention will now be described.

[1567] System configuration

[1568] The system includes the following main components:

[1569] 1. Server:

[1570] Singer voice usage rights management system: Provides an interface for singers to set and register their own voice usage policies.

[1571] Music data storage and management system: Music data and related metadata created by creators are stored in a database.

[1572] Revenue sharing system: Revenue is calculated based on music playback data and distributed to each party.

[1573] Emotion engine: Analyzes user emotions in real time and uses that data to recommend songs and influence revenue sharing.

[1574] Unauthorized use monitoring system: Monitors whether a singer's voice is being used illegally.

[1575] 2. User Device:

[1576] Singer's device: Register your voice usage rights on the platform and check the revenue sharing results.

[1577] Creator's device: Create and upload songs using the singer's voice.

[1578] Listener's device: In addition to playing music, the device's camera and microphone are used to collect and analyze emotional data.

[1579] Hardware and software used

[1580] Hardware:

[1581] Smartphone (camera, microphone)

[1582] Servers (high-performance processors, storage)

[1583] software:

[1584] Python

[1585] OpenCV (image analysis)

[1586] DeepFace (emotional analysis)

[1587] librosa (audio analysis)

[1588] requests (API communication)

[1589] Program processing explanation

[1590] The server first provides a platform for managing singers' voice usage rights and allows singers to set their own voice usage policies. Creators use this platform to create and upload songs, and the song data is stored on the server. The system checks whether songs comply with the singer's set policies when uploaded and also monitors them to prevent unauthorized use.

[1591] When a listener plays a song, the user device uses the smartphone's camera and microphone to collect emotional data from the user. The collected data is analyzed in real time using an emotion engine. Based on the results of this analysis, songs that fit the listener's mood at that time are recommended.

[1592] Finally, revenue will be calculated based on playback data and sentiment analysis data and distributed appropriately to singers, creators, and other stakeholders. The revenue distribution results will be notified to each participant.

[1593] Examples of concrete examples and prompts

[1594] As a concrete example, consider a scenario in which a user plays a song using a smartphone. As the user plays the song, the smartphone camera captures the user's facial expression, and the emotion engine determines that the user is "happy." Based on this result, the system recommends songs that are suitable for "happy." Below is an example of an input prompt sentence for the generative AI model.

[1595] Prompt statement:

[1596] "What songs would you recommend to a user who is in a positive mood? Based on the results of the emotion analysis, the user has been determined to be "happy." Please recommend some songs based on that determination."

[1597] In this way, the present invention provides a personalized music experience that responds to the user's emotions and realizes transparency and fairness in revenue sharing.

[1598] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1599] Step 1:

[1600] A user launches a smartphone app and creates an account. The input is the user's basic information (name, email address, etc.), and the output is a notification that the user account has been created. In this step, the user information is registered in the database.

[1601] Step 2:

[1602] A singer registers the rights to use their voice on the platform. The input is the singer's sample voice data and usage policy (e.g., only pop genres can be used). The output is a notification that registration is complete. The server saves the input data in the database and adds it to the policy management table.

[1603] Step 3:

[1604] A creator creates a song and uploads it to the server. The input is the song data and metadata (title, genre, etc.), and the output is a notification that the upload is complete. The server stores the song data in a database and checks the usage policy.

[1605] Step 4:

[1606] A listener uses the app to play a song. The input is the listener's song selection, and the output is the song streaming. During this step, the smartphone's camera and microphone collect the user's emotional data (facial expressions, voice).

[1607] Step 5:

[1608] The device's emotion engine analyzes the collected emotion data. The input is the user's facial expression image and voice data, and the output is the emotion analysis result (e.g., happy, sad). Here, OpenCV and DeepFace are used for image analysis, and librosa is used to analyze voice data.

[1609] Step 6:

[1610] Based on the sentiment analysis results, the server recommends songs. The input is the sentiment analysis results and the listener's playback history, and the output is a list of recommended songs. In this step, the generative AI model uses the collected data to select songs that correspond to the prompt.

[1611] Step 7:

[1612] The listener plays the recommended song. The input is a selection from a list of recommended songs, and the output is a streaming playback of the song. The playback data is sent to the server and used for revenue sharing.

[1613] Step 8:

[1614] The server calculates revenue based on the playback data and emotion data. The inputs are the number of plays, song usage, emotion data, etc., and the output is the revenue sharing result. In this step, the revenue sharing system calculates the revenue for each party and records it in the database.

[1615] Step 9:

[1616] The revenue distribution result is notified to each participant. The input is the revenue distribution result, and the output is a notification message. The server sends this notification to each participant's terminal, realizing fair revenue distribution.

[1617] In this way, the system analyzes users' emotions in real time, recommends songs based on the results, and ensures fair revenue distribution.

[1618] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1619] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1620] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1621] [Fourth embodiment]

[1622] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1623] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1624] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1625] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1626] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1627] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1628] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1629] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1630] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1631] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1632] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1633] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1634] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1635] This invention is a next-generation music streaming system that properly manages the rights to use singers' voices and achieves fair revenue distribution. The system includes a platform for managing voice usage rights, a function for distributing revenue based on music playback, a function for managing and checking singers' voice policies, and a monitoring function for preventing unauthorized use.

[1636] System Overview

[1637] This system consists of three main components: a server and user terminals (singers, creators, and listeners). Each component has a specific function and operates by communicating with each other.

[1638] 1. Server Functions

[1639] The server is responsible for the following main functions: managing the rights to use singers' voices, storing and managing song data, calculating revenue distribution and notifying the results. Specifically, it performs the following processes:

[1640] Accepting user registration: Provide an interface for singers to register information regarding the use of their voice.

[1641] Storing music data: Music data and related metadata created by creators are stored in a database.

[1642] Usage policy checks: When a song is uploaded, we check that it complies with the usage policy set by the artist.

[1643] Revenue sharing calculation: Calculate revenue based on playback data and distribute it to each party (singer, creator, original author).

[1644] Unauthorized use monitoring: Monitor singers' voices for unauthorized use.

[1645] 2. User device (singer) functions

[1646] Singers register their voice usage rights on the platform, set policies, and receive revenue sharing results through the system. Specifically, they perform the following operations:

[1647] Register voice usage rights: Upload sample voices and set usage policies.

[1648] Check the revenue sharing results: Receive notifications from the server and check the revenue sharing results.

[1649] 3. User Device (Creator) Functions

[1650] Creators create and upload songs using singers' voices. When uploading, they are required to ensure that the usage policy is adhered to. Specifically, they perform the following operations:

[1651] Voice selection: Select the singer's voice you want to use on the platform.

[1652] Uploading music: Upload the created music data and related metadata to the server.

[1653] 4. User terminal (listener) functions

[1654] Listeners search for and play songs on the platform by:

[1655] Song search: Enter keywords to search for songs.

[1656] Play Song: Select a song from the search results and stream it.

[1657] Specific examples

[1658] One day, singer A registers the rights to use his or her voice on a platform and sets a policy that it can only be used in the pop genre. In response, creator B creates a pop song using singer A's voice and uploads it to the platform. The server checks whether the uploaded song complies with singer A's policy and, if there are no problems, stores it in the database. Later, listener C searches for the pop song on the platform and plays it. Based on the playback data, the server calculates revenue and distributes it to singer A, creator B, and the author of the original song. Singer A and creator B are notified of the revenue distribution results.

[1659] In this way, the present invention provides functions for protecting the rights of singers' voices and achieving fair distribution of revenue, thereby building a music streaming system that can be used safely by all parties involved.

[1660] The processing flow will be explained below.

[1661] 1. User Registration and Contract Process

[1662] Step 1:

[1663] The singer (user) accesses the registration form.

[1664] The singer accesses the registration form provided on the system's web page.

[1665] Step 2:

[1666] The singer (user) enters their name, profile, voice sample, and desired usage policy.

[1667] Singers fill out a form with their name, profile information, a sample audio file of their voice, and voice usage policies (e.g., allowed genres, etc.).

[1668] Step 3:

[1669] The device sends the input to the server.

[1670] When the submit button on the registration form is clicked, the terminal transmits the information entered by the singer to the server.

[1671] Step 4:

[1672] The server stores the received information in a database.

[1673] The server stores the received registration information in a database and generates a unique ID for the singer.

[1674] Step 5:

[1675] The server will send a confirmation email to the singer.

[1676] The server will send a confirmation email to the singer's email address to notify them of the completion of registration.

[1677] 2. Creators' music production and release

[1678] Step 1:

[1679] The creator (user) logs in and accesses the dashboard.

[1680] Creators can log in by entering their account information and use the dashboard.

[1681] Step 2:

[1682] The creator (user) selects the singer's voice they want to use.

[1683] Creators select the voice they want to use from a list of singers provided on their dashboard.

[1684] Step 3:

[1685] The device sends the selection to the server.

[1686] Once the creator selects a voice, the content is sent to the server.

[1687] Step 4:

[1688] The server checks the policy of the selected singer's voice.

[1689] The server verifies that the selected singer's voice complies with the specified usage policy.

[1690] Step 5:

[1691] The device downloads a voice sample.

[1692] Once the server has completed the usage policy check, a voice sample is downloaded to the creator's device.

[1693] Step 6:

[1694] Creators (users) create music using AI tools.

[1695] Creators use AI tools (e.g., DAWs and AI voice plug-ins) to create music.

[1696] Step 7:

[1697] Creators (users) upload their completed songs to the platform.

[1698] Enter your completed music file and its metadata (e.g. title, genre, usage policy, etc.) into the upload form.

[1699] Step 8:

[1700] The device sends the uploaded content to the server.

[1701] Clicking the upload button sends the music file and metadata to the server.

[1702] Step 9:

[1703] The server checks the metadata and stores it in a database.

[1704] The server checks the metadata of the song to ensure it does not violate usage policies, and if there are no problems, stores the song data in a database.

[1705] 3. Listeners play music

[1706] Step 1:

[1707] The listener (user) logs in and searches for songs.

[1708] Listeners log in by entering their account information and then enter keywords into the search bar.

[1709] Step 2:

[1710] The device sends the search keywords to the server.

[1711] When you click the search button, the search keywords are sent to the server.

[1712] Step 3:

[1713] The server searches the database for the corresponding song and returns the results to the terminal.

[1714] The server searches the database for songs that match the keywords and sends the results to the listener's device.

[1715] Step 4:

[1716] The listener (user) selects the song to play.

[1717] Listeners select the song they want to play from the search results and click the play button.

[1718] Step 5:

[1719] The server begins streaming the selected song.

[1720] The server streams the selected songs and sends them to the listener's device.

[1721] 4. Revenue Sharing

[1722] Step 1:

[1723] The server periodically obtains the playback data.

[1724] The server periodically obtains the playback data from the playback history table.

[1725] Step 2:

[1726] The server performs the revenue calculation.

[1727] The server calculates the total revenue for each song based on the number of plays.

[1728] Step 3:

[1729] The server generates data for distributing revenue to each participant (author, singer, creator).

[1730] The server generates data for distributing the profits to each participant and calculates them based on the distribution ratio.

[1731] Step 4:

[1732] The server stores the revenue sharing results in a database and sends notifications to each participant.

[1733] The server stores the distribution results in a database and sends notifications to participants via email or other means.

[1734] The above is a specific processing flow for implementing the present invention, which realizes the protection of the singer's voice rights and the fair distribution of profits.

[1735] Example 1

[1736] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1737] In today's music streaming services, the rights of artists and other creators are often not properly protected. Furthermore, revenue distribution is often opaque and not fair. Furthermore, unauthorized use of artists' voices is common and often goes undetected. To solve these problems, a system is needed that properly manages the rights to use artists' voices, distributes revenue fairly, and prevents unauthorized use.

[1738] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1739] In this invention, the server includes a means for producing and releasing music on a platform for managing the rights to use singers' voices, a means for appropriately distributing revenue based on music playback, a means for managing and checking policies for singers' voices, a monitoring means for preventing unauthorized use, a server means for providing an interface for registering information regarding the use of users' voices, a database means for storing music sheet data and related metadata, and a server means for automatically performing policy checks, thereby enabling the protection of the rights of singers and creators, the fair distribution of revenue, and the prevention of unauthorized use.

[1740] A "singer" is an individual or group of individuals who provides their voice in a musical composition.

[1741] "Usage rights" are legal authorizations that permit the use of a particular vocal or musical work.

[1742] "Platform" refers to an online service or software that enables users to publish their content and interact with other users.

[1743] "Music" is an art form created by combining sounds and voices, and is a means of expression through the ears.

[1744] "Playback" is the act of listening to music or audio using a device.

[1745] "Revenue" is the monetary benefit derived from the provision of a service or consumption of a work.

[1746] A "policy" is a rule or guideline that prescribes a particular purpose or behavior.

[1747] "Monitoring" is the act of continually checking a particular activity or situation and responding when a problem arises.

[1748] An "interface" is the method or means by which a user interacts with a system or software.

[1749] A "server" is a computer system that provides data over a network.

[1750] A "database" is a system for efficiently storing, managing, and searching large amounts of data.

[1751] "Metadata" refers to data that contains information about specific data, such as the title of a song or the name of an artist.

[1752] A "policy check" is a process that verifies whether certain conditions are met based on a registered policy.

[1753] "Revenue sharing" is the act of distributing earned revenue fairly among multiple rights holders and related parties.

[1754] This is a next-generation music streaming system that properly manages the rights to use singers' voices and ensures fair revenue distribution. The system consists of three main components: a server and user terminals (singers, creators, and listeners).

[1755] Server Roles and Functions

[1756] The server is responsible for the central functions of the system. It performs the following main processes:

[1757] Accepting user registrations:

[1758] The server provides an interface for singers to register information about the use of their voice. This interface is a web-based application that singers can access from their terminals.

[1759] Save song data:

[1760] The music data and related metadata created by creators are stored in a database, which uses a common database management system such as MySQL or PostgreSQL.

[1761] Usage policy check:

[1762] The server automatically checks whether the uploaded song complies with the artist's usage policy, using a specific algorithm.

[1763] Revenue Share Calculation:

[1764] The server calculates revenue based on song playback data and distributes it to each party (singer, creator, original author), taking into account the number of plays, playback time, advertising revenue, etc.

[1765] Abuse Monitoring:

[1766] The server monitors the singer's voice for fraudulent use, which is achieved by using an AI model to check for voice pattern matches.

[1767] Roles and functions of user devices

[1768] The role of the singer

[1769] Singers will register the rights to use their voice and receive notification of the revenue sharing results. Specifically, they will:

[1770] Voice usage rights registration:

[1771] Singers upload their own sample voices from their devices and set usage policies.

[1772] Check your revenue sharing results:

[1773] The singer receives a notification from the server and checks the revenue sharing results.

[1774] The role of creators

[1775] Creators create and upload songs using singers' voices. Specifically, they do the following:

[1776] Voice Selection:

[1777] Creators choose the singer's voice they want to use on the platform.

[1778] Upload your song:

[1779] Creators upload the music data and related metadata they have created to the server, where they are checked to ensure that the usage policy is being followed.

[1780] The role of the listener

[1781] Listeners search for and play songs on the platform by:

[1782] Song search:

[1783] Listeners can input keywords to search for songs.

[1784] Play music:

[1785] Listeners can select songs from the search results and stream them.

[1786] Specific examples

[1787] One day, singer A registers the rights to use his or her voice on a platform and sets a policy that it can only be used in the pop genre. In response, creator B creates a pop song using singer A's voice and uploads it to the platform. The server checks whether the uploaded song complies with singer A's policy and, if there are no problems, stores it in the database. Later, listener C searches for the pop song on the platform and plays it. Based on the playback data, the server calculates revenue and distributes it to singer A, creator B, and the author of the original song. Singer A and creator B are notified of the revenue distribution results.

[1788] Example prompt:

[1789] "Please explain the user experience when Singer A sets a policy that allows music to be used only in the pop genre."

[1790] In this way, the invention provides a concrete means to protect singers' vocal rights and ensure fair distribution of revenue, creating a music streaming system that can be used safely by all parties involved.

[1791] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1792] Step 1: User registration and rights registration

[1793] Input: The singer uses the device to enter their name, contact information, sample voice (audio file), and usage policy.

[1794] What happens: A singer accesses the registration form, fills in the required information, uploads a sample voice, sets usage policies, and clicks the submit button.

[1795] Data processing / calculation: The server receives the input data, parses the usage policy and saves it in the database. It also saves the sample voice audio file.

[1796] Output: The server confirms that user registration and rights registration have been completed, and sends a registration completion notification to the singer's terminal.

[1797] Step 2: Choose your creator's voice

[1798] Input: Creators use their device to enter search criteria for the voice they want to use (e.g. genre, singer name).

[1799] How it works: Creators log in to the platform and enter criteria such as genre and artist name into the search form.

[1800] Data processing / calculation: The server queries the database based on the search criteria and returns filtered sample voice information of the corresponding singer.

[1801] Output: As a search result, a list of available singers and sample voice information are displayed on the creator's device.

[1802] Step 3: Upload your music

[1803] Input: Enter the audio file and metadata (e.g. title, genre, description) of the song created by the creator.

[1804] What happens: Creator selects a music file, enters relevant metadata into the form, and clicks the upload button.

[1805] Data processing / calculation: The server receives the uploaded audio file and metadata, checks the integrity of the file, and then verifies whether the song is permitted based on the usage policy.

[1806] Output: If the song data complies with the policy, the server saves the song data in the database and sends an upload completion notification to the creator's device. If it does not comply, it returns an error message.

[1807] Step 4: Search and play songs

[1808] Input: A listener uses a device to enter keywords to search for a song.

[1809] Specific actions: Listeners access the platform, enter keywords into the search form, and click the search button.

[1810] Data processing / calculation: The server queries the database based on the search keywords and generates a list of relevant songs.

[1811] Output: As a search result, a list of matching songs is displayed on the listener's device. When the listener selects a song and clicks the play button, the server sends the streaming data and plays the song.

[1812] Step 5: Calculate revenue share

[1813] Input: The server compiles song playback data (e.g., number of plays, playback time, advertising revenue).

[1814] What it does: The server records playback data every time a song is played on the platform. It aggregates the playback data over specific time intervals.

[1815] Data processing / calculation: The server uses a revenue sharing algorithm to calculate revenue, taking into account the number of plays, play time, ad revenue, etc.

[1816] Output: The server stores the revenue sharing results in a database and sends notifications to the singer, creator, and original author.

[1817] Step 6: Monitor for fraud

[1818] Input: Periodically scan all song data in the database.

[1819] Specific operation: The server executes a scheduled task at regular intervals to analyze all songs.

[1820] Data processing / calculation: The server uses AI models to detect unregistered usage patterns and unauthorized copying.

[1821] Output: If any unauthorized use is detected, the server will send a warning notice to the singer and the system administrator.

[1822] (Application example 1)

[1823] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1824] With the recent spread of music streaming services, fairness in the rights to use singers' voices and revenue distribution has become increasingly important. However, existing systems have insufficient management of rights to use singers' voices and revenue distribution, resulting in fraudulent use and unfair revenue distribution. Furthermore, there is a need for systems that make it easy for users to search and play music on their smart devices while also providing appropriate rights management. However, there is a lack of systems that combine an interactive, easy-to-use user interface with advanced voice analysis technology.

[1825] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1826] In this invention, the server includes means for producing and releasing music on a platform for managing the rights to use a singer's voice, means for appropriately distributing revenue based on music playback, means for managing and checking policies for the singer's voice, means for monitoring to prevent unauthorized use, means for using smart contracts to manage users' voice rights, means for providing streaming playback of music, means for notifying users of revenue distribution results, and means for providing a user interface that operates on a smartphone or a head-mounted display, thereby ensuring the appropriate use of a singer's voice and fairness in revenue distribution, and further improving the user experience on smart devices.

[1827] The "platform for managing singers' voice usage rights" is an online system that allows singers to register rights to use their voice and manage the scope and conditions of that use.

[1828] "Means for producing and releasing music" refers to a system that allows music creators to create music and publish that music data on the platform.

[1829] "Means for appropriate distribution of revenue" is a function for calculating revenue based on song playback data and distributing it fairly to the parties involved.

[1830] "Means for managing and checking singers' voice policies" refers to a system for checking whether the conditions and scope of voice usage set by singers are being properly observed.

[1831] "Monitoring measures to prevent unauthorized use" refers to a monitoring function to prevent the singer's voice from being used without permission.

[1832] "Means using smart contracts" refers to a mechanism that uses blockchain technology to automate the management of audio rights and revenue distribution, ensuring transparency.

[1833] "Means for providing streaming music playback" refers to a function that allows users to listen to music in real time over the Internet.

[1834] The "means for notifying the profit distribution results" is a function for promptly notifying the parties concerned of the profit calculation results.

[1835] A "user interface that runs on a smartphone or head-mounted display" is an operation screen or interface that allows users to use the system through a device such as a smartphone or VR device.

[1836] The system embodying this invention comprises the following main components: a server, a singer terminal, a creator terminal, and a listener terminal.

[1837] Server Features

[1838] The server provides the following functionality:

[1839] 1. Accepting user registration: The server provides an interface that allows singers, creators, and listeners to create accounts. In this case, AWS Cognito is used to manage account information.

[1840] 2. Managing singers' voice usage rights: Provides a function for singers to register their voice usage rights and set usage policies. Transparency and security are ensured by using smart contracts and blockchain technology (e.g., Ethereum).

[1841] 3. Storage and management of music data: Music data created by creators is stored in a database on the server (e.g., MySQL, PostgreSQL).

[1842] 4. Usage policy check: When a song is uploaded, the server checks whether it complies with the artist's usage policy. It uses voice analysis technology such as Google Cloud Speech-to-Text to scrutinize the audio data.

[1843] 5. Unauthorized use monitoring: Monitor singers' voices for unauthorized use and issue alerts if necessary.

[1844] 6. Providing a music streaming service: Using the Spotify SDK, we provide a fast, high-quality playback experience for listeners to stream music.

[1845] 7. Revenue share calculation: Based on the song playback data, revenue is calculated and distributed fairly to the parties involved. This process is performed using AWS Lambda and DynamoDB.

[1846] 8. Notification of Revenue Distribution Results: The singer, creator, and original author will be notified of the revenue distribution results via email.

[1847] User device functions

[1848] 1. Singer terminal: Singers can register their voice usage rights, manage the policies they have set, and check the revenue distribution results.

[1849] 2. Creator Terminal: Creators can create and upload music and ensure that usage policies are properly adhered to.

[1850] 3. Listener device: Listeners have the ability to search for songs and play them in streaming.

[1851] Specific examples

[1852] For example:

[1853] One day, a singer registers the rights to use her voice on a platform and sets a policy that it can only be used in the pop genre. In response, a creator creates a pop song using the singer's voice and uploads it to the platform. The server checks whether the uploaded song complies with the singer's policy and, if there are no problems, stores it in the database. A listener then searches for and plays the pop song on the platform. Based on the playback data, the server calculates revenue and distributes it to the singer, creator, and the copyright holder of the original song. The singer and creator are notified of the revenue distribution results.

[1854] The following prompt example is also provided:

[1855] Singer: "I only want my voice to be used in songs in the pop genre."

[1856] Creator: "I've created a new pop song and I want to use the singer's voice."

[1857] Listener: "I want to hear the latest pop songs."

[1858] The system ensures the proper use of singers' voices and fair revenue distribution, while also improving the user experience on smart devices.

[1859] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1860] Step 1:

[1861] A singer uses a terminal to register the rights to use their voice on the platform. At this time, the singer uploads a sample of their voice and sets a policy that it can only be used in the pop genre. The server receives the sample voice and policy data as input and registers it in a smart contract using blockchain technology to ensure transparency and security.

[1862] Step 2:

[1863] Creators use their devices to create songs using singers' voices and upload them to the platform. Creators then send the song data to the server based on the policy. The server stores the song data and metadata in a MySQL or PostgreSQL database and uses Google Cloud Speech-to-Text technology to check whether the voice data complies with the policy.

[1864] Step 3:

[1865] The server checks whether the uploaded song complies with the usage policy set by the artist. If it is confirmed, the song data is officially stored in the platform's database. Conversely, if there is a policy violation, the creator is notified and the upload is stopped.

[1866] Step 4:

[1867] Listeners use their devices to search for and stream songs on the platform. Listeners enter keywords, and the server searches the database for songs and outputs the results. The song data is then streamed to the listener's device using the Spotify SDK.

[1868] Step 5:

[1869] The server collects listeners' song play data, recording the number of plays and play time. This data is later used to calculate revenue distribution. The server processes the data in real time using AWS Lambda and DynamoDB to calculate how revenue should be distributed to each party.

[1870] Step 6:

[1871] The server distributes revenue based on the calculation results. The revenue is distributed to the singer, creator, and original song author based on a predetermined revenue distribution ratio. The distribution results are sent to the relevant parties via email or other notification methods.

[1872] Step 7:

[1873] After the server completes all procedures, it displays the revenue distribution results in real time on the user's dashboard, allowing all parties involved to instantly see how the revenue has been distributed.

[1874] Examples:

[1875] Singer: "I only want my voice to be used in songs in the pop genre."

[1876] Creator: "I've created a new pop song and I want to use the singer's voice."

[1877] Listener: "I want to hear the latest pop songs."

[1878] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1879] This is a next-generation music streaming system that recognizes users' emotions to improve the music experience and distributes revenue fairly, while properly managing the rights to use singers' voices. In addition to the functions of conventional music streaming systems, this system uses an emotion engine to recognize users' emotions and adjusts song recommendations and revenue distribution based on these.

[1880] System Overview

[1881] This system includes three main components: a server, user terminals (singers, creators, and listeners), and an emotion engine. These components have specific functions and operate while communicating with each other.

[1882] 1. Server Functions

[1883] The server manages the rights to use singers' voices, stores and manages song data, calculates revenue distribution and notifies the results, and processes emotion data using an emotion engine. Specifically, it performs the following processes:

[1884] Accepting user registration: Provide an interface for singers to register information regarding the use of their voice.

[1885] Storing music data: Music data and related metadata created by creators are stored in a database.

[1886] Usage policy checks: When a song is uploaded, we check that it complies with the usage policy set by the artist.

[1887] Revenue sharing calculation: Calculate revenue based on playback data and distribute it to each party (singer, creator, original author).

[1888] Unauthorized use monitoring: Monitor singers' voices for unauthorized use.

[1889] Emotional Data Analysis: Emotional data generated by the emotion engine is collected and analyzed to improve user experience and adjust revenue sharing.

[1890] 2. User device (singer) functions

[1891] Singers register their voice usage rights on the platform, set policies, and receive revenue sharing results through the system. Specifically, they perform the following operations:

[1892] Register voice usage rights: Upload sample voices and set usage policies.

[1893] Check the revenue sharing results: Receive notifications from the server and check the revenue sharing results.

[1894] 3. User Device (Creator) Functions

[1895] Creators create and upload songs using singers' voices. When uploading, they are required to ensure that the usage policy is adhered to. Specifically, they perform the following operations:

[1896] Voice selection: Select the singer's voice you want to use on the platform.

[1897] Uploading music: Upload the created music data and related metadata to the server.

[1898] 4. User terminal (listener) functions

[1899] Listeners search for and play songs on the platform, and the system recognizes their emotions and recommends appropriate songs. Specifically, they perform the following actions:

[1900] Song search: Enter keywords to search for songs.

[1901] Play Song: Select a song from the search results and stream it.

[1902] Emotion recognition: The emotion engine analyzes the user's emotions using the device's camera and voice input. Based on the analysis results, it recommends songs that suit the user.

[1903] Specific examples

[1904] One day, Singer A registers the rights to use his or her voice on the platform and sets a policy that it can only be used in the pop genre. In response, Creator B creates a pop song using Singer A's voice and uploads it to the platform. The server checks whether the uploaded song complies with Singer A's policy and, if there are no problems, stores it in the database. Listener C then searches for and plays a pop song on the platform. The emotion engine analyzes Listener C's facial expression and recognizes that he or she is relaxed, so songs in a similar pop genre are recommended.

[1905] Based on the playback data, the server calculates revenue and distributes it to singer A, creator B, and the copyright holder of the original song. The revenue distribution results are notified to singer A and creator B. In addition, revenue adjustments are also performed based on emotional data, realizing optimal revenue distribution according to the user's emotions.

[1906] In this way, the present invention not only protects the rights of singers' voices and ensures fair distribution of revenue, but also builds a system that provides a more personalized music experience through a recommendation function based on user emotions.

[1907] The processing flow will be explained below.

[1908] 1. User Registration and Contract Process

[1909] Step 1:

[1910] The singer (user) accesses the registration form.

[1911] The singer accesses the registration form provided on the system's web page.

[1912] Step 2:

[1913] The singer (user) enters their name, profile, voice sample, and desired usage policy.

[1914] Singers fill out a form with their name, profile information, a sample audio file of their voice, and usage policies (e.g., allowed genres, etc.).

[1915] Step 3:

[1916] The device sends the input to the server.

[1917] When the registration form submit button is clicked, the terminal sends the entered information to the server.

[1918] Step 4:

[1919] The server stores the received information in a database.

[1920] The server stores the received information in a database and generates a unique ID for the singer.

[1921] Step 5:

[1922] The server will send a confirmation email to the singer.

[1923] The server will send a confirmation email to the singer's email address to notify them of the completion of registration.

[1924] 2. Creators' music production and release

[1925] Step 1:

[1926] The creator (user) logs in and accesses the dashboard.

[1927] Creators log in on the system's web page and access their dashboard.

[1928] Step 2:

[1929] The creator (user) selects the singer's voice they want to use.

[1930] Creators select the voice they want to use from a list of singers provided on their dashboard.

[1931] Step 3:

[1932] The device sends the selection to the server.

[1933] Once the creator selects a voice, that information is sent to a server.

[1934] Step 4:

[1935] The server checks the policy of the selected singer's voice.

[1936] The server checks whether the selected singer's voice complies with the established usage policy.

[1937] Step 5:

[1938] The device downloads a voice sample.

[1939] Once the policy check is complete, the server will have the device download a voice sample file.

[1940] Step 6:

[1941] Creators (users) create music using AI tools.

[1942] Creators use AI tools to create and complete songs.

[1943] Step 7:

[1944] Creators (users) upload their completed songs to the platform.

[1945] Enter the completed music file and its metadata through the upload form.

[1946] Step 8:

[1947] The device sends the uploaded content to the server.

[1948] Clicking the upload button sends the music file and metadata to the server.

[1949] Step 9:

[1950] The server checks the metadata and stores it in a database.

[1951] The server checks the uploaded metadata, verifies that it does not violate any policies, and then stores it in the database.

[1952] 3. Listeners play music

[1953] Step 1:

[1954] The listener (user) logs in and searches for songs.

[1955] Listeners log into the system and enter keywords into the search bar.

[1956] Step 2:

[1957] The device sends the search keywords to the server.

[1958] When you click the search button, the entered keywords are sent to the server.

[1959] Step 3:

[1960] The server searches the database for the corresponding song and returns the results to the terminal.

[1961] The server searches the database for songs that match the keywords and sends the results to the listener's device.

[1962] Step 4:

[1963] The listener (user) selects the song to play.

[1964] Listeners select the song they want to play from the search results and click the play button.

[1965] Step 5:

[1966] The server begins streaming the selected song.

[1967] The server streams the selected songs and sends them to the listener's device.

[1968] Step 6:

[1969] The device recognizes the user's emotions using an emotion engine.

[1970] During playback, the device uses an emotion engine to analyze emotions from the user's facial expressions and voice.

[1971] Step 7:

[1972] The server receives the emotional data and recommends appropriate songs.

[1973] It receives data from the emotion engine and recommends songs that suit the user.

[1974] 4. Revenue Sharing

[1975] Step 1:

[1976] The server periodically obtains the playback data.

[1977] The server periodically obtains the playback data from the playback history table.

[1978] Step 2:

[1979] The server performs the revenue calculation.

[1980] The server calculates the total revenue for each song based on the number of plays and sentiment data.

[1981] Step 3:

[1982] The server generates data for distributing revenue to each participant (author, singer, creator).

[1983] The server generates data for distributing the profits to each participant based on the distribution ratio.

[1984] Step 4:

[1985] The server stores the revenue sharing results in a database and sends notifications to each participant.

[1986] The server stores the distribution results in a database and notifies each party of the revenue results via email or other means.

[1987] In this way, the present invention is a system that not only protects the rights of singers' voices and ensures fair distribution of revenue, but also provides a more personalized music experience through a recommendation function based on user emotions.

[1988] Example 2

[1989] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1990] Conventional music streaming systems have problems such as difficulty in properly managing singers' audio rights and fair revenue distribution. Furthermore, they lack a function for recommending songs based on user emotions. Therefore, a system to improve the user experience is needed.

[1991] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for producing and releasing music on a platform for managing singers' audio rights, a means for appropriately distributing revenue based on music playback, a means for managing and checking audio usage policies, a monitoring means for preventing unauthorized use, a means for recognizing user emotions and recommending songs based on the emotions, and a means for adjusting revenue distribution based on emotion data. This enables appropriate management of singers' audio rights, fair revenue distribution, and song recommendations based on user emotions.

[1992] "Singer's audio rights" refers to the right of a singer to commercially exploit his or her voice.

[1993] "Platform" refers to a system that enables users such as singers, creators, and listeners to create, play, and manage music via the Internet.

[1994] "Means for appropriately distributing revenue based on music plays" refers to algorithms or functions that distribute revenue to each party based on playback data.

[1995] "Audio usage policy" refers to the rules and standards that a singer sets for the conditions under which their voice is used and that are managed and checked based on those conditions.

[1996] "Monitoring measures to prevent unauthorized use" refers to monitoring systems and algorithms that check whether a singer's voice is being used fraudulently.

[1997] "Means for recognizing user emotions and recommending songs based on that" refers to a system that analyzes user emotions using technologies such as emotion engines and recommends songs based on the results.

[1998] "Means for adjusting revenue sharing based on emotional data" refers to an algorithm or function that adjusts the revenue sharing rate by taking into account the emotional data of users.

[1999] "Music data" refers to music files and associated metadata, such as title, artist information, genre, and other information.

[2000] "Database" refers to a system for managing stored audio rights information, music data, usage policies, playback data, user emotional data, and the like.

[2001] "Revenue Share Results" means the revenue share results for each party calculated based on playback data and other data.

[2002] MODE FOR CARRYING OUT THE INVENTION

[2003] This invention is a next-generation music streaming system that appropriately manages singers' audio rights, recognizes users' emotions to improve the music experience, and distributes revenue fairly. The main components of this system include a server and user terminals (singers, creators, and listeners), each with specific functions and operating while communicating with each other.

[2004] Server Features

[2005] The server is configured to perform the following main tasks:

[2006] Accepting user registration: Provides an interface for singers to register the rights to use their voice. Singers can upload their voice samples and enter genre settings and usage policies. The server receives this data and stores it in a database.

[2007] Music data storage: Creators upload the music data and related metadata they have created, which is then stored and managed on the server.

[2008] Usage policy check: Checks whether the uploaded song complies with the usage policy set by the artist. If it does not comply with the policy, the upload will be rejected and an error message will be sent to the creator.

[2009] Revenue sharing calculation: Calculate revenue based on playback data. Notify each party (singer, creator, original author) of the revenue sharing results.

[2010] Unauthorized use monitoring: We monitor whether singers' voices are being used illegally. If any unauthorized use is detected, we will send a warning message to the singer and take appropriate measures.

[2011] Emotion data analysis: The server analyzes the emotion data collected by the emotion engine, and uses this data to improve user experience and adjust revenue sharing.

[2012] As specific examples of hardware and software, the server uses an Amazon Web Services (AWS) EC2 instance, the database uses AWS RDS (Relational Database Service), and the emotion engine uses the Emotion API from Azure Cognitive Services.

[2013] User device (singer) functions

[2014] On the singer's device, do the following:

[2015] Registering vocal usage rights: Singers access the system, upload their vocal samples, set usage policies, and specify specific genres and uses.

[2016] Check the revenue sharing results: Receive notifications from the server and check the revenue sharing results.

[2017] User device (creator) functions

[2018] On the creator's device, do the following:

[2019] Voice Selection: Creators access the system and select the singer's voice they wish to use. They then review the usage policy for the selected voice.

[2020] Uploading music: Upload the created music data and related metadata to the server.

[2021] User terminal (listener) functions

[2022] On the listener's device, do the following:

[2023] Song search: Listeners access the system and enter keywords to search for songs.

[2024] Song Playback: The listener selects a song and streams it. During playback, the emotion engine analyzes the listener's emotions.

[2025] Emotion Recognition: The system collects the listener's emotions from the device's camera and voice input. Based on the analysis results of the emotion engine, the system recommends songs that suit the user.

[2026] Specific examples

[2027] One day, Singer A registers the rights to use his or her voice on the platform, allowing it to be used in the pop genre. Creator B then creates a pop song using that voice and uploads it to the server. The server checks that the song complies with Singer A's usage policy and then saves it.

[2028] Listener C accesses the system, searches for a pop song, and plays it. The emotion engine analyzes Listener C's facial expression and recognizes that he or she is in a relaxed state. The system then recommends a pop song suitable for relaxation.

[2029] Finally, the server calculates revenue based on the playback data of listener C and distributes the revenue to singer A, creator B, and the original author. The revenue distribution results are sent to singer A and creator B via a notification function.

[2030] Prompt sentence for generative AI model

[2031] An example of a prompt to enter is: "When a user searches for a pop song they want to listen to in a relaxing state, what kind of songs should be recommended? How can you recommend them based on the analysis results of the emotion engine?"

[2032] In this way, the present invention realizes a system that provides a more personalized music experience by properly managing singers' audio rights, fair revenue distribution, and recommending songs based on user emotions.

[2033] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2034] The flow of this system's program processing

[2035] Step 1: Accepting user registration

[2036] Input: Singer enters their voice sample, usage policy and genre settings.

[2037] Processing: The server uploads the singer's vocal sample and stores the configured usage policy and genre information in a database.

[2038] Output: The singer's voice rights information is stored in a database.

[2039] Specific behavior:

[2040] Singers access the system using a terminal and upload a sample of their voice into a form.

[2041] The server receives this sample and stores it in a database along with the usage policy.

[2042] A registration completion message is sent to the singer.

[2043] Step 2: Save the song data

[2044] Input: Creators upload the music data and related metadata they have created to the server.

[2045] Processing: The server stores the uploaded song data and metadata in a database.

[2046] Output: Song data and metadata are stored in a database.

[2047] Specific behavior:

[2048] Creators use their devices to upload the completed music data to the system.

[2049] The server receives the song data and related metadata and stores it in a database.

[2050] Notify creator of upload completion message.

[2051] Step 3: Check the usage policy

[2052] Input: Song data uploaded by the creator and usage policy set by the singer.

[2053] Processing: The server checks whether the uploaded music data complies with the singer's usage policy.

[2054] Output: If the song data complies with the policy, it is saved in the database; if it does not, an error message is sent to the creator.

[2055] Specific behavior:

[2056] The server obtains the metadata of the song data and the singer's usage policy.

[2057] If the policy is violated, the song data will be refused upload and an error message will be sent to the creator.

[2058] If the policy is met, the music data is stored in the database.

[2059] Step 4: Music playback and emotion recognition

[2060] Input: Commands for listeners to search for and play songs, as well as camera footage and audio input from listeners.

[2061] Processing: The server streams the music and the emotion engine analyzes the listener's emotions.

[2062] Output: Songs are recommended in real time based on the listener's sentiment analysis.

[2063] Specific behavior:

[2064] Listeners can search for songs by entering keywords and start playing them.

[2065] The server streams the music data to the listener's device.

[2066] The emotion engine analyzes the listener's camera footage and audio input data to quantify their emotional state, such as relaxation or excitement.

[2067] Based on the analysis results, the server recommends the most suitable songs to the listener.

[2068] Step 5: Calculate revenue share

[2069] Input: Song playback and emotion data.

[2070] Processing: The server calculates revenue based on playback data and adjusts it based on sentiment data.

[2071] Output: The revenue distribution results are notified to each party (singer, creator, original author).

[2072] Specific behavior:

[2073] The server collects song playback data and tallys up the number of times and duration of playback.

[2074] A revenue sharing adjustment algorithm based on sentiment data is applied to calculate the final revenue sharing amount.

[2075] Notify each party of the revenue sharing results.

[2076] Step 6: Monitor for fraud

[2077] Input: Audio data registered on the server and newly uploaded audio data.

[2078] Processing: The server compares and monitors the audio data using an abuse monitoring algorithm.

[2079] Output: If any unauthorized use is detected, send a warning message to the relevant parties.

[2080] Specific behavior:

[2081] The server routinely monitors newly uploaded audio data.

[2082] The system compares registered voice data with new data and applies an algorithm to detect fraudulent use.

[2083] If unauthorized use is discovered, a warning message will be sent to the relevant singers and creators, urging them to take action.

[2084] In this way, the system of the present invention processes data at each step based on the user's actions to provide a well-curated and personalized music experience.

[2085] (Application example 2)

[2086] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2087] Conventional music streaming systems focused on managing the rights to use singers' voices and distributing revenue, but this made it difficult to provide personalized music experiences, such as song recommendations based on user emotions. There was also a need for improved transparency and fairness in revenue distribution. In addition, there was a need to streamline the revenue distribution process while preventing unauthorized use.

[2088] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2089] In this invention, the server includes a means for producing and releasing music on a platform for managing the rights to use singers' voices, a means for appropriately distributing revenue based on music playback, a means for managing and checking singers' voice policies, a monitoring means for preventing unauthorized use, a means including an emotion engine for analyzing users' emotions in real time, and a means for recommending songs based on the emotion analysis. This allows users to receive optimal song recommendations based on their current emotions, resulting in a more personalized music experience. It also improves the transparency and fairness of revenue distribution, enabling efficient revenue distribution while preventing unauthorized use.

[2090] "Right to use the singer's voice" means the singer's right to use his or her voice for a specific use or purpose, which is used under a controlled agreement.

[2091] "Means for producing and releasing music on the platform" refers to a set of functions and processes that allow singers and creators to produce music and release it through online services.

[2092] "Appropriate revenue distribution measures" refers to methods and systems for distributing revenue fairly and transparently to the parties involved based on the number of times the music is played or other metrics.

[2093] "Means for managing and checking the singer's voice policy" refers to a method or system for checking and managing whether the terms and rules of use set by the singer for his or her voice are being observed.

[2094] "Monitoring measures to prevent unauthorized use" refers to systems that monitor and detect violations to ensure that an artist's voice or music is not used in unauthorized ways or places.

[2095] An "emotion engine" refers to technology that analyzes a user's facial expressions, tone of voice, etc. to detect their emotional state in real time.

[2096] "Means for recommending music" refers to methods or systems that suggest music that matches the user's mood at the time based on the results of an analysis of the user's emotions, etc.

[2097] "Means for storing and managing generated music data" refers to a system that safely stores music data created by creators in digital format and provides management and access as needed.

[2098] "Means for notifying each participant of the results of revenue distribution" refers to a method or system for promptly notifying the participants of the calculated results of revenue distribution.

[2099] MODE FOR CARRYING OUT THE INVENTION

[2100] This invention provides a music streaming system that analyzes a user's emotions in real time to improve the music experience while properly managing the rights to use a singer's voice. Specific embodiments of the invention will now be described.

[2101] System configuration

[2102] The system includes the following main components:

[2103] 1. Server:

[2104] Singer voice usage rights management system: Provides an interface for singers to set and register their own voice usage policies.

[2105] Music data storage and management system: Music data and related metadata created by creators are stored in a database.

[2106] Revenue sharing system: Revenue is calculated based on music playback data and distributed to each party.

[2107] Emotion engine: Analyzes user emotions in real time and uses that data to recommend songs and influence revenue sharing.

[2108] Unauthorized use monitoring system: Monitors whether a singer's voice is being used illegally.

[2109] 2. User Device:

[2110] Singer's device: Register your voice usage rights on the platform and check the revenue sharing results.

[2111] Creator's device: Create and upload songs using the singer's voice.

[2112] Listener's device: In addition to playing music, the device's camera and microphone are used to collect and analyze emotional data.

[2113] Hardware and software used

[2114] Hardware:

[2115] Smartphone (camera, microphone)

[2116] Servers (high-performance processors, storage)

[2117] software:

[2118] Python

[2119] OpenCV (image analysis)

[2120] DeepFace (emotional analysis)

[2121] librosa (audio analysis)

[2122] requests (API communication)

[2123] Program processing explanation

[2124] The server first provides a platform for managing singers' voice usage rights and allows singers to set their own voice usage policies. Creators use this platform to create and upload songs, and the song data is stored on the server. The system checks whether songs comply with the singer's set policies when uploaded and also monitors them to prevent unauthorized use.

[2125] When a listener plays a song, the user device uses the smartphone's camera and microphone to collect emotional data from the user. The collected data is analyzed in real time using an emotion engine. Based on the results of this analysis, songs that fit the listener's mood at that time are recommended.

[2126] Finally, revenue will be calculated based on playback data and sentiment analysis data and distributed appropriately to singers, creators, and other stakeholders. The revenue distribution results will be notified to each participant.

[2127] Examples of concrete examples and prompts

[2128] As a concrete example, consider a scenario in which a user plays a song using a smartphone. As the user plays the song, the smartphone camera captures the user's facial expression, and the emotion engine determines that the user is "happy." Based on this result, the system recommends songs that are suitable for "happy." Below is an example of an input prompt sentence for the generative AI model.

[2129] Prompt statement:

[2130] "What songs would you recommend to a user who is in a positive mood? Based on the results of the emotion analysis, the user has been determined to be "happy." Please recommend some songs based on that determination."

[2131] In this way, the present invention provides a personalized music experience that responds to the user's emotions and realizes transparency and fairness in revenue sharing.

[2132] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2133] Step 1:

[2134] A user launches a smartphone app and creates an account. The input is the user's basic information (name, email address, etc.), and the output is a notification that the user account has been created. In this step, the user information is registered in the database.

[2135] Step 2:

[2136] A singer registers the rights to use their voice on the platform. The input is the singer's sample voice data and usage policy (e.g., only pop genres can be used). The output is a notification that registration is complete. The server saves the input data in the database and adds it to the policy management table.

[2137] Step 3:

[2138] A creator creates a song and uploads it to the server. The input is the song data and metadata (title, genre, etc.), and the output is a notification that the upload is complete. The server stores the song data in a database and checks the usage policy.

[2139] Step 4:

[2140] A listener uses the app to play a song. The input is the listener's song selection, and the output is the song streaming. During this step, the smartphone's camera and microphone collect the user's emotional data (facial expressions, voice).

[2141] Step 5:

[2142] The device's emotion engine analyzes the collected emotion data. The input is the user's facial expression image and voice data, and the output is the emotion analysis result (e.g., happy, sad). Here, OpenCV and DeepFace are used for image analysis, and librosa is used to analyze voice data.

[2143] Step 6:

[2144] Based on the sentiment analysis results, the server recommends songs. The input is the sentiment analysis results and the listener's playback history, and the output is a list of recommended songs. In this step, the generative AI model uses the collected data to select songs that correspond to the prompt.

[2145] Step 7:

[2146] The listener plays the recommended song. The input is a selection from a list of recommended songs, and the output is a streaming playback of the song. The playback data is sent to the server and used for revenue sharing.

[2147] Step 8:

[2148] The server calculates revenue based on the playback data and emotion data. The inputs are the number of plays, song usage, emotion data, etc., and the output is the revenue sharing result. In this step, the revenue sharing system calculates the revenue for each party and records it in the database.

[2149] Step 9:

[2150] The revenue distribution result is notified to each participant. The input is the revenue distribution result, and the output is a notification message. The server sends this notification to each participant's terminal, realizing fair revenue distribution.

[2151] In this way, the system analyzes users' emotions in real time, recommends songs based on the results, and ensures fair revenue distribution.

[2152] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2153] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2154] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2155] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2156] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2157] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2158] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2159] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2160] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2161] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2162] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2163] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2164] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2165] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2166] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2167] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[2168] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2169] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[2170] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[2171] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[2172] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[2173] The following is further disclosed regarding the above embodiment.

[2174] (Claim 1)

[2175] A platform for managing the rights to use singers' voices and a means to produce and release music;

[2176] a means for appropriately distributing revenue based on music plays;

[2177] A means of managing and checking singer voice policies,

[2178] monitoring measures to prevent unauthorized use;

[2179] A system including:

[2180] (Claim 2)

[2181] 10. The system according to claim 1, further comprising means for storing and managing the generated music data.

[2182] (Claim 3)

[2183] 10. The system of claim 1, further comprising means for notifying each participant of a revenue sharing result.

[2184] "Example 1"

[2185] (Claim 1)

[2186] A platform for managing the rights to use singers' voices and a means to produce and release music;

[2187] a means for appropriately distributing revenue based on music plays;

[2188] A means of managing and checking singer voice policies,

[2189] monitoring measures to prevent unauthorized use;

[2190] a server means for providing an interface for registering information regarding the use of a user's voice;

[2191] database means for storing musical note data and associated metadata;

[2192] server means for automatically performing policy checks;

[2193] A system including:

[2194] (Claim 2)

[2195] 10. The system of claim 1, further comprising server means for calculating and distributing revenues to participants based on the generated song data and playback data.

[2196] (Claim 3)

[2197] 10. The system of claim 1, further comprising server means for notifying each participant of a revenue sharing result.

[2198] "Application Example 1"

[2199] (Claim 1)

[2200] A platform for managing the rights to use singers' voices and a means to produce and release music;

[2201] a means for appropriately distributing revenue based on music plays;

[2202] A means of managing and checking singer voice policies,

[2203] monitoring measures to prevent unauthorized use;

[2204] A means for using smart contracts to manage users' audio rights;

[2205] a means for providing streaming music;

[2206] a means for notifying the revenue distribution results;

[2207] A means for providing a user interface that operates on a smartphone or a head-mounted display;

[2208] A system including:

[2209] (Claim 2)

[2210] 10. The system according to claim 1, further comprising means for storing and managing the generated music data.

[2211] (Claim 3)

[2212] 10. The system of claim 1, further comprising means for analyzing the audio data utilizing a generative AI model.

[2213] "Example 2: Combining Emotion Engines"

[2214] (Claim 1)

[2215] A platform for managing singers' audio rights, a means to produce and release music, and

[2216] a means for appropriately distributing revenue based on music plays;

[2217] A means to manage and check voice usage policies;

[2218] monitoring measures to prevent unauthorized use;

[2219] means for recognizing a user's emotions and recommending songs based thereon;

[2220] means for adjusting revenue sharing based on sentiment data;

[2221] A system including:

[2222] (Claim 2)

[2223] 10. The system according to claim 1, further comprising means for storing and managing the generated music data.

[2224] (Claim 3)

[2225] 10. The system of claim 1, further comprising means for notifying each participant of the revenue sharing results.

[2226] "Application example 2 when combining emotion engines"

[2227] (Claim 1)

[2228] A platform for managing the rights to use singers' voices and a means to produce and release music;

[2229] a means for appropriately distributing revenue based on music plays;

[2230] A means of managing and checking singer voice policies,

[2231] monitoring measures to prevent unauthorized use;

[2232] a means including an emotion engine for analyzing user emotions in real time;

[2233] A means for recommending songs based on sentiment analysis;

[2234] A system including:

[2235] (Claim 2)

[2236] 10. The system according to claim 1, further comprising means for storing and managing the generated music data.

[2237] (Claim 3)

[2238] 10. The system of claim 1, further comprising means for notifying each participant of a revenue sharing result. [Explanation of symbols]

[2239] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means to create and release music on a platform that manages the rights to use singers' voices; a means for appropriately distributing revenue based on music plays; A means of managing and checking singers' voice policies, monitoring measures to prevent unauthorized use; A system including:

2. 2. The system according to claim 1, further comprising means for storing and managing the generated music data.

3. The system of claim 1 , further comprising means for notifying each participant of a revenue sharing result.

Images