system

Abstract

We provide the system. [Solution] A means of receiving abstract words or images from the user, A method for searching for songs based on abstract words or images received, A system that includes means for providing users with playlists generated from search results.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, and includes steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In modern times, music preferences and usage scenes are diversified, but it is not easy for users to find music suitable for their own images and feelings. In addition, there is no system that automatically generates an appropriate playlist for users who are not familiar with music or users who need to select music for events (such as weddings and parties). Therefore, there is a demand for a system that allows users to easily obtain a music playlist based on abstract words or images.

Means for Solving the Problems

[0005] This invention provides a system that includes means for receiving abstract words or images from a user, means for searching for music based on the received abstract words or images, and means for providing the user with a playlist generated from the search results. Specifically, it receives data in JSON format from a user via a network, searches for music using an API of an external music streaming service, generates a playlist from the search results, and provides it to the user. This system makes it possible for users to easily obtain music playlists that match their theme, providing a highly convenient service for users who are unfamiliar with selecting music or who need music selected for a special event.

[0006] A "user" is the entity that uses this system to input abstract words or images and receives playlists.

[0007] "Abstract words and images" refer to words and concepts that express emotions, atmosphere, themes, etc., rather than specific song titles or artist names.

[0008] "Songs" refer to music content provided through music streaming services.

[0009] A "search method" is a means of identifying relevant songs using the API of a music streaming service based on abstract words or images.

[0010] A "playlist" is a collection of songs selected based on a specific theme or image, presented in a list format.

[0011] A "network" refers to the communication lines and communication protocols used to exchange data and information between a user's terminal and a server.

[0012] "JSON data" refers to data represented using the JavaScript® Object Notation (JSON) format. It is primarily used for data exchange on the web.

[0013] A "music streaming service" is a service that delivers music content via the internet, allowing users to instantly play songs.

[0014] "API" stands for Application Programming Interface, and it refers to an interface that allows different software programs to share functions and data.

[0015] A "song ID" is a unique identifier used to identify a specific song on a music streaming service.

[0016] A "response" is the data that a server returns in response to a user's request. [Brief explanation of the drawing]

[0017] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7]It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

Embodiments for Carrying Out the Invention

[0018] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0019] First, the language used in the following description will be explained.

[0020] In the following embodiments, the signed processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), and APU (Accelerated Processing Unit).

[0021] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0022] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0023] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0025] [First Embodiment]

[0026] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0027] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0030] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0033] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0037] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0038] The embodiments for carrying out the present invention will be described in detail below. The present invention is a system that receives abstract words or images from a user, generates a music playlist based on them, and provides it to the user.

[0039] System Overview

[0040] The system works in conjunction with a server, a user's device, and a music streaming service. Based on abstract words or images entered by the user, the server uses the music streaming service's API to search for songs and generate a playlist.

[0041] Server Processing

[0042] The server is built using a web framework (e.g., Flask) and has the following functions:

[0043] 1. Receiving the request:

[0044] The server receives requests from users. These requests are sent, for example, in JSON format.

[0045] 2. Acquisition of abstract words and images:

[0046] The server extracts abstract words and images (for example, "relax" and "sunset") from the request.

[0047] 3. Search for songs:

[0048] The server uses the music streaming service's API to search for songs related to abstract words or images. It then retrieves the IDs of the relevant songs from the search results.

[0049] 4. Create a playlist:

[0050] The server uses the ID of the retrieved song to generate a new playlist. The playlist name might be, for example, "Playlist for Relaxation and Sunsets."

[0051] 5. Add songs to the playlist:

[0052] The server adds songs obtained from the search results to the generated playlist.

[0053] 6. Generating the response:

[0054] The server generates a response containing the name and URL of the created playlist and returns it to the user.

[0055] User processing

[0056] 1. Input abstract words or images:

[0057] Users input words or images that represent the themes or emotions they are seeking.

[0058] 2. Submitting a request:

[0059] The user enters words and images, converts them to JSON format, and sends a POST request to the server.

[0060] 3. Receiving the response:

[0061] The user receives a response from the server. The response includes the name and URL of the generated playlist.

[0062] 4. Using playlists:

[0063] The user clicks the received playlist URL and plays the playlist on their music streaming service.

[0064] Specific example

[0065] For example, if a user enters abstract words like "relax" and "sunset," the server will search for music based on these words and generate a playlist named "Playlist for Relaxation and Sunset." The playlist will contain the selected music and be provided to the user via a URL. By clicking this URL, the user can play the playlist and enjoy a relaxing atmosphere.

[0066] This invention allows users to easily obtain music playlists based on abstract images, which can be particularly helpful for users unfamiliar with music selection or when preparing for special events.

[0067] The following describes the processing flow.

[0068] Step 1:

[0069] The user opens a webpage or application using their device and enters abstract words or images. These inputs may include words such as "relax" or "sunset."

[0070] Step 2:

[0071] The system converts abstract words and images entered by the user into JSON format and sends a POST request to the server.

[0072] Step 3:

[0073] The server receives a POST request. A web framework such as Flask is used to parse the request.

[0074] Step 4:

[0075] The server extracts abstract words and images from the request. Specifically, it retrieves the `terms` field from the JSON data in the request body.

[0076] Step 5:

[0077] The server uses the API of a music streaming service to search for songs based on extracted words and images. This uses libraries such as Spotipy.

[0078] Step 6:

[0079] The server analyzes the search results returned from the music streaming service and compiles a list of the IDs of the relevant songs.

[0080] Step 7:

[0081] The server uses the music streaming service's API to generate a new playlist. The playlist name might be, for example, "Playlist for Relaxation and Sunsets."

[0082] Step 8:

[0083] The song is added to the playlist generated by the server using the song ID obtained in step 6.

[0084] Step 9:

[0085] Prepare a response in JSON format containing the name and URL of the playlist generated by the server, and return it to the user.

[0086] Step 10:

[0087] The user receives a response from the server. The response includes the playlist name and URL.

[0088] Step 11:

[0089] The user clicks the response URL and plays the playlist on their music streaming service. The user can then enjoy the generated playlist.

[0090] (Example 1)

[0091] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0092] Traditionally, manually selecting music playlists that matched users' emotions or themes was time-consuming and laborious. Furthermore, finding appropriate music was difficult for users unfamiliar with music selection or when preparing for special events. This invention aims to solve this problem by providing a system that allows users to easily obtain music playlists based on abstract words or images.

[0093] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0094] In this invention, the server includes means for receiving abstract words or images from a user, means for searching for music based on the received abstract words or images, means for providing the user with a playlist generated from the search results, means for receiving and analyzing user requests through a web framework, means for searching for relevant music using a music streaming service API, means for generating a playlist from the search results and adding music, and means for generating the name and URL of the generated playlist as a response. This makes it possible for users to easily and quickly obtain music playlists that match their mood or theme.

[0095] A "user" is a person or group that uses the system to generate music playlists based on abstract words or images.

[0096] "Abstract words" are words that express emotions or themes, rather than referring to specific objects or actions.

[0097] An "image" is a concept or theme expressed using visual or non-visual means.

[0098] "Means" refer to a set of processes or functions performed to achieve a specific objective.

[0099] A "musical piece" refers to a musical work, primarily offered through music streaming services.

[0100] "Searching" is the act of finding information based on specific criteria.

[0101] A "playlist" is a list of songs that are played in a specific order.

[0102] A "web framework" is a software framework for developing web applications.

[0103] A "request" refers to a request made from a client (the user's device) to a server.

[0104] "JSON format" is an abbreviation for JavaScript Object Notation, and it is a lightweight text format for representing data.

[0105] "API" stands for Application Programming Interface, and it is an interface that allows external parties to access the functions of software.

[0106] "Response" refers to the answer that a server returns to a client in response to a request.

[0107] "URL" stands for Uniform Resource Locator, and it is an address that indicates the location of a resource on the web.

[0108] A "network" is a system that connects multiple information devices to communicate data.

[0109] "User's device" refers to a device that the user directly operates (such as a smartphone or personal computer).

[0110] A "music streaming service" is a service that provides music in real time via the internet.

[0111] The embodiments for carrying out the present invention will be described in detail below. The present invention is a system that receives abstract words or images from a user, generates a music playlist based on them, and provides it to the user.

[0112] System Overview

[0113] The system operates in conjunction with a server, user terminals, and a music streaming service. The server is built using a web framework, specifically one such as Flask. Based on abstract words or images entered by the user, the server searches for songs using the music streaming service's API and generates a playlist.

[0114] Server Processing

[0115] The server has the following functions:

[0116] 1. Receiving the request:

[0117] The server receives requests from users. These requests are sent, for example, in JSON format.

[0118] 2. Acquisition of abstract words and images:

[0119] The server extracts abstract words and images from the request. Examples include the word "relax" and an image like "sunset."

[0120] 3. Search for songs:

[0121] The server uses the music streaming service's API to search for songs related to abstract words or images. It then retrieves the IDs of the relevant songs from the search results.

[0122] 4. Create a playlist:

[0123] The server uses the IDs of the retrieved songs to generate a new playlist. The playlist name might be, for example, "Playlist for Relaxation and Sunsets."

[0124] 5. Add songs to the playlist:

[0125] The server adds songs obtained from the search results to the generated playlist.

[0126] 6. Generating the response:

[0127] The server generates a response containing the name and URL of the created playlist and returns it to the user.

[0128] User processing

[0129] 1. Input abstract words or images:

[0130] Users use their devices to input words or images that represent the themes or emotions they desire.

[0131] 2. Submitting a request:

[0132] The user enters words and images, converts them to JSON format, and sends a POST request to the server.

[0133] 3. Receiving the response:

[0134] The user receives a response from the server. The response includes the name and URL of the generated playlist.

[0135] 4. Using playlists:

[0136] The user clicks the received playlist URL and plays the playlist on their music streaming service.

[0137] Specific example

[0138] For example, if a user enters the words "relax" and "sunset," the server will search for songs based on these words and generate a playlist named "Playlist for Relaxation and Sunset." The playlist will contain the selected songs. The user can play this playlist and enjoy a relaxing atmosphere by clicking on the URL provided by the server.

[0139] Example of a prompt

[0140] An example of a prompt for a generative AI model is, "Generate a playlist with the themes of relaxation and sunsets. Please select songs that fit the theme."

[0141] This system allows users to easily obtain music playlists based on abstract images. This feature will be particularly useful for users who struggle with selecting music or when preparing for special events.

[0142] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0143] Step 1:

[0144] The server receives a JSON-formatted request sent from the user's terminal. The input includes abstract words and images entered by the user. Specifically, the server retrieves the JSON data using request.get_json(). The output is the retrieved JSON data, which is then passed to the next step.

[0145] Step 2:

[0146] The server parses the received JSON data and extracts abstract words and images entered by the user. The JSON data obtained in step 1 is used as input. Specifically, the data is parsed using `data = request.json`, and abstract words and images are extracted using `keywords = data['keywords']`. The extracted words and images are then passed to the next step as output.

[0147] Step 3:

[0148] The server uses the music streaming service's API to search for songs related to abstract words and images. The words and images extracted in step 2 are used as input. Specifically, the API is called using requests.get('https: / / api.music-service.com / v1 / search', headers=headers, params=params) to obtain the relevant song IDs. The obtained song IDs are then passed to the next step as output.

[0149] Step 4:

[0150] The server generates a new playlist based on the retrieved song ID. The song ID obtained in step 3 is used as input. Specifically, it creates the playlist by executing requests.post('https: / / api.music-service.com / v1 / users / {user_id} / playlists', headers=headers, json=playlist_data). The generated playlist ID is passed to the next step as output.

[0151] Step 5:

[0152] The server adds songs related to the generated playlist. The inputs used are the playlist ID generated in step 4 and the song IDs obtained in step 3. Specifically, it executes requests.post('https: / / api.music-service.com / v1 / playlists / {playlist_id} / tracks', headers=headers, json=tracks_data) to add the songs to the playlist. The output is the playlist URL, which is passed to the next step.

[0153] Step 6:

[0154] The server generates a response containing the name and URL of the generated playlist and returns it to the user. The input used is the playlist URL obtained in step 5. Specifically, the response data is constructed using response_data = {'playlist_name': playlist_name, 'url': playlist_url}, and the response is returned to the user using return jsonify(response_data). The output provides the user with information about the generated playlist.

[0155] Step 7:

[0156] The user receives a playlist URL from the server using their device. The server's response is used as input. Specifically, the response.json() function is used to parse the response data. The playlist URL is displayed on the user's device as output.

[0157] Step 8:

[0158] The user clicks the received playlist URL and plays the playlist on their music streaming service. The input used is the playlist URL received in step 7. Specifically, the user opens the URL using a browser or music streaming app and plays the playlist. The output is the playback of the song the user requested.

[0159] (Application Example 1)

[0160] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0161] Traditional music playlist generation systems assumed users were familiar with specific songs and artists, and lacked the ability to automatically generate playlists from abstract words or visual information. As a result, users had the problem of spending a lot of time finding music that suited their mood or situation. Furthermore, if users wanted to express a specific atmosphere or situation, they had to manually search for songs, which was a time-consuming process.

[0162] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0163] In this invention, the server includes means for receiving abstract words and visual information from a user, means for searching for music based on the received abstract words and visual information, means for providing the user with a playlist generated from the search results, means for creating a playlist using the API of a music provider, and means for returning the URL of the provider to the user. This makes it possible for the user to instantly obtain the optimal playlist based on abstract words and visual information.

[0164] A "user" is an individual or group that uses the system.

[0165] "Abstract words" are words that express concepts or emotions rather than concrete things.

[0166] "Visual information" refers to data related to vision, such as images and videos.

[0167] "Means" refer to methods or devices used to achieve a specific objective.

[0168] A "musical piece" refers to the individual works that make up a piece of music.

[0169] "Searching" refers to the act of finding specific information.

[0170] A "playlist" is a list of songs that are organized according to a specific order or theme.

[0171] "Providing" refers to the act of handing over a certain item or service to another person.

[0172] A "server" is a computer that provides services to other computers via a network.

[0173] A "network" is a system in which multiple computers are connected to each other and can exchange information.

[0174] "JSON formatted data" refers to data structured in JavaScript Object Notation format.

[0175] An "external music provider API" refers to an application programming interface provided by an external music distribution service.

[0176] This invention is a system that receives abstract words and visual information from a user, generates a music playlist based on that information, and provides it to the user. Specific embodiments for carrying out this invention are described in detail below.

[0177] The system operates in conjunction with the user's terminal, the server, and the music provider's API.

[0178] System Configuration

[0179] server

[0180] The server primarily has the following functions:

[0181] Request received:

[0182] The server receives requests from users that include abstract words and visual information. These requests are sent in JSON format.

[0183] Acquisition of abstract words and visual information:

[0184] The server parses the received JSON data and extracts abstract words and visual information. In this process, natural language processing (NLP) is used to process abstract words, and image processing is used to analyze visual information.

[0185] Search for songs:

[0186] The server uses the API of the music provider (e.g., Spotify) to search for songs related to abstract words or visual information. It then retrieves the IDs of the relevant songs.

[0187] Playlist generation:

[0188] A new playlist is generated based on the acquired song IDs. The playlist name is determined based on abstract words or visual information.

[0189] Add songs to the playlist:

[0190] Add songs obtained from search results to the playlist you created.

[0191] Response generation:

[0192] Generate a response containing the name and URL of the generated playlist, and send it back to the user.

[0193] User terminal

[0194] The user terminal has the following functions:

[0195] Inputting abstract words and visual information:

[0196] The user inputs abstract words and visual information to generate a playlist. This input is done through the application.

[0197] Sending a request:

[0198] The user terminal sends a request containing abstract words and visual information to the server in JSON format.

[0199] Response received:

[0200] Receive the response sent back from the server and display the information of the generated playlist.

[0201] Specific example

[0202] For example, if a user enters an abstract phrase like "a refreshing morning walk" and uploads a morning landscape image as visual information, the server analyzes these to search for relevant music. It then generates a playlist with a name like "Playlist for a refreshing morning walk" and provides the user with its URL. By clicking the provided URL, the user can listen to the appropriate music.

[0203] Example of a prompt

[0204] Abstract words: A refreshing morning walk

[0205] Visual information: Morning landscape image

[0206] This allows users to easily obtain music playlists that match their desired themes and emotions. This invention is particularly useful for users who are unfamiliar with music selection and for creating the right atmosphere for special events.

[0207] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0208] Step 1:

[0209] The user inputs abstract words or visual information into the application.

[0210] Input: Abstract words entered by the user (e.g., "A refreshing morning walk") and visual information (e.g., an image of a morning landscape).

[0211] Output: Request data in JSON format.

[0212] Specific operation: The smartphone app receives user input, converts it into JSON data, and prepares to send it to the server.

[0213] Step 2:

[0214] The user's terminal sends a request in JSON format to the server.

[0215] Input: Request data in JSON format entered by the user.

[0216] Output: Request data sent to the server.

[0217] Specific operation: The user's terminal sends JSON-formatted request data to the server via the network.

[0218] Step 3:

[0219] The server receives the request and extracts abstract words and visual information.

[0220] Input: Request data in JSON format.

[0221] Output: Abstract words and visual information.

[0222] Specific operation: The server parses the request, analyzes abstract words using an NLP model, and analyzes visual information using image processing (e.g., OpenCV).

[0223] Step 4:

[0224] The server calls the music provider's API based on the analysis results and searches for related songs.

[0225] Input: Keywords and parameters based on abstract words and visual information.

[0226] Output: A list of IDs for related songs.

[0227] Specific operation: The server calls the music provider's API (e.g., Spotify API), searches for and retrieves relevant songs based on the analysis results.

[0228] Step 5:

[0229] The server generates a new playlist based on the song ID it retrieves.

[0230] Input: List of song IDs.

[0231] Output: The ID and URL of the newly generated playlist.

[0232] Specific operation: The server calls the music provider's API again, creates a new playlist, and adds the retrieved songs to that playlist.

[0233] Step 6:

[0234] The server sends the generated playlist information back to the user.

[0235] Input: Playlist ID and URL.

[0236] Output: JSON-formatted response data sent to the user's terminal.

[0237] Specific operation: The server generates a JSON response containing the playlist name and URL, and sends it to the user's terminal.

[0238] Step 7:

[0239] The user's terminal receives the response and displays the information of the generated playlist.

[0240] Input: JSON response data from the server.

[0241] Output: The name and URL of the playlist to be displayed.

[0242] Specific operation: The user's terminal parses the response received from the server and displays the playlist name and URL to the user.

[0243] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0244] The embodiments for carrying out the present invention will be described in detail below. The present invention combines a system that receives abstract words or images from a user, generates a music playlist based on them, and provides it to the user, with an emotion engine that recognizes the user's emotions.

[0245] System Overview

[0246] The system works in conjunction with a server, a user's device, and a music streaming service. Based on abstract words, images, and emotions entered by the user, the server uses the music streaming service's API to search for songs and generate a playlist.

[0247] Server Processing

[0248] The server is built using a web framework (e.g., Flask) and has the following functions:

[0249] 1. Receiving the request:

[0250] The server receives requests from users. These requests are sent, for example, in JSON format.

[0251] 2. Acquisition of abstract words and images:

[0252] The server extracts abstract words and images (for example, "relax" and "sunset") from the request.

[0253] 3. Emotional analysis using an emotion engine:

[0254] The server uses an emotion engine to analyze user emotions based on their input. The emotion engine uses text-based analysis techniques to identify the user's emotions (e.g., happiness, sadness, excitement).

[0255] 4. Search for songs:

[0256] The server uses music streaming service APIs to search for songs based on abstract words, images, and sentiment analysis results. This uses libraries such as Spotipy.

[0257] 5. Create a playlist:

[0258] The server uses the ID of the retrieved song to generate a new playlist. The playlist name might be, for example, "Playlist for Relaxation and Sunsets."

[0259] 6. Add songs to the playlist:

[0260] The server adds songs to the generated playlist using the song IDs obtained in step 4.

[0261] 7. Generating the response:

[0262] The server prepares a response in JSON format, including the name and URL of the generated playlist, and returns it to the user.

[0263] User processing

[0264] 1. Input abstract words or images:

[0265] Users input words or images that represent the themes or emotions they are seeking.

[0266] 2. Submitting a request:

[0267] The user enters words and images, converts them to JSON format, and sends a POST request to the server.

[0268] 3. Receiving the response:

[0269] The user receives a response from the server. The response includes the name and URL of the generated playlist.

[0270] 4. Using playlists:

[0271] The user clicks the received playlist URL and plays the playlist on their music streaming service.

[0272] Specific example

[0273] For example, if a user enters abstract words like "relax" and "sunset," and the emotion engine interprets these words as "happiness," the server will search for songs based on this and generate a playlist named "Playlist for Relaxation and Sunset." The playlist contains the selected songs and is provided to the user via a URL. By clicking this URL, the user can play the playlist and enjoy a relaxed and happy atmosphere.

[0274] This invention allows users to easily access music playlists based on abstract images and emotions, which can be particularly helpful for users unfamiliar with music selection or when preparing for special events.

[0275] The following describes the processing flow.

[0276] Step 1:

[0277] The user opens a webpage or application using their device and enters an abstract word or image into a text field. This input could include, for example, "relax" or "sunset."

[0278] Step 2:

[0279] The system sends JSON data containing abstract words and images entered by the user to the server as a POST request.

[0280] Step 3:

[0281] The server receives a POST request and extracts JSON data from the request body. Specifically, the Flask framework is used to parse the data from the request.

[0282] Step 4:

[0283] The server retrieves abstract words and images (e.g., "Relaxing sunset" included in the terms field) from the JSON data.

[0284] Step 5:

[0285] The server uses the emotion engine to analyze the abstract words and images and identify the user's emotion. For example, the words "Relaxing" and "sunset" are classified as the emotion of "happiness".

[0286] Step 6:

[0287] The server uses the API of the music streaming service to search for music based on the identified emotion and abstract words. The Spotipy library is used to send the search query.

[0288] Step 7:

[0289] The server analyzes the search results returned from the music streaming service and compiles the IDs of the relevant music into a list. For example, the IDs of 10 songs are included in the list.

[0290] Step 8:

[0291] The server uses the API of the music streaming service to generate a new playlist. The name of the playlist shall be "Playlist for Relaxing and Sunset".

[0292] Step 9:

[0293] The server adds the music using the IDs of the music obtained in Step 7 to the generated playlist. Specifically, all the music IDs are added to the playlist.

[0294] Step 10:

[0295] The server prepares a response containing the name and URL of the generated playlist in JSON format and returns it to the user.

[0296] Step 11:

[0297] The user receives a response from the server and checks its contents. The response includes the name and URL of the generated playlist.

[0298] Step 12:

[0299] The user clicks the response URL and opens the playlist on their music streaming service. When the playlist is played, songs based on the user's selected emotions and images are played.

[0300] (Example 2)

[0301] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0302] Conventional music playlist generation systems have been unable to adequately respond to users' abstract words and images, making it difficult to provide playlists that align with their emotions and preferences. Furthermore, there were no systems that combined emotional analysis to address the diverse needs of users.

[0303] The identification processing performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for receiving abstract words or images from the user, means for analyzing emotions from the received abstract words or images, means for searching for music based on the analyzed emotions and abstract words or images, and means for providing the user with a playlist generated from the search results. This makes it possible to analyze the user's emotions based on their abstract input and automatically generate and provide a music playlist that suits them.

[0304] "Abstract words and images" refer to words and visual concepts that are difficult for users to describe as specifically as possible, but are used to express the state or mood.

[0305] "Means for analyzing emotions" refers to technologies and algorithms for inferring and identifying the emotions of users from the words and images input by users. This includes natural language processing and machine learning models.

[0306] "Means for performing music search" refers to technologies and methods for finding music that matches specific keywords or emotions by using APIs of external music databases or music streaming services.

[0307] "Means for generating a playlist" refers to technologies and methods for appropriately combining the searched music to create a single continuous music list. This enables continuous playback of music along a specific theme or emotion.

[0308] "Means for receiving" refers to interfaces and communication protocols for receiving data from the outside via a network. This includes web frameworks that process HTTP requests.

[0309] "Data in JSON format" is an abbreviation for JavaScript Object Notation (JSON), which is a lightweight data exchange format. It represents data in a format that is easy for humans to read and easy for machines to analyze.

[0310] Modes for carrying out the invention

[0311] The modes for carrying out the present invention will be described in detail below. The present invention is a system that receives abstract words and images from a user, generates a music playlist based on them, and provides it to the user, further combined with an emotion engine that recognizes the emotions of the user.

[0312] The system operates in conjunction with a server, user terminals, and music streaming services. This will be explained below, along with specific hardware and software implementations.

[0313] Server Processing

[0314] The server is built using a web framework (e.g., Flask). The main processing steps are as follows:

[0315] 1. Receiving the request:

[0316] The server receives requests from users. This is done by users sending data containing abstract keywords and images in JSON format through their devices. For example, a user might enter the keywords "relax" and "sunset."

[0317] 2. Acquisition of abstract words and images:

[0318] The server parses the received JSON data and extracts the words and images entered by the user. This process uses the json module from Python's standard library.

[0319] 3. Emotional analysis using an emotion engine:

[0320] The server uses the Hugging Face Transformers library to analyze emotions from the user's input text. This analysis associates the keywords "relax" and "sunset" with the emotion "happiness." The emotion engine uses a pre-trained model to determine the emotion of the input text.

[0321] 4. Search for songs:

[0322] The server uses the Spotipy library to access the Spotify API and search for songs based on the keywords "relax," "sunset," and "happiness." This search query targets Spotify's song database to retrieve the appropriate song ID.

[0323] 5. Create a playlist:

[0324] The server generates a new playlist using the retrieved song ID. The playlist name will be based on the keywords entered by the user, such as "Playlist for Relaxation and Sunsets." The generated playlist will then be linked to the user's Spotify account.

[0325] 6. Add songs to the playlist:

[0326] The server adds songs to the generated playlist. This process uses the Spotipy library to send song addition requests to the Spotify API.

[0327] 7. Generating the response:

[0328] The server creates a JSON response containing the name of the generated playlist and its URL. For example, it might be returned to the user using Flask's response functionality. The JSON response includes data such as:

[0329] json

[0330] {

[0331] "playlist_name": "Playlist for relaxation and sunsets",

[0332] "playlist_url": "https: / / open.spotify.com / playlist / ..."

[0333] }

[0334] User processing

[0335] 1. Input abstract words or images:

[0336] Users input words or images that represent the themes or emotions they are seeking. For example, they might use words like "relax" or "sunset."

[0337] 2. Submitting a request:

[0338] The user enters words and images, converts them to JSON format, and sends a POST request to the server.

[0339] 3. Receiving the response:

[0340] The user receives a response from the server. The response includes the name and URL of the generated playlist.

[0341] 4. Using playlists:

[0342] The user clicks the received playlist URL and plays the playlist on their music streaming service.

[0343] Specific example

[0344] For example, if a user enters abstract words like "relax" and "sunset," and the emotion engine interprets these words as "happiness," the server will search for songs based on this and generate a playlist named "Playlist for Relaxation and Sunset." The playlist contains the selected songs and is provided to the user via a URL. By clicking this URL, the user can play the playlist and enjoy a relaxed and happy atmosphere.

[0345] Example of a prompt

[0346] "Please create a playlist for relaxation and sunsets."

[0347] Based on this input, the generative AI model performs sentiment analysis based on the user's abstract words, selects appropriate songs, and creates a playlist.

[0348] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0349] Step 1:

[0350] The user inputs abstract words or images.

[0351] Detailed explanation:

[0352] Users input abstract words or images that represent a theme or emotion using web forms or mobile apps from their own devices. For example, they might use words like "relax" or "sunset."

[0353] Inputs and outputs:

[0354] Input: Abstract words or images entered by the user (e.g., "relax," "sunset").

[0355] Output: The terminal generates data in JSON format.

[0356] Step 2:

[0357] The terminal sends the generated JSON data to the server.

[0358] Detailed explanation:

[0359] The terminal converts the words and images entered by the user into JSON format and sends them to a specific endpoint on the server as an HTTP POST request.

[0360] Inputs and outputs:

[0361] Input: JSON data containing abstract words or images entered by the user.

[0362] Output: JSON data received by the server.

[0363] Step 3:

[0364] The server parses the JSON data and extracts abstract words and images.

[0365] Detailed explanation:

[0366] The server uses the Flask web framework to parse the received JSON data and extract the words and images entered by the user. This process uses Python's json module.

[0367] Inputs and outputs:

[0368] Input: JSON data received by the server.

[0369] Output: Abstract words or images (e.g., "relax," "sunset").

[0370] Step 4:

[0371] The server uses an emotion engine to analyze the user's emotions based on their input.

[0372] Detailed explanation:

[0373] The server uses the Hugging Face Transformers library to analyze the sentiment of the user's input text. This analysis associates "relaxed" and "sunset" with the emotion of "happiness."

[0374] Inputs and outputs:

[0375] Input: Abstract words or images (e.g., "relax," "sunset").

[0376] Output: Analyzed emotion (e.g., "happiness").

[0377] Step 5:

[0378] The server uses the music streaming service's API to search for songs.

[0379] Detailed explanation:

[0380] The server uses the Spotipy library to access the Spotify API and search for songs based on the keywords "relax," "sunset," and "happiness." This search query targets Spotify's song database to retrieve the appropriate song ID.

[0381] Inputs and outputs:

[0382] Input: Abstract words, images, or analyzed emotions (e.g., "relaxed," "sunset," "happy").

[0383] Output: A list of searched song IDs.

[0384] Step 6:

[0385] The server generates a playlist using the acquired song ID.

[0386] Detailed explanation:

[0387] The server uses the Spotipy library to generate a new playlist via the Spotify API. The playlist name will be "Playlist for Relaxation and Sunsets," based on keywords entered by the user.

[0388] Inputs and outputs:

[0389] Input: A list of searched song IDs.

[0390] Output: The ID and name of the generated playlist.

[0391] Step 7:

[0392] Add songs to the playlist generated by the server.

[0393] Detailed explanation:

[0394] The server adds the searched songs to the generated playlist. This process uses the Spotipy library to send song addition requests to the Spotify API.

[0395] Inputs and outputs:

[0396] Input: The ID of the generated playlist and a list of the searched song IDs.

[0397] Output: A playlist with the added songs.

[0398] Step 8:

[0399] The server prepares a JSON response containing the name and URL of the generated playlist and returns it to the user.

[0400] Detailed explanation:

[0401] The server creates a JSON response containing the name and URL of the generated playlist and returns it to the user. This response is returned using Flask's response functionality.

[0402] Inputs and outputs:

[0403] Input: The ID and name of the generated playlist.

[0404] Output: JSON response containing the playlist name and URL.

[0405] Step 9:

[0406] The user receives the playlist URL returned from the server and plays it.

[0407] Detailed explanation:

[0408] The user receives a response from the server via their device and clicks the provided playlist URL. This allows them to play the playlist on their music streaming service.

[0409] Inputs and outputs:

[0410] Input: JSON response containing the playlist name and URL.

[0411] Output: The music playlist that the user will play.

[0412] (Application Example 2)

[0413] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0414] In modern virtual stores and online shopping experiences, there is a lack of personalized service based on users' emotions and moods. In particular, the selection of music is often generic rather than tailored to an individual's mood or theme, which degrades the quality of the user's purchasing experience.

[0415] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0416] In this invention, the server includes means for receiving abstract words or images from the user, means for searching for music based on the received abstract words or images and the user's emotions, means for providing the user with a playlist generated from the search results, and means for automatically generating and providing music playlists to be played within the virtual store. This makes it possible to improve the quality of online shopping through a personalized music experience based on the user's emotions and mood.

[0417] "Abstract words and images" refer to vague words or visual concepts used to express the feelings or themes that the user is experiencing.

[0418] "Emotions" refer to the user's psychological and sensory state and are classified into emotional categories such as happiness, sadness, and excitement.

[0419] "Searching for music" refers to the process of finding appropriate songs from the music streaming service's database based on user input.

[0420] A "playlist" refers to a collection of songs selected based on a specific theme or emotion, and is a list of music arranged to be played sequentially.

[0421] A "virtual store" is a virtual commercial space operated on the internet where users can browse and purchase products online.

[0422] "Automatic generation" refers to the process by which a system creates specific deliverables or data based on user input and algorithms, without requiring manual operation.

[0423] "To provide" refers to giving information or services to users so that they can use them.

[0424] This invention is a system that receives abstract words or images from a user, generates a music playlist based on them, and plays it within a virtual store. The following describes in detail the configurations for implementing this system.

[0425] System Configuration

[0426] The system operates in conjunction with servers, user terminals, and music streaming services.

[0427] Server Processing

[0428] The server is built using a web framework (e.g., Flask) and includes the following software components:

[0429] 1. Receiving the Request: The server receives a request from the user. The request is sent in JSON format.

[0430] 2. Retrieving abstract words and images: The server extracts abstract words and images from the request. Examples include "relax" and "sunset."

[0431] 3. Sentiment Analysis: The server uses an emotion engine to analyze the user's emotions. The emotion engine utilizes text analysis libraries such as TextBlob to identify the user's emotions.

[0432] 4. Searching for songs: The server searches for songs using the API of a music streaming service (e.g., the Spotipy library).

[0433] 5. Playlist Generation: The server uses the IDs of the retrieved songs to generate a new playlist that will be played within the virtual store.

[0434] 6. Providing the playlist: The server returns the name and URL of the generated playlist to the user in JSON format.

[0435] User terminal processing

[0436] 1. Input of abstract words and images: Users input words or images in text format that represent the theme or emotion they are seeking.

[0437] 2. Sending the Request: The user sends a POST request to the server with the entered words and images in JSON format.

[0438] 3. Receiving the response: The user receives the response returned from the server.

[0439] 4. Playlist Usage: Users click on the received playlist URL, play the playlist on their device, and enjoy the music experience of the virtual store.

[0440] Specific example

[0441] For example, if a user enters "I want to enjoy a relaxed shopping experience," the server receives this query and uses an emotion analysis engine to analyze the emotions "relaxed" and "happy." Next, it uses the APIs of music streaming services such as Spotify to search for songs that suit these emotions and generates a playlist named "Relaxing Shopping Playlist." The user is then provided with the URL of this playlist, which they can click to play within the virtual store. In this way, users can enjoy a richer shopping experience through music that matches their emotions and themes.

[0442] Example of a prompt

[0443] The following are examples of prompts that the user will enter into the system:

[0444] I want to enjoy a relaxed shopping experience. Please suggest some music to help me spend a pleasant time in the soft evening sunlight.

[0445] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0446] Step 1:

[0447] User input

[0448] In order to obtain a specific musical experience in a virtual store, users input their mood or theme as abstract words or images in text format. For example, they might input the phrase, "I want to enjoy a relaxed shopping experience."

[0449] Input: Text representing the user's theme or emotions.

[0450] Output: Text containing abstract words and images.

[0451] Step 2:

[0452] Send a request

[0453] The user terminal converts the input words and images into JSON format and sends a POST request to the server. The request reaches the server via the network.

[0454] Input: Text of abstract words or images

[0455] Output: Request data in JSON format

[0456] Step 3:

[0457] Receiving a request

[0458] The server receives a JSON-formatted request sent from the user's terminal.

[0459] Input: Request data in JSON format

[0460] Output: Analysis results of the request data

[0461] Step 4:

[0462] Extraction of abstract words and images

[0463] The server extracts abstract words and images (for example, "relax") from the received JSON data.

[0464] Input: Request data in JSON format

[0465] Output: Text containing abstract words and images.

[0466] Step 5:

[0467] Emotion analysis

[0468] The server performs sentiment analysis on abstract words and images. It uses a sentiment engine like TextBlob to identify the user's emotions (for example, "happiness").

[0469] Input: Text of abstract words or images

[0470] Output: Results of emotion analysis (emotion categories)

[0471] Step 6:

[0472] Search for songs

[0473] The server uses the results of sentiment analysis to search for songs using the API of a music streaming service (e.g., Spotify). The search query combines abstract words with the analyzed sentiment.

[0474] Input: Abstract words or categories of emotions

[0475] Output: Search results (list of song IDs)

[0476] Step 7:

[0477] Playlist generation

[0478] The server generates a new playlist using the acquired song ID. The playlist name might be, for example, "Relaxing Purchase Playlist".

[0479] Input: List of song IDs

[0480] Output: Information about the generated playlist (playlist ID, name)

[0481] Step 8:

[0482] Playlists provided

[0483] The server sends a JSON response to the user's device containing the name and URL of the generated playlist.

[0484] Input: Information about the generated playlist (playlist ID, name)

[0485] Output: JSON response containing playlist information

[0486] Step 9:

[0487] Received response

[0488] The user's device receives the JSON response sent back from the server and obtains the playlist URL.

[0489] Input: JSON response containing playlist information

[0490] Output: Playlist URL

[0491] Step 10:

[0492] Play playlist

[0493] The user clicks on the URL of the acquired playlist and plays the playlist within the virtual store using their device.

[0494] Input: Playlist URL

[0495] Output: A music experience where a playlist is played.

[0496] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0497] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0498] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0499] [Second Embodiment]

[0500] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0501] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0502] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0503] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0504] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0505] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0506] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0507] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0508] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0509] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0510] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0511] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0512] The embodiments for carrying out the present invention will be described in detail below. The present invention is a system that receives abstract words or images from a user, generates a music playlist based on them, and provides it to the user.

[0513] System Overview

[0514] The system works in conjunction with a server, a user's device, and a music streaming service. Based on abstract words or images entered by the user, the server uses the music streaming service's API to search for songs and generate a playlist.

[0515] Server Processing

[0516] The server is built using a web framework (e.g., Flask) and has the following functions:

[0517] 1. Receiving the request:

[0518] The server receives requests from users. These requests are sent, for example, in JSON format.

[0519] 2. Acquisition of abstract words and images:

[0520] The server extracts abstract words and images (for example, "relax" and "sunset") from the request.

[0521] 3. Search for songs:

[0522] The server uses the music streaming service's API to search for songs related to abstract words or images. It then retrieves the IDs of the relevant songs from the search results.

[0523] 4. Create a playlist:

[0524] The server uses the ID of the retrieved song to generate a new playlist. The playlist name might be, for example, "Playlist for Relaxation and Sunsets."

[0525] 5. Add songs to the playlist:

[0526] The server adds songs obtained from the search results to the generated playlist.

[0527] 6. Generating the response:

[0528] The server generates a response containing the name and URL of the created playlist and returns it to the user.

[0529] User processing

[0530] 1. Input abstract words or images:

[0531] Users input words or images that represent the themes or emotions they are seeking.

[0532] 2. Submitting a request:

[0533] The user enters words and images, converts them to JSON format, and sends a POST request to the server.

[0534] 3. Receiving the response:

[0535] The user receives a response from the server. The response includes the name and URL of the generated playlist.

[0536] 4. Using playlists:

[0537] The user clicks the received playlist URL and plays the playlist on their music streaming service.

[0538] Specific example

[0539] For example, if a user enters abstract words like "relax" and "sunset," the server will search for music based on these words and generate a playlist named "Playlist for Relaxation and Sunset." The playlist will contain the selected music and be provided to the user via a URL. By clicking this URL, the user can play the playlist and enjoy a relaxing atmosphere.

[0540] This invention allows users to easily obtain music playlists based on abstract images, which can be particularly helpful for users unfamiliar with music selection or when preparing for special events.

[0541] The following describes the processing flow.

[0542] Step 1:

[0543] The user opens a webpage or application using their device and enters abstract words or images. These inputs may include words such as "relax" or "sunset."

[0544] Step 2:

[0545] The system converts abstract words and images entered by the user into JSON format and sends a POST request to the server.

[0546] Step 3:

[0547] The server receives a POST request. A web framework such as Flask is used to parse the request.

[0548] Step 4:

[0549] The server extracts abstract words and images from the request. Specifically, it retrieves the `terms` field from the JSON data in the request body.

[0550] Step 5:

[0551] The server uses the API of a music streaming service to search for songs based on extracted words and images. This uses libraries such as Spotipy.

[0552] Step 6:

[0553] The server analyzes the search results returned from the music streaming service and compiles a list of the IDs of the relevant songs.

[0554] Step 7:

[0555] The server uses the music streaming service's API to generate a new playlist. The playlist name might be, for example, "Playlist for Relaxation and Sunsets."

[0556] Step 8:

[0557] The song is added to the playlist generated by the server using the song ID obtained in step 6.

[0558] Step 9:

[0559] Prepare a response in JSON format containing the name and URL of the playlist generated by the server, and return it to the user.

[0560] Step 10:

[0561] The user receives a response from the server. The response includes the playlist name and URL.

[0562] Step 11:

[0563] The user clicks the response URL and plays the playlist on their music streaming service. The user can then enjoy the generated playlist.

[0564] (Example 1)

[0565] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0566] Traditionally, manually selecting music playlists that matched users' emotions or themes was time-consuming and laborious. Furthermore, finding appropriate music was difficult for users unfamiliar with music selection or when preparing for special events. This invention aims to solve this problem by providing a system that allows users to easily obtain music playlists based on abstract words or images.

[0567] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0568] In this invention, the server includes means for receiving abstract words or images from a user, means for searching for music based on the received abstract words or images, means for providing the user with a playlist generated from the search results, means for receiving and analyzing user requests through a web framework, means for searching for relevant music using a music streaming service API, means for generating a playlist from the search results and adding music, and means for generating the name and URL of the generated playlist as a response. This makes it possible for users to easily and quickly obtain music playlists that match their mood or theme.

[0569] A "user" is a person or group that uses the system to generate music playlists based on abstract words or images.

[0570] "Abstract words" are words that express emotions or themes, rather than referring to specific objects or actions.

[0571] An "image" is a concept or theme expressed using visual or non-visual means.

[0572] "Means" refer to a set of processes or functions performed to achieve a specific objective.

[0573] A "musical piece" refers to a musical work, primarily offered through music streaming services.

[0574] "Searching" is the act of finding information based on specific criteria.

[0575] A "playlist" is a list of songs that are played in a specific order.

[0576] A "web framework" is a software framework for developing web applications.

[0577] A "request" refers to a request made from a client (the user's device) to a server.

[0578] "JSON format" is an abbreviation for JavaScript Object Notation, and it is a lightweight text format for representing data.

[0579] "API" stands for Application Programming Interface, and it is an interface that allows external parties to access the functions of software.

[0580] "Response" refers to the answer that a server returns to a client in response to a request.

[0581] "URL" stands for Uniform Resource Locator, and it is an address that indicates the location of a resource on the web.

[0582] A "network" is a system that connects multiple information devices to communicate data.

[0583] "User's device" refers to a device that the user directly operates (such as a smartphone or personal computer).

[0584] A "music streaming service" is a service that provides music in real time via the internet.

[0585] The embodiments for carrying out the present invention will be described in detail below. The present invention is a system that receives abstract words or images from a user, generates a music playlist based on them, and provides it to the user.

[0586] System Overview

[0587] The system operates in conjunction with a server, user terminals, and a music streaming service. The server is built using a web framework, specifically one such as Flask. Based on abstract words or images entered by the user, the server searches for songs using the music streaming service's API and generates a playlist.

[0588] Server Processing

[0589] The server has the following functions:

[0590] 1. Receiving the request:

[0591] The server receives requests from users. These requests are sent, for example, in JSON format.

[0592] 2. Acquisition of abstract words and images:

[0593] The server extracts abstract words and images from the request. Examples include the word "relax" and an image like "sunset."

[0594] 3. Search for songs:

[0595] The server uses the music streaming service's API to search for songs related to abstract words or images. It then retrieves the IDs of the relevant songs from the search results.

[0596] 4. Create a playlist:

[0597] The server uses the IDs of the retrieved songs to generate a new playlist. The playlist name might be, for example, "Playlist for Relaxation and Sunsets."

[0598] 5. Add songs to the playlist:

[0599] The server adds songs obtained from the search results to the generated playlist.

[0600] 6. Generating the response:

[0601] The server generates a response containing the name and URL of the created playlist and returns it to the user.

[0602] User processing

[0603] 1. Input abstract words or images:

[0604] Users use their devices to input words or images that represent the themes or emotions they desire.

[0605] 2. Submitting a request:

[0606] The user enters words and images, converts them to JSON format, and sends a POST request to the server.

[0607] 3. Receiving the response:

[0608] The user receives a response from the server. The response includes the name and URL of the generated playlist.

[0609] 4. Using playlists:

[0610] The user clicks the received playlist URL and plays the playlist on their music streaming service.

[0611] Specific example

[0612] For example, if a user enters the words "relax" and "sunset," the server will search for songs based on these words and generate a playlist named "Playlist for Relaxation and Sunset." The playlist will contain the selected songs. The user can play this playlist and enjoy a relaxing atmosphere by clicking on the URL provided by the server.

[0613] Example of a prompt

[0614] An example of a prompt for a generative AI model is, "Generate a playlist with the themes of relaxation and sunsets. Please select songs that fit the theme."

[0615] This system allows users to easily obtain music playlists based on abstract images. This feature will be particularly useful for users who struggle with selecting music or when preparing for special events.

[0616] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0617] Step 1:

[0618] The server receives a JSON-formatted request sent from the user's terminal. The input includes abstract words and images entered by the user. Specifically, the server retrieves the JSON data using request.get_json(). The output is the retrieved JSON data, which is then passed to the next step.

[0619] Step 2:

[0620] The server parses the received JSON data and extracts abstract words and images entered by the user. The JSON data obtained in step 1 is used as input. Specifically, the data is parsed using `data = request.json`, and abstract words and images are extracted using `keywords = data['keywords']`. The extracted words and images are then passed to the next step as output.

[0621] Step 3:

[0622] The server uses the music streaming service's API to search for songs related to abstract words and images. The words and images extracted in step 2 are used as input. Specifically, the API is called using requests.get('https: / / api.music-service.com / v1 / search', headers=headers, params=params) to obtain the relevant song IDs. The obtained song IDs are then passed to the next step as output.

[0623] Step 4:

[0624] The server generates a new playlist based on the retrieved song ID. The song ID obtained in step 3 is used as input. Specifically, it creates the playlist by executing requests.post('https: / / api.music-service.com / v1 / users / {user_id} / playlists', headers=headers, json=playlist_data). The generated playlist ID is passed to the next step as output.

[0625] Step 5:

[0626] The server adds songs related to the generated playlist. The inputs used are the playlist ID generated in step 4 and the song IDs obtained in step 3. Specifically, it executes requests.post('https: / / api.music-service.com / v1 / playlists / {playlist_id} / tracks', headers=headers, json=tracks_data) to add the songs to the playlist. The output is the playlist URL, which is passed to the next step.

[0627] Step 6:

[0628] The server generates a response containing the name and URL of the generated playlist and returns it to the user. The input used is the playlist URL obtained in step 5. Specifically, the response data is constructed using response_data = {'playlist_name': playlist_name, 'url': playlist_url}, and the response is returned to the user using return jsonify(response_data). The output provides the user with information about the generated playlist.

[0629] Step 7:

[0630] The user receives a playlist URL from the server using their device. The server's response is used as input. Specifically, the response.json() function is used to parse the response data. The playlist URL is displayed on the user's device as output.

[0631] Step 8:

[0632] The user clicks the received playlist URL and plays the playlist on their music streaming service. The input used is the playlist URL received in step 7. Specifically, the user opens the URL using a browser or music streaming app and plays the playlist. The output is the playback of the song the user requested.

[0633] (Application Example 1)

[0634] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0635] Traditional music playlist generation systems assumed users were familiar with specific songs and artists, and lacked the ability to automatically generate playlists from abstract words or visual information. As a result, users had the problem of spending a lot of time finding music that suited their mood or situation. Furthermore, if users wanted to express a specific atmosphere or situation, they had to manually search for songs, which was a time-consuming process.

[0636] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0637] In this invention, the server includes means for receiving abstract words and visual information from a user, means for searching for music based on the received abstract words and visual information, means for providing the user with a playlist generated from the search results, means for creating a playlist using the API of a music provider, and means for returning the URL of the provider to the user. This makes it possible for the user to instantly obtain the optimal playlist based on abstract words and visual information.

[0638] A "user" is an individual or group that uses the system.

[0639] "Abstract words" are words that express concepts or emotions rather than concrete things.

[0640] "Visual information" refers to data related to vision, such as images and videos.

[0641] "Means" refer to methods or devices used to achieve a specific objective.

[0642] A "musical piece" refers to the individual works that make up a piece of music.

[0643] "Searching" refers to the act of finding specific information.

[0644] A "playlist" is a list of songs that are organized according to a specific order or theme.

[0645] "Providing" refers to the act of handing over a certain item or service to another person.

[0646] A "server" is a computer that provides services to other computers via a network.

[0647] A "network" is a system in which multiple computers are connected to each other and can exchange information.

[0648] "JSON formatted data" refers to data structured in JavaScript Object Notation format.

[0649] An "external music provider API" refers to an application programming interface provided by an external music distribution service.

[0650] This invention is a system that receives abstract words and visual information from a user, generates a music playlist based on that information, and provides it to the user. Specific embodiments for carrying out this invention are described in detail below.

[0651] The system operates in conjunction with the user's terminal, the server, and the music provider's API.

[0652] System Configuration

[0653] server

[0654] The server primarily has the following functions:

[0655] Request received:

[0656] The server receives requests from users that include abstract words and visual information. These requests are sent in JSON format.

[0657] Acquisition of abstract words and visual information:

[0658] The server parses the received JSON data and extracts abstract words and visual information. In this process, natural language processing (NLP) is used to process abstract words, and image processing is used to analyze visual information.

[0659] Search for songs:

[0660] The server uses the API of the music provider (e.g., Spotify) to search for songs related to abstract words or visual information. It then retrieves the IDs of the relevant songs.

[0661] Playlist generation:

[0662] A new playlist is generated based on the acquired song IDs. The playlist name is determined based on abstract words or visual information.

[0663] Add songs to the playlist:

[0664] Add songs obtained from search results to the playlist you created.

[0665] Response generation:

[0666] Generate a response containing the name and URL of the generated playlist, and send it back to the user.

[0667] User terminal

[0668] The user terminal has the following functions:

[0669] Inputting abstract words and visual information:

[0670] The user inputs abstract words and visual information to generate a playlist. This input is done through the application.

[0671] Sending a request:

[0672] The user terminal sends a request containing abstract words and visual information to the server in JSON format.

[0673] Response received:

[0674] Receive the response sent back from the server and display the information of the generated playlist.

[0675] Specific example

[0676] For example, if a user enters an abstract phrase like "a refreshing morning walk" and uploads a morning landscape image as visual information, the server analyzes these to search for relevant music. It then generates a playlist with a name like "Playlist for a refreshing morning walk" and provides the user with its URL. By clicking the provided URL, the user can listen to the appropriate music.

[0677] Example of a prompt

[0678] Abstract words: A refreshing morning walk

[0679] Visual information: Morning landscape image

[0680] This allows users to easily obtain music playlists that match their desired themes and emotions. This invention is particularly useful for users who are unfamiliar with music selection and for creating the right atmosphere for special events.

[0681] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0682] Step 1:

[0683] The user inputs abstract words or visual information into the application.

[0684] Input: Abstract words entered by the user (e.g., "A refreshing morning walk") and visual information (e.g., an image of a morning landscape).

[0685] Output: Request data in JSON format.

[0686] Specific operation: The smartphone app receives user input, converts it into JSON data, and prepares to send it to the server.

[0687] Step 2:

[0688] The user's terminal sends a request in JSON format to the server.

[0689] Input: Request data in JSON format entered by the user.

[0690] Output: Request data sent to the server.

[0691] Specific operation: The user's terminal sends JSON-formatted request data to the server via the network.

[0692] Step 3:

[0693] The server receives the request and extracts abstract words and visual information.

[0694] Input: Request data in JSON format.

[0695] Output: Abstract words and visual information.

[0696] Specific operation: The server parses the request, analyzes abstract words using an NLP model, and analyzes visual information using image processing (e.g., OpenCV).

[0697] Step 4:

[0698] The server calls the music provider's API based on the analysis results and searches for related songs.

[0699] Input: Keywords and parameters based on abstract words and visual information.

[0700] Output: A list of IDs for related songs.

[0701] Specific operation: The server calls the music provider's API (e.g., Spotify API), searches for and retrieves relevant songs based on the analysis results.

[0702] Step 5:

[0703] The server generates a new playlist based on the song ID it retrieves.

[0704] Input: List of song IDs.

[0705] Output: The ID and URL of the newly generated playlist.

[0706] Specific operation: The server calls the music provider's API again, creates a new playlist, and adds the retrieved songs to that playlist.

[0707] Step 6:

[0708] The server sends the generated playlist information back to the user.

[0709] Input: Playlist ID and URL.

[0710] Output: JSON-formatted response data sent to the user's terminal.

[0711] Specific operation: The server generates a JSON response containing the playlist name and URL, and sends it to the user's terminal.

[0712] Step 7:

[0713] The user's terminal receives the response and displays the information of the generated playlist.

[0714] Input: JSON response data from the server.

[0715] Output: The name and URL of the playlist to be displayed.

[0716] Specific operation: The user's terminal parses the response received from the server and displays the playlist name and URL to the user.

[0717] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0718] The embodiments for carrying out the present invention will be described in detail below. The present invention combines a system that receives abstract words or images from a user, generates a music playlist based on them, and provides it to the user, with an emotion engine that recognizes the user's emotions.

[0719] System Overview

[0720] The system works in conjunction with a server, a user's device, and a music streaming service. Based on abstract words, images, and emotions entered by the user, the server uses the music streaming service's API to search for songs and generate a playlist.

[0721] Server Processing

[0722] The server is built using a web framework (e.g., Flask) and has the following functions:

[0723] 1. Receiving the request:

[0724] The server receives requests from users. These requests are sent, for example, in JSON format.

[0725] 2. Acquisition of abstract words and images:

[0726] The server extracts abstract words and images (for example, "relax" and "sunset") from the request.

[0727] 3. Emotional analysis using an emotion engine:

[0728] The server uses an emotion engine to analyze user emotions based on their input. The emotion engine uses text-based analysis techniques to identify the user's emotions (e.g., happiness, sadness, excitement).

[0729] 4. Search for songs:

[0730] The server uses music streaming service APIs to search for songs based on abstract words, images, and sentiment analysis results. This uses libraries such as Spotipy.

[0731] 5. Create a playlist:

[0732] The server uses the ID of the retrieved song to generate a new playlist. The playlist name might be, for example, "Playlist for Relaxation and Sunsets."

[0733] 6. Add songs to the playlist:

[0734] The server adds songs to the generated playlist using the song IDs obtained in step 4.

[0735] 7. Generating the response:

[0736] The server prepares a response in JSON format, including the name and URL of the generated playlist, and returns it to the user.

[0737] User processing

[0738] 1. Input abstract words or images:

[0739] Users input words or images that represent the themes or emotions they are seeking.

[0740] 2. Submitting a request:

[0741] The user enters words and images, converts them to JSON format, and sends a POST request to the server.

[0742] 3. Receiving the response:

[0743] The user receives a response from the server. The response includes the name and URL of the generated playlist.

[0744] 4. Using playlists:

[0745] The user clicks the received playlist URL and plays the playlist on their music streaming service.

[0746] Specific example

[0747] For example, if a user enters abstract words like "relax" and "sunset," and the emotion engine interprets these words as "happiness," the server will search for songs based on this and generate a playlist named "Playlist for Relaxation and Sunset." The playlist contains the selected songs and is provided to the user via a URL. By clicking this URL, the user can play the playlist and enjoy a relaxed and happy atmosphere.

[0748] This invention allows users to easily access music playlists based on abstract images and emotions, which can be particularly helpful for users unfamiliar with music selection or when preparing for special events.

[0749] The following describes the processing flow.

[0750] Step 1:

[0751] The user opens a webpage or application using their device and enters an abstract word or image into a text field. This input could include, for example, "relax" or "sunset."

[0752] Step 2:

[0753] The system sends JSON data containing abstract words and images entered by the user to the server as a POST request.

[0754] Step 3:

[0755] The server receives a POST request and extracts JSON data from the request body. Specifically, the Flask framework is used to parse the data from the request.

[0756] Step 4:

[0757] The server retrieves abstract words and images from the JSON data (for example, "relax sunset" contained in the terms field).

[0758] Step 5:

[0759] The server uses an emotion engine to analyze abstract words and images to identify the user's emotions. For example, the words "relax" and "sunset" might be classified as the emotion "happiness."

[0760] Step 6:

[0761] The server uses the music streaming service's API to search for songs based on identified emotions and abstract words. It uses the Spotipy library to submit search queries.

[0762] Step 7:

[0763] The server analyzes the search results returned from the music streaming service and compiles a list of the IDs of related songs. For example, the list might contain the IDs of 10 songs.

[0764] Step 8:

[0765] The server uses the music streaming service's API to generate a new playlist. The playlist will be named "Playlist for Relaxation and Sunsets".

[0766] Step 9:

[0767] Add songs to the playlist generated by the server using the song IDs obtained in step 7. Specifically, add all song IDs to the playlist.

[0768] Step 10:

[0769] Prepare a response in JSON format containing the name and URL of the playlist generated by the server, and return it to the user.

[0770] Step 11:

[0771] The user receives a response from the server and checks its contents. The response includes the name and URL of the generated playlist.

[0772] Step 12:

[0773] The user clicks the response URL and opens the playlist on their music streaming service. When the playlist is played, songs based on the user's selected emotions and images are played.

[0774] (Example 2)

[0775] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0776] Conventional music playlist generation systems have been unable to adequately respond to users' abstract words and images, making it difficult to provide playlists that align with their emotions and preferences. Furthermore, there were no systems that combined emotional analysis to address the diverse needs of users.

[0777] The identification processing performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for receiving abstract words or images from the user, means for analyzing emotions from the received abstract words or images, means for searching for music based on the analyzed emotions and abstract words or images, and means for providing the user with a playlist generated from the search results. This makes it possible to analyze the user's emotions based on their abstract input and automatically generate and provide a music playlist that suits them.

[0778] "Abstract words and images" are words or visual concepts used to express a state or feeling that is difficult for the user to describe as concretely as possible.

[0779] "Methods for analyzing emotions" refer to technologies and algorithms used to infer and identify a user's emotions from the words and images they input. This includes natural language processing and machine learning models.

[0780] "Methods for searching for music" refers to technologies and methods that use external music databases or music streaming service APIs to find songs that match specific keywords or emotions.

[0781] "Methods for generating playlists" refer to technologies and methods that appropriately combine searched songs to create a single, continuous music list. This makes it possible to play music in sequence according to a specific theme or emotion.

[0782] "Means of receiving" refers to interfaces and communication protocols for receiving data from external sources via a network. This includes web frameworks that process HTTP requests.

[0783] "JSON format data" is an abbreviation for JavaScript Object Notation (JSON), a lightweight data exchange format. It represents data in a format that is easy for humans to read and easy for machines to parse.

[0784] Modes for carrying out the invention

[0785] The embodiments for carrying out the present invention will be described in detail below. The present invention combines a system that receives abstract words or images from a user, generates a music playlist based on them, and provides it to the user, with an emotion engine that recognizes the user's emotions.

[0786] The system operates in conjunction with a server, user terminals, and music streaming services. This will be explained below, along with specific hardware and software implementations.

[0787] Server Processing

[0788] The server is built using a web framework (e.g., Flask). The main processing steps are as follows:

[0789] 1. Receiving the request:

[0790] The server receives requests from users. This is done by users sending data containing abstract keywords and images in JSON format through their devices. For example, a user might enter the keywords "relax" and "sunset."

[0791] 2. Acquisition of abstract words and images:

[0792] The server parses the received JSON data and extracts the words and images entered by the user. This process uses the json module from Python's standard library.

[0793] 3. Emotional analysis using an emotion engine:

[0794] The server uses the Hugging Face Transformers library to analyze emotions from the user's input text. This analysis associates the keywords "relax" and "sunset" with the emotion "happiness." The emotion engine uses a pre-trained model to determine the emotion of the input text.

[0795] 4. Search for songs:

[0796] The server uses the Spotipy library to access the Spotify API and search for songs based on the keywords "relax," "sunset," and "happiness." This search query targets Spotify's song database to retrieve the appropriate song ID.

[0797] 5. Create a playlist:

[0798] The server generates a new playlist using the retrieved song ID. The playlist name will be based on the keywords entered by the user, such as "Playlist for Relaxation and Sunsets." The generated playlist will then be linked to the user's Spotify account.

[0799] 6. Add songs to the playlist:

[0800] The server adds songs to the generated playlist. This process uses the Spotipy library to send song addition requests to the Spotify API.

[0801] 7. Generating the response:

[0802] The server creates a JSON response containing the name of the generated playlist and its URL. For example, it might be returned to the user using Flask's response functionality. The JSON response includes data such as:

[0803] json

[0804] {

[0805] "playlist_name": "Playlist for relaxation and sunsets",

[0806] "playlist_url": "https: / / open.spotify.com / playlist / ..."

[0807] }

[0808] User processing

[0809] 1. Input abstract words or images:

[0810] Users input words or images that represent the themes or emotions they are seeking. For example, they might use words like "relax" or "sunset."

[0811] 2. Submitting a request:

[0812] The user enters words and images, converts them to JSON format, and sends a POST request to the server.

[0813] 3. Receiving the response:

[0814] The user receives a response from the server. The response includes the name and URL of the generated playlist.

[0815] 4. Using playlists:

[0816] The user clicks the received playlist URL and plays the playlist on their music streaming service.

[0817] Specific example

[0818] For example, if a user enters abstract words like "relax" and "sunset," and the emotion engine interprets these words as "happiness," the server will search for songs based on this and generate a playlist named "Playlist for Relaxation and Sunset." The playlist contains the selected songs and is provided to the user via a URL. By clicking this URL, the user can play the playlist and enjoy a relaxed and happy atmosphere.

[0819] Example of a prompt

[0820] "Please create a playlist for relaxation and sunsets."

[0821] Based on this input, the generative AI model performs sentiment analysis based on the user's abstract words, selects appropriate songs, and creates a playlist.

[0822] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0823] Step 1:

[0824] The user inputs abstract words or images.

[0825] Detailed explanation:

[0826] Users input abstract words or images that represent a theme or emotion using web forms or mobile apps from their own devices. For example, they might use words like "relax" or "sunset."

[0827] Inputs and outputs:

[0828] Input: Abstract words or images entered by the user (e.g., "relax," "sunset").

[0829] Output: The terminal generates data in JSON format.

[0830] Step 2:

[0831] The terminal sends the generated JSON data to the server.

[0832] Detailed explanation:

[0833] The terminal converts the words and images entered by the user into JSON format and sends them to a specific endpoint on the server as an HTTP POST request.

[0834] Inputs and outputs:

[0835] Input: JSON data containing abstract words or images entered by the user.

[0836] Output: JSON data received by the server.

[0837] Step 3:

[0838] The server parses the JSON data and extracts abstract words and images.

[0839] Detailed explanation:

[0840] The server uses the Flask web framework to parse the received JSON data and extract the words and images entered by the user. This process uses Python's json module.

[0841] Inputs and outputs:

[0842] Input: JSON data received by the server.

[0843] Output: Abstract words or images (e.g., "relax," "sunset").

[0844] Step 4:

[0845] The server uses an emotion engine to analyze the user's emotions based on their input.

[0846] Detailed explanation:

[0847] The server uses the Hugging Face Transformers library to analyze the sentiment of the user's input text. This analysis associates "relaxed" and "sunset" with the emotion of "happiness."

[0848] Inputs and outputs:

[0849] Input: Abstract words or images (e.g., "relax," "sunset").

[0850] Output: Analyzed emotion (e.g., "happiness").

[0851] Step 5:

[0852] The server uses the music streaming service's API to search for songs.

[0853] Detailed explanation:

[0854] The server uses the Spotipy library to access the Spotify API and search for songs based on the keywords "relax," "sunset," and "happiness." This search query targets Spotify's song database to retrieve the appropriate song ID.

[0855] Inputs and outputs:

[0856] Input: Abstract words, images, or analyzed emotions (e.g., "relaxed," "sunset," "happy").

[0857] Output: A list of searched song IDs.

[0858] Step 6:

[0859] The server generates a playlist using the acquired song ID.

[0860] Detailed explanation:

[0861] The server uses the Spotipy library to generate a new playlist via the Spotify API. The playlist name will be "Playlist for Relaxation and Sunsets," based on keywords entered by the user.

[0862] Inputs and outputs:

[0863] Input: A list of searched song IDs.

[0864] Output: The ID and name of the generated playlist.

[0865] Step 7:

[0866] Add songs to the playlist generated by the server.

[0867] Detailed explanation:

[0868] The server adds the searched songs to the generated playlist. This process uses the Spotipy library to send song addition requests to the Spotify API.

[0869] Inputs and outputs:

[0870] Input: The ID of the generated playlist and a list of the searched song IDs.

[0871] Output: A playlist with the added songs.

[0872] Step 8:

[0873] The server prepares a JSON response containing the name and URL of the generated playlist and returns it to the user.

[0874] Detailed explanation:

[0875] The server creates a JSON response containing the name and URL of the generated playlist and returns it to the user. This response is returned using Flask's response functionality.

[0876] Inputs and outputs:

[0877] Input: The ID and name of the generated playlist.

[0878] Output: JSON response containing the playlist name and URL.

[0879] Step 9:

[0880] The user receives the playlist URL returned from the server and plays it.

[0881] Detailed explanation:

[0882] The user receives a response from the server via their device and clicks the provided playlist URL. This allows them to play the playlist on their music streaming service.

[0883] Inputs and outputs:

[0884] Input: JSON response containing the playlist name and URL.

[0885] Output: The music playlist that the user will play.

[0886] (Application Example 2)

[0887] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0888] In modern virtual stores and online shopping experiences, there is a lack of personalized service based on users' emotions and moods. In particular, the selection of music is often generic rather than tailored to an individual's mood or theme, which degrades the quality of the user's purchasing experience.

[0889] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0890] In this invention, the server includes means for receiving abstract words or images from the user, means for searching for music based on the received abstract words or images and the user's emotions, means for providing the user with a playlist generated from the search results, and means for automatically generating and providing music playlists to be played within the virtual store. This makes it possible to improve the quality of online shopping through a personalized music experience based on the user's emotions and mood.

[0891] "Abstract words and images" refer to vague words or visual concepts used to express the feelings or themes that the user is experiencing.

[0892] "Emotions" refer to the user's psychological and sensory state and are classified into emotional categories such as happiness, sadness, and excitement.

[0893] "Searching for music" refers to the process of finding appropriate songs from the music streaming service's database based on user input.

[0894] A "playlist" refers to a collection of songs selected based on a specific theme or emotion, and is a list of music arranged to be played sequentially.

[0895] A "virtual store" is a virtual commercial space operated on the internet where users can browse and purchase products online.

[0896] "Automatic generation" refers to the process by which a system creates specific deliverables or data based on user input and algorithms, without requiring manual operation.

[0897] "To provide" refers to giving information or services to users so that they can use them.

[0898] This invention is a system that receives abstract words or images from a user, generates a music playlist based on them, and plays it within a virtual store. The following describes in detail the configurations for implementing this system.

[0899] System Configuration

[0900] The system operates in conjunction with servers, user terminals, and music streaming services.

[0901] Server Processing

[0902] The server is built using a web framework (e.g., Flask) and includes the following software components:

[0903] 1. Receiving the Request: The server receives a request from the user. The request is sent in JSON format.

[0904] 2. Retrieving abstract words and images: The server extracts abstract words and images from the request. Examples include "relax" and "sunset."

[0905] 3. Sentiment Analysis: The server uses an emotion engine to analyze the user's emotions. The emotion engine utilizes text analysis libraries such as TextBlob to identify the user's emotions.

[0906] 4. Searching for songs: The server searches for songs using the API of a music streaming service (e.g., the Spotipy library).

[0907] 5. Playlist Generation: The server uses the IDs of the retrieved songs to generate a new playlist that will be played within the virtual store.

[0908] 6. Providing the playlist: The server returns the name and URL of the generated playlist to the user in JSON format.

[0909] User terminal processing

[0910] 1. Input of abstract words and images: Users input words or images in text format that represent the theme or emotion they are seeking.

[0911] 2. Sending the Request: The user sends a POST request to the server with the entered words and images in JSON format.

[0912] 3. Receiving the response: The user receives the response returned from the server.

[0913] 4. Playlist Usage: Users click on the received playlist URL, play the playlist on their device, and enjoy the music experience of the virtual store.

[0914] Specific example

[0915] For example, if a user enters "I want to enjoy a relaxed shopping experience," the server receives this query and uses an emotion analysis engine to analyze the emotions "relaxed" and "happy." Next, it uses the APIs of music streaming services such as Spotify to search for songs that suit these emotions and generates a playlist named "Relaxing Shopping Playlist." The user is then provided with the URL of this playlist, which they can click to play within the virtual store. In this way, users can enjoy a richer shopping experience through music that matches their emotions and themes.

[0916] Example of a prompt

[0917] The following are examples of prompts that the user will enter into the system:

[0918] I want to enjoy a relaxed shopping experience. Please suggest some music to help me spend a pleasant time in the soft evening sunlight.

[0919] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0920] Step 1:

[0921] User input

[0922] In order to obtain a specific musical experience in a virtual store, users input their mood or theme as abstract words or images in text format. For example, they might input the phrase, "I want to enjoy a relaxed shopping experience."

[0923] Input: Text representing the user's theme or emotions.

[0924] Output: Text containing abstract words and images.

[0925] Step 2:

[0926] Send a request

[0927] The user terminal converts the input words and images into JSON format and sends a POST request to the server. The request reaches the server via the network.

[0928] Input: Text of abstract words or images

[0929] Output: Request data in JSON format

[0930] Step 3:

[0931] Receiving a request

[0932] The server receives a JSON-formatted request sent from the user's terminal.

[0933] Input: Request data in JSON format

[0934] Output: Analysis results of the request data

[0935] Step 4:

[0936] Extraction of abstract words and images

[0937] The server extracts abstract words and images (for example, "relax") from the received JSON data.

[0938] Input: Request data in JSON format

[0939] Output: Text containing abstract words and images.

[0940] Step 5:

[0941] Emotion analysis

[0942] The server performs sentiment analysis on abstract words and images. It uses a sentiment engine like TextBlob to identify the user's emotions (for example, "happiness").

[0943] Input: Text of abstract words or images

[0944] Output: Results of emotion analysis (emotion categories)

[0945] Step 6:

[0946] Search for songs

[0947] The server uses the results of sentiment analysis to search for songs using the API of a music streaming service (e.g., Spotify). The search query combines abstract words with the analyzed sentiment.

[0948] Input: Abstract words or categories of emotions

[0949] Output: Search results (list of song IDs)

[0950] Step 7:

[0951] Playlist generation

[0952] The server generates a new playlist using the acquired song ID. The playlist name might be, for example, "Relaxing Purchase Playlist".

[0953] Input: List of song IDs

[0954] Output: Information about the generated playlist (playlist ID, name)

[0955] Step 8:

[0956] Playlists provided

[0957] The server sends a JSON response to the user's device containing the name and URL of the generated playlist.

[0958] Input: Information about the generated playlist (playlist ID, name)

[0959] Output: JSON response containing playlist information

[0960] Step 9:

[0961] Received response

[0962] The user's device receives the JSON response sent back from the server and obtains the playlist URL.

[0963] Input: JSON response containing playlist information

[0964] Output: Playlist URL

[0965] Step 10:

[0966] Play playlist

[0967] The user clicks on the URL of the acquired playlist and plays the playlist within the virtual store using their device.

[0968] Input: Playlist URL

[0969] Output: A music experience where a playlist is played.

[0970] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0971] The data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of the data generation model 58 is ChatGPT (Internet Search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0972] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0973] [Third Embodiment]

[0974] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0975] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0976] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0977] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0978] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0979] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0980] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0981] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0982] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0983] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0984] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0985] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0986] The embodiments for carrying out the present invention will be described in detail below. The present invention is a system that receives abstract words or images from a user, generates a music playlist based on them, and provides it to the user.

[0987] System Overview

[0988] The system works in conjunction with a server, a user's device, and a music streaming service. Based on abstract words or images entered by the user, the server uses the music streaming service's API to search for songs and generate a playlist.

[0989] Server Processing

[0990] The server is built using a web framework (e.g., Flask) and has the following functions:

[0991] 1. Receiving the request:

[0992] The server receives requests from users. These requests are sent, for example, in JSON format.

[0993] 2. Acquisition of abstract words and images:

[0994] The server extracts abstract words and images (for example, "relax" and "sunset") from the request.

[0995] 3. Search for songs:

[0996] The server uses the music streaming service's API to search for songs related to abstract words or images. It then retrieves the IDs of the relevant songs from the search results.

[0997] 4. Create a playlist:

[0998] The server uses the ID of the retrieved song to generate a new playlist. The playlist name might be, for example, "Playlist for Relaxation and Sunsets."

[0999] 5. Add songs to the playlist:

[1000] The server adds songs obtained from the search results to the generated playlist.

[1001] 6. Generating the response:

[1002] The server generates a response containing the name and URL of the created playlist and returns it to the user.

[1003] User processing

[1004] 1. Input abstract words or images:

[1005] Users input words or images that represent the themes or emotions they are seeking.

[1006] 2. Submitting a request:

[1007] The user enters words and images, converts them to JSON format, and sends a POST request to the server.

[1008] 3. Receiving the response:

[1009] The user receives a response from the server. The response includes the name and URL of the generated playlist.

[1010] 4. Using playlists:

[1011] The user clicks the received playlist URL and plays the playlist on their music streaming service.

[1012] Specific example

[1013] For example, if a user enters abstract words like "relax" and "sunset," the server will search for music based on these words and generate a playlist named "Playlist for Relaxation and Sunset." The playlist will contain the selected music and be provided to the user via a URL. By clicking this URL, the user can play the playlist and enjoy a relaxing atmosphere.

[1014] This invention allows users to easily obtain music playlists based on abstract images, which can be particularly helpful for users unfamiliar with music selection or when preparing for special events.

[1015] The following describes the processing flow.

[1016] Step 1:

[1017] The user opens a webpage or application using their device and enters abstract words or images. These inputs may include words such as "relax" or "sunset."

[1018] Step 2:

[1019] The system converts abstract words and images entered by the user into JSON format and sends a POST request to the server.

[1020] Step 3:

[1021] The server receives a POST request. A web framework such as Flask is used to parse the request.

[1022] Step 4:

[1023] The server extracts abstract words and images from the request. Specifically, it retrieves the `terms` field from the JSON data in the request body.

[1024] Step 5:

[1025] The server uses the API of a music streaming service to search for songs based on extracted words and images. This uses libraries such as Spotipy.

[1026] Step 6:

[1027] The server analyzes the search results returned from the music streaming service and compiles a list of the IDs of the relevant songs.

[1028] Step 7:

[1029] The server uses the music streaming service's API to generate a new playlist. The playlist name might be, for example, "Playlist for Relaxation and Sunsets."

[1030] Step 8:

[1031] The song is added to the playlist generated by the server using the song ID obtained in step 6.

[1032] Step 9:

[1033] Prepare a response in JSON format containing the name and URL of the playlist generated by the server, and return it to the user.

[1034] Step 10:

[1035] The user receives a response from the server. The response includes the playlist name and URL.

[1036] Step 11:

[1037] The user clicks the response URL and plays the playlist on their music streaming service. The user can then enjoy the generated playlist.

[1038] (Example 1)

[1039] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1040] Traditionally, manually selecting music playlists that matched users' emotions or themes was time-consuming and laborious. Furthermore, finding appropriate music was difficult for users unfamiliar with music selection or when preparing for special events. This invention aims to solve this problem by providing a system that allows users to easily obtain music playlists based on abstract words or images.

[1041] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1042] In this invention, the server includes means for receiving abstract words or images from a user, means for searching for music based on the received abstract words or images, means for providing the user with a playlist generated from the search results, means for receiving and analyzing user requests through a web framework, means for searching for relevant music using a music streaming service API, means for generating a playlist from the search results and adding music, and means for generating the name and URL of the generated playlist as a response. This makes it possible for users to easily and quickly obtain music playlists that match their mood or theme.

[1043] A "user" is a person or group that uses the system to generate music playlists based on abstract words or images.

[1044] "Abstract words" are words that express emotions or themes, rather than referring to specific objects or actions.

[1045] An "image" is a concept or theme expressed using visual or non-visual means.

[1046] "Means" refer to a set of processes or functions performed to achieve a specific objective.

[1047] A "musical piece" refers to a musical work, primarily offered through music streaming services.

[1048] "Searching" is the act of finding information based on specific criteria.

[1049] A "playlist" is a list of songs that are played in a specific order.

[1050] A "web framework" is a software framework for developing web applications.

[1051] A "request" refers to a request made from a client (the user's device) to a server.

[1052] "JSON format" is an abbreviation for JavaScript Object Notation, and it is a lightweight text format for representing data.

[1053] "API" stands for Application Programming Interface, and it is an interface that allows external parties to access the functions of software.

[1054] "Response" refers to the answer that a server returns to a client in response to a request.

[1055] "URL" stands for Uniform Resource Locator, and it is an address that indicates the location of a resource on the web.

[1056] A "network" is a system that connects multiple information devices to communicate data.

[1057] "User's device" refers to a device that the user directly operates (such as a smartphone or personal computer).

[1058] A "music streaming service" is a service that provides music in real time via the internet.

[1059] The embodiments for carrying out the present invention will be described in detail below. The present invention is a system that receives abstract words or images from a user, generates a music playlist based on them, and provides it to the user.

[1060] System Overview

[1061] The system operates in conjunction with a server, user terminals, and a music streaming service. The server is built using a web framework, specifically one such as Flask. Based on abstract words or images entered by the user, the server searches for songs using the music streaming service's API and generates a playlist.

[1062] Server Processing

[1063] The server has the following functions:

[1064] 1. Receiving the request:

[1065] The server receives requests from users. These requests are sent, for example, in JSON format.

[1066] 2. Acquisition of abstract words and images:

[1067] The server extracts abstract words and images from the request. Examples include the word "relax" and an image like "sunset."

[1068] 3. Search for songs:

[1069] The server uses the music streaming service's API to search for songs related to abstract words or images. It then retrieves the IDs of the relevant songs from the search results.

[1070] 4. Create a playlist:

[1071] The server uses the IDs of the retrieved songs to generate a new playlist. The playlist name might be, for example, "Playlist for Relaxation and Sunsets."

[1072] 5. Add songs to the playlist:

[1073] The server adds songs obtained from the search results to the generated playlist.

[1074] 6. Generating the response:

[1075] The server generates a response containing the name and URL of the created playlist and returns it to the user.

[1076] User processing

[1077] 1. Input abstract words or images:

[1078] Users use their devices to input words or images that represent the themes or emotions they desire.

[1079] 2. Submitting a request:

[1080] The user enters words and images, converts them to JSON format, and sends a POST request to the server.

[1081] 3. Receiving the response:

[1082] The user receives a response from the server. The response includes the name and URL of the generated playlist.

[1083] 4. Using playlists:

[1084] The user clicks the received playlist URL and plays the playlist on their music streaming service.

[1085] Specific example

[1086] For example, if a user enters the words "relax" and "sunset," the server will search for songs based on these words and generate a playlist named "Playlist for Relaxation and Sunset." The playlist will contain the selected songs. The user can play this playlist and enjoy a relaxing atmosphere by clicking on the URL provided by the server.

[1087] Example of a prompt

[1088] An example of a prompt for a generative AI model is, "Generate a playlist with the themes of relaxation and sunsets. Please select songs that fit the theme."

[1089] This system allows users to easily obtain music playlists based on abstract images. This feature will be particularly useful for users who struggle with selecting music or when preparing for special events.

[1090] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1091] Step 1:

[1092] The server receives a JSON-formatted request sent from the user's terminal. The input includes abstract words and images entered by the user. Specifically, the server retrieves the JSON data using request.get_json(). The output is the retrieved JSON data, which is then passed to the next step.

[1093] Step 2:

[1094] The server parses the received JSON data and extracts abstract words and images entered by the user. The JSON data obtained in step 1 is used as input. Specifically, the data is parsed using `data = request.json`, and abstract words and images are extracted using `keywords = data['keywords']`. The extracted words and images are then passed to the next step as output.

[1095] Step 3:

[1096] The server uses the music streaming service's API to search for songs related to abstract words and images. The words and images extracted in step 2 are used as input. Specifically, the API is called using requests.get('https: / / api.music-service.com / v1 / search', headers=headers, params=params) to obtain the relevant song IDs. The obtained song IDs are then passed to the next step as output.

[1097] Step 4:

[1098] The server generates a new playlist based on the retrieved song ID. The song ID obtained in step 3 is used as input. Specifically, it creates the playlist by executing requests.post('https: / / api.music-service.com / v1 / users / {user_id} / playlists', headers=headers, json=playlist_data). The generated playlist ID is passed to the next step as output.

[1099] Step 5:

[1100] The server adds songs related to the generated playlist. The inputs used are the playlist ID generated in step 4 and the song IDs obtained in step 3. Specifically, it executes requests.post('https: / / api.music-service.com / v1 / playlists / {playlist_id} / tracks', headers=headers, json=tracks_data) to add the songs to the playlist. The output is the playlist URL, which is passed to the next step.

[1101] Step 6:

[1102] The server generates a response containing the name and URL of the generated playlist and returns it to the user. The input used is the playlist URL obtained in step 5. Specifically, the response data is constructed using response_data = {'playlist_name': playlist_name, 'url': playlist_url}, and the response is returned to the user using return jsonify(response_data). The output provides the user with information about the generated playlist.

[1103] Step 7:

[1104] The user receives a playlist URL from the server using their device. The server's response is used as input. Specifically, the response.json() function is used to parse the response data. The playlist URL is displayed on the user's device as output.

[1105] Step 8:

[1106] The user clicks the received playlist URL and plays the playlist on their music streaming service. The input used is the playlist URL received in step 7. Specifically, the user opens the URL using a browser or music streaming app and plays the playlist. The output is the playback of the song the user requested.

[1107] (Application Example 1)

[1108] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1109] Traditional music playlist generation systems assumed users were familiar with specific songs and artists, and lacked the ability to automatically generate playlists from abstract words or visual information. As a result, users had the problem of spending a lot of time finding music that suited their mood or situation. Furthermore, if users wanted to express a specific atmosphere or situation, they had to manually search for songs, which was a time-consuming process.

[1110] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1111] In this invention, the server includes means for receiving abstract words and visual information from a user, means for searching for music based on the received abstract words and visual information, means for providing the user with a playlist generated from the search results, means for creating a playlist using the API of a music provider, and means for returning the URL of the provider to the user. This makes it possible for the user to instantly obtain the optimal playlist based on abstract words and visual information.

[1112] A "user" is an individual or group that uses the system.

[1113] "Abstract words" are words that express concepts or emotions rather than concrete things.

[1114] "Visual information" refers to data related to vision, such as images and videos.

[1115] "Means" refer to methods or devices used to achieve a specific objective.

[1116] A "musical piece" refers to the individual works that make up a piece of music.

[1117] "Searching" refers to the act of finding specific information.

[1118] A "playlist" is a list of songs that are organized according to a specific order or theme.

[1119] "Providing" refers to the act of handing over a certain item or service to another person.

[1120] A "server" is a computer that provides services to other computers via a network.

[1121] A "network" is a system in which multiple computers are connected to each other and can exchange information.

[1122] "JSON formatted data" refers to data structured in JavaScript Object Notation format.

[1123] An "external music provider API" refers to an application programming interface provided by an external music distribution service.

[1124] This invention is a system that receives abstract words and visual information from a user, generates a music playlist based on that information, and provides it to the user. Specific embodiments for carrying out this invention are described in detail below.

[1125] The system operates in conjunction with the user's terminal, the server, and the music provider's API.

[1126] System Configuration

[1127] server

[1128] The server primarily has the following functions:

[1129] Request received:

[1130] The server receives requests from users that include abstract words and visual information. These requests are sent in JSON format.

[1131] Acquisition of abstract words and visual information:

[1132] The server parses the received JSON data and extracts abstract words and visual information. In this process, natural language processing (NLP) is used to process abstract words, and image processing is used to analyze visual information.

[1133] Search for songs:

[1134] The server uses the API of the music provider (e.g., Spotify) to search for songs related to abstract words or visual information. It then retrieves the IDs of the relevant songs.

[1135] Playlist generation:

[1136] A new playlist is generated based on the acquired song IDs. The playlist name is determined based on abstract words or visual information.

[1137] Add songs to the playlist:

[1138] Add songs obtained from search results to the playlist you created.

[1139] Response generation:

[1140] Generate a response containing the name and URL of the generated playlist, and send it back to the user.

[1141] User terminal

[1142] The user terminal has the following functions:

[1143] Inputting abstract words and visual information:

[1144] The user inputs abstract words and visual information to generate a playlist. This input is done through the application.

[1145] Sending a request:

[1146] The user terminal sends a request containing abstract words and visual information to the server in JSON format.

[1147] Response received:

[1148] Receive the response sent back from the server and display the information of the generated playlist.

[1149] Specific example

[1150] For example, if a user enters an abstract phrase like "a refreshing morning walk" and uploads a morning landscape image as visual information, the server analyzes these to search for relevant music. It then generates a playlist with a name like "Playlist for a refreshing morning walk" and provides the user with its URL. By clicking the provided URL, the user can listen to the appropriate music.

[1151] Example of a prompt

[1152] Abstract words: A refreshing morning walk

[1153] Visual information: Morning landscape image

[1154] This allows users to easily obtain music playlists that match their desired themes and emotions. This invention is particularly useful for users who are unfamiliar with music selection and for creating the right atmosphere for special events.

[1155] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1156] Step 1:

[1157] The user inputs abstract words or visual information into the application.

[1158] Input: Abstract words entered by the user (e.g., "A refreshing morning walk") and visual information (e.g., an image of a morning landscape).

[1159] Output: Request data in JSON format.

[1160] Specific operation: The smartphone app receives user input, converts it into JSON data, and prepares to send it to the server.

[1161] Step 2:

[1162] The user's terminal sends a request in JSON format to the server.

[1163] Input: Request data in JSON format entered by the user.

[1164] Output: Request data sent to the server.

[1165] Specific operation: The user's terminal sends JSON-formatted request data to the server via the network.

[1166] Step 3:

[1167] The server receives the request and extracts abstract words and visual information.

[1168] Input: Request data in JSON format.

[1169] Output: Abstract words and visual information.

[1170] Specific operation: The server parses the request, analyzes abstract words using an NLP model, and analyzes visual information using image processing (e.g., OpenCV).

[1171] Step 4:

[1172] The server calls the music provider's API based on the analysis results and searches for related songs.

[1173] Input: Keywords and parameters based on abstract words and visual information.

[1174] Output: A list of IDs for related songs.

[1175] Specific operation: The server calls the music provider's API (e.g., Spotify API), searches for and retrieves relevant songs based on the analysis results.

[1176] Step 5:

[1177] The server generates a new playlist based on the song ID it retrieves.

[1178] Input: List of song IDs.

[1179] Output: The ID and URL of the newly generated playlist.

[1180] Specific operation: The server calls the music provider's API again, creates a new playlist, and adds the retrieved songs to that playlist.

[1181] Step 6:

[1182] The server sends the generated playlist information back to the user.

[1183] Input: Playlist ID and URL.

[1184] Output: JSON-formatted response data sent to the user's terminal.

[1185] Specific operation: The server generates a JSON response containing the playlist name and URL, and sends it to the user's terminal.

[1186] Step 7:

[1187] The user's terminal receives the response and displays the information of the generated playlist.

[1188] Input: JSON response data from the server.

[1189] Output: The name and URL of the playlist to be displayed.

[1190] Specific operation: The user's terminal parses the response received from the server and displays the playlist name and URL to the user.

[1191] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1192] The embodiments for carrying out the present invention will be described in detail below. The present invention combines a system that receives abstract words or images from a user, generates a music playlist based on them, and provides it to the user, with an emotion engine that recognizes the user's emotions.

[1193] System Overview

[1194] The system works in conjunction with a server, a user's device, and a music streaming service. Based on abstract words, images, and emotions entered by the user, the server uses the music streaming service's API to search for songs and generate a playlist.

[1195] Server Processing

[1196] The server is built using a web framework (e.g., Flask) and has the following functions:

[1197] 1. Receiving the request:

[1198] The server receives requests from users. These requests are sent, for example, in JSON format.

[1199] 2. Acquisition of abstract words and images:

[1200] The server extracts abstract words and images (for example, "relax" and "sunset") from the request.

[1201] 3. Emotional analysis using an emotion engine:

[1202] The server uses an emotion engine to analyze user emotions based on their input. The emotion engine uses text-based analysis techniques to identify the user's emotions (e.g., happiness, sadness, excitement).

[1203] 4. Search for songs:

[1204] The server uses music streaming service APIs to search for songs based on abstract words, images, and sentiment analysis results. This uses libraries such as Spotipy.

[1205] 5. Create a playlist:

[1206] The server uses the ID of the retrieved song to generate a new playlist. The playlist name might be, for example, "Playlist for Relaxation and Sunsets."

[1207] 6. Add songs to the playlist:

[1208] The server adds songs to the generated playlist using the song IDs obtained in step 4.

[1209] 7. Generating the response:

[1210] The server prepares a response in JSON format, including the name and URL of the generated playlist, and returns it to the user.

[1211] User processing

[1212] 1. Input abstract words or images:

[1213] Users input words or images that represent the themes or emotions they are seeking.

[1214] 2. Submitting a request:

[1215] The user enters words and images, converts them to JSON format, and sends a POST request to the server.

[1216] 3. Receiving the response:

[1217] The user receives a response from the server. The response includes the name and URL of the generated playlist.

[1218] 4. Using playlists:

[1219] The user clicks the received playlist URL and plays the playlist on their music streaming service.

[1220] Specific example

[1221] For example, if a user enters abstract words like "relax" and "sunset," and the emotion engine interprets these words as "happiness," the server will search for songs based on this and generate a playlist named "Playlist for Relaxation and Sunset." The playlist contains the selected songs and is provided to the user via a URL. By clicking this URL, the user can play the playlist and enjoy a relaxed and happy atmosphere.

[1222] This invention allows users to easily access music playlists based on abstract images and emotions, which can be particularly helpful for users unfamiliar with music selection or when preparing for special events.

[1223] The following describes the processing flow.

[1224] Step 1:

[1225] The user opens a webpage or application using their device and enters an abstract word or image into a text field. This input could include, for example, "relax" or "sunset."

[1226] Step 2:

[1227] The system sends JSON data containing abstract words and images entered by the user to the server as a POST request.

[1228] Step 3:

[1229] The server receives a POST request and extracts JSON data from the request body. Specifically, the Flask framework is used to parse the data from the request.

[1230] Step 4:

[1231] The server retrieves abstract words and images from the JSON data (for example, "relax sunset" contained in the terms field).

[1232] Step 5:

[1233] The server uses an emotion engine to analyze abstract words and images to identify the user's emotions. For example, the words "relax" and "sunset" might be classified as the emotion "happiness."

[1234] Step 6:

[1235] The server uses the music streaming service's API to search for songs based on identified emotions and abstract words. It uses the Spotipy library to submit search queries.

[1236] Step 7:

[1237] The server analyzes the search results returned from the music streaming service and compiles a list of the IDs of related songs. For example, the list might contain the IDs of 10 songs.

[1238] Step 8:

[1239] The server uses the music streaming service's API to generate a new playlist. The playlist will be named "Playlist for Relaxation and Sunsets".

[1240] Step 9:

[1241] Add songs to the playlist generated by the server using the song IDs obtained in step 7. Specifically, add all song IDs to the playlist.

[1242] Step 10:

[1243] Prepare a response in JSON format containing the name and URL of the playlist generated by the server, and return it to the user.

[1244] Step 11:

[1245] The user receives a response from the server and checks its contents. The response includes the name and URL of the generated playlist.

[1246] Step 12:

[1247] The user clicks the response URL and opens the playlist on their music streaming service. When the playlist is played, songs based on the user's selected emotions and images are played.

[1248] (Example 2)

[1249] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1250] Conventional music playlist generation systems have been unable to adequately respond to users' abstract words and images, making it difficult to provide playlists that align with their emotions and preferences. Furthermore, there were no systems that combined emotional analysis to address the diverse needs of users.

[1251] The identification processing performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for receiving abstract words or images from the user, means for analyzing emotions from the received abstract words or images, means for searching for music based on the analyzed emotions and abstract words or images, and means for providing the user with a playlist generated from the search results. This makes it possible to analyze the user's emotions based on their abstract input and automatically generate and provide a music playlist that suits them.

[1252] "Abstract words and images" are words or visual concepts used to express a state or feeling that is difficult for the user to describe as concretely as possible.

[1253] "Methods for analyzing emotions" refer to technologies and algorithms used to infer and identify a user's emotions from the words and images they input. This includes natural language processing and machine learning models.

[1254] "Methods for searching for music" refers to technologies and methods that use external music databases or music streaming service APIs to find songs that match specific keywords or emotions.

[1255] "Methods for generating playlists" refer to technologies and methods that appropriately combine searched songs to create a single, continuous music list. This makes it possible to play music in sequence according to a specific theme or emotion.

[1256] "Means of receiving" refers to interfaces and communication protocols for receiving data from external sources via a network. This includes web frameworks that process HTTP requests.

[1257] "JSON format data" is an abbreviation for JavaScript Object Notation (JSON), a lightweight data exchange format. It represents data in a format that is easy for humans to read and easy for machines to parse.

[1258] Modes for carrying out the invention

[1259] The embodiments for carrying out the present invention will be described in detail below. The present invention combines a system that receives abstract words or images from a user, generates a music playlist based on them, and provides it to the user, with an emotion engine that recognizes the user's emotions.

[1260] The system operates in conjunction with a server, user terminals, and music streaming services. This will be explained below, along with specific hardware and software implementations.

[1261] Server Processing

[1262] The server is built using a web framework (e.g., Flask). The main processing steps are as follows:

[1263] 1. Receiving the request:

[1264] The server receives requests from users. This is done by users sending data containing abstract keywords and images in JSON format through their devices. For example, a user might enter the keywords "relax" and "sunset."

[1265] 2. Acquisition of abstract words and images:

[1266] The server parses the received JSON data and extracts the words and images entered by the user. This process uses the json module from Python's standard library.

[1267] 3. Emotional analysis using an emotion engine:

[1268] The server uses the Hugging Face Transformers library to analyze emotions from the user's input text. This analysis associates the keywords "relax" and "sunset" with the emotion "happiness." The emotion engine uses a pre-trained model to determine the emotion of the input text.

[1269] 4. Search for songs:

[1270] The server uses the Spotipy library to access the Spotify API and search for songs based on the keywords "relax," "sunset," and "happiness." This search query targets Spotify's song database to retrieve the appropriate song ID.

[1271] 5. Create a playlist:

[1272] The server generates a new playlist using the retrieved song ID. The playlist name will be based on the keywords entered by the user, such as "Playlist for Relaxation and Sunsets." The generated playlist will then be linked to the user's Spotify account.

[1273] 6. Add songs to the playlist:

[1274] The server adds songs to the generated playlist. This process uses the Spotipy library to send song addition requests to the Spotify API.

[1275] 7. Generating the response:

[1276] The server creates a JSON response containing the name of the generated playlist and its URL. For example, it might be returned to the user using Flask's response functionality. The JSON response includes data such as:

[1277] json

[1278] {

[1279] "playlist_name": "Playlist for relaxation and sunsets",

[1280] "playlist_url": "https: / / open.spotify.com / playlist / ..."

[1281] }

[1282] User processing

[1283] 1. Input abstract words or images:

[1284] Users input words or images that represent the themes or emotions they are seeking. For example, they might use words like "relax" or "sunset."

[1285] 2. Submitting a request:

[1286] The user enters words and images, converts them to JSON format, and sends a POST request to the server.

[1287] 3. Receiving the response:

[1288] The user receives a response from the server. The response includes the name and URL of the generated playlist.

[1289] 4. Using playlists:

[1290] The user clicks the received playlist URL and plays the playlist on their music streaming service.

[1291] Specific example

[1292] For example, if a user enters abstract words like "relax" and "sunset," and the emotion engine interprets these words as "happiness," the server will search for songs based on this and generate a playlist named "Playlist for Relaxation and Sunset." The playlist contains the selected songs and is provided to the user via a URL. By clicking this URL, the user can play the playlist and enjoy a relaxed and happy atmosphere.

[1293] Example of a prompt

[1294] "Please create a playlist for relaxation and sunsets."

[1295] Based on this input, the generative AI model performs sentiment analysis based on the user's abstract words, selects appropriate songs, and creates a playlist.

[1296] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1297] Step 1:

[1298] The user inputs abstract words or images.

[1299] Detailed explanation:

[1300] Users input abstract words or images that represent a theme or emotion using web forms or mobile apps from their own devices. For example, they might use words like "relax" or "sunset."

[1301] Inputs and outputs:

[1302] Input: Abstract words or images entered by the user (e.g., "relax," "sunset").

[1303] Output: The terminal generates data in JSON format.

[1304] Step 2:

[1305] The terminal sends the generated JSON data to the server.

[1306] Detailed explanation:

[1307] The terminal converts the words and images entered by the user into JSON format and sends them to a specific endpoint on the server as an HTTP POST request.

[1308] Inputs and outputs:

[1309] Input: JSON data containing abstract words or images entered by the user.

[1310] Output: JSON data received by the server.

[1311] Step 3:

[1312] The server parses the JSON data and extracts abstract words and images.

[1313] Detailed explanation:

[1314] The server uses the Flask web framework to parse the received JSON data and extract the words and images entered by the user. This process uses Python's json module.

[1315] Inputs and outputs:

[1316] Input: JSON data received by the server.

[1317] Output: Abstract words or images (e.g., "relax," "sunset").

[1318] Step 4:

[1319] The server uses an emotion engine to analyze the user's emotions based on their input.

[1320] Detailed explanation:

[1321] The server uses the Hugging Face Transformers library to analyze the sentiment of the user's input text. This analysis associates "relaxed" and "sunset" with the emotion of "happiness."

[1322] Inputs and outputs:

[1323] Input: Abstract words or images (e.g., "relax," "sunset").

[1324] Output: Analyzed emotion (e.g., "happiness").

[1325] Step 5:

[1326] The server uses the music streaming service's API to search for songs.

[1327] Detailed explanation:

[1328] The server uses the Spotipy library to access the Spotify API and search for songs based on the keywords "relax," "sunset," and "happiness." This search query targets Spotify's song database to retrieve the appropriate song ID.

[1329] Inputs and outputs:

[1330] Input: Abstract words, images, or analyzed emotions (e.g., "relaxed," "sunset," "happy").

[1331] Output: A list of searched song IDs.

[1332] Step 6:

[1333] The server generates a playlist using the acquired song ID.

[1334] Detailed explanation:

[1335] The server uses the Spotipy library to generate a new playlist via the Spotify API. The playlist name will be "Playlist for Relaxation and Sunsets," based on keywords entered by the user.

[1336] Inputs and outputs:

[1337] Input: A list of searched song IDs.

[1338] Output: The ID and name of the generated playlist.

[1339] Step 7:

[1340] Add songs to the playlist generated by the server.

[1341] Detailed explanation:

[1342] The server adds the searched songs to the generated playlist. This process uses the Spotipy library to send song addition requests to the Spotify API.

[1343] Inputs and outputs:

[1344] Input: The ID of the generated playlist and a list of the searched song IDs.

[1345] Output: A playlist with the added songs.

[1346] Step 8:

[1347] The server prepares a JSON response containing the name and URL of the generated playlist and returns it to the user.

[1348] Detailed explanation:

[1349] The server creates a JSON response containing the name and URL of the generated playlist and returns it to the user. This response is returned using Flask's response functionality.

[1350] Inputs and outputs:

[1351] Input: The ID and name of the generated playlist.

[1352] Output: JSON response containing the playlist name and URL.

[1353] Step 9:

[1354] The user receives the playlist URL returned from the server and plays it.

[1355] Detailed explanation:

[1356] The user receives a response from the server via their device and clicks the provided playlist URL. This allows them to play the playlist on their music streaming service.

[1357] Inputs and outputs:

[1358] Input: JSON response containing the playlist name and URL.

[1359] Output: The music playlist that the user will play.

[1360] (Application Example 2)

[1361] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1362] In modern virtual stores and online shopping experiences, there is a lack of personalized service based on users' emotions and moods. In particular, the selection of music is often generic rather than tailored to an individual's mood or theme, which degrades the quality of the user's purchasing experience.

[1363] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1364] In this invention, the server includes means for receiving abstract words or images from the user, means for searching for music based on the received abstract words or images and the user's emotions, means for providing the user with a playlist generated from the search results, and means for automatically generating and providing music playlists to be played within the virtual store. This makes it possible to improve the quality of online shopping through a personalized music experience based on the user's emotions and mood.

[1365] "Abstract words and images" refer to vague words or visual concepts used to express the feelings or themes that the user is experiencing.

[1366] "Emotions" refer to the user's psychological and sensory state and are classified into emotional categories such as happiness, sadness, and excitement.

[1367] "Searching for music" refers to the process of finding appropriate songs from the music streaming service's database based on user input.

[1368] A "playlist" refers to a collection of songs selected based on a specific theme or emotion, and is a list of music arranged to be played sequentially.

[1369] A "virtual store" is a virtual commercial space operated on the internet where users can browse and purchase products online.

[1370] "Automatic generation" refers to the process by which a system creates specific deliverables or data based on user input and algorithms, without requiring manual operation.

[1371] "To provide" refers to giving information or services to users so that they can use them.

[1372] This invention is a system that receives abstract words or images from a user, generates a music playlist based on them, and plays it within a virtual store. The following describes in detail the configurations for implementing this system.

[1373] System Configuration

[1374] The system operates in conjunction with servers, user terminals, and music streaming services.

[1375] Server Processing

[1376] The server is built using a web framework (e.g., Flask) and includes the following software components:

[1377] 1. Receiving the Request: The server receives a request from the user. The request is sent in JSON format.

[1378] 2. Retrieving abstract words and images: The server extracts abstract words and images from the request. Examples include "relax" and "sunset."

[1379] 3. Sentiment Analysis: The server uses an emotion engine to analyze the user's emotions. The emotion engine utilizes text analysis libraries such as TextBlob to identify the user's emotions.

[1380] 4. Searching for songs: The server searches for songs using the API of a music streaming service (e.g., the Spotipy library).

[1381] 5. Playlist Generation: The server uses the IDs of the retrieved songs to generate a new playlist that will be played within the virtual store.

[1382] 6. Providing the playlist: The server returns the name and URL of the generated playlist to the user in JSON format.

[1383] User terminal processing

[1384] 1. Input of abstract words and images: Users input words or images in text format that represent the theme or emotion they are seeking.

[1385] 2. Sending the Request: The user sends a POST request to the server with the entered words and images in JSON format.

[1386] 3. Receiving the response: The user receives the response returned from the server.

[1387] 4. Playlist Usage: Users click on the received playlist URL, play the playlist on their device, and enjoy the music experience of the virtual store.

[1388] Specific example

[1389] For example, if a user enters "I want to enjoy a relaxed shopping experience," the server receives this query and uses an emotion analysis engine to analyze the emotions "relaxed" and "happy." Next, it uses the APIs of music streaming services such as Spotify to search for songs that suit these emotions and generates a playlist named "Relaxing Shopping Playlist." The user is then provided with the URL of this playlist, which they can click to play within the virtual store. In this way, users can enjoy a richer shopping experience through music that matches their emotions and themes.

[1390] Example of a prompt

[1391] The following are examples of prompts that the user will enter into the system:

[1392] I want to enjoy a relaxed shopping experience. Please suggest some music to help me spend a pleasant time in the soft evening sunlight.

[1393] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1394] Step 1:

[1395] User input

[1396] In order to obtain a specific musical experience in a virtual store, users input their mood or theme as abstract words or images in text format. For example, they might input the phrase, "I want to enjoy a relaxed shopping experience."

[1397] Input: Text representing the user's theme or emotions.

[1398] Output: Text containing abstract words and images.

[1399] Step 2:

[1400] Send a request

[1401] The user terminal converts the input words and images into JSON format and sends a POST request to the server. The request reaches the server via the network.

[1402] Input: Text of abstract words or images

[1403] Output: Request data in JSON format

[1404] Step 3:

[1405] Receiving a request

[1406] The server receives a JSON-formatted request sent from the user's terminal.

[1407] Input: Request data in JSON format

[1408] Output: Analysis results of the request data

[1409] Step 4:

[1410] Extraction of abstract words and images

[1411] The server extracts abstract words and images (for example, "relax") from the received JSON data.

[1412] Input: Request data in JSON format

[1413] Output: Text containing abstract words and images.

[1414] Step 5:

[1415] Emotion analysis

[1416] The server performs sentiment analysis on abstract words and images. It uses a sentiment engine like TextBlob to identify the user's emotions (for example, "happiness").

[1417] Input: Text of abstract words or images

[1418] Output: Results of emotion analysis (emotion categories)

[1419] Step 6:

[1420] Search for songs

[1421] The server uses the results of sentiment analysis to search for songs using the API of a music streaming service (e.g., Spotify). The search query combines abstract words with the analyzed sentiment.

[1422] Input: Abstract words or categories of emotions

[1423] Output: Search results (list of song IDs)

[1424] Step 7:

[1425] Playlist generation

[1426] The server generates a new playlist using the acquired song ID. The playlist name might be, for example, "Relaxing Purchase Playlist".

[1427] Input: List of song IDs

[1428] Output: Information about the generated playlist (playlist ID, name)

[1429] Step 8:

[1430] Playlists provided

[1431] The server sends a JSON response to the user's device containing the name and URL of the generated playlist.

[1432] Input: Information about the generated playlist (playlist ID, name)

[1433] Output: JSON response containing playlist information

[1434] Step 9:

[1435] Received response

[1436] The user's device receives the JSON response sent back from the server and obtains the playlist URL.

[1437] Input: JSON response containing playlist information

[1438] Output: Playlist URL

[1439] Step 10:

[1440] Play playlist

[1441] The user clicks on the URL of the acquired playlist and plays the playlist within the virtual store using their device.

[1442] Input: Playlist URL

[1443] Output: A music experience where a playlist is played.

[1444] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1445] The data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of the data generation model 58 is ChatGPT (Internet Search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1446] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1447] [Fourth Embodiment]

[1448] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1449] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1450] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1451] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1452] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1453] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1454] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1455] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1456] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1457] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1458] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1459] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1460] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1461] The embodiments for carrying out the present invention will be described in detail below. The present invention is a system that receives abstract words or images from a user, generates a music playlist based on them, and provides it to the user.

[1462] System Overview

[1463] The system works in conjunction with a server, a user's device, and a music streaming service. Based on abstract words or images entered by the user, the server uses the music streaming service's API to search for songs and generate a playlist.

[1464] Server Processing

[1465] The server is built using a web framework (e.g., Flask) and has the following functions:

[1466] 1. Receiving the request:

[1467] The server receives requests from users. These requests are sent, for example, in JSON format.

[1468] 2. Acquisition of abstract words and images:

[1469] The server extracts abstract words and images (for example, "relax" and "sunset") from the request.

[1470] 3. Search for songs:

[1471] The server uses the music streaming service's API to search for songs related to abstract words or images. It then retrieves the IDs of the relevant songs from the search results.

[1472] 4. Create a playlist:

[1473] The server uses the ID of the retrieved song to generate a new playlist. The playlist name might be, for example, "Playlist for Relaxation and Sunsets."

[1474] 5. Add songs to the playlist:

[1475] The server adds songs obtained from the search results to the generated playlist.

[1476] 6. Generating the response:

[1477] The server generates a response containing the name and URL of the created playlist and returns it to the user.

[1478] User processing

[1479] 1. Input abstract words or images:

[1480] Users input words or images that represent the themes or emotions they are seeking.

[1481] 2. Submitting a request:

[1482] The user enters words and images, converts them to JSON format, and sends a POST request to the server.

[1483] 3. Receiving the response:

[1484] The user receives a response from the server. The response includes the name and URL of the generated playlist.

[1485] 4. Using playlists:

[1486] The user clicks the received playlist URL and plays the playlist on their music streaming service.

[1487] Specific example

[1488] For example, if a user enters abstract words like "relax" and "sunset," the server will search for music based on these words and generate a playlist named "Playlist for Relaxation and Sunset." The playlist will contain the selected music and be provided to the user via a URL. By clicking this URL, the user can play the playlist and enjoy a relaxing atmosphere.

[1489] This invention allows users to easily obtain music playlists based on abstract images, which can be particularly helpful for users unfamiliar with music selection or when preparing for special events.

[1490] The following describes the processing flow.

[1491] Step 1:

[1492] The user opens a webpage or application using their device and enters abstract words or images. These inputs may include words such as "relax" or "sunset."

[1493] Step 2:

[1494] The system converts abstract words and images entered by the user into JSON format and sends a POST request to the server.

[1495] Step 3:

[1496] The server receives a POST request. A web framework such as Flask is used to parse the request.

[1497] Step 4:

[1498] The server extracts abstract words and images from the request. Specifically, it retrieves the `terms` field from the JSON data in the request body.

[1499] Step 5:

[1500] The server uses the API of a music streaming service to search for songs based on extracted words and images. This uses libraries such as Spotipy.

[1501] Step 6:

[1502] The server analyzes the search results returned from the music streaming service and compiles a list of the IDs of the relevant songs.

[1503] Step 7:

[1504] The server uses the music streaming service's API to generate a new playlist. The playlist name might be, for example, "Playlist for Relaxation and Sunsets."

[1505] Step 8:

[1506] The song is added to the playlist generated by the server using the song ID obtained in step 6.

[1507] Step 9:

[1508] Prepare a response in JSON format containing the name and URL of the playlist generated by the server, and return it to the user.

[1509] Step 10:

[1510] The user receives a response from the server. The response includes the playlist name and URL.

[1511] Step 11:

[1512] The user clicks the response URL and plays the playlist on their music streaming service. The user can then enjoy the generated playlist.

[1513] (Example 1)

[1514] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1515] Traditionally, manually selecting music playlists that matched users' emotions or themes was time-consuming and laborious. Furthermore, finding appropriate music was difficult for users unfamiliar with music selection or when preparing for special events. This invention aims to solve this problem by providing a system that allows users to easily obtain music playlists based on abstract words or images.

[1516] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1517] In this invention, the server includes means for receiving abstract words or images from a user, means for searching for music based on the received abstract words or images, means for providing the user with a playlist generated from the search results, means for receiving and analyzing user requests through a web framework, means for searching for relevant music using a music streaming service API, means for generating a playlist from the search results and adding music, and means for generating the name and URL of the generated playlist as a response. This makes it possible for users to easily and quickly obtain music playlists that match their mood or theme.

[1518] A "user" is a person or group that uses the system to generate music playlists based on abstract words or images.

[1519] "Abstract words" are words that express emotions or themes, rather than referring to specific objects or actions.

[1520] An "image" is a concept or theme expressed using visual or non-visual means.

[1521] "Means" refer to a set of processes or functions performed to achieve a specific objective.

[1522] A "musical piece" refers to a musical work, primarily offered through music streaming services.

[1523] "Searching" is the act of finding information based on specific criteria.

[1524] A "playlist" is a list of songs that are played in a specific order.

[1525] A "web framework" is a software framework for developing web applications.

[1526] A "request" refers to a request made from a client (the user's device) to a server.

[1527] "JSON format" is an abbreviation for JavaScript Object Notation, and it is a lightweight text format for representing data.

[1528] "API" stands for Application Programming Interface, and it is an interface that allows external parties to access the functions of software.

[1529] "Response" refers to the answer that a server returns to a client in response to a request.

[1530] "URL" stands for Uniform Resource Locator, and it is an address that indicates the location of a resource on the web.

[1531] A "network" is a system that connects multiple information devices to communicate data.

[1532] "User's device" refers to a device that the user directly operates (such as a smartphone or personal computer).

[1533] A "music streaming service" is a service that provides music in real time via the internet.

[1534] The embodiments for carrying out the present invention will be described in detail below. The present invention is a system that receives abstract words or images from a user, generates a music playlist based on them, and provides it to the user.

[1535] System Overview

[1536] The system operates in conjunction with a server, user terminals, and a music streaming service. The server is built using a web framework, specifically one such as Flask. Based on abstract words or images entered by the user, the server searches for songs using the music streaming service's API and generates a playlist.

[1537] Server Processing

[1538] The server has the following functions:

[1539] 1. Receiving the request:

[1540] The server receives requests from users. These requests are sent, for example, in JSON format.

[1541] 2. Acquisition of abstract words and images:

[1542] The server extracts abstract words and images from the request. Examples include the word "relax" and an image like "sunset."

[1543] 3. Search for songs:

[1544] The server uses the music streaming service's API to search for songs related to abstract words or images. It then retrieves the IDs of the relevant songs from the search results.

[1545] 4. Create a playlist:

[1546] The server uses the IDs of the retrieved songs to generate a new playlist. The playlist name might be, for example, "Playlist for Relaxation and Sunsets."

[1547] 5. Add songs to the playlist:

[1548] The server adds songs obtained from the search results to the generated playlist.

[1549] 6. Generating the response:

[1550] The server generates a response containing the name and URL of the created playlist and returns it to the user.

[1551] User processing

[1552] 1. Input abstract words or images:

[1553] Users use their devices to input words or images that represent the themes or emotions they desire.

[1554] 2. Submitting a request:

[1555] The user enters words and images, converts them to JSON format, and sends a POST request to the server.

[1556] 3. Receiving the response:

[1557] The user receives a response from the server. The response includes the name and URL of the generated playlist.

[1558] 4. Using playlists:

[1559] The user clicks the received playlist URL and plays the playlist on their music streaming service.

[1560] Specific example

[1561] For example, if a user enters the words "relax" and "sunset," the server will search for songs based on these words and generate a playlist named "Playlist for Relaxation and Sunset." The playlist will contain the selected songs. The user can play this playlist and enjoy a relaxing atmosphere by clicking on the URL provided by the server.

[1562] Example of a prompt

[1563] An example of a prompt for a generative AI model is, "Generate a playlist with the themes of relaxation and sunsets. Please select songs that fit the theme."

[1564] This system allows users to easily obtain music playlists based on abstract images. This feature will be particularly useful for users who struggle with selecting music or when preparing for special events.

[1565] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1566] Step 1:

[1567] The server receives a JSON-formatted request sent from the user's terminal. The input includes abstract words and images entered by the user. Specifically, the server retrieves the JSON data using request.get_json(). The output is the retrieved JSON data, which is then passed to the next step.

[1568] Step 2:

[1569] The server parses the received JSON data and extracts abstract words and images entered by the user. The JSON data obtained in step 1 is used as input. Specifically, the data is parsed using `data = request.json`, and abstract words and images are extracted using `keywords = data['keywords']`. The extracted words and images are then passed to the next step as output.

[1570] Step 3:

[1571] The server uses the music streaming service's API to search for songs related to abstract words and images. The words and images extracted in step 2 are used as input. Specifically, the API is called using requests.get('https: / / api.music-service.com / v1 / search', headers=headers, params=params) to obtain the relevant song IDs. The obtained song IDs are then passed to the next step as output.

[1572] Step 4:

[1573] The server generates a new playlist based on the retrieved song ID. The song ID obtained in step 3 is used as input. Specifically, it creates the playlist by executing requests.post('https: / / api.music-service.com / v1 / users / {user_id} / playlists', headers=headers, json=playlist_data). The generated playlist ID is passed to the next step as output.

[1574] Step 5:

[1575] The server adds songs related to the generated playlist. The inputs used are the playlist ID generated in step 4 and the song IDs obtained in step 3. Specifically, it executes requests.post('https: / / api.music-service.com / v1 / playlists / {playlist_id} / tracks', headers=headers, json=tracks_data) to add the songs to the playlist. The output is the playlist URL, which is passed to the next step.

[1576] Step 6:

[1577] The server generates a response containing the name and URL of the generated playlist and returns it to the user. The input used is the playlist URL obtained in step 5. Specifically, the response data is constructed using response_data = {'playlist_name': playlist_name, 'url': playlist_url}, and the response is returned to the user using return jsonify(response_data). The output provides the user with information about the generated playlist.

[1578] Step 7:

[1579] The user receives a playlist URL from the server using their device. The server's response is used as input. Specifically, the response.json() function is used to parse the response data. The playlist URL is displayed on the user's device as output.

[1580] Step 8:

[1581] The user clicks the received playlist URL and plays the playlist on their music streaming service. The input used is the playlist URL received in step 7. Specifically, the user opens the URL using a browser or music streaming app and plays the playlist. The output is the playback of the song the user requested.

[1582] (Application Example 1)

[1583] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1584] Traditional music playlist generation systems assumed users were familiar with specific songs and artists, and lacked the ability to automatically generate playlists from abstract words or visual information. As a result, users had the problem of spending a lot of time finding music that suited their mood or situation. Furthermore, if users wanted to express a specific atmosphere or situation, they had to manually search for songs, which was a time-consuming process.

[1585] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1586] In this invention, the server includes means for receiving abstract words and visual information from a user, means for searching for music based on the received abstract words and visual information, means for providing the user with a playlist generated from the search results, means for creating a playlist using the API of a music provider, and means for returning the URL of the provider to the user. This makes it possible for the user to instantly obtain the optimal playlist based on abstract words and visual information.

[1587] A "user" is an individual or group that uses the system.

[1588] "Abstract words" are words that express concepts or emotions rather than concrete things.

[1589] "Visual information" refers to data related to vision, such as images and videos.

[1590] "Means" refer to methods or devices used to achieve a specific objective.

[1591] A "musical piece" refers to the individual works that make up a piece of music.

[1592] "Searching" refers to the act of finding specific information.

[1593] A "playlist" is a list of songs that are organized according to a specific order or theme.

[1594] "Providing" refers to the act of handing over a certain item or service to another person.

[1595] A "server" is a computer that provides services to other computers via a network.

[1596] A "network" is a system in which multiple computers are connected to each other and can exchange information.

[1597] "JSON formatted data" refers to data structured in JavaScript Object Notation format.

[1598] An "external music provider API" refers to an application programming interface provided by an external music distribution service.

[1599] This invention is a system that receives abstract words and visual information from a user, generates a music playlist based on that information, and provides it to the user. Specific embodiments for carrying out this invention are described in detail below.

[1600] The system operates in conjunction with the user's terminal, the server, and the music provider's API.

[1601] System Configuration

[1602] server

[1603] The server primarily has the following functions:

[1604] Request received:

[1605] The server receives requests from users that include abstract words and visual information. These requests are sent in JSON format.

[1606] Acquisition of abstract words and visual information:

[1607] The server parses the received JSON data and extracts abstract words and visual information. In this process, natural language processing (NLP) is used to process abstract words, and image processing is used to analyze visual information.

[1608] Search for songs:

[1609] The server uses the API of the music provider (e.g., Spotify) to search for songs related to abstract words or visual information. It then retrieves the IDs of the relevant songs.

[1610] Playlist generation:

[1611] A new playlist is generated based on the acquired song IDs. The playlist name is determined based on abstract words or visual information.

[1612] Add songs to the playlist:

[1613] Add songs obtained from search results to the playlist you created.

[1614] Response generation:

[1615] Generate a response containing the name and URL of the generated playlist, and send it back to the user.

[1616] User terminal

[1617] The user terminal has the following functions:

[1618] Inputting abstract words and visual information:

[1619] The user inputs abstract words and visual information to generate a playlist. This input is done through the application.

[1620] Sending a request:

[1621] The user terminal sends a request containing abstract words and visual information to the server in JSON format.

[1622] Response received:

[1623] Receive the response sent back from the server and display the information of the generated playlist.

[1624] Specific example

[1625] For example, if a user enters an abstract phrase like "a refreshing morning walk" and uploads a morning landscape image as visual information, the server analyzes these to search for relevant music. It then generates a playlist with a name like "Playlist for a refreshing morning walk" and provides the user with its URL. By clicking the provided URL, the user can listen to the appropriate music.

[1626] Example of a prompt

[1627] Abstract words: A refreshing morning walk

[1628] Visual information: Morning landscape image

[1629] This allows users to easily obtain music playlists that match their desired themes and emotions. This invention is particularly useful for users who are unfamiliar with music selection and for creating the right atmosphere for special events.

[1630] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1631] Step 1:

[1632] The user inputs abstract words or visual information into the application.

[1633] Input: Abstract words entered by the user (e.g., "A refreshing morning walk") and visual information (e.g., an image of a morning landscape).

[1634] Output: Request data in JSON format.

[1635] Specific operation: The smartphone app receives user input, converts it into JSON data, and prepares to send it to the server.

[1636] Step 2:

[1637] The user's terminal sends a request in JSON format to the server.

[1638] Input: Request data in JSON format entered by the user.

[1639] Output: Request data sent to the server.

[1640] Specific operation: The user's terminal sends JSON-formatted request data to the server via the network.

[1641] Step 3:

[1642] The server receives the request and extracts abstract words and visual information.

[1643] Input: Request data in JSON format.

[1644] Output: Abstract words and visual information.

[1645] Specific operation: The server parses the request, analyzes abstract words using an NLP model, and analyzes visual information using image processing (e.g., OpenCV).

[1646] Step 4:

[1647] The server calls the music provider's API based on the analysis results and searches for related songs.

[1648] Input: Keywords and parameters based on abstract words and visual information.

[1649] Output: A list of IDs for related songs.

[1650] Specific operation: The server calls the music provider's API (e.g., Spotify API), searches for and retrieves relevant songs based on the analysis results.

[1651] Step 5:

[1652] The server generates a new playlist based on the song ID it retrieves.

[1653] Input: List of song IDs.

[1654] Output: The ID and URL of the newly generated playlist.

[1655] Specific operation: The server calls the music provider's API again, creates a new playlist, and adds the retrieved songs to that playlist.

[1656] Step 6:

[1657] The server sends the generated playlist information back to the user.

[1658] Input: Playlist ID and URL.

[1659] Output: JSON-formatted response data sent to the user's terminal.

[1660] Specific operation: The server generates a JSON response containing the playlist name and URL, and sends it to the user's terminal.

[1661] Step 7:

[1662] The user's terminal receives the response and displays the information of the generated playlist.

[1663] Input: JSON response data from the server.

[1664] Output: The name and URL of the playlist to be displayed.

[1665] Specific operation: The user's terminal parses the response received from the server and displays the playlist name and URL to the user.

[1666] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1667] The embodiments for carrying out the present invention will be described in detail below. The present invention combines a system that receives abstract words or images from a user, generates a music playlist based on them, and provides it to the user, with an emotion engine that recognizes the user's emotions.

[1668] System Overview

[1669] The system works in conjunction with a server, a user's device, and a music streaming service. Based on abstract words, images, and emotions entered by the user, the server uses the music streaming service's API to search for songs and generate a playlist.

[1670] Server Processing

[1671] The server is built using a web framework (e.g., Flask) and has the following functions:

[1672] 1. Receiving the request:

[1673] The server receives requests from users. These requests are sent, for example, in JSON format.

[1674] 2. Acquisition of abstract words and images:

[1675] The server extracts abstract words and images (for example, "relax" and "sunset") from the request.

[1676] 3. Emotional analysis using an emotion engine:

[1677] The server uses an emotion engine to analyze user emotions based on their input. The emotion engine uses text-based analysis techniques to identify the user's emotions (e.g., happiness, sadness, excitement).

[1678] 4. Search for songs:

[1679] The server uses music streaming service APIs to search for songs based on abstract words, images, and sentiment analysis results. This uses libraries such as Spotipy.

[1680] 5. Create a playlist:

[1681] The server uses the ID of the retrieved song to generate a new playlist. The playlist name might be, for example, "Playlist for Relaxation and Sunsets."

[1682] 6. Add songs to the playlist:

[1683] The server adds songs to the generated playlist using the song IDs obtained in step 4.

[1684] 7. Generating the response:

[1685] The server prepares a response in JSON format, including the name and URL of the generated playlist, and returns it to the user.

[1686] User processing

[1687] 1. Input abstract words or images:

[1688] Users input words or images that represent the themes or emotions they are seeking.

[1689] 2. Submitting a request:

[1690] The user enters words and images, converts them to JSON format, and sends a POST request to the server.

[1691] 3. Receiving the response:

[1692] The user receives a response from the server. The response includes the name and URL of the generated playlist.

[1693] 4. Using playlists:

[1694] The user clicks the received playlist URL and plays the playlist on their music streaming service.

[1695] Specific example

[1696] For example, if a user enters abstract words like "relax" and "sunset," and the emotion engine interprets these words as "happiness," the server will search for songs based on this and generate a playlist named "Playlist for Relaxation and Sunset." The playlist contains the selected songs and is provided to the user via a URL. By clicking this URL, the user can play the playlist and enjoy a relaxed and happy atmosphere.

[1697] This invention allows users to easily access music playlists based on abstract images and emotions, which can be particularly helpful for users unfamiliar with music selection or when preparing for special events.

[1698] The following describes the processing flow.

[1699] Step 1:

[1700] The user opens a webpage or application using their device and enters an abstract word or image into a text field. This input could include, for example, "relax" or "sunset."

[1701] Step 2:

[1702] The system sends JSON data containing abstract words and images entered by the user to the server as a POST request.

[1703] Step 3:

[1704] The server receives a POST request and extracts JSON data from the request body. Specifically, the Flask framework is used to parse the data from the request.

[1705] Step 4:

[1706] The server retrieves abstract words and images from the JSON data (for example, "relax sunset" contained in the terms field).

[1707] Step 5:

[1708] The server uses an emotion engine to analyze abstract words and images to identify the user's emotions. For example, the words "relax" and "sunset" might be classified as the emotion "happiness."

[1709] Step 6:

[1710] The server uses the music streaming service's API to search for songs based on identified emotions and abstract words. It uses the Spotipy library to submit search queries.

[1711] Step 7:

[1712] The server analyzes the search results returned from the music streaming service and compiles a list of the IDs of related songs. For example, the list might contain the IDs of 10 songs.

[1713] Step 8:

[1714] The server uses the music streaming service's API to generate a new playlist. The playlist will be named "Playlist for Relaxation and Sunsets".

[1715] Step 9:

[1716] Add songs to the playlist generated by the server using the song IDs obtained in step 7. Specifically, add all song IDs to the playlist.

[1717] Step 10:

[1718] Prepare a response in JSON format containing the name and URL of the playlist generated by the server, and return it to the user.

[1719] Step 11:

[1720] The user receives a response from the server and checks its contents. The response includes the name and URL of the generated playlist.

[1721] Step 12:

[1722] The user clicks the response URL and opens the playlist on their music streaming service. When the playlist is played, songs based on the user's selected emotions and images are played.

[1723] (Example 2)

[1724] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1725] Conventional music playlist generation systems have been unable to adequately respond to users' abstract words and images, making it difficult to provide playlists that align with their emotions and preferences. Furthermore, there were no systems that combined emotional analysis to address the diverse needs of users.

[1726] The identification processing performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for receiving abstract words or images from the user, means for analyzing emotions from the received abstract words or images, means for searching for music based on the analyzed emotions and abstract words or images, and means for providing the user with a playlist generated from the search results. This makes it possible to analyze the user's emotions based on their abstract input and automatically generate and provide a music playlist that suits them.

[1727] "Abstract words and images" are words or visual concepts used to express a state or feeling that is difficult for the user to describe as concretely as possible.

[1728] "Methods for analyzing emotions" refer to technologies and algorithms used to infer and identify a user's emotions from the words and images they input. This includes natural language processing and machine learning models.

[1729] "Methods for searching for music" refers to technologies and methods that use external music databases or music streaming service APIs to find songs that match specific keywords or emotions.

[1730] "Methods for generating playlists" refer to technologies and methods that appropriately combine searched songs to create a single, continuous music list. This makes it possible to play music in sequence according to a specific theme or emotion.

[1731] "Means of receiving" refers to interfaces and communication protocols for receiving data from external sources via a network. This includes web frameworks that process HTTP requests.

[1732] "JSON format data" is an abbreviation for JavaScript Object Notation (JSON), a lightweight data exchange format. It represents data in a format that is easy for humans to read and easy for machines to parse.

[1733] Modes for carrying out the invention

[1734] The embodiments for carrying out the present invention will be described in detail below. The present invention combines a system that receives abstract words or images from a user, generates a music playlist based on them, and provides it to the user, with an emotion engine that recognizes the user's emotions.

[1735] The system operates in conjunction with a server, user terminals, and music streaming services. This will be explained below, along with specific hardware and software implementations.

[1736] Server Processing

[1737] The server is built using a web framework (e.g., Flask). The main processing steps are as follows:

[1738] 1. Receiving the request:

[1739] The server receives requests from users. This is done by users sending data containing abstract keywords and images in JSON format through their devices. For example, a user might enter the keywords "relax" and "sunset."

[1740] 2. Acquisition of abstract words and images:

[1741] The server parses the received JSON data and extracts the words and images entered by the user. This process uses the json module from Python's standard library.

[1742] 3. Emotional analysis using an emotion engine:

[1743] The server uses the Hugging Face Transformers library to analyze emotions from the user's input text. This analysis associates the keywords "relax" and "sunset" with the emotion "happiness." The emotion engine uses a pre-trained model to determine the emotion of the input text.

[1744] 4. Search for songs:

[1745] The server uses the Spotipy library to access the Spotify API and search for songs based on the keywords "relax," "sunset," and "happiness." This search query targets Spotify's song database to retrieve the appropriate song ID.

[1746] 5. Create a playlist:

[1747] The server generates a new playlist using the retrieved song ID. The playlist name will be based on the keywords entered by the user, such as "Playlist for Relaxation and Sunsets." The generated playlist will then be linked to the user's Spotify account.

[1748] 6. Add songs to the playlist:

[1749] The server adds songs to the generated playlist. This process uses the Spotipy library to send song addition requests to the Spotify API.

[1750] 7. Generating the response:

[1751] The server creates a JSON response containing the name of the generated playlist and its URL. For example, it might be returned to the user using Flask's response functionality. The JSON response includes data such as:

[1752] json

[1753] {

[1754] "playlist_name": "Playlist for relaxation and sunsets",

[1755] "playlist_url": "https: / / open.spotify.com / playlist / ..."

[1756] }

[1757] User processing

[1758] 1. Input abstract words or images:

[1759] Users input words or images that represent the themes or emotions they are seeking. For example, they might use words like "relax" or "sunset."

[1760] 2. Submitting a request:

[1761] The user enters words and images, converts them to JSON format, and sends a POST request to the server.

[1762] 3. Receiving the response:

[1763] The user receives a response from the server. The response includes the name and URL of the generated playlist.

[1764] 4. Using playlists:

[1765] The user clicks the received playlist URL and plays the playlist on their music streaming service.

[1766] Specific example

[1767] For example, if a user enters abstract words like "relax" and "sunset," and the emotion engine interprets these words as "happiness," the server will search for songs based on this and generate a playlist named "Playlist for Relaxation and Sunset." The playlist contains the selected songs and is provided to the user via a URL. By clicking this URL, the user can play the playlist and enjoy a relaxed and happy atmosphere.

[1768] Example of a prompt

[1769] "Please create a playlist for relaxation and sunsets."

[1770] Based on this input, the generative AI model performs sentiment analysis based on the user's abstract words, selects appropriate songs, and creates a playlist.

[1771] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1772] Step 1:

[1773] The user inputs abstract words or images.

[1774] Detailed explanation:

[1775] Users input abstract words or images that represent a theme or emotion using web forms or mobile apps from their own devices. For example, they might use words like "relax" or "sunset."

[1776] Inputs and outputs:

[1777] Input: Abstract words or images entered by the user (e.g., "relax," "sunset").

[1778] Output: The terminal generates data in JSON format.

[1779] Step 2:

[1780] The terminal sends the generated JSON data to the server.

[1781] Detailed explanation:

[1782] The terminal converts the words and images entered by the user into JSON format and sends them to a specific endpoint on the server as an HTTP POST request.

[1783] Inputs and outputs:

[1784] Input: JSON data containing abstract words or images entered by the user.

[1785] Output: JSON data received by the server.

[1786] Step 3:

[1787] The server parses the JSON data and extracts abstract words and images.

[1788] Detailed explanation:

[1789] The server uses the Flask web framework to parse the received JSON data and extract the words and images entered by the user. This process uses Python's json module.

[1790] Inputs and outputs:

[1791] Input: JSON data received by the server.

[1792] Output: Abstract words or images (e.g., "relax," "sunset").

[1793] Step 4:

[1794] The server uses an emotion engine to analyze the user's emotions based on their input.

[1795] Detailed explanation:

[1796] The server uses the Hugging Face Transformers library to analyze the sentiment of the user's input text. This analysis associates "relaxed" and "sunset" with the emotion of "happiness."

[1797] Inputs and outputs:

[1798] Input: Abstract words or images (e.g., "relax," "sunset").

[1799] Output: Analyzed emotion (e.g., "happiness").

[1800] Step 5:

[1801] The server uses the music streaming service's API to search for songs.

[1802] Detailed explanation:

[1803] The server uses the Spotipy library to access the Spotify API and search for songs based on the keywords "relax," "sunset," and "happiness." This search query targets Spotify's song database to retrieve the appropriate song ID.

[1804] Inputs and outputs:

[1805] Input: Abstract words, images, or analyzed emotions (e.g., "relaxed," "sunset," "happy").

[1806] Output: A list of searched song IDs.

[1807] Step 6:

[1808] The server generates a playlist using the acquired song ID.

[1809] Detailed explanation:

[1810] The server uses the Spotipy library to generate a new playlist via the Spotify API. The playlist name will be "Playlist for Relaxation and Sunsets," based on keywords entered by the user.

[1811] Inputs and outputs:

[1812] Input: A list of searched song IDs.

[1813] Output: The ID and name of the generated playlist.

[1814] Step 7:

[1815] Add songs to the playlist generated by the server.

[1816] Detailed explanation:

[1817] The server adds the searched songs to the generated playlist. This process uses the Spotipy library to send song addition requests to the Spotify API.

[1818] Inputs and outputs:

[1819] Input: The ID of the generated playlist and a list of the searched song IDs.

[1820] Output: A playlist with the added songs.

[1821] Step 8:

[1822] The server prepares a JSON response containing the name and URL of the generated playlist and returns it to the user.

[1823] Detailed explanation:

[1824] The server creates a JSON response containing the name and URL of the generated playlist and returns it to the user. This response is returned using Flask's response functionality.

[1825] Inputs and outputs:

[1826] Input: The ID and name of the generated playlist.

[1827] Output: JSON response containing the playlist name and URL.

[1828] Step 9:

[1829] The user receives the playlist URL returned from the server and plays it.

[1830] Detailed explanation:

[1831] The user receives a response from the server via their device and clicks the provided playlist URL. This allows them to play the playlist on their music streaming service.

[1832] Inputs and outputs:

[1833] Input: JSON response containing the playlist name and URL.

[1834] Output: The music playlist that the user will play.

[1835] (Application Example 2)

[1836] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1837] In modern virtual stores and online shopping experiences, there is a lack of personalized service based on users' emotions and moods. In particular, the selection of music is often generic rather than tailored to an individual's mood or theme, which degrades the quality of the user's purchasing experience.

[1838] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1839] In this invention, the server includes means for receiving abstract words or images from the user, means for searching for music based on the received abstract words or images and the user's emotions, means for providing the user with a playlist generated from the search results, and means for automatically generating and providing music playlists to be played within the virtual store. This makes it possible to improve the quality of online shopping through a personalized music experience based on the user's emotions and mood.

[1840] "Abstract words and images" refer to vague words or visual concepts used to express the feelings or themes that the user is experiencing.

[1841] "Emotions" refer to the user's psychological and sensory state and are classified into emotional categories such as happiness, sadness, and excitement.

[1842] "Searching for music" refers to the process of finding appropriate songs from the music streaming service's database based on user input.

[1843] A "playlist" refers to a collection of songs selected based on a specific theme or emotion, and is a list of music arranged to be played sequentially.

[1844] A "virtual store" is a virtual commercial space operated on the internet where users can browse and purchase products online.

[1845] "Automatic generation" refers to the process by which a system creates specific deliverables or data based on user input and algorithms, without requiring manual operation.

[1846] "To provide" refers to giving information or services to users so that they can use them.

[1847] This invention is a system that receives abstract words or images from a user, generates a music playlist based on them, and plays it within a virtual store. The following describes in detail the configurations for implementing this system.

[1848] System Configuration

[1849] The system operates in conjunction with servers, user terminals, and music streaming services.

[1850] Server Processing

[1851] The server is built using a web framework (e.g., Flask) and includes the following software components:

[1852] 1. Receiving the Request: The server receives a request from the user. The request is sent in JSON format.

[1853] 2. Retrieving abstract words and images: The server extracts abstract words and images from the request. Examples include "relax" and "sunset."

[1854] 3. Sentiment Analysis: The server uses an emotion engine to analyze the user's emotions. The emotion engine utilizes text analysis libraries such as TextBlob to identify the user's emotions.

[1855] 4. Searching for songs: The server searches for songs using the API of a music streaming service (e.g., the Spotipy library).

[1856] 5. Playlist Generation: The server uses the IDs of the retrieved songs to generate a new playlist that will be played within the virtual store.

[1857] 6. Providing the playlist: The server returns the name and URL of the generated playlist to the user in JSON format.

[1858] User terminal processing

[1859] 1. Input of abstract words and images: Users input words or images in text format that represent the theme or emotion they are seeking.

[1860] 2. Sending the Request: The user sends a POST request to the server with the entered words and images in JSON format.

[1861] 3. Receiving the response: The user receives the response returned from the server.

[1862] 4. Playlist Usage: Users click on the received playlist URL, play the playlist on their device, and enjoy the music experience of the virtual store.

[1863] Specific example

[1864] For example, if a user enters "I want to enjoy a relaxed shopping experience," the server receives this query and uses an emotion analysis engine to analyze the emotions "relaxed" and "happy." Next, it uses the APIs of music streaming services such as Spotify to search for songs that suit these emotions and generates a playlist named "Relaxing Shopping Playlist." The user is then provided with the URL of this playlist, which they can click to play within the virtual store. In this way, users can enjoy a richer shopping experience through music that matches their emotions and themes.

[1865] Example of a prompt

[1866] The following are examples of prompts that the user will enter into the system:

[1867] I want to enjoy a relaxed shopping experience. Please suggest some music to help me spend a pleasant time in the soft evening sunlight.

[1868] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1869] Step 1:

[1870] User input

[1871] In order to obtain a specific musical experience in a virtual store, users input their mood or theme as abstract words or images in text format. For example, they might input the phrase, "I want to enjoy a relaxed shopping experience."

[1872] Input: Text representing the user's theme or emotions.

[1873] Output: Text containing abstract words and images.

[1874] Step 2:

[1875] Send a request

[1876] The user terminal converts the input words and images into JSON format and sends a POST request to the server. The request reaches the server via the network.

[1877] Input: Text of abstract words or images

[1878] Output: Request data in JSON format

[1879] Step 3:

[1880] Receiving a request

[1881] The server receives a JSON-formatted request sent from the user's terminal.

[1882] Input: Request data in JSON format

[1883] Output: Analysis results of the request data

[1884] Step 4:

[1885] Extraction of abstract words and images

[1886] The server extracts abstract words and images (for example, "relax") from the received JSON data.

[1887] Input: Request data in JSON format

[1888] Output: Text containing abstract words and images.

[1889] Step 5:

[1890] Emotion analysis

[1891] The server performs sentiment analysis on abstract words and images. It uses a sentiment engine like TextBlob to identify the user's emotions (for example, "happiness").

[1892] Input: Text of abstract words or images

[1893] Output: Results of emotion analysis (emotion categories)

[1894] Step 6:

[1895] Search for songs

[1896] The server uses the results of sentiment analysis to search for songs using the API of a music streaming service (e.g., Spotify). The search query combines abstract words with the analyzed sentiment.

[1897] Input: Abstract words or categories of emotions

[1898] Output: Search results (list of song IDs)

[1899] Step 7:

[1900] Playlist generation

[1901] The server generates a new playlist using the acquired song ID. The playlist name might be, for example, "Relaxing Purchase Playlist".

[1902] Input: List of song IDs

[1903] Output: Information about the generated playlist (playlist ID, name)

[1904] Step 8:

[1905] Playlists provided

[1906] The server sends a JSON response to the user's device containing the name and URL of the generated playlist.

[1907] Input: Information about the generated playlist (playlist ID, name)

[1908] Output: JSON response containing playlist information

[1909] Step 9:

[1910] Received response

[1911] The user's device receives the JSON response sent back from the server and obtains the playlist URL.

[1912] Input: JSON response containing playlist information

[1913] Output: Playlist URL

[1914] Step 10:

[1915] Play playlist

[1916] The user clicks on the URL of the acquired playlist and plays the playlist within the virtual store using their device.

[1917] Input: Playlist URL

[1918] Output: A music experience where a playlist is played.

[1919] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1920] The data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of the data generation model 58 is ChatGPT (Internet Search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1921] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[1922] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1923] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[1924] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[1925] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[1926] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[1927] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[1928] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[1929] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[1930] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[1931] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[1932] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1933] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[1934] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[1935] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[1936] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[1937] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[1938] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[1939] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted as being incorporated by reference.

[1940] The following is further disclosed regarding the embodiments described above.

[1941] (Claim 1)

[1942] A means of receiving abstract words and images from users,

[1943] A method for searching for songs based on abstract words or images received,

[1944] A system that includes means for providing users with playlists generated from search results.

[1945] (Claim 2)

[1946] The system according to claim 1, wherein the means for receiving abstract words or images is to receive data in JSON format from a user via a network.

[1947] (Claim 3)

[1948] The system according to claim 1, wherein the means for searching for music is to use an API of an external music streaming service to search for music.

[1949] "Example 1"

[1950] (Claim 1)

[1951] A means of receiving abstract words and images from users,

[1952] A method for searching for songs based on abstract words or images received,

[1953] A means of providing users with playlists generated from search results,

[1954] A means of receiving and analyzing user requests through a web framework,

[1955] A means of searching for related songs using the API of a music streaming service,

[1956] A method for generating a playlist from search results and adding songs,

[1957] A system that includes a means for generating the name and URL of the generated playlist as a response.

[1958] (Claim 2)

[1959] The system according to claim 1, wherein the means for receiving abstract words or images is to receive data in JSON format from a user via a network.

[1960] (Claim 3)

[1961] The system according to claim 1, wherein the means for searching for music is to use an API of an external music streaming service to search for music.

[1962] "Application Example 1"

[1963] (Claim 1)

[1964] Means of receiving abstract words and visual information from users,

[1965] A means of searching for songs based on abstract words and visual information received,

[1966] A means of providing users with playlists generated from search results,

[1967] Methods for creating playlists using the API of music providers,

[1968] A system that includes a means of returning the URL of the provider to the user.

[1969] (Claim 2)

[1970] The system according to claim 1, wherein the means for receiving abstract words or visual information is to receive data in JSON format from a user via a network.

[1971] (Claim 3)

[1972] The system according to claim 1, wherein the means for searching for music is to use an API of an external music provider to search for music.

[1973] "Example 2 of combining an emotion engine"

[1974] (Claim 1)

[1975] A means of receiving abstract words and images from users,

[1976] A means of analyzing emotions from abstract words and images received,

[1977] A method for searching for songs based on analyzed emotions and abstract words and images,

[1978] A means of providing users with playlists generated from search results,

[1979] A system that includes this.

[1980] (Claim 2)

[1981] The system according to claim 1, wherein the means for receiving abstract words or images is to receive data in JSON format from a user via a network.

[1982] (Claim 3)

[1983] The system according to claim 1, wherein the means for searching for music is to use an API of an external music streaming service to search for music.

[1984] "Application example 2 when combining with an emotional engine"

[1985] (Claim 1)

[1986] A means of receiving abstract words and images from users,

[1987] A means of searching for music based on abstract words and images received and the user's emotions,

[1988] A means of providing users with playlists generated from search results,

[1989] A means of automatically generating and providing music playlists to be played within virtual stores,

[1990] A system that includes this.

[1991] (Claim 2)

[1992] The system according to claim 1, wherein the means for receiving abstract words or images is to receive data in JSON format from a user via a network.

[1993] (Claim 3)

[1994] The system according to claim 1, wherein the means for searching for music is to use an API of an external music streaming service to search for music. [Explanation of symbols]

[1995] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of receiving abstract words and images from users, A method for searching for songs based on abstract words or images received, A system that includes means for providing users with playlists generated from search results.

2. The system according to claim 1, wherein the means for receiving abstract words or images is to receive data in JSON format from a user via a network.

3. The system according to claim 1, wherein the means for searching for music is to use an API of an external music streaming service to search for music.

Images