Information processing device and information processing method

Abstract

This invention provides a technology for suggesting effective songs according to the atmosphere of the participants in a karaoke room. [Solution] An information processing device according to one embodiment includes: an acquisition unit that acquires in real time atmosphere information indicating the atmosphere of a space when multiple users are singing karaoke in a space, based on data output from a measuring means that measures the reactions of other users when one user is singing; a selection unit that selects songs based on the atmosphere information and playlists obtained from the user terminals of the multiple users; and a presentation unit that presents the selected songs.

Description

Technical Field

[0004] ,

[0006] , , , ,

[0005] , , ,

[0007] , ,

[0001] The present invention relates to a technique for proposing singing music to karaoke participants.

Background Art

[0002] Techniques for proposing music for singing are known. For example, Patent Document 1 discloses an invention that notifies recommended singing music based on a playlist of music stored in a mobile terminal possessed by each participant when a plurality of participants enjoy karaoke simultaneously.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The invention described in Patent Document 1 simply selected the music common in the playlists of a plurality of people as the recommended music.

[0005] On the other hand, the present invention provides a technique for effectively proposing music according to the atmosphere of participants in a karaoke room.

Means for Solving the Problems

[0006] An information processing apparatus according to an aspect of the present disclosure includes an acquisition unit that acquires in real time atmosphere information indicating the atmosphere of a space obtained based on data output from a measurement unit that measures the reactions of other users when a certain user is singing when a plurality of users are performing karaoke in a certain space, a selection unit that selects music based on the atmosphere information and a playlist obtained from the user terminals of the plurality of users, and a presentation unit that presents the selected music.

[0007] An information processing method relating to another aspect of the present disclosure includes the steps of: acquiring in real time atmosphere information indicating the atmosphere of a space when a user is singing in a space where multiple users are performing karaoke; selecting a song based on the atmosphere information and a playlist obtained from the user terminals of the multiple users; and presenting the selected song. [Effects of the Invention]

[0008] According to the present invention, it is possible to suggest songs that are effective in accordance with the atmosphere of the participants in the karaoke room. [Brief explanation of the drawing]

[0009] [Figure 1] A diagram illustrating the system configuration of an information processing system 1 according to one embodiment. [Figure 2] A diagram illustrating the functional configuration of information processing system 1. [Figure 3] A diagram illustrating the hardware configuration of the information processing device 10. [Figure 4] A sequence chart illustrating the method for estimating atmosphere and selecting music in Information Processing System 1. [Figure 5] A diagram illustrating a database of 1000 karaoke participants. [Figure 6] A diagram illustrating the song selection criteria list 2000. [Figure 7] A diagram illustrating the display screen of user terminal 20. [Figure 8] A diagram illustrating the display screen of the user terminal 20 in a modified example. [Modes for carrying out the invention]

[0010] 1. Structure Figure 1 is a diagram illustrating the system configuration of an information processing system 1 according to one embodiment. In this example, the information processing system 1 (or simply referred to as the system) is a system for selecting and suggesting songs that are more effective in accordance with the atmosphere of a space when multiple participants (an example of users) are performing karaoke in a room in a karaoke establishment. Karaoke refers to the act of users singing along to the performance of music. This is often provided to users as an entertainment service in a space such as a room installed in a specific store or facility (hereinafter referred to as the "karaoke space"). Atmosphere refers to the sensory state of a place, consisting of emotions, moods, and stress levels of individual or group users. Here, it is used as a term to indicate the mental state of each user or group in a space where karaoke is being performed.

[0011] Karaoke, a popular form of entertainment, is enjoyed in a wide variety of situations. For example, it's often used by friends of the same age, groups with shared hobbies, and families. It's also frequently used by acquaintances who aren't particularly close, by communities of diverse ages, and within companies, all under various pretexts. In fact, karaoke spaces often serve as a place for communication among people with diverse values, fostering camaraderie and building relationships. However, even in such cases, when going to karaoke with people of different ages or hobbies, differences in musical tastes among participants can lead to situations where only some people are enjoying themselves.

[0012] Conventional related technologies have mainly focused on providing a karaoke space, and the introduction of technologies to proactively address the aforementioned challenges is still insufficient. Especially recently, considering the improvement in service quality accompanying technological innovations such as sensing technology, AI-related technologies, and applications to smart devices, more effectively improving the atmosphere of the karaoke space is an essential challenge. Therefore, the present invention aims to analyze the situation in the karaoke space or the reactions of participants in real time, select the optimal song based on this information, and provide a space that all participants can enjoy. The outline of the system in this invention is as follows.

[0013] Information processing system 1 includes an information processing device 10, a user terminal 20, and a karaoke terminal 30. In this example, each component of the system is connected to one another via a network 9. In this example, network 9 is a computer network such as the Internet or a mobile network.

[0014] The information processing device 10 is an information processing device or server device in the information processing system 1. In this example, the information processing device 10 monitors the status of users who are present in a specific karaoke space and identifies the playlists of songs that these users usually listen to. The information processing device 10 supports the creation of a more comfortable karaoke space by selecting and suggesting songs that match the current atmosphere based on the user's status and playlist. Specifically, the information processing device 10 can analyze the user's facial expressions, voice, tone, and movements (or gestures, etc.) using cameras, microphones, and motion sensors installed in the space, and can identify the user's age, emotions, and level of excitement. Further details will be described later.

[0015] The user terminal 20 is a terminal owned and used by a user of the information processing system 1, that is, a participant in karaoke. The user terminal 20 includes, for example, a smartphone, tablet, or personal computer. In this example, the user terminal 20 is carried by a user who enters the karaoke space and is used as a device for processing song selection, such as providing the user's song playlist to the information processing device 10. The user terminal 20 has an application used in the technology provided by the present invention (hereinafter simply referred to as the "song suggestion app") pre-installed on it. This song suggestion app can provide, for example, a service that suggests songs in the information processing system 1 to a user participating in karaoke, a function to obtain the user's song playlist by linking with a dedicated music listening app different from the song suggestion app, or a function to perform song distribution and listening within the song suggestion app.

[0016] The karaoke terminal 30 is a terminal owned and used by a store operator or other entity that provides karaoke services to users in the information processing system 1. The karaoke terminal 30 includes, for example, a smartphone, tablet, or personal computer. In this example, the karaoke terminal 30 can work in conjunction with various devices to analyze the user's state in the karaoke space in real time. For example, the karaoke terminal 30 acquires measurement data measured by a camera, microphone, and motion sensor and outputs it to the information processing device 10. Alternatively, it can output control signals to devices such as lighting or a mirror ball (MB in Figure 1) in response to a request from the information processing device 10, and operate the devices. In addition, the karaoke terminal 30 can work in conjunction with an input device for users to request songs from the karaoke equipment, a so-called Denmoku (registered trademark), and can receive operations from users to execute functions in the information processing system 1. The karaoke terminal 30 itself includes various forms, such as a terminal installed in a karaoke space or a device managed by each store.

[0017] FIG. 2 is a diagram illustrating the functional configuration of the information processing system 1. In this embodiment, the information processing apparatus 10 includes functional blocks (or components) such as an acquisition unit 11, a selection unit 12, a presentation unit 13, a specification unit 14, a reception unit 15, an output unit 16, a provision unit 17, a storage unit 191, and a control unit 192. In this example, the storage unit 191 stores various data, programs, and software including, for example, a database. In this example, the control unit 192 performs various controls.

[0018] The acquisition unit 11 acquires in real time atmosphere information indicating the atmosphere of the room obtained based on data output from a measurement unit 40 that measures the reactions of other users when a certain user is singing. In this example, the atmosphere information is information indicating the mental states of a plurality of users cohabiting in a space or each user, or the atmosphere in the space composed of those users, and includes, for example, information corresponding to parameters such as emotion, excitement level (or degree of interest), and age. The emotion among the atmosphere information refers to a predetermined subjective mental state for each user and is a parameter classified into items such as joy, anger, sorrow, and happiness. The excitement level is an index obtained by quantifying the ups and downs of the mood and the degree of interest in a song during karaoke or singing. The age refers to the age of each user estimated by a camera or the like. The method of using the atmosphere information will be described later.

[0019] The selection unit 12 selects a song based on the atmosphere information and a playlist obtained from the user terminals 20 of a plurality of users. For example, when the atmosphere information includes information regarding the age of the user, the selection unit 12 can select a song from the songs included in the playlist based on the age of the user. Specific examples of the method of selecting a song by the selection unit 12 will be described later. The presentation unit 13 presents the selected song. In this example, the presentation unit 13 can present song candidates selected for each user via the user terminal 20 and encourage singing.

[0020] The identification unit 14 identifies an indicator related to the atmosphere (hereinafter referred to as "song indicator") associated with the songs included in the playlist. In this example, the song indicator is assigned to each song based on predetermined criteria and is managed by a database that records a list of songs. As an example of how the song indicator is used, the selection unit 12 selects songs according to the combination of the atmosphere indicated by the atmosphere information and the atmosphere indicated by the song indicator. In this example, the song indicator has variables that correspond to (or are common to) emotion, excitement level, and age. Under specific judgment criteria corresponding to the atmosphere information and the song indicator corresponding to the song, the song selection process becomes executable.

[0021] The reception unit 15 receives a designation regarding the user who will sing (hereinafter referred to as the "singing user") from among multiple users. In this example, the singing user refers to a user who is waiting in line to sing, considering which song to sing, or simply present in the karaoke space, separate from the user who is currently singing (hereinafter referred to as the "user in progress"). Therefore, in this invention, it is used together with wording that includes information about the order of future singing, for example, as the next singing user, the second singing user, or the nth singing user. As an example of processing related to the singing user, the selection unit 12 selects a song using the aforementioned atmosphere information of the user who will sing. The selection unit 12 also selects a song based on the playlist of the next user to sing.

[0022] The output unit 16 outputs control signals corresponding to the atmosphere information to an external device 60 installed in the room where karaoke is being performed, in order to control the atmosphere. In this example, if the external device 60 is a light fixture installed in the room, the output unit 16 outputs a control signal related to the brightness of the light fixture. The control method for the external device 60 will be described later.

[0023] The provisioning unit 17 provides input means 50, which can input user requests, with options for user feedback regarding atmosphere information. In this example, the feedback includes various evaluations from the user in the form of questions, surveys, or polls. The information processing device 10 can reflect the user feedback in the selection of music and control of the atmosphere. For example, the selection unit 12 selects music based on the atmosphere information corresponding to the options. The output unit 16 also outputs a control signal to control the external device 60 based on the atmosphere information corresponding to the options.

[0024] The measurement means 40 is a functional element for measuring the state of users in the karaoke space, particularly the atmosphere of the space, and includes various devices for performing measurements on the target, such as a camera, microphone, and motion sensor. In this example, the camera is installed in the karaoke space to photograph users in the space. The microphone detects the user's voice, singing, or conversation and records it as audio data. This includes microphones installed in the space to collect sound, separate from the singing microphones that are pre-installed as part of the karaoke equipment. The motion sensor (hereinafter simply referred to as "sensor") can capture the user's movements, particularly hand gestures, reactions, and other gestures, and record them together with image data.

[0025] The input means 50 is a functional element for receiving various requests from the user in the information processing system 1, and includes terminal equipment such as input terminals placed in each room of a karaoke establishment. In this example, the input means 50 can display information for receiving user feedback, i.e., evaluations, regarding the atmosphere in the karaoke space. For example, the user selects information representing the current atmosphere from the questionnaire-style options displayed on the input means 50 and inputs it. This information is acquired by the information processing device 10 and used, for example, for processing related to song selection and suggestion in the information processing system 1. It should be noted that the introduction of the input means 50 is often envisioned to be used in conjunction with standard karaoke controllers that allow operations such as song searching and song reservation requests.

[0026] External devices 60 are equipment and devices pre-installed within the karaoke space, and for example, they implement functions to effectively control the atmosphere of the room. In this example, external devices 60 include, for example, lighting equipment, air conditioning equipment, or equipment related to performance effects. External devices 60 are operated and managed for each room by karaoke terminals 30 that receive requests from the information processing device 10. Furthermore, external devices 60 effectively change the atmosphere within the karaoke space by receiving requests from users and responding to control according to predetermined conditions.

[0027] Figure 3 illustrates the hardware configuration of the information processing device 10. Physically, the information processing device 10 is configured as a computer including a processor 101, memory 102, storage 103, communication device 104, input device (optional), display device (optional), and a bus connecting these. Each of these devices operates on power supplied from a battery (not shown). In the following description, the term "device" can be read as a circuit, device, unit, etc. The hardware configuration of the information processing device 10 may include one or more of the devices shown in Figure 3, or it may be configured without some of the devices. Alternatively, multiple devices with different enclosures may be connected via communication to constitute the information processing device 10.

[0028] Each function in the information processing device 10 is realized by loading predetermined software (programs) onto hardware such as the processor 101 and memory 102, which allows the processor 101 to perform calculations, control communication by the communication device 104, and control at least one of the reading and writing of data in the memory 102 and storage 103.

[0029] The processor 101 controls the entire computer, for example, by running the operating system. The processor 101 may consist of a central processing unit (CPU) that includes interfaces with peripheral devices, control units, arithmetic units, registers, etc. Alternatively, a baseband signal processing unit or a call processing unit may be implemented by the processor 101.

[0030] The processor 101 reads programs (program code), software modules, data, etc., from at least one of the storage 103 and the communication device 104 into the memory 102 and executes various processes accordingly. The program used is one that causes the computer to execute at least a part of the operations described later. The functional blocks of the information processing device 10 are stored in the memory 102 and may be realized by control programs that run on the processor 101. Various processes may be executed by one processor 101, or they may be executed simultaneously or sequentially by two or more processors 101. The processor 101 may be implemented by one or more chips. The program may also be transmitted to the information processing device 10 via a telecommunications line.

[0031] Memory 102 is a computer-readable recording medium and may consist of at least one of the following: ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), RAM (Random Access Memory), etc. Memory 102 may also be called a register, cache, main memory, etc. Memory 102 can store executable programs (program code), software modules, etc., for carrying out the method according to this embodiment.

[0032] The storage 103 is a computer-readable recording medium and may consist of at least one of the following: an optical disc such as a CD-ROM (Compact Disc ROM), a hard disk drive, a flexible disk, a magneto-optical disk (e.g., a compact disc, a digital multipurpose disc, a Blu-ray® disc), a smart card, flash memory (e.g., a card, a stick, a key drive), a floppy® disk, a magnetic strip, etc. The storage 103 may also be called an auxiliary storage device.

[0033] The communication device 104 is hardware (transceiver / receiver device) for communicating between computers via at least one of a wired network and a wireless network, and is also referred to as a network device, network controller, network card, communication module, etc.

[0034] Each device, such as the processor 101 and memory 102, is connected by a bus for communicating information. The bus may be configured using a single bus, or different buses may be used for each device.

[0035] The information processing device 10 may include hardware such as a microprocessor, a digital signal processor (DSP), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA (Field Programmable Gate Array), and some or all of the functional blocks may be implemented by such hardware. For example, the processor 101 may be implemented using at least one of these hardware components.

[0036] In this example, the program stored in storage 103 includes a program (hereinafter referred to as the "server program") that causes the computer to function as a server in the information processing system 1. When the processor 101 is executing the server program, the processor 101, memory 102, storage 103, and communication device 104 are examples of functional blocks for operating the information processing device 10. The processor 101 is an example of a selection unit 12, a specification unit 14, and a control unit 192. At least one of the memory 102 and storage 103 is an example of a storage unit 191. The communication device 104 is an example of an acquisition unit 11, a presentation unit 13, a reception unit 15, an output unit 16, and a provision unit 17.

[0037] While a detailed explanation will be omitted, the user terminal 20 and the karaoke terminal 30 are computers having a processor, memory, storage, communication device, input device, and output device; specifically, they are, for example, smartphones, tablet terminals, or personal computers. In this example, the programs stored in the storage of the user terminal 20 and the karaoke terminal 30 include programs that allow the computers to function as clients in the information processing system 1 (hereinafter referred to as "client programs"). The configuration of the information processing system 1 has now been described. Next, the operation of the information processing system 1 will be described.

[0038] 2.Operation Figure 4 is a sequence chart illustrating the method for estimating the atmosphere and selecting songs in the information processing system 1. Here, it is assumed that multiple users are singing karaoke in a certain space. In this example, the function of suggesting songs to the singing users is provided as a function of a dedicated song suggestion application (an example of a client program) that is pre-installed on the user terminal 20. For example, each user may operate a song selection terminal (for example, a so-called Denmoku®) or prepare a microphone after entering the karaoke space, and as part of this process, they may launch the song suggestion application on the user terminal 20.

[0039] In step S101, the user terminal 20 receives pre-specified information from the user. In this example, the pre-specified information includes the operation of inputting basic information into the song suggestion application in the information processing system 1 before starting karaoke. For example, the user terminal 20 can request the user to input information such as the location of the karaoke session (e.g., the store and room number), the duration of the session (e.g., start and end times), the number of participating users, and the age group of each user on the display screen of the song suggestion application. The user inputs the pre-specified information according to the presented request. This process is applied to each user in the same karaoke space. As a result, the information processing system 1 can compare the age group of participants specified by the user with the age group of participants determined based on the analysis results of images taken by the camera (an example of the measurement means 40), and identify the age of each user participating in the karaoke session.

[0040] In step S102, the information processing device 10 acquires various data from the user terminal 20. The data output by the user terminal 20 includes data related to the pre-specified settings entered in step S101 and data related to the playlist of the user who owns the user terminal 20. When each user enters the karaoke space they have been directed to, they notify the information processing device 10 via the user terminal 20 of the location where they will be singing karaoke. This includes processes that allow the information processing device 10 to identify the karaoke space where the user is located, such as inputting room information into the app or reading a QR code (registered trademark) on an operating terminal pre-installed in the karaoke space.

[0041] Furthermore, regarding user playlist data, the user terminal 20, for example, triggers the execution of an API (Application Programming Interface) integration function in the music suggestion app when it receives pre-specified input from the user. In this example, the music suggestion app can obtain data on playlists of songs that the user frequently listens to from the music listening app. The criteria for songs to be extracted as playlists are predefined in the information processing system 1 from perspectives such as the number of times they have been listened to, recent listening history, or whether they have been registered as favorites, and can be anything. The user terminal 20 provides the information processing device 10 with data on playlists along with the pre-specified information.

[0042] In step S103, the information processing device 10 records the user's playlist in the database. In steps S101 to S102, the information processing device 10 can obtain data from each user terminal 20 of each user staying in the same karaoke space. In this example, based on the obtained pre-specified information, the information processing device 10 identifies the playlist for each of the multiple users in a room in a karaoke establishment and records it in the database. Here, we will describe the database for managing user playlists for each karaoke space.

[0043] Figure 5 illustrates the karaoke participant database 1000. The karaoke participant database 1000 contains multiple records related to users who are using the song suggestion app among the users participating in karaoke. Each record corresponds to information for each user (or member) belonging to a group of users in the same karaoke space, with the group being treated as a single unit. Each record includes a group ID, member ID, attribute information, playlist information, mood, excitement level, and room information. The group ID is identification information attached to a group consisting of multiple users in the same karaoke space, identified based on pre-specified information obtained from the user terminal 20. The member ID is unique identification information for each user registered as a member of the song suggestion app. Attribute information is information related to the attributes of the app member, including, for example, age and gender. This information is pre-entered and recorded in the app during member registration, etc., and shared with the information processing device 10 as appropriate. Playlist information is data shared from music listening apps, etc., recording a list (i.e., playlist) of songs that each user frequently listens to. In this example, each song in the list is accompanied by information such as the song title, artist, number of views, most recent play, favorite status, or other song-related information. Emotion and excitement levels will be described later. Room information is identification information assigned to a specific karaoke space where multiple user groups are staying. Room information is identified based on pre-specified information obtained from the user terminal 20 and has unique information such as a specific room number in a particular karaoke establishment.

[0044] Here, in the karaoke participant database 1000, each song in the playlist is associated with a unique piece of data called a "song index" (not shown in the diagram). In this example, the information processing system 1 can obtain the song index defined for each song by linking with an existing music listening service (or app, etc.). The song index includes indicators such as excitement level, danceability, and popularity. These are all examples of indicators that show the atmosphere of a song. In this example, the excitement level is an index based on a perspective common to the user's excitement level included in the atmosphere information. The excitement level and danceability are defined within a range of values, for example, from 0.0 to 1.0. The higher the value, the greater the excitement level and danceability are considered to be. Popularity is a ranking of songs supported by a specific age group, and includes indicators (or rankings, etc.) according to the number of plays for each age group. These indicators are important indicators when the information processing device 10 selects songs. The method of song selection will be described later.

[0045] The karaoke participant database 1000 is primarily intended to collect and record data about user groups performing karaoke at any given time, and the information processing device 10 can update and manage the data as needed. The information processing device 10 records data obtained from user terminals 20 and karaoke terminals 30, associating it with the users performing karaoke in each specific space. Next, the process by which the information processing device 10 acquires atmosphere information will be explained.

[0046] Returning to Figure 4, in step S104, the karaoke terminal 30 performs measurements in the karaoke space using the measurement means 40. This process is initiated, for example, when the target user group completes registration at the store counter (i.e., when they are about to enter the space where they will perform karaoke). The karaoke terminal 30 works in conjunction with a registration terminal installed in the store to activate the measurement means 40 in the karaoke space that the user group is about to enter. Alternatively, if a user performs a specific operation (for example, searching for a song or reserving a song to sing) on ​​an operation terminal pre-placed in the karaoke space, the measurement by the measurement means 40 may be started. Now, the measurement method by the measurement means 40 in the karaoke space will be described.

[0047] As a prerequisite, the measurement means 40 is an individual or integrated device including, for example, a camera, microphone, and motion sensor. These devices are pre-installed in the karaoke space and can output the measured data to the karaoke terminal 30 via a network or the like. Each of the devices constituting the measurement means 40 can be connected to and cooperate with one another. Therefore, the karaoke terminal 30 can record data acquired from the measurement means 40, such as images, sounds, and actions, in an associated manner.

[0048] Here, let's consider the situation immediately after the karaoke terminal 30 begins measurement by the measurement means 40. In this example, the user group has already entered the karaoke space, and pre-selection or playlist sharing has been completed by the user terminal 20. In this state, the karaoke terminal 30 identifies each user in the group based on the data acquired from the measurement means 40. For example, the karaoke terminal 30 identifies the face of each user based on the image captured by the camera or the sound collected by the microphone, and also estimates the age of each user. In this example, the karaoke terminal 30 links the users in the space with the users recorded in the database, referring to attribute information of each user shared in advance from the information processing device 10. If possible, it is preferable that users register their facial photos etc. in the app in advance using the user terminal 20, and that each user is identified from the facial information etc. However, if privacy is a concern, the karaoke terminal 30 only needs to be able to identify at least the total number of users in the space and the attribute information of each individual (e.g., age or gender). The karaoke terminal 30 may identify each user based on data such as the number of participants and their age groups, which have been pre-selected from the user terminal 20. Alternatively, any other method may be used.

[0049] Next, consider the case where a specific user has already started singing. In this case as well, the measurement means 40 continues to measure using the camera, microphone, and motion sensor. The measurement by the measurement means 40 continues while karaoke is being performed, that is, while the participants are in the karaoke space. In this example, the karaoke terminal 30 identifies the user who is singing and other users (an example of other users) based on the data acquired from the camera and microphone, etc. The measurement data acquired in real time by the karaoke terminal 30 from the measurement means 40 is provided to the information processing device 10 via the network 9.

[0050] In step S105, the information processing device 10 acquires measurement data from the karaoke terminal 30. Based on the identification information assigned to the karaoke terminal 30, the information processing device 10 identifies the correspondence between the karaoke store and its room number, which has been registered in advance. This allows the information processing device 10 to identify which karaoke space (or which user group) the measurement data acquired from the karaoke terminal 30 belongs to. The information processing device 10 records the history of the measurement data in association with the user group in the karaoke participant database 1000. In steps S104 to S105, the measurement by the measurement means 40 and the transmission of data from the karaoke terminal 30 to the information processing device 10 are carried out continuously and periodically while karaoke is being performed.

[0051] In step S106, the information processing device 10 estimates the atmosphere in the karaoke room based on the measurement data. Here, "atmosphere" refers to, for example, the emotions and excitement levels of individual users, and in particular to the atmosphere of the singing users who are about to sing (i.e., users other than the user currently singing). Therefore, the information processing device 10 identifies the mental state of each user by applying emotion recognition technology (Affective Computing). More specifically, the process can be broadly divided into the following two processes.

[0052] In the first stage, the information processing device 10 analyzes the participant's facial expressions, voice, tone, or movements based on data obtained from a camera, microphone, or motion sensor. In this example, the information processing device 10 obtains information about each user's emotions as a result of the analysis by applying various analysis methods. For example, in facial expression analysis, the information processing device 10 extracts each user's facial expression from the facial image and identifies emotions such as joy, anger, or sadness. In voice analysis, the information processing device 10 analyzes the tone and intonation of the voice from the voice data and estimates stress or excitement levels. In motion analysis, the information processing device 10 analyzes motion data obtained from the motion sensor and estimates emotions such as anxiety or tension.

[0053] In the second stage, the information processing device 10 determines each user's emotion and excitement level based on the analysis results. In this example, the database of the information processing device 10 has predefined correspondences between the analysis results and the emotion and excitement levels. Therefore, the information processing device 10 can determine each user's emotion and excitement level by applying the analysis results obtained in the first stage to the predefined correspondences. In this example, emotions are classified into predetermined categories such as happiness, sadness, anger, confusion, disgust, surprise, calmness, fear, and unknown. The excitement level is an indicator of the user's mood excitement or interest in the music playing in the room, and is defined within a range of values ​​such as 0.0 to 1.0. A higher excitement level indicates a more excited mood, while a lower excitement level indicates a lower tension or calmer state. However, a low excitement level does not necessarily mean that the user is depressed; it is possible to determine the level of stress the user is experiencing based on information such as whether the user's emotion is "disgust" or "calmness."

[0054] Referring again to Figure 5, the karaoke participant database 1000 includes items that record emotion and excitement level among the records corresponding to each user. The information processing device 10 can analyze the user's mood in real time and update each record in the database with the results. Alternatively, the information processing device 10 may periodically record the history of emotional changes in the database in chronological order. The information recorded in the karaoke participant database 1000 is used when recommending songs to sing to the user. Next, the method of selecting and suggesting recommended songs will be explained. In the following, it is assumed that the participants have already started karaoke and the user is in the middle of singing.

[0055] In step S107, the information processing device 10 receives a designation for the next singing user from the user terminal 20. In this example, the next singing user may be waiting to sing after the current singer has finished, or they may be searching for a song to sing. Such waiting is often seen in karaoke, but in this embodiment, the singer is designated and recommended songs are extracted during this time. The information processing device 10 may receive the designation for the next singing user from any user terminal 20 in the same room, and in this case, it is also possible to implement a system in which another user suggests recommended songs to the next singer. The following explanation will describe a situation in which the current singer is user U1 and user U2 is waiting as the next singing user.

[0056] In step S108, the information processing device 10 extracts recommended song candidates from the playlist information in the karaoke participant database 1000. The information processing device 10 first determines the population of songs to be selected. In this example, the information processing device 10 extracts songs that are common to the playlist information of all or some of the users belonging to the target group. The information processing device 10 determines the extracted songs as the population of songs to be selected (hereinafter referred to as the "song population").

[0057] Furthermore, when extracting the song population, the population may not consist of songs common to all playlists, but rather songs common to at least the next singing user (e.g., user U2) and a specific person (e.g., the third singing user or user U3 with a low level of enthusiasm). Alternatively, the information processing device 10 may identify only the playlist of the next singing user as the population. Regarding the extraction requirements, the population may not be limited to songs that are exactly the same between playlists, but may also be defined as songs that share a common genre (e.g., K-POP or 90s J-POP), and any such criteria may be defined in advance in a database or similar.

[0058] Next, the information processing device 10 identifies (or obtains) the song indicators for each song included in the song population in order to further narrow down the song population and extract song candidates based on each user's mood information. The information processing device 10 narrows down the song population according to the combination of mood information (the mood indicated by it) and song indicators (the mood indicated by it). Here, a list of selection criteria for which the information processing device 10 narrows down the songs to be sung will be described.

[0059] Figure 6 is an example of the Song Selection Criteria List 2000. The Song Selection Criteria List 2000 (an example of criteria) includes a list of multiple items related to the combination of atmosphere information (in this example, the emotions of specific karaoke participants) and the atmosphere of the room (in this example, the level of excitement). Each item represents content corresponding to the combination of the participants' emotions and the level of excitement. Here, "content" refers to the conditions for narrowing down the song indicators. In the example in Figure 6, the participants' emotions are classified into three categories, and the level of excitement in the room is divided into two requirements. The participants' emotions are classified into three categories, from the first emotional group to the third emotional group, from the perspective of emotional items or stress levels. The level of excitement is distinguished by two different requirements: whether all participants maintain a numerical level within a common range (hereinafter referred to as "the first requirement"), or whether there is at least one participant with a low level of excitement (hereinafter referred to as "the second requirement"). The Song Selection Criteria List 2000 shows song indicators defined for each song according to each pattern, namely the song selection criteria related to the level of excitement, ease of dancing, and popularity.

[0060] For example, consider a case where users U1 and U2 belong to the first emotional group, which is relatively low-stress, while user U3 is not enjoying themselves much and belongs to the second emotional group. In this case, whether to select the first or second emotional group as the baseline emotional group may be determined according to the settings implemented in the information processing device 10. For example, whether to prioritize the mood of user U2, who is the next singer, or to prioritize consideration for user U3, who has a low level of excitement, is determined by the system settings or by the operator's specifications. In this example, if the information processing device 10 decides to use the second emotional group as the basis for song selection (i.e., considers user U3's emotions), it similarly identifies the requirement regarding the level of excitement as the second requirement. The information processing device 10 obtains the song selection criteria determined by the combination of the second emotional group and the second requirement: "excitement level within the range of the previous song + 0.3, ease of dancing within the range of the previous song + 0.3, and popularity within the top 10% of the age group of some participants (i.e., user U3)." Furthermore, the song selection criteria may include conditions related to the age range of the participants.

[0061] The information processing device 10 extracts song candidates that meet the conditions from the song population in accordance with the song selection criteria described above. In this example, the information processing device 10 may assign priorities to the songs that meet the conditions, such as songs that are ranked higher among the multiple song candidates. For example, the information processing device 10 can determine (i.e., select) three song candidates in the order of song S22, song S31, and song S25. According to this example, songs that match the emotions of the target user can be selected as candidates from the user's playlist.

[0062] Returning to Figure 4, in step S109, the information processing device 10 presents song candidates for the specified singing user to the user terminal 20. Basically, when a user terminal 20 specifies the next singing user, the information processing device 10 will present song candidates corresponding to that singing user. Therefore, the following explanation will use the display screen on the user terminal 20 corresponding to the processing from step S107 to step S109 as an example.

[0063] Figure 7 is an example of the display screen of the user terminal 20. Figure 7 is an example of a schematic diagram using the display screen of the user terminal 20 for specifying a singer and presenting song candidates. In this example, the user terminal 20 presents the user with two types of display screens corresponding to the following display contents. Field F1 represents an operation screen that accepts specification regarding the next singing user (indicated as "singer" in the figure) in step S107. For example, if user U2 is operating the user terminal 20 in the figure, by selecting the option "You", user U2 can view song candidates. Field F2 is a display screen that shows a list of song candidates extracted by the information processing device 10 from the playlist information. The user terminal 20 displays the candidates songs S22, S31, and S25 extracted from the song population on the screen in a predetermined arrangement. In this example, the information processing device 10 can present song S31 as a song that falls in the top 10% of the age group (e.g., 40s) of user U3, who has a low level of excitement, based on the song selection criteria. This allows the information processing device 10 to suggest songs that control the atmosphere within the karaoke space to the next singing user.

[0064] As a result, the information processing device 10 can select and suggest songs more effectively according to the atmosphere in the karaoke space. Therefore, it is possible to avoid situations where only some people can enjoy themselves, given the diverse musical preferences and age groups of participants in the various situations of karaoke. Furthermore, the information processing system 1 can analyze the situation in the karaoke space and the reactions of the participants in real time and select the most suitable songs based on this information. This is expected to provide an environment where all participants can enjoy themselves.

[0065] Returning to Figure 4, in step S110, the information processing device 10 outputs a control instruction to the external device 60 to the karaoke terminal 30. In this example, if there is a participant (e.g., user U3) who is not enjoying themselves in the karaoke space, the information processing device 10 can use the room's lighting or effects functions to control the atmosphere.

[0066] For example, the information processing device 10 obtains the functions of an external device 60 that can be controlled in the room in advance from the karaoke terminal 30. The information processing device 10 generates a control signal to control the functions of the external device 60 according to the tempo of the song being sung at the time or the emotions of the participants. Alternatively, a control instruction to the karaoke terminal 30 may be generated. The information processing device 10 controls the external device 60 via the karaoke terminal 30. For example, when the room is lively, disco-style lighting (e.g., a mirror ball) can be activated, or during relaxation time, the brightness of the lights can be adjusted to a softer color tone. In this way, the information processing device 10 can control the external device 60 in a way that allows participants to enjoy themselves more according to the atmosphere in the karaoke space.

[0067] 3. Variant The present invention is not limited to the embodiments described above, and various modifications are possible. Several modifications are described below. Two or more of the items described below may be combined and applied.

[0068] (1) Information processing system 1 The hardware and network configurations in the information processing system 1 are not limited to those illustrated in the embodiments. The information processing system 1 may have any hardware and network configurations as long as they can realize the required functions. For example, multiple physical devices may cooperate to function as the information processing system 1. For example, at least some of the functions of the information processing device 10 may be implemented in the user terminal 20 or the karaoke terminal 30. For example, the user terminal 20 or the karaoke terminal 30 may have at least some of the functions of the information processing device 10 related to atmosphere estimation or song selection.

[0069] (2) Information processing device 10 Some of the functions of the information processing device 10 may be implemented on other servers. These servers may be, for example, physical servers or virtual servers (including so-called clouds). Furthermore, the correspondence between functional elements and hardware is not limited to those illustrated in the embodiments. For example, at least some of the functions described in the embodiments as being implemented on the information processing device 10 may be implemented on other devices or systems, and conversely, at least some of the functions described as being implemented on other devices or systems may be implemented on the information processing device 10. In this example, at least some of the functional elements implemented by the information processing device 10 may be implemented on other devices or systems, for example, AI, applications, or other software. The information processing device 10 may use machine learning models to estimate the atmosphere of the participants.

[0070] (3) User terminal 20, Karaoke terminal 30 The user terminal 20 and the karaoke terminal 30 are not limited to those illustrated in the embodiment. The user terminal 20 and the karaoke terminal 30 may perform the above-described processing using any display screen, input device, or UI. The user terminal 20 may accept user specifications based on any display screen in the song suggestion application, present song candidates, or provide various services. The karaoke terminal 30 can provide information about a specific karaoke space to the information processing device 10. In this example, the karaoke terminal 30 may output data acquired from the measurement means 40, input means 50, or external device 60 to the information processing device 10 for each room (or group).

[0071] (4) Measuring means 40, input means 50, external device 60 The measuring means 40, input means 50, and external device 60 are not limited to those exemplified in the embodiment. The measuring means 40 may be any device. The measuring means 40 may also be various sensors mounted on the user terminal 20, in which case the information processing device 10 may acquire measurement data directly from the user terminal 20. The input means 50 may be any device, and may include the UI (User Interface) of the user terminal 20 instead of a pre-installed Denmoku (registered trademark) in the room. The external device 60 may be any device or equipment, and the functions of the external device 60 and the method of controlling the atmosphere may be predefined in a database in the information processing device 10.

[0072] (5) Method for estimating the atmosphere and selecting music The sequence chart shown in Figure 4 is merely an example of operation, and the operation of the information processing system 1 is not limited thereto. Some of the illustrated operations may be changed or omitted, the order may be changed, or new operations may be added. In step S106, the estimation of the atmosphere in the karaoke space may be performed by any method. In this example, the information processing device 10 may estimate the atmosphere based on a user request entered into the input means 50 installed in the room.

[0073] Alternatively, this process may involve obtaining feedback from the user terminal 20 instead of the input means 50. Figure 8 illustrates the display screen of the user terminal 20 in a modified example. Figure 8 is an example of a schematic diagram using the display screen of the user terminal 20 for inputting feedback. In this example, the information processing device 10 outputs options regarding song requests to the user terminal 20. Field F3 is an input screen for receiving user input regarding song requests. In this example, the user selects the option "More upbeat" in field F3, for example, if the atmosphere in the room is subdued. In step S108, the information processing device 10 reflects this request when selecting a song for the next singing user. For example, the information processing device 10 extracts candidates from the next singing user's playlist, focusing on songs with a high level of excitement.

[0074] In step S109, the information processing device 10 may present song candidates to the user terminals 20 of users other than the next singing user (hereinafter referred to as "other user terminals 20"). Referring again to Figure 8, in this example, field F4 is an input screen for presenting song candidates to other user terminals 20 and accepting user requests. For example, on the user terminal 20 of a user other than the next singing user (e.g., user U2), (e.g., user U3), song candidates extracted from the song population are presented. User U3 can select the song they want to listen to from the song candidates (e.g., song S31). Based on such voting, the information processing device 10 can update user U2's song candidates or notify users of which songs are desired by the participants. In addition to selecting song candidates, this voting function may also be used for feedback regarding the atmosphere of the room.

[0075] In step S110, the information processing device 10 may control the external device 60 based on the user's mood feedback described above. Alternatively, the information processing device 10 may display an operation screen or the like for controlling the external device 60 to the user terminal 20.

[0076] (6) Estimation of atmosphere The atmosphere information referenced when extracting song candidates in step S108 is not limited to representing the atmosphere of a specific user. The information processing device 10 may quantify (or index) the atmosphere for multiple (e.g., all) users in the room and process these statistically. The information processing device 10 uses the representative value (e.g., mean or mode) from the statistical processing as the atmosphere information for the room (overall) to extract song candidates.

[0077] (7) Singing order The method for determining the singing order is not limited to the methods illustrated in the embodiments. For example, the karaoke terminal 30 or the information processing device 10 may automatically determine the singing order among users. Any method may be used to determine the singing order, such as by age or in alphabetical order.

[0078] (8) Database (data) The database (or the data itself) of the information processing system 1 shown in Figures 5 and 6 is not limited to those illustrated in the embodiments. Any data may be registered in the database in this example. Any information may be attached to the playlist information or atmosphere information in the karaoke participant database 1000. For example, tags, keywords, or labels may be attached to identify the playlist information or atmosphere information. The playlist information may include information about the songs in the user's playlist, such as the song title, artist name, playback time, or release year for each song. The atmosphere information may include any indicators and may record information about the atmosphere of the individual user or the entire room.

[0079] The song selection criteria list 2000 may specify a method for selecting songs based on any pattern of atmospheric information. In addition to the combination of emotion and excitement level described above, the user's age may also be included as a criterion. Furthermore, the definition of song indicators included in the song selection criteria may be defined in any way. If a song that fits the song selection criteria list 2000 is not included in the song population, the information processing device 10 may refer to a database of songs other than playlist information and extract a song that fits the conditions.

[0080] (9) Others The various programs executed by processor 101 may be provided by downloading them over a network such as the Internet, or they may be provided recorded on a computer-readable non-temporary recording medium such as a DVD-ROM. Each processor may be, for example, a CPU, an MPU (Micro Processing Unit), or a GPU (Graphics Processing Unit).

[0081] The block diagrams used in the description of the above embodiments show functional units. These functional blocks (components) are realized by any combination of at least one of hardware and software. Furthermore, the method of realizing each functional block is not particularly limited. That is, each functional block may be realized using one device that is physically or logically coupled, or it may be realized using two or more physically or logically separated devices that are directly or indirectly connected (for example, using wired or wireless connections). A functional block may also be realized by combining the above one device or the above multiple devices with software.

[0082] Functions include, but are not limited to, judgment, decision, determination, calculation, calculation, processing, derivation, investigation, search, confirmation, reception, transmission, output, access, resolution, selection, selection, establishment, comparison, assumption, expectation, assumption, broadcasting, notifying, communicating, forwarding, configuring, reconfiguring, allocating (mapping), and assigning. For example, a functional block (configuration part) that enables transmission is called a transmitting unit or transmitter. As mentioned above, the method of implementation is not particularly limited.

[0083] For example, the information processing device 10 in one embodiment of the present disclosure may function as a computer that performs the processing of the present disclosure.

[0084] Each aspect or embodiment described in this disclosure may be applied to at least one of the following systems: LTE (Long Term Evolution), LTE-A (LTE-Advanced), SUPER 3G, IMT-Advanced, 4G (4th generation mobile communication system), 5G (5th generation mobile communication system), FRA (Future Radio Access), NR (New Radio), W-CDMA (registered trademark), GSM (registered trademark), CDMA2000, UMB (Ultra Mobile Broadband), IEEE 802.11 (Wi-Fi (registered trademark)), IEEE 802.16 (WiMAX (registered trademark)), IEEE 802.20, UWB (Ultra-WideBand), Bluetooth (registered trademark), and other appropriate systems, as well as next-generation systems extended based thereon. Furthermore, multiple systems may be applied in combination (for example, a combination of at least one of LTE and LTE-A with 5G).

[0085] The processing procedures, sequences, flowcharts, etc., of each aspect or embodiment described in this disclosure may be reordered, provided they do not contradict each other. For example, the methods described in this disclosure present various step elements in an exemplary order and are not limited to the specific order presented.

[0086] Input and output information may be stored in a specific location (e.g., memory) or managed using a management table. Input and output information may be overwritten, updated, or appended to. Output information may be deleted. Input information may be sent to other devices.

[0087] The determination may be made by a value represented by 1 bit (0 or 1), by a boolean value (true or false), or by a numerical comparison (for example, a comparison with a predetermined value).

[0088] Although the present disclosure has been described in detail above, it will be clear to those skilled in the art that the present disclosure is not limited to the embodiments described herein. The present disclosure can be implemented in modified and altered forms without departing from the intent and scope of the present disclosure as defined by the claims. Therefore, the descriptions in the present disclosure are illustrative and not intended to be restrictive in any way.

[0089] Software should be broadly interpreted to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executable files, execution threads, procedures, functions, etc., whether they are called software, firmware, middleware, microcode, hardware description languages, or by any other name. Furthermore, software, instructions, information, etc., may be transmitted and received via a transmission medium. For example, if software is transmitted from a website, server, or other remote source using at least one of wired technologies (such as coaxial cable, fiber optic cable, twisted pair, or digital subscriber line (DSL)) and wireless technologies (such as infrared or microwave), at least one of these wired and wireless technologies is included in the definition of a transmission medium.

[0090] The information, signals, etc., described herein may be represented using any of the following different technologies. For example, data, instructions, commands, information, signals, bits, symbols, chips, etc., which may be referred to throughout the above description, may be represented by voltage, current, electromagnetic waves, magnetic fields or magnetic particles, optical fields or photons, or any combination thereof. Terms used herein and terms necessary for understanding this disclosure may be replaced with terms having the same or similar meaning.

[0091] Furthermore, the information, parameters, etc., described in this disclosure may be expressed using absolute values, relative values ​​from a predetermined value, or corresponding other information.

[0092] In this disclosure, the phrase "based on" does not mean "based solely on" unless otherwise specified. In other words, the phrase "based on" means both "based solely on" and "based at least on."

[0093] Any reference to elements using designations such as “First,” “Second,” etc., as used in this disclosure does not generally limit the quantity or order of those elements. These designations may be used in this disclosure as a convenient way to distinguish between two or more elements. Accordingly, references to the First and Second elements do not imply that only two elements may be employed, or that the First element must precede the Second element in any way.

[0094] In the above-described configuration of each device, the term "part" may be replaced with "means," "circuit," "device," etc.

[0095] Where the terms “include,” “including,” and variations thereof are used in this disclosure, these terms are intended to be inclusive, as is the term “comprising.” Furthermore, the term “or” as used in this disclosure is not intended to mean exclusive OR.

[0096] In this disclosure, if articles are added by translation, such as a, an, and the in English, this disclosure may include the fact that the noun following these articles is plural.

[0097] In this disclosure, the term "A and B are different" may mean "A and B are different from each other." The term may also mean "A and B are each different from C." Terms such as "separate" and "combine" may be interpreted similarly to "different." [Explanation of symbols]

[0098] 1...Information processing system, 10...Information processing device, 20...User terminal, 30...Karaoke terminal, 40...Measurement means, 50...Input means, 60...External device, 9...Network, 11...Acquisition unit, 12...Selection unit, 13...Presentation unit, 14...Specification unit, 15...Reception unit, 16...Output unit, 17...Provision unit, 191...Storage unit, 192...Control unit, 101...Processor, 102...Memory, 103...Storage, 104...Communication device, 1000...Karaoke participant database, 2000...Song selection criteria list, F...Field, S...Song, U...User

Claims

1. In a space where multiple users are performing karaoke, an acquisition unit acquires in real time atmosphere information indicating the atmosphere of the space, based on data output from a measurement means that measures the reactions of other users while one user is singing. A selection unit that selects songs based on the aforementioned atmosphere information and playlists obtained from the user terminals of the multiple users, A display unit that presents the selected music and An information processing device having

2. It has a special unit that identifies an indicator related to the atmosphere associated with the songs included in the aforementioned playlist, The selection unit selects the song according to the combination of the atmosphere indicated by the atmosphere information and the atmosphere indicated by the index. The information processing apparatus according to claim 1.

3. The system has a reception unit that receives designations for the user who will perform the song from among the aforementioned multiple users, The selection unit selects the song using the atmosphere of the user performing the song as the atmosphere information. The information processing apparatus according to claim 2.

4. The system has a reception unit that receives designations for the user who will perform the song from among the aforementioned multiple users, The selection unit selects the song based on the playlist of the user who will sing next. The information processing apparatus according to claim 2.

5. The aforementioned atmosphere information includes information regarding the user's age, The selection unit selects songs from the playlist based on the user's age. The information processing apparatus according to claim 1.

6. To control the atmosphere, it has an output unit that outputs a control signal corresponding to the atmosphere information to an external device installed in the room where the karaoke is being performed. The information processing apparatus according to claim 1.

7. The external device is a light fixture installed in the room. The output unit outputs the control signal relating to the brightness of the lighting. The information processing apparatus according to claim 6.

8. The input means, which is capable of receiving the user's request, has a providing unit that provides options for user feedback regarding the atmosphere information. The selection unit selects the song based on the atmosphere information corresponding to the selection. The information processing apparatus according to claim 1.

9. The input means, which is capable of receiving the user's request, has a providing unit that provides options for user feedback regarding the atmosphere information. The output unit outputs the control signal to control the external device based on the atmosphere information corresponding to the selection. The information processing apparatus according to claim 6.

10. Computers In a space where multiple users are performing karaoke, the step of acquiring in real time atmosphere information indicating the atmosphere of the space based on data output from a measurement device that measures the reactions of other users when one user is singing, The steps include selecting a song based on the aforementioned atmosphere information and the playlists obtained from the user terminals of the multiple users, The step of presenting the selected song and An information processing method having

Images