System

Abstract

A system is provided.SOLUTION: The system includes a means for generating a rhythm, a means for transmitting a notification signal according to a specific rhythm to a user terminal, a means for receiving a timing of a tap from the user terminal, a means for evaluating a score of the user based on the received timing of the tap, and a means for notifying the user terminal of the evaluated score.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional tapping games primarily compete on users' reflexes and simple speed, but have the drawback of making it difficult to measure rhythmic sense and accuracy. Furthermore, there is no means for accurately evaluating the timing discrepancy of a user's taps, which can result in a lack of fairness and enjoyment in the game. The present invention aims to provide a game system that encourages users to tap according to a specific rhythm and evaluates the accuracy of that tapping, allowing players to compete in a more refined sense of rhythm. [Means for solving the problem]

[0005] The present invention includes a means for the game server to generate a rhythm and transmit a notification signal according to the specific rhythm to the user terminal. The system also includes a means for the user terminal to record the timing of tapping and transmit the recorded timing to the game server. The game server evaluates the user's score based on the recorded tapping timing and notifies the user terminal of the evaluation result. This allows for a fair and accurate evaluation of the user's sense of rhythm and tapping accuracy.

[0006] A "game server" is a computer system that generates rhythms, sends notification signals to user terminals, receives tap timings, and evaluates them.

[0007] A "rhythm" is a patterned temporal instruction that occurs at specific time intervals.

[0008] A "user terminal" is a device such as a smartphone or tablet that a user uses as an interface.

[0009] A "notification signal" is a signal or message sent from a game server to a user terminal to prompt the user to take a specific action.

[0010] "Tap timing" is time information about the moment when the user touches the terminal.

[0011] The "means for evaluating the score" is an algorithm or program that calculates the score by comparing the timing of the user's taps with the rhythm.

[0012] A "recording means" is a memory or software that stores the timing of a user's taps. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2]1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0014] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0015] First, the terms used in the following description will be explained.

[0016] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0017] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0018] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0019] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0020] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0021] [First embodiment]

[0022] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0023] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0024] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0025] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0026] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0027] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0028] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0029] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0030] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0031] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0032] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0033] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0034] This invention is a game system in which users compete for points by tapping in accordance with a specific rhythm. Below, each component of the system, its interactions, and the specific flow of operation will be described in detail.

[0035] server

[0036] The server plays a central role in the game. First, the server randomly generates a rhythm at the start of the game. This is the pattern of time intervals that the user must follow. This rhythm information becomes the basis for the user to tap accurately.

[0037] The server then sends a rhythmic notification signal to the user's device, allowing the user to receive tapping instructions in real time. The server records the timing of the user's taps and analyzes the tap interval data.

[0038] The server then evaluates the user's score based on the recorded timing of the taps. The evaluation is done by calculating the difference between the user's tap timing and the pre-generated rhythm. The closer the user's taps are to the rhythm, the higher the score.

[0039] Terminal

[0040] The user terminal functions as a user interface. When the game starts, the terminal displays the message "Game Start" to the user. When the terminal receives a rhythm notification signal from the server, it sends the message "Tap now!" to the user.

[0041] The device has a function to record the timing of taps, and saves the time each time the user taps. The saved tap timings are sent to the server in real time. At the end of the game, the device receives the evaluation results from the server and displays them to the user.

[0042] User

[0043] The user taps the device in accordance with the instructions on the device. The user's goal is to tap as accurately as possible to the rhythm instructed by the server. As the game progresses, the user taps multiple times, and the timing of each tap is sent to the server.

[0044] Specific examples

[0045] For example, suppose the rhythm randomly generated by the server at the start of the game is a pattern of "1.1 seconds, 0.8 seconds, 1.2 seconds." According to this rhythm, the first tap must be made 1.1 seconds after the start of the game, the next tap 0.8 seconds after that, and then 1.2 seconds after that.

[0046] 1. Based on these time intervals, the server will sequentially send a "Tap now!" notification signal to the device.

[0047] 2. The user follows the instructions on the device and taps at the specified timing.

[0048] 3. The device records the time of each tap and transmits it to the server in real time.

[0049] 4. After the game ends, the server compares the user's tap timing with the rhythm and calculates the score.

[0050] 5. Finally, the device displays the score to the user and provides feedback on the game results.

[0051] In this way, the user, terminal, and server work together to realize a new type of game system that measures the user's sense of rhythm and accuracy.

[0052] The processing flow will be explained below.

[0053] Step 1:

[0054] The server initializes the game and sets the game duration and rhythm pattern, which is randomly generated, for example, "1.1 seconds, 0.8 seconds, 1.2 seconds."

[0055] Step 2:

[0056] The user starts up the terminal and prepares to start the game. The terminal connects to the server and waits for a signal to start the game.

[0057] Step 3:

[0058] The server sends a signal to the terminal to start the game and simultaneously records the start time of the game. The terminal displays the message "Game Start" to the user.

[0059] Step 4:

[0060] The server sends a notification signal saying "Tap now!" to the device at specified intervals according to the rhythm pattern. The device displays this notification on its screen.

[0061] Step 5:

[0062] The user sees the notification and taps the device at the specified timing. The timing of each tap is recorded.

[0063] Step 6:

[0064] The device sends the timing of the user's taps to the server in real time, and the server receives and records them.

[0065] Step 7:

[0066] When the game duration ends, the server sends a signal to the terminal to end the game, causing the terminal to display a message saying "Game End." At the same time, the server starts the process of evaluating the score based on the user's tap timing data.

[0067] Step 8:

[0068] The server compares the rhythm pattern with the user's tap timing and calculates a score based on the accuracy of each tap. The closer the user's taps are to the rhythm, the higher the score.

[0069] Step 9:

[0070] The server sends the evaluated score to the user's terminal, which displays the score to the user and provides feedback on the game result.

[0071] Step 10:

[0072] The user checks their score on the terminal and ends the game.

[0073] The above are the processing steps of the present invention. By tapping along a specific rhythm, users can compete with each other in terms of rhythmic sense and tapping accuracy.

[0074] Example 1

[0075] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0076] Rhythm games require a system that allows users to tap accurately in time with the rhythm and evaluate the results in real time. However, current systems face the challenge of urging users to tap accurately in accordance with randomly generated rhythms and accurately recording and evaluating the timing of their taps. To solve this challenge, improvements to the user interface and faster data communication are required.

[0077] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0078] In this invention, the server includes means for generating a rhythm, means for transmitting a notification signal according to a specific rhythm to a user terminal, means for receiving tap timings from the user terminal, means for evaluating a user's score based on the received tap timings, means for notifying the user terminal of the evaluated score, means for sequentially transmitting notification signals according to the rhythm, means for evaluating the score by comparing the received tap timings with a randomly generated rhythm, and means for notifying the user terminal of the score result. This allows the user to tap faithfully to the specified rhythm, and the results can be accurately evaluated and displayed in real time.

[0079] A "game server" is a central system that generates rhythms, receives tap timings, evaluates scores, and notifies users of the results of their scores in a game in which users tap in accordance with the rhythm.

[0080] The "rhythm" refers to a pattern of time intervals that serves as a basis for the user to perform tapping.

[0081] The "notification signal" is a signal sent from the server to the user terminal to notify the timing of tapping in accordance with the rhythm.

[0082] A "user terminal" is a device operated by a user playing a game, which receives notification signals from the server and records the timing of taps.

[0083] "Tap timing" refers to the exact time when a user taps the device, and this data is used for evaluation on the server.

[0084] The "score" is an evaluation value calculated based on how closely the user's tap timing matches the rhythm.

[0085] "Random" means that something is determined randomly without following any particular rule, and is used here in the context of rhythm generation.

[0086] The "evaluation means" is a function that compares the received tap timing with the rhythm and calculates the user's score.

[0087] The "notification means" is a function for transmitting the evaluated score to the user terminal and displaying it to the user.

[0088] This invention is a game system in which users compete for points by tapping in accordance with a specific rhythm. Below, each component of the system, its interactions, and the specific flow of operation will be described in detail.

[0089] server

[0090] The server plays a central role in the game. When the game starts, the server randomly generates a specific rhythm. This rhythm is the pattern of time intervals that the user must follow. The rhythm is generated using a random number generation algorithm and a library of time patterns (e.g., the random module). This rhythm information becomes the basis for the user to tap accurately.

[0091] Next, the server sequentially sends rhythmic notification signals to the user's device, allowing the user to receive tapping instructions in real time. Communication protocols such as HTTP requests and WebSockets are used to send the notification signals. Specifically, the server sends a "tap" notification 1.1 seconds later.

[0092] The server records the timing of the user's taps and evaluates the user's score based on that data. The evaluation is done by calculating the difference between the user's tap timing and a pre-generated rhythm. This evaluation uses absolute difference calculations and a statistical library (e.g., NumPy). The closer the user's taps are to the rhythm, the higher the score will be.

[0093] Terminal

[0094] The user device functions as a user interface. When the game starts, the device displays the message "Game Start" to the user. When the device receives a rhythm notification signal from the server, it sends the message "Tap now!" to the user. To display the notification, a UI library (e.g., React, SwiftUI) is used to draw the message on the screen.

[0095] The device has a function to record tap timing, and saves the time every time the user taps in milliseconds. The saved tap timing is sent to the server in real time. This communication also uses HTTP requests and WebSockets. At the end of the game, the device receives the evaluation result from the server and displays it to the user. A UI library is used again for this, and a message such as "Your score is 85 points" is displayed.

[0096] User

[0097] The user taps the device in accordance with the instructions of the device. The user's goal is to tap as accurately as possible to the rhythm instructed by the server. The timing of the taps is recorded on the device and transmitted to the server in real time.

[0098] Specific operation example

[0099] For example, suppose the rhythm randomly generated by the server at the start of the game is a pattern of "1.1 seconds, 0.8 seconds, 1.2 seconds." In this case, the user must tap the first time 1.1 seconds after the start of the game, then tap the next time 0.8 seconds later, and then tap the next time 1.2 seconds later.

[0100] 1. Based on these time intervals, the server will sequentially send a "Tap now!" notification signal to the device.

[0101] 2. The user follows the instructions on the device and taps at the specified timing.

[0102] 3. The device records the time of each tap and transmits it to the server in real time.

[0103] 4. After the game ends, the server compares the user's tap timing with the rhythm and calculates the score.

[0104] 5. Finally, the device displays the score to the user and provides feedback on the game results.

[0105] Examples of prompt statements

[0106] Here are some example prompts to input to a generative AI model:

[0107] "You are now participating in a musical note game. Please tap according to the following rhythm: 1.1 seconds, 0.8 seconds, 1.2 seconds. Please tap accurately at each timing."

[0108] In this way, by realizing this game system through cooperation between the server, terminals, and users, the users' sense of rhythm can be measured and enjoyed.

[0109] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0110] Step 1:

[0111] The server generates the rhythm.

[0112] Input: Rhythm generation algorithm (e.g., random module)

[0113] Processing: To generate rhythms, a random number generation algorithm is used to randomly determine time intervals (e.g., 1.1 seconds, 0.8 seconds, 1.2 seconds).

[0114] Output: Generated rhythm pattern (e.g. "1.1 seconds, 0.8 seconds, 1.2 seconds")

[0115] Step 2:

[0116] The server transmits a rhythm notification signal to the user terminal.

[0117] Input: Generated rhythm pattern

[0118] Processing: The server sequentially sends "Tap now!" notification signals after a specified time (e.g., 1.1 seconds, then 0.8 seconds, then 1.2 seconds) based on the generated rhythm pattern. This transmission uses a communication protocol such as an HTTP request or WebSocket.

[0119] Output: Notification signal to the user device (e.g. "Tap now!")

[0120] Step 3:

[0121] The device displays a notification signal.

[0122] Input: Notification signal from the server

[0123] Processing: The device displays the notification signal received from the server in the user interface. Specifically, a UI library (e.g., React, SwiftUI) is used to display the message "Tap now!" on the screen.

[0124] Output: The message that is displayed to the user (e.g., "Tap now!")

[0125] Step 4:

[0126] The user taps.

[0127] Input: The message displayed on the terminal

[0128] Action: The user taps the screen in response to the instruction from the device, "Tap now!"

[0129] Output: User's tap action

[0130] Step 5:

[0131] The device records the timing of your taps.

[0132] Input: User tap action

[0133] Processing: The device records the exact moment the user tapped, down to the millisecond, using the system clock or a timer library (e.g., Date.now(), System.currentTimeMillis()).

[0134] Output: Recorded tap timing

[0135] Step 6:

[0136] The device sends the tap timing to the server.

[0137] Input: Recorded tap timing

[0138] Processing: The device sends the recorded tap timings to the server in real time via HTTP requests or WebSockets in JSON format.

[0139] Output: Tap timing data to the server

[0140] Step 7:

[0141] The server evaluates the tap timing.

[0142] Input: Received tap timing, generated rhythm pattern

[0143] Processing: The server compares the user's tap timing with the generated rhythm and calculates the user's score. Absolute difference calculations and statistical libraries (e.g., NumPy) are used. For example, a high score is awarded if the tap timing is within ±50 milliseconds of the rhythm.

[0144] Output: Calculated score

[0145] Step 8:

[0146] The server transmits the evaluation results to the terminal.

[0147] Input: Calculated score

[0148] Processing: The server sends the score results in JSON format to the device, again using an HTTP request or WebSocket.

[0149] Output: Score result data to the terminal

[0150] Step 9:

[0151] The terminal displays the evaluation results.

[0152] Input: Score result data from the server

[0153] Processing: The device displays the received score result to the user. This is done by using the UI library again, and displays a message such as "Your score is 85 points."

[0154] Output: Scoring results displayed to the user

[0155] In this way, the rhythm tap game system is realized by the server, terminals, and users operating in cooperation with each other.

[0156] (Application example 1)

[0157] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0158] In autonomous vehicles, there is a need for a means to check the driver's attention and reaction time to improve safety. In conventional autonomous driving systems, the driver may monitor the driving situation for long periods of time without operating the vehicle, which can lead to a decline in attention. Therefore, there is a lack of effective methods to maintain the driver's attention and promote appropriate reactions.

[0159] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0160] In this invention, the server includes means for generating a rhythm, means for transmitting a notification signal according to a specific rhythm to a user terminal, means for receiving tap timing from the user terminal, means for evaluating a user's score based on the received tap timing, means for notifying the user terminal of the evaluated score, means for requesting the user to perform an operation according to the rhythm while the vehicle is in automatic driving mode and measuring the reaction speed and accuracy, and means for evaluating the user's attention and confirming safety. This allows the driver to be periodically reminded of the driver's attention even during automatic driving, making it possible to maintain appropriate reaction speed and accuracy.

[0161] A "rhythm" is a time pattern that serves as a reference for a user to perform an operation according to a specific time interval.

[0162] A "notification signal" is a signal that is sent from a server to a user terminal and prompts the user to perform an operation according to a specific rhythm.

[0163] "Tap timing" is the time when the user operates the terminal in accordance with the specified rhythm.

[0164] A "game server" is a central system that generates rhythms, sends notification signals, receives tap timings, and evaluates scores.

[0165] A "user terminal" is a device that provides a user interface, displays notification signals from the server, and records user operations.

[0166] "Score evaluation" is a process of evaluating the accuracy of the user's operation based on the difference between the received tap timing and rhythm, and expressing the result as a score.

[0167] "Vehicle is in automatic driving mode" means that the vehicle is under the control of an automatic driving system.

[0168] The "response speed" is the time it takes for a user to complete an operation in response to a notification signal.

[0169] "Accuracy" is a measure of how closely the user's operation matches a specific rhythm.

[0170] "Attention" refers to the concentration required for a user to perform a specified operation accurately and quickly.

[0171] server

[0172] The server provides the core functionality of the invention. First, the server generates a rhythm, which is a pattern of time intervals that the user should follow. The rhythm is generated randomly. The generated rhythm is then sent to the user terminal as a notification signal according to the specific rhythm.

[0173] Next, the server receives the tap timing data sent from the user device. Based on the received data, it calculates the difference between the tap timing and the rhythm to evaluate the user's score. It then notifies the user device of the evaluated score. In addition, in an autonomous vehicle, the server requests the user to operate the vehicle according to the rhythm while the vehicle is being driven automatically, measures the reaction speed and accuracy, and evaluates the user's attention.

[0174] Terminal

[0175] The user terminal functions as a user interface. When it receives a rhythm notification signal from the server, the terminal displays the message "Tap now!" to the user. Each time the user taps, the terminal stores the time, and the stored tap timing is sent to the server in real time. The terminal displays the evaluation results from the server to the user, providing feedback on the driver's attention and reaction speed.

[0176] User

[0177] The user taps according to the rhythm notification signal from the device. In an autonomous vehicle, the application is required to periodically prompt the user to operate according to the rhythm and maintain their attention.

[0178] Hardware and Software

[0179] The system is implemented using devices such as smartphones and head-mounted displays. The software uses Python to implement server-side and device-side programs. The server-side uses an API to exchange data in real time using the HTTP protocol.

[0180] Specific examples

[0181] For example, if the rhythm randomly generated by the server is "1.1 seconds, 0.8 seconds, 1.2 seconds," the server will send a notification signal to the device accordingly. The user taps at the right time according to the instruction "Tap now!", and the timing data is sent from the device to the server. Based on the received data, the server evaluates the user's reaction speed and accuracy, calculates a score, and notifies the device.

[0182] Prompt Sentence Examples

[0183] Design an application that measures a driver's reaction time by tapping at specific intervals according to the instruction "Tap now!". The system uses a smartphone or head-mounted display, and sends the recorded timing to a server for evaluation. Please explain the specific implementation method.

[0184] This allows the driver to maintain appropriate reaction speed and attention even during automated driving, improving safety.

[0185] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0186] Step 1:

[0187] The server generates the rhythm. The server generates a random time interval pattern (rhythm) and stores that information for the next processing step. The inputs are the set parameters and the random generation algorithm, and the output is the generated rhythm pattern.

[0188] Step 2:

[0189] The server sends the generated rhythm to the user's device as a notification signal. Based on the rhythm data, the server sends a message saying "Tap now!", which the device displays. The input is the generated rhythm pattern, and the output is a notification signal and a display message to the user's device.

[0190] Step 3:

[0191] The user device receives the notification signal and prompts the user to operate in accordance with the rhythm. The device displays the message "Tap now!" in accordance with the rhythm, prompting the user to tap at a specific timing. The input is the notification signal from the server, and the output is the message displayed to the user.

[0192] Step 4:

[0193] The user taps according to the notification message. The user taps the device at the specified timing, and the time of the tap is recorded. The input is the user's tap operation, and the output is the recorded tap timing.

[0194] Step 5:

[0195] The user terminal transmits the tap timing to the server. The terminal transmits the recorded tap timing to the server in real time, and the tap data is accumulated on the server. The input is the recorded tap timing, and the output is the data transmitted to the server.

[0196] Step 6:

[0197] The server evaluates the score based on the received tap timing. The server calculates the difference between the tap timing and the generated rhythm and evaluates the accuracy of the user's operation. The input is the received tap timing and rhythm pattern, and the output is the calculated score.

[0198] Step 7:

[0199] The server notifies the user terminal of the evaluated score. The server then sends the evaluation result to the user terminal, which then displays it to the user. The input is the calculated score, and the output is a notification signal and a display message to the user terminal.

[0200] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0201] This invention combines a game system in which users compete for points by tapping in accordance with a specific rhythm with an emotion engine that recognizes the user's emotions. Below, we will explain each component of this system, its interactions, and the specific operation flow.

[0202] server

[0203] The server plays an important role in the game. First, the server randomly generates a rhythm when the game starts. This rhythm is a pattern of time intervals that the user must follow, and is set, for example, as "1.1 seconds, 0.8 seconds, 1.2 seconds." The generated rhythm is then sent to the user's device.

[0204] Next, the server sequentially sends rhythmic notification signals to the user's device. The notification signals include the timing at which the user should tap. The timing of the taps is recorded on the device and transmitted to the server in real time. Based on this, tap timing data is collected and saved.

[0205] The server then evaluates the user's score based on the recorded timing of the taps, assessing how closely the user's taps match the rhythm, and calculates the score based on the results. The calculated score is then sent to the user's device and notified to the user.

[0206] Emotion Engine

[0207] The emotion engine is used to recognize the user's emotional state and adjust the difficulty of the game. The emotion engine uses sensors such as cameras and microphones to analyze the user's facial expressions and tone of voice, and has the ability to extract emotions in real time.

[0208] The emotion engine sends the user's emotional state to the server and can adaptively adjust the difficulty of the rhythm based on that data. For example, if the user is stressed, the rhythm can be made easier, while if the user is relaxed, the rhythm can be made more difficult. This makes the game experience more personalized and enjoyable.

[0209] User terminal

[0210] The user device functions as an interface, receiving notification signals from the server and displaying them to the user. When the user taps, the device records the timing of the tap and sends it to the server in real time. It also analyzes the user's facial expressions and voice based on data from the emotion engine.

[0211] User

[0212] The user follows the instructions on the device and taps the device in time with the rhythm. The emotion engine analyzes the user's emotional state and adjusts the difficulty of the rhythm based on that, so the game experience is tailored to each individual user's emotional state. After the game is over, the user can check their score sent from the server on their device.

[0213] Specific examples

[0214] For example, suppose User A starts a game. The server generates a rhythm of "1.1 seconds, 0.8 seconds, 1.2 seconds," and the emotion engine analyzes User A's facial expressions and voice to determine that he or she is relaxed. Based on this data, the server maintains the rhythm as normal and sends a notification signal to User A.

[0215] During the game, User A taps according to the rhythm. The device records the tap timing and sends it to the server. After the game ends, the server analyzes the tap timing data, calculates User A's score, and notifies the device.

[0216] On the other hand, if User B is feeling stressed, the emotion engine detects this and adaptively adjusts the difficulty of the rhythm by widening the intervals between rhythms, allowing User B to enjoy the game. After the game ends, the server evaluates the tap timing based on the adjusted rhythm, calculates the appropriate score, and notifies the user.

[0217] In this way, a more personalized gaming experience can be provided by recognizing and adaptively adjusting the user's emotional state in real time.

[0218] The processing flow will be explained below.

[0219] Step 1:

[0220] The server initializes the game, sets the game duration, and randomly generates rhythm patterns.

[0221] Step 2:

[0222] The server transmits the generated rhythm pattern to the user terminal and issues a signal to start the game.

[0223] Step 3:

[0224] The device displays the message "Game Start" to the user, and the emotion engine begins to recognize and record the user's emotional state.

[0225] Step 4:

[0226] The server sends a notification signal to the user terminal saying "Tap now!" at specified time intervals based on the rhythm pattern. The notification includes the timing of the rhythm.

[0227] Step 5:

[0228] The device displays the received notification signal on the screen and prompts the user to tap. The emotion engine analyzes the user's facial expressions and voice and transmits the user's emotional state to the server in real time.

[0229] Step 6:

[0230] The user follows the notification from the device and taps the device at the specified timing. The timing of each tap is recorded.

[0231] Step 7:

[0232] The device records the tap timing and transmits the data in real time to the server, which then analyzes the data.

[0233] Step 8:

[0234] The server adjusts the difficulty of the rhythm based on the user's emotional state, for example, making the rhythm easier if the user is feeling stressed.

[0235] Step 9:

[0236] When the game duration ends, the server sends a signal to the terminal to end the game, causing the terminal to display a message saying "Game End." The server evaluates the user's score based on the tap timing data and emotion data.

[0237] Step 10:

[0238] The server compares the rhythmic patterns with the user's tap timing and emotional state, and calculates a score based on the accuracy of each tap. The closer the user's taps are to the rhythm, the higher the score, taking into account their emotional state.

[0239] Step 11:

[0240] The server sends the evaluated score to the user's terminal, which displays the score to the user and provides feedback on the game result.

[0241] Step 12:

[0242] The user checks their score on the terminal and ends the game.

[0243] Example 2

[0244] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0245] In conventional rhythm games, fixed difficulty settings result in a uniform playing experience for all users, making it difficult to provide an optimal game experience tailored to each individual user's emotional state. As a result, the level of stress or relaxation felt by the user during the game is not reflected, which can lead to a decrease in motivation to play and a decrease in satisfaction. Furthermore, because the difficulty setting for rhythm games is done manually, dynamic difficulty adjustment is not possible, making it difficult to appropriately adjust the game to suit the user's skill level and emotional state.

[0246] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0247] In this invention, the server includes means for generating a rhythm, means for transmitting a notification signal according to a specific rhythm to a user terminal, means for receiving tap timings from the user terminal, means for evaluating a user's score based on the received tap timings, means for notifying the user terminal of the evaluated score, means for recognizing the user's emotional state, and means for adaptively adjusting the difficulty of the rhythm based on the recognized emotional state. This makes it possible to dynamically adjust the difficulty of the game according to the user's emotional state, thereby providing an optimal gaming experience for each individual user.

[0248] A "rhythm" is a time pattern that allows a user to take action at specified time intervals.

[0249] A "user terminal" is a device that allows a user to make rhythmic inputs.

[0250] The "notification signal" is a signal that notifies the user of an action that follows the rhythm.

[0251] The "tap timing" refers to the moment in time when the user operates the user terminal.

[0252] "Evaluating the score" means determining how closely the timing of the user's taps matches the rhythm.

[0253] "Emotional state" refers to the user's psychological and sensory state.

[0254] "Recognizing an emotional state" means identifying a user's psychological state by analyzing the user's facial expressions, voice, etc.

[0255] "Adaptively adjusting the difficulty level" means dynamically changing the difficulty level of the game depending on the emotional state of the user.

[0256] The present invention combines a game system in which users compete for points by tapping in accordance with a specific rhythm with an emotion engine that recognizes the emotional state of the user. The system of the present invention comprises a server, a user terminal, an emotion engine, and a user. Specific embodiments for carrying out the present invention are described below.

[0257] server

[0258] The server plays a central role in this system. At the start of the game, the server randomly generates a rhythm pattern using Python's random module, for example. This rhythm pattern indicates the time intervals that the user must follow, and is set in the format of, for example, "1.1 seconds, 0.8 seconds, 1.2 seconds." After generating the rhythm, the server transmits the generated rhythm pattern to the user's device. The server then transmits notification signals sequentially according to each timing of the rhythm and receives the user's tap timing in real time. The server then evaluates the score based on the received tap timing and notifies the user's device of the evaluation result.

[0259] Emotion Engine

[0260] The emotion engine is responsible for analyzing the user's emotional state using sensors such as the user's camera and microphone. Specifically, the emotion engine collects and analyzes the user's facial expressions and tone of voice in real time. As a result of the analysis, it extracts the user's emotional state, such as whether they are relaxed or stressed. This emotional state is sent to the server, which adaptively adjusts the difficulty of the rhythm based on this data. For example, if the user is stressed, the interval between rhythms is increased, and if they are relaxed, the interval is decreased.

[0261] User terminal

[0262] The user device functions as an interface that allows the user to tap along to the rhythm. First, the user device displays the rhythm pattern received from the server to the user. It also analyzes the user's facial expressions and voice based on data obtained from the emotion engine and sends the analysis results to the server. When the user taps, the timing of the tap is recorded and sent to the server in real time. After the game ends, the user device displays the score sent from the server to the user.

[0263] User

[0264] Users tap according to the rhythm patterns displayed on their device. The emotion engine analyzes the user's emotional state and adjusts the difficulty of the rhythm based on the results, providing an optimal gaming experience for each individual user. After the game is over, users can check their score on their device.

[0265] Specific examples

[0266] For example, consider the case where user A starts a game. The server generates a rhythm pattern of "1.1 seconds, 0.8 seconds, 1.2 seconds," and the emotion engine analyzes that user A is relaxed. Based on this information, the server keeps the rhythm difficulty at normal and sends a notification signal to user A. During the game, user A taps according to the rhythm, and the tap timing is recorded on the device and sent to the server. After the game ends, the server analyzes the tap timing data, calculates a score, and notifies user A's device. On the other hand, if user B is feeling stressed, the emotion engine detects this and the server adaptively adjusts the difficulty of the rhythm. By widening the interval between the rhythms, user B can enjoy the game. After the game ends, the server calculates a score based on the adjusted rhythm and notifies the user's device.

[0267] Prompt Sentence Examples

[0268] Below are some example prompts to input to the generative AI model:

[0269] Describe a game system in which a user taps to a specific rhythm. The system has a built-in emotion engine that recognizes the user's emotions. The server randomly generates rhythms, and the emotion engine adjusts the difficulty of the rhythm according to the user's emotional state. If the user is relaxed, the rhythm becomes more difficult, and if the user is stressed, the rhythm becomes easier. Please explain the process in detail.

[0270] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0271] Step 1: Prepare to start the game

[0272] The server detects when the user taps the start button. Based on this input, the server generates a rhythm pattern using Python's random module. This rhythm pattern consists of time intervals such as "1.1 seconds, 0.8 seconds, 1.2 seconds." The generated rhythm pattern is sent to the user's device.

[0273] Specific operation: The server detects when the user starts the game → generates a rhythm (random) → sends it to the user's device

[0274] Step 2: Receiving and displaying rhythm patterns

[0275] The user terminal receives the rhythm pattern sent from the server. Based on this input, the rhythm pattern is presented to the user visually or audibly. Specific output may include displaying the rhythm interval on the screen or announcing the rhythm to the user via voice.

[0276] Specific operation: The user device receives the rhythm pattern → displays it on the screen or gives a voice notification

[0277] Step 3: Sending a rhythmic notification signal

[0278] The server sequentially transmits notification signals according to the rhythmic pattern. These notification signals include the timing at which the user should tap, and the user terminal conveys this information to the user in real time. The user taps based on these notification signals.

[0279] Specific operation: The server generates a timing signal according to the rhythm pattern and sends it to the user terminal.

[0280] Step 4: Recognizing your emotional state

[0281] The emotion engine analyzes data input from the user's camera and microphone to recognize the user's emotional state. Using facial recognition and voice analysis software, it determines whether the user is relaxed or stressed. This emotional data is then sent to a server.

[0282] Specific operation: The emotion engine collects data from the camera and microphone → analyzes the emotional state → sends it to the server

[0283] Step 5: Adjust your rhythm according to your emotional state

[0284] The server receives the emotion data sent from the emotion engine and adjusts the difficulty of the rhythm based on this data. For example, if the user is feeling stressed, the server may adjust the rhythm by widening the intervals between rhythms. The adjusted rhythm pattern is then retransmitted to the user's device.

[0285] Specific operation: The server receives emotion data → adjusts the rhythm pattern → sends it to the user's device

[0286] Step 6: Record your tap timing

[0287] When the user taps in time with the rhythm according to the instructions on the device, the user's device records the timing of the tapping. This recorded data is sent to the server in real time, and the server evaluates the accuracy of the tapping based on this data.

[0288] Specific operation: User taps in time with the rhythm → User device records tap timing → Sends to server

[0289] Step 7: Calculating and reporting scores

[0290] The server calculates the user's score based on the recorded tap timing. It evaluates the error between the acquired tap data and the rhythm pattern and calculates the score. This score is sent to the user's device and notified to the user.

[0291] Specific operation: The server evaluates the tap data → calculates the score → sends it to the user's device

[0292] Step 8: Ending the game and displaying the results

[0293] When the game ends, the user terminal displays the score sent from the server to the user, allowing the user to check their game results.

[0294] Specific operation: The server recognizes the end of the game → sends the score → the user device displays the score

[0295] (Application example 2)

[0296] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0297] Conventional rhythm game systems have been insufficient in personalizing the game experience according to the user's emotional state, and have provided a uniform level of difficulty even when the user is in a different emotional state, resulting in limited stress relief and relaxation effects. In particular, reducing customer stress and providing a comfortable environment while waiting in line at physical stores and other locations is required, but no adaptive rhythm game system has been available to achieve this.

[0298] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0299] In this invention, the server includes a means for generating a rhythm, a means for transmitting a notification signal according to a specific rhythm to a user terminal, a means for receiving tap timings from the user terminal, a means for evaluating a user's score based on the received tap timings, a means for notifying the user terminal of the evaluated score, and a means for adjusting the difficulty of the rhythm based on emotion analysis data. This makes it possible to personalize the rhythm game according to the user's emotional state, thereby improving the comfort of waiting times at physical stores.

[0300] A "rhythm" is a pattern in which a user should perform an action according to a specified time interval.

[0301] A "game server" is a computer system that generates rhythms, sends notification signals, receives tap timing, evaluates scores, and so on.

[0302] A "user terminal" is a device operated by a user, which communicates with the game server, records tap timing, and receives notification signals.

[0303] The "notification signal" is a signal that indicates the timing at which the user should tap in accordance with the rhythm.

[0304] The "tap timing" refers to the moment when the user operates the terminal.

[0305] The "score" is a score that is evaluated based on how well the user's tap timing matches the rhythm.

[0306] "Emotion analysis data" refers to data that indicates the emotional state of a user inferred from biometric information and behavioral information.

[0307] "Rhythm difficulty" refers to the difficulty of the game, determined by the complexity and speed of the rhythm.

[0308] This invention relates to a rhythm game system that allows customers to spend their waiting time in a fun and meaningful way in a brick-and-mortar store. This system is composed of a game server, a user terminal, and an emotion analysis engine, and the roles of each are explained below.

[0309] Game Server

[0310] The server plays the central role in this system. It is equipped with a rhythm-generating program that randomly generates a different rhythm for each user. The generated rhythm is sent to the user's device as a notification signal. The server also evaluates the user's score based on tap timing data received from the user's device and notifies the user of the result. Furthermore, by adjusting the difficulty of the rhythm based on emotion analysis data, the server provides a game experience that suits the user's emotional state.

[0311] User terminal

[0312] The user device primarily functions as an interface. It displays notification signals sent from the server to the user, and the user taps along to the rhythm. The tapping timing is recorded in real time and sent to the server. The device also has a built-in camera and microphone, and these sensors are used to analyze the user's emotional state. The analyzed emotional data is sent to the server and used to adjust the difficulty of the rhythm.

[0313] Sentiment Analysis Engine

[0314] The emotion analysis engine is a program that recognizes the user's emotional state in real time. It uses facial recognition and voice analysis technology to analyze the user's facial expressions and tone of voice to extract emotional states such as "relaxed" or "stressed." This emotional data is sent from the user's device to the server and used to adjust the game's difficulty level.

[0315] Hardware and software used

[0316] Hardware: Smart glasses, smartphones

[0317] Software: Flask (Python web framework), OpenCV (image analysis library), Emotion Recognition Model (emotion analysis engine)

[0318] Specific examples

[0319] For example, consider the case where a customer wearing smart glasses is waiting in line at a brick-and-mortar store. The camera captures the customer's face, and the image data is sent to the server. The server detects "stress" through its emotion analysis engine, and reduces the difficulty of the rhythm accordingly, generating a simple rhythm of "1.8 seconds, 2.0 seconds, 1.7 seconds." This rhythm is sent to the smart glasses as a notification signal, and the customer taps according to the rhythm.

[0320] Prompt Sentence Examples

[0321] Capture camera images and analyze the emotional state of customers. Based on the emotional state, generate rhythm patterns for a rhythm game that can be enjoyed while customers wait.

[0322] As described above, the present invention provides a rhythm game that is personalized according to the user's emotional state, making waiting times at physical stores more comfortable.

[0323] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0324] Step 1:

[0325] The customer wears the smart glasses and the camera image is captured by the terminal.

[0326] Input: Customer's face image

[0327] How it works: The camera in the smart glasses captures the customer's face and generates image data.

[0328] Step 2:

[0329] The device transmits the captured facial image to the server.

[0330] Input: A captured face image

[0331] Output: Facial image data sent to the server

[0332] Specific operation: The device transfers the captured facial image to the server in real time.

[0333] Step 3:

[0334] The server uses an emotion analysis engine to analyze the facial image and recognize the customer's emotional state.

[0335] Input: Facial image data

[0336] Output: Perceived emotional state (e.g., stressed, relaxed)

[0337] Specific operation: The server starts the emotion analysis engine and analyzes facial expressions from the facial image to extract the emotional state.

[0338] Step 4:

[0339] The server adjusts the difficulty of the rhythm based on the emotional state and generates a rhythm pattern.

[0340] Input: Perceived emotional state

[0341] Output: Rhythm pattern (e.g. 1.8 seconds, 2.0 seconds, 1.7 seconds)

[0342] Specific operation: When the emotional state is "stressed," the server executes a program that generates a simple rhythm pattern.

[0343] Step 5:

[0344] The server transmits the generated rhythm pattern to the terminal as a notification signal.

[0345] Input: Rhythm pattern

[0346] Output: Notification signal sent to the device

[0347] Specific operation: The server formats the generated rhythm pattern as a notification signal and sends it to the terminal.

[0348] Step 6:

[0349] The terminal receives the notification signal and displays tapping instructions to the customer according to the rhythm pattern.

[0350] Input: Notification signal

[0351] Output: Tap instructions shown to the customer

[0352] Specific operation: The device decodes the notification signal and displays instructions on the smart glasses display instructing when to tap.

[0353] Step 7:

[0354] The customer taps the smart glasses in rhythm.

[0355] Input: Tap instructions

[0356] Output: Tap timing

[0357] Specific operation: The customer taps the device according to the tap instructions displayed by the smart glasses.

[0358] Step 8:

[0359] The device records the tap timing and transmits it to the server in real time.

[0360] Input: Tap timing

[0361] Output: Tap timing data sent to the server

[0362] Specific operation: The terminal accurately measures the customer's tap timing and sends the data to the server.

[0363] Step 9:

[0364] The server compares the received tap timing with the rhythm pattern and evaluates the customer's score.

[0365] Input: Tap timing data, rhythm pattern

[0366] Output: Estimated score

[0367] Specific operation: The server evaluates the degree of agreement between the tap timing and the rhythm pattern, and calculates the score using a score calculation program.

[0368] Step 10:

[0369] The server notifies the terminal of the evaluated score, and the terminal displays the score to the customer.

[0370] Input: Estimated score

[0371] Output: Scoring information displayed to the customer

[0372] Specific operation: The server sends the score data to the terminal, which receives it and displays it to the customer.

[0373] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0374] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0375] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0376] [Second embodiment]

[0377] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0378] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0379] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0380] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0381] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0382] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0383] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0384] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0385] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0386] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0387] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0388] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0389] This invention is a game system in which users compete for points by tapping in accordance with a specific rhythm. Below, each component of the system, its interactions, and the specific flow of operation will be described in detail.

[0390] server

[0391] The server plays a central role in the game. First, the server randomly generates a rhythm at the start of the game. This is the pattern of time intervals that the user must follow. This rhythm information becomes the basis for the user to tap accurately.

[0392] The server then sends a rhythmic notification signal to the user's device, allowing the user to receive tapping instructions in real time. The server records the timing of the user's taps and analyzes the tap interval data.

[0393] The server then evaluates the user's score based on the recorded timing of the taps. The evaluation is done by calculating the difference between the user's tap timing and the pre-generated rhythm. The closer the user's taps are to the rhythm, the higher the score.

[0394] Terminal

[0395] The user terminal functions as a user interface. When the game starts, the terminal displays the message "Game Start" to the user. When the terminal receives a rhythm notification signal from the server, it sends the message "Tap now!" to the user.

[0396] The device has a function to record the timing of taps, and saves the time each time the user taps. The saved tap timings are sent to the server in real time. At the end of the game, the device receives the evaluation results from the server and displays them to the user.

[0397] User

[0398] The user taps the device in accordance with the instructions on the device. The user's goal is to tap as accurately as possible to the rhythm instructed by the server. As the game progresses, the user taps multiple times, and the timing of each tap is sent to the server.

[0399] Specific examples

[0400] For example, suppose the rhythm randomly generated by the server at the start of the game is a pattern of "1.1 seconds, 0.8 seconds, 1.2 seconds." According to this rhythm, the first tap must be made 1.1 seconds after the start of the game, the next tap 0.8 seconds after that, and then 1.2 seconds after that.

[0401] 1. Based on these time intervals, the server will sequentially send a "Tap now!" notification signal to the device.

[0402] 2. The user follows the instructions on the device and taps at the specified timing.

[0403] 3. The device records the time of each tap and transmits it to the server in real time.

[0404] 4. After the game ends, the server compares the user's tap timing with the rhythm and calculates the score.

[0405] 5. Finally, the device displays the score to the user and provides feedback on the game results.

[0406] In this way, the user, terminal, and server work together to realize a new type of game system that measures the user's sense of rhythm and accuracy.

[0407] The processing flow will be explained below.

[0408] Step 1:

[0409] The server initializes the game and sets the game duration and rhythm pattern, which is randomly generated, for example, "1.1 seconds, 0.8 seconds, 1.2 seconds."

[0410] Step 2:

[0411] The user starts up the terminal and prepares to start the game. The terminal connects to the server and waits for a signal to start the game.

[0412] Step 3:

[0413] The server sends a signal to the terminal to start the game and simultaneously records the start time of the game. The terminal displays the message "Game Start" to the user.

[0414] Step 4:

[0415] The server sends a notification signal saying "Tap now!" to the device at specified intervals according to the rhythm pattern. The device displays this notification on its screen.

[0416] Step 5:

[0417] The user sees the notification and taps the device at the specified timing. The timing of each tap is recorded.

[0418] Step 6:

[0419] The device sends the timing of the user's taps to the server in real time, and the server receives and records them.

[0420] Step 7:

[0421] When the game duration ends, the server sends a signal to the terminal to end the game, causing the terminal to display a message saying "Game End." At the same time, the server starts the process of evaluating the score based on the user's tap timing data.

[0422] Step 8:

[0423] The server compares the rhythm pattern with the user's tap timing and calculates a score based on the accuracy of each tap. The closer the user's taps are to the rhythm, the higher the score.

[0424] Step 9:

[0425] The server sends the evaluated score to the user's terminal, which displays the score to the user and provides feedback on the game result.

[0426] Step 10:

[0427] The user checks their score on the terminal and ends the game.

[0428] The above are the processing steps of the present invention. By tapping along a specific rhythm, users can compete with each other in terms of rhythmic sense and tapping accuracy.

[0429] Example 1

[0430] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0431] Rhythm games require a system that allows users to tap accurately in time with the rhythm and evaluate the results in real time. However, current systems face the challenge of urging users to tap accurately in accordance with randomly generated rhythms and accurately recording and evaluating the timing of their taps. To solve this challenge, improvements to the user interface and faster data communication are required.

[0432] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0433] In this invention, the server includes means for generating a rhythm, means for transmitting a notification signal according to a specific rhythm to a user terminal, means for receiving tap timings from the user terminal, means for evaluating a user's score based on the received tap timings, means for notifying the user terminal of the evaluated score, means for sequentially transmitting notification signals according to the rhythm, means for evaluating the score by comparing the received tap timings with a randomly generated rhythm, and means for notifying the user terminal of the score result. This allows the user to tap faithfully to the specified rhythm, and the results can be accurately evaluated and displayed in real time.

[0434] A "game server" is a central system that generates rhythms, receives tap timings, evaluates scores, and notifies users of the results of their scores in a game in which users tap in accordance with the rhythm.

[0435] The "rhythm" refers to a pattern of time intervals that serves as a basis for the user to perform tapping.

[0436] The "notification signal" is a signal sent from the server to the user terminal to notify the timing of tapping in accordance with the rhythm.

[0437] A "user terminal" is a device operated by a user playing a game, which receives notification signals from the server and records the timing of taps.

[0438] "Tap timing" refers to the exact time when a user taps the device, and this data is used for evaluation on the server.

[0439] The "score" is an evaluation value calculated based on how closely the user's tap timing matches the rhythm.

[0440] "Random" means that something is determined randomly without following any particular rule, and is used here in the context of rhythm generation.

[0441] The "evaluation means" is a function that compares the received tap timing with the rhythm and calculates the user's score.

[0442] The "notification means" is a function for transmitting the evaluated score to the user terminal and displaying it to the user.

[0443] This invention is a game system in which users compete for points by tapping in accordance with a specific rhythm. Below, each component of the system, its interactions, and the specific flow of operation will be described in detail.

[0444] server

[0445] The server plays a central role in the game. When the game starts, the server randomly generates a specific rhythm. This rhythm is the pattern of time intervals that the user must follow. The rhythm is generated using a random number generation algorithm and a library of time patterns (e.g., the random module). This rhythm information becomes the basis for the user to tap accurately.

[0446] Next, the server sequentially sends rhythmic notification signals to the user's device, allowing the user to receive tapping instructions in real time. Communication protocols such as HTTP requests and WebSockets are used to send the notification signals. Specifically, the server sends a "tap" notification 1.1 seconds later.

[0447] The server records the timing of the user's taps and evaluates the user's score based on that data. The evaluation is done by calculating the difference between the user's tap timing and a pre-generated rhythm. This evaluation uses absolute difference calculations and a statistical library (e.g., NumPy). The closer the user's taps are to the rhythm, the higher the score will be.

[0448] Terminal

[0449] The user device functions as a user interface. When the game starts, the device displays the message "Game Start" to the user. When the device receives a rhythm notification signal from the server, it sends the message "Tap now!" to the user. To display the notification, a UI library (e.g., React, SwiftUI) is used to draw the message on the screen.

[0450] The device has a function to record tap timing, and saves the time every time the user taps in milliseconds. The saved tap timing is sent to the server in real time. This communication also uses HTTP requests and WebSockets. At the end of the game, the device receives the evaluation result from the server and displays it to the user. A UI library is used again for this, and a message such as "Your score is 85 points" is displayed.

[0451] User

[0452] The user taps the device in accordance with the instructions of the device. The user's goal is to tap as accurately as possible to the rhythm instructed by the server. The timing of the taps is recorded on the device and transmitted to the server in real time.

[0453] Specific operation example

[0454] For example, suppose the rhythm randomly generated by the server at the start of the game is a pattern of "1.1 seconds, 0.8 seconds, 1.2 seconds." In this case, the user must tap the first time 1.1 seconds after the start of the game, then tap the next time 0.8 seconds later, and then tap the next time 1.2 seconds later.

[0455] 1. Based on these time intervals, the server will sequentially send a "Tap now!" notification signal to the device.

[0456] 2. The user follows the instructions on the device and taps at the specified timing.

[0457] 3. The device records the time of each tap and transmits it to the server in real time.

[0458] 4. After the game ends, the server compares the user's tap timing with the rhythm and calculates the score.

[0459] 5. Finally, the device displays the score to the user and provides feedback on the game results.

[0460] Examples of prompt statements

[0461] Here are some example prompts to input to a generative AI model:

[0462] "You are now participating in a musical note game. Please tap according to the following rhythm: 1.1 seconds, 0.8 seconds, 1.2 seconds. Please tap accurately at each timing."

[0463] In this way, by realizing this game system through cooperation between the server, terminals, and users, the users' sense of rhythm can be measured and enjoyed.

[0464] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0465] Step 1:

[0466] The server generates the rhythm.

[0467] Input: Rhythm generation algorithm (e.g., random module)

[0468] Processing: To generate rhythms, a random number generation algorithm is used to randomly determine time intervals (e.g., 1.1 seconds, 0.8 seconds, 1.2 seconds).

[0469] Output: Generated rhythm pattern (e.g. "1.1 seconds, 0.8 seconds, 1.2 seconds")

[0470] Step 2:

[0471] The server transmits a rhythm notification signal to the user terminal.

[0472] Input: Generated rhythm pattern

[0473] Processing: The server sequentially sends "Tap now!" notification signals after a specified time (e.g., 1.1 seconds, then 0.8 seconds, then 1.2 seconds) based on the generated rhythm pattern. This transmission uses a communication protocol such as an HTTP request or WebSocket.

[0474] Output: Notification signal to the user device (e.g. "Tap now!")

[0475] Step 3:

[0476] The device displays a notification signal.

[0477] Input: Notification signal from the server

[0478] Processing: The device displays the notification signal received from the server in the user interface. Specifically, a UI library (e.g., React, SwiftUI) is used to display the message "Tap now!" on the screen.

[0479] Output: The message that is displayed to the user (e.g., "Tap now!")

[0480] Step 4:

[0481] The user taps.

[0482] Input: The message displayed on the terminal

[0483] Action: The user taps the screen in response to the instruction from the device, "Tap now!"

[0484] Output: User's tap action

[0485] Step 5:

[0486] The device records the timing of your taps.

[0487] Input: User tap action

[0488] Processing: The device records the exact moment the user tapped, down to the millisecond, using the system clock or a timer library (e.g., Date.now(), System.currentTimeMillis()).

[0489] Output: Recorded tap timing

[0490] Step 6:

[0491] The device sends the tap timing to the server.

[0492] Input: Recorded tap timing

[0493] Processing: The device sends the recorded tap timings to the server in real time via HTTP requests or WebSockets in JSON format.

[0494] Output: Tap timing data to the server

[0495] Step 7:

[0496] The server evaluates the tap timing.

[0497] Input: Received tap timing, generated rhythm pattern

[0498] Processing: The server compares the user's tap timing with the generated rhythm and calculates the user's score. Absolute difference calculations and statistical libraries (e.g., NumPy) are used. For example, a high score is awarded if the tap timing is within ±50 milliseconds of the rhythm.

[0499] Output: Calculated score

[0500] Step 8:

[0501] The server transmits the evaluation results to the terminal.

[0502] Input: Calculated score

[0503] Processing: The server sends the score results in JSON format to the device, again using an HTTP request or WebSocket.

[0504] Output: Score result data to the terminal

[0505] Step 9:

[0506] The terminal displays the evaluation results.

[0507] Input: Score result data from the server

[0508] Processing: The device displays the received score result to the user. This is done by using the UI library again, and displays a message such as "Your score is 85 points."

[0509] Output: Scoring results displayed to the user

[0510] In this way, the rhythm tap game system is realized by the server, terminals, and users operating in cooperation with each other.

[0511] (Application example 1)

[0512] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0513] In autonomous vehicles, there is a need for a means to check the driver's attention and reaction time to improve safety. In conventional autonomous driving systems, the driver may monitor the driving situation for long periods of time without operating the vehicle, which can lead to a decline in attention. Therefore, there is a lack of effective methods to maintain the driver's attention and promote appropriate reactions.

[0514] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0515] In this invention, the server includes means for generating a rhythm, means for transmitting a notification signal according to a specific rhythm to a user terminal, means for receiving tap timing from the user terminal, means for evaluating a user's score based on the received tap timing, means for notifying the user terminal of the evaluated score, means for requesting the user to perform an operation according to the rhythm while the vehicle is in automatic driving mode and measuring the reaction speed and accuracy, and means for evaluating the user's attention and confirming safety. This allows the driver to be periodically reminded of the driver's attention even during automatic driving, making it possible to maintain appropriate reaction speed and accuracy.

[0516] A "rhythm" is a time pattern that serves as a reference for a user to perform an operation according to a specific time interval.

[0517] A "notification signal" is a signal that is sent from a server to a user terminal and prompts the user to perform an operation according to a specific rhythm.

[0518] "Tap timing" is the time when the user operates the terminal in accordance with the specified rhythm.

[0519] A "game server" is a central system that generates rhythms, sends notification signals, receives tap timings, and evaluates scores.

[0520] A "user terminal" is a device that provides a user interface, displays notification signals from the server, and records user operations.

[0521] "Score evaluation" is a process of evaluating the accuracy of the user's operation based on the difference between the received tap timing and rhythm, and expressing the result as a score.

[0522] "Vehicle is in automatic driving mode" means that the vehicle is under the control of an automatic driving system.

[0523] The "response speed" is the time it takes for a user to complete an operation in response to a notification signal.

[0524] "Accuracy" is a measure of how closely the user's operation matches a specific rhythm.

[0525] "Attention" refers to the concentration required for a user to perform a specified operation accurately and quickly.

[0526] server

[0527] The server provides the core functionality of the invention. First, the server generates a rhythm, which is a pattern of time intervals that the user should follow. The rhythm is generated randomly. The generated rhythm is then sent to the user terminal as a notification signal according to the specific rhythm.

[0528] Next, the server receives the tap timing data sent from the user device. Based on the received data, it calculates the difference between the tap timing and the rhythm to evaluate the user's score. It then notifies the user device of the evaluated score. In addition, in an autonomous vehicle, the server requests the user to operate the vehicle according to the rhythm while the vehicle is being driven automatically, measures the reaction speed and accuracy, and evaluates the user's attention.

[0529] Terminal

[0530] The user terminal functions as a user interface. When it receives a rhythm notification signal from the server, the terminal displays the message "Tap now!" to the user. Each time the user taps, the terminal stores the time, and the stored tap timing is sent to the server in real time. The terminal displays the evaluation results from the server to the user, providing feedback on the driver's attention and reaction speed.

[0531] User

[0532] The user taps according to the rhythm notification signal from the device. In an autonomous vehicle, the application is required to periodically prompt the user to operate according to the rhythm and maintain their attention.

[0533] Hardware and Software

[0534] The system is implemented using devices such as smartphones and head-mounted displays. The software uses Python to implement server-side and device-side programs. The server-side uses an API to exchange data in real time using the HTTP protocol.

[0535] Specific examples

[0536] For example, if the rhythm randomly generated by the server is "1.1 seconds, 0.8 seconds, 1.2 seconds," the server will send a notification signal to the device accordingly. The user taps at the right time according to the instruction "Tap now!", and the timing data is sent from the device to the server. Based on the received data, the server evaluates the user's reaction speed and accuracy, calculates a score, and notifies the device.

[0537] Prompt Sentence Examples

[0538] Design an application that measures a driver's reaction time by tapping at specific intervals according to the instruction "Tap now!". The system uses a smartphone or head-mounted display, and sends the recorded timing to a server for evaluation. Please explain the specific implementation method.

[0539] This allows the driver to maintain appropriate reaction speed and attention even during automated driving, improving safety.

[0540] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0541] Step 1:

[0542] The server generates the rhythm. The server generates a random time interval pattern (rhythm) and stores that information for the next processing step. The inputs are the set parameters and the random generation algorithm, and the output is the generated rhythm pattern.

[0543] Step 2:

[0544] The server sends the generated rhythm to the user's device as a notification signal. Based on the rhythm data, the server sends a message saying "Tap now!", which the device displays. The input is the generated rhythm pattern, and the output is a notification signal and a display message to the user's device.

[0545] Step 3:

[0546] The user device receives the notification signal and prompts the user to operate in accordance with the rhythm. The device displays the message "Tap now!" in accordance with the rhythm, prompting the user to tap at a specific timing. The input is the notification signal from the server, and the output is the message displayed to the user.

[0547] Step 4:

[0548] The user taps according to the notification message. The user taps the device at the specified timing, and the time of the tap is recorded. The input is the user's tap operation, and the output is the recorded tap timing.

[0549] Step 5:

[0550] The user terminal transmits the tap timing to the server. The terminal transmits the recorded tap timing to the server in real time, and the tap data is accumulated on the server. The input is the recorded tap timing, and the output is the data transmitted to the server.

[0551] Step 6:

[0552] The server evaluates the score based on the received tap timing. The server calculates the difference between the tap timing and the generated rhythm and evaluates the accuracy of the user's operation. The input is the received tap timing and rhythm pattern, and the output is the calculated score.

[0553] Step 7:

[0554] The server notifies the user terminal of the evaluated score. The server then sends the evaluation result to the user terminal, which then displays it to the user. The input is the calculated score, and the output is a notification signal and a display message to the user terminal.

[0555] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0556] This invention combines a game system in which users compete for points by tapping in accordance with a specific rhythm with an emotion engine that recognizes the user's emotions. Below, we will explain each component of this system, its interactions, and the specific operation flow.

[0557] server

[0558] The server plays an important role in the game. First, the server randomly generates a rhythm when the game starts. This rhythm is a pattern of time intervals that the user must follow, and is set, for example, as "1.1 seconds, 0.8 seconds, 1.2 seconds." The generated rhythm is then sent to the user's device.

[0559] Next, the server sequentially sends rhythmic notification signals to the user's device. The notification signals include the timing at which the user should tap. The timing of the taps is recorded on the device and transmitted to the server in real time. Based on this, tap timing data is collected and saved.

[0560] The server then evaluates the user's score based on the recorded timing of the taps, assessing how closely the user's taps match the rhythm, and calculates the score based on the results. The calculated score is then sent to the user's device and notified to the user.

[0561] Emotion Engine

[0562] The emotion engine is used to recognize the user's emotional state and adjust the difficulty of the game. The emotion engine uses sensors such as cameras and microphones to analyze the user's facial expressions and tone of voice, and has the ability to extract emotions in real time.

[0563] The emotion engine sends the user's emotional state to the server and can adaptively adjust the difficulty of the rhythm based on that data. For example, if the user is stressed, the rhythm can be made easier, while if the user is relaxed, the rhythm can be made more difficult. This makes the game experience more personalized and enjoyable.

[0564] User terminal

[0565] The user device functions as an interface, receiving notification signals from the server and displaying them to the user. When the user taps, the device records the timing of the tap and sends it to the server in real time. It also analyzes the user's facial expressions and voice based on data from the emotion engine.

[0566] User

[0567] The user follows the instructions on the device and taps the device in time with the rhythm. The emotion engine analyzes the user's emotional state and adjusts the difficulty of the rhythm based on that, so the game experience is tailored to each individual user's emotional state. After the game is over, the user can check their score sent from the server on their device.

[0568] Specific examples

[0569] For example, suppose User A starts a game. The server generates a rhythm of "1.1 seconds, 0.8 seconds, 1.2 seconds," and the emotion engine analyzes User A's facial expressions and voice to determine that he or she is relaxed. Based on this data, the server maintains the rhythm as normal and sends a notification signal to User A.

[0570] During the game, User A taps according to the rhythm. The device records the tap timing and sends it to the server. After the game ends, the server analyzes the tap timing data, calculates User A's score, and notifies the device.

[0571] On the other hand, if User B is feeling stressed, the emotion engine detects this and adaptively adjusts the difficulty of the rhythm by widening the intervals between rhythms, allowing User B to enjoy the game. After the game ends, the server evaluates the tap timing based on the adjusted rhythm, calculates the appropriate score, and notifies the user.

[0572] In this way, a more personalized gaming experience can be provided by recognizing and adaptively adjusting the user's emotional state in real time.

[0573] The processing flow will be explained below.

[0574] Step 1:

[0575] The server initializes the game, sets the game duration, and randomly generates rhythm patterns.

[0576] Step 2:

[0577] The server transmits the generated rhythm pattern to the user terminal and issues a signal to start the game.

[0578] Step 3:

[0579] The device displays the message "Game Start" to the user, and the emotion engine begins to recognize and record the user's emotional state.

[0580] Step 4:

[0581] The server sends a notification signal to the user terminal saying "Tap now!" at specified time intervals based on the rhythm pattern. The notification includes the timing of the rhythm.

[0582] Step 5:

[0583] The device displays the received notification signal on the screen and prompts the user to tap. The emotion engine analyzes the user's facial expressions and voice and transmits the user's emotional state to the server in real time.

[0584] Step 6:

[0585] The user follows the notification from the device and taps the device at the specified timing. The timing of each tap is recorded.

[0586] Step 7:

[0587] The device records the tap timing and transmits the data in real time to the server, which then analyzes the data.

[0588] Step 8:

[0589] The server adjusts the difficulty of the rhythm based on the user's emotional state, for example, making the rhythm easier if the user is feeling stressed.

[0590] Step 9:

[0591] When the game duration ends, the server sends a signal to the terminal to end the game, causing the terminal to display a message saying "Game End." The server evaluates the user's score based on the tap timing data and emotion data.

[0592] Step 10:

[0593] The server compares the rhythmic patterns with the user's tap timing and emotional state, and calculates a score based on the accuracy of each tap. The closer the user's taps are to the rhythm, the higher the score, taking into account their emotional state.

[0594] Step 11:

[0595] The server sends the evaluated score to the user's terminal, which displays the score to the user and provides feedback on the game result.

[0596] Step 12:

[0597] The user checks their score on the terminal and ends the game.

[0598] Example 2

[0599] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0600] In conventional rhythm games, fixed difficulty settings result in a uniform playing experience for all users, making it difficult to provide an optimal game experience tailored to each individual user's emotional state. As a result, the level of stress or relaxation felt by the user during the game is not reflected, which can lead to a decrease in motivation to play and a decrease in satisfaction. Furthermore, because the difficulty setting for rhythm games is done manually, dynamic difficulty adjustment is not possible, making it difficult to appropriately adjust the game to suit the user's skill level and emotional state.

[0601] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0602] In this invention, the server includes means for generating a rhythm, means for transmitting a notification signal according to a specific rhythm to a user terminal, means for receiving tap timings from the user terminal, means for evaluating a user's score based on the received tap timings, means for notifying the user terminal of the evaluated score, means for recognizing the user's emotional state, and means for adaptively adjusting the difficulty of the rhythm based on the recognized emotional state. This makes it possible to dynamically adjust the difficulty of the game according to the user's emotional state, thereby providing an optimal gaming experience for each individual user.

[0603] A "rhythm" is a time pattern that allows a user to take action at specified time intervals.

[0604] A "user terminal" is a device that allows a user to make rhythmic inputs.

[0605] The "notification signal" is a signal that notifies the user of an action that follows the rhythm.

[0606] The "tap timing" refers to the moment in time when the user operates the user terminal.

[0607] "Evaluating the score" means determining how closely the timing of the user's taps matches the rhythm.

[0608] "Emotional state" refers to the user's psychological and sensory state.

[0609] "Recognizing an emotional state" means identifying a user's psychological state by analyzing the user's facial expressions, voice, etc.

[0610] "Adaptively adjusting the difficulty level" means dynamically changing the difficulty level of the game depending on the emotional state of the user.

[0611] The present invention combines a game system in which users compete for points by tapping in accordance with a specific rhythm with an emotion engine that recognizes the emotional state of the user. The system of the present invention comprises a server, a user terminal, an emotion engine, and a user. Specific embodiments for carrying out the present invention are described below.

[0612] server

[0613] The server plays a central role in this system. At the start of the game, the server randomly generates a rhythm pattern using Python's random module, for example. This rhythm pattern indicates the time intervals that the user must follow, and is set in the format of, for example, "1.1 seconds, 0.8 seconds, 1.2 seconds." After generating the rhythm, the server transmits the generated rhythm pattern to the user's device. The server then transmits notification signals sequentially according to each timing of the rhythm and receives the user's tap timing in real time. The server then evaluates the score based on the received tap timing and notifies the user's device of the evaluation result.

[0614] Emotion Engine

[0615] The emotion engine is responsible for analyzing the user's emotional state using sensors such as the user's camera and microphone. Specifically, the emotion engine collects and analyzes the user's facial expressions and tone of voice in real time. As a result of the analysis, it extracts the user's emotional state, such as whether they are relaxed or stressed. This emotional state is sent to the server, which adaptively adjusts the difficulty of the rhythm based on this data. For example, if the user is stressed, the interval between rhythms is increased, and if they are relaxed, the interval is decreased.

[0616] User terminal

[0617] The user device functions as an interface that allows the user to tap along to the rhythm. First, the user device displays the rhythm pattern received from the server to the user. It also analyzes the user's facial expressions and voice based on data obtained from the emotion engine and sends the analysis results to the server. When the user taps, the timing of the tap is recorded and sent to the server in real time. After the game ends, the user device displays the score sent from the server to the user.

[0618] User

[0619] Users tap according to the rhythm patterns displayed on their device. The emotion engine analyzes the user's emotional state and adjusts the difficulty of the rhythm based on the results, providing an optimal gaming experience for each individual user. After the game is over, users can check their score on their device.

[0620] Specific examples

[0621] For example, consider the case where user A starts a game. The server generates a rhythm pattern of "1.1 seconds, 0.8 seconds, 1.2 seconds," and the emotion engine analyzes that user A is relaxed. Based on this information, the server keeps the rhythm difficulty at normal and sends a notification signal to user A. During the game, user A taps according to the rhythm, and the tap timing is recorded on the device and sent to the server. After the game ends, the server analyzes the tap timing data, calculates a score, and notifies user A's device. On the other hand, if user B is feeling stressed, the emotion engine detects this and the server adaptively adjusts the difficulty of the rhythm. By widening the interval between the rhythms, user B can enjoy the game. After the game ends, the server calculates a score based on the adjusted rhythm and notifies the user's device.

[0622] Prompt Sentence Examples

[0623] Below are some example prompts to input to the generative AI model:

[0624] Describe a game system in which a user taps to a specific rhythm. The system has a built-in emotion engine that recognizes the user's emotions. The server randomly generates rhythms, and the emotion engine adjusts the difficulty of the rhythm according to the user's emotional state. If the user is relaxed, the rhythm becomes more difficult, and if the user is stressed, the rhythm becomes easier. Please explain the process in detail.

[0625] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0626] Step 1: Prepare to start the game

[0627] The server detects when the user taps the start button. Based on this input, the server generates a rhythm pattern using Python's random module. This rhythm pattern consists of time intervals such as "1.1 seconds, 0.8 seconds, 1.2 seconds." The generated rhythm pattern is sent to the user's device.

[0628] Specific operation: The server detects when the user starts the game → generates a rhythm (random) → sends it to the user's device

[0629] Step 2: Receiving and displaying rhythm patterns

[0630] The user terminal receives the rhythm pattern sent from the server. Based on this input, the rhythm pattern is presented to the user visually or audibly. Specific output may include displaying the rhythm interval on the screen or announcing the rhythm to the user via voice.

[0631] Specific operation: The user device receives the rhythm pattern → displays it on the screen or gives a voice notification

[0632] Step 3: Sending a rhythmic notification signal

[0633] The server sequentially transmits notification signals according to the rhythmic pattern. These notification signals include the timing at which the user should tap, and the user terminal conveys this information to the user in real time. The user taps based on these notification signals.

[0634] Specific operation: The server generates a timing signal according to the rhythm pattern and sends it to the user terminal.

[0635] Step 4: Recognizing your emotional state

[0636] The emotion engine analyzes data input from the user's camera and microphone to recognize the user's emotional state. Using facial recognition and voice analysis software, it determines whether the user is relaxed or stressed. This emotional data is then sent to a server.

[0637] Specific operation: The emotion engine collects data from the camera and microphone → analyzes the emotional state → sends it to the server

[0638] Step 5: Adjust your rhythm according to your emotional state

[0639] The server receives the emotion data sent from the emotion engine and adjusts the difficulty of the rhythm based on this data. For example, if the user is feeling stressed, the server may adjust the rhythm by widening the intervals between rhythms. The adjusted rhythm pattern is then retransmitted to the user's device.

[0640] Specific operation: The server receives emotion data → adjusts the rhythm pattern → sends it to the user's device

[0641] Step 6: Record your tap timing

[0642] When the user taps in time with the rhythm according to the instructions on the device, the user's device records the timing of the tapping. This recorded data is sent to the server in real time, and the server evaluates the accuracy of the tapping based on this data.

[0643] Specific operation: User taps in time with the rhythm → User device records tap timing → Sends to server

[0644] Step 7: Calculating and reporting scores

[0645] The server calculates the user's score based on the recorded tap timing. It evaluates the error between the acquired tap data and the rhythm pattern and calculates the score. This score is sent to the user's device and notified to the user.

[0646] Specific operation: The server evaluates the tap data → calculates the score → sends it to the user's device

[0647] Step 8: Ending the game and displaying the results

[0648] When the game ends, the user terminal displays the score sent from the server to the user, allowing the user to check their game results.

[0649] Specific operation: The server recognizes the end of the game → sends the score → the user device displays the score

[0650] (Application example 2)

[0651] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0652] Conventional rhythm game systems have been insufficient in personalizing the game experience according to the user's emotional state, and have provided a uniform level of difficulty even when the user is in a different emotional state, resulting in limited stress relief and relaxation effects. In particular, reducing customer stress and providing a comfortable environment while waiting in line at physical stores and other locations is required, but no adaptive rhythm game system has been available to achieve this.

[0653] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0654] In this invention, the server includes a means for generating a rhythm, a means for transmitting a notification signal according to a specific rhythm to a user terminal, a means for receiving tap timings from the user terminal, a means for evaluating a user's score based on the received tap timings, a means for notifying the user terminal of the evaluated score, and a means for adjusting the difficulty of the rhythm based on emotion analysis data. This makes it possible to personalize the rhythm game according to the user's emotional state, thereby improving the comfort of waiting times at physical stores.

[0655] A "rhythm" is a pattern in which a user should perform an action according to a specified time interval.

[0656] A "game server" is a computer system that generates rhythms, sends notification signals, receives tap timing, evaluates scores, and so on.

[0657] A "user terminal" is a device operated by a user, which communicates with the game server, records tap timing, and receives notification signals.

[0658] The "notification signal" is a signal that indicates the timing at which the user should tap in accordance with the rhythm.

[0659] The "tap timing" refers to the moment when the user operates the terminal.

[0660] The "score" is a score that is evaluated based on how well the user's tap timing matches the rhythm.

[0661] "Emotion analysis data" refers to data that indicates the emotional state of a user inferred from biometric information and behavioral information.

[0662] "Rhythm difficulty" refers to the difficulty of the game, determined by the complexity and speed of the rhythm.

[0663] This invention relates to a rhythm game system that allows customers to spend their waiting time in a fun and meaningful way in a brick-and-mortar store. This system is composed of a game server, a user terminal, and an emotion analysis engine, and the roles of each are explained below.

[0664] Game Server

[0665] The server plays the central role in this system. It is equipped with a rhythm-generating program that randomly generates a different rhythm for each user. The generated rhythm is sent to the user's device as a notification signal. The server also evaluates the user's score based on tap timing data received from the user's device and notifies the user of the result. Furthermore, by adjusting the difficulty of the rhythm based on emotion analysis data, the server provides a game experience that suits the user's emotional state.

[0666] User terminal

[0667] The user device primarily functions as an interface. It displays notification signals sent from the server to the user, and the user taps along to the rhythm. The tapping timing is recorded in real time and sent to the server. The device also has a built-in camera and microphone, and these sensors are used to analyze the user's emotional state. The analyzed emotional data is sent to the server and used to adjust the difficulty of the rhythm.

[0668] Sentiment Analysis Engine

[0669] The emotion analysis engine is a program that recognizes the user's emotional state in real time. It uses facial recognition and voice analysis technology to analyze the user's facial expressions and tone of voice to extract emotional states such as "relaxed" or "stressed." This emotional data is sent from the user's device to the server and used to adjust the game's difficulty level.

[0670] Hardware and software used

[0671] Hardware: Smart glasses, smartphones

[0672] Software: Flask (Python web framework), OpenCV (image analysis library), Emotion Recognition Model (emotion analysis engine)

[0673] Specific examples

[0674] For example, consider the case where a customer wearing smart glasses is waiting in line at a brick-and-mortar store. The camera captures the customer's face, and the image data is sent to the server. The server detects "stress" through its emotion analysis engine, and reduces the difficulty of the rhythm accordingly, generating a simple rhythm of "1.8 seconds, 2.0 seconds, 1.7 seconds." This rhythm is sent to the smart glasses as a notification signal, and the customer taps according to the rhythm.

[0675] Prompt Sentence Examples

[0676] Capture camera images and analyze the emotional state of customers. Based on the emotional state, generate rhythm patterns for a rhythm game that can be enjoyed while customers wait.

[0677] As described above, the present invention provides a rhythm game that is personalized according to the user's emotional state, making waiting times at physical stores more comfortable.

[0678] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0679] Step 1:

[0680] The customer wears the smart glasses and the camera image is captured by the terminal.

[0681] Input: Customer's face image

[0682] How it works: The camera in the smart glasses captures the customer's face and generates image data.

[0683] Step 2:

[0684] The device transmits the captured facial image to the server.

[0685] Input: A captured face image

[0686] Output: Facial image data sent to the server

[0687] Specific operation: The device transfers the captured facial image to the server in real time.

[0688] Step 3:

[0689] The server uses an emotion analysis engine to analyze the facial image and recognize the customer's emotional state.

[0690] Input: Facial image data

[0691] Output: Perceived emotional state (e.g., stressed, relaxed)

[0692] Specific operation: The server starts the emotion analysis engine and analyzes facial expressions from the facial image to extract the emotional state.

[0693] Step 4:

[0694] The server adjusts the difficulty of the rhythm based on the emotional state and generates a rhythm pattern.

[0695] Input: Perceived emotional state

[0696] Output: Rhythm pattern (e.g. 1.8 seconds, 2.0 seconds, 1.7 seconds)

[0697] Specific operation: When the emotional state is "stressed," the server executes a program that generates a simple rhythm pattern.

[0698] Step 5:

[0699] The server transmits the generated rhythm pattern to the terminal as a notification signal.

[0700] Input: Rhythm pattern

[0701] Output: Notification signal sent to the device

[0702] Specific operation: The server formats the generated rhythm pattern as a notification signal and sends it to the terminal.

[0703] Step 6:

[0704] The terminal receives the notification signal and displays tapping instructions to the customer according to the rhythm pattern.

[0705] Input: Notification signal

[0706] Output: Tap instructions shown to the customer

[0707] Specific operation: The device decodes the notification signal and displays instructions on the smart glasses display instructing when to tap.

[0708] Step 7:

[0709] The customer taps the smart glasses in rhythm.

[0710] Input: Tap instructions

[0711] Output: Tap timing

[0712] Specific operation: The customer taps the device according to the tap instructions displayed by the smart glasses.

[0713] Step 8:

[0714] The device records the tap timing and transmits it to the server in real time.

[0715] Input: Tap timing

[0716] Output: Tap timing data sent to the server

[0717] Specific operation: The terminal accurately measures the customer's tap timing and sends the data to the server.

[0718] Step 9:

[0719] The server compares the received tap timing with the rhythm pattern and evaluates the customer's score.

[0720] Input: Tap timing data, rhythm pattern

[0721] Output: Estimated score

[0722] Specific operation: The server evaluates the degree of agreement between the tap timing and the rhythm pattern, and calculates the score using a score calculation program.

[0723] Step 10:

[0724] The server notifies the terminal of the evaluated score, and the terminal displays the score to the customer.

[0725] Input: Estimated score

[0726] Output: Scoring information displayed to the customer

[0727] Specific operation: The server sends the score data to the terminal, which receives it and displays it to the customer.

[0728] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0729] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0730] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0731] [Third embodiment]

[0732] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0733] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0734] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0735] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0736] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0737] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0738] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0739] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0740] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0741] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0742] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0743] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0744] This invention is a game system in which users compete for points by tapping in accordance with a specific rhythm. Below, each component of the system, its interactions, and the specific flow of operation will be described in detail.

[0745] server

[0746] The server plays a central role in the game. First, the server randomly generates a rhythm at the start of the game. This is the pattern of time intervals that the user must follow. This rhythm information becomes the basis for the user to tap accurately.

[0747] The server then sends a rhythmic notification signal to the user's device, allowing the user to receive tapping instructions in real time. The server records the timing of the user's taps and analyzes the tap interval data.

[0748] The server then evaluates the user's score based on the recorded timing of the taps. The evaluation is done by calculating the difference between the user's tap timing and the pre-generated rhythm. The closer the user's taps are to the rhythm, the higher the score.

[0749] Terminal

[0750] The user terminal functions as a user interface. When the game starts, the terminal displays the message "Game Start" to the user. When the terminal receives a rhythm notification signal from the server, it sends the message "Tap now!" to the user.

[0751] The device has a function to record the timing of taps, and saves the time each time the user taps. The saved tap timings are sent to the server in real time. At the end of the game, the device receives the evaluation results from the server and displays them to the user.

[0752] User

[0753] The user taps the device in accordance with the instructions on the device. The user's goal is to tap as accurately as possible to the rhythm instructed by the server. As the game progresses, the user taps multiple times, and the timing of each tap is sent to the server.

[0754] Specific examples

[0755] For example, suppose the rhythm randomly generated by the server at the start of the game is a pattern of "1.1 seconds, 0.8 seconds, 1.2 seconds." According to this rhythm, the first tap must be made 1.1 seconds after the start of the game, the next tap 0.8 seconds after that, and then 1.2 seconds after that.

[0756] 1. Based on these time intervals, the server will sequentially send a "Tap now!" notification signal to the device.

[0757] 2. The user follows the instructions on the device and taps at the specified timing.

[0758] 3. The device records the time of each tap and transmits it to the server in real time.

[0759] 4. After the game ends, the server compares the user's tap timing with the rhythm and calculates the score.

[0760] 5. Finally, the device displays the score to the user and provides feedback on the game results.

[0761] In this way, the user, terminal, and server work together to realize a new type of game system that measures the user's sense of rhythm and accuracy.

[0762] The processing flow will be explained below.

[0763] Step 1:

[0764] The server initializes the game and sets the game duration and rhythm pattern, which is randomly generated, for example, "1.1 seconds, 0.8 seconds, 1.2 seconds."

[0765] Step 2:

[0766] The user starts up the terminal and prepares to start the game. The terminal connects to the server and waits for a signal to start the game.

[0767] Step 3:

[0768] The server sends a signal to the terminal to start the game and simultaneously records the start time of the game. The terminal displays the message "Game Start" to the user.

[0769] Step 4:

[0770] The server sends a notification signal saying "Tap now!" to the device at specified intervals according to the rhythm pattern. The device displays this notification on its screen.

[0771] Step 5:

[0772] The user sees the notification and taps the device at the specified timing. The timing of each tap is recorded.

[0773] Step 6:

[0774] The device sends the timing of the user's taps to the server in real time, and the server receives and records them.

[0775] Step 7:

[0776] When the game duration ends, the server sends a signal to the terminal to end the game, causing the terminal to display a message saying "Game End." At the same time, the server starts the process of evaluating the score based on the user's tap timing data.

[0777] Step 8:

[0778] The server compares the rhythm pattern with the user's tap timing and calculates a score based on the accuracy of each tap. The closer the user's taps are to the rhythm, the higher the score.

[0779] Step 9:

[0780] The server sends the evaluated score to the user's terminal, which displays the score to the user and provides feedback on the game result.

[0781] Step 10:

[0782] The user checks their score on the terminal and ends the game.

[0783] The above are the processing steps of the present invention. By tapping along a specific rhythm, users can compete with each other in terms of rhythmic sense and tapping accuracy.

[0784] Example 1

[0785] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0786] Rhythm games require a system that allows users to tap accurately in time with the rhythm and evaluate the results in real time. However, current systems face the challenge of urging users to tap accurately in accordance with randomly generated rhythms and accurately recording and evaluating the timing of their taps. To solve this challenge, improvements to the user interface and faster data communication are required.

[0787] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0788] In this invention, the server includes means for generating a rhythm, means for transmitting a notification signal according to a specific rhythm to a user terminal, means for receiving tap timings from the user terminal, means for evaluating a user's score based on the received tap timings, means for notifying the user terminal of the evaluated score, means for sequentially transmitting notification signals according to the rhythm, means for evaluating the score by comparing the received tap timings with a randomly generated rhythm, and means for notifying the user terminal of the score result. This allows the user to tap faithfully to the specified rhythm, and the results can be accurately evaluated and displayed in real time.

[0789] A "game server" is a central system that generates rhythms, receives tap timings, evaluates scores, and notifies users of the results of their scores in a game in which users tap in accordance with the rhythm.

[0790] The "rhythm" refers to a pattern of time intervals that serves as a basis for the user to perform tapping.

[0791] The "notification signal" is a signal sent from the server to the user terminal to notify the timing of tapping in accordance with the rhythm.

[0792] A "user terminal" is a device operated by a user playing a game, which receives notification signals from the server and records the timing of taps.

[0793] "Tap timing" refers to the exact time when a user taps the device, and this data is used for evaluation on the server.

[0794] The "score" is an evaluation value calculated based on how closely the user's tap timing matches the rhythm.

[0795] "Random" means that something is determined randomly without following any particular rule, and is used here in the context of rhythm generation.

[0796] The "evaluation means" is a function that compares the received tap timing with the rhythm and calculates the user's score.

[0797] The "notification means" is a function for transmitting the evaluated score to the user terminal and displaying it to the user.

[0798] This invention is a game system in which users compete for points by tapping in accordance with a specific rhythm. Below, each component of the system, its interactions, and the specific flow of operation will be described in detail.

[0799] server

[0800] The server plays a central role in the game. When the game starts, the server randomly generates a specific rhythm. This rhythm is the pattern of time intervals that the user must follow. The rhythm is generated using a random number generation algorithm and a library of time patterns (e.g., the random module). This rhythm information becomes the basis for the user to tap accurately.

[0801] Next, the server sequentially sends rhythmic notification signals to the user's device, allowing the user to receive tapping instructions in real time. Communication protocols such as HTTP requests and WebSockets are used to send the notification signals. Specifically, the server sends a "tap" notification 1.1 seconds later.

[0802] The server records the timing of the user's taps and evaluates the user's score based on that data. The evaluation is done by calculating the difference between the user's tap timing and a pre-generated rhythm. This evaluation uses absolute difference calculations and a statistical library (e.g., NumPy). The closer the user's taps are to the rhythm, the higher the score will be.

[0803] Terminal

[0804] The user device functions as a user interface. When the game starts, the device displays the message "Game Start" to the user. When the device receives a rhythm notification signal from the server, it sends the message "Tap now!" to the user. To display the notification, a UI library (e.g., React, SwiftUI) is used to draw the message on the screen.

[0805] The device has a function to record tap timing, and saves the time every time the user taps in milliseconds. The saved tap timing is sent to the server in real time. This communication also uses HTTP requests and WebSockets. At the end of the game, the device receives the evaluation result from the server and displays it to the user. A UI library is used again for this, and a message such as "Your score is 85 points" is displayed.

[0806] User

[0807] The user taps the device in accordance with the instructions of the device. The user's goal is to tap as accurately as possible to the rhythm instructed by the server. The timing of the taps is recorded on the device and transmitted to the server in real time.

[0808] Specific operation example

[0809] For example, suppose the rhythm randomly generated by the server at the start of the game is a pattern of "1.1 seconds, 0.8 seconds, 1.2 seconds." In this case, the user must tap the first time 1.1 seconds after the start of the game, then tap the next time 0.8 seconds later, and then tap the next time 1.2 seconds later.

[0810] 1. Based on these time intervals, the server will sequentially send a "Tap now!" notification signal to the device.

[0811] 2. The user follows the instructions on the device and taps at the specified timing.

[0812] 3. The device records the time of each tap and transmits it to the server in real time.

[0813] 4. After the game ends, the server compares the user's tap timing with the rhythm and calculates the score.

[0814] 5. Finally, the device displays the score to the user and provides feedback on the game results.

[0815] Examples of prompt statements

[0816] Here are some example prompts to input to a generative AI model:

[0817] "You are now participating in a musical note game. Please tap according to the following rhythm: 1.1 seconds, 0.8 seconds, 1.2 seconds. Please tap accurately at each timing."

[0818] In this way, by realizing this game system through cooperation between the server, terminals, and users, the users' sense of rhythm can be measured and enjoyed.

[0819] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0820] Step 1:

[0821] The server generates the rhythm.

[0822] Input: Rhythm generation algorithm (e.g., random module)

[0823] Processing: To generate rhythms, a random number generation algorithm is used to randomly determine time intervals (e.g., 1.1 seconds, 0.8 seconds, 1.2 seconds).

[0824] Output: Generated rhythm pattern (e.g. "1.1 seconds, 0.8 seconds, 1.2 seconds")

[0825] Step 2:

[0826] The server transmits a rhythm notification signal to the user terminal.

[0827] Input: Generated rhythm pattern

[0828] Processing: The server sequentially sends "Tap now!" notification signals after a specified time (e.g., 1.1 seconds, then 0.8 seconds, then 1.2 seconds) based on the generated rhythm pattern. This transmission uses a communication protocol such as an HTTP request or WebSocket.

[0829] Output: Notification signal to the user device (e.g. "Tap now!")

[0830] Step 3:

[0831] The device displays a notification signal.

[0832] Input: Notification signal from the server

[0833] Processing: The device displays the notification signal received from the server in the user interface. Specifically, a UI library (e.g., React, SwiftUI) is used to display the message "Tap now!" on the screen.

[0834] Output: The message that is displayed to the user (e.g., "Tap now!")

[0835] Step 4:

[0836] The user taps.

[0837] Input: The message displayed on the terminal

[0838] Action: The user taps the screen in response to the instruction from the device, "Tap now!"

[0839] Output: User's tap action

[0840] Step 5:

[0841] The device records the timing of your taps.

[0842] Input: User tap action

[0843] Processing: The device records the exact moment the user tapped, down to the millisecond, using the system clock or a timer library (e.g., Date.now(), System.currentTimeMillis()).

[0844] Output: Recorded tap timing

[0845] Step 6:

[0846] The device sends the tap timing to the server.

[0847] Input: Recorded tap timing

[0848] Processing: The device sends the recorded tap timings to the server in real time via HTTP requests or WebSockets in JSON format.

[0849] Output: Tap timing data to the server

[0850] Step 7:

[0851] The server evaluates the tap timing.

[0852] Input: Received tap timing, generated rhythm pattern

[0853] Processing: The server compares the user's tap timing with the generated rhythm and calculates the user's score. Absolute difference calculations and statistical libraries (e.g., NumPy) are used. For example, a high score is awarded if the tap timing is within ±50 milliseconds of the rhythm.

[0854] Output: Calculated score

[0855] Step 8:

[0856] The server transmits the evaluation results to the terminal.

[0857] Input: Calculated score

[0858] Processing: The server sends the score results in JSON format to the device, again using an HTTP request or WebSocket.

[0859] Output: Score result data to the terminal

[0860] Step 9:

[0861] The terminal displays the evaluation results.

[0862] Input: Score result data from the server

[0863] Processing: The device displays the received score result to the user. This is done by using the UI library again, and displays a message such as "Your score is 85 points."

[0864] Output: Scoring results displayed to the user

[0865] In this way, the rhythm tap game system is realized by the server, terminals, and users operating in cooperation with each other.

[0866] (Application example 1)

[0867] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0868] In autonomous vehicles, there is a need for a means to check the driver's attention and reaction time to improve safety. In conventional autonomous driving systems, the driver may monitor the driving situation for long periods of time without operating the vehicle, which can lead to a decline in attention. Therefore, there is a lack of effective methods to maintain the driver's attention and promote appropriate reactions.

[0869] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0870] In this invention, the server includes means for generating a rhythm, means for transmitting a notification signal according to a specific rhythm to a user terminal, means for receiving tap timing from the user terminal, means for evaluating a user's score based on the received tap timing, means for notifying the user terminal of the evaluated score, means for requesting the user to perform an operation according to the rhythm while the vehicle is in automatic driving mode and measuring the reaction speed and accuracy, and means for evaluating the user's attention and confirming safety. This allows the driver to be periodically reminded of the driver's attention even during automatic driving, making it possible to maintain appropriate reaction speed and accuracy.

[0871] A "rhythm" is a time pattern that serves as a reference for a user to perform an operation according to a specific time interval.

[0872] A "notification signal" is a signal that is sent from a server to a user terminal and prompts the user to perform an operation according to a specific rhythm.

[0873] "Tap timing" is the time when the user operates the terminal in accordance with the specified rhythm.

[0874] A "game server" is a central system that generates rhythms, sends notification signals, receives tap timings, and evaluates scores.

[0875] A "user terminal" is a device that provides a user interface, displays notification signals from the server, and records user operations.

[0876] "Score evaluation" is a process of evaluating the accuracy of the user's operation based on the difference between the received tap timing and rhythm, and expressing the result as a score.

[0877] "Vehicle is in automatic driving mode" means that the vehicle is under the control of an automatic driving system.

[0878] The "response speed" is the time it takes for a user to complete an operation in response to a notification signal.

[0879] "Accuracy" is a measure of how closely the user's operation matches a specific rhythm.

[0880] "Attention" refers to the concentration required for a user to perform a specified operation accurately and quickly.

[0881] server

[0882] The server provides the core functionality of the invention. First, the server generates a rhythm, which is a pattern of time intervals that the user should follow. The rhythm is generated randomly. The generated rhythm is then sent to the user terminal as a notification signal according to the specific rhythm.

[0883] Next, the server receives the tap timing data sent from the user device. Based on the received data, it calculates the difference between the tap timing and the rhythm to evaluate the user's score. It then notifies the user device of the evaluated score. In addition, in an autonomous vehicle, the server requests the user to operate the vehicle according to the rhythm while the vehicle is being driven automatically, measures the reaction speed and accuracy, and evaluates the user's attention.

[0884] Terminal

[0885] The user terminal functions as a user interface. When it receives a rhythm notification signal from the server, the terminal displays the message "Tap now!" to the user. Each time the user taps, the terminal stores the time, and the stored tap timing is sent to the server in real time. The terminal displays the evaluation results from the server to the user, providing feedback on the driver's attention and reaction speed.

[0886] User

[0887] The user taps according to the rhythm notification signal from the device. In an autonomous vehicle, the application is required to periodically prompt the user to operate according to the rhythm and maintain their attention.

[0888] Hardware and Software

[0889] The system is implemented using devices such as smartphones and head-mounted displays. The software uses Python to implement server-side and device-side programs. The server-side uses an API to exchange data in real time using the HTTP protocol.

[0890] Specific examples

[0891] For example, if the rhythm randomly generated by the server is "1.1 seconds, 0.8 seconds, 1.2 seconds," the server will send a notification signal to the device accordingly. The user taps at the right time according to the instruction "Tap now!", and the timing data is sent from the device to the server. Based on the received data, the server evaluates the user's reaction speed and accuracy, calculates a score, and notifies the device.

[0892] Prompt Sentence Examples

[0893] Design an application that measures a driver's reaction time by tapping at specific intervals according to the instruction "Tap now!". The system uses a smartphone or head-mounted display, and sends the recorded timing to a server for evaluation. Please explain the specific implementation method.

[0894] This allows the driver to maintain appropriate reaction speed and attention even during automated driving, improving safety.

[0895] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0896] Step 1:

[0897] The server generates the rhythm. The server generates a random time interval pattern (rhythm) and stores that information for the next processing step. The inputs are the set parameters and the random generation algorithm, and the output is the generated rhythm pattern.

[0898] Step 2:

[0899] The server sends the generated rhythm to the user's device as a notification signal. Based on the rhythm data, the server sends a message saying "Tap now!", which the device displays. The input is the generated rhythm pattern, and the output is a notification signal and a display message to the user's device.

[0900] Step 3:

[0901] The user device receives the notification signal and prompts the user to operate in accordance with the rhythm. The device displays the message "Tap now!" in accordance with the rhythm, prompting the user to tap at a specific timing. The input is the notification signal from the server, and the output is the message displayed to the user.

[0902] Step 4:

[0903] The user taps according to the notification message. The user taps the device at the specified timing, and the time of the tap is recorded. The input is the user's tap operation, and the output is the recorded tap timing.

[0904] Step 5:

[0905] The user terminal transmits the tap timing to the server. The terminal transmits the recorded tap timing to the server in real time, and the tap data is accumulated on the server. The input is the recorded tap timing, and the output is the data transmitted to the server.

[0906] Step 6:

[0907] The server evaluates the score based on the received tap timing. The server calculates the difference between the tap timing and the generated rhythm and evaluates the accuracy of the user's operation. The input is the received tap timing and rhythm pattern, and the output is the calculated score.

[0908] Step 7:

[0909] The server notifies the user terminal of the evaluated score. The server then sends the evaluation result to the user terminal, which then displays it to the user. The input is the calculated score, and the output is a notification signal and a display message to the user terminal.

[0910] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0911] This invention combines a game system in which users compete for points by tapping in accordance with a specific rhythm with an emotion engine that recognizes the user's emotions. Below, we will explain each component of this system, its interactions, and the specific operation flow.

[0912] server

[0913] The server plays an important role in the game. First, the server randomly generates a rhythm when the game starts. This rhythm is a pattern of time intervals that the user must follow, and is set, for example, as "1.1 seconds, 0.8 seconds, 1.2 seconds." The generated rhythm is then sent to the user's device.

[0914] Next, the server sequentially sends rhythmic notification signals to the user's device. The notification signals include the timing at which the user should tap. The timing of the taps is recorded on the device and transmitted to the server in real time. Based on this, tap timing data is collected and saved.

[0915] The server then evaluates the user's score based on the recorded timing of the taps, assessing how closely the user's taps match the rhythm, and calculates the score based on the results. The calculated score is then sent to the user's device and notified to the user.

[0916] Emotion Engine

[0917] The emotion engine is used to recognize the user's emotional state and adjust the difficulty of the game. The emotion engine uses sensors such as cameras and microphones to analyze the user's facial expressions and tone of voice, and has the ability to extract emotions in real time.

[0918] The emotion engine sends the user's emotional state to the server and can adaptively adjust the difficulty of the rhythm based on that data. For example, if the user is stressed, the rhythm can be made easier, while if the user is relaxed, the rhythm can be made more difficult. This makes the game experience more personalized and enjoyable.

[0919] User terminal

[0920] The user device functions as an interface, receiving notification signals from the server and displaying them to the user. When the user taps, the device records the timing of the tap and sends it to the server in real time. It also analyzes the user's facial expressions and voice based on data from the emotion engine.

[0921] User

[0922] The user follows the instructions on the device and taps the device in time with the rhythm. The emotion engine analyzes the user's emotional state and adjusts the difficulty of the rhythm based on that, so the game experience is tailored to each individual user's emotional state. After the game is over, the user can check their score sent from the server on their device.

[0923] Specific examples

[0924] For example, suppose User A starts a game. The server generates a rhythm of "1.1 seconds, 0.8 seconds, 1.2 seconds," and the emotion engine analyzes User A's facial expressions and voice to determine that he or she is relaxed. Based on this data, the server maintains the rhythm as normal and sends a notification signal to User A.

[0925] During the game, User A taps according to the rhythm. The device records the tap timing and sends it to the server. After the game ends, the server analyzes the tap timing data, calculates User A's score, and notifies the device.

[0926] On the other hand, if User B is feeling stressed, the emotion engine detects this and adaptively adjusts the difficulty of the rhythm by widening the intervals between rhythms, allowing User B to enjoy the game. After the game ends, the server evaluates the tap timing based on the adjusted rhythm, calculates the appropriate score, and notifies the user.

[0927] In this way, a more personalized gaming experience can be provided by recognizing and adaptively adjusting the user's emotional state in real time.

[0928] The processing flow will be explained below.

[0929] Step 1:

[0930] The server initializes the game, sets the game duration, and randomly generates rhythm patterns.

[0931] Step 2:

[0932] The server transmits the generated rhythm pattern to the user terminal and issues a signal to start the game.

[0933] Step 3:

[0934] The device displays the message "Game Start" to the user, and the emotion engine begins to recognize and record the user's emotional state.

[0935] Step 4:

[0936] The server sends a notification signal to the user terminal saying "Tap now!" at specified time intervals based on the rhythm pattern. The notification includes the timing of the rhythm.

[0937] Step 5:

[0938] The device displays the received notification signal on the screen and prompts the user to tap. The emotion engine analyzes the user's facial expressions and voice and transmits the user's emotional state to the server in real time.

[0939] Step 6:

[0940] The user follows the notification from the device and taps the device at the specified timing. The timing of each tap is recorded.

[0941] Step 7:

[0942] The device records the tap timing and transmits the data in real time to the server, which then analyzes the data.

[0943] Step 8:

[0944] The server adjusts the difficulty of the rhythm based on the user's emotional state, for example, making the rhythm easier if the user is feeling stressed.

[0945] Step 9:

[0946] When the game duration ends, the server sends a signal to the terminal to end the game, causing the terminal to display a message saying "Game End." The server evaluates the user's score based on the tap timing data and emotion data.

[0947] Step 10:

[0948] The server compares the rhythmic patterns with the user's tap timing and emotional state, and calculates a score based on the accuracy of each tap. The closer the user's taps are to the rhythm, the higher the score, taking into account their emotional state.

[0949] Step 11:

[0950] The server sends the evaluated score to the user's terminal, which displays the score to the user and provides feedback on the game result.

[0951] Step 12:

[0952] The user checks their score on the terminal and ends the game.

[0953] Example 2

[0954] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0955] In conventional rhythm games, fixed difficulty settings result in a uniform playing experience for all users, making it difficult to provide an optimal game experience tailored to each individual user's emotional state. As a result, the level of stress or relaxation felt by the user during the game is not reflected, which can lead to a decrease in motivation to play and a decrease in satisfaction. Furthermore, because the difficulty setting for rhythm games is done manually, dynamic difficulty adjustment is not possible, making it difficult to appropriately adjust the game to suit the user's skill level and emotional state.

[0956] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0957] In this invention, the server includes means for generating a rhythm, means for transmitting a notification signal according to a specific rhythm to a user terminal, means for receiving tap timings from the user terminal, means for evaluating a user's score based on the received tap timings, means for notifying the user terminal of the evaluated score, means for recognizing the user's emotional state, and means for adaptively adjusting the difficulty of the rhythm based on the recognized emotional state. This makes it possible to dynamically adjust the difficulty of the game according to the user's emotional state, thereby providing an optimal gaming experience for each individual user.

[0958] A "rhythm" is a time pattern that allows a user to take action at specified time intervals.

[0959] A "user terminal" is a device that allows a user to make rhythmic inputs.

[0960] The "notification signal" is a signal that notifies the user of an action that follows the rhythm.

[0961] The "tap timing" refers to the moment in time when the user operates the user terminal.

[0962] "Evaluating the score" means determining how closely the timing of the user's taps matches the rhythm.

[0963] "Emotional state" refers to the user's psychological and sensory state.

[0964] "Recognizing an emotional state" means identifying a user's psychological state by analyzing the user's facial expressions, voice, etc.

[0965] "Adaptively adjusting the difficulty level" means dynamically changing the difficulty level of the game depending on the emotional state of the user.

[0966] The present invention combines a game system in which users compete for points by tapping in accordance with a specific rhythm with an emotion engine that recognizes the emotional state of the user. The system of the present invention comprises a server, a user terminal, an emotion engine, and a user. Specific embodiments for carrying out the present invention are described below.

[0967] server

[0968] The server plays a central role in this system. At the start of the game, the server randomly generates a rhythm pattern using Python's random module, for example. This rhythm pattern indicates the time intervals that the user must follow, and is set in the format of, for example, "1.1 seconds, 0.8 seconds, 1.2 seconds." After generating the rhythm, the server transmits the generated rhythm pattern to the user's device. The server then transmits notification signals sequentially according to each timing of the rhythm and receives the user's tap timing in real time. The server then evaluates the score based on the received tap timing and notifies the user's device of the evaluation result.

[0969] Emotion Engine

[0970] The emotion engine is responsible for analyzing the user's emotional state using sensors such as the user's camera and microphone. Specifically, the emotion engine collects and analyzes the user's facial expressions and tone of voice in real time. As a result of the analysis, it extracts the user's emotional state, such as whether they are relaxed or stressed. This emotional state is sent to the server, which adaptively adjusts the difficulty of the rhythm based on this data. For example, if the user is stressed, the interval between rhythms is increased, and if they are relaxed, the interval is decreased.

[0971] User terminal

[0972] The user device functions as an interface that allows the user to tap along to the rhythm. First, the user device displays the rhythm pattern received from the server to the user. It also analyzes the user's facial expressions and voice based on data obtained from the emotion engine and sends the analysis results to the server. When the user taps, the timing of the tap is recorded and sent to the server in real time. After the game ends, the user device displays the score sent from the server to the user.

[0973] User

[0974] Users tap according to the rhythm patterns displayed on their device. The emotion engine analyzes the user's emotional state and adjusts the difficulty of the rhythm based on the results, providing an optimal gaming experience for each individual user. After the game is over, users can check their score on their device.

[0975] Specific examples

[0976] For example, consider the case where user A starts a game. The server generates a rhythm pattern of "1.1 seconds, 0.8 seconds, 1.2 seconds," and the emotion engine analyzes that user A is relaxed. Based on this information, the server keeps the rhythm difficulty at normal and sends a notification signal to user A. During the game, user A taps according to the rhythm, and the tap timing is recorded on the device and sent to the server. After the game ends, the server analyzes the tap timing data, calculates a score, and notifies user A's device. On the other hand, if user B is feeling stressed, the emotion engine detects this and the server adaptively adjusts the difficulty of the rhythm. By widening the interval between the rhythms, user B can enjoy the game. After the game ends, the server calculates a score based on the adjusted rhythm and notifies the user's device.

[0977] Prompt Sentence Examples

[0978] Below are some example prompts to input to the generative AI model:

[0979] Describe a game system in which a user taps to a specific rhythm. The system has a built-in emotion engine that recognizes the user's emotions. The server randomly generates rhythms, and the emotion engine adjusts the difficulty of the rhythm according to the user's emotional state. If the user is relaxed, the rhythm becomes more difficult, and if the user is stressed, the rhythm becomes easier. Please explain the process in detail.

[0980] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0981] Step 1: Prepare to start the game

[0982] The server detects when the user taps the start button. Based on this input, the server generates a rhythm pattern using Python's random module. This rhythm pattern consists of time intervals such as "1.1 seconds, 0.8 seconds, 1.2 seconds." The generated rhythm pattern is sent to the user's device.

[0983] Specific operation: The server detects when the user starts the game → generates a rhythm (random) → sends it to the user's device

[0984] Step 2: Receiving and displaying rhythm patterns

[0985] The user terminal receives the rhythm pattern sent from the server. Based on this input, the rhythm pattern is presented to the user visually or audibly. Specific output may include displaying the rhythm interval on the screen or announcing the rhythm to the user via voice.

[0986] Specific operation: The user device receives the rhythm pattern → displays it on the screen or gives a voice notification

[0987] Step 3: Sending a rhythmic notification signal

[0988] The server sequentially transmits notification signals according to the rhythmic pattern. These notification signals include the timing at which the user should tap, and the user terminal conveys this information to the user in real time. The user taps based on these notification signals.

[0989] Specific operation: The server generates a timing signal according to the rhythm pattern and sends it to the user terminal.

[0990] Step 4: Recognizing your emotional state

[0991] The emotion engine analyzes data input from the user's camera and microphone to recognize the user's emotional state. Using facial recognition and voice analysis software, it determines whether the user is relaxed or stressed. This emotional data is then sent to a server.

[0992] Specific operation: The emotion engine collects data from the camera and microphone → analyzes the emotional state → sends it to the server

[0993] Step 5: Adjust your rhythm according to your emotional state

[0994] The server receives the emotion data sent from the emotion engine and adjusts the difficulty of the rhythm based on this data. For example, if the user is feeling stressed, the server may adjust the rhythm by widening the intervals between rhythms. The adjusted rhythm pattern is then retransmitted to the user's device.

[0995] Specific operation: The server receives emotion data → adjusts the rhythm pattern → sends it to the user's device

[0996] Step 6: Record your tap timing

[0997] When the user taps in time with the rhythm according to the instructions on the device, the user's device records the timing of the tapping. This recorded data is sent to the server in real time, and the server evaluates the accuracy of the tapping based on this data.

[0998] Specific operation: User taps in time with the rhythm → User device records tap timing → Sends to server

[0999] Step 7: Calculating and reporting scores

[1000] The server calculates the user's score based on the recorded tap timing. It evaluates the error between the acquired tap data and the rhythm pattern and calculates the score. This score is sent to the user's device and notified to the user.

[1001] Specific operation: The server evaluates the tap data → calculates the score → sends it to the user's device

[1002] Step 8: Ending the game and displaying the results

[1003] When the game ends, the user terminal displays the score sent from the server to the user, allowing the user to check their game results.

[1004] Specific operation: The server recognizes the end of the game → sends the score → the user device displays the score

[1005] (Application example 2)

[1006] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1007] Conventional rhythm game systems have been insufficient in personalizing the game experience according to the user's emotional state, and have provided a uniform level of difficulty even when the user is in a different emotional state, resulting in limited stress relief and relaxation effects. In particular, reducing customer stress and providing a comfortable environment while waiting in line at physical stores and other locations is required, but no adaptive rhythm game system has been available to achieve this.

[1008] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1009] In this invention, the server includes a means for generating a rhythm, a means for transmitting a notification signal according to a specific rhythm to a user terminal, a means for receiving tap timings from the user terminal, a means for evaluating a user's score based on the received tap timings, a means for notifying the user terminal of the evaluated score, and a means for adjusting the difficulty of the rhythm based on emotion analysis data. This makes it possible to personalize the rhythm game according to the user's emotional state, thereby improving the comfort of waiting times at physical stores.

[1010] A "rhythm" is a pattern in which a user should perform an action according to a specified time interval.

[1011] A "game server" is a computer system that generates rhythms, sends notification signals, receives tap timing, evaluates scores, and so on.

[1012] A "user terminal" is a device operated by a user, which communicates with the game server, records tap timing, and receives notification signals.

[1013] The "notification signal" is a signal that indicates the timing at which the user should tap in accordance with the rhythm.

[1014] The "tap timing" refers to the moment when the user operates the terminal.

[1015] The "score" is a score that is evaluated based on how well the user's tap timing matches the rhythm.

[1016] "Emotion analysis data" refers to data that indicates the emotional state of a user inferred from biometric information and behavioral information.

[1017] "Rhythm difficulty" refers to the difficulty of the game, determined by the complexity and speed of the rhythm.

[1018] This invention relates to a rhythm game system that allows customers to spend their waiting time in a fun and meaningful way in a brick-and-mortar store. This system is composed of a game server, a user terminal, and an emotion analysis engine, and the roles of each are explained below.

[1019] Game Server

[1020] The server plays the central role in this system. It is equipped with a rhythm-generating program that randomly generates a different rhythm for each user. The generated rhythm is sent to the user's device as a notification signal. The server also evaluates the user's score based on tap timing data received from the user's device and notifies the user of the result. Furthermore, by adjusting the difficulty of the rhythm based on emotion analysis data, the server provides a game experience that suits the user's emotional state.

[1021] User terminal

[1022] The user device primarily functions as an interface. It displays notification signals sent from the server to the user, and the user taps along to the rhythm. The tapping timing is recorded in real time and sent to the server. The device also has a built-in camera and microphone, and these sensors are used to analyze the user's emotional state. The analyzed emotional data is sent to the server and used to adjust the difficulty of the rhythm.

[1023] Sentiment Analysis Engine

[1024] The emotion analysis engine is a program that recognizes the user's emotional state in real time. It uses facial recognition and voice analysis technology to analyze the user's facial expressions and tone of voice to extract emotional states such as "relaxed" or "stressed." This emotional data is sent from the user's device to the server and used to adjust the game's difficulty level.

[1025] Hardware and software used

[1026] Hardware: Smart glasses, smartphones

[1027] Software: Flask (Python web framework), OpenCV (image analysis library), Emotion Recognition Model (emotion analysis engine)

[1028] Specific examples

[1029] For example, consider the case where a customer wearing smart glasses is waiting in line at a brick-and-mortar store. The camera captures the customer's face, and the image data is sent to the server. The server detects "stress" through its emotion analysis engine, and reduces the difficulty of the rhythm accordingly, generating a simple rhythm of "1.8 seconds, 2.0 seconds, 1.7 seconds." This rhythm is sent to the smart glasses as a notification signal, and the customer taps according to the rhythm.

[1030] Prompt Sentence Examples

[1031] Capture camera images and analyze the emotional state of customers. Based on the emotional state, generate rhythm patterns for a rhythm game that can be enjoyed while customers wait.

[1032] As described above, the present invention provides a rhythm game that is personalized according to the user's emotional state, making waiting times at physical stores more comfortable.

[1033] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1034] Step 1:

[1035] The customer wears the smart glasses and the camera image is captured by the terminal.

[1036] Input: Customer's face image

[1037] How it works: The camera in the smart glasses captures the customer's face and generates image data.

[1038] Step 2:

[1039] The device transmits the captured facial image to the server.

[1040] Input: A captured face image

[1041] Output: Facial image data sent to the server

[1042] Specific operation: The device transfers the captured facial image to the server in real time.

[1043] Step 3:

[1044] The server uses an emotion analysis engine to analyze the facial image and recognize the customer's emotional state.

[1045] Input: Facial image data

[1046] Output: Perceived emotional state (e.g., stressed, relaxed)

[1047] Specific operation: The server starts the emotion analysis engine and analyzes facial expressions from the facial image to extract the emotional state.

[1048] Step 4:

[1049] The server adjusts the difficulty of the rhythm based on the emotional state and generates a rhythm pattern.

[1050] Input: Perceived emotional state

[1051] Output: Rhythm pattern (e.g. 1.8 seconds, 2.0 seconds, 1.7 seconds)

[1052] Specific operation: When the emotional state is "stressed," the server executes a program that generates a simple rhythm pattern.

[1053] Step 5:

[1054] The server transmits the generated rhythm pattern to the terminal as a notification signal.

[1055] Input: Rhythm pattern

[1056] Output: Notification signal sent to the device

[1057] Specific operation: The server formats the generated rhythm pattern as a notification signal and sends it to the terminal.

[1058] Step 6:

[1059] The terminal receives the notification signal and displays tapping instructions to the customer according to the rhythm pattern.

[1060] Input: Notification signal

[1061] Output: Tap instructions shown to the customer

[1062] Specific operation: The device decodes the notification signal and displays instructions on the smart glasses display instructing when to tap.

[1063] Step 7:

[1064] The customer taps the smart glasses in rhythm.

[1065] Input: Tap instructions

[1066] Output: Tap timing

[1067] Specific operation: The customer taps the device according to the tap instructions displayed by the smart glasses.

[1068] Step 8:

[1069] The device records the tap timing and transmits it to the server in real time.

[1070] Input: Tap timing

[1071] Output: Tap timing data sent to the server

[1072] Specific operation: The terminal accurately measures the customer's tap timing and sends the data to the server.

[1073] Step 9:

[1074] The server compares the received tap timing with the rhythm pattern and evaluates the customer's score.

[1075] Input: Tap timing data, rhythm pattern

[1076] Output: Estimated score

[1077] Specific operation: The server evaluates the degree of agreement between the tap timing and the rhythm pattern, and calculates the score using a score calculation program.

[1078] Step 10:

[1079] The server notifies the terminal of the evaluated score, and the terminal displays the score to the customer.

[1080] Input: Estimated score

[1081] Output: Scoring information displayed to the customer

[1082] Specific operation: The server sends the score data to the terminal, which receives it and displays it to the customer.

[1083] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1084] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1085] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1086] [Fourth embodiment]

[1087] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1088] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1089] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1090] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1091] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1092] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1093] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1094] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1095] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1096] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1097] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1098] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1099] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1100] This invention is a game system in which users compete for points by tapping in accordance with a specific rhythm. Below, each component of the system, its interactions, and the specific flow of operation will be described in detail.

[1101] server

[1102] The server plays a central role in the game. First, the server randomly generates a rhythm at the start of the game. This is the pattern of time intervals that the user must follow. This rhythm information becomes the basis for the user to tap accurately.

[1103] The server then sends a rhythmic notification signal to the user's device, allowing the user to receive tapping instructions in real time. The server records the timing of the user's taps and analyzes the tap interval data.

[1104] The server then evaluates the user's score based on the recorded timing of the taps. The evaluation is done by calculating the difference between the user's tap timing and the pre-generated rhythm. The closer the user's taps are to the rhythm, the higher the score.

[1105] Terminal

[1106] The user terminal functions as a user interface. When the game starts, the terminal displays the message "Game Start" to the user. When the terminal receives a rhythm notification signal from the server, it sends the message "Tap now!" to the user.

[1107] The device has a function to record the timing of taps, and saves the time each time the user taps. The saved tap timings are sent to the server in real time. At the end of the game, the device receives the evaluation results from the server and displays them to the user.

[1108] User

[1109] The user taps the device in accordance with the instructions on the device. The user's goal is to tap as accurately as possible to the rhythm instructed by the server. As the game progresses, the user taps multiple times, and the timing of each tap is sent to the server.

[1110] Specific examples

[1111] For example, suppose the rhythm randomly generated by the server at the start of the game is a pattern of "1.1 seconds, 0.8 seconds, 1.2 seconds." According to this rhythm, the first tap must be made 1.1 seconds after the start of the game, the next tap 0.8 seconds after that, and then 1.2 seconds after that.

[1112] 1. Based on these time intervals, the server will sequentially send a "Tap now!" notification signal to the device.

[1113] 2. The user follows the instructions on the device and taps at the specified timing.

[1114] 3. The device records the time of each tap and transmits it to the server in real time.

[1115] 4. After the game ends, the server compares the user's tap timing with the rhythm and calculates the score.

[1116] 5. Finally, the device displays the score to the user and provides feedback on the game results.

[1117] In this way, the user, terminal, and server work together to realize a new type of game system that measures the user's sense of rhythm and accuracy.

[1118] The processing flow will be explained below.

[1119] Step 1:

[1120] The server initializes the game and sets the game duration and rhythm pattern, which is randomly generated, for example, "1.1 seconds, 0.8 seconds, 1.2 seconds."

[1121] Step 2:

[1122] The user starts up the terminal and prepares to start the game. The terminal connects to the server and waits for a signal to start the game.

[1123] Step 3:

[1124] The server sends a signal to the terminal to start the game and simultaneously records the start time of the game. The terminal displays the message "Game Start" to the user.

[1125] Step 4:

[1126] The server sends a notification signal saying "Tap now!" to the device at specified intervals according to the rhythm pattern. The device displays this notification on its screen.

[1127] Step 5:

[1128] The user sees the notification and taps the device at the specified timing. The timing of each tap is recorded.

[1129] Step 6:

[1130] The device sends the timing of the user's taps to the server in real time, and the server receives and records them.

[1131] Step 7:

[1132] When the game duration ends, the server sends a signal to the terminal to end the game, causing the terminal to display a message saying "Game End." At the same time, the server starts the process of evaluating the score based on the user's tap timing data.

[1133] Step 8:

[1134] The server compares the rhythm pattern with the user's tap timing and calculates a score based on the accuracy of each tap. The closer the user's taps are to the rhythm, the higher the score.

[1135] Step 9:

[1136] The server sends the evaluated score to the user's terminal, which displays the score to the user and provides feedback on the game result.

[1137] Step 10:

[1138] The user checks their score on the terminal and ends the game.

[1139] The above are the processing steps of the present invention. By tapping along a specific rhythm, users can compete with each other in terms of rhythmic sense and tapping accuracy.

[1140] Example 1

[1141] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1142] Rhythm games require a system that allows users to tap accurately in time with the rhythm and evaluate the results in real time. However, current systems face the challenge of urging users to tap accurately in accordance with randomly generated rhythms and accurately recording and evaluating the timing of their taps. To solve this challenge, improvements to the user interface and faster data communication are required.

[1143] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1144] In this invention, the server includes means for generating a rhythm, means for transmitting a notification signal according to a specific rhythm to a user terminal, means for receiving tap timings from the user terminal, means for evaluating a user's score based on the received tap timings, means for notifying the user terminal of the evaluated score, means for sequentially transmitting notification signals according to the rhythm, means for evaluating the score by comparing the received tap timings with a randomly generated rhythm, and means for notifying the user terminal of the score result. This allows the user to tap faithfully to the specified rhythm, and the results can be accurately evaluated and displayed in real time.

[1145] A "game server" is a central system that generates rhythms, receives tap timings, evaluates scores, and notifies users of the results of their scores in a game in which users tap in accordance with the rhythm.

[1146] The "rhythm" refers to a pattern of time intervals that serves as a basis for the user to perform tapping.

[1147] The "notification signal" is a signal sent from the server to the user terminal to notify the timing of tapping in accordance with the rhythm.

[1148] A "user terminal" is a device operated by a user playing a game, which receives notification signals from the server and records the timing of taps.

[1149] "Tap timing" refers to the exact time when a user taps the device, and this data is used for evaluation on the server.

[1150] The "score" is an evaluation value calculated based on how closely the user's tap timing matches the rhythm.

[1151] "Random" means that something is determined randomly without following any particular rule, and is used here in the context of rhythm generation.

[1152] The "evaluation means" is a function that compares the received tap timing with the rhythm and calculates the user's score.

[1153] The "notification means" is a function for transmitting the evaluated score to the user terminal and displaying it to the user.

[1154] This invention is a game system in which users compete for points by tapping in accordance with a specific rhythm. Below, each component of the system, its interactions, and the specific flow of operation will be described in detail.

[1155] server

[1156] The server plays a central role in the game. When the game starts, the server randomly generates a specific rhythm. This rhythm is the pattern of time intervals that the user must follow. The rhythm is generated using a random number generation algorithm and a library of time patterns (e.g., the random module). This rhythm information becomes the basis for the user to tap accurately.

[1157] Next, the server sequentially sends rhythmic notification signals to the user's device, allowing the user to receive tapping instructions in real time. Communication protocols such as HTTP requests and WebSockets are used to send the notification signals. Specifically, the server sends a "tap" notification 1.1 seconds later.

[1158] The server records the timing of the user's taps and evaluates the user's score based on that data. The evaluation is done by calculating the difference between the user's tap timing and a pre-generated rhythm. This evaluation uses absolute difference calculations and a statistical library (e.g., NumPy). The closer the user's taps are to the rhythm, the higher the score will be.

[1159] Terminal

[1160] The user device functions as a user interface. When the game starts, the device displays the message "Game Start" to the user. When the device receives a rhythm notification signal from the server, it sends the message "Tap now!" to the user. To display the notification, a UI library (e.g., React, SwiftUI) is used to draw the message on the screen.

[1161] The device has a function to record tap timing, and saves the time every time the user taps in milliseconds. The saved tap timing is sent to the server in real time. This communication also uses HTTP requests and WebSockets. At the end of the game, the device receives the evaluation result from the server and displays it to the user. A UI library is used again for this, and a message such as "Your score is 85 points" is displayed.

[1162] User

[1163] The user taps the device in accordance with the instructions of the device. The user's goal is to tap as accurately as possible to the rhythm instructed by the server. The timing of the taps is recorded on the device and transmitted to the server in real time.

[1164] Specific operation example

[1165] For example, suppose the rhythm randomly generated by the server at the start of the game is a pattern of "1.1 seconds, 0.8 seconds, 1.2 seconds." In this case, the user must tap the first time 1.1 seconds after the start of the game, then tap the next time 0.8 seconds later, and then tap the next time 1.2 seconds later.

[1166] 1. Based on these time intervals, the server will sequentially send a "Tap now!" notification signal to the device.

[1167] 2. The user follows the instructions on the device and taps at the specified timing.

[1168] 3. The device records the time of each tap and transmits it to the server in real time.

[1169] 4. After the game ends, the server compares the user's tap timing with the rhythm and calculates the score.

[1170] 5. Finally, the device displays the score to the user and provides feedback on the game results.

[1171] Examples of prompt statements

[1172] Here are some example prompts to input to a generative AI model:

[1173] "You are now participating in a musical note game. Please tap according to the following rhythm: 1.1 seconds, 0.8 seconds, 1.2 seconds. Please tap accurately at each timing."

[1174] In this way, by realizing this game system through cooperation between the server, terminals, and users, the users' sense of rhythm can be measured and enjoyed.

[1175] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1176] Step 1:

[1177] The server generates the rhythm.

[1178] Input: Rhythm generation algorithm (e.g., random module)

[1179] Processing: To generate rhythms, a random number generation algorithm is used to randomly determine time intervals (e.g., 1.1 seconds, 0.8 seconds, 1.2 seconds).

[1180] Output: Generated rhythm pattern (e.g. "1.1 seconds, 0.8 seconds, 1.2 seconds")

[1181] Step 2:

[1182] The server transmits a rhythm notification signal to the user terminal.

[1183] Input: Generated rhythm pattern

[1184] Processing: The server sequentially sends "Tap now!" notification signals after a specified time (e.g., 1.1 seconds, then 0.8 seconds, then 1.2 seconds) based on the generated rhythm pattern. This transmission uses a communication protocol such as an HTTP request or WebSocket.

[1185] Output: Notification signal to the user device (e.g. "Tap now!")

[1186] Step 3:

[1187] The device displays a notification signal.

[1188] Input: Notification signal from the server

[1189] Processing: The device displays the notification signal received from the server in the user interface. Specifically, a UI library (e.g., React, SwiftUI) is used to display the message "Tap now!" on the screen.

[1190] Output: The message that is displayed to the user (e.g., "Tap now!")

[1191] Step 4:

[1192] The user taps.

[1193] Input: The message displayed on the terminal

[1194] Action: The user taps the screen in response to the instruction from the device, "Tap now!"

[1195] Output: User's tap action

[1196] Step 5:

[1197] The device records the timing of your taps.

[1198] Input: User tap action

[1199] Processing: The device records the exact moment the user tapped, down to the millisecond, using the system clock or a timer library (e.g., Date.now(), System.currentTimeMillis()).

[1200] Output: Recorded tap timing

[1201] Step 6:

[1202] The device sends the tap timing to the server.

[1203] Input: Recorded tap timing

[1204] Processing: The device sends the recorded tap timings to the server in real time via HTTP requests or WebSockets in JSON format.

[1205] Output: Tap timing data to the server

[1206] Step 7:

[1207] The server evaluates the tap timing.

[1208] Input: Received tap timing, generated rhythm pattern

[1209] Processing: The server compares the user's tap timing with the generated rhythm and calculates the user's score. Absolute difference calculations and statistical libraries (e.g., NumPy) are used. For example, a high score is awarded if the tap timing is within ±50 milliseconds of the rhythm.

[1210] Output: Calculated score

[1211] Step 8:

[1212] The server transmits the evaluation results to the terminal.

[1213] Input: Calculated score

[1214] Processing: The server sends the score results in JSON format to the device, again using an HTTP request or WebSocket.

[1215] Output: Score result data to the terminal

[1216] Step 9:

[1217] The terminal displays the evaluation results.

[1218] Input: Score result data from the server

[1219] Processing: The device displays the received score result to the user. This is done by using the UI library again, and displays a message such as "Your score is 85 points."

[1220] Output: Scoring results displayed to the user

[1221] In this way, the rhythm tap game system is realized by the server, terminals, and users operating in cooperation with each other.

[1222] (Application example 1)

[1223] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1224] In autonomous vehicles, there is a need for a means to check the driver's attention and reaction time to improve safety. In conventional autonomous driving systems, the driver may monitor the driving situation for long periods of time without operating the vehicle, which can lead to a decline in attention. Therefore, there is a lack of effective methods to maintain the driver's attention and promote appropriate reactions.

[1225] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1226] In this invention, the server includes means for generating a rhythm, means for transmitting a notification signal according to a specific rhythm to a user terminal, means for receiving tap timing from the user terminal, means for evaluating a user's score based on the received tap timing, means for notifying the user terminal of the evaluated score, means for requesting the user to perform an operation according to the rhythm while the vehicle is in automatic driving mode and measuring the reaction speed and accuracy, and means for evaluating the user's attention and confirming safety. This allows the driver to be periodically reminded of the driver's attention even during automatic driving, making it possible to maintain appropriate reaction speed and accuracy.

[1227] A "rhythm" is a time pattern that serves as a reference for a user to perform an operation according to a specific time interval.

[1228] A "notification signal" is a signal that is sent from a server to a user terminal and prompts the user to perform an operation according to a specific rhythm.

[1229] "Tap timing" is the time when the user operates the terminal in accordance with the specified rhythm.

[1230] A "game server" is a central system that generates rhythms, sends notification signals, receives tap timings, and evaluates scores.

[1231] A "user terminal" is a device that provides a user interface, displays notification signals from the server, and records user operations.

[1232] "Score evaluation" is a process of evaluating the accuracy of the user's operation based on the difference between the received tap timing and rhythm, and expressing the result as a score.

[1233] "Vehicle is in automatic driving mode" means that the vehicle is under the control of an automatic driving system.

[1234] The "response speed" is the time it takes for a user to complete an operation in response to a notification signal.

[1235] "Accuracy" is a measure of how closely the user's operation matches a specific rhythm.

[1236] "Attention" refers to the concentration required for a user to perform a specified operation accurately and quickly.

[1237] server

[1238] The server provides the core functionality of the invention. First, the server generates a rhythm, which is a pattern of time intervals that the user should follow. The rhythm is generated randomly. The generated rhythm is then sent to the user terminal as a notification signal according to the specific rhythm.

[1239] Next, the server receives the tap timing data sent from the user device. Based on the received data, it calculates the difference between the tap timing and the rhythm to evaluate the user's score. It then notifies the user device of the evaluated score. In addition, in an autonomous vehicle, the server requests the user to operate the vehicle according to the rhythm while the vehicle is being driven automatically, measures the reaction speed and accuracy, and evaluates the user's attention.

[1240] Terminal

[1241] The user terminal functions as a user interface. When it receives a rhythm notification signal from the server, the terminal displays the message "Tap now!" to the user. Each time the user taps, the terminal stores the time, and the stored tap timing is sent to the server in real time. The terminal displays the evaluation results from the server to the user, providing feedback on the driver's attention and reaction speed.

[1242] User

[1243] The user taps according to the rhythm notification signal from the device. In an autonomous vehicle, the application is required to periodically prompt the user to operate according to the rhythm and maintain their attention.

[1244] Hardware and Software

[1245] The system is implemented using devices such as smartphones and head-mounted displays. The software uses Python to implement server-side and device-side programs. The server-side uses an API to exchange data in real time using the HTTP protocol.

[1246] Specific examples

[1247] For example, if the rhythm randomly generated by the server is "1.1 seconds, 0.8 seconds, 1.2 seconds," the server will send a notification signal to the device accordingly. The user taps at the right time according to the instruction "Tap now!", and the timing data is sent from the device to the server. Based on the received data, the server evaluates the user's reaction speed and accuracy, calculates a score, and notifies the device.

[1248] Prompt Sentence Examples

[1249] Design an application that measures a driver's reaction time by tapping at specific intervals according to the instruction "Tap now!". The system uses a smartphone or head-mounted display, and sends the recorded timing to a server for evaluation. Please explain the specific implementation method.

[1250] This allows the driver to maintain appropriate reaction speed and attention even during automated driving, improving safety.

[1251] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1252] Step 1:

[1253] The server generates the rhythm. The server generates a random time interval pattern (rhythm) and stores that information for the next processing step. The inputs are the set parameters and the random generation algorithm, and the output is the generated rhythm pattern.

[1254] Step 2:

[1255] The server sends the generated rhythm to the user's device as a notification signal. Based on the rhythm data, the server sends a message saying "Tap now!", which the device displays. The input is the generated rhythm pattern, and the output is a notification signal and a display message to the user's device.

[1256] Step 3:

[1257] The user device receives the notification signal and prompts the user to operate in accordance with the rhythm. The device displays the message "Tap now!" in accordance with the rhythm, prompting the user to tap at a specific timing. The input is the notification signal from the server, and the output is the message displayed to the user.

[1258] Step 4:

[1259] The user taps according to the notification message. The user taps the device at the specified timing, and the time of the tap is recorded. The input is the user's tap operation, and the output is the recorded tap timing.

[1260] Step 5:

[1261] The user terminal transmits the tap timing to the server. The terminal transmits the recorded tap timing to the server in real time, and the tap data is accumulated on the server. The input is the recorded tap timing, and the output is the data transmitted to the server.

[1262] Step 6:

[1263] The server evaluates the score based on the received tap timing. The server calculates the difference between the tap timing and the generated rhythm and evaluates the accuracy of the user's operation. The input is the received tap timing and rhythm pattern, and the output is the calculated score.

[1264] Step 7:

[1265] The server notifies the user terminal of the evaluated score. The server then sends the evaluation result to the user terminal, which then displays it to the user. The input is the calculated score, and the output is a notification signal and a display message to the user terminal.

[1266] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1267] This invention combines a game system in which users compete for points by tapping in accordance with a specific rhythm with an emotion engine that recognizes the user's emotions. Below, we will explain each component of this system, its interactions, and the specific operation flow.

[1268] server

[1269] The server plays an important role in the game. First, the server randomly generates a rhythm when the game starts. This rhythm is a pattern of time intervals that the user must follow, and is set, for example, as "1.1 seconds, 0.8 seconds, 1.2 seconds." The generated rhythm is then sent to the user's device.

[1270] Next, the server sequentially sends rhythmic notification signals to the user's device. The notification signals include the timing at which the user should tap. The timing of the taps is recorded on the device and transmitted to the server in real time. Based on this, tap timing data is collected and saved.

[1271] The server then evaluates the user's score based on the recorded timing of the taps, assessing how closely the user's taps match the rhythm, and calculates the score based on the results. The calculated score is then sent to the user's device and notified to the user.

[1272] Emotion Engine

[1273] The emotion engine is used to recognize the user's emotional state and adjust the difficulty of the game. The emotion engine uses sensors such as cameras and microphones to analyze the user's facial expressions and tone of voice, and has the ability to extract emotions in real time.

[1274] The emotion engine sends the user's emotional state to the server and can adaptively adjust the difficulty of the rhythm based on that data. For example, if the user is stressed, the rhythm can be made easier, while if the user is relaxed, the rhythm can be made more difficult. This makes the game experience more personalized and enjoyable.

[1275] User terminal

[1276] The user device functions as an interface, receiving notification signals from the server and displaying them to the user. When the user taps, the device records the timing of the tap and sends it to the server in real time. It also analyzes the user's facial expressions and voice based on data from the emotion engine.

[1277] User

[1278] The user follows the instructions on the device and taps the device in time with the rhythm. The emotion engine analyzes the user's emotional state and adjusts the difficulty of the rhythm based on that, so the game experience is tailored to each individual user's emotional state. After the game is over, the user can check their score sent from the server on their device.

[1279] Specific examples

[1280] For example, suppose User A starts a game. The server generates a rhythm of "1.1 seconds, 0.8 seconds, 1.2 seconds," and the emotion engine analyzes User A's facial expressions and voice to determine that he or she is relaxed. Based on this data, the server maintains the rhythm as normal and sends a notification signal to User A.

[1281] During the game, User A taps according to the rhythm. The device records the tap timing and sends it to the server. After the game ends, the server analyzes the tap timing data, calculates User A's score, and notifies the device.

[1282] On the other hand, if User B is feeling stressed, the emotion engine detects this and adaptively adjusts the difficulty of the rhythm by widening the intervals between rhythms, allowing User B to enjoy the game. After the game ends, the server evaluates the tap timing based on the adjusted rhythm, calculates the appropriate score, and notifies the user.

[1283] In this way, a more personalized gaming experience can be provided by recognizing and adaptively adjusting the user's emotional state in real time.

[1284] The processing flow will be explained below.

[1285] Step 1:

[1286] The server initializes the game, sets the game duration, and randomly generates rhythm patterns.

[1287] Step 2:

[1288] The server transmits the generated rhythm pattern to the user terminal and issues a signal to start the game.

[1289] Step 3:

[1290] The device displays the message "Game Start" to the user, and the emotion engine begins to recognize and record the user's emotional state.

[1291] Step 4:

[1292] The server sends a notification signal to the user terminal saying "Tap now!" at specified time intervals based on the rhythm pattern. The notification includes the timing of the rhythm.

[1293] Step 5:

[1294] The device displays the received notification signal on the screen and prompts the user to tap. The emotion engine analyzes the user's facial expressions and voice and transmits the user's emotional state to the server in real time.

[1295] Step 6:

[1296] The user follows the notification from the device and taps the device at the specified timing. The timing of each tap is recorded.

[1297] Step 7:

[1298] The device records the tap timing and transmits the data in real time to the server, which then analyzes the data.

[1299] Step 8:

[1300] The server adjusts the difficulty of the rhythm based on the user's emotional state, for example, making the rhythm easier if the user is feeling stressed.

[1301] Step 9:

[1302] When the game duration ends, the server sends a signal to the terminal to end the game, causing the terminal to display a message saying "Game End." The server evaluates the user's score based on the tap timing data and emotion data.

[1303] Step 10:

[1304] The server compares the rhythmic patterns with the user's tap timing and emotional state, and calculates a score based on the accuracy of each tap. The closer the user's taps are to the rhythm, the higher the score, taking into account their emotional state.

[1305] Step 11:

[1306] The server sends the evaluated score to the user's terminal, which displays the score to the user and provides feedback on the game result.

[1307] Step 12:

[1308] The user checks their score on the terminal and ends the game.

[1309] Example 2

[1310] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1311] In conventional rhythm games, fixed difficulty settings result in a uniform playing experience for all users, making it difficult to provide an optimal game experience tailored to each individual user's emotional state. As a result, the level of stress or relaxation felt by the user during the game is not reflected, which can lead to a decrease in motivation to play and a decrease in satisfaction. Furthermore, because the difficulty setting for rhythm games is done manually, dynamic difficulty adjustment is not possible, making it difficult to appropriately adjust the game to suit the user's skill level and emotional state.

[1312] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1313] In this invention, the server includes means for generating a rhythm, means for transmitting a notification signal according to a specific rhythm to a user terminal, means for receiving tap timings from the user terminal, means for evaluating a user's score based on the received tap timings, means for notifying the user terminal of the evaluated score, means for recognizing the user's emotional state, and means for adaptively adjusting the difficulty of the rhythm based on the recognized emotional state. This makes it possible to dynamically adjust the difficulty of the game according to the user's emotional state, thereby providing an optimal gaming experience for each individual user.

[1314] A "rhythm" is a time pattern that allows a user to take action at specified time intervals.

[1315] A "user terminal" is a device that allows a user to make rhythmic inputs.

[1316] The "notification signal" is a signal that notifies the user of an action that follows the rhythm.

[1317] The "tap timing" refers to the moment in time when the user operates the user terminal.

[1318] "Evaluating the score" means determining how closely the timing of the user's taps matches the rhythm.

[1319] "Emotional state" refers to the user's psychological and sensory state.

[1320] "Recognizing an emotional state" means identifying a user's psychological state by analyzing the user's facial expressions, voice, etc.

[1321] "Adaptively adjusting the difficulty level" means dynamically changing the difficulty level of the game depending on the emotional state of the user.

[1322] The present invention combines a game system in which users compete for points by tapping in accordance with a specific rhythm with an emotion engine that recognizes the emotional state of the user. The system of the present invention comprises a server, a user terminal, an emotion engine, and a user. Specific embodiments for carrying out the present invention are described below.

[1323] server

[1324] The server plays a central role in this system. At the start of the game, the server randomly generates a rhythm pattern using Python's random module, for example. This rhythm pattern indicates the time intervals that the user must follow, and is set in the format of, for example, "1.1 seconds, 0.8 seconds, 1.2 seconds." After generating the rhythm, the server transmits the generated rhythm pattern to the user's device. The server then transmits notification signals sequentially according to each timing of the rhythm and receives the user's tap timing in real time. The server then evaluates the score based on the received tap timing and notifies the user's device of the evaluation result.

[1325] Emotion Engine

[1326] The emotion engine is responsible for analyzing the user's emotional state using sensors such as the user's camera and microphone. Specifically, the emotion engine collects and analyzes the user's facial expressions and tone of voice in real time. As a result of the analysis, it extracts the user's emotional state, such as whether they are relaxed or stressed. This emotional state is sent to the server, which adaptively adjusts the difficulty of the rhythm based on this data. For example, if the user is stressed, the interval between rhythms is increased, and if they are relaxed, the interval is decreased.

[1327] User terminal

[1328] The user device functions as an interface that allows the user to tap along to the rhythm. First, the user device displays the rhythm pattern received from the server to the user. It also analyzes the user's facial expressions and voice based on data obtained from the emotion engine and sends the analysis results to the server. When the user taps, the timing of the tap is recorded and sent to the server in real time. After the game ends, the user device displays the score sent from the server to the user.

[1329] User

[1330] Users tap according to the rhythm patterns displayed on their device. The emotion engine analyzes the user's emotional state and adjusts the difficulty of the rhythm based on the results, providing an optimal gaming experience for each individual user. After the game is over, users can check their score on their device.

[1331] Specific examples

[1332] For example, consider the case where user A starts a game. The server generates a rhythm pattern of "1.1 seconds, 0.8 seconds, 1.2 seconds," and the emotion engine analyzes that user A is relaxed. Based on this information, the server keeps the rhythm difficulty at normal and sends a notification signal to user A. During the game, user A taps according to the rhythm, and the tap timing is recorded on the device and sent to the server. After the game ends, the server analyzes the tap timing data, calculates a score, and notifies user A's device. On the other hand, if user B is feeling stressed, the emotion engine detects this and the server adaptively adjusts the difficulty of the rhythm. By widening the interval between the rhythms, user B can enjoy the game. After the game ends, the server calculates a score based on the adjusted rhythm and notifies the user's device.

[1333] Prompt Sentence Examples

[1334] Below are some example prompts to input to the generative AI model:

[1335] Describe a game system in which a user taps to a specific rhythm. The system has a built-in emotion engine that recognizes the user's emotions. The server randomly generates rhythms, and the emotion engine adjusts the difficulty of the rhythm according to the user's emotional state. If the user is relaxed, the rhythm becomes more difficult, and if the user is stressed, the rhythm becomes easier. Please explain the process in detail.

[1336] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1337] Step 1: Prepare to start the game

[1338] The server detects when the user taps the start button. Based on this input, the server generates a rhythm pattern using Python's random module. This rhythm pattern consists of time intervals such as "1.1 seconds, 0.8 seconds, 1.2 seconds." The generated rhythm pattern is sent to the user's device.

[1339] Specific operation: The server detects when the user starts the game → generates a rhythm (random) → sends it to the user's device

[1340] Step 2: Receiving and displaying rhythm patterns

[1341] The user terminal receives the rhythm pattern sent from the server. Based on this input, the rhythm pattern is presented to the user visually or audibly. Specific output may include displaying the rhythm interval on the screen or announcing the rhythm to the user via voice.

[1342] Specific operation: The user device receives the rhythm pattern → displays it on the screen or gives a voice notification

[1343] Step 3: Sending a rhythmic notification signal

[1344] The server sequentially transmits notification signals according to the rhythmic pattern. These notification signals include the timing at which the user should tap, and the user terminal conveys this information to the user in real time. The user taps based on these notification signals.

[1345] Specific operation: The server generates a timing signal according to the rhythm pattern and sends it to the user terminal.

[1346] Step 4: Recognizing your emotional state

[1347] The emotion engine analyzes data input from the user's camera and microphone to recognize the user's emotional state. Using facial recognition and voice analysis software, it determines whether the user is relaxed or stressed. This emotional data is then sent to a server.

[1348] Specific operation: The emotion engine collects data from the camera and microphone → analyzes the emotional state → sends it to the server

[1349] Step 5: Adjust your rhythm according to your emotional state

[1350] The server receives the emotion data sent from the emotion engine and adjusts the difficulty of the rhythm based on this data. For example, if the user is feeling stressed, the server may adjust the rhythm by widening the intervals between rhythms. The adjusted rhythm pattern is then retransmitted to the user's device.

[1351] Specific operation: The server receives emotion data → adjusts the rhythm pattern → sends it to the user's device

[1352] Step 6: Record your tap timing

[1353] When the user taps in time with the rhythm according to the instructions on the device, the user's device records the timing of the tapping. This recorded data is sent to the server in real time, and the server evaluates the accuracy of the tapping based on this data.

[1354] Specific operation: User taps in time with the rhythm → User device records tap timing → Sends to server

[1355] Step 7: Calculating and reporting scores

[1356] The server calculates the user's score based on the recorded tap timing. It evaluates the error between the acquired tap data and the rhythm pattern and calculates the score. This score is sent to the user's device and notified to the user.

[1357] Specific operation: The server evaluates the tap data → calculates the score → sends it to the user's device

[1358] Step 8: Ending the game and displaying the results

[1359] When the game ends, the user terminal displays the score sent from the server to the user, allowing the user to check their game results.

[1360] Specific operation: The server recognizes the end of the game → sends the score → the user device displays the score

[1361] (Application example 2)

[1362] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1363] Conventional rhythm game systems have been insufficient in personalizing the game experience according to the user's emotional state, and have provided a uniform level of difficulty even when the user is in a different emotional state, resulting in limited stress relief and relaxation effects. In particular, reducing customer stress and providing a comfortable environment while waiting in line at physical stores and other locations is required, but no adaptive rhythm game system has been available to achieve this.

[1364] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1365] In this invention, the server includes a means for generating a rhythm, a means for transmitting a notification signal according to a specific rhythm to a user terminal, a means for receiving tap timings from the user terminal, a means for evaluating a user's score based on the received tap timings, a means for notifying the user terminal of the evaluated score, and a means for adjusting the difficulty of the rhythm based on emotion analysis data. This makes it possible to personalize the rhythm game according to the user's emotional state, thereby improving the comfort of waiting times at physical stores.

[1366] A "rhythm" is a pattern in which a user should perform an action according to a specified time interval.

[1367] A "game server" is a computer system that generates rhythms, sends notification signals, receives tap timing, evaluates scores, and so on.

[1368] A "user terminal" is a device operated by a user, which communicates with the game server, records tap timing, and receives notification signals.

[1369] The "notification signal" is a signal that indicates the timing at which the user should tap in accordance with the rhythm.

[1370] The "tap timing" refers to the moment when the user operates the terminal.

[1371] The "score" is a score that is evaluated based on how well the user's tap timing matches the rhythm.

[1372] "Emotion analysis data" refers to data that indicates the emotional state of a user inferred from biometric information and behavioral information.

[1373] "Rhythm difficulty" refers to the difficulty of the game, determined by the complexity and speed of the rhythm.

[1374] This invention relates to a rhythm game system that allows customers to spend their waiting time in a fun and meaningful way in a brick-and-mortar store. This system is composed of a game server, a user terminal, and an emotion analysis engine, and the roles of each are explained below.

[1375] Game Server

[1376] The server plays the central role in this system. It is equipped with a rhythm-generating program that randomly generates a different rhythm for each user. The generated rhythm is sent to the user's device as a notification signal. The server also evaluates the user's score based on tap timing data received from the user's device and notifies the user of the result. Furthermore, by adjusting the difficulty of the rhythm based on emotion analysis data, the server provides a game experience that suits the user's emotional state.

[1377] User terminal

[1378] The user device primarily functions as an interface. It displays notification signals sent from the server to the user, and the user taps along to the rhythm. The tapping timing is recorded in real time and sent to the server. The device also has a built-in camera and microphone, and these sensors are used to analyze the user's emotional state. The analyzed emotional data is sent to the server and used to adjust the difficulty of the rhythm.

[1379] Sentiment Analysis Engine

[1380] The emotion analysis engine is a program that recognizes the user's emotional state in real time. It uses facial recognition and voice analysis technology to analyze the user's facial expressions and tone of voice to extract emotional states such as "relaxed" or "stressed." This emotional data is sent from the user's device to the server and used to adjust the game's difficulty level.

[1381] Hardware and software used

[1382] Hardware: Smart glasses, smartphones

[1383] Software: Flask (Python web framework), OpenCV (image analysis library), Emotion Recognition Model (emotion analysis engine)

[1384] Specific examples

[1385] For example, consider the case where a customer wearing smart glasses is waiting in line at a brick-and-mortar store. The camera captures the customer's face, and the image data is sent to the server. The server detects "stress" through its emotion analysis engine, and reduces the difficulty of the rhythm accordingly, generating a simple rhythm of "1.8 seconds, 2.0 seconds, 1.7 seconds." This rhythm is sent to the smart glasses as a notification signal, and the customer taps according to the rhythm.

[1386] Prompt Sentence Examples

[1387] Capture camera images and analyze the emotional state of customers. Based on the emotional state, generate rhythm patterns for a rhythm game that can be enjoyed while customers wait.

[1388] As described above, the present invention provides a rhythm game that is personalized according to the user's emotional state, making waiting times at physical stores more comfortable.

[1389] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1390] Step 1:

[1391] The customer wears the smart glasses and the camera image is captured by the terminal.

[1392] Input: Customer's face image

[1393] How it works: The camera in the smart glasses captures the customer's face and generates image data.

[1394] Step 2:

[1395] The device transmits the captured facial image to the server.

[1396] Input: A captured face image

[1397] Output: Facial image data sent to the server

[1398] Specific operation: The device transfers the captured facial image to the server in real time.

[1399] Step 3:

[1400] The server uses an emotion analysis engine to analyze the facial image and recognize the customer's emotional state.

[1401] Input: Facial image data

[1402] Output: Perceived emotional state (e.g., stressed, relaxed)

[1403] Specific operation: The server starts the emotion analysis engine and analyzes facial expressions from the facial image to extract the emotional state.

[1404] Step 4:

[1405] The server adjusts the difficulty of the rhythm based on the emotional state and generates a rhythm pattern.

[1406] Input: Perceived emotional state

[1407] Output: Rhythm pattern (e.g. 1.8 seconds, 2.0 seconds, 1.7 seconds)

[1408] Specific operation: When the emotional state is "stressed," the server executes a program that generates a simple rhythm pattern.

[1409] Step 5:

[1410] The server transmits the generated rhythm pattern to the terminal as a notification signal.

[1411] Input: Rhythm pattern

[1412] Output: Notification signal sent to the device

[1413] Specific operation: The server formats the generated rhythm pattern as a notification signal and sends it to the terminal.

[1414] Step 6:

[1415] The terminal receives the notification signal and displays tapping instructions to the customer according to the rhythm pattern.

[1416] Input: Notification signal

[1417] Output: Tap instructions shown to the customer

[1418] Specific operation: The device decodes the notification signal and displays instructions on the smart glasses display instructing when to tap.

[1419] Step 7:

[1420] The customer taps the smart glasses in rhythm.

[1421] Input: Tap instructions

[1422] Output: Tap timing

[1423] Specific operation: The customer taps the device according to the tap instructions displayed by the smart glasses.

[1424] Step 8:

[1425] The device records the tap timing and transmits it to the server in real time.

[1426] Input: Tap timing

[1427] Output: Tap timing data sent to the server

[1428] Specific operation: The terminal accurately measures the customer's tap timing and sends the data to the server.

[1429] Step 9:

[1430] The server compares the received tap timing with the rhythm pattern and evaluates the customer's score.

[1431] Input: Tap timing data, rhythm pattern

[1432] Output: Estimated score

[1433] Specific operation: The server evaluates the degree of agreement between the tap timing and the rhythm pattern, and calculates the score using a score calculation program.

[1434] Step 10:

[1435] The server notifies the terminal of the evaluated score, and the terminal displays the score to the customer.

[1436] Input: Estimated score

[1437] Output: Scoring information displayed to the customer

[1438] Specific operation: The server sends the score data to the terminal, which receives it and displays it to the customer.

[1439] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1440] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1441] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1442] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1443] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1444] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1445] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1446] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1447] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1448] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1449] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1450] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1451] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1452] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1453] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1454] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1455] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1456] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1457] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1458] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1459] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1460] The following is further disclosed regarding the above embodiment.

[1461] (Claim 1)

[1462] A game server has a means for generating a rhythm;

[1463] means for transmitting a notification signal to a user terminal according to a specific rhythm;

[1464] means for receiving tap timing from a user terminal;

[1465] means for assessing a user's score based on the timing of the received taps;

[1466] The system includes a means for notifying the user terminal of the evaluated score.

[1467] (Claim 2)

[1468] 2. The system of claim 1, wherein the game server randomly generates the rhythm.

[1469] (Claim 3)

[1470] a means for displaying a notification to the user on the user terminal to prompt the user to tap in accordance with a specific rhythm;

[1471] means for recording the timing of a user's taps;

[1472] 10. The system of claim 1, further comprising means for transmitting the recorded tap timings to a game server.

[1473] "Example 1"

[1474] (Claim 1)

[1475] A game server has a means for generating a rhythm;

[1476] means for transmitting a notification signal to a user terminal according to a specific rhythm;

[1477] means for receiving tap timing from a user terminal;

[1478] means for assessing a user's score based on the timing of the received taps;

[1479] means for notifying a user terminal of the evaluated score;

[1480] means for sequentially transmitting rhythmic notification signals;

[1481] means for comparing the timing of the received taps with a randomly generated rhythm to assess a score;

[1482] A system including a means for notifying a user terminal of the score results.

[1483] (Claim 2)

[1484] 2. The system of claim 1, wherein the game server randomly generates a specific rhythm.

[1485] (Claim 3)

[1486] a means for displaying a notification to the user on the user terminal to prompt the user to tap in accordance with a specific rhythm;

[1487] means for recording the timing of a user's taps;

[1488] means for transmitting the recorded tap timing to a game server;

[1489] 2. The system according to claim 1, further comprising means for displaying a message saying "Tap now" at a timing corresponding to the rhythm.

[1490] "Application Example 1"

[1491] (Claim 1)

[1492] A game server has a means for generating a rhythm;

[1493] means for transmitting a notification signal to a user terminal according to a specific rhythm;

[1494] means for receiving tap timing from a user terminal;

[1495] means for assessing a user's score based on the timing of the received taps;

[1496] means for notifying a user terminal of the evaluated score;

[1497] a means for requesting a user to perform an operation according to a rhythm during automatic driving of the vehicle and measuring the reaction speed and accuracy of the operation;

[1498] A means of assessing user attention and ensuring safety

[1499] A system including:

[1500] (Claim 2)

[1501] 2. The system of claim 1, wherein the game server randomly generates the rhythm.

[1502] (Claim 3)

[1503] a means for displaying a notification to the user on the user terminal to prompt the user to tap in accordance with a specific rhythm;

[1504] means for recording the timing of a user's taps;

[1505] 10. The system of claim 1, further comprising means for transmitting the recorded tap timings to a game server.

[1506] "Example 2: Combining Emotion Engines"

[1507] (Claim 1)

[1508] a means for generating a rhythm;

[1509] means for transmitting a notification signal to a user terminal according to a specific rhythm;

[1510] means for receiving tap timing from a user terminal;

[1511] means for assessing a user's score based on the timing of the received taps;

[1512] means for notifying a user terminal of the evaluated score;

[1513] means for recognizing the emotional state of a user;

[1514] The system includes a means for adaptively adjusting rhythmic difficulty based on a perceived emotional state.

[1515] (Claim 2)

[1516] 10. The system of claim 1, wherein the rhythm is generated randomly.

[1517] (Claim 3)

[1518] means for displaying a notification to the user prompting the user to tap according to a specific rhythm;

[1519] means for recording the timing of a user's taps;

[1520] means for transmitting the recorded tap timings to a server;

[1521] 10. The system of claim 1, further comprising means for analyzing the user's facial expressions and voice.

[1522] "Application example 2 when combining emotion engines"

[1523] (Claim 1)

[1524] A game server has a means for generating a rhythm;

[1525] means for transmitting a notification signal to a user terminal according to a specific rhythm;

[1526] means for receiving tap timing from a user terminal;

[1527] means for assessing a user's score based on the timing of the received taps;

[1528] means for notifying a user terminal of the evaluated score;

[1529] The system includes a means for adjusting the difficulty of the rhythm based on emotion analysis data.

[1530] (Claim 2)

[1531] 2. The system of claim 1, wherein the game server randomly generates the rhythm.

[1532] (Claim 3)

[1533] a means for displaying a notification to the user on the user terminal to prompt the user to tap in accordance with a specific rhythm;

[1534] means for recording the timing of a user's taps;

[1535] means for transmitting the recorded tap timing to a game server;

[1536] means for analyzing the emotional state of a user;

[1537] 10. The system of claim 1, further comprising means for transmitting the sentiment analysis data to the game server. [Explanation of symbols]

[1538] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A game server has a means for generating a rhythm; means for transmitting a notification signal to a user terminal according to a specific rhythm; means for receiving tap timing from a user terminal; means for assessing a user's score based on the timing of the received taps; The system includes a means for notifying the user terminal of the evaluated score.

2. 2. The system of claim 1, wherein the game server randomly generates the rhythm.

3. a means for displaying a notification to the user in the user terminal to prompt the user to tap in accordance with a specific rhythm; means for recording the timing of a user's taps; 10. The system of claim 1, further comprising means for transmitting the recorded tap timings to a game server.

Images