Generation method, generation system, control program, and game system

Abstract

To generate or display an instruction mark on the basis of an appropriate operation even when there is a gap between an operation in operation data to be a reference and a desired operation.SOLUTION: In a generation method generating, on the basis of operation data D1, instruction data D2 which is generated on the basis of movement data D1 of a reference dance movement going with music and is used in a game displaying instruction marks MP1, MP2, MT1, MT2 instructing a player about play movements, and is for displaying the instruction marks MP1, MP2, MT1, MT2 at an instruction timing when the play movement is performed, specific timing at which a specific movement of the reference dance movements is performed is compared with the instruction timing. When the specific timing differs from the instruction timing, the instruction data D2 is generated so that the instruction marks MP1, MP2, MT1, MT2 showing a specific movement at the specific timing are displayed at the compared instruction timing.SELECTED DRAWING: Figure 3

Description

Technical Field

[0004] ,

[0001] The present invention relates to a generation method for generating instruction data based on operation data, a generation system, a control program, and a game system.

Background Art

[0002] Patent Document 1 discloses a game device for a music game in which a ghost mode for performing a ghost display using input motion data recorded in a previous game play is provided. The game device records input motion data related to the game play every time a player plays the game. Further, the game device controls display of a display body showing the motion content of the input motion that the player performed as an operation input in a past game play using the recorded input motion data. Then, in a game play in which the ghost mode is selected and started, a ghost display is performed using the ghost data recorded for the play music of the game play. Then, a display body that reproduces the input timing related to each operation input during play and the motion content of the input motion is displayed.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In a dance game, which is an example of a music game, instructional markers can be created to guide the player's actions based on motion data of a human dancer's movements in time with the music. For example, instructional markers that mimic a dancer's pose can be created based on the coordinates of each part of the dancer's body in the motion data. However, because the dancer is human, it is difficult for them to perform the dance perfectly in time with the music. Therefore, the dancer may not be able to assume the correct pose at the time when the instructional marker should be displayed. In other words, there may be a discrepancy between the movements in the reference motion data and the desired movements. Therefore, the generation method disclosed below aims to generate or display instructional markers based on appropriate movements, even when there is a discrepancy between the movements in the reference motion data and the desired movements. [Means for solving the problem]

[0005] A generation method according to one embodiment is used in a game that generates motion data of a standard dance movement synchronized with music and displays an instruction sign to instruct the player to perform a play action, and generates instruction data for displaying the instruction sign at the instruction timing when the play action should be performed, based on the motion data, wherein the method compares a specific timing in which a specific movement among the standard dance movements is performed with the instruction timing, and if the specific timing is different from the instruction timing, generates the instruction data such that an instruction sign representing the specific movement at the specific timing is displayed at the compared instruction timing.

[0006] Furthermore, a generation system according to another embodiment is used in a game that generates motion data of standard dance movements synchronized with music and displays instruction signs to instruct the player to perform play movements, and is a generation system equipped with a computer that generates instruction data for displaying the instruction signs at the instruction timing when the play movements should be performed, the computer compares a specific timing in which a specific movement among the standard dance movements is performed with the instruction timing, and if the specific timing is different from the instruction timing, generates the instruction data such that an instruction sign representing the specific movement at the specific timing is displayed at the compared instruction timing.

[0007] Furthermore, a control program according to another embodiment is used in a game that generates motion data of a reference dance movement synchronized with music and displays an instruction sign to instruct the player to perform a play action, and generates instruction data for displaying the instruction sign at the instruction timing when the play action should be performed, based on the motion data, and is a control program for a generation system equipped with a computer, which causes the computer to compare a specific timing in which a specific movement among the reference dance movements is performed with the instruction timing, and if the specific timing is different from the instruction timing, causes the computer to generate the instruction data such that an instruction sign representing the specific movement at the specific timing is displayed at the compared instruction timing.

[0008] Furthermore, a game system according to another embodiment provides a game that is generated based on motion data of standard dance movements synchronized with music and displays instruction signs that instruct the player to perform play actions, and is a game system equipped with a computer, wherein the computer displays dance objects that move in time with the music based on the motion data, and displays the instruction signs at predetermined instruction timings, and if the specific timing at which the dance object performs a specific action among the standard dance movements differs from the instruction timing, the computer displays an instruction sign that represents the specific action at that instruction timing. [Effects of the Invention]

[0009] This allows for the generation or display of instruction signs based on appropriate actions, even when there is a discrepancy between the actions in the reference action data and the desired actions. [Brief explanation of the drawing]

[0010] [Figure 1] A schematic diagram of the game system. [Figure 2] A schematic diagram of an example game screen. [Figure 3] A block diagram outlining the game system. [Figure 4] A schematic diagram to explain the timing discrepancy. [Figure 5] A flowchart outlining the generation process. [Modes for carrying out the invention]

[0011] Hereinafter, exemplary embodiments for carrying out the present invention will be described in detail with reference to the drawings. However, the dimensions, materials, shapes, and relative positions of components described in the following embodiments can be arbitrarily set and modified according to the configuration of the apparatus or method to which the present invention is applied or various conditions. Furthermore, unless otherwise specified, the scope of the present invention is not limited to the embodiments specifically described below.

[0012] [First Embodiment] Figure 1 shows a game system 100 as an example of a generation system. The game system 100 provides a game that displays instruction signs to the player to guide them through gameplay. These instruction signs are generated based on motion data of a reference dance movement synchronized with music. For example, the reference dance movement is a standard dance movement that the player should reproduce, and can be a dance movement performed by a professional dancer in time with the music, or a dance movement performed by another player playing the game in time with the music. The game system 100 comprises a game server 10 and a game machine 60 that provides the game to the player. The game machine 60 is configured to connect to the game server 10 via a predetermined network 50.

[0013] The game server 10 is configured as a single logical server device by combining multiple server units, which are computers. However, the game server 10 may be configured as a single server unit. Alternatively, the game server 10 may be logically configured using cloud computing. Furthermore, the game server 10 may function in cooperation with external servers.

[0014] Furthermore, the game server 10 provides various services for game devices, including game services, to a game machine 60, which is an example of a game device. This service includes a distribution service that delivers and updates game programs or data to the game machine 60 via the network 50. Through this distribution service, the game server 10 appropriately delivers various programs or data necessary for providing games to each game machine 60.

[0015] The game service may include a service in which the game server 10 receives user identification information from the game machine 60 to identify the user and authenticates the user. The game service may also include a service in which the game server 10 receives and stores play data, including the play results of the authenticated user, from the game machine 60. Furthermore, the game service may include a service in which the game server 10 provides the stored play data to the game machine 60. In addition, the game service may include a service in which the game server 10 matches users when multiple users play the same game via the network 50. The game service may also include a service in which the game server 10 collects fees from users.

[0016] Furthermore, the game server 10 may provide web services to users of the game console 60 via the network 50. The web services include information provision services that provide various kinds of information, community services that provide a place for interaction such as information dissemination, exchange, and sharing by users, and services that assign user identification information (hereinafter also referred to as user ID) that uniquely identifies each user. In addition, the web services may include a service in which the game server 10 receives the user ID and authenticates the user.

[0017] Network 50 is configured to connect game server 10 and game console 60. For example, network 50 is configured to achieve network communication using the TCP / IP protocol. Specifically, a local area network LAN connects game server 10 and the internet 51. The internet 51, which acts as a WAN, and the local area network LAN are connected via router 53. Game console 60 is also configured to connect to the internet 51. Alternatively, a local server may be installed between game console 60 and the router 53 of the store or other location where game console 60 is installed. Game console 60 can communicate with game server 10 via this local server.

[0018] Further, the game server 10 and the game machine 60 may be interconnected by the Internet 51 instead of or in addition to the local area network LAN. Note that the network 50 may be any of a dedicated line, a telephone line, a corporate network, a mobile communication network, other communication lines, and combinations thereof, regardless of whether it is wired or wireless.

[0019] The game machine 60 is a commercial (or business-use) game device that allows a user to play a game within a range corresponding to a consideration in exchange for payment of a fee that is a predetermined consideration. This game machine 60 may be called an arcade game machine. Also, the game machine 60 is a computer device and is installed in a store or the like mainly for the purpose of repeatedly allowing a large number of users to play a game to increase profits. As an example, the game machine 60 provides a music game in which a player performs a dance motion in accordance with the music of the game.

[0020] [Overview of the Game] Next, referring to FIGS. 2 and 3, an overview of the music game provided by the game machine 60 will be described. FIG. 2 is a schematic diagram showing the game screen GS, and FIG. 3 is a schematic block diagram of the game system 100. As shown in FIG. 3, the game machine 60 includes a display device 64 on which an instruction sign and a dance object DO are displayed. The display device 64 is, for example, a display such as a liquid crystal display, an organic EL display, and an inorganic EL display, or a touch panel.

[0021] Furthermore, the display device 64 may be a light-emitting device (for example, a light-emitting diode) arranged in a matrix, or a projector that projects a game screen onto a part of the housing. Also, the game machine 60 includes an audio output device 65 that outputs game music, sound effects, etc. reproduced during the game. This audio output device 65 is, for example, a speaker. The display device 64 and the audio output device 65 are both provided on the housing of the game machine 60, but may be separate from the housing respectively.

[0022] As shown in Figure 2, the game provided by the game machine 60 displays multiple types of instruction signs. For example, the instruction signs include pose markers MP1 and MP2, and touch markers MT1 and MT2. The pose markers MP1 and MP2 have a shape that mimics the silhouette of a dancer. The pose markers MP1 and MP2 represent the dancer's pose or dance movement, and represent the play action that the player should perform. The touch markers MT1 and MT2 represent the positions where the dancer's arms and legs have moved, and indicate the positions where the player should move their arms and legs. Furthermore, the instruction signs include trace markers (not shown) that represent the trajectory of the dancer's arm and leg movements, and indicate the trajectory where the player should move their arms and legs.

[0023] Additionally, a timeline TL is displayed at the top of the game screen GS. The pose marker MP1 is displayed on the timeline TL. This pose marker MP1 moves along the timeline TL, repeating predetermined actions to instruct the player on how to move their body. In the example in Figure 2, the pose marker MP1 moves from the right edge of the game screen GS to the left. The player then performs the dance action instructed by the pose marker MP1 when the pose marker MP1 reaches the area enclosed by the judgment frame FL on the timeline TL.

[0024] Additionally, the pose marker MP2 is displayed at the bottom of the timeline TL. This pose marker MP2 is represented as an image that mimics a fixed pose to instruct the player to strike a signature pose. Here, a signature pose is a pose performed at a timing corresponding to a characteristic part of the music, and one example is an action that temporarily stops body movement. One of the two pose marker MP2s moves from the right edge of the game screen GS towards the center of the game screen GS, and the other moves from the left edge of the game screen GS towards the center of the game screen GS. The player takes the pose indicated by the pose marker MP2 when the pose marker MP2 reaches the center of the game screen GS.

[0025] Note that there may be one or more pause marker MP2s. Furthermore, the initial display position and movement direction of the pause marker MP2s are arbitrary. For example, a pause marker MP2 may move from the top or bottom edge of the game screen GS toward the center of the game screen GS. Alternatively, a pause marker MP2 may be displayed in the center of the game screen GS and move toward the top or bottom edge of the game screen GS. Or, a pause marker MP2 may be displayed in the center of the game screen GS and move toward the right or left edge of the game screen GS.

[0026] The indicator markers represent specific actions. For example, a specific action is the endpoint of a movement, the starting point of a movement, or a series of movements including the endpoint or starting point. For example, in the case of a right foot step, the specific action is the movement from when the raised right foot is brought down, or the series of movements from when the right foot is raised until it is brought down. A specific action may also be a final pose, or a series of movements including a final pose. For example, a final pose is a posture that stops a movement, such as a keep or freeze, or a posture with the arms or legs extended. For example, pose markers MP1 and MP2 have shapes that mimic a dancer performing a specific action, thereby representing the specific action. Touch markers MT1 and MT2 indicate the position of the hands or feet when performing a specific action, thereby representing the specific action. Furthermore, trace markers (not shown) indicate the trajectory of the hand or foot movement when performing a specific action, thereby representing the specific action.

[0027] Below the timeline TL, the dance object DO is displayed. Touch markers MT1 and MT2 are displayed around the dance object DO. Specifically, touch marker MT2 indicates where the player should move their hands and is displayed near the hands of the dance object DO. Touch marker MT1 indicates where the player should move their feet and is displayed near the feet (especially at the feet) of the dance object DO. In Figure 2, the main dancer is shown as a single dance object DO. However, the dance object DO may include backup dancers in addition to the main dancer. Furthermore, the dance object DO may include moving items or objects such as animals that move in time with the music.

[0028] In the music game provided by game system 100, play actions are instructed by instruction markers at predetermined timings (hereinafter also referred to as "instruction timings") based on instruction data D2. For example, instruction timings correspond to the timing of each beat in each measure of the music (for example, the downbeat for the first beat). The instruction timing and the timing of the player's play action are then compared. As a result, if the discrepancy between the instruction timing and the timing of the play action is small, the player is evaluated as having performed an appropriate play action and receives a high score. Conversely, if the discrepancy between the instruction timing and the timing of the play action is large, the player is evaluated as having failed to perform the play action and receives a low score or no score at all.

[0029] In addition to large discrepancies, a low evaluation result will also be given if the player fails to perform the action at the indicated timing. Furthermore, the actions that are evaluated are not limited to those indicated by the instruction signs.

[0030] Above the timeline TL, the viewer gauge VG is displayed. The viewer gauge VG increases when the player performs the appropriate play action at the timing indicated by the instruction signs. On the other hand, if the player fails to perform the play action, the viewer gauge VG will not change or will shrink. When the viewer gauge VG reaches the passing line, the game stage is cleared. Furthermore, above the timeline TL, to the right of the viewer gauge VG in Figure 2, the score SC is displayed. The score SC represents the player's score and increases when the player performs the appropriate play action at the timing indicated by the instruction signs. On the other hand, if the player fails to perform the play action, the score SC will not increase or will decrease.

[0031] Furthermore, the player's performance may be represented by a circle or other shape instead of the viewer gauge VG. Also, the player's performance may be represented by letters (e.g., A, B, C, D) instead of the score SC. In addition, the image representing the player's performance (viewer gauge VG and score SC in the example of Figure 2) may be displayed at any position on the game screen GS. Furthermore, the image representing the player's performance may be displayed on the results screen after the game has ended.

[0032] Additionally, the combo count (CN) is displayed at the bottom of the game screen (GS). The combo count (CN) increases when the player performs the correct dance moves consecutively. If the player fails a dance move, the combo count (CN) resets to zero or decreases.

[0033] Furthermore, the indicator signs may change in various ways over time. For example, changes in the indicator signs may include a gradual decrease in area, a gradual change in color, a gradual increase or decrease in density, a gradual increase or decrease in brightness, and a gradual increase or decrease in the number of objects. In addition, the indicator signs may include letters, figures, patterns, numbers, and symbols, as well as combinations thereof.

[0034] [Game System Configuration] Referring to Figure 3, the control system of the game system 100 will be described. The game server 10 comprises a server control unit 11 that controls the game server 10, and a server storage unit 12, which is an example of a storage means that stores the control program PG1 of the game server 10. The server control unit 11 is configured as a computer that combines a processor that performs various calculations and operation controls according to a predetermined program with other peripheral devices.

[0035] As an example, the processor of the server control unit 11 is a CPU (Central Processing Unit) or MPU (Micro-Processing Unit), and it controls the entire game server 10 based on the program stored in the server memory unit 12, and also comprehensively controls various processes. The server memory unit 12 also includes RAM (Random Access Memory), which is the system work memory for the processor to operate, and storage devices such as ROM (Read Only Memory), HDD (Hard Disk Drive), and SSD (Solid State Drive) that store programs and system software. In the following, we will explain using as an example the case in which the CPU performs various calculations, control, and discrimination operations according to the control program stored in the ROM or HDD.

[0036] The server control unit 11 is connected via wired or wireless connection to an interface (not shown) including a keyboard or various switches for inputting predetermined commands and data. The server control unit 11 is also connected via wired or wireless connection to a monitor (not shown) that displays the device's input status, settings, measurement results, and various other information. Furthermore, the server control unit 11 can perform control according to programs stored on portable recording media such as CDs (Compact Discs), DVDs (Digital Versatile Discs), CF (Compact Flash) cards, and USB (Universal Serial Bus) memory, or external storage media such as servers on the Internet.

[0037] The server storage unit 12 is a storage device such as a hard disk or semiconductor memory device, which includes a non-volatile storage medium, an example of a computer-readable non-temporary storage medium. The server storage unit 12 may store all data on a single storage unit, or it may distribute and store data across multiple storage units. The server storage unit 12 stores a control program PG1 as an example of a computer program that causes the server control unit 11 to execute various necessary processes. The server control unit 11 generates instruction data D2 for displaying instruction indicators by executing the control program PG1. The server storage unit 12 also stores a game program (not shown). The server control unit 11 then provides game services to a player playing the game machine 60 by executing the game program.

[0038] Furthermore, the server storage unit 12 stores motion data D1 of a reference dance movement synchronized with the music. For example, motion data D1 is generated based on a dance video of a dancer or player performing a reference dance movement. That is, motion data D1 of the dancer or player is obtained from the analysis results of the dance video. This motion data D1 is information describing the movements from the start to the end of the dance. As an example, motion data D1 is bone information obtained by bone analysis. Alternatively, motion data D1 may be data indicating the position, orientation, and displacement of each part of the dancer's or player's body. In the following explanation, an example using motion data D1 of a dancer's reference dance movement will be particularly described.

[0039] Here, bone information refers to information indicating bones that connect points representing the position coordinates of parts of the dancer's body, and in bone information, bones are connected to each other by connection points. Based on the position, orientation, and displacement of the dancer's bones in the dance video, the dancer's reference dance movements can be recorded as motion data D1. The bone information represents the dancer's bones at each point in time as time progresses from the start of the dance. However, the bone information may include some information that predicts the position, orientation, and displacement of the bones. The server control unit 11 may also obtain motion data D1 by analyzing a video captured by the game machine 60's camera 63 or a video acquired from an external recording device. Alternatively, bone information may be created by analyzing a dance video obtained by filming dance movements in advance, and the operator may store this bone information in the server storage unit 12.

[0040] Furthermore, the server memory unit 12 stores game data for the music game. The game data includes instruction data D2 and music data D3. The music data D3 consists of each song played during the game, but may also include data for sound effects and sound effects. The instruction data D2 is data that defines the operation procedure of the game, and includes data such as the display timing, display position, and movement speed of instruction signs, and the type of instruction sign. In addition, the game data stores image data. The image data includes data for images such as instruction signs, effect images, icons, various objects, and backgrounds.

[0041] [Game console configuration] As shown in Figure 3, the game console 60 comprises a game console control unit 61 that controls the game console 60, and a game console memory unit 62 that stores the game console program PG2. The game console control unit 61 is a computer that combines a processor that performs various calculations and operation controls according to a predetermined program with other peripheral devices. The processor is, for example, a CPU or MPU, and controls the entire game console 60 based on the game console program PG2, as well as comprehensively controlling various processes. The game console memory unit 62 includes RAM, which is a system work memory for the processor to operate, and storage devices such as ROM, HDD, and SSD that store programs and system software. The game console control unit 61 can also be controlled according to a program stored on a portable recording medium such as a CD, DVD, CF card, and USB memory, or an external storage medium such as a server on the Internet.

[0042] The game machine storage unit 62 is a storage device such as a hard disk and a semiconductor storage device, which includes a non-volatile storage medium, an example of a computer-readable non-temporary storage medium. The game machine storage unit 62 also records game data acquired from the game server 10 along with the game machine program PG2. The game data is used to allow the user to play a predetermined game according to the game machine program PG2. The game data may also include image data, music data D3, and instruction data D2, similar to the game data stored in the server storage unit 12. Furthermore, the game data may also include data for dance objects DO.

[0043] Furthermore, the game machine 60 has a camera 63 as an example of a detection device that detects the player's movements during gameplay. The camera 63 is, for example, a camera and may consist of multiple cameras. The cameras are positioned so that the player's entire body is included in the shooting range. The camera 63 may also capture the player's movements and store them as gameplay video in the game machine's storage unit 62. In addition, the game machine 60 is equipped with a display device 64 that displays the game screen. The game machine 60 is also equipped with an audio output device 65 that outputs sounds such as music, judgment sounds, evaluation sounds, and sound effects that are output during gameplay.

[0044] The game machine control unit 61 analyzes the gameplay video captured by the camera 63 to obtain player motion information (e.g., bone information). Alternatively, the camera 63 may be a camera that captures markers worn on the player's body. In this case, the game machine control unit 61 analyzes the video showing the markers to obtain motion information. Alternatively, the detection device may include sensors that detect the player's body movements. In this case, the detection device may consist of inertial sensors such as a gyro sensor and an accelerometer that detect the player's body movements, and a recording device that records the detected movements. Furthermore, the detection device may consist of a magnetic or mechanical sensor that detects the player's body movements, and a recording device that records the detected movements.

[0045] Furthermore, the game machine control unit 61 causes the player to play the game based on the game machine program PG2. The game machine control unit 61 then performs various calculations and controls necessary for the progress of the game in order to control the game's progression. For example, the game machine control unit 61 displays the instruction timing and dance movements on the display device 64 based on the position and movement speed of the pause marker MP1 moving toward the judgment frame FL. In addition, the game machine control unit 61 has a display control unit 66 as a logical device, which is an example of a display control means. This logical device is realized by a combination of the hardware resources of the game machine control unit 61 and the game machine program PG2 as a software resource.

[0046] [Display control means] The game system 100 provides a game that displays instruction signs, which are generated based on the operation data D1, to instruct the player to perform a play action. To this end, the display control unit 66 displays a dance object DO that moves in time with the music based on the operation data D1. Furthermore, the display control unit 66 displays instruction signs at predetermined instruction timings based on instruction data D2. For example, the instruction signs are pause markers MP1 and MP2, and touch markers MT1 and MT2. In this way, the display control unit 66 instructs the player on the timing to perform a play action. The player may dance by imitating the movements of the dance object DO, or they may dance according to the instructions of the instruction signs.

[0047] As an example, the display control unit 66 generates a game screen GS in which the dance object DO moves according to the motion data D1, using image data of the dance object DO that has been previously acquired from the game server 10. At this time, the display control unit 66 may generate a game screen GS in which the dance object DO moves, as well as the dance object DOs that act as backup dancers. The game screen GS also includes images or videos such as instruction signs, backgrounds, and performance images. Alternatively, the display control unit 66 may have previously acquired video data of the dance object DO that moves to mimic the movements of a dancer performing a standard dance from the game server 10.

[0048] As an example of how the dance object DO is displayed, the dance object DO has bones set up, and the display control unit 66 changes the relative position and orientation of adjacent bones of the dance object DO according to the dancer's motion data D1 that has been previously obtained from the game server 10. As a result, the external shape of the dance object DO also changes. The appearance of the dance object DO is then represented by a texture that changes in accordance with the changes in the position and orientation of the bones. On the game screen GS, the dancer's reference dance movements are reflected in the movement of the dance object DO. For example, if the arms move up and down in the reference dance movement, the arms of the dance object DO also move up and down. The player moves their body to perform the dance movements indicated by the instruction signs while looking at the dance object DO displayed on the display device 64.

[0049] As an example of how instruction markers are displayed, the display control unit 66 displays pause markers MP1 and MP2 as shown in Figure 2. Specifically, the display control unit 66 reads the instruction timings included in the instruction data D2 that has been previously acquired from the game server 10 and stored in the game machine storage unit 62. The display control unit 66 then displays and moves pause marker MP1 so that it reaches the judgment frame FL and is displayed at the instruction timing. The display control unit 66 also displays and moves pause marker MP2 so that it reaches the center of the screen or the dance object DO and is displayed at the instruction timing.

[0050] Furthermore, as an example of the display mode of the instruction sign, the display control unit 66 causes the touch markers MT1 and MT2 to be displayed on the display device 64. Specifically, the display control unit 66 reads the instruction timing included in the instruction data D2. Then, at or immediately before the instruction timing, it causes the touch markers MT1 and MT2 to be displayed at the position indicated in the instruction data D2. In addition, the game machine control unit 61 may cause the audio output device 65 to output a sound indicating the instruction timing. The game machine control unit 61 may also indicate the instruction timing by turning on or off a light-emitting device provided on the game machine 60. Furthermore, the game machine control unit 61 may also indicate the instruction timing by vibrating a vibration device provided on the game machine 60.

[0051] The display control unit 66 displays an indicator sign representing a specific action at the indicated timing if the specific timing at which the dance object DO performs a specific action from the reference dance movements differs from the indicated timing. To explain this, first, the reference dance movements performed by the dancer include a specific action. This specific action is also reflected in the movement data D1. The specific timing at which this specific action is performed may differ from the indicated timing indicated by the indicator sign. This is because the dancer is a human and may not be able to perform the specific action at a timing that perfectly matches the indicated timing. As a result, the specific timing at which the dance object DO, which operates based on the movement data D1, performs the specific action may differ from the indicated timing.

[0052] In this case, the display control unit 66 displays an instruction sign that represents a specific action at the instruction timing. That is, instead of displaying an instruction sign at a specific timing, the display control unit 66 displays an instruction sign at the instruction timing. For example, in the judgment frame FL, the display control unit 66 moves the pose marker MP1 so that it is displayed at the instruction timing. As a result, at the timing when the pose marker MP1 reaches the judgment frame FL (i.e., the instruction timing), the pose of the dance object DO and the pose represented by the pose marker MP1 are different.

[0053] This allows the player to refer to instructional markers while using the dance object DO as a model. Therefore, they can perform specific actions at appropriate timings that differ from the specific actions of the dance object DO, thereby improving their score. Alternatively, the player may perform actions according to the dance object DO. In this case, the player can reproduce the standard dance actions performed by the dancer. The standard dance actions performed by the dancer are their own unique expression. Therefore, for the player, reproducing the standard dance actions allows them to perform a more attractive dance, even if it results in a lower score. Furthermore, for the model dancer, it allows them to perform the standard dance actions without being subject to strict timing constraints. Therefore, they can perform more attractive, for example, more dynamic, standard dance actions. By utilizing the action data D1 of such standard dance actions, the dance object DO that performs more attractive dance actions is displayed.

[0054] The display control unit 66 displays an instruction sign according to instruction data D2, which is written to display an instruction sign representing a specific action at the instruction timing. This ensures that the instruction sign is displayed at the instruction timing even if the specific timing differs from the instruction timing. Alternatively, the display control unit 66 may determine whether the specific timing at which the dance object DO performs the specific action differs from the instruction timing. In this case, if the display control unit 66 determines that the two timings are different, it displays an instruction sign representing the specific action at the instruction timing.

[0055] The display control unit 66 may display multiple play options that the player can select as the play target. Examples of play options include game stages, music, or difficulty levels. These play options can be identified by target identification information that uniquely identifies the play target selected by the player. The target identification information may also be identification information that identifies the stage, music, or difficulty level.

[0056] [Evaluation methods] The game machine control unit 61 also functions as an evaluation tool, evaluating the timing of play actions in relation to the instruction timing. Specifically, the game machine control unit 61 analyzes the gameplay video captured by the camera 63 to obtain player action information. Based on the analysis results, the game machine control unit 61 evaluates the timing of the player's play actions. For example, the game machine control unit 61 compares the timing at which the player's hand reaches the touch marker MT2, or the timing at which it is predicted to reach it, with the instruction timing. Based on the comparison results, the game machine control unit 61 evaluates the play action. Similarly, the game machine control unit 61 evaluates the timing at which the player's foot reaches the touch marker MT1.

[0057] Furthermore, the game machine control unit 61 compares the timing of the player's actions with the instruction timing when the pause marker MP1 reaches the judgment frame FL. Based on the comparison result, the game machine control unit 61 evaluates the actions. Alternatively, the game machine control unit 61 evaluates the actions by comparing the timing of the actions performed with the instruction timing by the pause marker MP2. The game machine control unit 61 can also evaluate whether the player's actions are appropriate by comparing the player's action information with the action data D1.

[0058] The game machine control unit 61 stores a high evaluation in the game machine memory unit 62 if the operation timing is close to the instructed timing. On the other hand, the game machine control unit 61 stores a low evaluation in the game machine memory unit 62 if the operation timing is far from the instructed timing. Furthermore, the game machine control unit 61 may display an image or text (for example, the words "Perfect," "Good," or "Bad") indicating the evaluation result on the display device 64. The game machine control unit 61 may also output audio indicating the evaluation result from the audio output device 65. Furthermore, the game machine control unit 61 may indicate the evaluation result by turning on or off a light-emitting device provided on the game machine 60. The game machine control unit 61 may also indicate the evaluation result by vibrating a vibration device provided on the game machine 60.

[0059] Furthermore, if the game machine control unit 61 is highly rated in terms of gameplay, it evaluates the game in a way that improves the user's score and adds a high score to the user's score. If the game machine control unit 61 is highly rated in terms of gameplay, it either does not add any points to the user's score or adds a low score. In addition, if the game machine control unit 61 is highly rated in terms of gameplay, it may also deduct points from the user's score. Finally, the game machine control unit 61 displays the score SC as the game score on the display device 64.

[0060] [Server control means] As shown in Figure 3, the server control unit 11 includes a game control unit 13 as an example of game control means, a comparison unit 14 as an example of comparison means, and a generation unit 15 as an example of generation means. The game control unit 13 is a logical device and is realized by a combination of the hardware resources of the server control unit 11 and a game program (not shown) as a software resource. The comparison unit 14 and the generation unit 15 are also logical devices and are realized by a combination of the hardware resources of the server control unit 11 and a control program PG1 as a software resource.

[0061] The game control unit 13 is a logical device for providing game services. The game control unit 13 transmits instruction data D2 and data necessary for providing the game, such as game data, to the game machine 60. Furthermore, the game control unit 13 obtains user data from the game machine 60, including the user's personal information (e.g., user ID) and information related to game play, such as game scores. The game control unit 13 then stores the user data in the server storage unit 12. The game control unit 13 may also perform user authentication.

[0062] The game system 100, as a generation system, generates instruction data D2 used in the game, which is instruction data D2 for displaying instruction indicators at instruction timings when play actions should be performed. That is, the game system 100 generates at least a portion of the instruction data D2 based on the action data D1. To this end, the server control unit 11 has a comparison unit 14 and a generation unit 15.

[0063] Alternatively, the comparison unit 14 and the generation unit 15 may be located in another computer device, such as an external server, as another example of the generation system. Furthermore, the other computer device may be a stationary or book-type personal computer, or a mobile terminal device such as a mobile phone (including a smartphone). In this case, the game control unit 13 acquires instruction data D2 from the other computer device and stores it in the server storage unit 12. The storage unit of the other computer device stores operation data D1, music data D3, and control program PG1.

[0064] [Means of comparison] The comparison unit 14 compares the specific timing at which a particular movement in the reference dance movement is performed with the instructed timing. As described above, there may be a time axis shift in the movement data D1 between the specific timing and the instructed timing. For example, as shown in Figure 4, a game is described in which a specific movement is performed simultaneously with the first, second, third, and fourth beats of a measure of four-beat music. In this example, in the reference dance movement, i.e., the dancer's reference dance movement, the right foot step on the first beat is performed at the appropriate timing. In this example, the appropriate timing is the timing simultaneous with each beat, and the instructed timing corresponds to that timing.

[0065] However, in the standard dance sequence, the left foot step on the second beat is performed earlier than the correct timing. Also, the right foot step on the third beat is performed later than the correct timing. Furthermore, the final pose on the fourth beat is performed later than the correct timing. Therefore, the specific movements from the second to the fourth beat are out of sync with the correct timing.

[0066] In instruction data D2, we want to display instruction markers representing specific actions at the same time as the second through fourth beats. However, in the case of the fourth beat, the dancer's reference dance movement is delayed by the length of one eighth note (240 ticks) in striking the final pose. Therefore, if we generate instruction markers, such as pose markers MP1 and MP2, using action data D1 based on the reference dance movement at the instruction timing, they will appear as if the dancer is in the process of striking the final pose. On the other hand, if we generate instruction markers according to the specific timing of action data D1 based on the reference dance movement, they will be out of sync with the music's progression, resulting in instructions that feel unnatural to the player.

[0067] Therefore, as a preprocessing step to eliminate the discrepancy, the comparison unit 14 compares the specific timing with the instructed timing to determine whether the specific timing is different from the instructed timing. Specifically, the comparison unit 14 determines whether there is a discrepancy on the time axis of the operation data D1 between the two timings. Here, the comparison unit 14 determines that the specific timing is different from the instructed timing if there is a difference between the two timings that exceeds a predetermined range (for example, 60 ticks). The comparison unit 14 then calculates the length of the discrepancy on the time axis of the operation data D1 as the amount of discrepancy. Note that the time axis of the operation data D1 coincides with the time axis of the music data D3 (i.e., the data for each song), and the length of time from the start to the end of both data is the same.

[0068] For example, the comparison unit 14 calculates the number of ticks corresponding to the length of the discrepancy as the amount of discrepancy. In this case, the comparison unit 14 determines that there is no difference between the two timings if the difference between the specific timing and the instructed timing is within a predetermined range. For example, the comparison unit 14 determines that the difference between the two timings is within a predetermined range if the two timings match, or if the difference between the two timings is small (e.g., within 60 ticks). Note that the time axis from the start to the end of the music matches the time axis from the start to the end of the operation data D1. Therefore, the timing of each beat on the time axis of the operation data D1 matches the timing of each beat on the music data D3. Alternatively, the comparison between the specific timing and the instructed timing and the calculation of the discrepancy amount may be performed visually by the operator, and the comparison results may be stored in the server storage unit 12.

[0069] In this specification, the tick count is the length on the time axis of the motion data D1, where the length of a quarter note is 480 ticks. Therefore, 1 tick is 1 / 120th the length of one sixteenth note, and 240 ticks is the length of one eighth note. In the example in Figure 4, on the second beat, the dancer's reference dance movement shows the left foot step being 120 ticks earlier. On the third beat, the dancer's reference dance movement shows the right foot step being 120 ticks later.

[0070] [Generation means] The generation unit 15 generates instruction data D2 such that, if a specific timing differs from the instructed timing, an instruction indicator representing a specific operation at the specific timing is displayed at the compared instructed timing. For example, the comparison unit 14 determines that the two timings are different if the specific timing differs from the instructed timing by more than a predetermined range. In this case, the generation unit 15 refers to the amount of difference between the two timings and identifies the specific operation that will serve as the basis for the instruction indicator in the operation data D1. For example, it identifies an operation that is earlier or later by a length corresponding to the amount of difference as the specific operation that will serve as the basis for the instruction indicator.

[0071] The generation unit 15 then creates instruction markers based on specific actions. For example, the generation unit 15 changes the relative position and orientation of adjacent bones of pose markers MP1 and MP2 based on bone information corresponding to the specific action. This also changes the external shape of pose markers MP1 and MP2. The generation unit 15 then applies a texture representing the silhouette of the dancer to the appearance of pose markers MP1 and MP2. This ensures that the specific action is reflected in the movement of pose markers MP1 and MP2.

[0072] Furthermore, the generation unit 15 generates instruction data D2 so that the created instruction marker is displayed at the desired position at the instruction timing. This allows the instruction marker to be created based on the motion timing that provides the best visual appearance of the motion in the motion data D1. Alternatively, the operator may create the instruction marker and generate instruction data D2 so that it is displayed at the desired position at the instruction timing. In this case, the operator identifies a specific motion that is either earlier or later by a length corresponding to the amount of deviation as the specific motion that will serve as the basis for the instruction marker. The operator then uses an editing program that generates pose markers MP1 and MP2 from bone information to create the instruction marker based on the specific motion.

[0073] In the example in Figure 4, on the fourth beat, the dancer's reference dance movement is 240 ticks behind the instructed timing of the fourth beat. Therefore, the generation unit 15 identifies the movement in the movement data D1 that is 240 ticks behind the instructed timing. The generation unit 15 then creates an instruction marker based on the identified movement. Furthermore, the generation unit 15 generates instruction data D2 so that the created instruction marker is displayed at the desired position (e.g., judgment frame FL) at the instructed timing of the fourth beat. Similarly, for the second beat, the generation unit 15 identifies the movement in the movement data D1 that is 120 ticks earlier than the second beat. Also, for the third beat, the generation unit 15 identifies the movement in the movement data D1 that is 120 ticks later than the third beat.

[0074] This allows for the generation of instruction markers based on appropriate movements, even when there is a discrepancy between the movements in the reference movement data D1 and the desired movements. Furthermore, even when the movement is still in progress at the instruction timing in the movement data D1 based on the reference dance movements, instruction markers can be generated based on appropriate movements. For example, instruction markers that mimic the silhouette of a dancer can be made into the appropriate shape. Also, instruction markers that specify the position of a part of the dancer's body can be displayed in the appropriate position. In addition, instruction markers that trace a trajectory following the dancer's movements can be displayed along the appropriate trajectory.

[0075] Furthermore, if the difference between the specific timing and the instructed timing is within a predetermined range, the generation unit 15 generates instruction data D2 such that an instruction indicator representing a reference dance movement at the instructed timing is displayed at the instructed timing. That is, if the specific timing and the instructed timing coincide, or if the difference between the two timings is small, the generation unit 15 generates instruction data D2 such that an instruction indicator representing a specific movement at the instructed timing on the time axis of the motion data D1 is displayed at the instructed timing. For example, if the difference between the two timings is less than 60 ticks, it is considered to be within a predetermined range. This allows the generation unit 15 to generate an instruction indicator based on a visually appealing movement at the instructed timing in the motion data D1.

[0076] Furthermore, the generation unit 15 uses the difference between a specific timing and an instructed timing on the time axis of the operation data D1 as a correction value. The generation unit 15 then either adds the correction value to the specific timing or subtracts the correction value from the specific timing. In this way, the generation unit 15 sets the operation timing of the specific operation to match the instructed timing and generates the instructed data D2.

[0077] For example, in the example shown in Figure 4, the generation unit 15 subtracts 120 Tick1 as a correction value from the specific timing of the second beat. That is, the generation unit 15 sets -120 Tick as the offset Tick number for the specific timing of the second beat. As a result, the generation unit 15 sets the timing of the left foot step, which is the specific action at the specific timing, to match the instruction timing that coincides with the second beat. The generation unit 15 then creates an instruction marker that represents the specific action at that timing. As a result, the generation unit 15 generates instruction data D2 so that the instruction marker representing the specific action at the previously misaligned specific timing is displayed at the instruction timing.

[0078] As a result, the generation unit 15 can create an instruction mark for the instructed timing based on a specific action at a specific timing that is 120 ticks earlier than the instructed timing. Similarly, for the third beat, the generation unit 15 creates an instruction mark based on a specific action (i.e., a right foot step) at a specific timing that is 120 ticks later than the instructed timing. This allows for the objective identification of the amount of deviation and enables the creation of an instruction mark based on a specific action at an accurate specific timing. Note that the correction value is not limited to the number of ticks, but may also be the number of notes, the number of beats, or time such as seconds. Furthermore, since only the timing of the specific action is shifted, the timing of other actions (for example, the timing of the specific action where no deviation occurs) is used as is for creating the instruction mark.

[0079] Furthermore, the generation unit 15 extracts musical feature points from the music data D3 (i.e., data for each song). The generation unit 15 then identifies a first timing corresponding to the musical feature point as the instruction timing. Furthermore, the generation unit 15 generates instruction data D2 based on the identified instruction timing. In other words, the generation unit 15 analyzes the music data D3 to extract musical feature points, uses the identified timing as the instruction timing, and compares it with a specific timing of a specific action. The generation unit 15 then generates instruction data D2 using the comparison result. In this way, the generation unit 15 can automatically identify the instruction timing. As an example, the musical feature point is the downbeat, but it may be another timing, such as the timing when the musical style changes.

[0080] Furthermore, the generation unit 15 extracts motion feature points from the motion data D1. Then, if the second timing corresponding to the motion feature point is earlier or later than the first timing, the generation unit 15 identifies the second timing as a specific timing. Furthermore, the generation unit 15 generates instruction data D2 based on the identified specific timing. In other words, the generation unit 15 analyzes the motion data D1 to extract motion feature points, identifies the timing of the motion feature point as a specific timing, and compares it with the instruction timing. Then, the generation unit 15 generates instruction data D2 using the comparison result. In this way, the generation unit 15 can automatically identify specific actions and specific timings. As an example, a motion feature point is an action that stops movement, and is either the start point or the end point of movement.

[0081] In this way, by automatically identifying the instruction timing, specific actions, and specific timings, the generation unit 15 can generate instruction data D2 from music data D3 and action data D1 in real time. For example, when the reference dance movements of a dancer or another player are filmed, the generation unit 15 can generate instruction data D2 at the same time as filming. Furthermore, by transmitting the generated instruction data D2 to the game machine 60, instruction indicators can be displayed on the game machine 60 at the same time as the dancer or other player's dance. This allows the player to play the game as if dancing along with the dancer or other player.

[0082] Furthermore, the generation unit 15 generates a dance object DO that operates in time with the music based on the operation data D1. This dance object DO is then included in the instruction data D2 and transmitted to the game machine 60, where it is displayed by the display control unit 66 along with an instruction sign. Since this dance object DO is generated based on the operation data D1, the specific operation of the instruction sign differs from the operation of the dance object DO at the instruction timing. This allows the player to use the dance object DO, which demonstrates the dancer's standard dance movements, as a model, while also dancing at appropriate instruction timings that differ from the standard dance movements, following the instructions of the instruction sign.

[0083] As an example, a dance object DO has bones set up, and the generation unit 15 generates the dance object DO according to the motion data D1, changing the relative position and orientation of adjacent bones of the dance object DO. This also changes the external shape of the dance object DO. Here, the appearance of the dance object DO is represented by a texture that changes in accordance with the changes in the position and orientation of the bones. The display control unit 66 can change the texture of the dance object DO. Therefore, the display control unit 66 can display multiple types of dance object DO. Alternatively, the display control unit 66 may generate the dance object DO instead of the generation unit 15.

[0084] The generation unit 15 generates instruction markers as dance images that mimic at least a part of the body of a dancer performing a reference dance movement. For example, the dance images are pose markers MP1 and MP2, which are silhouettes of the dancer's body. Alternatively, the dance images may mimic the dancer's hands or feet. For example, dance images mimicking hands or feet are used as touch markers MT1 and MT2.

[0085] Furthermore, the display control unit 66 operates the dance object DO to mimic the movements of a dancer performing a reference dance. Specifically, the generation unit 15 generates the dance object DO according to the motion data D1, changing the relative position and orientation of adjacent bones of the dance object DO, and includes this in the instruction data D2. The display control unit 66 then operates the generated dance object DO according to the instruction data D2.

[0086] [Generation Process Flow] Next, the process of generating instruction data D2 will be explained with reference to the flowchart shown in Figure 5. As a preprocessing step, motion data D1 is generated based on a dance video in which the dancer's or player's reference dance movements are filmed. Here, the generation of motion data D1 may be performed by the server control unit 11 or by another computer device operated by the operator. The server storage unit 12 then stores the generated motion data D1. The game server 10 then acquires the generated motion data D1 (S101). That is, the game server 10 acquires the motion data D1 generated by the server control unit 11 or receives and acquires the motion data D1 from another computer device.

[0087] The comparison unit 14 of the server control unit 11 compares a specific timing at which a specific operation among the reference dance operations is performed with the instruction timing (S102). If the timings are not different, that is, if the difference between the specific timing and the instruction timing is within a predetermined range (NO in S103), the generation unit 15 creates an instruction indicator that represents the reference dance operation at the instruction timing (S104). In other words, the generation unit 15 generates instruction data D2 so that the created instruction indicator is displayed at the instruction timing. To put it another way, the generation unit 15 generates instruction data D2 so that an instruction indicator that represents a specific operation at the instruction timing on the time axis of the operation data D1 is displayed at the instruction timing.

[0088] Furthermore, the generation unit 15 generates a dance object DO that operates in time with the music based on the operation data D1 (S105). Then, the generation unit 15 generates instruction data D2 so that the instruction indicator and the dance object DO are displayed (S106). As a result, if the creation of instruction indicators for all instruction timings is completed (YES in S107), the generation process of instruction data D2 is completed. On the other hand, if the creation of instruction indicators is not completed (NO in S107), the next instruction timing is compared with the specific timing (S102).

[0089] On the other hand, if the instruction timing and the specific timing are different, that is, if the difference between the specific timing and the instruction timing is outside a predetermined range (YES in S103), the generation unit 15 uses the difference between the specific timing and the instruction timing on the time axis of the operation data D1 as a correction value. Then, the generation unit 15 adds the correction value to the specific timing or subtracts the correction value from the specific timing (S108). Subsequently, the generation unit 15 creates an instruction indicator that represents the specific operation at the specific timing, which is to be displayed at the compared instruction timing (S104).

[0090] Furthermore, the generation unit 15 generates a dance object DO that operates in time with the music based on the operation data D1 (S105). Then, the generation unit 15 generates instruction data D2 so that the instruction indicator and the dance object DO are displayed (S106). As a result, if the creation of instruction indicators for all instruction timings is completed (YES in S107), the generation process of instruction data D2 is completed. On the other hand, if the creation of instruction indicators is not completed (NO in S107), the next instruction timing is compared with the specific timing (S102).

[0091] According to the embodiments described above, even if there is a discrepancy between the movement in the reference movement data D1 and the desired movement, instruction signs can be generated based on the appropriate movement. Furthermore, even if the movement is still in progress at the instruction timing in the movement data D1 based on the reference dance movement, instruction signs can be generated based on the appropriate movement. For example, an instruction sign that mimics the silhouette of a dancer can be made into an appropriate shape. In addition, an instruction sign that specifies the position of a part of the dancer's body can be displayed in an appropriate position. Moreover, an instruction sign that traces a trajectory according to the dancer's movement can be displayed in an appropriate trajectory.

[0092] Furthermore, this embodiment allows for more dynamic dance movements to be instructed to the player. That is, the dancer performs a reference dance movement aiming to perfectly match the instructed timing. However, aiming for perfect timing can restrict the dancer's freedom of expression, causing the reference dance movement to lose its dynamism. In this respect, this embodiment allows for the display of instruction markers at the appropriate timing even if the dancer's dance movement (especially a specific movement) is out of sync with the instructed timing. Therefore, the dancer no longer needs to aim for perfect timing, increasing their freedom of expression and maintaining the dynamism of the reference dance movement. By creating instruction markers based on such a reference dance movement, the player can be instructed to perform more dynamic dance movements.

[0093] Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the embodiments described above. Inventions modified within the scope that does not contradict the present invention, and inventions equivalent to the present invention are also included in the present invention. Furthermore, each embodiment and each variation, as well as the technical means included in each embodiment or each variation, can be appropriately combined within the scope that does not contradict the present invention.

[0094] For example, at least a portion of the logical device installed in the game server 10 can be installed in the game machine 60 or another computer device. In this case, at least two devices—the game machine 60, the game server 10, and the other computer device—work together to function as a generation system or game system 100.

[0095] The following describes various embodiments derived from the above-described embodiments and modifications. Reference numerals shown in the accompanying drawings are included to facilitate understanding of each embodiment. However, these reference numerals are not intended to limit the present invention to the illustrated forms.

[0096] A generation method used in a game that generates instruction data D2 for displaying instruction indicators MP1, MP2, MT1, and MT2 that are generated based on motion data D1 of standard dance movements synchronized with music and that instruct the player to perform play movements, and for generating instruction data D2 for displaying the instruction indicators MP1, MP2, MT1, and MT2 at the instruction timing when the play movements should be performed, compares a specific timing in which a specific movement among the standard dance movements is performed with the instruction timing, and if the specific timing is different from the instruction timing, generates the instruction data D2 so that the instruction indicators MP1, MP2, MT1, and MT2 that represent the specific movement at the specific timing are displayed at the compared instruction timing.

[0097] In a game where instruction indicators MP1, MP2, MT1, and MT2 are generated based on motion data D1 of standard dance movements synchronized with music and instruct the player to perform play movements, the generation system 100, which includes a computer 11, generates instruction data D2 for displaying the instruction indicators MP1, MP2, MT1, and MT2 at the instruction timing when the play movements should be performed, based on the motion data D1, and the computer 11 compares a specific timing in which a specific movement among the standard dance movements is performed with the instruction timing, and if the specific timing differs from the instruction timing, it generates the instruction data D2 so that the instruction indicators MP1, MP2, MT1, and MT2 representing the specific movement at the specific timing are displayed at the compared instruction timing.

[0098] The system is used in a game that generates instruction data MP1, MP2, MT1, and MT2 that are generated based on motion data D1 of reference dance movements synchronized with music and that instruct the player to perform play movements. The system generates instruction data D2 based on the motion data D1 to display the instruction data MP1, MP2, MT1, and MT2 at the instruction timing when the play movements should be performed. The control program PG1 of the generation system 100, which includes a computer 11, causes the computer 11 to compare a specific timing in which a specific movement among the reference dance movements is performed with the instruction timing. If the specific timing differs from the instruction timing, the system generates instruction data D2 such that the instruction data MP1, MP2, MT1, and MT2 representing the specific movement at the specific timing are displayed at the compared instruction timing.

[0099] This allows for the generation of instruction markers MP1, MP2, MT1, and MT2 based on appropriate movements, even when there is a discrepancy between the movements in the reference movement data D1 and the desired movements. Furthermore, even when the movements in the movement data D1 based on the reference dance movements are still in progress at the instruction timing, instruction markers MP1, MP2, MT1, and MT2 can be generated based on appropriate movements.

[0100] The game system 100 provides a game that displays instruction signs MP1, MP2, MT1, and MT2 that are generated based on motion data D1 of reference dance movements synchronized with music and that instruct the player to perform play movements, and includes a computer 11. The computer 11 displays a dance object DO that moves in time with the music based on the motion data D1, and displays the instruction signs MP1, MP2, MT1, and MT2 at predetermined instruction timings. If the specific timing at which the dance object DO performs a specific movement from the reference dance movements differs from the instruction timing, the computer 11 displays the instruction signs MP1, MP2, MT1, and MT2 that represent the specific movement at that instruction timing.

[0101] This allows the player to refer to instruction indicators MP1, MP2, MT1, and MT2 while using the dance object DO as a model. Therefore, specific actions can be performed at appropriate timings that differ from the specific actions of the dance object DO.

[0102] If the difference between the specified timing and the instruction timing is within a predetermined range, the instruction data D2 is generated so that the instruction indicators MP1, MP2, MT1, and MT2, which represent the reference dance movement at the instruction timing, are displayed at the instruction timing. As a result, if the movement at the instruction timing in the movement data D1 is a visually appealing movement, the instruction indicators MP1, MP2, MT1, and MT2 can be generated based on that movement.

[0103] The specific timing is set by adding or subtracting the difference between the specific timing and the operation timing corresponding to the instruction timing on the time axis of the operation data D1 as a correction value to the operation timing. This allows the amount of deviation to be objectively identified, and instruction signs MP1, MP2, MT1, and MT2 can be created based on the specific operation at the precise specific timing.

[0104] Musical feature points are extracted from the music data D3 of the aforementioned music, and the first timing corresponding to the musical feature points is used as the instruction timing to generate the instruction data D2. This allows the generation unit 15 to automatically identify the instruction timing.

[0105] The generation unit 15 extracts characteristic operation points from the operation data D1, and if the second timing corresponding to the characteristic operation point is earlier or later than the first timing, it generates the instruction data D2 with the second timing as the specific timing. This allows the generation unit 15 to automatically identify the specific operation and the specific timing.

[0106] A dance object DO is generated that is displayed together with the instruction indicators MP1, MP2, MT1, and MT2 and operates in time with the music based on the operation data D1, and the specific operation is different from the operation of the dance object DO at the indicated timing. This allows the player to use the dance object DO, which shows a reference dance operation by the dancer, as a model, while also dancing at appropriate indicated timings different from the reference dance operation, in accordance with the instructions given by the instruction indicators MP1, MP2, MT1, and MT2.

[0107] The instruction markers MP1 and MP2 are generated as dance images that mimic at least a part of the body of the dancer performing the reference dance movements, and the dance object DO is generated to mimic the movements of the dancer performing the reference dance movements. As a result, the player can use the dance object DO, which shows the reference dance movements performed by the dancer, as a model, and dance according to the instructions given by the instruction markers MP1, MP2, MT1, and MT2, at appropriate timings different from the reference dance movements. [Explanation of symbols]

[0108] 11: Server Control Unit (Computer) 100: Game System (Generation System) D1: Operation data D2: Instruction data D3: Music data MP1: Pause marker (instructional sign) MP2: Pause marker (instructional sign) MT1: Touch marker (instructional sign) MT2: Touch Marker (Instructional Sign)

Claims

1. A method for generating instruction data for displaying instruction signs that instruct the player to perform a play action, which is used in a game that generates instruction data based on motion data obtained by filming standard dance movements synchronized with music, and for which a computer generates instruction data for displaying the instruction signs at the instruction timing when the play action should be performed, based on the motion data, The aforementioned computer, The specific timing at which a specific movement among the aforementioned standard dance movements is performed is compared with the instructed timing. A generation method that generates instruction data such that, if the specified timing differs from the instruction timing, the instruction indicator representing the specified operation at the specified timing is displayed at the compared instruction timing.

2. The generation method according to claim 1, wherein, if the difference between the specific timing and the instruction timing is within a predetermined range, the computer generates the instruction data such that an instruction sign representing the reference dance movement at the instruction timing is displayed at the instruction timing.

3. The generation method according to claim 1 or 2, wherein the computer generates the instruction data by setting the operation timing of the specific operation by adding the difference between the specific timing and the instruction timing on the time axis of the operation data as a correction value to the specific timing or subtracting it from the specific timing.

4. The computer Musical feature points are extracted from the music data of the aforementioned music. The generation method according to any one of claims 1 to 3, wherein the first timing corresponding to the musical feature point is used as the instruction timing to generate the instruction data.

5. The computer From the aforementioned motion data, motion characteristic points are extracted, The generation method according to claim 4, wherein if the second timing corresponding to the aforementioned operational characteristic point is earlier or later than the first timing, the second timing is used as the specific timing to generate the instruction data.

6. The computer A dance object is generated that is displayed together with the instruction sign and operates in time with the music based on the operation data. The generation method according to any one of claims 1 to 5, wherein the specified operation is different from the operation of the dance object at the instructed timing.

7. The computer The instruction marker is generated as a dance image that mimics at least a part of the body of the dancer performing the aforementioned reference dance movement. The generation method according to claim 6, which generates the dance object so that it moves in a manner that imitates the movements of the dancer performing the aforementioned reference dance movements.

8. A generation system, comprising a computer, is used in a game that generates instruction data based on motion data obtained by filming standard dance movements synchronized with music and displays instruction signs that instruct the player to perform the play movements, and generates instruction data for displaying the instruction signs at the instruction timing when the play movements should be performed, based on the motion data. The aforementioned computer, The specific timing at which a specific movement among the aforementioned standard dance movements is performed is compared with the instructed timing. A generation system that generates instruction data such that, if the specified timing differs from the instruction timing, the instruction indicator representing the specified operation at the specified timing is displayed at the compared instruction timing.

9. A control program for a generation system equipped with a computer, which is used in a game that generates instruction data based on motion data obtained by filming standard dance movements synchronized with music and displays instruction signs that instruct the player to perform play actions, and generates instruction data for displaying the instruction signs at the instruction timing when the play actions should be performed, based on the motion data, To the aforementioned computer, The specific timing at which a specific movement among the aforementioned standard dance movements is performed is compared with the instructed timing. A control program that, if the specified timing differs from the instruction timing, generates instruction data such that the instruction indicator representing the specified operation at the specified timing is displayed at the compared instruction timing.

10. A game system is provided that includes a computer and displays instruction signs that are generated based on motion data obtained by filming standard dance movements synchronized with music, and that instruct the player to perform the movements. The aforementioned computer, Based on the aforementioned motion data, a dance object that moves in time with the music is displayed, and the instruction sign is displayed at a predetermined instruction timing. A game system that, if the specific timing at which the dance object performs a specific movement from the standard dance movements differs from the instructed timing, displays an instruction sign representing the specific movement at the instructed timing.

Images