Arithmetic device, arithmetic method, and computer program

Abstract

To provide an arithmetic device for estimating, from musical score data, changes in posture of a user making a series of movements according to the musical score data.SOLUTION: An arithmetic device 1 is provided, comprising an arithmetic unit 11 configured to estimate changes in posture of a user according to musical score data using a model designed to output changes in posture of a suer upon receipt of musical score data as an input.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to a computing device, a computing method, and a computer program. [Background technology]

[0002] Content is known in which a player performs a series of movements by performing instructed movements at instructed timing. For example, such content may include music accompanied by movement instructions, and the user performs a series of movements, such as dancing or gymnastics, by performing the instructed movements in time with the music. The movements are movements that move at least a part of the player's body to the instructed position. For example, in a music game, a movement may involve stepping on multiple panels at the player's feet, identified by arrows, in accordance with the arrows that appear as they move from bottom to top on the screen. Such content is used in dance games, rhythmic gymnastics, and the like.

[0003] In a dance game or the like, music score data, which is data representing instructions for successive movements and the timing of performing those movements, may be created manually or automatically. For example, Japanese Patent Application Laid-Open Publication No. 2019-201939 (hereinafter referred to as Patent Document 1) discloses a device that automatically creates music score data. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2019-201939 Summary of the Invention

[0005] Depending on the user's changes in posture when performing a series of actions instructed by the music score data, the series of actions may be difficult or easy. However, since the music score data instructs the actions and the timing of performing the actions, the music score data alone does not specify what posture the user should use to perform the actions. Therefore, it is desirable to be able to understand from the music score data the changes in the user's posture when performing a series of actions in accordance with the music score data.

[0006] Here, the calculation device includes a calculation unit configured to estimate a change in the user's posture relative to the music score data using a model that outputs a change in the user's posture when music score data is given as input.

[0007] The calculation method is a method for estimating a user's posture change relative to music score data, and includes estimating a user's posture change relative to music score data using a model that outputs a user's posture change when music score data is given as input.

[0008] The computer program is a computer program that causes a computer to perform a process of estimating a user's posture change in response to music score data, and causes the computer to estimate a user's posture change in response to music score data using a model that outputs a user's posture change when music score data is given as input.

[0009] Further details will be described in the following embodiments. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a schematic diagram showing an example of how a computing device according to an embodiment is used. [Figure 2] FIG. 2 is a schematic block diagram illustrating an example of the configuration of the arithmetic unit. [Figure 3] FIG. 3 is a diagram showing an example of a part of a model used in a computing device. [Figure 4]FIG. 4 is a diagram for explaining the actions of the player. [Figure 5] FIG. 5 is a flowchart illustrating an example of a method for determining a sequence of actions for a set of instructions. [Figure 6] FIG. 6 is a diagram illustrating an example of a transition path determined in the arithmetic device. [Figure 7] FIG. 7 is a diagram illustrating another example of a method for calculating a node evaluation value. [Figure 8] FIG. 8 is a diagram for explaining another example of a method for calculating the transition probability of an edge. [Figure 9] FIG. 9 is a diagram showing an example of an input musical score. [Figure 10] FIG. 10 is a diagram showing posture changes estimated by the computing device when the user attributes are changed in the musical score of FIG. DETAILED DESCRIPTION OF THE INVENTION

[0011] <1. Overview of the arithmetic device, arithmetic method, and computer program>

[0012] (1) A calculation device according to an embodiment includes a calculation unit configured to estimate a user's posture change in response to music score data using a model that outputs a user's posture change when music score data is given as input.

[0013] A posture change refers to a change in the posture of a user performing a series of actions in accordance with music score data. Music score data refers to data set for content that indicates instructions for successive actions and the timing of performing those actions.

[0014] By using a model that outputs a user's posture change when music score data is given as input, it becomes possible to easily obtain the user's posture change by inputting music score data.

[0015] (2) Preferably, the model has a plurality of nodes that define the user's posture and movements, and a plurality of edges connecting the nodes, and estimating the user's posture changes includes determining the user's posture changes in response to the group of instructions by determining a path of node transition appropriate for the group of instructions that includes a plurality of movement instructions contained in the music score data.

[0016] The model may be, for example, a hidden Markov model or a graph in graph theory. By using a model having multiple nodes that define the user's posture and movements and edges connecting the nodes, the path of node transitions is determined taking into account the user's posture and movements, and the transitions between movements. By determining the path of node transitions, the user's posture changes can be obtained. Therefore, it is possible to obtain the user's posture changes from the music score data.

[0017] (3) Preferably, the calculation unit uses a judgment value in the process of determining the path, and the judgment value is based on at least one of the transition probability of the edge, the evaluation value of the node, and the degree of agreement between the action specified by the node and the action specified by the instruction. Thus, the path of node transition is determined using at least one of the transition probability of the edge, the evaluation value of the node, and the degree of agreement between the action specified by the node and the action specified by the instruction. As a result, the user's posture change is determined taking into account the user's posture and actions, and the transition between actions.

[0018] (4) Preferably, the transition probability is expressed as a value indicating the likelihood of a change in the posture defined by each node between the nodes, thereby determining a transition path for nodes that are likely to change their posture.

[0019] (5) Preferably, the transition probability is obtained based on at least one of the direction of movement of at least a part of the body between nodes, the amount of movement, the amount of change in angle of the body, and the presence or absence of a specific movement in at least one of a first coordinate system that is the reference for instructions and a second coordinate system that is defined according to the user's posture. This allows a transition path of nodes that is likely to cause a change in posture for the user to be determined.

[0020] (6) Preferably, the evaluation value is expressed as a value indicating the ease with which the node can assume a posture defined by the node, thereby determining a transition path for a node that is prone to changing posture.

[0021] (7) Preferably, the evaluation value is obtained based on at least one of the degree of understanding of the position of at least a part of the body in the posture defined by the node in a coordinate system that is the reference for instructions, the positional relationship of a plurality of body parts, and the orientation of the body. This allows the path of node transition to be determined using the evaluation value that represents the ease with which the user can assume the posture.

[0022] (8) Preferably, the process of determining the path includes a process of adjusting at least one of the transition probability and the evaluation value, so that the path of node transition is determined using flexible at least one of the transition probability and the evaluation value.

[0023] (9) Preferably, the adjusting process includes adjusting at least one of the transition probability and the evaluation value according to the user attributes. This allows the node transition path to be determined using at least one of the transition probability and the evaluation value according to the user attributes. As a result, a posture change suitable for the user is determined.

[0024] (10) Preferably, the calculation unit is configured to further execute a process of evaluating the group of instructions using the judgment value, thereby making it possible to objectively indicate a series of movements in accordance with the group of instructions in terms of posture change using numerical values.

[0025] (11) A calculation method according to an embodiment is a method for estimating a user's posture change relative to music score data, and includes estimating a user's posture change relative to the music score data using a model that outputs a user's posture change when music score data is given as input. By using the model that outputs a user's posture change when music score data is given as input, it is possible to easily obtain a user's posture change by inputting music score data.

[0026] (11) A computer program according to an embodiment is a computer program that causes a computer to perform a process of determining a user's posture change in response to music score data, and causes the computer to estimate a user's posture change in response to the music score data using a model that outputs a user's posture change when music score data is given as input. By using the model that outputs a user's posture change when music score data is given as input, the computer can easily determine a user's posture change by inputting the music score data.

[0027] 2. Examples of arithmetic devices, arithmetic methods, and computer programs

[0028] FIG. 1 is a schematic diagram showing an example of how a calculation device 1 according to this embodiment is used. Referring to FIG. 1, the calculation device 1 is connected to a music score data generation device 9, as an example, and performs calculations using music score data D obtained from the music score data generation device 9. The music score data D is data that represents instructions for successive actions and the timing to perform the actions, which is set for content such as music, and includes an instruction group that includes multiple instructions for actions. A user, who is a player, performs a series of actions in accordance with the instructions in the instruction group included in the music score data, which causes changes in posture, resulting in the user playing a music game such as dancing, performing exercises, or the like.

[0029] The calculation device 1 is connected to a controller 50 of a dance game device 5, which is an example of a device that instructs a user, such as a player, to perform movements such as dancing, and transmits a signal corresponding to the calculation result to the controller 50. The signal corresponding to the calculation result may be, for example, music score data D. Another example of a device that instructs a user to perform movements is a device that instructs a user to perform physical movements for treatment such as rehabilitation, which involves physical movements.

[0030] Dance game device 5 has display 51, speaker 52, and operation unit 53. Music M is output from speaker 52. Operation unit 53 is located at the feet of a player standing facing display surface 51a (front) of display 51. This allows a player of a dance game using dance game device 5 to stand in front of display 51 and perform actions by stepping on operation unit 53 with both feet.

[0031] The operation unit 53 has a first panel 31, a second panel 32, a third panel 33, a fourth panel 34, and a fifth panel 35, and the other panels 31 to 34 are arranged on the left, right, front, and rear of the fifth panel 35, respectively. Here, "front" refers to the side closer to the display surface 51a of the display 51, "rear" refers to the side farther from the display 51, "left" refers to the left side as viewed from the display 51, and "right" refers to the right side as viewed from the display 51. As an example, a leftward arrow 31a, a rightward arrow 32a, a forward arrow 33a, and a backward arrow 34a are depicted on the panels 31 to 34, respectively, and the panels 31 to 34 are identified by the leftward arrow 31a, the rightward arrow 32a, the forward arrow 33a, and the backward arrow 34a, respectively.

[0032] The display 51 displays instructions for the player to perform an action based on the music score data D. The action is to step on at least one of the panels 31 to 35 at a specific timing. The instructions for the action include the display of an arrow indicating the direction of the panel to be stepped on. For example, in the example of FIG. 1, instruction I1 represents a left-pointing arrow and specifies that the first panel 31 should be stepped on. Instruction I2 represents a downward-pointing arrow and specifies that the fourth panel 34 should be stepped on. Instruction I3 represents a right-pointing arrow and specifies that the second panel 32 should be stepped on. This allows the player to know which panel to step on as indicated by the instructions on the display 51.

[0033] The action instruction also includes an instruction on the timing of stepping. The timing of stepping is indicated, for example, by the position of the instruction displayed on the display 51. For example, in the example of FIG. 1, multiple instructions I1, I2, and I3 are displayed in order, moving from the bottom to the top of the display 51 in the direction of arrow A. At this time, a display position indicating the action timing is specified on the display 51. In the example of FIG. 1, a line P1 represents the display position indicating the action timing. As an example, the action instruction includes the timing when the instruction moving in the direction of arrow A is displayed overlapping with the line P1, as an instruction on the timing of stepping. This allows the player to know the panel to be stepped on indicated by the instruction on the display 51 and the timing of stepping on that panel.

[0034] In the music game, a group of instructions consisting of a plurality of instructions based on the music score data D is displayed on the display 51 in time with the music M. The player then steps on the indicated panels at the indicated timing. As a result, the player changes their posture in time with the music M. The changes in posture form a series of movements such as dancing.

[0035] The calculation device 1 acquires score data D from the score data generation device 9. The calculation device 1 determines a change in the player's posture when performing a series of actions in accordance with an instruction group having multiple action instructions included in the score data D. As another example, the score data D may be stored in the calculation device 1 in advance, or may be input from an operation device 15 (FIG. 2) of the calculation device 1, etc.

[0036] Fig. 2 is a schematic block diagram showing an example of the configuration of the arithmetic device 1. Referring to Fig. 2, the arithmetic device 1 is configured as a computer having an arithmetic unit 11 (processor) and a memory 12. The arithmetic unit 11 is, for example, a CPU. The memory 12 includes a flash memory, an EEPROM, a ROM, a RAM, etc. Alternatively, the memory 12 may be a primary storage device or a secondary storage device.

[0037] The memory 12 stores a computer program 121 executed by the calculation unit 11. The calculation unit 11 executes the computer program 121 to perform calculation processing for estimating a change in the player's posture (hereinafter abbreviated as calculation processing).

[0038] The memory 12 may store the model 122. Alternatively, the model 122 may be stored in a device different from the computing device 1, and the computing device 1 may read the model 122 from the different device by communicating with it.

[0039] Model 122 is a model that outputs changes in the user's posture when music score data is given as input. As an example, model 122 is a model that represents the transitions in the player's state and has nodes and edges. A model that has nodes and edges expresses the player's posture and its changes as a probabilistic finite-state automaton, and may be, for example, a hidden Markov model in which a group of instructions is an observation sequence and the player's posture during state transitions is a hidden state. As another example, it may be a graph in graph theory.

[0040] As another example, the model 122 may be a learning model that has been machine-learned to output the user's movements (postures) when music score data is given as input.

[0041] The nodes define the player's posture and actions. The player's posture is indicated, for example, by the position of the right foot, the position of the left foot, and the direction of the body. The position of each foot is indicated by the panel on which it is placed. The direction of the body is indicated by the direction the player is standing in front. The player's actions are indicated by the position where the stepping action is performed. The position where the stepping action is performed is indicated by the panel that is the target of the stepping action.

[0042] The edges are directed edges that define the transition from the start node to the end node. Since each node represents a player's posture, the transition between the nodes represented by the edges represents a change in the player's posture.

[0043] 3 is a diagram showing an example of model 122A, which is a part of model 122. Model 122A has nodes N1 to N13 and edges connecting between them. Model 122A is a part of model 122, with node N1 as the start node, and state transitions from node N1.

[0044] Each node has four parameters P1, P2, P3, and P4. Parameter P1 indicates the panel on which the left foot is placed. Parameter P2 indicates the panel on which the right foot is placed. Parameter P3 indicates the direction of the body. Parameter P4 indicates the action that is the target of stepping. Parameters P1, P2, and P3 are parameters that indicate the player's posture. Parameter P4 is a parameter that indicates the player's action. The four parameters are expressed, for example, as (P1, P2, P3, P4).

[0045] For example, for node N1, parameters P1, P2, and P4 are all the fifth panel 35. For node N1, parameters P1 and P2 indicate a posture in which both the right foot and the left foot are placed on the fifth panel 35. For node N1, parameter P3 indicates a posture in which the user stands facing forward. For node N1, parameter P4 indicates an action of stepping on the fifth panel 35.

[0046] Node N2 has parameter P1 for the third panel 33 and parameter P2 for the second panel 32. Based on parameters P1 and P2, node N2 indicates a posture in which the left foot is placed on the third panel 33 and the right foot is placed on the second panel 32. Based on parameter P3, node N2 indicates a posture in which the user stands facing diagonally forward to the right. Based on parameter P4, node N2 indicates an action of stepping on the third panel 33 and the second panel 32. In this case, the third panel 33 is stepped on with the left foot, and the second panel 32 is stepped on with the right foot.

[0047] Node N3 has parameter P1 for the fourth panel 34 and parameter P2 for the second panel 32. Based on parameters P1 and P2, node N3 indicates a posture in which the left foot is placed on the fourth panel 34 and the right foot is placed on the second panel 32. Based on parameter P3, node N3 indicates a posture in which the user stands facing diagonally forward to the left. Based on parameter P4, node N3 indicates the action of stepping on the second panel 32. In this case, the action is to place the left foot on the fourth panel 34 and step on the second panel 32 with the right foot.

[0048] Node N4 has parameter P1 for the second panel 32 and parameter P2 for the fourth panel 34. Based on parameters P1 and P2, node N4 indicates a posture in which the left foot is placed on the second panel 32 and the right foot is placed on the fourth panel 34. Based on parameter P3, node N4 indicates a posture in which the user stands facing diagonally backward to the right. Based on parameter P4, node N4 indicates a motion of stepping on the second panel 32. In this case, the motion is to place the right foot on the fourth panel 34 and step on the second panel 32 with the left foot.

[0049] Model 122A has an edge connecting node N1 and node N2. Model 122A indicates that a transition is possible between node N1 and node N2. In the transition from node N1 to node N2, the left foot moves from the fifth panel 35 to the third panel 33, and the right foot moves from the fifth panel 35 to the second panel 32. Therefore, the transition from node N1 to node N2 indicates a posture change in which the player's posture transitions from a posture facing the display 51 (facing forward) to a posture facing in a direction between the third panel 33 and the second panel 32 (facing diagonally forward to the right). In addition, the transition from node N1 to node N2 indicates a posture change in which the player jumps from a standing state with both feet on the fifth panel 35 and lands with their left and right feet on the third panel 33 and the second panel 32, respectively.

[0050] Model 122A has an edge connecting node N2 and node N3. Model 122A indicates that a transition between node N2 and node N3 is possible. In the transition from node N2 to node N3, the left foot moves from the third panel 33 to the fourth panel 34, and the right foot remains on the second panel 32. Therefore, the transition from node N2 to node N3 indicates a posture change in which the player's posture transitions from a posture facing the direction between the third panel 33 and the second panel 32 (facing diagonally forward to the right) to a posture facing the direction between the first panel 31 and the third panel 33 (facing diagonally forward to the left). In addition, the transition from node N2 to node N3 indicates a posture change in which the player's body orientation is rotated around the right foot, which is placed on the second panel 32, to move the left foot from the third panel 33 to the fourth panel 34.

[0051] Model 122A has an edge connecting node N2 and node N4. Model 122A indicates that a transition is possible between node N2 and node N4. In the transition from node N2 to node N4, the left foot moves from the third panel 33 to the second panel 32, and the right foot moves from the second panel 32 to the fourth panel 34. Therefore, the transition from node N2 to node N4 indicates a posture change in which the player's posture transitions from a posture facing in the direction between the third panel 33 and the second panel 32 (facing diagonally forward to the right) to a posture facing in the direction between the second panel 32 and the fourth panel 34 (facing diagonally backward to the right).

[0052] Preferably, each edge has a transition probability between nodes. The transition probability of an edge connecting a first node and a second node indicates the probability of transition from the first node to the second node. Specifically, the transition probability of an edge connecting a first node and a second node represents the probability of transition of an edge having the second node as its end node with respect to all edges (other edges) having the first node as its start node. In other words, the transition probability of an edge connecting a first node and a second node represents the ease with which a player's posture changes from the posture defined by the first node to the posture defined by the second node with respect to other edges, with a higher transition probability indicating that the change is more likely than other edges, and a lower transition probability indicating that the change is more difficult than other edges.

[0053] As a first example, the transition probability of an edge (the transition probability between nodes represented by an edge) is related to the direction of movement of at least one of the player's feet. For a typical player, it is easy to move the foot forward. It is also possible to move the foot sideways. Moving the foot backward is more difficult than in other directions. Therefore, a transition probability is set for an edge connecting a first node and a second node according to the direction of foot movement from the posture defined for the first node to the posture defined for the second node. Specifically, the transition probability is set higher the closer the foot movement direction is to the front of the player, and lower the further away from the front.

[0054] The ease of posture change will be explained using Figure 4. In the example of Figure 4, player P is standing with his / her front right side facing forward in relation to the XY coordinate system (first coordinate system) that serves as the basis for instructions from dance game device 5, with first panel 31 on the left, second panel 32 on the right, third panel 33 in front, and fourth panel 34 behind.

[0055] Furthermore, when considering the direction of foot movement for the player, an xy coordinate system (second coordinate system) is set based on the player's body. As an example, the origin is set to the player's center of gravity, the x axis is set to the player's left-right direction, and the y axis is set to the player's front-to-back direction, with the direction in which the x coordinate increases being to the right and the direction in which the y coordinate increases being to the front (the direction in which the hips face). In other words, in this example, the xy coordinate system is set by rotating it to the right from the XY coordinate system.

[0056] The transition probability of the edge connecting the first node and the second node is set according to the direction in the xy coordinate system set for the posture defined in the first node of the movement of the foot from the posture defined in the first node to the posture defined in the second node.

[0057] The posture of player P in Figure 4 is the posture defined for the first node, and the foot moves along movement path M1 to the posture defined for the second node. Movement path M1 represents the movement direction and amount of movement of the foot, and indicates a movement angle θ1 and a movement amount LM relative to the y-axis direction. In this case, the transition probability of the edge connecting the first node and the second node is set according to the absolute value of the movement angle θ1. In other words, the smaller the absolute value of the movement angle θ1, the higher the transition probability is set, and the larger the absolute value of the movement angle θ1, the lower the transition probability is set.

[0058] As a second example, the ease of posture change is related to the amount of movement of at least a part of the player's body. For a typical player, movements with large amounts of movement are difficult, and movements with small amounts of movement are easy. The amount of movement may be the amount of movement of the feet, or the amount of movement of the player's body itself, for example, the amount of movement of the center of gravity. Therefore, a transition probability is set for the edge connecting the first node and the second node according to the amount of movement of at least a part of the body from the posture defined for the first node to the posture defined for the second node. Specifically, the transition probability is set higher as the amount of movement is larger, and lower as the amount of movement is smaller. In the example of Figure 4, the transition probability is set higher as the amount of movement LM is smaller, and lower as it is larger.

[0059] As a third example, the ease of posture change is related to the amount of change in the player's body angle. The amount of change in body angle refers to the amount of rotation of the body around the center of gravity, and one example is the amount of change in the direction of the hips. For a typical player, movements that involve large body angle changes are difficult, but movements that involve small angle changes are easy. In the example of Figure 4, player P's posture is defined as the posture defined in the first node, and he rotates his body by an amount θ2 to the posture defined in the second node. In this case, the transition probability of the edge connecting the first node and the second node is set according to the absolute value of the amount of rotation θ2. In other words, the smaller the absolute value of the amount of rotation θ2, the higher the transition probability is set, and the larger the absolute value of the amount of rotation θ2, the lower the transition probability is set.

[0060] As a fourth example, the ease of posture change is related to the presence or absence of a specific action. A specific action is a special action that is difficult for an average player to perform, such as moving both feet at the same time or moving both feet in different directions. An example of an action that moves both feet at the same time is a jumping action. An example of an action that moves both feet in different directions is switching the positions of the left and right feet. In this case, the transition probability is set lower the more specific actions are included, and higher the fewer specific actions are included.

[0061] Preferably, the transition probability is set based on at least one of the first to fourth examples. Preferably, the transition probability is set based on two or more of the first to fourth examples. Thereby, the transition probability is determined according to the ease of posture change.

[0062] When the transition probability is set based on all of the first to fourth examples, the transition probability Pt is calculated by the following formula (1), for example. Pt=log(1 / E) …(1) where E=E1+E2+E3+E4

[0063] E1 to E4 in formula (1) are values ​​related to those shown in the first to fourth examples, respectively. Specifically, E1 is a value related to the movement directions of the left and right feet shown in the first example, and is, for example, a value calculated using a movement angle θ1 using a normal distribution with a maximum value at θ1=0° multiplied by -1. E1 is a continuous value greater than 0 and equal to or less than 1 for each of the left and right feet.

[0064] E2 is a value related to the amount of movement of at least a part of the body shown in the second example, and is, for example, a value obtained by multiplying the amount of movement LM of the left and right feet and the amount of movement of the center of gravity by a predetermined weight. E2 is a continuous value greater than or equal to 0 and less than 2 × the weight for each of the amount of movement of the left and right feet and the amount of movement of the center of gravity.

[0065] E3 is a value related to the amount of change in the angle of the body shown in the third example, and is, for example, a value obtained by multiplying the amount of rotation θ2 by a predefined weight. E3 is a continuous value equal to or greater than 0.

[0066] E4 is a value related to the presence or absence of a specific action shown in the fourth example, and is obtained by adding a predetermined fixed value when, for example, there is an action of moving both feet simultaneously and an action of stepping on the same panel with both feet. E4 is a continuous value greater than 0 and less than the fixed value for each action.

[0067] By selecting edges based on the transition probability, it is possible to select a node's movement path according to the ease of change. Also, by referring to the transition probability of each edge included in the node's movement path, it is possible to evaluate the ease of change of a series of actions.

[0068] Preferably, each node has an evaluation value. The evaluation value of a node is an index value that indicates how easy it is to take the posture defined by the node. As an example, a higher evaluation value of a node indicates that the posture defined by the node is easier to take, and a lower evaluation value indicates that the posture is more difficult to take. In other words, the evaluation value of a node is an index value that indicates how stable the player's body is when stationary in the corresponding posture.

[0069] As a first example, the ease of assuming a posture (the evaluation value of a node) is related to the position where the player places their left and right feet, i.e., the degree of grasp of the panel on which they place their feet. A posture where the player places their feet on a panel with a low degree of grasp is unstable and can be said to be a difficult posture. A posture where the player places their feet on a panel with a high degree of grasp is stable and can be said to be an easy posture.

[0070] The degree of grasp of a panel can be determined by an XY coordinate system (first coordinate system) that serves as the basis for instructions from dance game device 5. This is because the player looks at the group of instructions displayed on display 51 and performs the instructed movements, so their line of sight is directed toward display 51. Therefore, as an example, the closer the panel is to the front (with a larger Y coordinate) in the XY coordinate system, the higher the grasp degree, and the closer the panel is to the back (with a smaller Y coordinate), the lower the grasp degree. In other words, the closer the panel on which the player places their feet to display 51, the higher the evaluation value is set, and the farther the panel is from display 51, the lower the evaluation value is set.

[0071] As a second example, the ease of posture is related to the positional relationship between the left and right feet. A posture in which the left and right feet are placed on panels that are not aligned in the front-to-back (Y-axis) or left-to-right (X-axis) directions in the XY coordinate system can be said to be difficult to assume. This is because the degree of comprehension decreases because the user must simultaneously distinguish between the front-to-back and left-to-right panels. In other words, a high evaluation value is assigned when the panels on which the left and right feet are placed are aligned in the front-to-back or left-to-right directions, and a low evaluation value is assigned when they are not aligned.

[0072] As a third example, the ease of posture is related to the orientation of the player's body relative to the display 51, i.e., the Y-axis direction in the XY coordinate system, as indicated by angle θ3 in the example of FIG. 4. If the orientation of the body (the orientation of the waist) differs from the Y-axis direction, the orientation of the player's face and body will differ. The larger the angle θ3 formed between the orientation of the face and the orientation of the body, the more unstable the posture. In other words, a high evaluation value is set when angle θ3 is small, and a low evaluation value is set when angle θ3 is large.

[0073] Preferably, the evaluation value is set based on at least one of the first to third examples. Preferably, the evaluation value is set based on two or more of the first to third examples. This allows the evaluation value to correspond to the ease of adopting a posture.

[0074] When the evaluation value is set based on all of the first to third examples, the evaluation value Ev is calculated, for example, by the following formula (2): Note that in formula (2), the evaluation value Ev is calculated as the total product of Ev1 to Ev3, but it may also be calculated as the total sum of Ev1 to Ev3. Ev = Ev1 × Ev2 × Ev3 … (2)

[0075] Ev1 to Ev3 in formula (2) are values ​​related to those shown in the first to third examples, respectively. Specifically, Ev1 is a value related to the degree of grasp of the panel on which each foot is placed, as shown in the first example. As an example, it is a value calculated using a normal distribution in which the panel (third panel 33) with the largest Y coordinate in the XY coordinate system has the largest value and the panel (fourth panel 34) with the smallest Y coordinate has the smallest value. Ev1 is a continuous value greater than 0 and equal to or less than 1 for each of the left and right feet.

[0076] Ev2 is a value related to the positional relationship between the left and right feet shown in the second example. As an example, if the X coordinate or Y coordinate of the panel on which the left and right feet are placed in the XY coordinate system matches, it is 1, and if they do not match, it is a value obtained as a predefined fixed value other than 1. Ev2 is a discrete value that is 1 or a fixed value.

[0077] Ev3 is a value related to the body orientation shown in the third example. As an example, it is a value calculated using a normal distribution with a maximum at 0° using the angle θ3. Ev3 is a continuous value greater than 0 and less than or equal to 1.

[0078] 2, the arithmetic device 1 may be connected to a display 14. The display 14 is an example of an output unit for outputting the calculation results, such as a series of actions in response to a group of user instructions. This allows the player to know the posture changes estimated by the arithmetic device 1.

[0079] The arithmetic device 1 may also be connected to an operation device 15. The operation device 15 is an example of an input unit for receiving input of user attributes, such as a touch pen, a mouse, or a touch panel. The user attributes are attributes related to the player's dance game, and include at least one of dominant foot, physique, proficiency, movement habits, posture characteristics, injury location, etc. This enables the adjustment process described below.

[0080] The calculation processing executed by the calculation unit 11 includes a path determination processing 111. The path determination processing 111 includes using a model 122 to determine a path of node transition suitable for a group of instructions included in the music score data D. By using the model 122 that outputs a user's posture change when music score data is given as input, it becomes possible to easily obtain the user's posture change.

[0081] Preferably, in the path determination process 111, the calculation unit 11 determines a path for node transition using a judgment value. The judgment value is a value obtained based on at least one of the transition probability, the evaluation value, and the degree of agreement between the action specified by the node and the action specified by the instruction. The calculation unit 11 can, for example, determine the path with the largest judgment value as the path to be adopted.

[0082] The degree of agreement with the instruction is expressed as a value greater than 0, which indicates a complete mismatch, and less than 1, which indicates a complete match, and is a value obtained for each node for each instruction. When the model 122 is a hidden Markov model, the degree of agreement with the instruction is used to obtain an output probability for each node by further considering the evaluation value for each node. For example, the output probability is a value obtained by multiplying the evaluation value by the degree of agreement.

[0083] As an example, in the path determination process 111, the calculation unit 11 determines a path for a node in response to a group of instructions consisting of multiple instructions, using at least one of the output probability calculated from the degree of agreement of each node obtained in response to the instructions, the evaluation value of each node, and the transition probability of the edge from the current node to each node. By using the evaluation value of the node, it is possible to determine a path taking into account the ease of posture. By using the transition probability, it is possible to determine a path according to the ease of transition. By using the output probability, it is possible to determine a path to an action that is easy to assume a posture among actions that are close to the instructions.

[0084] The judgment value may be, for example, a value obtained by scoring all possible paths for a set of instructions using at least one of the output probability, the evaluation value, and the transition probability. That is, the judgment value may be a value obtained by substituting the evaluation values ​​of the nodes corresponding to each of the instructions into an arithmetic expression with the evaluation value as a variable. By using this judgment value, the path is determined with an emphasis on the ease of assuming a posture to follow the instructions.

[0085] The judgment value may be a value obtained by substituting the transition probability of the edge corresponding to each of the instructions into an arithmetic expression with the transition probability as a variable. By using this judgment value, a path that is likely to lead to a transition of the action according to the instructions is determined.

[0086] The judgment value may be a value obtained by substituting the output probability of the node corresponding to each of the instructions into an arithmetic expression with the output probability as a variable. By using this judgment value, a path that makes it easier to adopt a posture that follows the instructions and performs actions that are close to the instructions is determined.

[0087] The judgment value may also be a combination thereof. As an example, the calculation unit 11 uses a value obtained by substituting the output probability, evaluation value, and transition probability for the nodes and edges corresponding to each of the multiple instructions into an arithmetic expression in which the output probability, evaluation value, and transition probability are variables, as the judgment value. By using this judgment value, a path is determined that makes it easier to adopt a posture that follows the instructions, makes it easier to transition in movement according to the instructions, and performs movements that are close to the instructions.

[0088] The judgment value may be a value obtained by adding up the products of the output probability, evaluation value, and transition probability for each instruction for all instructions in the group. Another example of the judgment value may be a value obtained by multiplying the output probability, evaluation value, and transition probability for all instructions. This makes it possible to determine the path for the group of instructions that has the highest value obtained by the output probability, evaluation value, and transition probability, i.e., the series of actions corresponding to the group of instructions.

[0089] Preferably, the path determination process 111 includes an adjustment process 112. The adjustment process 112 includes adjusting at least one of the transition probability and the evaluation value. The adjustment process 112 is performed, for example, in accordance with user attributes. For example, the user attributes are attributes of the player related to the dance game that are input from the operation device 15.

[0090] Adjusting at least one of the transition probability and the evaluation value may involve, for example, applying a specific parameter based on a user attribute to at least one of the transition probability and the evaluation value.

[0091] When adjusting the transition probability, in the case of a transition probability related to the movement direction of the foot, the intrinsic parameter may be the variance of a normal distribution of values ​​according to the movement angle θ1. In the case of a transition probability related to the movement amount of at least a part of the body or the change amount of the angle of the body, the intrinsic parameter may be a weight. In the case of a transition probability related to the presence or absence of a specific motion, the intrinsic parameter may be a fixed value. This allows the transition probability to be a value according to the user attributes.

[0092] When adjusting the evaluation value, if the evaluation value is related to the degree of grip on the panel on which the feet are placed, the intrinsic parameter may be the variance of a normal distribution of values ​​corresponding to the degree of grip on the panel. If the evaluation value is related to the positional relationship between the left and right feet, the intrinsic parameter may be a fixed value. If the evaluation value is related to the orientation of the body, the intrinsic parameter may be the variance of a normal distribution of values ​​corresponding to the angle θ3 between the orientation of the face and the orientation of the body, or a limit value. This allows the evaluation value to be set to a value corresponding to the user attributes, or to 0 if it exceeds the limit value corresponding to the user attributes.

[0093] By adjusting at least one of the transition probabilities and the evaluation value according to user attributes, the path can be determined according to the player's attributes related to the dance game, such as dominant foot, physique, skill level, movement habits, posture characteristics, injury locations, etc. As a result, a path suited to the player is determined, making it easier for the player to perform a series of movements.

[0094] As another example, adjustment process 112 is performed in response to user input. For example, when dance game device 5 is used for training such as rehabilitation, the user may input the load to be applied to the player, and adjustment process 112 may be performed in response to the input load. In this way, a path according to the set load is determined, allowing the player to perform effective movements.

[0095] As another example, the adjustment process 112 may include adjusting a threshold value of the score for determining a route when scoring each route using a judgment value in the route determination process 111. As an example, in the route determination process 111, the calculation unit 11 scores each path using all of the output probability, evaluation value, and transition probability, and determines that the route to be adopted is one whose obtained score is within the range adjusted by the adjustment process 112. This allows a route to be determined with a score according to user attributes, etc.

[0096] Preferably, the calculation process executed by the calculation unit 11 includes evaluation process 113. Evaluation process 113 includes evaluating the music score data D using a judgment value, and specifically, may be outputting an evaluation value. One example of the evaluation value is a numerical value based on the judgment value. The numerical value based on the judgment value is, for example, the judgment value itself. Another example of the numerical value based on the judgment value may be a value obtained by comparing the judgment value with a threshold value.

[0097] As another example, evaluating the music score data D may involve selecting music score data D suitable for the player based on the evaluation value from among multiple pieces of music score data D for the specified content. This allows the player to use music score data D that allows for easy posture changes, such as music score data D that corresponds to the player's level of proficiency.

[0098] 5 is a flowchart showing an example of a calculation method according to this embodiment, which is a method for estimating a change in a player's posture in response to a group of instructions from dance game device 5. The calculation method shown in the flowchart in FIG. 5 is a method employed in the process of determining a transition path of nodes in calculation device 1 in response to a group of instructions from dance game device 5.

[0099] 5, the arithmetic device 1 acquires the music score data D from the music score data generating device 9 (step S101). The arithmetic device 1 also receives input of the user attributes of the player (step S103).

[0100] The calculation unit 11 of the calculation device 1 refers to the model 122 in the memory 12 and adjusts the evaluation value included in each node according to the input user attribute (step S105). Also, the calculation unit 11 adjusts the transition probability included in each edge according to the input user attribute (step S107). Also, for each instruction in the instruction group included in the music score data D, the calculation unit 11 calculates the output probability using the degree of agreement for all nodes (step S109).

[0101] The calculation unit 11 extracts node transitions that can be taken according to each instruction, calculates a judgment value for each transition path obtained by combining the extracted transitions (step S111), and extracts the transition path with the largest judgment value from among the extracted transition paths (step S113).

[0102] When the calculation unit 11 of the calculation device 1 executes the path determination process 111, the calculation device 1 obtains an output such as that shown in Fig. 6. The output of Fig. 6 may be output using the display 14 or the like. This allows the player to perform a series of actions in accordance with the obtained output.

[0103] Fig. 6 is a diagram showing an example of a transition path determined by the arithmetic device 1. Instructions 61 to 65 in Fig. 6 represent instructions obtained from the music score data D, and a series of these instructions 61 to 65 are given as an instruction group to the arithmetic device 1. The instructions 61 to 65 are in the form of arrows and represent the first panel 31 to the fifth panel 35.

[0104] Outputs 41 to 45 in Fig. 6 are the calculation results of the player's movements corresponding to instructions 61 to 65, respectively. The successive outputs 41 to 45 represent the change in the player's posture in response to the group of instructions 61 to 65. Using parameters PP1 to PP5, the outputs 41 to 45 represent the foot to be placed on each of the first to fifth panels 31 to 35, the foot performing the stepping motion, and the orientation of the body.

[0105] Specifically, parameter PP1 represents the foot placed on the third panel 33, parameter PP2 represents the foot placed on the first panel 31, parameter PP3 represents the foot placed on the fifth panel 35, parameter PP4 represents the foot placed on the second panel 32, and parameter PP5 represents the foot placed on the fourth panel 34. R (uppercase R) or r (lowercase R) represents the right foot, L (uppercase L) or l (lowercase L) represents the left foot, R or L (uppercase) represents the foot that performs a stepping motion, and r or l (lowercase) represents the foot that does not require a stepping motion. A circle with a triangle indicates the player's center of gravity. The triangle represents the orientation of the body. An upward-pointing triangle indicates facing forward, a right-pointing triangle indicates facing right, a left-pointing triangle indicates facing left, and a downward-pointing triangle indicates facing backward.

[0106] 6, in response to instruction 61, output 41 indicates an action to face the body forward, place the left foot on first panel 31 and the right foot on second panel 32, and step with the right foot. By following this output 41, second panel 32 is stepped on with the right foot, and instruction 61 is fulfilled.

[0107] In response to instruction 62, output 42 indicates the movement of turning the body to the left, placing the right foot on third panel 33 and the left foot on fourth panel 34, and stepping on both ends. By following this output 42, third panel 33 is stepped on with the right foot, and instruction 62 is fulfilled.

[0108] In response to instruction 63, output 43 indicates the action of stepping with the left foot while maintaining the body facing left and keeping the right foot on third panel 33 and the left foot on fourth panel 34. By following this output 43, the fourth panel 34 is stepped on with the left foot, and instruction 63 is fulfilled.

[0109] In response to instruction 64, output 44 indicates the action of placing both feet on third panel 33 and stepping with both feet. By following this output 44, third panel 33 is stepped on with both feet, and instruction 64 is fulfilled.

[0110] In response to instruction 65, output 45 indicates the action of placing both feet on fifth panel 35 and stepping with both feet. By following this output 45, fifth panel 35 is stepped on with both feet.

[0111] In response to instruction 66, output 46 indicates the action of placing the left foot on first panel 31 and the right foot on second panel 32, and stepping with both feet. By following this output 46, the first panel 31 and the second panel 32 are both stepped on, and instruction 66 is fulfilled.

[0112] 6, the action following the output 45 for the instruction 65 is not an action that satisfies the instruction 65. In this way, the node transition path determined by the calculation device 1 may not instruct an action that completely matches the instruction group. This is because, as an example, when the calculation unit 11 executes the path determination process 111, in the process of transitioning from instruction 64 to instruction 65 and from instruction 65 to instruction 66, the judgment value was higher for the path that passes through the node where both feet are placed on the fifth panel 35 and where both feet are stepped on, than for the path that passes through the node where both feet are stepped on the third panel 33 and the fourth panel 34, respectively, in accordance with instruction 65.

[0113] Determining such a path may result in the player performing a movement that is slightly different from the movement instructed by the group of instructions. However, determining such a path allows for a posture change that is easier to adopt and transition to, taking into account the previous and following postures, rather than a movement that completely matches the movement instructed by the group of instructions. Furthermore, by adjusting the user attributes, etc., as described above, it is possible to achieve a posture change that is appropriate for the user attributes, etc.

[0114] Another example of the method for calculating the evaluation value of a node will be described below. As another example, the evaluation value of a node may be calculated by taking into account the user attributes of each player in advance. In other words, as another example of the calculation of the evaluation value of a node, parameters unique to the player are used.

[0115] This is because whether or not a certain static posture is stable varies depending on factors such as the player's level of proficiency with the game. For example, it is known that the fourth panel 34, which is located opposite the display 51, is particularly difficult for beginner gamers who need to check their feet. Therefore, in the case of such user attributes, the evaluation value of a node whose feet are on the fourth panel 34 should be lower than the evaluation value of a node whose feet are not on the fourth panel 34. On the other hand, an advanced player who is accustomed to the game can easily step on the fourth panel 34. Therefore, in the case of such user attributes, the evaluation value of a node whose feet are on the fourth panel 34 does not need to be significantly different from the evaluation values ​​of the other nodes.

[0116] In another example, the evaluation value of the node is calculated using, for example, an intrinsic parameter PG_σ 2 ,PG_s,PG_w,PT,Pθ_σ 2 , Pθ_w, Pθ_l are used. These inherent parameters are set based on the user information of the player. These inherent parameters are used to calculate the evaluation value of the node.

[0117] Specifically, the intrinsic parameter PG_σ 2 , PG_s, PG_w are examples of intrinsic parameters for evaluating the degree of recognition of the foot positions for both the left and right feet, and the intrinsic parameters PG_σ 2 is an intrinsic parameter that represents the variance of the normal distribution. 2 is a value greater than 0, and the larger the value, the higher the degree of grasp for any position. The intrinsic parameter PG_s is an intrinsic parameter that represents the minimum value of the degree of grasp for the foot position. The intrinsic parameter PG_s is a value greater than 0 and less than 1. The intrinsic parameter PG_w is an intrinsic parameter that represents the maximum value of the degree of grasp for the foot position. The intrinsic parameter PG_w is a value greater than 0 and less than 1.

[0118] The intrinsic parameter PT is an example of an intrinsic parameter for evaluating the mismatch of the positions of the feet, and is an intrinsic parameter that represents the stability in a posture in which the axes of the feet do not match. The intrinsic parameter PT is a value greater than 0 and less than or equal to 1.

[0119] Intrinsic parameter Pθ_σ 2 , Pθ_w, Pθ_l are examples of intrinsic parameters for evaluating the influence of the body orientation θ, and the intrinsic parameters Pθ_σ 2 is an intrinsic parameter that represents the variance of the normal distribution. 2 is a value greater than 0, and the larger the value, the more stable the posture is in any orientation. The intrinsic parameter Pθ_w is an intrinsic parameter that represents the maximum influence of the body orientation θ. The intrinsic parameter Pθ_w is a value greater than or equal to 0 and less than or equal to 1. The intrinsic parameter Pθ_l is an intrinsic parameter for evaluating the influence of the body orientation θ, and represents the limit value of the angle at which the neck can be rotated. The intrinsic parameter Pθ_l is a value greater than or equal to 0° and less than or equal to 360°.

[0120] In another example, the evaluation values ​​of the following first to fourth items are used as factors that affect the stability of the static posture, and an equation for obtaining the evaluation value of node n is defined. Figure 7 shows the definition equations for the evaluation values ​​of the first to fourth items. The evaluation values ​​of the four items are generally calculated by the player as follows: 1) The first panel 31 is the easiest to grasp, and the fourth panel 34 is the hardest to grasp. 2) The posture of having the feet on a panel that does not properly grasp its position is unstable. 3) A posture in which one foot is on either the third panel 33 or the fourth panel 34 and the other foot is on either the first panel 31 or the second panel 32 is unstable. 4) The greater the neck rotation angle, the more unstable the posture, and there is a limit to that angle. It is set based on the assumption that

[0121] The evaluation value for the first item is the left foot position recognition level S_ln, which indicates the degree to which the player recognizes the position of the panel on which the left foot is placed. The left foot position recognition level S_ln is defined as shown in equation (3) in Fig. 7 using the smaller angle α_l between the front direction (the y-axis direction in the XY coordinate system (first coordinate system)) facing dance game device 5 and the vector (left foot position vector) pointing from the origin pointing to the position of the left foot when the center of fifth panel 35 is the origin.

[0122] The left foot position knowledge level S_ln is greater than the value of the intrinsic parameter PG_s and less than or equal to the value of the intrinsic parameter PG_w. The larger the value of the left foot position knowledge level S_ln, the more accurately the left foot is grasped. Therefore, the larger the value of the left foot position knowledge level S_ln, the more it contributes to increasing the evaluation value of the node.

[0123] The second evaluation value is the right foot position recognition level S_rn, which indicates the degree to which the player has a grasp of the position of the panel on which the right foot is placed. The right foot position recognition level S_rn is defined as shown in equation (4) in Fig. 7 using the smaller angle α_r between the front direction facing the dance game device 5 and the vector from the origin pointing toward the right foot position (right foot position vector).

[0124] The right foot position knowledge level S_rn is greater than the value of the intrinsic parameter PG_s and less than or equal to the value of the intrinsic parameter PG_w, and the larger the value, the more accurately the position is understood. Therefore, the larger the value of the right foot position knowledge level S_rn, the more it contributes to increasing the evaluation value of the node.

[0125] The third evaluation value is the mismatch S_tn of the axes of both feet, which indicates the degree of stability in a posture in which one foot is on the x-axis and the other foot is on the y-axis in an XY coordinate system (first coordinate system). The mismatch S_tn of the axes of both feet is defined as in equation (5) of FIG. 7. As shown in equation (5), the mismatch S_tn of the axes of both feet is either the value of the intrinsic parameter PT or 1.

[0126] The mismatch of both feet's axes S_tn indicates that the corresponding posture is more stable as the value approaches 1 from 0. Therefore, the mismatch of both feet's axes S_tn contributes to increasing the evaluation value of the node as the value increases.

[0127] The fourth evaluation value is the influence of body orientation S_θn, which is an evaluation value indicating the degree of stability of posture at body orientation θn. The influence of body orientation S_θn is defined as equation (6) in Figure 7. The influence of body orientation S_θn is equal to or greater than 0 and equal to or less than the value of the intrinsic parameter Pθ_w.

[0128] The larger the value of the body orientation influence S_θn, the more stable the posture is at that body orientation θ. Therefore, the larger the value of the body orientation influence S_θn, the more it contributes to increasing the evaluation value of the node.

[0129] In this example, the evaluation value En of node n is calculated by the following formula (7) using the evaluation values ​​of the first to fourth items, for example. That is, the evaluation value En is obtained by the total product of the evaluation values ​​of the first to fourth items. En=S_ln×S_rn×S_tn×S_θn …(7) As a result, the evaluation value En of the node n according to the other example takes into consideration user attributes such as the player's game proficiency.

[0130] Another example of a method for calculating the transition probability of an edge will now be described. The transition probability of an edge may also be calculated by taking into consideration the user attributes of each player in advance, similar to the evaluation value of a node. That is, another example of calculating the transition probability of an edge uses parameters specific to each player.

[0131] This is because the ease of posture change also varies depending on the player's game proficiency and movement preferences. For example, a player who is right-footed prefers movements with their right foot. Therefore, the transition probability of an edge corresponding to a movement that moves the right foot significantly should be higher than the transition probability of a movement that moves the left foot significantly.

[0132] In another example, the transition probability of an edge is calculated using, for example, an intrinsic parameter PM_σ 2 , PM_s, PM_w, PL, PR, PO, PA, PJ, and PS are used. These intrinsic parameters are each set based on the player's user information. In another example, these intrinsic parameters are used to set the evaluation value of the edge itself for the player, and the transition probability of the edge is calculated based on the relationship with the evaluation values ​​of other edges. The evaluation value of an edge is an evaluation value of the transition of an action from the start node to the end node, and as an example, the higher the value, the easier the transition, and the lower the value, the more difficult the transition.

[0133] Intrinsic parameter PM_σ 2 , PM_s, PM_w are examples of intrinsic parameters for evaluating the degree of understanding of the movement direction of the left and right feet, and the intrinsic parameters PM_σ 2 is an intrinsic parameter that represents the variance of the normal distribution. 2 is a value greater than 0, and the larger the value, the higher the degree of grasp in any direction. The intrinsic parameter PM_s is an intrinsic parameter that represents the minimum value of the degree of grasp in the direction of foot movement. The intrinsic parameter PM_s is a value greater than 0 and less than 1. The intrinsic parameter PM_w is an intrinsic parameter that represents the maximum value of the degree of grasp in the direction of foot movement. The intrinsic parameter PM_w is a value greater than 0 and less than 1.

[0134] The intrinsic parameter PL is an example of an intrinsic parameter for evaluating the movement distance of the left foot, and is an intrinsic parameter that represents the weight of the movement distance of the left foot. The intrinsic parameter PL is a value greater than 0.

[0135] The intrinsic parameter PR is an example of an intrinsic parameter for evaluating the movement distance of the right foot, and is an intrinsic parameter that represents the weight of the movement distance of the right foot. The intrinsic parameter PR is a value greater than 0.

[0136] The intrinsic parameter PO is an example of an intrinsic parameter for evaluating the movement distance of the center of gravity, and is an intrinsic parameter that represents the weight of the movement distance of the center of gravity. The intrinsic parameter PR is a value greater than 0.

[0137] The intrinsic parameter PA is an example of an intrinsic parameter for evaluating the amount of change in the body orientation θ, and is an intrinsic parameter that represents the weight of the amount of change in the body orientation θ. The intrinsic parameter PA is a value greater than 0.

[0138] The intrinsic parameter PJ is an example of an intrinsic parameter for evaluating a movement performed with both feet simultaneously, and is an intrinsic parameter that represents resistance to a jumping movement. The intrinsic parameter PJ has a value of 1 or greater.

[0139] The intrinsic parameter PS is an example of an intrinsic parameter for evaluating foot replacement on the same panel, and is an intrinsic parameter that represents resistance to the foot replacement action on the same panel. The intrinsic parameter PS has a value of 1 or greater.

[0140] In another example, the evaluation values ​​of the following items 1 to 8 are used as factors that affect the probability of transition of an action, and a formula is defined to obtain the transition probability Pe of the edge e connecting the start node to the end node. Figure 8 shows the definition formulas for each of the evaluation values ​​of items 1 to 8, and the definition formula for the transition probability Pe using these evaluation values. The evaluation values ​​of the eight items are, for a typical player, 1) Moving your feet forward is a natural movement, but moving your feet backward is a difficult movement. 2) The more the feet and center of gravity move, the more difficult the movement. 3) Movements that involve large changes in body orientation are difficult. 4) Jumping with both feet is more difficult than jumping with one foot at a time. 5) Repeatedly hitting the same panel with both feet is a difficult task. Regarding 5), if the foot that stepped first is not lifted away from the panel, it is difficult to detect that the subsequent foot has stepped on the panel again.

[0141] The evaluation value of the first item is the left foot movement direction knowledge degree M_dle, which is an evaluation value indicating the player's degree of confidence in the direction in which to move the left foot. The left foot movement direction knowledge degree M_dle is defined as shown in equation (8) in Figure 8 using a vector yp = (0, 1) in the positive y-axis direction in an xy coordinate system (second coordinate system) based on the player's body, a left foot position vector l(e, s) = (xp_ls, yp_ls) at the start node, a left foot position vector l(e, g) = (xp_lg, yp_lg) at the end node, and the smaller angle βl between the vector yp and the vector (l(e, g) - l(e, s)).

[0142] The left foot movement direction knowledge degree M_dle is greater than the value of the intrinsic parameter PM_s and less than or equal to the value of the intrinsic parameter PM_w. The larger the value of the left foot movement direction knowledge degree M_dle, the more confident the player is in moving the left foot in that direction. Therefore, the larger the value of the left foot movement direction knowledge degree M_dle, the more it contributes to increasing the evaluation value of the edge itself.

[0143] The evaluation value of the second item is the right foot movement direction knowledge degree M_dre, which is an evaluation value indicating the player's degree of confidence in the direction of moving the right foot. The right foot movement direction knowledge degree M_dre is defined as shown in equation (9) in Figure 8 using a vector yp = (0, 1) in the positive y-axis direction in the second coordinate system, a right foot position vector r(e, s) = (xp_rs, yp_rs) at the start node, a left foot position vector r(e, g) = (xp_rg, yp_rg) at the end node, and the smaller angle βr between the vector yp and the vector (r(e, g) - r(e, s)).

[0144] The right foot movement direction knowledge degree M_dre is greater than the value of the intrinsic parameter PM_s and less than or equal to the value of the intrinsic parameter PM_w. The greater the value of the right foot movement direction knowledge degree M_dre, the more confident the player can move their right foot in that direction. Therefore, the greater the value of the right foot movement direction knowledge degree M_dre, the more it contributes to increasing the evaluation value of the edge itself.

[0145] The evaluation value of the third item is the left foot movement distance M_mle, which is an evaluation value indicating the movement distance of the left foot measured in the XY coordinate system (first coordinate system) that serves as the basis for instructions from the dance game device 5. The left foot movement distance M_mle is defined as shown in equation (10) of Fig. 8 using the coordinates of the left foot at the start node (xg_ls, yg_ls) and the coordinates of the left foot at the end node (xg_lg, yg_lg) in the first coordinate system.

[0146] The larger the value of the left foot movement distance M_mle, the more difficult the transition of movement becomes. Therefore, a larger value of the left foot movement distance M_mle contributes to increasing the evaluation value of the edge itself.

[0147] The evaluation value of the fourth item is the right foot movement distance M_mre, which is an evaluation value indicating the movement distance of the right foot measured in the first coordinate system. The right foot movement distance M_mre is defined as shown in equation (11) in Fig. 8 using the coordinates (xg_rs, yg_rs) of the right foot at the start node and the coordinates (xg_rg, yg_rg) of the right foot at the end node in the first coordinate system.

[0148] The larger the value of the right foot movement distance M_mre, the more difficult the transition of movement becomes. Therefore, a larger value of the right foot movement distance M_mre contributes to increasing the evaluation value of the edge itself.

[0149] The fifth evaluation value is the center of gravity movement distance M_moe, which is an evaluation value indicating the movement distance of the center of gravity measured in the first coordinate system. The center of gravity movement distance M_moe is defined as shown in equation (12) in Figure 8 using the coordinates (xg_os, yg_os) of the center of gravity at the start node and the coordinates (xg_og, yg_og) of the center of gravity at the end node.

[0150] The larger the value of the center of gravity movement distance M_moe, the more difficult the transition of motion becomes. Therefore, the larger the value of the center of gravity movement distance M_moe, the more it contributes to increasing the evaluation value of the edge itself.

[0151] The sixth evaluation value is the body orientation change amount M_mθe, which is an evaluation value indicating the change amount of the body orientation θ. The body orientation change amount M_mθe is defined as shown in equation (13) in Fig. 8 using the body orientation θs at the start node and the body orientation θg at the end node.

[0152] The larger the body orientation change amount M_mθe, the more difficult the transition of motion becomes. Therefore, a larger body orientation change amount M_mθe contributes to increasing the evaluation value of the edge itself.

[0153] The seventh item, the evaluation value of the simultaneous two-foot action M_je, is an evaluation value that indicates the player's confidence in the action that causes the state of pressing the panel with both feet at the same time, i.e., the jump action. The simultaneous two-foot action M_je is defined as equation (14) in Figure 8 using the number p of panel pressing actions occurring at the end node.

[0154] The simultaneous two-foot action M_je is either 1 or the value of the intrinsic parameter PJ, as shown in equation (14). The closer the value of the simultaneous two-foot action M_je is to 1, the more confident the player is in the action, and the larger the value is from 1, the less confident the player is. Therefore, the larger the value of the simultaneous two-foot action M_je is, the more it contributes to increasing the evaluation value of the edge itself.

[0155] The index value of the eighth item is the foot swap on the same panel M_se, which is an evaluation value indicating the degree of confidence of the player in an action in which one foot is on the panel on which the other foot is on at the end node. The foot swap on the same panel M_se is defined as shown in equation (15) in Figure 8 using the coordinates (xg_ls, yg_ls) of the left foot at the start node, the coordinates (xg_rs, yg_rs) of the right foot at the start node, and the coordinates (xg_lg, yg_lg) and coordinates (xg_rg, yg_rg) of the left foot at the end node in the first coordinate system.

[0156] As shown in equation (15), the foot swap M_se on the same panel is either 1 or the value of the intrinsic parameter PS. The closer the value of the foot swap M_se on the same panel is to 1, the more confident the player is in the corresponding action, and the larger the value is from 1, the less confident the player is. Therefore, a larger value of the foot swap M_se on the same panel contributes to increasing the evaluation value of the edge itself.

[0157] In this example, the transition probability Pe of edge e is calculated by equation (17) using the evaluation values ​​of the first to eighth items and the evaluation value Me of the edge obtained by equation (16) in FIG. 8. In equation (16), the evaluation value Me of the edge is expressed using the index value Te of the difficulty of the motion transition expressed by equation (18) below. From equation (16), the lower the index value Te of the difficulty of the motion transition, the higher the evaluation value Me of the edge. The index value Te is an index value obtained by summing up the values ​​of the evaluation values ​​of the first to eighth items of edge e in the direction that decreases the transition probability. Te=-M_dle-M_dre+M_mle+M_mre+M_moe+M_mθe+M_je+M_se …(18)

[0158] The transition probability Pe of edge e is calculated from equation (17) as the ratio of the evaluation value Me of edge e to the sum of the evaluation values ​​M of a set E of edges that includes edge e and has the same start node as edge e. As a result, the transition probability Pe of edge e in another example takes into account user attributes such as the player's game proficiency.

[0159] The arithmetic device 1 according to the embodiment can further evaluate the music score data D by outputting a value based on the obtained judgment value as an evaluation value. This makes it possible to objectively indicate the ease of assuming a posture for a series of actions in accordance with a group of instructions, the ease of transitioning between actions, and the like, i.e., the difficulty level of the music score data D, using numerical values. Furthermore, by adjusting the user attributes, etc., as described above when obtaining the judgment value, it is possible to evaluate whether a posture change in accordance with a group of instructions is difficult or easy for the user, i.e., the difficulty level for the user, using an objective value.

[0160] As another example, when content such as music is selected, the calculation device 1 may select suitable score data D based on the evaluation value from among multiple score data D corresponding to the content and acquired from the score data generation device 9. Suitable score data D may be, for example, score data with the highest judgment value for the difficulty of posture change, score data with the lowest judgment value, or score data with a judgment value within a range preset for the user attributes. This allows suitable score data D to be selected objectively.

[0161] Fig. 9 is a diagram showing an example of an input musical score. S1 to S6 in Fig. 9 correspond to the first to sixth steps in a series of actions performed according to the input musical score. Fig. 10 is a diagram showing posture changes estimated by a calculation device when the user attributes are changed in the musical score of Fig. 9. S1 to S6 in Fig. 10 represent the actions of the first to sixth steps corresponding to S1 to S6 in the input musical score of Fig. 9.

[0162] In FIG. 10, the movements of the first to sixth steps are represented by parameters PP1 to PP5, similar to FIG. 6, which represent the foot placed on the first to fifth panels 31 to 35, the foot performing the stepping motion, and the direction of the body. That is, in FIG. 10, parameter PP1 represents the foot placed on the third panel 33, parameter PP2 represents the foot placed on the first panel 31, parameter PP3 represents the foot placed on the fifth panel 35, parameter PP4 represents the foot placed on the second panel 32, and parameter PP5 represents the foot placed on the fourth panel 34. R (uppercase R) or r (lowercase R) indicates the right foot, L (uppercase L) or l (lowercase L) indicates the left foot, R or L (uppercase) represents the foot performing the stepping motion, and r or l (lowercase) represents the foot not requiring the stepping motion. The circle with a triangle indicates the player's center of gravity. The triangle represents the direction of the body. An upward-facing triangle indicates facing forward, a right-facing triangle indicates facing right, a left-facing triangle indicates facing left, and a downward-facing triangle indicates facing backward.

[0163] FIG. 10 shows schematic diagrams of the "standard" model, "right" model, "beginner" model, and "perfect" model for the music score in FIG. 9. The "standard" model is a reference model. The "right" model is a model designed for right-footed players. The "beginner" model is a model designed for beginner players. The "perfect" model is a model that is open to all physical movements.

[0164] As shown in FIG. 10, the "standard" model's right foot does not move from the second panel 32 in steps 1 to 6. On the other hand, the "right" model's left foot does not move from the third panel 33 in steps 3 to 5. Furthermore, in step 6 of the "right" model, the right foot moves to the first panel 31. The "beginner" model often steps 3 to 5, stepping on the same panel with both feet simultaneously. This is thought to be because the model estimated the stepping motion of stepping on the fourth panel 34 and the third panel 33 with the body angle at 0°. The "perfect" model performs a jump in step 4. This motion is thought to be an estimation intended to move the left foot onto the first panel 31 in advance of the instruction to step on the first panel 31 in step 6. As described above, the calculation method according to this embodiment allows for estimation of posture changes suited to the player by changing the user attributes.

[0165] <3. Notes> This invention automatically determines whether the created musical score is actually playable. It also determines how to move the body in a natural way without straining or falling. It also makes it possible to create games tailored to physical parameters. As a result, in the entertainment field, it can provide reference material for creating content such as music games. Furthermore, the present invention is not limited to the above-described embodiment, and various modifications are possible. For example, the present invention can be applied not only to dance games that involve stepping, but also to dance games that involve using the hands, Twister, bouldering, and other games. Furthermore, the present invention is not limited to games, and can also be applied to rehabilitation therapy, etc. By applying the present invention to rehabilitation therapy, it is possible to propose exercise content and exercise load that take physical constraints into account. [Explanation of symbols]

[0166] 1: Arithmetic device 5: Dance game device 9: Music score data generator 11: Calculation unit (processor) 12: Memory 14: Display 15: Operating device 31: First panel 31a: Left Arrow 32: Second panel 32a: Right Arrow 33: Third panel 33a: forward arrow 34: Fourth Panel 34a: Backward Arrow 35: 5th panel 41: Output 42: Output 43: Output 44: Output 45: Output 46: Output 50: Controller 51: Display 51a:Display surface 52: Speaker 53: Operation section 61: Instructions 62: Instructions 63: Instructions 64: Instructions 65: Instructions 66: Instructions 111: Route determination processing 112: Adjustment process 113: Evaluation process 121: Computer Programs 122: Model 122A: Model A: Arrow D: Music score data I1: Instructions I2 :Instruction I3: Instructions LM: Travel amount M: Music M1: Travel route N1: Node N2: Node N3: Node N4: Node N5: Node N6: Node N7: Node N8: Node N9: Node N10: Node N11: Node N12: Node N13: Node P: Player P1: Parameter P2: Parameter P3: Parameter PP1: Parameter PP2: Parameters PP3: Parameters PP4: Parameters PP5: Parameters θ1: Movement angle θ2: Amount of rotation θ3: Angle

Claims

1. A method for performing a game comprising: providing a computer-readable recording medium having a computer program for performing a game according to claim 1; and using a computer-readable recording medium for performing a game according to claim 1; the model has a plurality of nodes and a plurality of edges connecting the nodes, each of the plurality of nodes defining a posture of the user and a movement in the posture; estimating the change in posture of the user includes determining a transition path of the node suitable for the group of instructions using a judgment value, thereby determining a change in posture of the user with respect to the group of instructions; The judgment value is a value obtained based on the degree of agreement indicating the degree of agreement / inconsistency between the movement in the posture defined by the node and the movement defined by the instruction. Computing device.

2. The judgment value is a value obtained based on at least one of the transition probability of the edge and the evaluation value of the node. The computing device of claim 1 .

3. The transition probability is expressed as a value indicating the likelihood of the change in the posture defined by each of the nodes between the nodes. The computing device according to claim 2 .

4. The transition probability is obtained based on at least one of a direction of movement of at least a part of the body between the nodes, an amount of movement, an amount of change in angle of the body, and the presence or absence of a specific motion in at least one of a first coordinate system that is a reference for the instruction and a second coordinate system that is defined according to the posture of the user. The computing device according to claim 3 .

5. The evaluation value is expressed as a value indicating the ease of taking the posture defined by the node. The computing device according to any one of claims 2 to 4.

6. The evaluation value is obtained based on at least one of the degree of understanding of the position of at least a part of the body in the posture defined by the node, the positional relationship of a plurality of body parts, and the orientation of the body in the coordinate system that is the reference for the instruction. The computing device according to claim 5 .

7. The process of determining the route includes a process of adjusting at least one of the transition probability and the evaluation value. The computing device according to any one of claims 2 to 6.

8. The adjusting process includes adjusting at least one of the transition probability and the evaluation value according to a user attribute. The computing device according to claim 7.

9. The calculation unit and further performing a process of evaluating the set of instructions using the determination value. The computing device according to any one of claims 1 to 8.

10. A calculation unit configured to estimate a user's posture change in response to music score data using a model that outputs a user's posture change when music score data is given as input, the model has a plurality of nodes that define the posture and movement of the user, and a plurality of edges that connect the nodes; estimating the change in the user's posture includes determining a path of transition of the nodes appropriate for an instruction group having a plurality of instructions for actions, which is included in the music score data, thereby determining a change in the user's posture in response to the instruction group; the calculation unit uses a determination value based on a transition probability of the edge in the process of determining the path, The transition probability is The posture change is expressed by a value indicating the ease of change of the posture defined by each of the nodes between the nodes, The coordinate system is obtained based on at least one of a direction of movement of at least a part of the body between the nodes, an amount of movement, an amount of change in angle of the body, and the presence or absence of a specific movement in at least one of a first coordinate system that is a reference for the instruction and a second coordinate system that is defined according to the posture of the user. Computing device.

11. A calculation unit configured to estimate a user's posture change in response to music score data using a model that outputs a user's posture change when music score data is given as input, the model has a plurality of nodes that define the posture and movement of the user, and a plurality of edges that connect the nodes; estimating the change in the user's posture includes determining a path of transition of the nodes appropriate for an instruction group having a plurality of instructions for actions, which is included in the music score data, thereby determining a change in the user's posture in response to the instruction group; the calculation unit uses a judgment value based on an evaluation value of the node in the process of determining the route, The evaluation value is is expressed by a value indicating the ease of taking the posture defined by the node, The information is obtained based on at least one of the degree of understanding of the position of at least a part of the body in the posture defined by the node, the positional relationship of a plurality of body parts, and the orientation of the body in the coordinate system that is the reference for the instruction. Computing device.

12. A calculation unit configured to estimate a user's posture change in response to music score data using a model that outputs a user's posture change when music score data is given as input, the model has a plurality of nodes that define the posture and movement of the user, and a plurality of edges that connect the nodes; estimating the change in the user's posture includes determining a path of transition of the nodes appropriate for an instruction group having a plurality of instructions for actions, which is included in the music score data, thereby determining a change in the user's posture in response to the instruction group; the calculation unit, in the process of determining the path, uses a determination value based on at least one of a transition probability of the edge and an evaluation value of the node, The process of determining the route includes a process of adjusting at least one of the transition probability and the evaluation value. Computing device.

13. The adjustment process includes adjusting at least the transition probability and the evaluation value according to a user attribute. This also includes adjusting the other The computing device of claim 12.

14. A calculation unit configured to estimate a user's posture change in response to music score data using a model that outputs a user's posture change when music score data is given as input, the model has a plurality of nodes that define the posture and movement of the user, and a plurality of edges that connect the nodes; estimating the change in the user's posture includes determining a path of transition of the nodes appropriate for an instruction group having a plurality of instructions for actions, which is included in the music score data, thereby determining a change in the user's posture in response to the instruction group; The calculation unit In the process of determining the path, a determination value based on at least one of a transition probability of the edge, an evaluation value of the node, and a degree of agreement between an action specified by the node and an action specified by the instruction is used, and further performing a process of evaluating the set of instructions using the determination value. Computing device.

15. A method for estimating a change in a user's posture in response to music score data including an instruction group having a plurality of movement instructions, comprising: The method comprises: estimating a change in the user's posture relative to the music score data using a model that outputs a change in the user's posture when the music score data is given as an input; the model has a plurality of nodes and a plurality of edges connecting the nodes, each of the plurality of nodes defining a posture of the user and a movement in the posture; estimating the change in posture of the user includes determining a transition path of the node suitable for the group of instructions using a judgment value, thereby determining a change in posture of the user with respect to the group of instructions; The judgment value is a value obtained based on the degree of agreement indicating the degree of agreement / inconsistency between the movement in the posture defined by the node and the movement defined by the instruction. Calculation method.

16. A computer program that causes a computer to perform a process of estimating a change in a user's posture in response to music score data that includes an instruction group having a plurality of movement instructions, The computer, using a model that outputs a posture change of the user when the music score data is given as an input, to estimate a posture change of the user with respect to the music score data; the model has a plurality of nodes and a plurality of edges connecting the nodes, each of the plurality of nodes defining a posture of the user and a movement in the posture; estimating the change in posture of the user includes determining a transition path of the node suitable for the group of instructions using a judgment value, thereby determining a change in posture of the user with respect to the group of instructions; The judgment value is a value obtained based on the degree of agreement indicating the degree of agreement / inconsistency between the movement in the posture defined by the node and the movement defined by the instruction. Computer program.

Images