Martial Arts Movement Recording Output System

Abstract

This system provides a mechanism for accurately and easily recording competitive movements in combat sports such as boxing, where striking is the primary method of attack, allowing athletes to efficiently recall those movements during shadow boxing or other practice sessions at a later date, thereby enhancing learning effectiveness. [Solution] A system was devised that uses musical notation, called "Kenfu," to record the movements of martial arts. The system inputs the movements during the competition through an interface, through image recognition of handwritten Kenfu, or through analysis of competition videos, and outputs the movements by reproducing them with 3D computer graphics and communicating them with sound.

Description

Technical Field

[0001] The present invention relates to the practice of combat sports that use strikes, such as boxing, as the main method of attack.

Background Art

[0002] Currently, the process of improving the competitive ability of those engaged in combat sports that use strikes, such as boxing, as the main method of attack depends on verbal communication in the gym and repetitive practice using one's own body. There is no written language in the world of combat sports. In order to efficiently learn and master competitive movements, it is desirable to establish a more efficient learning method that is not limited to primitive communication by word of mouth or body.

[0003] As a prior art, for the purpose of performing effective walking training for persons with walking disabilities, Japanese Patent Application Laid-Open No. 09-276348 focuses on the commonality between human walking movements and music in that periodicity, the length and strength of movements that appear continuously in time series have important meanings, encodes the walking rhythm using musical notes, and performs time-series reproduction of the encoded signals by a computer.

[0004] The above invention encodes using musical symbols for the physical movement of walking. Encoding using musical symbols in the movements of combat sports such as boxing, which have commonalities in terms of physical movements, can contribute to effective improvement of competitive ability. In combat sports such as boxing, it is necessary to perform movements that are difficult for the opponent to predict, and since the movements are completed in a short time, periodicity and the length of movements that appear continuously in time series do not have important meanings. However, on the other hand, the idea of originally capturing continuous sound waves as a collection of 12 discontinuous sounds as musical scales is common to combat sports in that it names movements with certain characteristics, such as jabs and straight punches, in the originally infinitely conceivable modes of strikes using hands and feet and simulates their identity. Therefore, the above invention can be developed by encoding the movements of combat sports as "punch scores" based on musical symbols.

Prior Art Documents

[0005] [Patent Document 1] Japanese Patent Application Publication No. 09-276348 [Overview of the project] [Problems that the invention aims to solve]

[0006] In striking-based martial arts, the specific meaning of acquiring techniques that contribute to improving competitive ability is mastering combinations. Athletes learn new combinations through opportunities such as hitting mitts with their instructors and self-study by watching videos outside of the training area, such as at home. However, since the number of combinations an athlete can accurately remember at one time is limited, a method for recording these combinations is necessary. One possible method for recording combinations is to take notes using natural language, but with this method, a considerable amount of information must be written to accurately record the movements, and when reading notes written in natural language, it is not possible to intuitively grasp the combinations written.

[0007] Given the widespread use of camera-equipped mobile devices today, it might seem possible to record combination movements as video. However, this presents several challenges: it requires the athlete or a third party to physically perform the movements, and it necessitates controlling a certain spatial area for filming. For example, if an athlete comes up with a new combination while traveling on a train and wants to use it in their next practice, they would need to record the movement on the train. However, actually performing batting or other movements on a train is impractical. It would also require the presence of a separate cameraman, but even if such a person exists, there may not be enough space to maintain a certain distance and film the movements with a camera. Furthermore, filming in public spaces can be difficult due to privacy concerns. Thus, there are limitations to the situations in which video recording of combinations can be used.

[0008] Furthermore, information about combinations recorded as video, which is a non-standard data format, has a larger data capacity compared to recording it as standard data. This limits the amount that can be recorded and also places a burden on information processing.

[0009] Furthermore, while combination information recorded as video can only be output as video, in striking-based martial arts, where observing the opponent is essential, it is impossible to simultaneously review combinations by watching the video and observe the opponent. Therefore, visually recorded combination information cannot be effectively utilized in the practice of striking-based martial arts. In addition, video is not ideal in terms of overall readability.

[0010] The invention described in Patent Document 1 above only represents two states: the feet touching the ground and the feet leaving the ground. Therefore, it cannot represent the diverse and complex movements of striking-based martial arts. Consequently, the invention described in Patent Document 1 above cannot solve the above-mentioned problems. [Means for solving the problem]

[0011] To solve the above problem, the invention described in claim 1 comprises encoding means for converting individual movements of a person performing martial arts techniques into individual code information including musical notes based on predetermined conversion rules, recording means for recording the code information, and motion output means for outputting an image of a person performing martial arts movements to a display means, wherein the motion output means outputs a video of a series of multiple movements of the martial arts based on the time-series output of the multiple code information recorded in the recording means.

[0012] The invention described in claim 2 is characterized in that, in addition to the configuration described in claim 1, it comprises a code output means for outputting the code information as an image of the code to the display means, wherein the code output means outputs the image of the code to the display means together with the person image, or separately from the person image.

[0013] The invention described in claim 3, in addition to the configuration described in claim 1 or 2, comprises a sound conversion means that converts the coded information into sound information based on a predetermined conversion rule, and a sound output means that outputs the sound information converted by the sound conversion means as a sound that can be heard by humans, wherein the sound conversion means converts each of the movements of the martial arts into a state of sound to be output, including the pitch, rhythm, and timbre of the sound.

[0014] The invention described in claim 4 is characterized in that it has the configuration described in claim 1, 2, or 3, wherein the martial art is boxing. [Effects of the Invention]

[0015] Through the aforementioned means, input of competitive movements can be achieved in just a few operations by providing an input interface that follows predetermined rules, thus enabling accurate and concise recording of competitive movements, which can be intuitively understood.

[0016] Furthermore, since this method allows for input of movements without actually using the body or controlling a certain spatial area, there is no need to consider the privacy of those around, and there are no restrictions on the situations in which it can be used.

[0017] By converting athletic actions into coded information, they can be recorded as standardized data. This requires significantly less data capacity compared to recording them in a non-standardized data format, thus enabling the storage of athletic actions with considerably lower capacity, eliminating the need to consider the quantitative constraints of storage media. Furthermore, processing standardized data places less information processing burden on the user compared to processing non-standardized data, which contributes to the analysis of athletic actions.

[0018] Since information about the actions performed during the competition is recorded as code, the output is not limited to playback as video; it is also possible to use sound for output that does not depend on visuals. Furthermore, by displaying the code information as an image according to predetermined rules, it is possible to provide output that is superior to viewing video in terms of overall readability.

Brief Description of Drawings

[0019] [Figure 1] It is a functional block diagram showing the configuration of the system. [Figure 2] It is an example of a sequence diagram showing the behavior of the system. [Figure 3] It is a conceptual diagram showing an example of a boxing score database. [Figure 4] It is a diagram showing an example of the correspondence between positions on a five-line musical score and punches in boxing. [Figure 5] It is a diagram showing an example of the rules of a boxing score. [Figure 6] It is a diagram exemplifying the expression of fighting techniques in a boxing score. [Figure 7] It is a diagram showing an example of a boxing score input screen. [Figure 8] It is a diagram showing an example of an image display screen of a boxing score. [Figure 9] It is a diagram showing an example of the output of a boxing score by 3DCG.

Modes for Carrying Out the Invention

[0020] (1. Configuration of the System) The fighting technique action recording and output system 100 shown in FIG. 1 is a computer device used by a user (a user of the fighting technique action recording and output system 100. mainly a person who practices fighting techniques). This computer device can be, for example, a home game console, a personal computer, a tablet, a smartphone, etc.

[0021] An example of the configuration of the martial arts movement recording output system 100 is shown based on Figure 1. The input unit 10 is implemented by means of operating devices such as a keyboard, mouse, and touch panel display, and grasps martial arts movements as information for encoding. The input unit 10 can also be implemented by an image communication interface connected to a camera that takes still images and videos, or an audio communication interface connected to a microphone that collects sound. The encoding unit 20 is implemented as a functional means by a computer program implemented in the device being processed by the device's CPU, etc., and converts the various information received by the input unit 10 into a martial arts score according to predetermined conversion rules illustrated in Figure 5. The recording unit 30 is implemented by a storage medium built into or connected to the device, such as RAM, ROM, EEPROM, etc., and holds the input and encoded martial arts scores as data. The output unit 40 is implemented by a display means for image display, such as an LCD, and displays the martial arts score as an image, and also reproduces the movements with a person represented as 3DCG. Furthermore, the output unit 40 is also equipped with a speaker for sound output, and outputs the martial arts movements represented by the kendo notation as sound to the user (the user of the martial arts movement recording output system 100). Note that the input unit 10 and the output unit 40 may be integrated into a single configuration, such as a touch panel display.

[0022] The martial arts motion recording and output system 100 is either a standalone computer device or a computer device that connects to a server via a network and constitutes a client terminal in a client-server system. That is, the input unit 10, encoding unit 20, recording unit 30, and output unit 40 can all be built on a client terminal such as a web browser or native application, but the encoding unit 20 and recording unit 30 can also be built in a form where they are performed on a server and communicate with the client.

[0023] Figure 2 shows an example of a configuration in which a martial arts movement recording output system 100 is connected to a client terminal 101 and a server 102 via a data communication network (not shown). In the martial arts movement recording output system 100 shown in Figure 2, the encoding unit 20 and recording unit 30 are provided on the server 102, and the input unit 10 and output unit 40 are provided on the client terminal 101. In Figure 2, the source video input to the input unit 10 of the client terminal 101 is supplied to the encoding unit 20 of the server 102, encoded into a martial arts score, and recorded in the recording unit 30 of the server 102. The martial arts score recorded in the recording unit 30 is transmitted to the client terminal 101 and output from the output unit 40.

[0024] (2. Rules for Fist Records) Before detailing each part, let's outline the rules of the boxing technique notation. Boxing notation expresses the movements of striking martial arts such as boxing as a combination of three independent elements: offensive movements, defensive movements, and stepping movements.

[0025] Figure 5 shows an example of the rules for the boxing notation in this embodiment. The figure shows examples of notation for individual offensive movements, defensive movements, and stepping movements in the boxing notation. Multiple symbols for offensive movements in the boxing notation are displayed in the display column 51 of Figure 5. For example, the top symbol in the display column 51 is symbol 52a, which represents a lead straight, and the second symbol from the top is symbol 52b, which represents a rear straight.

[0026] As illustrated above, offensive movements are represented using musical notation, similar to musical notation. Since offensive movements in striking martial arts have many types and classifications, when representing them in kendo notation, it is necessary to change the notation method according to the type and classification of the offensive movement. Therefore, unlike in musical notation where the position of a note on the staff represents a specific note in a scale, in kendo notation, the position of the note on the staff corresponds to the trajectory of the strike and whether it is left or right. This allows for the distinction between commonly known strikes such as jabs and straights based on the position of the note on the staff.

[0027] Figure 4 shows an example of the correspondence between the position of a punch on a musical staff and a punch in boxing, in the punch notation of this embodiment. In Figure 4, for example, the punch on the second line from the top, shown on the far left, represents a "jab," and the punch between the second and third lines from the top, shown second from the left, represents a "straight."

[0028] Furthermore, although not shown in Figure 4, by changing the shape of the note head, which is the head portion of the tadpole, it is possible to distinguish whether the strike is to the head, abdomen, or legs. In addition, by distinguishing the orientation of the note stem, which is the rod portion of the tadpole, it is possible to distinguish between punches and kicks within the scope of musical notation, such as an upward direction signifying a punch and a downward direction signifying a kick.

[0029] Regarding defensive actions, existing notational representations cannot be directly referenced, so it is necessary to devise them independently. That is, as illustrated in the display column 53 in Figure 5, symbols are prepared for each of the defensive actions that are primarily used in practice. For example, the top of the display column 53 is symbol 54a, which represents a two-handed block, and the second from the top is symbol 54b, which represents a lead block, meaning a block using the left hand in an orthodox stance.

[0030] Stepping movements are often performed in parallel with offensive or defensive movements, particularly those involving punches. Therefore, it is essential that they be symbolically represented alongside these movements. As illustrated in Figure 5, in display column 55, a unique symbol is devised using the lower part of the musical staff. However, other rules can also be used if they allow for simultaneous representation. For example, the top symbol in display column 55 is 56a, representing a front step, and the second symbol from the top is 56b, representing a back step.

[0031] (3. Operation of the input section) The input unit 10 receives input for martial arts movements in three ways. The first method is for the user to directly manipulate coded information through the interface provided by the input unit. The following is an example of an implementation of the interface provided by the input unit as the first method.

[0032] The offensive action, though not illustrated, involves the user placing notes at their desired location by clicking or tapping on any point on the musical staff.

[0033] Figure 7 shows an example of the screen in the input mode for defense actions on the input screen. In the circular display area, the symbols shown in the display field 54 in Figure 5, which are defined as defense actions, are arranged. When the user clicks or taps any of the displayed symbols, the input unit 10 displays the symbol representing the defense action to be displayed. Step actions can also be input in the same way as defense actions.

[0034] The second method involves capturing stored image information or obtaining image information from a connected camera resource in a way that the client can recognize, based on a handwritten record of a martial arts score by the user in a notebook or other similar document, or a martial arts score that has been previously recorded on another information medium such as paper or a computer screen.

[0035] The third method involves acquiring video footage of a person performing a competitive action as video data, or as a camera resource acquired in real time.

[0036] (4. Operation of the encoding section) The encoding unit 20 receives various data acquired by the input unit 10 by the first to third method and converts them into coded information as a kendo score based on the rules of the kendo score. In the first method, the encoding unit 20 converts input operations obtained through the input interface into coded information of the kendo score. In the second method, the encoding unit 20 converts an image of the kendo score into coded information of the kendo score by optical recognition. In the third method, the encoding unit 20 converts the competitive movements in the video into coded information of the kendo score by analyzing the video.

[0037] The encoded martial arts movements will be represented as shown in Figure 6 as an example. The fist symbol shown in Figure 6 is a jab feint 61, a one-beat pause 62, a front step 63 while throwing a jab 64, a head slip 65 while throwing a straight punch 66, a left ducking 67, a right side step 68 while weaving 69, a one-beat pause 70 while taking a right side step 71.

[0038] (5. Operation of the recording unit) The recording unit 30 records the boxing record as coded information. The data structure is illustrated with Figure 3. In martial arts, the meaning of movements differs depending on the distance to the opponent, so the distance from the user to the opponent is defined by three different lengths and saved as distance symbols, known as distance31. Even when performing solo movements such as shadow boxing, the presence of an opponent remains a consideration, and the necessity of this remains unchanged.

[0039] For jabs and straight punches, even when performing the same type of strike, combinations with different rhythms can be of significant importance in competition. Therefore, to express the rhythmic differences that characterize these combinations of strikes, quarter notes are used for movements with a longer interval between movements, and eighth notes are used for movements with a shorter interval than those represented by quarter notes. Similarly, in defensive movements, the length of defensive movements that occur between offensive movements is also important. This information is described in duration32.

[0040] Offensive actions are recorded using five fields. `offense_pitch33a` stores the type of strike in martial arts, which is primarily determined by trajectory, similar to the pitch element in music. `offense_notehead33b` stores a value that controls the note head, representing the target of the strike on the staff, such as the head, abdomen, or legs. `offense_stem33c` stores a value that controls the note stem, representing whether the attack is a punch or a kick on the staff. `offense_quirk34d` stores a value that controls additional symbols used to express detailed distinctions in karate and Muay Thai, where a variety of kicking techniques exist and simply distinguishing positions on the staff by the trajectory of the strike is insufficient. For example, values ​​such as `dot`, `rounded`, and `diagonal-line` can represent dots, circles, or parallel lines. `offense_hat33e` controls whether a hat symbol is added to a specific offensive action. A "kenfu" (fist record) is a record of movements in martial arts. In martial arts, during the exchange of moves with an opponent, one attempts to intentionally create a false prediction about the opponent and then attack the opponent's vital points by performing an attack that contradicts that prediction. Therefore, in a kenfu, it is necessary to record not only the objective movements but also the subjective intention of which offensive movement is truly intended to hit a vital point, and thus the hat symbol is required.

[0041] Defensive actions are identified by type and origin. `defense_type34a` can take values ​​such as block, parry, headslip, weave, duck, and sway, allowing for the identification of major defensive actions in various martial arts, including boxing. `defense_origin34b` expresses whether the defensive action originates from the leg closer to the opponent or the leg further away. This difference in stance is known in martial arts as the distinction between orthodox and southpaw. While discussing only orthodox stances can be replaced with the distinction between left and right, understanding it as a southpaw reverses the meaning of left and right, requiring abstraction. Therefore, it is conceivable to distinguish them using specific values ​​such as lead and rear.

[0042] Step movements are identified by type and origin. step_type35a can take values ​​such as front, back, side, and pivot, thereby distinguishing the type of step. step_origin35b, similar to the defensive movements mentioned above, can take values ​​such as lead and rear, as it abstractly represents differences in stance such as orthodox and southpaw.

[0043] (5. Operation of the output section) The output unit 40 provides, firstly, an output in the form of an image, secondly, an output as the movement of a person represented as 3DCG, and thirdly, an output as sound.

[0044] Figure 8 shows an example of a screen for image output. On the display screen, card-like elements 81, 82, and 83, which contain boxing sequences representing competitive movements, are arranged vertically.

[0045] Figure 9 shows an example of the output of a character's movements as represented by 3DCG on the screen. On the display screen, a 3DCG character 91 is positioned at the back, and by clicking or tapping the play button 92 positioned at the front, the competitive movements represented by the game score 93 are performed on the screen by the 3DCG character 91.

[0046] In the third method, the output unit 40 outputs the punch notation as sound. When converting offensive movements into sound, the same pitch as on the musical staff is generated based on the position on the staff represented by offense_pitch33a. Furthermore, by assigning different instruments to vital points such as the head, abdomen, and legs, and to the striking methods of punches and kicks, the aforementioned pitch is generated by different instruments depending on the vital point to be targeted, according to the distinction between offense_notehead33b and offense_stem33c. When converting defensive movements into sound, unlike offensive movements, analogies based on musical pitch cannot be applied, so it is conceivable to assign them using percussion. However, defensive movements can also be assigned to instruments that have pitch. For step movements, if an instrument capable of forming a melody is selected for offensive movements and percussion is assigned for defensive movements, it is conceivable to assign elements that constitute a bass sound for musical balance. However, the instruments and sounds selected for step movements can also be considered separately.

[0047] (6. Usage patterns from the user's perspective) Next, an example of the operation of the martial arts movement recording output system 100 when a user inputs and outputs a record of their movements. This section explains the following. The user inputs the kendo notation as symbolic information through the interface provided by the input unit 10. Offensive movements are input by the user specifying any point on the musical staff that the interface has provided in advance, by clicking, tapping, or other means (not shown). Defensive movements are input by selecting one of the symbols representing defensive movements from the group of symbols that the interface has provided in advance, by clicking, tapping, or other means, as illustrated in Figure 7, in the defensive movement input mode. Step movements are input in the same way as defensive movements, by selecting one of the kendo notation symbols that the interface has provided in advance, by clicking, tapping, or other means, after the step movement input mode has been displayed. Alternatively, the user inputs the kendo notation by optically capturing the kendo notation displayed on a notebook or display screen through the client terminal's camera resources. Alternatively, the user inputs the kendo notation by saving video footage of themselves or others performing martial arts movements, such as by filming a practice session, or by capturing the footage immediately through the client terminal's camera resources without saving it.

[0048] The output unit 40 displays the punch sequences entered into the martial arts movement recording output system 100 on the screen, as illustrated in Figure 8. Users can visualize martial arts movements by viewing the displayed punch sequences from left to right, similar to musical notation, and use this to aid in repetitive learning. In the first punch sequence display area 81 in Figure 8, the movements of throwing punches in the order of lead straight, rear straight, and lead hook are displayed as images in the punch sequence. In the second punch sequence display area 82, the movements of throwing punches in the order of lead straight, lead straight, and rear straight are displayed as images in the punch sequence. In the third punch sequence display area 83, the movements of throwing punches in the order of lead straight, rear straight, and lead straight are displayed as images in the punch sequence. Furthermore, as illustrated in Figure 8, by arranging multiple punch sequences vertically or horizontally, users can view multiple different combinations at a glance.

[0049] In martial arts where striking is the primary method of attack, athletes typically practice alone, such as in shadow boxing. When a user practices alone, they perform the movements of the sport based on sounds that are continuously generated in a time-series manner by the output unit 40. This gives the user the same effect as a boxer performing combinations as instructed by a coach in a boxing gym, such as "one, two." Since the sounds are converted from punching patterns according to predetermined rules, the user intuitively recalls the movements of the martial art by listening to the generated sounds. However, even users who are not familiar with these sound conversion rules can intuitively understand the contents of the punching patterns by viewing the screen, as illustrated in Figure 9, because the output unit 40 displays images of punching patterns and videos of a person represented as 3DCG performing the movements described in the punching patterns in accordance with the sound output.

[0050] Although embodiments have been described above, the present invention is not limited to the above description, and various modifications are possible without departing from the spirit of the invention. Furthermore, one or more requirements or steps described in the above embodiments may be omitted. Such modifications, improvements, or omitted forms are also included within the technical scope of the present invention. [Explanation of Symbols]

[0051] 10 Input section 20 Encoding section 30 Records Section 40 Output section 100 Martial Arts Movement Recording Output System

Claims

1. An encoding means that converts the individual movements of a person performing a martial arts technique into individual coded information including musical notes based on predetermined conversion rules, Recording means for recording the aforementioned code information, It comprises an action output means that outputs an image of a person performing a martial arts action to a display means, The combat sports action recording output system is characterized in that the action output means outputs a video of a series of actions of the combat sports based on the output of a plurality of coded information recorded in the recording means in a time series.

2. The system includes a code output means that outputs the code information as an image of the code to the display means, The martial arts action recording output system according to claim 1, characterized in that the code output means outputs an image of the code to the display means together with the person image, or separately from the person image.

3. A sound conversion means that converts the aforementioned coded information into sound information based on a predetermined conversion rule, The sound conversion means comprises a sound output means that outputs the sound information converted by the sound conversion means as a sound that can be heard by humans, The martial arts action recording output system according to claim 1, characterized in that the sound conversion means converts each action of the martial arts into a state of output sound, including pitch, rhythm, and timbre.

4. The combat sport action recording output system according to claim 1, characterized in that the combat sport is boxing.

Images