Method and apparatus for evaluating taekwondo motion based on voice recognition using artificial intelligence

Abstract

An embodiment of the present invention relates to a method for evaluating a Taekwondo motion based on voice recognition performed by at least one server and may include the steps of: providing image information including continuous motions of an evaluation target to an evaluator terminal; obtaining voice evaluation information from the evaluator terminal while an image according to the image information is reproduced; deriving a frame corresponding to a time point at which the voice evaluation information is obtained from among multiple frames included in the image information; and generating image evaluation information by synthesizing the voice evaluation information and the image information based on the derived frame.

Description

Method and device for evaluating taekwondo movements based on voice recognition using artificial intelligence

[0001] The present invention relates to a method and device for evaluating Taekwondo movements based on voice recognition using artificial intelligence, and more particularly, to a method and device for an evaluator to evaluate a subject in real time using voice while the subject performs Taekwondo movements.

[0002] Taekwondo is a modern, traditional martial art characterized by fast and powerful kicking techniques, a martial art practiced without weapons, using only the hands and feet. It was adopted as a demonstration sport at the 1988 Summer Olympics and became an official event at the 2000 Summer Olympics.

[0003] Taekwondo can be broadly divided into three categories: poomsae (forms), sparring (sparring), and breaking (training). Taekwondo poomsae refers to a series of movements that combine attack and defense techniques, arranged according to prescribed formats, allowing practice without a leader. Through poomsae, practitioners can improve sparring skills, apply movement techniques, and hone specialized techniques not learned through basic movements.

[0004] Generally, poomsae is practiced according to the poomsae line, which is a line that indicates the position of the feet and the direction of their movement, and there are ten types: Taegeuk, Goryeo, Geumgang, Taebaek, Pyeongwon, Sipjin, Jitae, Cheonkwon, Hansu, and Ilyeo. In the past, the poomsae evaluation method had the judges sit in their respective seats, check the continuous movements of the person being evaluated, and directly record the scores. However, even after the evaluation was over, there was a problem that it was not possible to judge in detail where and how the continuous movements of the person being evaluated were performed, and which movements were performed appropriately or inadequately.

[0005] The purpose of the present invention to solve the above problems is to provide a method for evaluating Taekwondo movements based on voice recognition using artificial intelligence.

[0006] Another object of the present invention to solve the above problems is to provide a Taekwondo movement evaluation device based on voice recognition using artificial intelligence.

[0007] The problems to be solved by this application are not limited to the problems described above, and problems not mentioned can be clearly understood by a person having ordinary skill in the technical field to which this application belongs from this specification and the attached drawings.

[0008] In order to achieve the above object, a method for evaluating Taekwondo movements based on voice recognition using artificial intelligence, performed by at least one server according to one embodiment of the present invention, may include the steps of providing image information including continuous movements of an evaluation subject to an evaluator terminal, obtaining voice evaluation information from the evaluator terminal while an image according to the image information is played, deriving a frame corresponding to a time point at which the voice evaluation information is obtained from among a plurality of frames included in the image information, and generating image evaluation information by synthesizing the voice evaluation information and the image information based on the derived frame.

[0009] Here, the voice evaluation information includes voice information recognized as one word among a plurality of preset words including a first word and a second word, and the first word and the second word may indicate different evaluation scores.

[0010] Here, the method further includes a step of generating evaluation result information of the evaluation subject based on the image evaluation information, and the evaluation result information may include result information in which an evaluation score is applied according to one recognized word to an action corresponding to the frame among the continuous actions.

[0011] Here, the method further includes a step of providing the evaluation result information to the evaluator terminal, a step of obtaining correction information about the evaluation result information from the evaluator terminal, and a step of generating final result information based on the correction information and the evaluation result information, wherein the correction information may include information for changing an action to which an evaluation score according to the recognized word is applied to another action among the continuous actions.

[0012] An electronic device for performing a voice recognition-based Taekwondo movement evaluation method using artificial intelligence according to an embodiment of the present invention for achieving the above other objects includes a processor and a memory for storing at least one command executed by the processor, wherein the at least one command is executed to provide image information including continuous movements of an evaluation subject to an evaluator terminal, to acquire voice evaluation information from the evaluator terminal while an image according to the image information is played, to derive a frame corresponding to a time point at which the voice evaluation information is acquired from among a plurality of frames included in the image information, and to generate image evaluation information by synthesizing the voice evaluation information and the image information based on the derived frame.

[0013] The means of solving the problem are not limited to the above-described means of solving the problem, and means of solving the problem that are not mentioned can be clearly understood by a person having ordinary skill in the technical field to which the present application belongs from this specification and the attached drawings.

[0014] According to the present invention, by having an evaluator verbally evaluate the incorrect parts in real time while the subject of evaluation performs a series of movements, detailed evaluation information can be generated as to which movements were deducted points.

[0015] According to the present invention, point deduction information can be included in the same timeline according to the continuous movements of the subject of evaluation, thereby providing video information that records in detail which movement was deducted which point.

[0016] According to various embodiments, the effects are not limited to the effects described above, and effects not mentioned can be clearly understood by a person having ordinary skill in the art to which the present application belongs from the present specification and the attached drawings.

[0017] FIG. 1 is a block diagram of an electronic device that performs a voice recognition-based taekwondo movement evaluation method using artificial intelligence according to one embodiment of the present invention.

[0018] FIG. 2 is a diagram showing terminals on which a method for evaluating Taekwondo movements based on voice recognition using artificial intelligence according to one embodiment of the present invention is performed.

[0019] Figure 3 is a flowchart of a method for evaluating Taekwondo movements based on voice recognition using artificial intelligence according to one embodiment of the present invention.

[0020] FIG. 4 is a diagram for explaining a process of synthesizing voice evaluation information into image information according to one embodiment of the present invention.

[0021] FIG. 5 is a drawing for explaining image evaluation information according to one embodiment of the present invention.

[0022] Figure 6 is a flowchart of a method for generating final result information according to one embodiment of the present invention.

[0023] The present invention is susceptible to various modifications and embodiments. Specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, but rather to encompass all modifications, equivalents, and alternatives falling within the spirit and technical scope of the present invention. Throughout the description of each drawing, similar reference numerals have been used to designate similar components.

[0024] Terms such as "first," "second," "A," and "B" may be used to describe various components, but these components should not be limited by these terms. These terms are used solely to distinguish one component from another. For example, without departing from the scope of the present invention, the first component could be referred to as the "second component," and similarly, the second component could also be referred to as the "first component." The term "and / or" includes any combination of multiple related items listed or any one of multiple related items listed.

[0025] When a component is referred to as being "connected" or "connected" to another component, it should be understood that it may be directly connected or connected to that other component, but that there may be other components intervening. Conversely, when a component is referred to as being "directly connected" or "connected" to another component, it should be understood that there are no other components intervening.

[0026] The terminology used in this application is only used to describe specific embodiments and is not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, it should be understood that the terms "comprise" or "have" indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but do not exclude in advance the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0027] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and will not be interpreted in an idealized or overly formal sense unless explicitly defined herein.

[0028] Hereinafter, with reference to the attached drawings, preferred embodiments of the present invention will be described in more detail. In order to facilitate an overall understanding in describing the present invention, identical reference numerals will be used for identical components in the drawings, and redundant descriptions of identical components will be omitted.

[0029] FIG. 1 is a block diagram of an electronic device that performs a voice recognition-based taekwondo movement evaluation method using artificial intelligence according to one embodiment of the present invention.

[0030] Referring to FIG. 1, an electronic device (100) that performs a voice recognition-based Taekwondo movement evaluation method using artificial intelligence according to one embodiment of the present invention may be configured to include at least one processor (110), a memory (120), and a transmission / reception device (130). In addition, the electronic device (100) may further be configured to include at least one of an input interface device (140), an output interface device (150), and a storage device (160).

[0031] For example, at least one processor (110) may mean a central processing unit (CPU), a graphics processing unit (GPU), or a dedicated processor on which methods according to embodiments of the present invention are performed. Each of the memory (120) and the storage device (160) may be configured with at least one of a volatile storage medium and a non-volatile storage medium. For example, the memory (120) may be one of a read-only memory (ROM) and a random access memory (RAM), and the storage device (160) may be a flash memory, a hard disk drive (HDD), a solid state drive (SSD), or various memory cards (e.g., a micro SD card).

[0032] Additionally, the transceiver (130) can perform communication via a wireless network, and each component included in the electronic device (100) can communicate with each other by being connected by a bus (170).

[0033] For example, the memory (120) can store at least one instruction to be performed by the processor (110), and the at least one instruction can be executed to perform at least a part of the operation of the artificial intelligence-based voice recognition-based Taekwondo movement evaluation method described below.

[0034] FIG. 2 is a diagram showing terminals on which a method for evaluating Taekwondo movements based on voice recognition using artificial intelligence according to one embodiment of the present invention is performed.

[0035] Referring to FIG. 2, an electronic device (hereinafter, electronic device, 100) that performs a voice recognition-based Taekwondo movement evaluation method using artificial intelligence according to an embodiment of the present invention can obtain voice evaluation information on continuous movements of an evaluation subject from an evaluator terminal (200), generate image evaluation information of the evaluation subject based on the obtained voice evaluation information and image information on continuous movements of the evaluation subject, and provide the image evaluation information to an evaluation subject terminal (300).

[0036] Alternatively, the electronic device (100) can provide image information on the continuous movements of the subject of evaluation to the evaluator terminal (200), the evaluator terminal (200) can obtain voice evaluation information from the evaluator, the evaluator terminal (200) can generate image evaluation information based on the image information and voice evaluation information, and can transmit the image evaluation information to the electronic device (100).

[0037] Alternatively, the electronic device (100) may provide the generated image evaluation information to the evaluator terminal (200), obtain correction information from the evaluator terminal (200), and generate final result information based on the correction information and the image evaluation information. Alternatively, the electronic device (100) may provide at least one of the final result information and the image evaluation information to at least one of the evaluator terminal (200) and the subject terminal (300).

[0038] Here, the electronic device (100) may represent at least one server. Alternatively, the electronic device (100) may be represented as a Taekwondo movement evaluation server. The electronic device (100) may transmit and receive information with the evaluator terminal (200) and / or the subject terminal (300) via wired or wireless communication.

[0039] For example, the evaluator terminal (200) and / or the evaluator terminal (300) may be, but is not limited to, a communication-capable desktop computer, laptop computer, notebook, smart phone, tablet PC, mobile phone, smart watch, smart glass, digital camera, digital video recorder, digital video player, PDA (Personal Digital Assistant), etc., and may include any computing-capable electronic device.

[0040] For example, the wireless communication may include, but is not limited to, at least one of LTE, LTE-A (LTE Advance), CDMA (code division multiple access), WCDMA (wideband CDMA), UMTS (universal mobile telecommunications system), WiBro (Wireless Broadband), GSM (Global System for Mobile Communications), WiFi (wireless fidelity), Bluetooth, Bluetooth low energy (BLE), Zigbee, near field communication (NFC), Magnetic Secure Transmission, radio frequency (RF), or body area network (BAN).

[0041] For example, wired communication may include at least one of USB (universal serial bus), HDMI (high definition multimedia interface), RS-232 (recommended standard232), power line communication, or POTS (plain old telephone service).

[0042] For example, the network may include at least one of a telecommunications network, such as a computer network (e.g., a LAN or WAN), the Internet, or a telephone network.

[0043] Figure 3 is a flowchart of a method for evaluating Taekwondo movements based on voice recognition using artificial intelligence according to one embodiment of the present invention.

[0044] Referring to FIG. 3, in operation S310, the server may provide the evaluation terminal with video information including continuous movements of the evaluation subject. For example, the continuous movements may be movements representing a specific Taekwondo Poomsae and may be divided into multiple unit movements. The unit movements may be distinguished into different unit movements based on changes in the shape of the legs / hands, the direction in which the legs / hands are facing, the speed at which the legs / hands are performed, etc. In other words, a Taekwondo Poomsae may be composed of continuous unit movements. For example, the video information including continuous movements may represent information regarding a video recording the movements of the evaluation subject performing the Taekwondo Poomsae.

[0045] In the S320 operation, the server can obtain voice evaluation information from the evaluator terminal while the video is played according to the video information.

[0046] For example, an evaluator may evaluate a Taekwondo Poomsae performed by a subject, and may do so based on preset movement evaluation criteria. For example, the preset movement evaluation criteria may include information regarding the evaluation criteria for each movement among the subject's consecutive movements. For example, the information regarding the evaluation criteria for each movement may include an evaluation score corresponding to at least one of the following: the shape of the incorrect leg / hand, the direction of the incorrect leg / hand, and the speed at which the incorrect leg / hand is performed. For example, the movement evaluation criteria may be provided to the evaluator through the evaluator terminal, but may not be provided separately if the evaluator is familiar with them.

[0047] For example, the voice evaluation information may include voice information recognized as one word among a plurality of preset words including a first word and a second word. In other words, when an evaluator utters a voice indicating a specific word, the evaluator terminal can acquire the voice and determine whether a word corresponding to the voice exists among the plurality of preset words. Alternatively, the evaluator terminal can acquire the voice and transmit it to a server, and the server can determine whether a word corresponding to the voice exists among the plurality of preset words. Since various methods of voice recognition are known and are obvious to those skilled in the art, a detailed description thereof will be omitted.

[0048] For example, multiple words, including the first word and the second word, may each have different evaluation scores. Here, the evaluation scores may represent, but are not limited to, deducted points.

[0049] In other words, if the evaluator pronounces a specific word, the corresponding score (e.g., 3) may be deducted, and if the evaluator pronounces another specific word, another score (e.g., 5) corresponding to the other specific word may be deducted. In the above explanation, two deduction scores were described based on two words, but it can also be implemented with only one deduction score based on one word, or it can be implemented with three or more deduction scores based on three or more words.

[0050] In operation S330, the server can derive a frame corresponding to the time at which the voice evaluation information was acquired from among the multiple frames included in the video information. For example, the video information may consist of multiple still images, and each still image may be represented as a frame. For example, the server can derive frame or time zone information corresponding to the time at which the evaluator uttered a specific word during video playback.

[0051] In operation S340, the server can generate video evaluation information by synthesizing voice evaluation information and video information based on the derived frame. This will be described in detail later with reference to FIGS. 4 and 5.

[0052] FIG. 4 is a diagram for explaining a process of synthesizing voice evaluation information into image information according to one embodiment of the present invention, and FIG. 5 is a diagram for explaining image evaluation information according to one embodiment of the present invention.

[0053] Referring to FIGS. 4 and 5, the server can insert the evaluator's voice into the video information based on the derived frame. For example, the evaluator's voice can be inserted at the time when the voice evaluation information is acquired, can be inserted at a certain time before the time when the voice evaluation information is acquired, or can be inserted at the start time of a unit motion shown in a frame corresponding to the time when the voice evaluation information is synthesized. This is to effectively simultaneously check the points deducted according to the voice and the motion played according to the video when the video evaluation information in which the voice evaluation information is synthesized is played back. For example, the time point can be synthesized by pulling forward by a certain time in consideration of a person's judgment of the video and the speed of the voice response.

[0054] Figure 6 is a flowchart of a method for generating final result information according to one embodiment of the present invention.

[0055] Referring to FIG. 6, one embodiment may generate final result information that can be provided to the subject or evaluator based on the image evaluation information generated as described above, and the evaluation result may be determined based on the final result information. However, since one embodiment may perform only at least some of the operations described below, the evaluation result may also be determined based on the evaluation result information.

[0056] In operation S610, the server can generate evaluation result information of the subject of the evaluation based on the video evaluation information. For example, the evaluation result information may include video evaluation information, and may include result information in which an evaluation score is applied according to a single word recognized from the evaluator's voice for a motion corresponding to a frame derived from a series of motions. In other words, the evaluation result information may include result information in which the evaluator's voice-evaluated and deducted points are reflected for each motion included in the video, and may include which motion was deducted based on which voice or word, the number of times points were deducted based on which voice or word, and the final score. For example, the final score may represent a score deducted from a preset maximum score through the evaluator's voice, and the maximum score may be 100 points, but is not limited thereto.

[0057] For example, if the evaluator's voice is acquired slightly late, the intended motion for deduction may be incorrectly matched with the synthesized voice, i.e., the deducted motion. To address this situation, one embodiment may provide the evaluator with video evaluation information synthesized with voice evaluation information, allowing the evaluator to review and correct the information. To provide this solution, one embodiment may further include the following actions:

[0058] In operation S620, the server may provide evaluation result information to the evaluator terminal. For example, the server may provide evaluation result information and / or video evaluation information to the evaluator terminal. This allows the evaluator to review the result information or the video information synthesized with the negative points using their own voice to confirm that the evaluation was performed as intended.

[0059] In operation S630, the server can obtain correction information regarding the evaluation result information from the evaluator terminal. In other words, if some evaluations are not evaluated as intended, the evaluator can correct the evaluation results by providing correction information.

[0060] For example, if the evaluator's voice acquisition is slightly delayed and the voice is matched to a different action than the intended action, resulting in a deduction of points, the correction information may include information to change the action assigned a score based on a single word recognized through voice to another action among the consecutive actions. Here, the other action to be changed can be acquired through the evaluator's terminal.

[0061] In operation S640, the server can generate final result information based on the correction information and evaluation result information. For example, the server can delete audio information related to incorrectly synthesized motions based on the correction information from the video evaluation information included in the evaluation result information, and resynthesize or insert the deleted audio information based on other changed motions acquired through the evaluator terminal. This can generate final result information that has been reviewed by the evaluator. Thereafter, the server can provide the final result information to the evaluator terminal or the evaluation subject terminal.

[0062] In addition, the final result information may include information about the final score with the changes applied, similar to the evaluation result information described above, and may include information indicating pass or fail by comparing it with a preset threshold score, and may include information indicating a proficiency level corresponding to the score ranges through multiple score ranges.

[0063] The operations according to the embodiments of this specification can be implemented as computer-readable programs or codes on a computer-readable recording medium. Computer-readable recording media include all types of recording devices that store data readable by a computer system. Furthermore, computer-readable recording media can be distributed across network-connected computer systems, allowing computer-readable programs or codes to be stored and executed in a distributed manner.

[0064] When the embodiment is implemented in software, the above-described technique can be implemented as a module (process, function, etc.) that performs the above-described function. The module can be stored in memory and executed by the processor. The memory can be internal or external to the processor and connected to the processor by various well-known means.

[0065] Additionally, the computer-readable recording medium may include hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, etc. The program instructions may include not only machine language codes produced by a compiler, but also high-level language codes that can be executed by a computer using an interpreter, etc.

[0066] While some aspects of the present invention have been described in the context of a device, they may also represent a description of a corresponding method, wherein a block or device corresponds to a method step or a feature of a method step. Similarly, aspects described in the context of a method may also be described as a corresponding block or item or a feature of a corresponding device. Some or all of the method steps may be performed by (or using) a hardware device, such as, for example, a microprocessor, a programmable computer, or an electronic circuit. In some embodiments, one or more of the most significant method steps may be performed by such a device.

[0067] In embodiments, a programmable logic device (e.g., a field programmable gate array) may be used to perform some or all of the functions of the methods described herein. In embodiments, the field programmable gate array may operate in conjunction with a microprocessor to perform one of the methods described herein. In general, the methods are preferably performed by some hardware device.

[0068] Although the present invention has been described above with reference to preferred embodiments thereof, it will be understood by those skilled in the art that various modifications and changes may be made to the present invention without departing from the spirit and scope of the present invention as set forth in the claims below.

Claims

1. A method for evaluating Taekwondo movements based on voice recognition performed by at least one server, A step of providing image information including continuous movements of the subject of evaluation to the evaluator terminal; A step of obtaining voice evaluation information from the evaluator terminal while the video according to the above video information is played; A step of deriving a frame corresponding to the time at which the voice evaluation information is acquired among a plurality of frames included in the video information; and A step of generating image evaluation information by synthesizing the voice evaluation information and the image information based on the derived frame, A method for evaluating taekwondo movements based on voice recognition.

2. In claim 1, The above voice evaluation information includes voice information recognized as one word among a plurality of preset words including a first word and a second word, The first word and the second word represent different evaluation scores, A method for evaluating taekwondo movements based on voice recognition.

3. In claim 2, Further comprising a step of generating evaluation result information of the evaluation subject based on the above image evaluation information, The above evaluation result information includes result information in which an evaluation score is applied according to the recognized word to the motion corresponding to the frame among the continuous motions. A method for evaluating taekwondo movements based on voice recognition.

4. In claim 3, A step of providing the above evaluation result information to the evaluator terminal; A step of obtaining correction information regarding the evaluation result information from the evaluator terminal; and Further comprising a step of generating final result information based on the above modification information and the above evaluation result information, The above modification information includes information for changing an action to which an evaluation score is applied according to the recognized word to another action among the consecutive actions. A method for evaluating taekwondo movements based on voice recognition.

5. In an electronic device performing a voice recognition-based Taekwondo movement evaluation method, processor; and A memory that stores at least one instruction to be performed by the processor, At least one of the above commands, It is executed to provide image information including the continuous movements of the subject of evaluation to the evaluator terminal, It is executed to obtain voice evaluation information from the evaluator terminal while the video according to the above video information is played, It is executed to derive a frame corresponding to the time at which the voice evaluation information is acquired among a plurality of frames included in the above video information, Executes to generate image evaluation information by synthesizing the voice evaluation information and the image information based on the above-deduced frame. Electronic devices.

Images