A method for identifying and evaluating curling player's movements

Through computer vision technology, human posture estimation algorithm and dynamic programming algorithm are used to evaluate curling athletes' throwing movements, solving the problem of low evaluation efficiency in the existing technology, and achieving accurate evaluation of curling athletes' movements and intelligent auxiliary training.

CN116824703BActive Publication Date: 2024-11-15HARBIN INST OF TECH +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310854395.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-07-13
Publication Date
2024-11-15
Estimated Expiration
2043-07-13

AI Technical Summary

Technical Problem

The prior art is difficult to effectively evaluate and assist curling athletes' throwing movements, resulting in low training efficiency, relying on coaches' experience and lack of intelligent assistance.

Method used

Through computer vision technology, human posture estimation algorithm and dynamic programming algorithm are used to extract the skeleton time series information of curling athletes, design joint angle score function, and accurately evaluate and standardize the throwing movements of curling athletes.

Benefits of technology

The quantitative evaluation of curling athletes' throwing movements has been achieved, training efficiency has been improved, and training has been assisted by beginners and athletes, which has improved the intelligence level of curling exercises.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116824703B_ABST
    Figure CN116824703B_ABST
Patent Text Reader

Abstract

The present invention proposes a method for identifying and evaluating curling player actions. The method first predicts the confidence of human joint points and trunk affinity field through a human posture estimation algorithm, and extracts skeleton time series information in a curling player throwing video; secondly, through the normalization of the skeleton graph, the Euclidean distance of the pixel points between the skeleton graphs is used as the matching distance, and the skeleton sequence alignment based on the dynamic programming algorithm is performed to achieve the time series alignment between the skeleton sequence to be matched and the standard skeleton sequence; finally, the curling player throwing action evaluation rule is constructed, and the target joint angle is calculated through the skeleton graph of the curling player human skeleton detection, and the score function is designed by using the range of the joint angle to evaluate the action. The method realizes the evaluation of the curling player's throwing action, provides a reference for the training of curling players, and can play a role in assisting the curling player's training.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a method for intelligently evaluating curling players based on action recognition, which belongs to the cross-technical fields of artificial intelligence, image detection, etc., and in particular to a method for identifying and evaluating curling players' actions. Background Art

[0002] Curling is a sport that is both competitive and interesting. The result of each throw by a curler will affect the score and strategic deployment on the field, so the accuracy of the curler's throw is of utmost importance. Curlers need to undergo intensive training to master standard movements in order to throw the desired curling trajectory. In actual training, coaches evaluate the athletes' movements through their eyes and experience, and then correct the athletes' movements. Therefore, the level and state of the coaches also affect the athletes' training. With the help of computer vision-based motion recognition and evaluation methods to quantify the standardization of curling athletes' movements, it is possible to assist beginners and athletes in training, standardize curling throwing movements, and enable robots to spar and guide athletes, thereby improving the intelligence level of curling. Summary of the invention

[0003] The purpose of the present invention is to solve the problems in the prior art and propose a method for identifying and evaluating the movements of curling players. The method has strong applicability and can evaluate the accuracy of curling players' movements through computer vision.

[0004] The present invention is implemented by the following technical scheme. The present invention proposes a method for identifying and evaluating curling player movements, the method comprising:

[0005] Step 1: Use the human posture estimation algorithm to predict the confidence of human joints and the trunk affinity field, and extract the skeleton time series information in the curling athlete's throwing video;

[0006] Step 2: By normalizing the skeleton images, taking the Euclidean distance of the pixels between the skeleton images as the matching distance, the skeleton sequence alignment based on the dynamic programming algorithm is performed to achieve the temporal alignment between the skeleton sequence to be matched and the standard skeleton sequence;

[0007] Step 3: Construct the evaluation rules for the curling athlete's throwing action, calculate the target joint angle through the skeleton diagram of the curling athlete's human skeleton detection, and use the range of the joint angle to design a scoring function for action evaluation.

[0008] Furthermore, the step 1 is specifically as follows:

[0009] The first step is to detect the human joints of the curling athletes from the video based on the Openpose algorithm; first, the feature extraction network is constructed through the first ten layers of the VGG-19 network to obtain the feature map F; then the confidence S of the human joints is predicted1 =ρ 1 (F), each confidence map contains a certain type of joint points; then predict the trunk affinity field L 1 =φ 1 (F), that is, the actual connection between the two joints; where ρ 1 and φ 1 They represent the convolutional network for generating the joint confidence map and the convolutional network for generating the trunk affinity field in the first stage respectively;

[0010] The second step is to connect the human joint confidence prediction results and trunk affinity field prediction results of the previous stage with the original image features to accurately predict the human joint points;

[0011]

[0012] Among them, ρ t and φ t It is the convolutional network for generating the joint confidence map and the torso affinity field for reasoning in the tth stage;

[0013] The third step is to predict 18 human joints and lines from the curling player's throwing video based on the predictions in the first and second steps, and extract the curling player's human posture time series.

[0014] Furthermore, in step 2, the skeleton normalization method of the curling player's throwing action is specifically as follows: the pixel coordinates of the skeleton points of the curling player in a single frame are defined as a set X={x i |i=1,2,...,18} and Y={y i |i=1,2,...,18}, which represent the coordinate positions of 18 skeleton joints in the horizontal and vertical directions of the image respectively; a skeleton image is defined as [X,Y], and the pixel coordinates x of all the bone joints of the curling player skeleton are calculated. min 、x max and pixel coordinate y min ,y max , the coordinate range of the skeleton is obtained, and the range is used as the normalization standard to normalize the pixel values ​​of the skeleton; the normalization method is described as shown in formulas (2)-(5):

[0015]

[0016]

[0017]

[0018]

[0019] After normalization, the coordinates of the skeleton will be mapped to [0,1].

[0020] Further, in step 2, the distance between the two curling player skeleton images is determined as follows: first, the skeleton is preprocessed, the joint points of the two curling player skeleton images are screened, the common joint point coordinates of the two images are retained, and the coordinate positions of the remaining joint points are set to (0,0); the two skeleton images [X m ,Y m ] 0 ,[X n ,Y n ] 0 Normalize and get two normalized joint graphs [X m ,Y m ],[X n ,Y n ]; Calculate the pixel Euclidean distance between the same joint points corresponding to the two skeleton images:

[0021]

[0022] where x mi ∈X m ,y mi ∈Y m , x ni ∈X n ,y ni ∈Y n , i=1,2,...,18.

[0023] Further, in step 2, for a video skeleton sequence composed of multiple frames, the matching between two time series skeletons is completed based on the dynamic programming algorithm as follows: define two skeleton sequences as The time series lengths are T 1 and T 2 First, the two skeleton sequences are normalized, and then the distance matrix between the two skeleton graphs is calculated according to formula (6). The path with the minimum distance is calculated according to the dynamic programming algorithm to complete the matching between the two skeleton time series.

[0024] Furthermore, in step 3, the throwing action of the curling athlete is divided into four sub-actions, including squatting, starting and pulling back, sliding, and throwing; the joint angle ranges of the four sub-actions are defined as the rules for evaluating the throwing action of the curling athlete: the shoulder extension angle α in the squatting sub-action 1 The range is to Pedal knee angle β 1 The range is to Shoulder abduction angle γ 1 The range is to Shoulder extension angle α during start-up and pull-back 2 The range is to Pedal knee angle β 2 The range is to Shoulder abduction angle γ 2 The range is to Elbow angle when holding the kettle during sliding movements The range is to Shoulder abduction angle γ 3 The range is to Wrist angle θ during hand-out action 4 The range is greater than 85°.

[0025] Furthermore, in step 3, the score function S(α i )、S(β j ), S(γ k )and They satisfy Gaussian distribution respectively, as shown in equations (7)-(10):

[0026]

[0027]

[0028]

[0029]

[0030] Among them, i = 1, 2; j = 1, 2; k = 1, 2, 3;

[0031] Design the score function S(θ 4 ) as formula (11):

[0032]

[0033] Furthermore, in step 3, the scores of the joint angles in each sub-action interval of the video segment to be evaluated are averaged, and then the scores of the four sub-actions are averaged to obtain the score S of the curling athlete's throwing action. avg , as shown in formula (12):

[0034]

[0035] Among them, T 1 、T 2 、T3 、T 4 Indicates the number of frames of the video corresponding to the four sub-actions. and It is the score function corresponding to the joint angle of the tth frame in each sub-action interval.

[0036] The present invention proposes an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of a method for identifying and evaluating curling player movements when executing the computer program.

[0037] The present invention provides a computer-readable storage medium for storing computer instructions, which, when executed by a processor, implement the steps of a method for identifying and evaluating curling player movements.

[0038] Compared with the prior art, the present invention has the following beneficial effects:

[0039] The present invention extracts joint point information from the throwing video of curling players, designs a joint angle action evaluation algorithm process based on the human skeleton time series, realizes the evaluation of the throwing action of curling players, provides a reference for the training of curling players, and can play a role in assisting the training of curling players. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0041] Figure 1 The present invention is a flow chart of a method for identifying and evaluating curling player movements. DETAILED DESCRIPTION

[0042] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] The purpose of the present invention is to provide a method for identifying and evaluating the actions of curling players. The method has strong applicability and can evaluate the accuracy of curling player actions through computer vision. The purpose of the present invention is achieved through the following technical solutions: first, through a human posture estimation algorithm, the confidence of human joints and trunk affinity field are predicted, and the skeleton time series information in the curling player throwing video is extracted; secondly, through the normalization of the skeleton graph, the Euclidean distance of the pixel points between the skeleton graphs is used as the matching distance, and the skeleton sequence alignment based on the dynamic programming algorithm is performed to achieve the time series alignment between the skeleton sequence to be matched and the standard skeleton sequence; finally, the curling player throwing action evaluation rule is constructed, and the target joint angle is calculated through the skeleton graph of the curling player human skeleton detection, and the score function is designed using the range of the joint angle to evaluate the action.

[0044] Combination Figure 1 The present invention proposes a method for identifying and evaluating curling player movements, the method comprising:

[0045] Step 1: Use the human posture estimation algorithm to predict the confidence of human joints and the torso affinity field, and extract the skeleton time series information in the curling athlete's throwing video.

[0046] The first step is to detect the human joints of the curling athlete from the video based on the Openpose algorithm for human posture estimation. First, the first ten layers of the VGG-19 network are used to form a feature extraction network to obtain the feature map F; then the confidence S of the human joints is predicted. 1 =ρ 1 (F), each confidence map contains a certain type of joint points; then predict the trunk affinity field L 1 =φ 1 (F), that is, the part between two joints where there is actual connection. 1 and φ 1 They represent the convolutional network for generating the joint confidence map and the convolutional network for generating the torso affinity field in the first stage respectively.

[0047] The second step is to connect the human joint confidence prediction results of the previous stage, the torso affinity field prediction results and the original image features to accurately predict the human joint points.

[0048]

[0049] Among them, ρ t and φ t It is the convolutional network for generating the joint confidence map and the torso affinity field for reasoning in the tth stage.

[0050] Step 3: Based on the predictions in Steps 1 and 2, 18 human joints and lines are predicted from the curling athlete's throwing video, and the curling athlete's human posture time series is extracted. Due to different throwing forces and targets, curling athletes have different throwing action amplitudes and rhythms, resulting in human skeleton time series of different lengths.

[0051] Step 2: By normalizing the skeleton images, taking the Euclidean distance of the pixels between the skeleton images as the matching distance, skeleton sequence alignment based on the dynamic programming algorithm is performed to achieve the temporal alignment between the skeleton sequence to be matched and the standard skeleton sequence.

[0052] Step 1: Propose a skeleton normalization method for the curling player's throwing action. Define the pixel coordinates of the skeleton points of the curling player in a single frame as a set X = {x i |i=1,2,...,18} and Y={y i |i=1,2,...,18}, which represent the coordinate positions of 18 skeleton joints in the horizontal and vertical directions of the image. Define a skeleton image as [X,Y], and calculate the pixel coordinates x of all the skeleton joints of the curling player. min 、x max and pixel coordinate y min ,y max , the coordinate range of the skeleton is obtained, and the range is used as the normalization standard to normalize the pixel values ​​of the skeleton. The normalization method is described as shown in equations (2)-(5):

[0053]

[0054]

[0055]

[0056]

[0057] After normalization, the coordinates of the skeleton will be mapped to [0,1].

[0058] Step 2: Determine the distance between the two curling player skeleton images. First, pre-process the skeletons, filter the joint points of the two curling player skeleton images, retain the common joint point coordinates of the two images, and set the coordinate positions of the remaining joint points to (0,0). m ,Y m ] 0 ,[X n ,Y n ] 0 Normalize according to the method in step 1 to obtain two normalized joint images [X m ,Ym ],[X n ,Y n ] Calculate the pixel Euclidean distance between the same joint points corresponding to the two skeleton images:

[0059]

[0060] where x mi ∈X m ,y mi ∈Y m , x ni ∈X n ,y ni ∈Y n , i=1,2,...,18.

[0061] Step 3: For a video skeleton sequence composed of multiple frames, the matching alignment between the skeletons of the two time series is completed based on the dynamic programming algorithm. The two skeleton sequences are defined as The time series lengths are T 1 and T 2 First, the two skeleton sequences are normalized according to step 1, and then the distance matrix between the two skeleton images is calculated according to formula (6). The path with the minimum distance is calculated according to the dynamic programming algorithm to complete the matching alignment between the two skeleton time series.

[0062] Step 3: Construct the evaluation rules for the curling athlete's throwing action, calculate the target joint angle through the skeleton diagram of the curling athlete's human skeleton detection, and use the range of the joint angle to design a scoring function for action evaluation.

[0063] 1. Divide the throwing action of curling athletes into four sub-actions, including squatting, starting and pulling back, sliding, and throwing. Define the joint angle ranges of the four sub-actions as the rules for evaluating the throwing action of curling athletes: The shoulder extension angle α in the squatting sub-action 1 The range is to Pedal knee angle β 1 The range is to Shoulder abduction angle γ 1 The range is to Shoulder extension angle α during start-up and pull-back 2 The range is to Pedal knee angle β 2 The range is to Shoulder abduction angle γ 2 The range is to Elbow angle when holding the kettle during sliding movements The range is to Shoulder abduction angle γ 3 The range is to Wrist angle θ during hand-out action 4 The range is greater than 85°.

[0064] 2. Design the score function S(α) of the joint angle in the three sub-actions of squatting, starting and pulling back, and sliding i )、S(β j ), S(γ k )and They satisfy Gaussian distribution respectively, as shown in equations (7)-(10).

[0065]

[0066]

[0067]

[0068]

[0069] Among them, i=1,2; j=1,2; k=1,2,3.

[0070] Design the score function S(θ 4 ) as formula (11):

[0071]

[0072] 3. Average the scores of the joint angles in each sub-action interval of the video segment to be evaluated, and then average the scores of the four sub-actions to obtain the score S of the curling athlete's throwing action avg , as shown in formula (12):

[0073]

[0074] Among them, T 1 、T 2 、T 3 、T 4 Indicates the number of frames of the video corresponding to the four sub-actions. and It is the score function corresponding to the joint angle of the tth frame in each sub-action interval.

[0075] The present invention proposes an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of a method for identifying and evaluating curling player movements when executing the computer program.

[0076] The present invention provides a computer-readable storage medium for storing computer instructions, which, when executed by a processor, implement the steps of a method for identifying and evaluating curling player movements.

[0077] The memory in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), and direct RAM bus RAM (DRRAM). It should be noted that the memory of the method described in the present invention is intended to include, but is not limited to, these and any other suitable types of memory.

[0078] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (digital subscriber line, DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a high-density digital video disc (DVD)), or a semiconductor medium (eg, a solid state disc (SSD)).

[0079] In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in a processor or an instruction in the form of software. The steps of the method disclosed in conjunction with the embodiment of the present application can be directly embodied as a hardware processor for execution, or a combination of hardware and software modules in a processor for execution. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory and completes the steps of the above method in conjunction with its hardware. To avoid repetition, it is not described in detail here.

[0080] It should be noted that the processor in the embodiment of the present application can be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method embodiment can be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to perform, or the hardware and software modules in the decoding processor can be combined and performed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0081] The above is a detailed introduction to the curling player action recognition and evaluation method proposed in the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A method for identifying and evaluating curling player movements, Features: The method comprises: Step 1: Use the human posture estimation algorithm to predict the confidence of human joints and the trunk affinity field, and extract the skeleton time series information in the curling athlete's throwing video; Step 2: By normalizing the skeleton images, taking the Euclidean distance of the pixels between the skeleton images as the matching distance, the skeleton sequence alignment based on the dynamic programming algorithm is performed to achieve the temporal alignment between the skeleton sequence to be matched and the standard skeleton sequence; Step 3: construct the curling athlete throwing action evaluation rules, calculate the target joint angle through the skeleton diagram of the curling athlete's human skeleton detection, and use the range of the joint angle to design a scoring function for action evaluation; In step 3, the throwing action of the curling athlete is divided into four sub-actions, including squatting, starting and pulling back, sliding, and throwing; the joint angle ranges of the four sub-actions are defined as the rules for evaluating the throwing action of the curling athlete: the shoulder extension angle α in the squatting sub-action 1 The range is to Pedal knee angle β 1 The range is to Shoulder abduction angle γ 1 The range is to Shoulder extension angle α during start-up and pull-back 2 The range is to Pedal knee angle β 2 The range is to Shoulder abduction angle γ 2 The range is to Elbow angle when holding the kettle during sliding movements The range is to Shoulder abduction angle γ 3 The range is to Wrist angle θ during hand-out action 4 The range is greater than 85°; Design the score function S(α) of the joint angle in the three sub-actions of squatting, starting and pulling back, and sliding i )、S(β j ), S(γ k )and They satisfy Gaussian distribution respectively, as shown in equations (7)-(10): Among them, i = 1, 2; j = 1, 2; k = 1, 2, 3; Design the score function S(θ 4 ) as formula (11): The scores of the joint angles in each sub-action interval of the video segment to be evaluated are averaged, and then the scores of the four sub-actions are averaged to obtain the score S of the curling athlete's throwing action. avg , as shown in formula (12): Among them, T 1 、T 2 、T 3 、T 4 Indicates the number of frames of the video corresponding to the four sub-actions. and It is the score function corresponding to the joint angle of the tth frame in each sub-action interval.

2. The method according to claim 1, Features: The step 1 is specifically as follows: The first step is to detect the human joints of the curling athletes from the video based on the Openpose algorithm; first, the feature extraction network is constructed through the first ten layers of the VGG-19 network to obtain the feature map F; then the confidence S of the human joints is predicted 1 =ρ 1 (F), each confidence map contains a certain type of joint points; then predict the trunk affinity field L 1 =φ 1 (F), that is, the actual connection between the two joints; where ρ 1 and φ 1 They represent the convolutional network for generating the joint confidence map and the convolutional network for generating the trunk affinity field in the first stage respectively; The second step is to connect the human joint confidence prediction results and trunk affinity field prediction results of the previous stage with the original image features to accurately predict the human joint points; Among them, ρ t and φ t It is the convolutional network for generating the joint confidence map and the torso affinity field for reasoning in the tth stage; The third step is to predict 18 human joints and lines from the curling player's throwing video based on the predictions in the first and second steps, and extract the curling player's human posture time series.

3. The method according to claim 2, Features: In step 2, the skeleton normalization method of the curling player's throwing action is specifically as follows: the pixel coordinates of the skeleton points of the curling player in a single frame are defined as a set X = {x i |i=1,2,...,18} and Y={y i |i=1,2,...,18}, which represent the coordinate positions of 18 skeleton joints in the horizontal and vertical directions of the image respectively; a skeleton image is defined as [X,Y], and the pixel coordinates x of all the bone joints of the curling player skeleton are calculated. min 、x max and pixel coordinate y min ,y max , the coordinate range of the skeleton is obtained, and the range is used as the normalization standard to normalize the pixel values ​​of the skeleton; the normalization method is described as shown in formulas (2)-(5): After normalization, the coordinates of the skeleton will be mapped to [0,1].

4. The method according to claim 3, Features: In step 2, the distance between the two curling player skeleton images is determined as follows: first, the skeleton is preprocessed, the joint points of the two curling player skeleton images are screened, the common joint point coordinates of the two images are retained, and the coordinate positions of the remaining joint points are set to (0,0); the two skeleton images are [X m ,Y m ] 0 ,[X n ,Y n ] 0 Normalize and get two normalized joint graphs [X m ,Y m ],[X n ,Y n ]; Calculate the pixel Euclidean distance between the same joint points corresponding to the two skeleton images: where x mi ∈ X m and y mi ∈ Y m and x ni ∈ X n and y ni ∈ Y n and i = 1, 2, ..., 18.

5. The method according to claim 4, Features: In step 2, for a video skeleton sequence composed of multiple frames, the matching between two time series skeletons is completed based on the dynamic programming algorithm. Specifically, the two skeleton sequences are defined as The time series lengths are T 1 and T 2 First, the two skeleton sequences are normalized, and then the distance matrix between the two skeleton graphs is calculated according to formula (6). The path with the minimum distance is calculated according to the dynamic programming algorithm to complete the matching between the two skeleton time series.

6. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, It is characterized in that When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

7. A computer-readable storage medium for storing computer instructions, It is characterized in that When the computer instructions are executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.