A skeleton-based lightweight martial arts action recognition and scoring method and device

Abstract

The embodiment of the present application discloses a skeleton-based lightweight martial arts action recognition and scoring method and device; the method comprises: obtaining a student's martial arts action frame image and generating skeleton data from the martial arts action frame image; analyzing the joint types and the correlation between the joints of the skeleton data of the same frame image according to a preset graph convolutional neural network layer; obtaining aggregated data of all action frame images; performing martial arts classification according to the aggregated data to obtain a classification output result; comparing the classification output result with preset standard action data to perform martial arts action scoring; the embodiment of the present application intelligently scores martial arts actions, improves the accuracy of the scoring results, has a simple network structure, and is time-saving, thereby improving the efficiency of scoring and reducing the work pressure of teachers.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of motion recognition technology, and more particularly to a skeleton-based lightweight martial arts motion recognition and scoring method and device. Background Art

[0002] With the progress and development of education, in the education of students, it is advocated not only to carry out ideological quality education, but also to pay attention to students' physical health literacy. For this reason, some schools have included martial arts and gymnastics in the scope of examinations. Among them, manually scoring the martial arts and gymnastics movements of a large number of students is not objective enough, and the speed is slow, and the efficiency and accuracy are low. Therefore, the human body movements are recognized by the neural network model of deep learning to improve the scoring rate and accuracy of martial arts movements. However, the existing analysis of martial arts movements uses complex deep network models, which require a large number of parameters, require a considerable amount of time, and are inefficient. Summary of the invention

[0003] The embodiments of the present application provide a skeleton-based lightweight martial arts action recognition and scoring method and device to solve the problem that the existing analysis of martial arts actions uses a complex deep network model, requires a large number of parameters, consumes a considerable amount of time, and has low efficiency.

[0004] In a first aspect, an embodiment of the present application provides a skeleton-based lightweight martial arts action recognition and scoring method, the method comprising the following steps:

[0005] Obtaining a martial arts action frame image of a student, and generating skeleton data from the martial arts action frame image;

[0006] According to the preset graph convolutional neural network layer, the joint types and correlations between joints of the skeleton data of the same frame are analyzed;

[0007] Get the aggregate data of all action frames;

[0008] Classify martial arts according to the aggregated data to obtain a classification output result;

[0009] The classification output result is compared with preset standard action data to perform martial arts action scoring.

[0010] Furthermore, the step of obtaining a martial arts action frame diagram of a student and generating skeleton data from the martial arts action frame diagram includes:

[0011] Obtaining the student's martial arts action frame images, and dividing the video action into multiple martial arts action frame images;

[0012] Capturing the student's martial arts action data through a depth sensor, wherein each frame of the martial arts action frame image contains multiple joints of the human body, and generating skeleton data;

[0013] Among them, all the joints of the skeleton sequence in the skeleton data are represented by the set S:

[0014]

[0015] in, Indicates the joint type k at frame t, joint The position is expressed in three-dimensional coordinates P t,k To express it, the speed is the difference between the positions of two adjacent frames, expressed as V t,k express;

[0016] Embed location information:

[0017]

[0018] Among them, W1 and W2 are weight matrices, b1 and b2 are bias coefficients, and σ represents the activation function;

[0019] Embed the difference in position:

[0020]

[0021] Among them, W3 and W4 are weight matrices, b3 and b4 are bias coefficients, and σ represents the activation function;

[0022] Fusion and The kinetic parameters are obtained and recorded as Z t,k .

[0023] Furthermore, the method of analyzing the joint types and the correlation between the joints of the skeleton data of the same frame image according to the preset graph convolutional neural network layer includes:

[0024] We learn between joints in the same frame, merge the semantics of joint types into the graph convolutional neural network layer, and learn content-adaptive graphs.

[0025] Learn content-adaptive graph connections through joint type semantics and dynamics data to obtain the relationship between connection weights and joints;

[0026] For joints of type k, use vector j k To represent, we get the corresponding embedding

[0027] Among them, W5 and W6 are weight matrices, b5 and b6 are bias coefficients, and σ represents the activation function;

[0028] Then the joint of type k in frame t is represented as All joints in frame t are represented by Zt =(z t,1 ;...;z t,J ) means, according to:

[0029]

[0030] Model the edge weights between joints i and j in the same frame;

[0031] Among them, θ and All are conversion functions;

[0032] Calculate the affinity between all joints in a frame and get the normalized adjacency matrix G t ,according to:

[0033] Z′ t =G t Z t W y +Z t W z ;

[0034] Get the output weight matrix Z' t , share it with its frame, where W y and W z are transformation matrices.

[0035] Furthermore, before obtaining the aggregated data of all action frame graphs, the process further includes:

[0036] A lightweight multi-modal attention module is inserted into the graph convolutional neural network to adaptively recalibrate the activations of joints, frames, and channels of different data samples by providing different attentions from spatial, temporal, and feature perspectives.

[0037] Furthermore, the obtaining of aggregated data of all action frame graphs includes:

[0038] Use frame index semantics to model the relationship between frames, use two convolutional neural network layers to model the dependency between frames, enhance the learning feature representation capability, merge all joint information in a frame, and then aggregate the information of all frames to obtain aggregated data;

[0039] Here, we use the vector f t To represent the frame index, the embedding is calculated as

[0040]

[0041] Among them, W7 and W8 are weight matrices, b7 and b8 are bias coefficients, and σ represents the activation function;

[0042] Then the joint of type k in frame t is represented as

[0043] Furthermore, the martial arts classification is performed according to the aggregated data to obtain the classification output result, including:

[0044] Based on the traditional random vector function chain network to build a width learning system,

[0045] Use the width learning system to complete the classification task, map the aggregated data to features, obtain feature nodes, and then enhance the mapped features to obtain enhanced nodes, according to:

[0046]

[0047] Get the features of feature nodes and enhanced nodes, where W M , W E is the connection weight;

[0048] The output coefficients are obtained by calculating the pseudo-inverse of the corresponding mapping matrix, combining the feature nodes and the enhancement nodes, according to:

[0049] ο=[Z|H]W°;

[0050] Get the classification output results.

[0051] Furthermore, the comparing the classification output result with the preset standard action data to score the martial arts action includes:

[0052] Collect standard action data in advance;

[0053] Use frame-by-frame matching method to score martial arts movements and divide them into several categories;

[0054] Compare the classification output results with the standard action data. If the match is successful, it means that the action meets the standard and scores are given. Otherwise, no score is given. Finally, the total score is accumulated.

[0055] In a second aspect, the embodiment of the present application further provides a skeleton-based lightweight martial arts action recognition and scoring device, comprising:

[0056] An image acquisition module is used to acquire a student's martial arts action frame image and generate skeleton data from the martial arts action frame image;

[0057] An image analysis module is used to analyze the joint types and correlations between joints of the skeleton data of the same frame image according to a preset graph convolutional neural network layer;

[0058] A data acquisition module is used to obtain aggregated data of all action frame graphs;

[0059] A result output module, used for classifying martial arts according to the aggregated data to obtain a classification output result;

[0060] The scoring output module is used to compare the classification output results with preset standard action data to perform martial arts action scoring.

[0061] In a third aspect, an embodiment of the present application further provides a computer device, including: a memory and one or more processors;

[0062] The memory is used to store one or more programs;

[0063] When the one or more programs are executed by the one or more processors, the one or more processors implement the above-mentioned skeleton-based lightweight martial arts action recognition and scoring method.

[0064] In a fourth aspect, an embodiment of the present application further provides a storage medium comprising computer executable instructions, which, when executed by a computer processor, are used to execute a skeleton-based lightweight martial arts action recognition and scoring method as described above.

[0065] The embodiment of the present application obtains the student's martial arts action frame image and generates skeleton data from the martial arts action frame image; analyzes the joint types and the correlation between the joints of the skeleton data of the same frame image according to a preset graph convolutional neural network layer; obtains aggregated data of all action frame images; classifies martial arts according to the aggregated data to obtain a classification output result; compares the classification output result with preset standard action data to score the martial arts actions; intelligently scores the martial arts actions, thereby improving the accuracy of the scoring results, simplifying the network structure, and shortening the time consumption, thereby improving the efficiency of scoring and reducing the work pressure of teachers. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 It is a flowchart of a skeleton-based lightweight martial arts action recognition and scoring method provided in an embodiment of the present application;

[0067] Figure 2 It is a network framework diagram of a width learning system of a skeleton-based lightweight martial arts action recognition and scoring method provided in an embodiment of the present application;

[0068] Figure 3 It is a network framework diagram of a skeleton-based lightweight martial arts action recognition and scoring method provided in an embodiment of the present application;

[0069] Figure 4 It is a structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0070] In order to make the purpose, technical scheme and advantages of the present application clearer, the specific embodiments of the present application are further described in detail below in conjunction with the accompanying drawings. It is understood that the specific embodiments described herein are only used to explain the present application, rather than to limit the present application. It should also be noted that, for the convenience of description, only the part related to the present application but not all the contents are shown in the accompanying drawings. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow chart describes each operation (or step) as a sequential process, many of the operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of each operation can be rearranged. The process can be terminated when its operation is completed, but it can also have additional steps not included in the accompanying drawings. The process can correspond to a method, a function, a procedure, a subroutine, a subprogram, etc.

[0071] At present, human action recognition has been widely used in the field of computer vision in recent years, and it can cooperate with multiple different research directions in cross-field. Among the action recognition methods based on deep learning, they can be roughly divided into three categories: methods based on recurrent neural networks (RNNs), methods based on convolutional neural networks (CNNs), and methods based on graph convolutional networks (GCNs). Since the structure of the human body is in the form of a graph, but the methods based on RNNs and CNNs ignore the structural information of the human body, people are more inclined to use methods based on GCNs in the latest research methods. Compared with the traditional method of using RGB videos for recognition, the skeleton-based method has also attracted more and more attention due to its strong adaptability to dynamic environments and complex backgrounds. In skeleton data, each joint of the human body is identified by joint type, frame index, and three-dimensional position. Therefore, GCN is applied to skeleton-based human action recognition, with vertices representing joint points and edges representing the connections between joint points, which has shown good performance in current research.

[0072] In the study of human action recognition based on skeletons, researchers have developed a variety of very efficient network structures by continuously modifying and improving past network structures. Since the first application of spatiotemporal graph models to human action recognition, new research has been endlessly improving network structures. For example, the proposed spatial reasoning network (SRN) that learns the spatial structure within a frame through a residual graph network, and the proposed directed graph relationship modeling that uses the nodes and edges of the skeleton graph as a two-stream network input model and incorporates the features of the edges to improve the effect. By changing the connection relationship, a spatiotemporal relationship model of the skeleton sequence is established, such as the proposed spatiotemporal graph routing method (STGR) that learns the intrinsic high-order relationship between joints.

[0073] However, the above methods all use complex deep network models, which require a huge number of parameters. Since the amount of data in the NTU60 (skeletal behavior dataset) and NTU120 public datasets used in this direction is very large, coupled with the complex network structure, the related research has higher requirements for hardware and requires a considerable amount of time.

[0074] In this regard, the embodiment of the present application establishes a skeleton-based lightweight martial arts action recognition and scoring method to avoid the problem that the existing analysis of martial arts actions uses complex deep network models, requires a large number of parameters, consumes a considerable amount of time, and has low efficiency.

[0075] The skeleton-based lightweight martial arts action recognition and scoring method provided in the embodiment can be performed by a skeleton-based lightweight martial arts action recognition and scoring device, which can be implemented by software and / or hardware and integrated in a skeleton-based lightweight martial arts action recognition and scoring device. The skeleton-based lightweight martial arts action recognition and scoring device can be a computer or other device.

[0076] Figure 1 A flowchart of a skeleton-based lightweight martial arts action recognition and scoring method provided in an embodiment of the present application. Figure 1 , the method comprises the following steps:

[0077] Step 110, obtaining a martial arts action frame image of the student, and generating skeleton data from the martial arts action frame image;

[0078] Specifically, a martial arts action frame image of the student is obtained, and the video action is divided into multiple martial arts action frame images; the martial arts action data of the student is captured by a depth sensor, and each frame of the martial arts action frame image contains multiple joints of the human body, and skeleton data is generated; optionally, the action video or action data of the student can be captured by a camera or sensor, and then the video is divided into frames.

[0079] Among them, all the joints of the skeleton sequence in the skeleton data are represented by the set S:

[0080]

[0081] in, Indicates the joint type k at frame t, joint The position is expressed in three-dimensional coordinates P t,k To express it, the speed is the difference between the positions of two adjacent frames, expressed as V t,k express;

[0082] Embed / encode location information:

[0083]

[0084] Among them, W1 and W2 are weight matrices, b1 and b2 are bias coefficients, and σ represents the activation function;

[0085] Embed the difference in position:

[0086]

[0087] Among them, W3 and W4 are weight matrices, b3 and b4 are bias coefficients, and σ represents the activation function;

[0088] Fusion and The kinetic parameters are obtained and recorded as Z t,k .

[0089] Step 120: Analyze the joint types and correlations between joints of the skeleton data of the same frame according to the preset graph convolutional neural network layer.

[0090] Specifically, the joints in the same frame are learned, the joint type semantics are merged into the graph convolutional neural network layer, and the content adaptive graph is learned; the content adaptive graph connection is learned through the joint type semantics and dynamics data to obtain the relationship between the connection weight and the joints; for the joint of type k, the vector j is used k To represent, we get the corresponding embedding

[0091] Among them, W5 and W6 are weight matrices, b5 and b6 are bias coefficients, and σ represents the activation function;

[0092] Then the joint of type k in frame t is represented as All joints in frame t are represented by Z t =(z t,1 ;...;z t,J ) means, according to:

[0093]

[0094] Model the edge weights between joints i and j in the same frame;

[0095] Among them, θ and All are conversion functions;

[0096] Calculate the affinity between all joints in a frame and get the normalized adjacency matrix G t ,according to:

[0097] Z′ t =G t Z t W y +Zt W z ;

[0098] Get the output weight matrix Z' t , share it with its frame, where W y and W z are transformation matrices.

[0099] Optionally, a lightweight multi-modal attention module is inserted into the graph convolutional neural network to adaptively recalibrate the activations of joints, frames, and channels of different data samples by providing different attentions from spatial, temporal, and feature perspectives.

[0100] As described above, the embodiments of the present application combine semantic information with a graph convolutional neural network to improve the network learning ability; and use a lightweight multi-modal attention module to enhance network performance and maintain network stability and recognition accuracy.

[0101] Step 130: Obtain aggregated data of all action frame graphs.

[0102] Specifically, the above obtains the joint correlation in a frame image. For different frame images, the embodiment of the present application uses frame index semantics to model the relationship between frames, models the inter-frame dependency through two convolutional neural network layers, enhances the learning feature representation capability, merges all joint information in a frame image, and then aggregates the information of all frame images to obtain aggregated data;

[0103] Here, we use the vector f t To represent the frame index, the embedding is calculated as

[0104]

[0105] Among them, W7 and W8 are weight matrices, b7 and b8 are bias coefficients, and σ represents the activation function;

[0106] Then the joint of type k in frame t is represented as

[0107] Step 140, classifying martial arts according to the aggregated data to obtain a classification output result;

[0108] Specifically, a width learning system is constructed based on a traditional random vector function chain network, and the classification task is completed using the width learning system. The aggregated data is mapped to features to obtain feature nodes, and the mapped features are enhanced to obtain enhanced nodes, according to:

[0109]

[0110] Get the features of feature nodes and enhanced nodes, where W M , WE is the connection weight;

[0111] The output coefficients are obtained by calculating the pseudo-inverse of the corresponding mapping matrix, combining the feature nodes and the enhancement nodes, according to:

[0112] ο=[Z|H]W°;

[0113] Get the classification output results, among which, please refer to Figure 2 , is the network framework diagram of the width learning system, Y is the fusion of feature node Z and enhancement node H, and W is the connection weight.

[0114] Step 150: Compare the classification output result with preset standard action data to perform martial arts action scoring.

[0115] Specifically, standard action data is collected in advance; martial arts actions are scored using a frame-by-frame matching method, and martial arts actions are divided into several categories; optionally, corresponding labels are added to different martial arts actions.

[0116] Among them, the classification output results and standard action data are compared. Optionally, the above classification output results, that is, the student's martial arts action labels and standard action labels are compared. If the match is successful, it means that the action meets the standard and scores, otherwise no score is scored, and finally the total score is accumulated.

[0117] Exemplarily, the embodiment of the present application uses a frame-by-frame matching method to score martial arts movements, and divides the movements into fifteen types, including side-by-side fisting, bow step punching, kicking with a spring leg, bow step left and right palm pushing, blocking and pushing palm, upper leg kicking, horse stance push palm, side-by-side fisting, bow step double punching, fist holding and leg kicking, horse stance blocking and striking, bow step left and right punching, bow step turning around to punch, turning over and chopping, and kicking with a spring leg. Standard movement data is collected in advance, and the standard movement is compared with the classification output result. If the match is successful, it means that the movement meets the standard and is scored, otherwise no score is scored, and finally the total score is accumulated.

[0118] For example, please refer to Figure 3 , which is a network framework diagram of an embodiment of the present application.

[0119] As mentioned above, the embodiment of the present application assists teachers in scoring martial arts movements by proposing a skeleton-based lightweight martial arts action recognition and scoring method. In the current era of the combination of artificial intelligence and education, such intelligent auxiliary teaching methods reduce the work pressure for teachers and can also improve the accuracy of the scoring results. The action recognition network proposed in the embodiment of the present application combines semantic information to simplify the network structure, add attention modules, improve attention to important joints, and use a width learning system to ensure the stability and accuracy of the network. The skeleton-based lightweight martial arts action recognition network proposed in the embodiment of the present application, with its extremely short running time and relatively stable accuracy, can be used as an auxiliary reference tool for teachers in martial arts teaching and scoring.

[0120] Based on the above embodiments, Figure 4 This is a schematic diagram of the structure of a skeleton-based lightweight martial arts action recognition and scoring device provided in an embodiment of the present application. Figure 4 The skeleton-based lightweight martial arts action recognition and scoring device provided in this embodiment specifically includes: an image acquisition module 101, an image analysis module 102, a data acquisition module 103, a result output module 104 and a scoring output module 105.

[0121] The image acquisition module 101 is used to acquire the student's martial arts action frame image and generate skeleton data from the martial arts action frame image; the image analysis module 102 is used to analyze the joint types and the correlation between the joints of the skeleton data of the same frame image according to the preset graph convolutional neural network layer; the data acquisition module 103 is used to acquire the aggregated data of all action frame images; the result output module 104 is used to classify martial arts according to the aggregated data to obtain the classification output result; the scoring output module 105 is used to compare the classification output result with the preset standard action data to score the martial arts actions.

[0122] In the above, the martial arts action frame images of students are obtained, and skeleton data is generated from the martial arts action frame images; according to the preset graph convolutional neural network layer, the joint types of the skeleton data of the same frame image and the correlation between the joints are analyzed; the aggregated data of all action frame images are obtained; martial arts are classified according to the aggregated data to obtain the classification output result; the classification output result is compared with the preset standard action data to score the martial arts actions; the martial arts actions are scored intelligently, the accuracy of the scoring results is improved, the network structure is simple, the time consumption is short, the scoring efficiency is improved, and the work pressure of teachers is reduced.

[0123] The skeleton-based lightweight martial arts action recognition and scoring device provided in the embodiment of the present application can be used to execute the skeleton-based lightweight martial arts action recognition and scoring method provided in the above embodiment, and has corresponding functions and beneficial effects.

[0124] The embodiment of the present application also provides a computer device, which can integrate the skeleton-based lightweight martial arts action recognition and scoring device provided in the embodiment of the present application. Figure 4 Schematic diagram of a computer device provided in an embodiment of the present application. Figure 4 The computer device includes: an input device 43, an output device 44, a memory 42 and one or more processors 41; the memory 42 is used to store one or more programs; when the one or more programs are executed by the one or more processors 41, the one or more processors 41 implement the skeleton-based lightweight martial arts action recognition and scoring method provided in the above embodiment. The input device 43, the output device 44, the memory 42 and the processor 41 can be connected via a bus or other means. Figure 4 The example of connecting through bus is taken in the following.

[0125] The processor 41 executes various functional applications and data processing of the device by running the software programs, instructions and modules stored in the memory 41, that is, realizes the above-mentioned skeleton-based lightweight martial arts action recognition and scoring method.

[0126] The computer device provided above can be used to execute the skeleton-based lightweight martial arts action recognition and scoring method provided in the above embodiment, and has corresponding functions and beneficial effects.

[0127] The embodiment of the present application also provides a storage medium containing computer executable instructions, which are used to execute a skeleton-based lightweight martial arts action recognition and scoring method when executed by a computer processor. The skeleton-based lightweight martial arts action recognition and scoring method includes: obtaining a student's martial arts action frame image, and generating skeleton data from the martial arts action frame image; analyzing the joint types and correlations between joints of the skeleton data of the same frame image according to a preset graph convolutional neural network layer; obtaining aggregated data of all action frame images; performing martial arts classification according to the aggregated data to obtain a classification output result; and comparing the classification output result with preset standard action data to perform martial arts action scoring.

[0128] Storage medium - any of various types of memory devices or storage devices. The term "storage medium" is intended to include: installation media, such as CD-ROM, floppy disk or tape devices; computer device memory or random access memory, such as DRAM, DDRRAM, SRAM, EDORAM, Rambus RAM, etc.; non-volatile memory, such as flash memory, magnetic media (such as hard disk or optical storage); registers or other similar types of memory elements, etc. The storage medium may also include other types of memory or combinations thereof. In addition, the storage medium may be located in a first computer device in which the program is executed, or may be located in a different second computer device, which is connected to the first computer device via a network (such as the Internet). The second computer device can provide program instructions to the first computer for execution. The term "storage medium" may include two or more storage media that may reside in different locations (for example, in different computer devices connected via a network). The storage medium may store program instructions (for example, embodied as a computer program) that can be executed by one or more processors.

[0129] Of course, the storage medium containing computer executable instructions provided in the embodiment of the present application is not limited to the skeleton-based lightweight martial arts action recognition and scoring method as described above, and can also execute related operations in the skeleton-based lightweight martial arts action recognition and scoring method provided in any embodiment of the present application.

[0130] The skeleton-based lightweight martial arts action recognition and scoring device, storage medium and computer equipment provided in the above embodiments can execute the skeleton-based lightweight martial arts action recognition and scoring method provided in any embodiment of the present application. For technical details not described in detail in the above embodiments, please refer to the skeleton-based lightweight martial arts action recognition and scoring method provided in any embodiment of the present application.

[0131] The above are only preferred embodiments of the present application and the technical principles used. The present application is not limited to the specific embodiments described herein, and various obvious changes, readjustments and substitutions that can be made by those skilled in the art will not deviate from the scope of protection of the present application. Therefore, although the present application is described in more detail through the above embodiments, the present application is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present application, and the scope of the present application is determined by the scope of the claims.

Claims

1. A skeleton-based lightweight martial arts action recognition and scoring method, characterized in that: The method comprises the following steps: Obtaining a martial arts action frame image of a student, and generating skeleton data from the martial arts action frame image; According to the preset graph convolutional neural network layer, the joint types and correlations between joints of the skeleton data of the same frame are analyzed; Get the aggregate data of all action frames; Classify martial arts according to the aggregated data to obtain a classification output result; Comparing the classification output result with preset standard action data to perform martial arts action scoring; The step of obtaining a martial arts action frame diagram of a student and generating skeleton data from the martial arts action frame diagram includes: Obtaining the student's martial arts action frame images, and dividing the video action into multiple martial arts action frame images; Capturing the student's martial arts action data through a depth sensor, wherein each frame of the martial arts action frame image contains multiple joints of the human body, and generating skeleton data; Among them, all the joints of the skeleton sequence in the skeleton data are represented by the set S: in, Indicates the joint type k at frame t, joint The position is expressed in three-dimensional coordinates P t,k To express it, the speed is the difference between the positions of two adjacent frames, expressed as V t,k express; Embed location information: Among them, W1 and W2 are weight matrices, b1 and b2 are bias coefficients, and σ represents the activation function; Embed the difference in position: Among them, W3 and W4 are weight matrices, b3 and b4 are bias coefficients, and σ represents the activation function; Fusion and The kinetic parameters are obtained and recorded as Z t,k ; The method of analyzing the joint types and the correlation between the joints of the skeleton data of the same frame image according to the preset graph convolutional neural network layer includes: We learn between joints in the same frame, merge the semantics of joint types into the graph convolutional neural network layer, and learn content-adaptive graphs. Learn content-adaptive graph connections through joint type semantics and dynamics data to obtain the relationship between connection weights and joints; For joints of type k, use vector j k To represent, we get the corresponding embedding Among them, W5 and W6 are weight matrices, b5 and b6 are bias coefficients, and σ represents the activation function; Then the joint of type k in frame t is represented as All joints in frame t are represented by Z t =(z t,1 ;...;z t,J ) means, according to: Model the edge weights between joints i and j in the same frame; Among them, θ and All are conversion functions; Calculate the affinity between all joints in a frame and get the normalized adjacency matrix G t ,according to: WITH' t =G t WITH t IN y +Z t IN z ; Get the output weight matrix Z' t , share it with its frame, where W y and W z are transformation matrices; The obtaining of aggregated data of all action frame graphs includes: Use frame index semantics to model the relationship between frames, use two convolutional neural network layers to model the dependency between frames, enhance the learning feature representation capability, merge all joint information in a frame, and then aggregate the information of all frames to obtain aggregated data; Here, we use the vector f t To represent the frame index, the embedding is calculated as Among them, W7 and W8 are weight matrices, b7 and b8 are bias coefficients, and σ represents the activation function; Then the joint of type k in frame t is represented as 2. The skeleton-based lightweight martial arts action recognition scoring method according to claim 1, characterized in that: Before obtaining the aggregated data of all action frame graphs, the method further includes: A lightweight multi-modal attention module is inserted into the graph convolutional neural network to adaptively recalibrate the activations of joints, frames, and channels of different data samples by providing different attentions from spatial, temporal, and feature perspectives.

3. The skeleton-based lightweight martial arts action recognition scoring method according to claim 1 is characterized in that: The martial arts classification is performed according to the aggregated data to obtain the classification output result, including: Based on the traditional random vector function chain network to build a width learning system, Use the width learning system to complete the classification task, map the aggregated data to features, obtain feature nodes, and then enhance the mapped features to obtain enhanced nodes, according to: Get the features of feature nodes and enhanced nodes, where W M , W E is the connection weight; The output coefficients are obtained by calculating the pseudo-inverse of the corresponding mapping matrix, combining the feature nodes and the enhancement nodes, according to: ο=[Z|H]W o ; Get the classification output results.

4. The skeleton-based lightweight martial arts action recognition scoring method according to claim 1, characterized in that: The comparing the classification output result with the preset standard action data to score the martial arts action includes: Collect standard action data in advance; Use frame-by-frame matching method to score martial arts movements and divide them into several categories; Compare the classification output results with the standard action data. If the match is successful, it means that the action meets the standard and scores are given. Otherwise, no score is given. Finally, the total score is accumulated.

5. A skeleton-based lightweight martial arts action recognition and scoring device, characterized in that: The method for implementing the skeleton-based lightweight martial arts action recognition and scoring method as claimed in any one of claims 1 to 4 comprises: An image acquisition module is used to acquire a student's martial arts action frame image and generate skeleton data from the martial arts action frame image; An image analysis module is used to analyze the joint types and correlations between joints of the skeleton data of the same frame image according to a preset graph convolutional neural network layer; A data acquisition module is used to obtain aggregated data of all action frame graphs; A result output module, used for classifying martial arts according to the aggregated data to obtain a classification output result; The scoring output module is used to compare the classification output results with preset standard action data to perform martial arts action scoring.

6. A computer device, characterized in that: include: memory and one or more processors; The memory is used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement a skeleton-based lightweight martial arts action recognition and scoring method as described in any one of claims 1-4.

7. A storage medium containing computer executable instructions, characterized in that: The computer executable instructions are used to execute the skeleton-based lightweight martial arts action recognition and scoring method as described in any one of claims 1-4 when executed by a computer processor.

Images