Automatic motor skill assessment method and system based on image analysis

Abstract

The invention discloses an automatic exercise skill assessment method and system based on image analysis. The system comprises a data acquisition module, an exercise assessment module and a training guidance module. The data acquisition module is used for acquiring motion data and motion videos of a to-be-tested student; the motion evaluation module is used for acquiring a video frame set by using a target detection algorithm and an image threshold segmentation technology, acquiring a trajectory graph structure by establishing a three-dimensional coordinate system, acquiring a trajectory graph set, establishing a motion evaluation model according to the trajectory graph set by using a graph convolutional neural network and a multi-task learning strategy, and performing motion evaluation on the motion evaluation model. Inputting the trajectory diagram structure into an action evaluation model to obtain action evaluation information; the training guidance module is used for acquiring sports teaching videos and text description, extracting video key phrases by using a keyword extraction technology, and acquiring recommended videos through keyword tags and the video key phrases; therefore, the user can directly and conveniently find the motion defects of the to-be-tested student, and each motion motion of the to-be-tested student can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of graphic data reading, and in particular to a method and system for automatically evaluating sports skills based on image analysis. Background Art

[0002] In recent years, with the continuous reform and development of physical education, the assessment of sports skills has received great attention. At the same time, with the development of computer technology, the combination of sports skills assessment and computer technology has become increasingly close, providing new opportunities for the innovation and development of physical education.

[0003] At present, a Chinese invention with publication number CN114931743B discloses a motion evaluation method, a motion evaluation device, an electronic device and a readable storage medium, which obtain a first motion image of a user, and identify the first coordinates of the user's skeletal key points in the motion preparation stage based on the first motion image, and determine a benchmark action based on the first coordinates; obtain a second motion image of the user, and identify the second coordinates of the user's skeletal key points in the motion stage based on the second motion image; evaluate the standard degree of the motion action corresponding to the second coordinate based on the second coordinate and the benchmark action; generate the user's motion result based on the user information and the standard degree of the motion action. Although the accuracy of identifying the standard degree of the user's motion action is achieved and the user's motion effect is improved, no motion evaluation is performed on each motion part of the user, thereby obtaining the motion problems of each motion part, and providing the user with motion teaching guidance based on the motion problems, which is not conducive to the user recognizing and improving their own motion problems. Summary of the Invention

[0004] The technical problem solved by the present invention is that the existing technology does not perform exercise assessment on each moving part of the user, thereby obtaining the exercise problems of each moving part, and providing exercise teaching guidance to the user based on the exercise problems, which is not conducive to the user to understand and improve their own exercise problems.

[0005] To solve the above technical problems, in a first aspect, the present invention provides an automatic sports skill assessment system based on image analysis, comprising a data acquisition module, a sports assessment module and a training guidance module;

[0006] The data acquisition module is used to obtain the motion data and motion video of the trainee to be tested, wherein the motion data includes the trainee's facial image and basic information of the trainee;

[0007] The motion assessment module is used to use a target detection algorithm and an image threshold segmentation technique to obtain a target motion video of the trainee to be tested, cut the target motion video into video frames according to a preset time interval, obtain a video frame set, obtain the coordinates of the motion parts by establishing a three-dimensional coordinate system, and obtain a trajectory graph structure of each motion part through the motion part coordinates to obtain a trajectory atlas, use a graph convolutional neural network and a multi-task learning strategy to establish a motion assessment model based on the trajectory atlas structure, input the trajectory graph structure into the motion assessment model, obtain motion assessment information, and obtain the trainee's motion assessment information through the motion assessment information;

[0008] The training guidance module is used to use big data technology to obtain sports teaching videos and text descriptions from the network platform, obtain video descriptions based on the sports teaching videos and text descriptions, and use keyword extraction technology to obtain video keyword groups for each sports teaching video based on the video descriptions, obtain recommended videos through keyword tags and the video keyword groups, and recommend corresponding sports teaching videos to students.

[0009] As a preferred embodiment of the automatic sports skill assessment system based on image analysis of the present invention,

[0010] The data acquisition module includes a data acquisition unit and a video acquisition unit;

[0011] The data acquisition unit is used to collect the facial image and basic information of the students to be tested, wherein the basic information includes the student ID, name, gender and age of the students;

[0012] The video acquisition unit is used to collect the motion video of the trainee to be tested using a visual sensor.

[0013] As a preferred embodiment of the automatic sports skill assessment system based on image analysis of the present invention,

[0014] The motion assessment module includes an action assessment unit and a motion assessment unit;

[0015] The action evaluation unit is used to use a target detection algorithm to capture the target of the student to be tested in the action video according to the student's facial image, and use image segmentation to obtain the target motion video of the student to be tested, cut the target motion video into video frames according to a preset time interval to obtain a video frame set sorted by time, establish a three-dimensional coordinate system according to the image frames of the video frame set, the horizontal plane is the x-axis, the vertical plane is the y-axis, and the vertical longitudinal direction is the z-axis, and obtain the coordinates of the moving parts according to the three-dimensional coordinate system, the moving part coordinates include the left arm joint coordinates, the left hand joint coordinates, the right arm joint coordinates, the right hand joint coordinates, the left leg joint coordinates, the left foot joint coordinates, the right leg joint coordinates, the waist joint coordinates, the left shoulder joint coordinates and the right shoulder joint coordinates, connect the moving part coordinates in time series to obtain the motion trajectory graph of each moving part, construct a trajectory graph structure according to the motion trajectory graph, a node of the trajectory graph structure represents a moving part node coordinate and includes the moving part node coordinate information, an edge of the trajectory graph structure represents the connection relationship between two moving part nodes and includes edge information, the edge information includes the node distance and the node angle, and input the trajectory graph structure into the action evaluation model to obtain action evaluation information;

[0016] The process of establishing the action evaluation model includes:

[0017] Obtain a motion video set, perform data annotation on the motion video set to obtain evaluation labels, the evaluation labels include score labels and keyword labels, obtain a trajectory atlas based on the motion video set, the trajectory atlas is a collection of trajectory graph structures containing the motion video set, input the trajectory graph structure into the first graph convolution layer of the graph convolutional neural network, extract node features and edge features of the trajectory graph structure, construct a first node feature matrix based on the node features, construct a first adjacency matrix based on the edge features, input the first node feature matrix and the first adjacency matrix into the second graph convolution layer, use the second graph convolution layer to normalize the first adjacency matrix to obtain a second adjacency matrix, and update the node feature matrix based on the second adjacency matrix, repeat the graph convolution operation l times to obtain a feature matrix, set corresponding evaluation output layers for different motion parts, and input the feature matrix into the evaluation output layer for training, with the feature matrix as input and the evaluation label as output, until the output fit reaches an expected threshold, then stop training, and obtain the motion evaluation model;

[0018] The motion evaluation unit is used to retrieve the motion evaluation score and keyword tag in the motion evaluation information, perform weighted calculation on the motion evaluation score of each motion part according to a certain weight to obtain a motion evaluation score, and merge the motion evaluation score and keyword tag to obtain motion evaluation information;

[0019] As a preferred embodiment of the automatic sports skill assessment system based on image analysis of the present invention,

[0020] The training guidance module includes a video classification unit and a video guidance unit;

[0021] The video classification unit is used to use big data technology to obtain sports teaching videos and text descriptions from the network platform, use speech recognition technology to obtain voice descriptions based on the sports teaching videos, merge the text descriptions and the voice descriptions to obtain video descriptions, and use the Yake method to extract video keywords from the video descriptions, and arrange the video keywords from large to small according to keyword weights to obtain video keyword groups for each video description;

[0022] The video guidance unit is used to match keyword tags and video keyword groups, and obtain the relevance between keyword tags and each sports teaching video based on the first mapping relationship, and sort the sports teaching videos in descending order according to the relevance to obtain a video recommendation sequence, and recommend the sports teaching videos to students based on the video recommendation sequence.

[0023] As a preferred embodiment of the automatic sports skill assessment system based on image analysis of the present invention,

[0024] The operation function of the graph convolution operation is as follows:

[0025]

[0026] Among them, H (l+1) is the node feature matrix of layer l+1, H (l) is the node feature matrix of layer l, is the adjacency matrix of the self-loop, for The degree matrix, W (l) is the learnable weight matrix of the lth layer, and σ is the activation function.

[0027] As a preferred embodiment of the automatic sports skill assessment system based on image analysis of the present invention,

[0028] The keyword tags include action evaluation tags, which are keyword tags for evaluating actions of various joints.

[0029] As a preferred embodiment of the automatic sports skill assessment system based on image analysis of the present invention,

[0030] The evaluation output layer includes a left arm movement evaluation output layer, a left hand movement evaluation output layer, a right arm movement evaluation output layer, a right hand movement evaluation output layer, a left leg movement evaluation output layer, a left foot movement evaluation output layer, a right leg movement evaluation output layer, a waist movement evaluation output layer, a left shoulder movement evaluation output layer and a right shoulder movement evaluation output layer.

[0031] As a preferred embodiment of the automatic sports skill assessment system based on image analysis of the present invention,

[0032] The video keywords extracted by the Yake method include:

[0033] Preprocessing the video description to remove non-semantic characters, wherein the non-semantic characters include expression symbols and numbers;

[0034] Defining keyword features of the video description, wherein the keyword features include the position of the segmentation, the frequency of occurrence of the segmentation in the sentence, the frequency of occurrence of the segmentation in the video description, and the contextual relationship of the segmentation, wherein the contextual relationship of the segmentation is determined by the number of different candidate segments;

[0035] Obtaining the position score, sentence frequency score, text frequency score and candidate word score of each segmentation according to the keyword features, and obtaining the key weight of each segmentation according to the position score, sentence frequency score, text frequency score and candidate word score;

[0036] Arrange each segmentation in descending order according to the key weight, remove duplicate segmentations, extract the first n segmentations, define the segmentations as video keywords of the video description, and merge the n video keywords into the video keyword group of the sports teaching video, where n is a natural number greater than zero.

[0037] As a preferred embodiment of the automatic sports skill assessment system based on image analysis of the present invention,

[0038] Obtaining the coordinates of the moving part according to the three-dimensional coordinate system includes:

[0039] A motion part recognition model is established according to the image frame set, and the motion part recognition model is used to perform motion part area recognition on the image frames, and each motion part area is marked with a corresponding motion part label, and the motion part area is divided into n motion part sub-areas, and the center point of the motion part sub-area is used as the regional coordinate of the motion part sub-area, and the regional coordinates of each motion part sub-area are obtained, and the regional coordinates are merged to obtain the corresponding application part coordinates.

[0040] In a second aspect, the present invention provides a method for automatically evaluating motor skills based on image analysis, comprising the following steps:

[0041] Step S1, using sensors to collect motion data and motion videos;

[0042] Step S2, using a target detection algorithm and image segmentation to obtain a target motion video of the trainee target, cutting the target motion video into video frames according to a preset time interval to obtain a video frame set, obtaining the coordinates of the moving parts by establishing a three-dimensional coordinate system, and obtaining a trajectory graph structure of each moving part through the moving part coordinates;

[0043] Step S3: obtaining a trajectory atlas, establishing an action evaluation model based on the trajectory atlas structure using a graph convolutional neural network and a multi-task learning strategy, inputting the trajectory graph structure into the action evaluation model, obtaining action evaluation information, and obtaining the trainee's motion evaluation information based on the action evaluation information;

[0044] Step S4, using big data technology to obtain sports teaching videos and text descriptions from the network platform, obtaining video descriptions based on the sports teaching videos and text descriptions, and using keyword extraction technology to obtain video keyword groups for each sports teaching video based on the video descriptions;

[0045] Step S5, obtaining recommended videos based on the keyword tags and the video keyword group, and recommending corresponding sports teaching videos to the students.

[0046] The beneficial effects of the present invention are as follows: the present invention obtains the coordinates of the moving parts by establishing a three-dimensional coordinate system, and obtains the trajectory graph structure by using the coordinates of the moving parts, obtains a trajectory atlas, uses a graph convolutional neural network and a multi-task learning strategy to establish a motion evaluation model according to the trajectory atlas, and inputs the trajectory graph structure into the motion evaluation model to obtain motion evaluation information. The trajectory graph structure is used to evaluate each moving part of the trainee to be tested, and the motion evaluation scores and keyword labels of each moving part can be accurately and clearly obtained, which is conducive to the user directly and conveniently discovering the motion defects of each moving part of the trainee to be tested.

[0047] Big data technology is used to obtain sports teaching videos and text descriptions from the Internet platform, and speech recognition technology is used to obtain voice descriptions of sports teaching videos. The text descriptions and voice descriptions are merged to obtain video descriptions, and the Yake method is used to extract video keyword groups from sports teaching videos. The keyword tags and video keyword groups are matched to obtain video recommendation sequences. According to the video recommendation sequences, corresponding sports teaching videos are recommended to the trainees to be tested. Corresponding sports teaching videos are recommended according to the movement defects of the trainees to be tested, which is conducive to targeted guidance and improvement of the various sports movements of the trainees to be tested, and improving the overall sports skill level of the trainees to be tested. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1A schematic diagram of the basic process of an automatic sports skill assessment system based on image analysis provided by one embodiment of the present invention;

[0049] Figure 2 A schematic diagram of the basic flow of a method for automatic motor skill assessment based on image analysis provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0050] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings. It is obvious that the described embodiments are only part of the embodiments of the present invention, but not all of the embodiments.

[0051] Example 1, reference Figure 1 , as one embodiment of the present invention, provides an automatic sports skill assessment system based on image analysis, including a data acquisition module, a sports assessment module and a training guidance module;

[0052] The data acquisition module is used to obtain the motion data and motion video of the trainee to be tested, wherein the motion data includes the trainee's facial image and basic information of the trainee;

[0053] The motion assessment module is used to use a target detection algorithm and an image threshold segmentation technique to obtain a target motion video of the trainee to be tested, cut the target motion video into video frames according to a preset time interval, obtain a video frame set, obtain the coordinates of the motion parts by establishing a three-dimensional coordinate system, and obtain a trajectory graph structure of each motion part through the motion part coordinates to obtain a trajectory atlas, use a graph convolutional neural network and a multi-task learning strategy to establish a motion assessment model based on the trajectory atlas structure, input the trajectory graph structure into the motion assessment model, obtain motion assessment information, and obtain the trainee's motion assessment information through the motion assessment information;

[0054] The training guidance module is used to use big data technology to obtain sports teaching videos and text descriptions from the network platform, obtain video descriptions based on the sports teaching videos and text descriptions, and use keyword extraction technology to obtain video keyword groups for each sports teaching video based on the video descriptions, obtain recommended videos through keyword tags and the video keyword groups, and recommend corresponding sports teaching videos to students.

[0055] In this embodiment, the object detection algorithm is to find the objects of interest in the image and determine their categories and locations;

[0056] In this embodiment, the image threshold segmentation technology is a basic and important technology in image processing, which is used to separate the target object from the background in the image;

[0057] In this embodiment, the graph convolutional neural network is a convolutional neural network that can directly act on graph structure data. It aggregates the feature information of a node and its neighboring nodes and performs feature extraction and representation learning through convolution operations.

[0058] In this embodiment, the multi-task learning strategy is a powerful machine learning paradigm that improves the generalization ability and efficiency of the model by simultaneously learning multiple related tasks in the same model.

[0059] In this embodiment, the invention obtains the coordinates of the moving parts by establishing a three-dimensional coordinate system, and uses the coordinates of the moving parts to obtain a trajectory graph structure, obtains a trajectory atlas, uses a graph convolutional neural network and a multi-task learning strategy to establish a motion evaluation model according to the trajectory atlas, and inputs the trajectory graph structure into the motion evaluation model to obtain motion evaluation information. The trajectory graph structure is used to evaluate each moving part of the trainee to be tested, and the motion evaluation scores and keyword labels of each moving part can be accurately and clearly obtained, which is conducive to users directly and conveniently discovering the motion defects of each moving part of the trainee to be tested.

[0060] Big data technology is used to obtain sports teaching videos and text descriptions from the Internet platform, and speech recognition technology is used to obtain voice descriptions of sports teaching videos. The text descriptions and voice descriptions are merged to obtain video descriptions, and the Yake method is used to extract video keyword groups from sports teaching videos. The keyword tags and video keyword groups are matched to obtain video recommendation sequences. According to the video recommendation sequences, corresponding sports teaching videos are recommended to the trainees to be tested. Corresponding sports teaching videos are recommended according to the movement defects of the trainees to be tested, which is conducive to targeted guidance and improvement of the various sports movements of the trainees to be tested, and improving the overall sports skill level of the trainees to be tested.

[0061] The data acquisition module includes a data acquisition unit and a video acquisition unit;

[0062] The data acquisition unit is used to collect the facial image and basic information of the students to be tested, wherein the basic information includes the student ID, name, gender and age of the students;

[0063] The video acquisition unit is used to collect the motion video of the trainee to be tested using a visual sensor.

[0064] In this embodiment, a sensor is used to collect motion videos of the trainee to be tested, and detailed video data is provided for obtaining motion evaluation information and motion assessment information of the trainee to be tested using a motion recognition model based on the motion videos of the trainee to be tested.

[0065] The motion assessment module includes an action assessment unit and a motion assessment unit;

[0066] The action evaluation unit is used to use a target detection algorithm to capture the target of the student to be tested in the action video according to the student's facial image, and use image segmentation to obtain the target motion video of the student to be tested, cut the target motion video into video frames according to a preset time interval to obtain a video frame set sorted by time, establish a three-dimensional coordinate system according to the image frames of the video frame set, the horizontal plane is the x-axis, the vertical plane is the y-axis, and the vertical longitudinal direction is the z-axis, and obtain the coordinates of the moving parts according to the three-dimensional coordinate system, the moving part coordinates include the left arm joint coordinates, the left hand joint coordinates, the right arm joint coordinates, the right hand joint coordinates, the left leg joint coordinates, the left foot joint coordinates, the right leg joint coordinates, the waist joint coordinates, the left shoulder joint coordinates and the right shoulder joint coordinates, connect the moving part coordinates in time series to obtain the motion trajectory graph of each moving part, construct a trajectory graph structure according to the motion trajectory graph, a node of the trajectory graph structure represents a moving part node coordinate and includes the moving part node coordinate information, an edge of the trajectory graph structure represents the connection relationship between two moving part nodes and includes edge information, the edge information includes the node distance and the node angle, and input the trajectory graph structure into the action evaluation model to obtain action evaluation information;

[0067] The process of establishing the action evaluation model includes:

[0068] Obtain a motion video set, perform data annotation on the motion video set to obtain evaluation labels, the evaluation labels include score labels and keyword labels, obtain a trajectory atlas based on the motion video set, the trajectory atlas is a collection of trajectory graph structures containing the motion video set, input the trajectory graph structure into the first graph convolution layer of the graph convolutional neural network, extract node features and edge features of the trajectory graph structure, construct a first node feature matrix based on the node features, construct a first adjacency matrix based on the edge features, input the first node feature matrix and the first adjacency matrix into the second graph convolution layer, use the second graph convolution layer to normalize the first adjacency matrix to obtain a second adjacency matrix, and update the node feature matrix based on the second adjacency matrix, repeat the graph convolution operation l times to obtain a feature matrix, set corresponding evaluation output layers for different motion parts, and input the feature matrix into the evaluation output layer for training, with the feature matrix as input and the evaluation label as output, until the output fit reaches an expected threshold, then stop training, and obtain the motion evaluation model;

[0069] The motion evaluation unit is used to retrieve the motion evaluation score and keyword tag in the motion evaluation information, perform weighted calculation on the motion evaluation score of each motion part according to a certain weight to obtain the motion evaluation score, and merge the motion evaluation score and keyword tag to obtain the motion evaluation information.

[0070] In this embodiment, a three-dimensional coordinate system is established to obtain the coordinates of the moving parts, and the coordinates of the moving parts are used to obtain a trajectory graph structure, a trajectory atlas is obtained, a graph convolutional neural network and a multi-task learning strategy are used to establish a motion evaluation model based on the trajectory atlas, and the trajectory graph structure is input into the motion evaluation model to obtain motion evaluation information. The trajectory graph structure is used to evaluate each moving part of the student to be tested, and the motion evaluation scores and keyword labels of each moving part can be accurately and clearly obtained, which is conducive to users directly and conveniently discovering the motion defects of each moving part of the student to be tested.

[0071] The training guidance module includes a video classification unit and a video guidance unit;

[0072] The video classification unit is used to use big data technology to obtain sports teaching videos and text descriptions from the network platform, use speech recognition technology to obtain voice descriptions based on the sports teaching videos, merge the text descriptions and the voice descriptions to obtain video descriptions, and use the Yake method to extract video keywords from the video descriptions, and arrange the video keywords from large to small according to keyword weights to obtain video keyword groups for each video description;

[0073] The video guidance unit is used to match keyword tags and video keyword groups, and obtain the relevance between keyword tags and each sports teaching video based on the first mapping relationship, and sort the sports teaching videos in descending order according to the relevance to obtain a video recommendation sequence, and recommend the sports teaching videos to students based on the video recommendation sequence.

[0074] In this embodiment, big data technology is used to obtain sports teaching videos and text descriptions from the network platform, and speech recognition technology is used to obtain voice descriptions of the sports teaching videos. The text descriptions and voice descriptions are merged to obtain video descriptions, and the Yake method is used to extract video keyword groups of the sports teaching videos. The keyword tags and video keyword groups are matched to obtain video recommendation sequences, and corresponding sports teaching videos are recommended to the trainees to be tested based on the video recommendation sequences. Corresponding sports teaching videos are recommended based on the movement defects of the trainees to be tested, which is conducive to targeted guidance and improvement of each sports movement of the trainees to be tested, and improving the overall sports skill level of the trainees to be tested.

[0075] The operation function of the graph convolution operation is as follows:

[0076]

[0077] Among them, H (l+1) is the node feature matrix of layer l+1, H (l) is the node feature matrix of layer l, is the adjacency matrix of the self-loop, for The degree matrix, W (l) is the learnable weight matrix of the lth layer, and σ is the activation function.

[0078] In this embodiment, the convolution operation function is used to update the node feature matrix and obtain the feature matrix, which provides a specific and comprehensive feature basis for the evaluation output layer, which is conducive to ensuring the accuracy of the action evaluation model.

[0079] The keyword tags include action evaluation tags, which are keyword tags for evaluating actions of various joints.

[0080] In this embodiment, action evaluation labels are obtained based on keyword labels. The action evaluation labels obtained by evaluating each action of the trainee to be tested are conducive to directly and accurately understanding the action completion and action defects of the trainee to be tested, and provide effective data support for recommending sports teaching videos based on keyword labels.

[0081] The evaluation output layer includes a left arm movement evaluation output layer, a left hand movement evaluation output layer, a right arm movement evaluation output layer, a right hand movement evaluation output layer, a left leg movement evaluation output layer, a left foot movement evaluation output layer, a right leg movement evaluation output layer, a waist movement evaluation output layer, a left shoulder movement evaluation output layer and a right shoulder movement evaluation output layer.

[0082] In this embodiment, an evaluation output layer is constructed according to different movement parts, and each movement part of the trainee to be tested is evaluated separately to obtain the movement evaluation score and keyword label of each movement part. This is conducive to directly and accurately understanding the movement completion and movement defects of the trainee to be tested, and provides effective data support for recommending sports teaching videos based on keyword labels.

[0083] The video keywords extracted by the Yake method include:

[0084] Preprocessing the video description to remove non-semantic characters, wherein the non-semantic characters include expression symbols and numbers;

[0085] Defining keyword features of the video description, wherein the keyword features include the position of the segmentation, the frequency of occurrence of the segmentation in the sentence, the frequency of occurrence of the segmentation in the video description, and the contextual relationship of the segmentation, wherein the contextual relationship of the segmentation is determined by the number of different candidate segments;

[0086] Obtaining the position score, sentence frequency score, text frequency score and candidate word score of each segmentation according to the keyword features, and obtaining the key weight of each segmentation according to the position score, sentence frequency score, text frequency score and candidate word score;

[0087] Arrange each segmentation in descending order according to the key weight, remove duplicate segmentations, extract the first n segmentations, define the segmentations as video keywords of the video description, and merge the n video keywords into the video keyword group of the sports teaching video, where n is a natural number greater than zero.

[0088] In this embodiment, the key weights of each word are obtained by defining the keyword features of the video description, and each word is sorted according to the key weights. The first m words are extracted and defined as video keywords to form video keyword groups, providing a specific and accurate matching reference for matching video keyword groups according to keyword tags, and providing detailed data support for targeted video teaching guidance based on the movement defects of the students to be tested.

[0089] Obtaining the coordinates of the moving part according to the three-dimensional coordinate system includes:

[0090] A motion part recognition model is established according to an image frame set, and the motion part recognition model is used to perform motion part region recognition on the image frames, and each motion part region is marked with a corresponding motion part label, and the motion part region is divided into n motion part sub-regions, and the center point of the motion part sub-region is used as the regional coordinate of the motion part sub-region, and the regional coordinates of each motion part sub-region are obtained, and the regional coordinates are defined as motion part node coordinates, and the motion part node coordinates are merged to obtain the corresponding motion part coordinates.

[0091] In this embodiment, the process of establishing the motion part recognition model includes:

[0092] Obtain an image frame dataset, annotate the moving part area of ​​the image frame dataset, and input the annotated image frame dataset into the machine learning model for training, using the image frame as input and the moving part label as output. Stop training until the fitting degree reaches the expected threshold, and obtain the moving part recognition model.

[0093] In this embodiment, a motion part recognition model is used to identify motion parts based on an image frame set, obtain various motion part regions and motion part labels, and perform regional division on the motion part regions to obtain regional coordinates of the motion part sub-regions. The motion part coordinates are obtained based on the regional coordinates, providing a detailed coordinate information basis for obtaining the trajectory graph structure.

[0094] Example 2, reference Figure 2 , which is another embodiment of the present invention, differs from the first embodiment in that it provides a method for automatically evaluating sports skills based on image analysis, comprising the following steps:

[0095] Step S1, using sensors to collect motion data and motion videos;

[0096] Step S2, using a target detection algorithm and image segmentation to obtain a target motion video of the trainee target, cutting the target motion video into video frames according to a preset time interval to obtain a video frame set, obtaining the coordinates of the moving parts by establishing a three-dimensional coordinate system, and obtaining a trajectory graph structure of each moving part through the moving part coordinates;

[0097] Step S3: obtaining a trajectory atlas, establishing an action evaluation model based on the trajectory atlas structure using a graph convolutional neural network and a multi-task learning strategy, inputting the trajectory graph structure into the action evaluation model, obtaining action evaluation information, and obtaining the trainee's motion evaluation information based on the action evaluation information;

[0098] Step S4, using big data technology to obtain sports teaching videos and text descriptions from the network platform, obtaining video descriptions based on the sports teaching videos and text descriptions, and using keyword extraction technology to obtain video keyword groups for each sports teaching video based on the video descriptions;

[0099] Step S5, obtaining recommended videos based on the keyword tags and the video keyword group, and recommending corresponding sports teaching videos to the students.

[0100] In this embodiment, a three-dimensional coordinate system is established to obtain the coordinates of the moving parts, and the coordinates of the moving parts are used to obtain a trajectory graph structure, a trajectory atlas is obtained, a graph convolutional neural network and a multi-task learning strategy are used to establish a motion evaluation model based on the trajectory atlas, and the trajectory graph structure is input into the motion evaluation model to obtain motion evaluation information. The trajectory graph structure is used to evaluate each moving part of the student to be tested, and the motion evaluation scores and keyword labels of each moving part can be accurately and clearly obtained, which is conducive to users directly and conveniently discovering the motion defects of each moving part of the student to be tested.

[0101] Big data technology is used to obtain sports teaching videos and text descriptions from the Internet platform, and speech recognition technology is used to obtain voice descriptions of sports teaching videos. The text descriptions and voice descriptions are merged to obtain video descriptions, and the Yake method is used to extract video keyword groups from sports teaching videos. The keyword tags and video keyword groups are matched to obtain video recommendation sequences. According to the video recommendation sequences, corresponding sports teaching videos are recommended to the trainees to be tested. Corresponding sports teaching videos are recommended according to the movement defects of the trainees to be tested, which is conducive to targeted guidance and improvement of the various sports movements of the trainees to be tested, and improving the overall sports skill level of the trainees to be tested.

[0102] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a fully hardware embodiment, a fully software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code. The storage medium may be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0103] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. An automatic sports skill assessment system based on image analysis, characterized in that: It includes data acquisition module, motion assessment module and training guidance module; The data acquisition module is used to obtain the motion data and motion video of the trainee to be tested, wherein the motion data includes the trainee's facial image and basic information of the trainee; The motion assessment module is used to use a target detection algorithm and an image threshold segmentation technique to obtain a target motion video of the trainee to be tested, cut the target motion video into video frames according to a preset time interval, obtain a video frame set, obtain the coordinates of the motion parts by establishing a three-dimensional coordinate system, and obtain a trajectory graph structure of each motion part through the motion part coordinates to obtain a trajectory atlas, use a graph convolutional neural network and a multi-task learning strategy to establish a motion assessment model based on the trajectory atlas structure, input the trajectory graph structure into the motion assessment model, obtain motion assessment information, and obtain the trainee's motion assessment information through the motion assessment information; The training guidance module is used to use big data technology to obtain sports teaching videos and text descriptions from the network platform, obtain video descriptions based on the sports teaching videos and text descriptions, and use keyword extraction technology to obtain video keyword groups for each sports teaching video based on the video descriptions, obtain recommended videos through keyword tags and the video keyword groups, and recommend corresponding sports teaching videos to students.

2. The automatic sports skill assessment system based on image analysis according to claim 1, wherein: The data acquisition module includes a data acquisition unit and a video acquisition unit; The data acquisition unit is used to collect the facial image and basic information of the students to be tested, wherein the basic information includes the student ID, name, gender and age of the students; The video acquisition unit is used to collect the motion video of the trainee to be tested using a visual sensor.

3. The automatic sports skill assessment system based on image analysis according to claim 1, wherein: The motion assessment module includes an action assessment unit and a motion assessment unit; The action evaluation unit is used to use a target detection algorithm to capture the target of the student to be tested in the action video according to the student's facial image, and use image segmentation to obtain the target motion video of the student to be tested, cut the target motion video into video frames according to a preset time interval to obtain a video frame set sorted by time, establish a three-dimensional coordinate system according to the image frames of the video frame set, the horizontal plane is the x-axis, the vertical plane is the y-axis, and the vertical longitudinal direction is the z-axis, and obtain the coordinates of the moving parts according to the three-dimensional coordinate system, the moving part coordinates include the left arm joint coordinates, the left hand joint coordinates, the right arm joint coordinates, the right hand joint coordinates, the left leg joint coordinates, the left foot joint coordinates, the right leg joint coordinates, the waist joint coordinates, the left shoulder joint coordinates and the right shoulder joint coordinates, connect the moving part coordinates in time series to obtain the motion trajectory graph of each moving part, construct a trajectory graph structure according to the motion trajectory graph, a node of the trajectory graph structure represents a moving part node coordinate and includes the moving part node coordinate information, an edge of the trajectory graph structure represents the connection relationship between two moving part nodes and includes edge information, the edge information includes the node distance and the node angle, and input the trajectory graph structure into the action evaluation model to obtain action evaluation information; The process of establishing the action evaluation model includes: Obtain a motion video set, perform data annotation on the motion video set to obtain evaluation labels, the evaluation labels include score labels and keyword labels, obtain a trajectory atlas based on the motion video set, the trajectory atlas is a collection of trajectory graph structures containing the motion video set, input the trajectory graph structure into the first graph convolution layer of the graph convolutional neural network, extract node features and edge features of the trajectory graph structure, construct a first node feature matrix based on the node features, construct a first adjacency matrix based on the edge features, input the first node feature matrix and the first adjacency matrix into the second graph convolution layer, use the second graph convolution layer to normalize the first adjacency matrix to obtain a second adjacency matrix, and update the node feature matrix based on the second adjacency matrix, repeat the graph convolution operation l times to obtain a feature matrix, set corresponding evaluation output layers for different motion parts, and input the feature matrix into the evaluation output layer for training, with the feature matrix as input and the evaluation label as output, until the output fit reaches an expected threshold, then stop training, and obtain the motion evaluation model; The motion evaluation unit is used to retrieve the motion evaluation score and keyword tag in the motion evaluation information, perform weighted calculation on the motion evaluation score of each motion part according to a certain weight to obtain the motion evaluation score, and merge the motion evaluation score and keyword tag to obtain the motion evaluation information.

4. The automatic sports skill assessment system based on image analysis according to claim 1, wherein: The training guidance module includes a video classification unit and a video guidance unit; The video classification unit is used to use big data technology to obtain sports teaching videos and text descriptions from the network platform, use speech recognition technology to obtain voice descriptions based on the sports teaching videos, merge the text descriptions and the voice descriptions to obtain video descriptions, and use the Yake method to extract video keywords from the video descriptions, and arrange the video keywords from large to small according to keyword weights to obtain video keyword groups for each video description; The video guidance unit is used to match keyword tags and video keyword groups, and obtain the relevance between keyword tags and each sports teaching video based on the first mapping relationship, and sort the sports teaching videos in descending order according to the relevance to obtain a video recommendation sequence, and recommend the sports teaching videos to students based on the video recommendation sequence.

5. The automatic sports skill assessment system based on image analysis according to claim 3, wherein: The operation function of the graph convolution operation is as follows: Among them, H (l+1) is the node feature matrix of layer l+1, H (l) is the node feature matrix of layer l, is the adjacency matrix of the self-loop, for The degree matrix, W (l) is the learnable weight matrix of the lth layer, and σ is the activation function.

6. The automatic sports skill assessment system based on image analysis according to claim 3, wherein: The keyword tags include action evaluation tags, which are keyword tags for evaluating actions of various joints.

7. The automatic sports skill assessment system based on image analysis according to claim 3, wherein: The evaluation output layer includes a left arm movement evaluation output layer, a left hand movement evaluation output layer, a right arm movement evaluation output layer, a right hand movement evaluation output layer, a left leg movement evaluation output layer, a left foot movement evaluation output layer, a right leg movement evaluation output layer, a waist movement evaluation output layer, a left shoulder movement evaluation output layer and a right shoulder movement evaluation output layer.

8. The automatic sports skill assessment system based on image analysis according to claim 4, characterized in that: The video keywords extracted by the Yake method include: Preprocessing the video description to remove non-semantic characters, wherein the non-semantic characters include expression symbols and numbers; Defining keyword features of the video description, wherein the keyword features include the position of the segmentation, the frequency of occurrence of the segmentation in the sentence, the frequency of occurrence of the segmentation in the video description, and the contextual relationship of the segmentation, wherein the contextual relationship of the segmentation is determined by the number of different candidate segments; Obtaining the position score, sentence frequency score, text frequency score and candidate word score of each segmentation according to the keyword features, and obtaining the key weight of each segmentation according to the position score, sentence frequency score, text frequency score and candidate word score; Arrange each segmentation in descending order according to the key weight, remove duplicate segmentations, extract the first n segmentations, define the segmentations as video keywords of the video description, and merge the n video keywords into the video keyword group of the sports teaching video, where n is a natural number greater than zero.

9. The automatic sports skill assessment system based on image analysis according to claim 3, wherein: Obtaining the coordinates of the moving part according to the three-dimensional coordinate system includes: A motion part recognition model is established according to the image frame set, and the motion part recognition model is used to perform motion part area recognition on the image frames, and each motion part area is marked with a corresponding motion part label, and the motion part area is divided into n motion part sub-areas, and the center point of the motion part sub-area is used as the regional coordinate of the motion part sub-area, and the regional coordinates of each motion part sub-area are obtained, and the regional coordinates are merged to obtain the corresponding application part coordinates.

10. A method for automatic assessment of sports skills based on image analysis, characterized in that: The following steps are involved: Step S1, using sensors to collect motion data and motion videos; Step S2, using a target detection algorithm and image segmentation to obtain a target motion video of the trainee target, cutting the target motion video into video frames according to a preset time interval to obtain a video frame set, obtaining the coordinates of the moving parts by establishing a three-dimensional coordinate system, and obtaining a trajectory graph structure of each moving part through the moving part coordinates; Step S3: obtaining a trajectory atlas, establishing an action evaluation model based on the trajectory atlas structure using a graph convolutional neural network and a multi-task learning strategy, inputting the trajectory graph structure into the action evaluation model, obtaining action evaluation information, and obtaining the trainee's motion evaluation information based on the action evaluation information; Step S4, using big data technology to obtain sports teaching videos and text descriptions from the network platform, obtaining video descriptions based on the sports teaching videos and text descriptions, and using keyword extraction technology to obtain video keyword groups for each sports teaching video based on the video descriptions; Step S5: obtaining recommended videos based on the keyword tags and the video keyword group, and recommending corresponding sports teaching videos to the students.

Images