Skiing level evaluation method and system based on multi-source information fusion

Abstract

The invention belongs to the technical field of behavior recognition, and provides a skiing level evaluation method and system based on multi-source information fusion, and the method comprises the steps: synchronously collecting multi-source time sequence data and biological information when a skier completes a preset key action, and carrying out the standardization processing; constructing a data set, wherein the data set comprises a time sequence data set composed of multi-source time sequence data and a biological data set composed of biological information; constructing a multi-task neural network model, and synchronously completing a behavior recognition task and an identity recognition task; training the multi-task neural network model by using a training set containing excellent athlete sample information; utilizing the trained multi-task neural network model to extract high-level fusion features of a to-be-evaluated skier; and calculating the similarity between the high-level fusion features of the to-be-evaluated skier and a preset excellent skier, and evaluating the skiing level according to the similarity. According to the method, the problem that an evaluation system in the prior art cannot fully consider individual differences among skiers is solved.

Description

Technical Field

[0001] This invention belongs to the field of behavior recognition technology, and in particular relates to a skiing level evaluation method and system based on multi-source information fusion. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] In recent years, skiing has become increasingly popular, with a significant increase in the number of participants. Beginners make up the vast majority of these participants. While they are enthusiastic about skiing, a lack of experience and professional guidance often hinders their ability to perform well on the slopes, and they may even suffer injuries due to improper technique. Against this backdrop, ski instructors play a crucial role; however, current instructor resources are significantly insufficient in both quantity and capacity, making it difficult to meet the individualized needs of beginners.

[0004] With the continuous development and widespread application of computer technology, building such virtual coaching systems has become possible. Virtual coaching systems can be used to assess skiers' skill levels, identify shortcomings in their movements, and provide improvement suggestions and training plans, thereby helping beginners quickly improve their skills and reduce the risk of sports injuries.

[0005] Although much research has been conducted on skiing level assessment or movement analysis, the evaluation systems often have certain limitations: some evaluation methods are relatively crude and can only give a broad level rating; others focus only on comparison with standard movements and fail to fully consider individual differences among skiers, which affects the accuracy and comprehensiveness of the evaluation results. Summary of the Invention

[0006] The purpose of this invention is to provide a skiing level evaluation method and system based on multi-source information fusion, which simultaneously completes behavior recognition and identity recognition tasks, and extracts high-level fusion features based on a multi-task neural network model, so as to solve the problem that the evaluation system in the prior art fails to fully consider the individual differences among skiers.

[0007] To achieve the above objectives, the present invention adopts the following technical solution: The first aspect of this invention provides a skiing level evaluation method based on multi-source information fusion, comprising: While the skier completes the preset key movements, multi-source time-series data and biological information are collected simultaneously; the multi-source time-series data and biological information are standardized respectively; the multi-source time-series data are segmented into samples and associated with corresponding biological information; a dataset is constructed, which includes a time-series dataset composed of multi-source time-series data and a biological dataset composed of biological information; the dataset is divided into a training set and a test set according to the proportion. A multi-task neural network model is constructed to simultaneously complete behavior recognition and identity recognition tasks. The multi-task neural network model is trained using a training set containing sample information of outstanding athletes, so that the high-level fusion features learned by the multi-task neural network model can reflect the differences in skier skill levels. The high-level fusion features of the skier to be evaluated are extracted using the trained multi-task neural network model. The similarity between the high-level fusion features of the skier to be evaluated and the preset outstanding skiers is calculated, and the skiing level is evaluated based on the similarity.

[0008] As a further technical solution, the key actions include turning, gliding and jumping, and the corresponding key action types serve as supervision information for model training; the multi-source time-series data is acquired by inertial measurement unit, magnetometer and GPS, including acceleration, angular velocity, magnetic field, latitude and longitude, altitude and speed information; the biological information includes height, weight, age and leg-to-body ratio.

[0009] As a further technical solution, a fixed-size sliding window is used to segment the standardized multi-source time series data into samples, with a window size of 100 and a step size of 50.

[0010] As a further technical solution, the segmented samples are labeled. Specifically: the behavior recognition task is used to identify key behaviors during skiing and is labeled using key behavior categories; the identity recognition task is used to reflect the relationship between the level of skiing behavior completion and individual differences and is labeled using personnel numbers.

[0011] As a further technical solution, the multi-task neural network model includes a multi-source temporal data processing branch and a bioinformatics processing branch; the multi-source temporal data processing branch includes a temporal feature extraction layer, an attention layer, a bidirectional recurrent neural network layer, and a fully connected layer connected in sequence, wherein the fully connected layer includes a first fully connected layer and a second fully connected layer; the bioinformatics processing branch is an independent fully connected network, including a third fully connected layer and a fourth fully connected layer, used for high-level semantic extraction of bioinformatics.

[0012] As a further technical solution, the output of the first fully connected layer is fused with the high-level semantic features output by the third fully connected layer, and the fused features are then processed by the fourth fully connected layer to finally output high-level fused features as input for the identity recognition task; the temporal features output by the second fully connected layer are used as input for the behavior recognition task.

[0013] As a further technical solution, during the training process of the multi-task neural network model, the loss function of different tasks is calculated and feedback learning is performed, so that the high-level fusion features extracted by the multi-task neural network model satisfy the requirement that the feature similarity between skiers of the same level is higher than the feature similarity between skiers of different levels.

[0014] A second aspect of the present invention provides a skiing level evaluation system based on multi-source information fusion, comprising: The dataset construction module is configured to: synchronously collect multi-source time-series data and biological information when skiers complete preset key actions; standardize the multi-source time-series data and biological information respectively; segment the multi-source time-series data into samples and associate them with corresponding biological information; construct a dataset containing a time-series dataset composed of multi-source time-series data and a biological dataset composed of biological information; and divide the dataset into a training set and a test set according to a certain ratio. The skiing level evaluation module is configured to: construct a multi-task neural network model to simultaneously complete behavior recognition and identity recognition tasks; train the multi-task neural network model using a training set containing sample information of outstanding athletes, so that the high-level fusion features learned by the multi-task neural network model can reflect the differences in skier level; extract the high-level fusion features of the skier to be evaluated using the trained multi-task neural network model; calculate the similarity between the high-level fusion features of the skier to be evaluated and the preset outstanding skiers, and evaluate the skiing level based on the similarity.

[0015] A third aspect of the present invention provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps of a ski level evaluation method based on multi-source information fusion as described in the first aspect of the present invention.

[0016] A fourth aspect of the present invention provides an electronic device including a memory, a processor, and a program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of a skiing level evaluation method based on multi-source information fusion as described in the first aspect of the present invention.

[0017] The technical solution of the present invention has the following beneficial effects: This invention provides a method for evaluating skiing skills that takes into account individual differences based on multi-task learning. It constructs a feature library with high-level semantics through multi-task learning. This feature library contains features of elite athletes and ordinary beginner skiers. Unlike direct comparison with templates, this invention uses high-level fusion features for comparison, thereby improving the accuracy of the evaluation.

[0018] This invention provides a virtual coaching system suitable for a wide range of skiing beginners. It can be easily ported to mobile devices, which is conducive to its promotion to the public and solves the problem of limited manpower and energy.

[0019] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0020] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0021] Figure 1 This is a schematic diagram of the method flow of the first embodiment.

[0022] Figure 2 This is a schematic diagram of the ski level evaluation process in the first embodiment. Detailed Implementation

[0023] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0024] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0025] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0026] To meet the personalized needs of beginner skiers, there is an urgent need for a virtual coaching system that can intelligently assist them and effectively improve their skill level. However, virtual coaching systems need to assess skiing skill levels or analyze movements. Current evaluation systems often have limitations, such as only providing general skill levels or focusing solely on comparisons with standard movements, failing to fully consider individual differences among skiers. Therefore, this application proposes a skiing skill evaluation method and system based on multi-source information fusion for use in a virtual coaching system. A coach database is constructed using fixed movement and posture information from elite skiers. The system can compare user posture information with the coach database and provide guidance based on user behavior, thus realizing the virtual coaching function.

[0027] Example 1 like Figure 1 , Figure 2 As shown, this embodiment discloses a skiing level evaluation method based on multi-source information fusion; including: S1: Building the dataset S1.1: Simultaneously collect multi-source time-series data and biological information when the skier completes the preset key actions.

[0028] In this embodiment, the skiers being collected need to complete designated actions as required by the staff, such as turning, gliding or jumping, which are common key actions in skiing. The corresponding action types will be used as supervision information for subsequent model training.

[0029] Multi-source time-series data is acquired by an inertial measurement unit, magnetometer, and GPS. The inertial sensors include an accelerometer and a gyroscope. The multi-source time-series data includes acceleration, angular velocity, magnetic field, latitude and longitude, altitude, and velocity information. Modern smartphones are equipped with these devices, which meets the needs of the system for the general public, while avoiding the problems of complex deployment, environmental sensitivity, and high cost associated with solutions that rely on visual information.

[0030] In addition, biometrics were incorporated to account for individual differences, thereby making the evaluation results more accurate. Biometrics include height, weight, age, and leg-to-body ratio.

[0031] S1.2: Standardize the multi-source time-series data and bioinformatics respectively.

[0032] First, attitude determination is performed using Kalman filtering based on acceleration, angular velocity, and magnetic field data to obtain attitude features. Biological information is then standardized to map its representation to the same order of magnitude. Similarly, multi-source time-series data such as acceleration and angular velocity are also standardized.

[0033] In this embodiment, the Z-Score normalization method is used, as shown in the following formula:

[0034] in, The mean of the data. The standard deviation is denoted as .

[0035] In this embodiment, the standardized multi-source time-series data and biological information are respectively concatenated for feature extraction to obtain sample information. Among these, data such as acceleration and angular velocity acquired by sensors and their derived information are time-series features, while biological information is a non-time-series feature. Finally, multi-source time-series data with concatenated features can be obtained. and bioinformatics ,in and It is the feature dimension.

[0036] S1.3: Segment multi-source time-series data into samples and associate them with corresponding biological information.

[0037] In this embodiment, a sliding window of fixed size 100 is used to segment the standardized multi-source time-series data into samples with a step size of 50, constructing a sample set, and obtaining each sample as... .

[0038] The segmented samples are labeled, divided into two learning tasks: behavior recognition and identity recognition. Specifically, the behavior recognition task identifies key behaviors during skiing, which are crucial indicators of skiing skill level, and is labeled using key behavior categories. The identity recognition task reflects the relationship between skiing performance levels and individual differences, thereby making skiing skill assessments more accurate, and is labeled using personnel ID numbers.

[0039] S1.4: Construct the dataset by dividing it into training and test sets according to a set ratio.

[0040] Through the above steps, the dataset is constructed, comprising a time-series dataset composed of multi-source time-series data and a biological dataset composed of bioinformatics. The resulting time-series dataset is randomly shuffled, ensuring that the biological dataset maintains its correspondence with it. Further, the dataset is divided in a 4:1 ratio to obtain training and testing sets. The training set is then further divided in a 9:1 ratio to obtain training and validation sets. The time-series data input to the model has a dimension of [missing information]. Where B is the batch size and L is the time duration; the bioinformatics dimension is... .

[0041] S2: Constructing a multi-task neural network model S2.1: Construct a multi-task neural network model to simultaneously complete behavior recognition and identity recognition tasks.

[0042] In this embodiment, the multi-task neural network model includes a multi-source temporal data processing branch and a bioinformatics processing branch. The multi-source temporal data processing branch comprises a temporal feature extraction layer, an attention layer, a bidirectional recurrent neural network layer, and a fully connected layer connected in sequence. The fully connected layer includes a first fully connected layer and a second fully connected layer. The bioinformatics processing branch is an independent fully connected network, including a third fully connected layer and a fourth fully connected layer, used for high-level semantic extraction of bioinformatics.

[0043] Multi-source time-series data is input into the multi-source time-series data processing branch to perform the behavior recognition task; biological information is input into the biological information processing branch and fused with the time-series features of the intermediate layer to perform the identity recognition task.

[0044] Specifically, the task process includes: The dimension of the input is The time series data was transformed by dimensionality to obtain The output channel is set to 1 through a one-dimensional convolutional network layer with a kernel size of 1. The output is The temporal characteristics are then transformed again to obtain a dimension of... The temporal characteristics are then analyzed. The output is fed into the attention mechanism layer to learn different weights for different time points; a higher weight indicates a greater contribution of that information to the task. The output dimension remains [missing information]. The above output is then fed into a Bi-LSTM network layer, consisting of two layers, ultimately yielding a dimension of... The temporal characteristics are then analyzed. The output is passed through two fully connected layers. The features output from the first fully connected layer are fused with biometric information and used as input for the identity recognition task. The output from the second fully connected layer is used for the behavior recognition task.

[0045] The attention mechanism framework of the attention mechanism layer is (Q, K, V), where Q is the query, K is the key, and V is the value. Q and K are multiplied to obtain the similarity score between each query vector and all key vectors. This similarity score is the weight of each query vector. After normalization, it is multiplied by V to obtain the weighted feature representation at different time steps.

[0046] Furthermore, biometric information can also incorporate image information, such as uploading facial images. After modeling the facial image data and obtaining high-level feature information, it can be fused with temporal features and used as input for identity recognition tasks.

[0047] This embodiment uses a fully connected network to perform high-level semantic extraction of biometric information, including a third fully connected layer and a fourth fully connected layer. Specifically, the output of the first fully connected layer is fused with the high-level semantic features output by the third fully connected layer. The fused features are then processed by the fourth fully connected layer, and the final output high-level fused features are used as input for the identity recognition task; the temporal features output by the second fully connected layer are used as input for the behavior recognition task.

[0048] S2.2: Train the multi-task neural network model using a training set containing sample information of outstanding athletes, so that the high-level fusion features learned by the multi-task neural network model can reflect the differences in skier skill levels.

[0049] There should be a significant difference in performance between elite athletes and ordinary beginner skiers. The training set includes information on elite athletes, who will serve as the benchmark against which beginner skiers will be compared, thus determining a skier's skiing level or a specific skiing behavior. Unlike general evaluation methods, this embodiment uses high-level feature representations (high-level fusion features) in the model for evaluation. These high-level feature representations (high-level fusion features) not only better learn behavioral characteristics but also incorporate biometric information, making the evaluation results more accurate. By comparing the high-level feature representations (high-level fusion features) of beginner skiers and elite athletes, scores are calculated to determine the beginner skier's skiing level, thereby providing feedback.

[0050] S2.3: Calculate the loss function for different tasks and perform feedback learning on the network. Through multiple iterations, the model reaches its optimal state, ensuring that the high-level fusion features extracted by the multi-task neural network model satisfy the requirement that the feature similarity between skiers of the same skill level is higher than the feature similarity between skiers of different skill levels.

[0051] In this embodiment, the cross-entropy loss function is used, and its formula is as follows:

[0052] Where C represents the total number of categories. For real labels, This represents the probability of predicting the class as c.

[0053] S2.4: Input the data of the skier to be evaluated into the trained multi-task neural network model, and use the trained multi-task neural network model to extract the high-level fusion features of the skier to be evaluated.

[0054] S2.5: Calculate the similarity between the high-level integration features of the skier to be evaluated and the preset excellent athletes, and evaluate the skiing level based on the similarity.

[0055] In this embodiment, the similarity is calculated as follows:

[0056] Here, A and B represent the characteristics of different individuals.

[0057] The similarity value ranges from [-1, 1], and is mapped to [0, 1] using a linear mapping. The specific operation is as follows:

[0058] When obtaining a similarity value, a higher similarity to an excellent athlete indicates a higher level of skiing ability for the skier being evaluated.

[0059] When obtaining high-level integration characteristics, the similarity of characteristics between elite athletes should be higher, the similarity between elite athletes and ordinary beginner skiers should be lower, and the similarity between ordinary beginner skiers should be higher.

[0060] In this embodiment, a coach database is constructed using fixed movement posture information of outstanding athletes. The system can compare the key movement information of skiers with the coach database and provide guidance on behavioral performance, thereby realizing the virtual coach function.

[0061] Example 2 This embodiment discloses a skiing level evaluation system based on multi-source information fusion, including: The dataset construction module is configured to: synchronously collect multi-source time-series data and biological information when skiers complete preset key actions; standardize the multi-source time-series data and biological information respectively; segment the multi-source time-series data into samples and associate them with corresponding biological information; construct a dataset containing a time-series dataset composed of multi-source time-series data and a biological dataset composed of biological information; and divide the dataset into a training set and a test set according to a certain ratio. The skiing level evaluation module is configured to: construct a multi-task neural network model to simultaneously complete behavior recognition and identity recognition tasks; train the multi-task neural network model using a training set containing sample information of outstanding athletes, so that the high-level fusion features learned by the multi-task neural network model can reflect the differences in skier level; extract the high-level fusion features of the skier to be evaluated using the trained multi-task neural network model; calculate the similarity between the high-level fusion features of the skier to be evaluated and the preset outstanding skiers, and evaluate the skiing level based on the similarity.

[0062] Example 3 The purpose of this embodiment is to provide a computer-readable storage medium. A computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of a ski level evaluation method based on multi-source information fusion as described in Embodiment 1 of this disclosure.

[0063] Example 4 The purpose of this embodiment is to provide an electronic device. An electronic device includes a memory, a processor, and a program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps in a ski level evaluation method based on multi-source information fusion as described in Embodiment 1 of this disclosure.

[0064] The steps and methods involved in the apparatuses of Embodiments 2, 3, and 4 above correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.

[0065] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computer device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. The present invention is not limited to any particular combination of hardware and software.

[0066] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A skiing level evaluation method based on multi-source information fusion, characterized in that, include: Multi-source time-series data and biological information are collected simultaneously as skiers complete preset key movements; The multi-source time-series data and biological information are respectively standardized. Multi-source time-series data are segmented into samples and associated with corresponding biological information; a dataset is constructed, which includes a time-series dataset composed of multi-source time-series data and a biological dataset composed of biological information; the dataset is divided into training set and test set according to the proportions. A multi-task neural network model is constructed to simultaneously complete behavior recognition and identity recognition tasks; the multi-task neural network model is trained using a training set containing sample information of outstanding athletes, so that the high-level fusion features learned by the multi-task neural network model can reflect the differences in skier skill levels; the high-level fusion features of the skier to be evaluated are extracted using the trained multi-task neural network model. The similarity between the high-level integration features of the skier to be evaluated and the preset excellent skiers is calculated, and the skiing level is evaluated based on the similarity.

2. The skiing level evaluation method based on multi-source information fusion as described in claim 1, characterized in that, The key actions include turning, gliding, and jumping, and the corresponding key action types serve as supervisory information for model training; the multi-source time-series data is acquired by inertial measurement units, magnetometers, and GPS, including acceleration, angular velocity, magnetic field, latitude and longitude, altitude, and velocity information; the bio-information includes height, weight, age, and leg-to-body ratio.

3. The skiing level evaluation method based on multi-source information fusion as described in claim 1, characterized in that, The standardized multi-source time series data is segmented into samples using a fixed-size sliding window with a window size of 100 and a step size of 50.

4. The skiing level evaluation method based on multi-source information fusion as described in claim 3, characterized in that, The segmented samples are labeled. Specifically, the behavior recognition task is used to identify key behaviors during skiing and is labeled using key behavior categories; the identity recognition task is used to reflect the relationship between the level of skiing performance and individual differences and is labeled using personnel numbers.

5. The skiing level evaluation method based on multi-source information fusion as described in claim 1, characterized in that, The multi-task neural network model includes a multi-source temporal data processing branch and a bioinformatics processing branch. The multi-source temporal data processing branch includes a temporal feature extraction layer, an attention layer, a bidirectional recurrent neural network layer, and a fully connected layer connected in sequence. The fully connected layer includes a first fully connected layer and a second fully connected layer. The bioinformatics processing branch is an independent fully connected network, including a third fully connected layer and a fourth fully connected layer, used for high-level semantic extraction of bioinformatics.

6. The skiing level evaluation method based on multi-source information fusion as described in claim 5, characterized in that, The output of the first fully connected layer is fused with the high-level semantic features output by the third fully connected layer. The fused features are then processed by the fourth fully connected layer, and finally the high-level fused features are output as the input for the identity recognition task. The temporal features output by the second fully connected layer are used as input for the behavior recognition task.

7. The skiing level evaluation method based on multi-source information fusion as described in claim 1, characterized in that, During the training process of the multi-task neural network model, the loss function of different tasks is calculated and feedback learning is performed so that the high-level fusion features extracted by the multi-task neural network model satisfy the requirement that the feature similarity between skiers of the same level is higher than the feature similarity between skiers of different levels.

8. A skiing level evaluation system based on multi-source information fusion, characterized in that, include: The dataset construction module is configured to simultaneously collect multi-source time-series data and biological information when skiers complete preset key actions; The multi-source time-series data and biological information are standardized respectively; the multi-source time-series data are segmented into samples and associated with corresponding biological information; a dataset is constructed, which includes a time-series dataset composed of multi-source time-series data and a biological dataset composed of biological information; the dataset is divided into a training set and a test set according to the proportion. The skiing level evaluation module is configured to: construct a multi-task neural network model to simultaneously complete behavior recognition and identity recognition tasks; train the multi-task neural network model using a training set containing sample information of outstanding athletes, so that the high-level fusion features learned by the multi-task neural network model can reflect the differences in skiers' levels; and extract the high-level fusion features of the skiers to be evaluated using the trained multi-task neural network model. The similarity between the high-level integration features of the skier to be evaluated and the preset excellent skiers is calculated, and the skiing level is evaluated based on the similarity.

9. A computer-readable storage medium having a program stored thereon, characterized in that, When executed by the processor, the program implements the steps in the ski level evaluation method based on multi-source information fusion as described in any one of claims 1-7.

10. An electronic device comprising a memory, a processor, and a program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the ski level evaluation method based on multi-source information fusion as described in any one of claims 1-7.

Images