Snow sports information monitoring method and related device

Abstract

The present application provides a method and related device for monitoring snow sports information, which are used to monitor the information of snow sports. The method includes: using an image acquisition device to collect in real time images of a target person during movement in a preset area, which are recorded as real-time acquisition images; using one or more sensors to obtain in real time the motion sensing information of the target person; respectively performing feature extraction on the real-time acquisition images and the motion sensing information to obtain corresponding feature extraction results; performing data fusion on the feature extraction results, the environmental information, the snowboard structure information of the target person, and the body index measurement information to obtain a motion data fusion result of the target person; and inputting the motion data fusion result of the target person into a motion evaluation prediction model to obtain a motion evaluation prediction result of the target person. The present application can monitor in real time the multi-source information of a person during snow sports so as to realize the evaluation of the sports effect.

Description

Technical Field

[0001] This application relates to the technical fields of sports information monitoring and artificial intelligence, and particularly to a method for monitoring snow sports information and related devices. Background Art

[0002] Sports such as ski jumping, alpine skiing, and freestyle skiing are all common snow sports. To further improve the training efficiency of snow sports athletes, it is necessary to obtain information such as the real-time position, speed, acceleration, and skiing posture of snow sports athletes. On this basis, comprehensively and pertinently analyzing the movement process of athletes is very necessary for improving the performance of athletes.

[0003] Currently, generally, manual judgment of athletes' technical movements is carried out with the help of experience, which is time-consuming, laborious, and subjective. Moreover, traditional non-intelligent methods of collecting data have problems such as low collection efficiency and lack of real-time nature. There are problems of "difficult to quantify, difficult to record, difficult to supervise, and difficult to analyze" in the snow training of athletes. Therefore, to truly tap the potential of athletes and give full play to their competitive abilities, it is necessary to rely on advanced science and technology to track and monitor the whole process of athletes' snow sports, and monitor the movement trajectories and movement parameters of athletes.

[0004] Based on this, this application provides a method for monitoring snow sports information and related devices to solve the problems existing in the above-mentioned prior art. Summary of the Invention

[0005] The purpose of this application is to provide a method for monitoring snow sports information and related devices to monitor diversified information of a person during snow sports in real time so as to evaluate the exercise effect.

[0006] The purpose of this application is achieved by adopting the following technical solutions:

[0007] In a first aspect, this application provides a method for monitoring snow sports information for monitoring information of snow sports. The method includes:

[0008] Using an image acquisition device to collect images of a target person during movement in a preset area in real time, which is recorded as a real-time acquisition image. The preset area is the area where the target person makes technical movements of snow sports;

[0009] Using one or more sensors to obtain the motion sensing information of the target person in real time. The motion sensing information includes one or more of the following information: multi-dimensional force information between the shoe and the ski board, plantar flexible force information, ski board acceleration information, and the center of gravity movement trajectory information of the target person, joint acceleration information, joint angle information, and joint position information;

[0010] Feature extraction is respectively performed on the real-time acquired image and the motion sensing information to obtain the feature extraction results corresponding to the real-time acquired image and the motion sensing information; wherein, the feature extraction result corresponding to the real-time acquired image is used to indicate the human body posture of the target person, and the feature extraction result corresponding to the motion sensing information includes the kinematic and dynamic characteristics of the target person;

[0011] Data fusion is performed on the feature extraction results corresponding to the real-time acquired image and the motion sensing information, as well as the environmental information, the snowboard structure information of the target person, and the body index measurement information, to obtain the motion data fusion result of the target person;

[0012] The motion data fusion result of the target person is input into the motion evaluation prediction model to obtain the motion evaluation prediction result of the target person.

[0013] The beneficial effects of this technical solution are as follows: This application can achieve long-term, dynamic, and remote multi-point monitoring within the snow sports area, and can obtain real-time remote sensing information of the regional scene and target personnel during snow sports training or competitions, so as to obtain comprehensive motion sensing information in real time online (that is, the sensing information includes kinematic information and / or kinetic information, and correspondingly, the motion parameters include kinematic parameters and / or kinetic parameters). First, compared with the traditional manual observation method, this application has no time limit and can monitor the motion process of the target personnel at any time, without being restricted by time, liberating the monitoring personnel from repetitive work. Second, this application specifically quantifies the motion sensing information, so that relevant personnel such as athletes and coaches can obtain the motion information of the target personnel more objectively and scientifically, eliminating the subjectivity of traditional manual observation. Third, it can collect the entire dynamic motion process in real time, accurately record the real-time collected images and the motion sensing information of the target personnel, and can remotely view the motion data of the target personnel without place restrictions. On the other hand, through new technologies such as artificial intelligence, feature extraction is performed on the real-time collected images and the motion sensing information to extract the beneficial information in each signal / data to the greatest extent, so as to improve the utilization rate of the images and information. And the feature extraction results corresponding to the real-time collected images and the motion sensing information are fused with the environmental information, the snowboard structure information of the target personnel, and the body index measurement information to obtain the motion data fusion result of the target personnel. Then, the obtained motion data fusion result is input into the motion evaluation prediction model to obtain the motion evaluation prediction result for each target personnel, making the result more targeted and scientific, empowering precise teaching of snow sports, and improving the intelligent level of snow sports training management. The advantage of doing this is to use artificial intelligence technology (such as the motion evaluation prediction model, etc.) to intelligently obtain the motion evaluation prediction result, so that the coach can specifically propose a training plan for the target personnel, thus providing a snow sports information monitoring function featuring multi-source information collection and real-time data acquisition. This application fills the gap in the domestic real-time information monitoring technology for multi-source information of athletes and can provide accurate suggestions for sports coaches to a great extent.

[0014] In view of the characteristics of snow sports, this application integrates multi-dimensional force information between shoes and skis, plantar flexible force information, ski acceleration information, as well as the center of gravity movement trajectory information of the target person, joint acceleration information, joint angle information, joint position information, etc., to build a comprehensive real-time online acquisition platform for kinematic information and dynamic information, and realizes the real-time synchronous acquisition of parameters such as ski acceleration, three-dimensional force, plantar pressure, and human body posture and position. This application designs and constructs a data format and dataset architecture that are convenient for storage, management, and invocation, and constructs a dataset of sports science information for snow sports. Secondly, this application combines a deep neural network with an information fusion model through an effective multi-source heterogeneous data fusion method to generate multiple inferences, and can construct a characteristic parameter index system for snow sports related to the exercise effect; finally, this application can evaluate the exercise effect by studying the non-linear relationship between the characteristic parameters of snow sports and the exercise effect.

[0015] In some optional embodiments, the motion sensing information includes multi-dimensional force information between shoes and skis, plantar flexible force information, ski acceleration information, as well as the center of gravity movement trajectory information of the target person, joint acceleration information, joint angle information, and joint position information;

[0016] The process of obtaining the feature extraction result corresponding to the motion sensing information includes:

[0017] Based on the motion sensing information of the target person, obtain the data to be processed of multiple motion parameters of the target person; the multiple motion parameters include the three-dimensional force of the target person, ski acceleration, movement centroid position, velocities of each joint, angular velocities of each joint, accelerations of each joint, angular accelerations of each joint, and angles of each joint;

[0018] Normalize the data to be processed of the multiple motion parameters of the target person to obtain the normalized result of the motion parameters of the target person;

[0019] Based on the normalized result of the motion parameters of the target person, use the gradient of different colors to represent different numerical sizes, and convert and generate the stacked inertial signal image of the target person;

[0020] Extract features from the stacked inertial signal image of the target person to obtain the feature extraction result corresponding to the motion sensing information.

[0021] The beneficial effects of this technical solution are as follows: This application needs to collect multi-dimensional force information between the athlete's shoes and the snowboard, the flexible force information of the athlete's sole, the snowboard acceleration information, the target person's center of gravity movement trajectory information, joint acceleration information, joint angle information, and joint position information, and analyze to obtain the three-dimensional force, snowboard acceleration, movement center of mass position, speeds of each joint, angular speeds of each joint, accelerations of each joint, angular accelerations of each joint, and angles of each joint of the target person. The 3D data contained in these motion inertia type data (i.e., three-dimensional force, snowboard acceleration, movement center of mass position, speeds of each joint, angular speeds of each joint, accelerations of each joint, angular accelerations of each joint, and angles of each joint) respectively correspond to the three directions of X, Y, and Z. Normalize the above data so that the data value range is mapped to between [0, 1]. This normalization process can make the originally incomparable data comparable, while retaining the size relationship existing in the original data and eliminating the influence of dimension and data value range. The purpose of this normalization process also includes normalizing the indicators to facilitate subsequent data processing. For example, after normalization, it is very convenient to perform subsequent data stacking to obtain a stacked inertia signal image. In addition, this application uses data visualization technology to more accurately and clearly express the above information, providing a basis for coaches and athletes to further conduct reasonable and comprehensive motion analysis on relevant information.

[0022] In some optional embodiments, the motion evaluation prediction result of the target person is used to indicate the motion score and / or rating of the target person.

[0023] The beneficial effects of this technical solution are as follows: Select a scoring standard and / or rating standard that is the same as or similar to that of the official competition, specifically quantify the scoring standard and / or rating standard, and more clearly and explicitly express the influencing factors that affect the target person's competition score. For example, whether factors such as snowboard parameter information, environmental information, the height and weight of the target person, and technical movements have an impact on the competition score, and objectively select the most suitable snowboard information parameters for the athlete. Through scoring and / or rating, it is more directly expressed whether the target person has made progress in this technical movement and whether the training is effective, further helping the target person correct their posture and even correct wrong force application habits, force application time, movement amplitude, movement frequency, etc., so as to help the target person save effort and accurately make standard movements with standardized postures, improve training efficiency and effect, improve competition results, and further point out the improvement direction of the target person. It is beneficial for the target person to be oriented by the official competition results during normal snow sports training, and to specifically practice and correct their own movements to obtain a higher competition score and / or rating.

[0024] In some optional embodiments, the process of obtaining the environmental information includes:

[0025] Obtain the environmental information in real time by using one or more environmental sensors; the environmental information includes one or more of temperature information, humidity information, and air pressure information;

[0026] The process of obtaining the snowboard structure information of the target person includes:

[0027] Perform 3D scanning on the snowboard of the target person by using a 3D scanning device to obtain the snowboard structure information of the target person;

[0028] The process of obtaining the body index measurement information of the target person includes:

[0029] Measure the body index measurement information of the target person by using a height and weight measuring instrument; the body index measurement information of the target person includes the height information and weight information of the target person.

[0030] The beneficial effects of this technical solution are as follows: On the one hand, it realizes the application of new technologies such as 3D scanning devices and sensors in the monitoring field, and promotes the double improvement of monitoring quality and efficiency. On the other hand, it quantifies and records the influence of factors such as environmental status, snowboard structure, and athlete's body index on snow sports training, so as to reasonably monitor the influence of factors such as environmental status, snowboard structure, and athlete's body index on snow sports project training, which is beneficial for athletes to carry out sports training more scientifically and effectively.

[0031] In some optional implementation manners, the training process of the motion evaluation prediction model includes:

[0032] Obtain a training set, the training set includes multiple training data, and each training data includes the motion data fusion result of a sample person and the labeled data of the motion evaluation prediction result of the sample person;

[0033] For each training data in the training set, perform the following processing:

[0034] Input the motion data fusion result of the sample person in the training data into a preset neural network to obtain the predicted data of the motion evaluation prediction result of the sample person;

[0035] Update the model parameters of the neural network based on the predicted data and the labeled data of the motion evaluation prediction result of the sample person;

[0036] Detect whether the preset training end condition is satisfied; if so, use the trained neural network as the motion evaluation prediction model; if not, continue to train the neural network by using the next training data.

[0037] The beneficial effects of this technical solution are as follows: By modeling and statistically analyzing relevant motion data, establishing an appropriate number of neuron computing nodes and a multi-layer operation hierarchy, and selecting appropriate input and output layers, a preset neural network can be obtained. Through the learning and optimization of this preset neural network, a functional relationship from input to output can be established, and a motion evaluation prediction model can be obtained to master the development trend of relevant indicators, find complex coupling relationships between various characteristic parameters, analyze relevant kinematic and dynamic characteristic parameters, and study the non-linear relationship between the characteristic parameters of the ski jumping event and the motion effect, which has important guiding significance for improving the motion of ski jumping athletes and enhancing their performance.

[0038] In some optional embodiments, the method further includes:

[0039] Record the feature extraction results corresponding to the real-time acquired image and the motion sensing information, as well as the environmental information, the snowboard structure information of the target person, and the body index measurement information as the data to be fused. When part or all of the data in the data to be fused changes, perform data fusion on the changed data to be fused to obtain a new motion data fusion result;

[0040] Input the new motion data fusion result into the motion evaluation prediction model to obtain a new motion evaluation prediction result;

[0041] Compare the motion evaluation prediction result of the target person with the new motion evaluation prediction result to obtain a comparison result;

[0042] Generate an evaluation improvement suggestion based on the comparison result and send it to a preset user device.

[0043] The beneficial effects of this technical solution are as follows: Based on the comparison result, an evaluation improvement suggestion is generated, enabling each subject to obtain a personalized exercise prescription, guiding each athlete's snow sports training more scientifically, reasonably, and pertinently, which plays an important role in improving the technical level of the athletes' snow sports. Among them, under the condition that other conditions remain unchanged, by changing single or multiple factors such as the snowboard structure parameters, environmental parameters, the height and weight of the target person, or a certain technical action, and comparing the previous motion evaluation prediction result of the target person with the latest motion evaluation prediction result of the target person one by one, the relevant factors affecting the motion evaluation prediction result of the target person can be found. It helps athletes choose the most suitable snowboard for themselves, judge the influence of environmental factors and height and weight factors on personal performance, and at the same time helps coaches guide or helps athletes adjust their motion postures.

[0044] In a second aspect, the present application provides a snow sports information monitoring device for monitoring snow sports information. The device includes:

[0045] An image acquisition module, configured to use an image acquisition device to collect in real time images of a target person during movement in a preset area, denoted as real-time acquisition images, where the preset area is an area where the target person performs technical actions of snow sports;

[0046] An information acquisition module, configured to use one or more sensors to acquire in real time the motion sensing information of the target person; the motion sensing information includes one or more of the following information: multi-dimensional force information between the shoes and the skis, plantar flexible force information, ski acceleration information, and the target person's center of gravity movement trajectory information, joint acceleration information, joint angle information, and joint position information;

[0047] A feature extraction module, configured to perform feature extraction on the real-time acquisition images and the motion sensing information respectively to obtain feature extraction results corresponding to the real-time acquisition images and the motion sensing information; wherein, the feature extraction result corresponding to the real-time acquisition images is used to indicate the human posture of the target person, and the feature extraction result corresponding to the motion sensing information includes the kinematic and dynamic characteristics of the target person;

[0048] A data fusion module, configured to perform data fusion on the feature extraction results corresponding to the real-time acquisition images and the motion sensing information, as well as the environmental information, the target person's ski structure information, and body index measurement information, to obtain a motion data fusion result of the target person;

[0049] An evaluation and prediction module, configured to input the motion data fusion result of the target person into a motion evaluation and prediction model to obtain a motion evaluation and prediction result of the target person.

[0050] In some optional implementation manners, the motion sensing information includes multi-dimensional force information between the shoes and the board, plantar flexible force information, ski acceleration information, and the person's center of gravity movement trajectory information, joint acceleration information, joint angle information, and joint position information;

[0051] The process of obtaining the feature extraction result corresponding to the motion sensing information includes:

[0052] Based on the motion sensing information of the target person, obtain the data to be processed of multiple motion parameters of the target person; the multiple motion parameters include the three-dimensional force of the target person, the position of the center of motion mass, the speed of each joint, the angular speed of each joint, the acceleration of each joint, the angular acceleration of each joint, and the angle of each joint;

[0053] Perform normalization processing on the data to be processed of the multiple motion parameters of the target person to obtain a normalized result of the motion parameters of the target person;

[0054] Based on the normalization result of the motion parameters of the target person, using the gradual change of different colors to represent different numerical magnitudes, convert and generate the stacked inertial signal image of the target person;

[0055] Extract features from the stacked inertial signal image of the target person to obtain the feature extraction result corresponding to the motion sensing information.

[0056] In some optional embodiments, the motion evaluation prediction result of the target person is used to indicate the motion score and / or rating of the target person.

[0057] In some optional embodiments, the process of obtaining the environmental information includes:

[0058] Use one or more environmental sensors to obtain the environmental information in real time; the environmental information includes one or more of temperature information, humidity information, and air pressure information;

[0059] The process of obtaining the snowboard structure information of the target person includes:

[0060] Use a 3D scanning device to perform 3D scanning on the snowboard of the target person to obtain the snowboard structure information of the target person;

[0061] The process of obtaining the body index measurement information of the target person includes:

[0062] Use a height and weight measuring instrument to measure the body index measurement information of the target person; the body index measurement information of the target person includes the height information and weight information of the target person.

[0063] In some optional embodiments, the training process of the motion evaluation prediction model includes:

[0064] Obtain a training set, the training set includes a plurality of training data, and each training data includes the motion data fusion result of a sample person and the labeled data of the motion evaluation prediction result of the sample person;

[0065] For each training data in the training set, perform the following processing:

[0066] Input the motion data fusion result of the sample person in the training data into a preset neural network to obtain the prediction data of the motion evaluation prediction result of the sample person;

[0067] Based on the prediction data and labeled data of the motion evaluation prediction result of the sample person, update the model parameters of the neural network;

[0068] Check whether the preset training end condition is satisfied; if so, use the trained neural network as the motion evaluation prediction model; if not, continue to train the neural network using the next training data.

[0069] In some optional embodiments, the device further includes a recommendation generation module, and the recommendation generation module is configured to:

[0070] Record the feature extraction results corresponding to the real-time acquired image and the motion sensing information, as well as the environmental information, the snowboard structure information of the target person, and the body index measurement information as the data to be fused. When part or all of the data in the data to be fused changes, perform data fusion on the changed data to be fused to obtain a new motion data fusion result;

[0071] Input the new motion data fusion result into the motion evaluation prediction model to obtain a new motion evaluation prediction result;

[0072] Compare the motion evaluation prediction result of the target person with the new motion evaluation prediction result to obtain a comparison result;

[0073] Generate an evaluation improvement recommendation based on the comparison result and send it to a preset user device.

[0074] In a third aspect, the present application provides a snow sports information monitoring device, which includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the steps of any one of the above methods are implemented.

[0075] In a fourth aspect, the present application provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the steps of any one of the above methods are implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] The present application will be further described below with reference to the drawings and embodiments.

[0077] Figure 1 The flowchart of a snow sports information monitoring method provided by an embodiment of the present application is shown.

[0078] Figure 2 The structural diagram of a snow sports information monitoring device provided by an embodiment of the present application is shown.

[0079] Figure 3 The flowchart of a process for obtaining feature extraction results provided by an embodiment of the present application is shown.

[0080] Figure 4It shows a schematic flowchart of generating evaluation improvement suggestions provided by an embodiment of the present application.

[0081] Figure 5 It shows a schematic flowchart of data fusion provided by an embodiment of the present application.

[0082] Figure 6 It shows a schematic flowchart of obtaining a 3D human pose provided by an embodiment of the present application.

[0083] Figure 7 It shows a schematic structural diagram of a snow sports information monitoring device provided by an embodiment of the present application.

[0084] Figure 8 It shows a schematic block diagram of a snow sports information monitoring device provided by an embodiment of the present application.

[0085] Figure 9 It shows a schematic structural diagram of a program product provided by an embodiment of the present application. Detailed implementation manners

[0086] Next, the technical solutions in the present application will be described in combination with the specification drawings and the detailed implementation manners of the present application. It should be noted that, on the premise of no conflict, any combination can be formed among the following described implementation manners or technical features.

[0087] In the present application, "at least one" means one or more, and "multiple" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects before and after. "At least one (item)" or its similar expression refers to any combination of these items, including any combination of single item (item) or plural items (items). For example, at least one (item) of a, b or c can represent: a, b, c, a and b, a and c, b and c, a and b and c, where a, b and c can be single or multiple. It should be noted that "at least one (item)" can also be interpreted as "one (item) or more items (items)".

[0088] It should also be noted that in the present application, words such as "exemplary" or "for example" are used to represent examples, illustrations or explanations. Any implementation manner or design solution described as "exemplary" or "for example" in the present application should not be interpreted as more preferred or more advantageous than other implementation manners or design solutions. Exactly speaking, using words such as "exemplary" or "for example" aims to present relevant concepts in a specific manner.

[0089] Method Embodiment

[0090] See Figure 1 and Figure 2 , Figure 1 FIG. shows a schematic flow chart of a snow sports information monitoring method provided by an embodiment of the present application, Figure 2 FIG. shows a schematic structural diagram of a snow sports information monitoring device provided by an embodiment of the present application.

[0091] An embodiment of the present application provides a snow sports information monitoring method for monitoring information of snow sports. The method includes:

[0092] Step S101: Use an image acquisition device to collect images of a target person during movement in a preset area in real time, denoted as real-time acquisition images. The preset area is an area where the target person makes technical movements of snow sports;

[0093] Step S102: Use one or more sensors to obtain the motion sensing information of the target person in real time; the motion sensing information includes one or more of the following information: multi-dimensional force information between the shoe and the ski board, plantar flexible force information, ski board acceleration information, and the center of gravity movement trajectory information, joint acceleration information, joint angle information, and joint position information of the target person;

[0094] Step S103: Extract features from the real-time acquisition images and the motion sensing information respectively to obtain feature extraction results corresponding to the real-time acquisition images and the motion sensing information; among them, the feature extraction result corresponding to the real-time acquisition image is used to indicate the human posture of the target person, and the feature extraction result corresponding to the motion sensing information includes the kinematic and dynamic characteristics of the target person;

[0095] Step S104: Perform data fusion on the feature extraction results corresponding to the real-time acquisition images and the motion sensing information, the environmental information, the ski board structure information of the target person, and the body index measurement information to obtain a motion data fusion result of the target person;

[0096] Step S105: Input the motion data fusion result of the target person into a motion evaluation prediction model to obtain a motion evaluation prediction result of the target person.

[0097] This application can achieve long-term, dynamic, and remote multi-point monitoring within the snow sports area, enabling real-time remote acquisition of the motion sensing information parameters of the regional scene and target personnel during snow sports training or competitions, thereby obtaining comprehensive motion sensing information in real-time online (that is, motion sensing information includes kinematic information and / or kinetic information, and correspondingly, motion parameters include kinematic parameters and / or kinetic parameters). First, compared with the traditional manual observation method, this application has no time limit and can monitor the motion process of the target personnel at any time, liberating the monitoring personnel from repetitive work. Second, this application specifically quantifies the motion sensing information so that relevant personnel such as athletes and coaches can obtain the motion information of the target personnel more objectively and scientifically, eliminating the subjectivity of traditional manual observation. Third, it can collect the entire dynamic motion process in real-time, accurately record the real-time collected images, and the motion sensing information of the target personnel can be remotely viewed without location restrictions. On the other hand, through new technologies such as artificial intelligence, feature extraction is performed on the real-time collected images and the motion sensing information to maximize the extraction of favorable information in each signal / data, so as to improve the utilization rate of images and information. And the feature extraction results corresponding to the real-time collected images and the motion sensing information are fused with the environmental information, the snowboard structure information of the target personnel, and the body index measurement information to obtain the motion data fusion result of the target personnel. After that, the obtained motion data fusion result is input into the motion evaluation prediction model to obtain the motion evaluation prediction result for each target personnel, making the result more targeted and scientific, empowering precise teaching of snow sports, and improving the intelligent level of snow sports training management. The advantage of doing this is to use artificial intelligence technology (such as the motion evaluation prediction model, etc.) to intelligently obtain the motion evaluation prediction result, so that the coach can specifically propose a training plan for the target personnel, thereby providing a snow sports information monitoring function characterized by multi-source information collection and real-time data acquisition. This application fills the gap in the domestic technology of real-time information monitoring of multi-source information of athletes and can greatly provide accurate suggestions for sports coaches.

[0098] In view of the characteristics of snow sports, this application integrates multi-dimensional force information between shoes and skis, plantar flexible force information, ski acceleration information, as well as the center-of-gravity movement trajectory information of the target person, joint acceleration information, joint angle information, joint position information, etc., to build a comprehensive real-time online acquisition platform for kinematic information and dynamic information, and realize the real-time synchronous acquisition of parameters such as ski acceleration, three-dimensional force, plantar pressure, and human body posture and position. This application designs and constructs a data format and dataset architecture that are convenient for storage, management, and invocation, and constructs a dataset of sports science information for snow sports. Secondly, this application combines a deep neural network with an information fusion model through a designed effective multi-source heterogeneous data fusion method to generate multiple inferences, and can build a characteristic parameter index system for snow sports related to the exercise effect; finally, this application can realize the evaluation of the exercise effect by studying the non-linear relationship between the characteristic parameters of snow sports and the exercise effect.

[0099] Kinematic is a branch discipline of theoretical mechanics, which studies the motion of objects from a geometric perspective. Here, "motion" refers to mechanical motion, that is, the change in the position of an object. It is a branch of mechanics that describes and studies the law of change of an object's position over time from a geometric perspective (referring to not involving the physical properties of the object itself and the forces applied to the object). What it studies is the motion itself, mainly expressing the magnitude and direction relationships among the quantities of an object's velocity, acceleration, and spatial position. Pure kinematic research does not involve the mass of an object, nor does it involve forces; often, an object is abstracted as a particle or a certain geometric shape, and the velocity, acceleration, and relative position relationships between characteristic points are studied. The equations listed with angles, velocities, accelerations, etc. are kinematic equations.

[0100] Dynamics is a branch discipline of theoretical mechanics, which mainly studies the relationship between the forces acting on an object and the object's motion. The equation of equality listed according to the longitudinal force situation or the lateral force situation is the dynamic equation.

[0101] Dynamics and kinematics are closely related. The bridge connecting the two is "Newton's second law: F = m × a", where a belongs to the kinematic category and F is the force. Kinematics mainly describes the motion state of an object (displacement, velocity, acceleration) and does not involve forces; conversely, dynamics is mainly based on the word "motion". Due to "motion", acceleration a is generated (the acceleration a of uniform linear motion is 0), and thus there is an inertial force m×a. The prerequisite for dynamic analysis must be kinematic analysis.

[0102] Kinematic mainly studies the knowledge of the pose-time relationship of the object being studied and does not involve the forces that cause pose changes at all. It is divided into forward kinematics and inverse kinematics. Dynamics (motion mechanics) mainly studies: what kind of motion trajectory an object will generate under a given acting force, or what kind of force should be applied to achieve a specified motion trajectory.

[0103] In the embodiments of the present application, the multi-dimensional force information between the shoes and the ski board and the plantar flexible force information belong to the relevant information of dynamics, and the three-dimensional force belongs to the motion parameters related to dynamics.

[0104] The ski board acceleration information, the center-of-gravity movement trajectory information of the target person, the joint acceleration information, the joint angle information, and the joint position information belong to the relevant information of kinematics. The ski board acceleration, the position of the moving center of mass, the speeds of each joint, the angular speeds of each joint, the accelerations of each joint, the angular accelerations of each joint, and the angles of each joint belong to the motion parameters related to kinematics.

[0105] The multiple motion parameters of the target person include the above-mentioned motion parameters related to kinematics and the motion parameters related to dynamics. Therefore, (normalizing the data to be processed of the motion parameters, generating a stacked inertial signal image by using the normalization result, and extracting features from the stacked inertial signal image), the feature extraction result corresponding to the motion sensing information includes kinematic and dynamic features. Among them, the kinematic features are the features corresponding to the ski board acceleration, the position of the moving center of mass, the speeds of each joint, the angular speeds of each joint, the accelerations of each joint, the angular accelerations of each joint, and the angles of each joint, and the dynamic features are the features corresponding to the three-dimensional force.

[0106] In the present application, the motion sensing information of the target person refers to the information collected by one or more sensors corresponding to the target person. Some sensors can be worn on the target person, and some sensors can be set on the clothes, shoes or ski board.

[0107] Snow sports can include, for example, ski jumping, alpine skiing, figure skiing, freestyle skiing, etc. Freestyle skiing can include, for example, big air, aerials, moguls, halfpipe and slopestyle and other sub-events. Ski jumping can include, for example, three height categories of 70 meters, 90 meters and 120 meters, and can also be divided into individual events and team events. Figure skiing can be divided into snowboard skiing and alpine skiing, and figure skiing can also include, for example, snow ballet, tricks, downhill and other sub-events.

[0108] As an example, the scoring criteria for freestyle skiing mainly involve the skills in one or both of the following aspects: aerials and moguls. The scoring elements in the aspect of aerials mainly include takeoff, movement, landing; the scoring elements in the aspect of moguls mainly include rotation, takeoff, speed.

[0109] As an example, ski jumping is a snow sports event in which a ski board is used as a tool, and the speed obtained by the weight of the person on a dedicated ski jump platform through the starting slope is used to compare the jumping distance and the action posture, and the jumping distance is calculated in meters of flight.

[0110] Therefore, one or more image acquisition devices are placed on the venue of the snow sports project, and the real-time acquisition images or real-time acquisition videos of the preset area are obtained in real time by using the image acquisition devices, so as to more intuitively observe the real-time environmental state of the points in this area and the movement conditions of the athletes. The preset area is the area where the target person makes the technical movements of the snow sports.

[0111] One or more sensors are used to obtain in real time various types of motion sensing information of different joint parts of the target person.

[0112] The embodiments of the present application do not limit the image acquisition device, which may be, for example, a camera, a video camera, a camera, a scanner, or other devices with a photographing function (such as a mobile phone, a tablet computer, etc.). Among them, the camera may include, for example, an optical camera and / or an infrared camera.

[0113] In the embodiments of the present application, the target person may be, for example, an athlete, a coach, a sparring partner, etc.

[0114] In the embodiments of the present application, the snow sports may include one or more of the following: ski jumping, alpine skiing, figure skiing, freestyle skiing, etc.

[0115] The technical movements of each snow sport may not be completely the same. That is to say, there may be some same or similar situations in the technical movements of each snow sport.

[0116] In a practical application, the technical movements of figure skiing may include: skiing down, rotating, jumping, flipping in the air, etc.

[0117] In a practical application, the technical movements of ski jumping may include: gliding, taking off, flying in the air, and skiing in the landing termination area.

[0118] As an example, in the women's freestyle skiing big air final, the motion recognition result of athlete A is "the type of technical movement is a left-deflected off-axis spin, the degree of rotation in the air is 1620 degrees, the difficulty rating is the highest, the completion rate is 99%, and the score is 94.50".

[0119] As another example, in the ski jumping competition, the motion recognition result of athlete B is "the ski jump height is 90 meters, the difficulty rating is medium, the completion rate is 86%, and the score is 83.20". As another example, in the snowboard competition, the motion recognition result of athlete C is "the type of technical movement is Back side 5 40 inner turn 540, the difficulty rating is high, the completion rate is 95%, and the score is 90.00".

[0120] In some embodiments, the center-of-gravity movement trajectory information of the target person can be represented, for example, by a trajectory curve or trajectory points; the multi-dimensional force information between the shoe and the ski can be represented, for example, by a vector or a scalar; the ski acceleration information can be represented, for example, by three-dimensional coordinate data; the sole flexural force information can be represented, for example, by a vector or a scalar; the joint acceleration information can be represented, for example, by three-dimensional coordinate data; the joint angle information can be represented, for example, by a numerical value; and the joint position information can be represented, for example, by three-dimensional coordinate data.

[0121] In the embodiments of the present application, the joint pose information of each joint of the target person, the pose information of the ski, the pose information of the camera, etc. can also be obtained. Among them, the pose information can include three pose angles (i.e., pitch angle, roll angle, and yaw angle).

[0122] The embodiments of the present application do not limit the preset area. A snow sports event can include one preset area or multiple preset areas.

[0123] The embodiments of the present application do not limit the accuracy and data type of the inertial sensor. For example, it can be a low-precision, medium-precision, or high-precision inertial sensor, or an acceleration data type sensor or an angle type inertial sensor.

[0124] The embodiments of the present application do not limit the extracted features. For example, they can be information such as edges, shapes, contours, and local features.

[0125] The embodiments of the present application do not limit the data fusion method. For example, it can adopt data-level fusion, feature-level fusion, decision-level fusion, etc.

[0126] The feature-level fusion in the embodiments of the present application refers to extracting feature vectors that can reflect the attributes of the target person from the data collected by the above-mentioned sensors and image acquisition devices. Information fusion of the features of the target person at this level is feature-level fusion. The reason why this method is feasible is that some key feature information can replace all data information.

[0127] The embodiments of the present application do not limit the technical actions. For example, they can include skiing downhill, rotating, jumping, flipping, assisting in skiing, taking off, flying in the air, and landing and sliding in the termination area.

[0128] See Figure 3 , Figure 3 which shows a schematic flow chart of obtaining a feature extraction result provided by the present application.

[0129] In some optional embodiments, the motion sensing information includes multi-dimensional force information between the shoe and the ski board, plantar flexible force information, ski board acceleration information, target person's center of gravity movement trajectory information, joint acceleration information, joint angle information, and joint position information;

[0130] The process of obtaining the feature extraction result corresponding to the motion sensing information includes:

[0131] Step S201: Based on the motion sensing information of the target person, obtain the data to be processed of multiple motion parameters of the target person; the multiple motion parameters include the target person's three-dimensional force, ski board acceleration, movement center of mass position, velocities of each joint, angular velocities of each joint, accelerations of each joint, angular accelerations of each joint, and angles of each joint;

[0132] Step S202: Perform normalization processing on the data to be processed of the multiple motion parameters of the target person to obtain the normalized result of the motion parameters of the target person;

[0133] Step S203: Based on the normalized result of the motion parameters of the target person, use different color gradients to represent different numerical magnitudes, and convert and generate the stacked inertial signal image of the target person;

[0134] Step S204: Perform feature extraction on the stacked inertial signal image of the target person to obtain the feature extraction result corresponding to the motion sensing information.

[0135] Therefore, this application needs to collect multi-dimensional force information between the athlete's shoe and the ski board, plantar flexible force information, ski board acceleration information, target person's center of gravity movement trajectory information, joint acceleration information, joint angle information, and joint position information, and analyze to obtainThe three-dimensional force, snowboard acceleration, position of the center of mass of movement, velocities of each joint, angular velocities of each joint, accelerations of each joint, angular accelerations of each joint, and angles of each joint of the target person. The 3D data contained in each joint among these types of motion inertia data (i.e., three-dimensional force, snowboard acceleration, position of the center of mass of movement, velocities of each joint, angular velocities of each joint, accelerations of each joint, angular accelerations of each joint, and angles of each joint) respectively correspond to the three directions of X, Y, and Z. Normalize the above data so that the data value range is mapped to between [0, 1]. This normalization process can make the originally incomparable data comparable, while retaining the magnitude relationship existing in the original data and eliminating the influence of dimension and data value range. The purpose of performing normalization here also includes normalizing the indicators to facilitate subsequent data processing. For example, after normalization, it is very convenient to perform subsequent data stacking to obtain a stacked inertia signal image. In addition, this application uses data visualization technology to more accurately and clearly express the above information, providing a basis for coaches and athletes to further conduct reasonable and comprehensive motion analysis on relevant information.

[0136] In some embodiments, the position of the center of mass of movement can be represented, for example, by a trajectory curve or a trajectory point; the velocities of each joint, angular velocities of each joint, accelerations of each joint, angular accelerations of each joint, and angles of each joint can be represented by three-dimensional data.

[0137] The embodiments of this application do not limit the image format. For example, it can include JPG, JPEG, RAW, PNG, GIF, etc.

[0138] In some embodiments, the joint center information can be represented, for example, by a two-dimensional trajectory curve or a trajectory point; the joint length information can be represented, for example, by one-dimensional data.

[0139] In some embodiments, capturing the technical action posture can be, for example, using a human body posture model: connecting key nodes to describe the pose of the target person.

[0140] The embodiments of this application do not limit the number of key nodes. For example, it can be 2, 4, 5, 6, 7, 8, 10, 12, 16, 18, 20, 24, 28, 32, 36, 38, etc.

[0141] In some optional embodiments, the motion evaluation and prediction result of the target person is used to indicate the motion score and / or rating of the target person.

[0142] Select scoring and / or grading criteria that are identical or similar to those used in official competitions, specifically quantify them, and more clearly articulate the factors that influence the target participant's competition score. For example, factors such as ski parameters, environmental information, the target participant's height, weight, and technique can influence competition scores, allowing for the objective selection of the most suitable ski parameters for the athlete. Scoring and / or grading provide a more direct indication of the target participant's progress in the technique and the effectiveness of their training. This can further help the target participant correct their form and even incorrect force application habits, timing, range of motion, and frequency of movement. This helps the target participant achieve accurate and precise movements with proper form, improves training efficiency and effectiveness, and enhances competition results, further highlighting areas for improvement. This can help the target participant use official competition results as a guide in their regular snow sports training, allowing them to practice and correct their movements to achieve higher scores and / or ratings.

[0143] In some optional implementations, the process of obtaining the environmental information may include:

[0144] Using one or more environmental sensors to obtain the environmental information in real time; the environmental information includes one or more of temperature information, humidity information and air pressure information;

[0145] The process of obtaining the target person's ski board structure information may include:

[0146] Performing a 3D scan on the target person's skis using a 3D scanning device to obtain structural information of the target person's skis;

[0147] The process of obtaining the target person's physical indicator measurement information may include:

[0148] The body index measurement information of the target person is obtained by measuring with a height and weight measuring instrument; the body index measurement information of the target person includes the height information and weight information of the target person.

[0149] By installing environmental sensors, environmental parameters such as temperature, humidity and air pressure can be obtained in real time; the ski board parameters are measured through 3D scanning equipment. The ski board coefficient mainly includes the length, width, hardness, board shape, curvature, radian, weight and arch structure of the ski board to obtain the ski board structural coefficient; the height and weight measuring instrument can be used to measure the athlete's height, weight and other data.

[0150] On the one hand, the application of new technologies such as 3D scanning devices and sensors in the monitoring field has been realized, promoting the double improvement of monitoring quality and efficiency. On the other hand, the impacts of factors such as the environmental state, snowboard structure, and athlete's physical indicators on snow sports training are quantified and recorded, so as to reasonably monitor the impacts of factors such as the environmental state, snowboard structure, and athlete's physical indicators on snow sports training, which is beneficial for athletes to carry out sports training more scientifically and effectively.

[0151] In the embodiments of this application, the target personnel are not limited, for example, they can be athletes, coaches, sparring partners, etc.

[0152] In the embodiments of this application, the environmental sensors are not limited, and for example, they can include temperature and humidity sensors, barometers, etc.

[0153] In the embodiments of this application, the height and weight measuring instrument is not limited, and for example, it can be an ultrasonic height and weight measuring instrument, etc.

[0154] In the embodiments of this application, the snowboard is not limited, for example, it can be a single-board, double-board, biathlon board, jump board, freestyle board, etc.

[0155] In the embodiments of this application, the snowboard structure information is not limited, and for example, it can include one or more of length, width, hardness, board shape, curvature, arc, weight, and arch structure.

[0156] In some optional embodiments, the training process of the motion evaluation prediction model includes:

[0157] Obtain a training set, the training set includes a plurality of training data, and each training data includes the motion data fusion result of a sample person and the labeled data of the motion evaluation prediction result of the sample person;

[0158] For each training data in the training set, perform the following processing:

[0159] Input the motion data fusion result of the sample person in the training data into a preset neural network to obtain the prediction data of the motion evaluation prediction result of the sample person;

[0160] Based on the prediction data and labeled data of the motion evaluation prediction result of the sample person, update the model parameters of the neural network;

[0161] Detect whether the preset training end condition is satisfied; if so, use the trained neural network as the motion evaluation prediction model; if not, continue to train the neural network with the next training data.

[0162] In the embodiments of this application, the neural network can, for example, adopt a machine learning network or a deep learning network.

[0163] By modeling and statistically analyzing relevant motion data, establishing an appropriate number of neuron computing nodes and a multi-layer operation hierarchy, and selecting appropriate input and output layers, a preset neural network can be obtained. Through the learning and optimization of the preset neural network, a functional relationship from input to output can be established, and a motion evaluation prediction model can be obtained to grasp the development trend of relevant indicators, find the complex coupling relationship between various characteristic parameters, analyze relevant kinematic and dynamic characteristic parameters, and study the non-linear relationship between the characteristic parameters of the ski jumping event and the motion effect, which has important guiding significance for improving the motion of ski jumpers and enhancing their performance.

[0164] Among them, the motion evaluation prediction model can be trained by a large amount of training data, and can predict corresponding output data (i.e., the motion evaluation prediction result of the target person) for different input data (i.e., the fusion result of the motion data of the target person), with a wide range of applications and a high level of intelligence.

[0165] According to the internal description of the feature extraction result in a specific semantics, or the recombination of feature attributes. Feature-level fusion of images is performed, features of the images are extracted, and information such as edges, shapes, contours, and local features is comprehensively processed. In order to analyze relevant kinematic and dynamic characteristic parameters, study the non-linear relationship between the characteristic parameters of snow sports events and the motion effect, which has important guiding significance for improving the motion of snow athletes.

[0166] In some optional embodiments, the motion evaluation prediction model of the embodiments of the present application can be trained by the above training process. In some other optional embodiments, the embodiments of the present application can use a pre-trained motion evaluation prediction model.

[0167] The embodiments of the present application do not limit the acquisition method of the labeled data. For example, the manual labeling method can be used, or the automatic labeling or semi-automatic labeling method can be used.

[0168] The embodiments of the present application do not limit the training process of the motion evaluation prediction model. For example, the above supervised learning training method can be used, or the semi-supervised learning training method can be used, or the unsupervised learning training method can be used.

[0169] The embodiments of the present application do not limit the preset training end condition. For example, it can be that the number of training times reaches a preset number (the preset number is, for example, 1 time, 3 times, 10 times, 100 times, 1000 times, 10000 times, etc.), or it can be that all the training data in the training set have completed one or more trainings.

[0170] See Figure 4 、 Figure 5 、Figure 6 , Figure 4 shows a schematic flowchart of a process for generating evaluation improvement suggestions provided by an embodiment of the present application. Figure 5 shows a schematic flowchart of a process for data fusion provided by an embodiment of the present application. Figure 6 shows a schematic flowchart of a process for obtaining a 3D human body posture provided by an embodiment of the present application.

[0171] In some optional embodiments, the method may further include:

[0172] Step S401: Denote the feature extraction results corresponding to the real-time acquisition image and the motion sensing information, as well as the environmental information, the snowboard structure information of the target person, and the body index measurement information as the data to be fused. When part or all of the data in the data to be fused changes, perform data fusion on the changed data to be fused to obtain a new motion data fusion result;

[0173] Step S402: Input the new motion data fusion result into the motion evaluation prediction model to obtain a new motion evaluation prediction result;

[0174] Step S403: Compare the motion evaluation prediction result of the target person with the new motion evaluation prediction result to obtain a comparison result;

[0175] Step S404: Generate an evaluation improvement suggestion based on the comparison result and send it to a preset user device.

[0176] In an embodiment of the present application, the preset user device may, for example, include user devices of coaches, athletes, technical officials, dietitians, equipment suppliers, etc. The embodiment of the present application does not limit the user device, which may, for example, be a smart terminal device such as a mobile phone, a tablet computer, a laptop computer, a desktop computer, a smart wearable device, or the user device may be a workstation or a console.

[0177] The ways to send evaluation improvement suggestions are, for example, SMS push, email push, in-app push, phone notification, etc., and the applications are, for example, WeChat APP, Alipay APP, mini-programs, etc. Based on the comparison results, evaluation improvement suggestions are generated, so that each subject can obtain a personalized exercise prescription, guiding each athlete to conduct snow sports training more scientifically, reasonably and pertinently, which plays an important role in improving the snow sports technical level of athletes. Among them, under the condition that other conditions remain unchanged, by only changing one or more factors such as snowboard structure parameters, environmental parameters, the height and weight of the target person or a certain technical movement, and comparing the previous exercise evaluation prediction results of the target person with the latest exercise evaluation prediction results of the target person one by one, the relevant factors affecting the exercise evaluation prediction results of the target person can be found. It helps athletes choose the most suitable snowboard for themselves and judge the influence of environmental factors and height and weight factors on their personal performance, and at the same time helps coaches to guide or helps athletes to adjust their exercise postures.

[0178] In some optional implementation manners, the exercise evaluation prediction results of the target person may include, for example, one or more of the type of technical movement, movement parameters, difficulty rating, completion degree and score.

[0179] As an example, in freestyle skiing practice, when the snowboard structure parameters, the technical movement of athlete A, etc. remain unchanged, only the environmental temperature is decreased from -5 degrees Celsius to -10 degrees Celsius (that is, the temperature parameter in the environmental parameters is changed). By comparing the previous and latest exercise evaluation prediction results of athlete A one by one, it is found that the score of the latest exercise evaluation prediction result is 1.5 points lower. Therefore, it is concluded that the overall score of this athlete will decrease at -10 degrees Celsius. Therefore, the content of the generated evaluation improvement suggestion can be that it is recommended that the athlete needs to make preparations in advance, as the temperature change may affect his / her competition score result.

[0180] As another example, in snowboard skiing competition, when the environmental parameters and the technical movement of athlete B, etc. remain unchanged, only the snowboard length parameter is increased from 155 cm to 160 cm (that is, the snowboard structure parameter is changed). By comparing the previous and latest exercise evaluation prediction results of athlete B one by one, it is found that the score of athlete B in the latest exercise evaluation prediction result is 1 point lower. Therefore, the content of the generated evaluation improvement suggestion can be that it is recommended that this athlete continue to use a 155-cm snowboard.

[0181] As another example, in freestyle skiing practice, with the snowboard structure parameters, environmental parameters, and the height and weight of athlete C remaining unchanged, only one technical movement during the movement of athlete C is changed. By comparing the motion evaluation and prediction results of athlete C before and after one by one, it is found that the score of the latest motion evaluation and prediction result is 1 point higher. It is concluded that the change in this technical movement can increase the overall score of athlete C. Therefore, the content of the generated evaluation improvement suggestion can be to suggest that the coach guide the athlete to perfect the changed technical movement.

[0182] As yet another example, in ski jumping practice, with the ski jump height, snowboard structure parameters, environmental parameters, and the technical movements of athlete D remaining unchanged, only the weight of athlete D's ski suit is increased by 2 kilograms. By comparing the motion evaluation and prediction results of the current two times one by one, it is found that the jumping distance of athlete D in the latest motion evaluation and prediction result has increased by 0.5 meters. Therefore, the content of the generated evaluation improvement suggestion can be to suggest customizing a heavier ski suit suitable for this athlete.

[0183] As yet another example, in snowboard skiing practice, with the snowboard structure parameters, environmental parameters, and the technical movements of athlete E (to distinguish from athletes A, B, C, and D) remaining unchanged, the diet structure of athlete E is changed for a period of time, and protein foods are appropriately increased. By comparing the motion evaluation and prediction results of athlete E before and after many times, it is found that the score of athlete E in the latest motion evaluation and prediction result is continuously improving. Therefore, the content of the generated evaluation improvement suggestion can be to suggest that the dietitian change the diet structure of athlete E and appropriately increase the intake of protein foods for athlete E. Continue to refer to Figure 5 , Figure 5 shows a schematic flowchart of data fusion provided by an embodiment of the present application.

[0184] In some embodiments, each motion sensing information is processed separately, and then fused to generate a stacked inertial signal image, and features are extracted through a convolutional network. Secondly, the images collected by the image acquisition device in real time are processed to obtain the final 3D human pose. Then, the feature extraction results of the convolutional network and the 3D human pose are fused.

[0185] Continue to refer to Figure 6 [ , Figure 6 shows a schematic flowchart of obtaining a 3D human pose provided by an embodiment of the present application.

[0186] In some embodiments, a three-dimensional human body model is built from a two-dimensional image (i.e., the real-time captured image). For the two-dimensional image captured by an image acquisition device, first, a target detection network is used to perform target detection on the image to obtain an image with a detection box containing the target person; subsequently, a High-Resolution Network (HRNet) is used for 2D joint detection; based on the detected 2D joint coordinates, 3D joints P are constructed through polar geometric transformation; then, the 3D joints P, shape parameters β, and twist angle Φ are sent to a hybrid inverse kinematics (Hybrik) module to solve for the relative rotation, i.e., the pose parameter θ; finally, the pose parameter θ is input into the static pose T, and the reconstructed human body mesh M is obtained through regression, thereby generating a 3D human body pose.

[0187] In some embodiments, the High-Resolution Network includes multi-resolution parallel branches.

[0188] In some embodiments, the High-Resolution Network can take the high-resolution convolutional branch as the first step, gradually add high-to-low resolution branches one by one to form a new stage, and then connect the multi-resolution branches in parallel. The resolution of the next-level parallel branch includes the resolution of the previous level and a lower resolution.

[0189] In some embodiments, the High-Resolution Network can adopt repeated multi-scale feature information fusion.

[0190] In some embodiments, the human body mesh M can be output through the SMPL model.

[0191] In some embodiments, the SMPL model parameters can be input into 3D software to obtain a visualized SMPL model, that is, to generate a 3D human body pose.

[0192] In some embodiments, the SMPL model parameters can include the number of vertices and the number of faces.

[0193] The embodiments of the present application do not limit the number of vertices of the SMPL model, which can be, for example, 6890.

[0194] The embodiments of the present application do not limit the number of faces of the SMPL model, which can be, for example, 13776.

[0195] In some embodiments, the shape parameters β and the twist angle Φ can be obtained through full connection after being calculated from the 2D joint coordinates.

[0196] In some embodiments, the base template and the shape parameters β can be input to adjust the human body figure, and the human body mesh M in the static pose T is output.

[0197] In some embodiments, the base template is the mean value obtained from a large number of real human body meshes.

[0198] In some embodiments, the entire mesh is represented by endpoints, and each endpoint includes three spatial coordinates of x, y, and z.

[0199] The embodiments of the present application do not limit the number of endpoints, which can be, for example, 6890.

[0200] The embodiments of the present application do not limit the pose parameters, which can be, for example, the rotation angles of 24 joints.

[0201] Device embodiments

[0202] See Figure 7 , Figure 7 which shows a schematic structural diagram of a snow sports information monitoring device provided by an embodiment of the present application.

[0203] The embodiments of the present application also provide a snow sports information monitoring device for monitoring information on snow sports. The device includes:

[0204] An image acquisition module 101, configured to use an image acquisition device to collect in real time an image of a target person during movement in a preset area, denoted as a real-time acquisition image. The preset area is an area where the target person makes a technical movement of snow sports;

[0205] An information acquisition module 102, configured to use one or more sensors to acquire in real time the motion sensing information of the target person; the motion sensing information includes one or more of the following information: multi-dimensional force information between the shoe and the ski board, plantar flexible force information, ski board acceleration information, and target person's center of gravity movement trajectory information, joint acceleration information, joint angle information, and joint position information;

[0206] A feature extraction module 103, configured to perform feature extraction on the real-time acquisition image and the motion sensing information respectively to obtain feature extraction results corresponding to the real-time acquisition image and the motion sensing information; wherein, the feature extraction result corresponding to the real-time acquisition image is used to indicate the human pose of the target person, and the feature extraction result corresponding to the motion sensing information includes the kinematic and dynamic characteristics of the target person;

[0207] A data fusion module 104, configured to perform data fusion on the feature extraction results corresponding to the real-time acquisition image and the motion sensing information, as well as the environmental information, the ski board structure information of the target person, and the body index measurement information, to obtain a motion data fusion result of the target person;

[0208] The evaluation and prediction module 105 is configured to input the motion data fusion result of the target person into a motion evaluation and prediction model to obtain the motion evaluation and prediction result of the target person.

[0209] In some optional embodiments, the motion sensing information includes multi-dimensional force information between the shoe and the ski board, plantar flexible force information, ski board acceleration information, and the target person's center of gravity movement trajectory information, joint acceleration information, joint angle information, and joint position information.

[0210] The process of obtaining the feature extraction result corresponding to the motion sensing information includes:

[0211] Based on the motion sensing information of the target person, obtain the data to be processed of multiple motion parameters of the target person; the multiple motion parameters include the three-dimensional force of the target person, ski board acceleration, position of the center of motion mass, speed of each joint, angular velocity of each joint, acceleration of each joint, angular acceleration of each joint, and angle of each joint.

[0212] Perform normalization processing on the data to be processed of the multiple motion parameters of the target person to obtain the normalized result of the motion parameters of the target person.

[0213] Based on the normalized result of the motion parameters of the target person, use different color gradients to represent different numerical magnitudes, and convert and generate the stacked inertial signal image of the target person.

[0214] Perform feature extraction on the stacked inertial signal image of the target person to obtain the feature extraction result corresponding to the motion sensing information.

[0215] In some optional embodiments, the motion evaluation and prediction result of the target person is used to indicate the motion score and / or rating of the target person.

[0216] In some optional embodiments, the process of obtaining the environmental information includes:

[0217] Use one or more environmental sensors to obtain the environmental information in real time; the environmental information includes one or more of temperature information, humidity information, and air pressure information.

[0218] The process of obtaining the ski board structure information of the target person includes:

[0219] Use a 3D scanning device to perform 3D scanning on the ski board of the target person to obtain the ski board structure information of the target person.

[0220] The process of obtaining the body index measurement information of the target person includes:

[0221] Measure the body index measurement information of the target person using a height and weight measuring instrument; the body index measurement information of the target person includes the height information and weight information of the target person.

[0222] In some optional embodiments, the training process of the motion evaluation prediction model includes:

[0223] Obtain a training set, the training set includes a plurality of training data, and each training data includes the motion data fusion result of a sample person and the labeled data of the motion evaluation prediction result of the sample person;

[0224] For each training data in the training set, perform the following processing:

[0225] Input the motion data fusion result of the sample person in the training data into a preset neural network to obtain the prediction data of the motion evaluation prediction result of the sample person;

[0226] Update the model parameters of the neural network based on the prediction data and the labeled data of the motion evaluation prediction result of the sample person;

[0227] Detect whether the preset training end condition is satisfied; if so, use the trained neural network as the motion evaluation prediction model; if not, continue to train the neural network using the next training data.

[0228] In some optional embodiments, the device further includes a suggestion generation module 106, and the suggestion generation module 106 is used for:

[0229] Record the feature extraction results of the real-time acquisition image and the motion sensing information, as well as the environment information, the snowboard structure information and the body index measurement information of the target person as the data to be fused. When part or all of the data in the data to be fused changes, perform data fusion on the changed data to be fused to obtain a new motion data fusion result;

[0230] Input the new motion data fusion result into the motion evaluation prediction model to obtain a new motion evaluation prediction result;

[0231] Compare the motion evaluation prediction result of the target person with the new motion evaluation prediction result to obtain a comparison result;

[0232] Generate an evaluation improvement suggestion based on the comparison result and send it to a preset user device.

[0233] Device Embodiment

[0234] The embodiments of the present application also provide a snow sports information monitoring device, which includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the steps of any of the above methods are implemented. The specific implementation manners are the same as those described in the above method embodiments and achieve the same technical effects, and some contents will not be repeated here.

[0235] See Figure 8 , Figure 8 which shows a structural block diagram of a snow sports information monitoring device 200 provided by the embodiments of the present application.

[0236] The snow sports information monitoring device 200 may, for example, include at least one memory 210, at least one processor 220, and a bus 230 connecting different platform systems.

[0237] The memory 210 may include a readable medium in the form of a volatile memory, such as a random access memory (RAM) 211 and / or a cache memory 212, and may further include a read-only memory (ROM) 213.

[0238] Among them, the memory 210 also stores a computer program, and the computer program can be executed by the processor 220, so that the processor 220 implements the steps of any of the above methods.

[0239] The memory 210 may further include a utility 214 having at least one program module 215. Such program modules 215 include, but are not limited to: an operating system, one or more application programs, other program modules, and program data. The implementation of a network environment may be included in each or some combination of these examples.

[0240] Correspondingly, the processor 220 can execute the above computer program and can also execute the utility 214.

[0241] The processor 220 may adopt one or more application specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field programmable gate arrays (FPGAs), or other electronic components.

[0242] The bus 230 can be one or more representing several types of bus structures, including a memory bus or a memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus of any bus structure using multiple bus structures.

[0243] The snow sports information monitoring device 200 can also communicate with one or more external devices 240 such as a keyboard, a pointing device, a Bluetooth device, etc., and can also communicate with one or more devices capable of interacting with the snow sports information monitoring device 200, and / or communicate with any device (such as a router, a modem, etc.) that enables the snow sports information monitoring device 200 to communicate with one or more other computing devices. Such communication can be carried out through the input / output interface 250. Moreover, the snow sports information monitoring device 200 can also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through the network adapter 260. The network adapter 260 can communicate with other modules of the snow sports information monitoring device 200 through the bus 230. It should be understood that although not shown in the figure, other hardware and / or software modules can be used in combination with the snow sports information monitoring device 200, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms, etc.

[0244] Medium embodiments

[0245] The embodiments of the present application also provide a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the steps of any of the above methods are implemented. The specific implementation manners are the same as those recorded in the above method embodiments and the achieved technical effects are consistent. Some contents will not be repeated.

[0246] See Figure 9 , Figure 9 which shows a schematic structural diagram of a program product provided by the embodiments of the present application.

[0247] The program product is used to implement any of the above methods. The program product may be a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In the embodiments of the present application, the readable storage medium may be any tangible medium that contains or stores a program, and the program may be used by or in combination with an instruction execution system, apparatus, or device. The program product may adopt any combination of one or more readable media. The readable media may be a readable signal medium or a readable storage medium. The readable storage medium may, for example, be but is not limited to an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0248] The computer-readable storage medium may include a data signal propagated in a baseband or as part of a carrier wave, in which the readable program code is carried. Such a propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The readable storage medium may also be any readable medium that can send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical fiber cable, RF, etc., or any suitable combination of the above. The program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc., and also including conventional procedural programming languages such as the C language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, executed as a stand-alone software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., by using an Internet service provider to connect through the Internet).

[0249] This application is described from the perspectives of purpose of use, efficacy, progress, and novelty, and has met the functional improvement and usage requirements emphasized by the Patent Law. The above description and accompanying drawings of this application are only preferred embodiments of this application and do not limit this application. Therefore, all those that are similar or identical to the structure, device, features, etc. of this application, that is, all equivalent substitutions or modifications made according to the scope of the patent application of this application, shall fall within the scope of protection of the patent application of this application.

Claims

1. A snow sports information monitoring method, characterized in that: For monitoring information of snow sports, the method comprises: Using an image acquisition device to acquire real-time images of a target person in a preset area while the target person is exercising, which is recorded as a real-time acquired image, wherein the preset area is an area where the target person performs technical movements of snow sports; Using one or more sensors to obtain motion sensing information of the target person in real time; the motion sensing information includes one or more of the following information: multi-dimensional force information between the shoe and the snowboard, sole flexibility force information, snowboard acceleration information, and target person's center of gravity movement trajectory information, joint acceleration information, joint angle information, and joint position information; performing feature extraction on the real-time collected image and the motion sensing information, respectively, to obtain feature extraction results corresponding to the real-time collected image and the motion sensing information; wherein the feature extraction results corresponding to the real-time collected image are used to indicate the human posture of the target person, and the feature extraction results corresponding to the motion sensing information include kinematic and dynamic characteristics of the target person; Performing data fusion on the real-time collected image, the feature extraction results corresponding to the motion sensing information, the environmental information, the ski board structure information, and the physical index measurement information of the target person to obtain a motion data fusion result of the target person; Inputting the motion data fusion result of the target person into the motion evaluation prediction model to obtain the motion evaluation prediction result of the target person; The process of obtaining the feature extraction result corresponding to the motion sensing information includes: Based on the motion sensing information of the target person, obtaining data to be processed of multiple motion parameters of the target person; Normalizing the to-be-processed data of the plurality of motion parameters of the target person to obtain a normalized result of the motion parameters of the target person; Based on the normalized results of the motion parameters of the target person, different color gradients are used to represent different numerical values, and a stacked inertial signal image of the target person is converted and generated; Feature extraction is performed on the stacked inertial signal image of the target person to obtain a feature extraction result corresponding to the motion sensing information.

2. The snow sports information monitoring method according to claim 1, characterized in that: The motion sensing information includes multi-dimensional force information between the shoe and the ski, sole flexibility force information, ski acceleration information, and target person's center of gravity movement trajectory information, joint acceleration information, joint angle information, and joint position information; The multiple motion parameters include the three-dimensional force of the target person, snowboard acceleration, motion center of mass position, joint velocity, joint angular velocity, joint acceleration, joint angular acceleration and joint angle.

3. The snow sports information monitoring method according to claim 1, characterized in that: The motion evaluation prediction result of the target person is used to indicate the motion score and / or rating of the target person.

4. The snow sports information monitoring method according to claim 1, characterized in that: The process of obtaining the environmental information includes: Using one or more environmental sensors to obtain the environmental information in real time; the environmental information includes one or more of temperature information, humidity information and air pressure information; The process of obtaining the target person's ski board structure information includes: Performing a 3D scan on the target person's skis using a 3D scanning device to obtain structural information of the target person's skis; The process of obtaining the target person's physical indicator measurement information includes: The body index measurement information of the target person is obtained by measuring with a height and weight measuring instrument; the body index measurement information of the target person includes the height information and weight information of the target person.

5. The snow sports information monitoring method according to claim 1, characterized in that: The training process of the motion evaluation prediction model includes: Acquire a training set, the training set including a plurality of training data, each of the training data including a motion data fusion result of a sample person and labeled data of a motion evaluation prediction result of the sample person; For each training data in the training set, perform the following processing: Inputting the fusion results of the motion data of the sample persons in the training data into a preset neural network to obtain prediction data of the motion evaluation prediction results of the sample persons; Updating the model parameters of the neural network based on the predicted data and the labeled data of the motion evaluation prediction results of the sample person; Detect whether a preset training end condition is met; if so, use the trained neural network as the motion evaluation prediction model; if not, continue training the neural network using the next training data.

6. The snow sports information monitoring method according to claim 1, characterized in that: The method further comprises: Recording the feature extraction results corresponding to the real-time collected image and the motion sensing information, as well as the environmental information, the target person's ski structure information, and the physical index measurement information as data to be fused; when part or all of the data to be fused changes, performing data fusion on the changed data to obtain a new motion data fusion result; Inputting the new motion data fusion result into the motion evaluation prediction model to obtain a new motion evaluation prediction result; Comparing the target person's motion evaluation prediction result with the new motion evaluation prediction result to obtain a comparison result; Based on the comparison results, an evaluation improvement suggestion is generated and sent to a preset user device.

7. A snow sports information monitoring device, characterized in that: Used for monitoring information of snow sports, the device comprises: An image acquisition module, configured to use an image acquisition device to acquire real-time images of a target person in a preset area while the target person is performing a technical action in a snow sport, which are recorded as real-time acquired images. The preset area is the area where the target person is performing a technical action in a snow sport; an information acquisition module for acquiring, in real time, motion sensing information of the target person using one or more sensors; the motion sensing information includes one or more of the following: multi-dimensional force information between the shoe and the snowboard, sole flexibility force information, snowboard acceleration information, and target person's center of gravity movement trajectory information, joint acceleration information, joint angle information, and joint position information; a feature extraction module, configured to perform feature extraction on the real-time collected image and the motion sensing information, respectively, to obtain feature extraction results corresponding to the real-time collected image and the motion sensing information; wherein the feature extraction results corresponding to the real-time collected image are used to indicate the human posture of the target person, and the feature extraction results corresponding to the motion sensing information include kinematic and dynamic characteristics of the target person; a data fusion module for fusing the real-time collected image, the feature extraction results corresponding to the motion sensing information, the environmental information, the ski structure information and the physical index measurement information of the target person, to obtain a motion data fusion result of the target person; An evaluation prediction module is used to input the motion data fusion result of the target person into a motion evaluation prediction model to obtain a motion evaluation prediction result of the target person; The process of obtaining the feature extraction result corresponding to the motion sensing information includes: Based on the motion sensing information of the target person, obtaining data to be processed of multiple motion parameters of the target person; Normalizing the to-be-processed data of the plurality of motion parameters of the target person to obtain a normalized result of the motion parameters of the target person; Based on the normalized results of the motion parameters of the target person, different color gradients are used to represent different numerical values, and a stacked inertial signal image of the target person is converted and generated; Feature extraction is performed on the stacked inertial signal image of the target person to obtain a feature extraction result corresponding to the motion sensing information.

8. The snow sports information monitoring device according to claim 7, characterized in that: The motion sensing information includes multi-dimensional force information between the shoe and the ski, sole flexibility force information, ski acceleration information, and target person's center of gravity movement trajectory information, joint acceleration information, joint angle information, and joint position information; The multiple motion parameters include the three-dimensional force of the target person, snowboard acceleration, motion center of mass position, joint velocity, joint angular velocity, joint acceleration, joint angular acceleration and joint angle.

9. A snow sports information monitoring device, characterized in that: The snow sports information monitoring device includes a memory and a processor, the memory stores a computer program, and the processor implements the steps of the method according to any one of claims 1 to 6 when executing the computer program.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images