Portable three-dimensional sports technology comparative analysis method and device

Abstract

A multi-view analysis device collects motion videos of a tested person at different angles, is connected with a mobile device and uploads the videos to a cloud end; in the cloud, two-dimensional pixel coordinates of key points are obtained through a two-dimensional human body posture estimation module, a three-dimensional reconstruction module converts the two-dimensional pixel coordinates of the key points of the motion videos at different angles into three-dimensional space coordinates, and an optimization algorithm module adjusts the three-dimensional space coordinates; marking key frames for analysis and calculation; comparing the calculation result data of the key frame with the optimal range in the self-built model module to obtain a relative comparison result; inputting a relative comparison result into a scheme library to feed back scheme result data in real time; the data error is small, the precision of sports technology analysis data is higher, depth information is provided, accurate reference is provided, the deployment process is short in time consumption, deployment is simple, and carrying is convenient.

Description

Technical Field

[0001] The invention belongs to the technical field of sports artificial intelligence and discloses a portable three-dimensional sports technology comparison and analysis method and equipment. Background Art

[0002] With the rapid development of artificial intelligence in the field of sports analysis, national team athletes have taken the lead in applying AI technology to improve performance and training efficiency. Currently, 2D and 3D human posture estimation technology has become an important tool for analyzing athlete movements, helping coaches and athletes optimize the training process; However, in three-dimensional sports technology analysis, multiple photographic devices are usually required and deployed in a multi-angle and long-distance manner. Each camera device needs to be operated independently, and complex internal and external camera parameter calibration must be performed before each shooting. This not only increases the cost of equipment and operation, but also makes the deployment process time-consuming and difficult to promote on a large scale to grassroots sports people. Two-dimensional sports technology analysis only requires one camera device, which has the advantages of simple deployment and low cost, but two-dimensional technology can only capture planar motion information. For projects involving complex movements such as rotation, the lack of depth information leads to large data errors and it is difficult to provide accurate reference. Therefore, it is necessary to introduce a portable three-dimensional sports technology comparative analysis method and equipment. Summary of the invention

[0003] The present invention is based on the above technical problems and proposes a portable three-dimensional sports technology comparison and analysis method and equipment for sports artificial intelligence technology; it not only has a small data error and provides accurate reference, but also the deployment process is time-saving.

[0004] The present invention is achieved in that: A portable three-dimensional sports technology comparative analysis method and device, the method uses a multi-eye analysis device, a two-dimensional human posture estimation module, a three-dimensional reconstruction module, an optimization algorithm module, a self-built model module and a solution library module; characterized in that the method comprises the following steps: Step 1: The multi-view analysis device collects motion videos of the person being tested from different angles, connects to a mobile device and uploads them to the cloud; Step 2: The 2D human posture estimation module in the cloud obtains the 2D pixel coordinates of the key points. The 3D reconstruction module converts the 2D pixel coordinates of the key points in the motion video at different angles into 3D space coordinates. The optimization algorithm module adjusts the 3D space coordinates. Step 3: Mark key frames for analysis and calculation; Step 4: Compare the calculated result data of the key frame with the optimal range in the self-built model module to obtain a relative comparison result; Step 5: Input the relative comparison results into the solution library to provide real-time feedback on solution result data.

[0005] According to an achievable method of the present invention, in step 1, the multi-eye analysis device collects motion videos of the person under test at different angles, connects to a mobile device and uploads the videos to the cloud. The specific operation method includes: The user unfolds the portable, foldable multi-eye analysis device and places it at a position where each camera can see the person under test, collects the time stamp aligned video of the person under test, stores it in the storage unit, connects the mobile device to the portable, foldable multi-eye analysis device sending unit, reads the time stamp aligned video of multiple cameras in the storage unit, and sends it to the cloud; A multi-camera, used for simultaneously collecting time-stamp aligned videos at different angles; a storage unit, used for storing time-stamp aligned videos of multiple cameras; and a sending unit, used for connecting to a mobile device to send time-stamp aligned videos of multiple cameras; Mobile devices include smartphones, tablets, and desktop computers. Mobile device content is tagged with φ β , where β=0, 1, 2, ..., p; p is a positive integer representing the maximum value that β can take in the mobile device content.

[0006] According to an achievable method of the present invention, in step 2, the two-dimensional pixel coordinates of the key points are obtained in the cloud through the two-dimensional human posture estimation module, the three-dimensional reconstruction module converts the two-dimensional pixel coordinates of the key points of the motion video at different angles into three-dimensional space coordinates, and the optimization algorithm module adjusts the three-dimensional space coordinates. The operation method includes: The cloud-based 2D human posture estimation module predicts the timestamp-aligned videos of multiple cameras and outputs the 2D pixel coordinates of the key points of each frame of each video. The 3D reconstruction module includes a matching algorithm unit, a limit constraint unit, a geometric constraint unit and a scale estimation unit; the matching algorithm unit is used to match and identify objects in each video, and the limit constraint unit and the geometric constraint unit are used to convert the 2D pixel coordinates into 3D space coordinates according to the 2D pixel coordinates of key points of the video at different angles and the internal and external parameter information of the camera; the scale estimation unit is used to determine the movement distance or length of other objects; The preliminary three-dimensional space coordinates are adjusted by optimizing the algorithm module to minimize the error between the two-dimensional projection and the actually detected two-dimensional key points under different angles of motion video.

[0007] According to an achievable manner of aspects of the present invention, the method for marking key frames for analysis and calculation in step 3 includes: Calculate time, distance, speed, angle and angular velocity by marking key frames; obtain key frame time points by marking key frames, and calculate the time interval t between key frames by using the key frame time points; calculate the distance s by the pixel coordinate difference of the object during the time interval t; calculate the speed by using the distance s and the time interval t using v=s / t; Through the three-dimensional pixel coordinates xyz of the three key points, the two vectors, the dot product of the two vectors, and the length of the two vectors are calculated. The cosine value COS(θ) of the angle is calculated through the dot product of the two vectors and the length of the two vectors, and the angle θ is calculated using arccos(cos(θ)). Through the angle θ and the time interval t, the angular velocity ω is calculated using ω=∆θ / ∆t.

[0008] According to an achievable manner of aspects of the present invention, the operation method of comparing the calculation result data of the key frame with the optimal range in the self-built model module in step 4 to obtain a relative comparison result includes: Collect target reference level motion videos at different angles, mark key frames, extract time, distance, speed, angle, angular velocity and other data, use mean, median and standard deviation fusion algorithms to determine the optimal range of data, and compare it with the optimal range of data in the self-built model module to obtain relative comparison results of the data.

[0009] According to an achievable manner of the present invention, the method for operating the real-time feedback scheme result data in the scheme library relative to the comparison result in step 5 includes: The solution result data includes scores, shortcomings and training solutions, etc. The solution result data content is marked as Us, where s=0, 1, 2, ..., l; l is a positive integer, representing the maximum value of s in the solution result data content; the solution library module is used to convert detailed training instructions based on the relative comparison results; through the solution library historical solutions, the scores, shortcomings and training solutions are output according to the relative comparison results; according to the comparison results, the performance of the person being tested is quantitatively scored, including comprehensive scores, such as the overall score of the action completion and sub-item scores, such as speed, posture, balance, etc.; through animation rendering and visual technology, analysis videos and three-dimensional animations are output.

[0010] A cloud system sends the data results of the measured person to a solution library through a two-dimensional human posture estimation module and a three-dimensional reconstruction module; and performs three-dimensional sports technology through cloud computing and analysis.

[0011] Based on any of the above aspects, the beneficial effects of the present invention are: The multi-eye analysis device of the present invention collects motion videos of the person being measured at different angles, connects to a mobile device and uploads them to the cloud; the two-dimensional pixel coordinates of key points are obtained in the cloud through a two-dimensional human posture estimation module, the three-dimensional reconstruction module converts the two-dimensional pixel coordinates of key points in the motion videos at different angles into three-dimensional space coordinates, and the optimization algorithm module adjusts the three-dimensional space coordinates.

[0012] On the one hand, this method has a small data error and provides an accurate reference; through the convenient and foldable multi-eye analysis equipment, there is no need to calibrate the internal and external parameters of the camera every time, and the deployment is simple and easy to carry; compared with traditional three-dimensional sports technology analysis, it has low cost and high portability; through the three-dimensional reconstruction module, the two-dimensional pixel coordinates of the key points of the motion video at different angles are converted into three-dimensional space coordinates; at a low cost and high portability, the data analysis is more accurate than the traditional two-dimensional sports technology, has depth information, and can be used in projects involving complex movements such as rotation.

[0013] The present invention marks key frames for analysis and calculation; compares the calculation result data of the key frames with the optimal range in the self-built model module to obtain a relative comparison result; the relative comparison result is input into the solution library to feed back the solution result data in real time; on the other hand, the deployment process of this method is time-consuming. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The present invention is further described using the accompanying drawings, but the embodiments in the accompanying drawings do not constitute any limitation to the present invention. A person skilled in the art can obtain other drawings based on the following drawings without creative work.

[0015] Figure 1 What is shown is a workflow diagram of a portable three-dimensional sports technology comparative analysis method in an embodiment; Figure 2 What is shown is a workflow diagram of a portable, foldable multi-eye analysis device in an embodiment; Figure 3 What is shown is a workflow diagram of a three-dimensional reconstruction module in an embodiment; Figure 4 What is shown is a workflow diagram of marking key frames in an embodiment; Figure 5 What is shown is the workflow diagram of the self-built model module in the embodiment; Figure 6 What is shown is a workflow diagram of the solution library module in the embodiment; Figure 7 What is shown is a structural schematic diagram of a portable three-dimensional sports technology comparison and analysis system in an embodiment. Specific implementation methods

[0016] The above contents in combination with the implementation of the present invention are merely examples and explanations of the concept of the present invention. The technical personnel in the relevant technical field may make various modifications or supplements to the specific embodiments described or replace them in a similar manner. As long as they do not deviate from the concept of the invention or exceed the scope defined by the claims, they shall all fall within the protection scope of the present invention.

[0017] Combination Figure 1 A portable three-dimensional sports technology comparison and analysis method and device, the method uses a multi-eye analysis device, a two-dimensional human posture estimation module, a three-dimensional reconstruction module, an optimization algorithm module, a self-built model module and a solution library module; characterized in that the method includes the following steps: Step 1: The multi-view analysis device collects motion videos of the person being tested from different angles, connects to a mobile device and uploads them to the cloud; In a specific embodiment of the present invention, in step 1, the multi-eye analysis device collects motion videos of the person under test at different angles, connects to a mobile device and uploads the videos to the cloud. The specific operation method includes: The user unfolds the portable, foldable multi-eye analysis device and places it at a position where each camera can see the person under test, collects the time stamp alignment video of the person under test, and stores it in the storage unit. The mobile device connects to the sending unit of the portable, foldable multi-eye analysis device, reads the time stamp alignment videos of multiple cameras in the storage unit, and sends them to the cloud; the multi-eye camera is used to collect time stamp alignment videos of different angles at the same time, the storage unit is used to store the time stamp alignment videos of multiple cameras, and the sending unit is used to connect to the mobile device to send the time stamp alignment videos of multiple cameras; Mobile devices include smartphones, tablets, and desktop computers. Mobile device content is tagged with φ β , where β=0, 1, 2, ..., p; p is a positive integer, representing the maximum value that β can take in the mobile device content; It should be noted that, in a specific embodiment, for example, the user selects the shot put sport, unfolds the portable, foldable multi-eye analysis device, places it next to the person being tested, checks whether the shot put of the person being tested is within the multi-eye camera, and ensures that the person being tested exists in each camera; starts collecting, multiple cameras simultaneously start recording the shot put action, collects the video of the person being tested with the timestamp aligned, and stores it in the storage unit; the user's mobile device is connected to the portable, foldable multi-eye analysis device, and selects the shot video to upload to the cloud; It should be noted that, in another specific embodiment, for example, the user selects golf, unfolds the portable, foldable multi-eye analysis device, places it next to the person being tested, checks whether the golf swing of the person being tested is within the multi-eye camera, and ensures that the person being tested exists in each camera; starts collecting data, and multiple cameras simultaneously start recording the swing action, collects the video of the person being tested with the timestamp aligned, and stores it in the storage unit; the user's mobile device is connected to the portable, foldable multi-eye analysis device, and selects the shot video to upload to the cloud.

[0018] Step 2: The 2D human posture estimation module in the cloud obtains the 2D pixel coordinates of the key points. The 3D reconstruction module converts the 2D pixel coordinates of the key points in the motion video at different angles into 3D space coordinates. The optimization algorithm module adjusts the 3D space coordinates. In a specific embodiment of the present invention, the two-dimensional pixel coordinates of the key points are obtained in the cloud through the two-dimensional human posture estimation module in step 2, the three-dimensional reconstruction module converts the two-dimensional pixel coordinates of the key points of the motion video at different angles into three-dimensional space coordinates, and the optimization algorithm module adjusts the three-dimensional space coordinates. The operation method includes: The cloud-based 2D human posture estimation module predicts the timestamp-aligned videos of multiple cameras and outputs the 2D pixel coordinates of the key points of each frame of each video. The 3D reconstruction module includes a matching algorithm unit, a limit constraint unit, a geometric constraint unit and a scale estimation unit; the matching algorithm unit is used to match and identify objects in each video, and the limit constraint unit and the geometric constraint unit are used to convert the 2D pixel coordinates into 3D space coordinates according to the 2D pixel coordinates of key points of the video at different angles and the internal and external parameter information of the camera; the scale estimation unit is used to determine the movement distance or length of other objects; It should be noted that, in a specific embodiment, for example, a portable, foldable binocular analysis device, respectively photographs a certain three-dimensional point P in the same scene, and obtains two-dimensional projection points P1 and P2 on the image planes of two cameras. Given a point P1 in one perspective, its corresponding point P2 in another perspective must be on an epipolar line, thereby limiting the search space in perspective 2 to this epipolar line. The intrinsic and extrinsic parameters of the camera are known, so the three-dimensional position of the point is restored by triangulation, and the geometric relationship between the two cameras is used to back-project the corresponding two-dimensional key point coordinates into the three-dimensional space to solve the three-dimensional coordinates of the key point; The preliminary 3D space coordinates are adjusted by optimizing the algorithm module to minimize the error between the 2D projection and the actually detected 2D key points under different angles of motion video. It should be noted that, in a specific embodiment, for example, the preliminary three-dimensional reconstruction obtains the initial position P0 of each key point in the three-dimensional space through triangulation or other geometric methods; for each key point P0, projection is performed in each camera perspective, and the difference between the projection point and the actual detected two-dimensional key point is calculated to construct an error function; a nonlinear least squares optimization algorithm is applied to adjust the three-dimensional coordinates of the key point through iterative optimization to minimize the total error; this process may involve dozens to hundreds of iterations until the error converges below a certain threshold; after the optimization process is completed, the refined three-dimensional coordinates are obtained.

[0019] Step 3: Mark key frames for analysis and calculation; In a specific embodiment of the present invention, the method for marking key frames for analysis and calculation in step 3 includes: Calculate time, distance, speed, angle and angular velocity by marking key frames; obtain key frame time points by marking key frames, and calculate the time interval t between key frames by using the key frame time points; calculate the distance s by the pixel coordinate difference of the object during the time interval t; calculate the speed by using the distance s and the time interval t using v=s / t; Through the three-dimensional pixel coordinates xyz of the three key points, the two vectors, the dot product of the two vectors, and the length of the two vectors are calculated. The cosine value COS(θ) of the angle is calculated through the dot product of the two vectors and the length of the two vectors, and the angle θ is calculated using arccos(cos(θ)). Through the angle θ and the time interval t, the angular velocity ω is calculated using ω=∆θ / ∆t.

[0020] Step 4: Compare the calculated result data of the key frame with the optimal range in the self-built model module to obtain a relative comparison result; In a specific embodiment of the present invention, the operation method of comparing the calculation result data of the key frame with the optimal range in the self-built model module to obtain a relative comparison result in step 4 includes: Collect target reference level motion videos at different angles, mark key frames, extract time, distance, speed, angle, angular velocity and other data, use mean, median and standard deviation fusion algorithms to determine the optimal range of data, and compare it with the optimal range of data in the self-built model module to obtain relative comparison results of the data.

[0021] Step 5: Input the relative comparison results into the solution library to provide real-time feedback of solution result data; In a specific embodiment of the present invention, the operation method for inputting the relative comparison result into the solution library in step 5 to provide real-time feedback of solution result data includes: The solution result data includes scores, shortcomings and training solutions, etc. The solution result data content is marked as Us, where s=0, 1, 2, ..., l; l is a positive integer, representing the maximum value of s in the solution result data content; the solution library module is used to convert detailed training instructions based on the relative comparison results; through the solution library historical solutions, the scores, shortcomings and training solutions are output according to the relative comparison results; according to the comparison results, the performance of the person being tested is quantitatively scored, including comprehensive scores, such as the overall score of the action completion and sub-item scores, such as speed, posture, balance, etc.; through animation rendering and visual technology, analysis videos and three-dimensional animations are output.

[0022] A cloud system sends the data results of the measured person to a solution library through a two-dimensional human posture estimation module and a three-dimensional reconstruction module; and performs three-dimensional sports technology through cloud computing and analysis.

[0023] The multi-eye analysis device of the present invention collects motion videos of the person under test at different angles, connects to a mobile device and uploads the videos to the cloud; the two-dimensional pixel coordinates of key points are obtained in the cloud through a two-dimensional human posture estimation module, the three-dimensional reconstruction module converts the two-dimensional pixel coordinates of key points in the motion videos at different angles into three-dimensional space coordinates, and the optimization algorithm module adjusts the three-dimensional space coordinates; on the one hand, this method has a small data error and provides an accurate reference; key frames are marked for analysis and calculation; the calculation result data of the key frames are compared with the optimal range in the self-built model module to obtain a relative comparison result; the relative comparison result is input into a solution library to feed back solution result data in real time; on the other hand, this method has a short deployment process.

[0024] The above contents are merely examples and explanations of the structure of the present invention. The technicians in this technical field may make various modifications or additions to the specific embodiments described or replace them in a similar manner. As long as they do not deviate from the structure of the invention or exceed the scope defined by the claims, they should all fall within the protection scope of the present invention.

Claims

1. A portable three-dimensional sports technology comparative analysis method and equipment, which uses a multi-eye analysis device, a two-dimensional human posture estimation module, a three-dimensional reconstruction module, an optimization algorithm module, a self-built model module and a solution library module; It is characterized in that The method comprises the following steps: Step 1: The multi-view analysis device collects motion videos of the person being tested from different angles, connects to a mobile device and uploads them to the cloud; Step 2: The 2D pixel coordinates of key points are obtained through the 2D human posture estimation module in the cloud. The 3D reconstruction module converts the 2D pixel coordinates of key points in motion videos at different angles into 3D space coordinates. The optimization algorithm module adjusts the 3D space coordinates. Step 3: Mark key frames for analysis and calculation; Step 4: Compare the calculated result data of the key frame with the optimal range in the self-built model module to obtain a relative comparison result; Step 5: Input the relative comparison results into the solution library to provide real-time feedback on solution result data.

2. A portable three-dimensional sports technology comparative analysis method and device according to claim 1, Features: In step 1, the multi-view analysis device collects motion videos of the person under test at different angles, connects to a mobile device and uploads them to the cloud. The specific operation method includes: The user unfolds the portable, foldable multi-eye analysis device and places it at a position where each camera can see the person under test, collects the time stamp aligned video of the person under test, stores it in the storage unit, connects the mobile device to the portable, foldable multi-eye analysis device sending unit, reads the time stamp aligned video of multiple cameras in the storage unit, and sends it to the cloud; Mobile devices include smartphones, tablets, and desktop computers. Mobile device content is tagged with φ β , where β=0, 1, 2, ..., p; p is a positive integer representing the maximum value that β can take in the mobile device content.

3. A portable three-dimensional sports technology comparative analysis method and device according to claim 1, Features: In step 2, the two-dimensional pixel coordinates of key points are obtained in the cloud through the two-dimensional human posture estimation module, the three-dimensional reconstruction module converts the two-dimensional pixel coordinates of key points of the motion video at different angles into three-dimensional space coordinates, and the optimization algorithm module adjusts the three-dimensional space coordinates. The operation method includes: The cloud-based 2D human posture estimation module predicts the timestamp-aligned videos of multiple cameras and outputs the 2D pixel coordinates of the key points of each frame of each video. The 3D reconstruction module includes a matching algorithm unit, a limit constraint unit, a geometric constraint unit and a scale estimation unit; the matching algorithm unit is used to match and identify objects in each video, and the limit constraint unit and the geometric constraint unit are used to convert the 2D pixel coordinates into 3D space coordinates according to the 2D pixel coordinates of key points of the video at different angles and the internal and external parameter information of the camera; the scale estimation unit is used to determine the movement distance or length of other objects; The preliminary three-dimensional space coordinates are adjusted by optimizing the algorithm module to minimize the error between the two-dimensional projection and the actually detected two-dimensional key points under different angles of motion video.

4. A portable three-dimensional sports technology comparative analysis method and device according to claim 1, Features: The method for marking key frames for analysis and calculation in step 3 includes: Calculate time, distance, speed, angle and angular velocity by marking key frames; obtain key frame time points by marking key frames, and calculate the time interval t between key frames by using the key frame time points; calculate the distance s by the pixel coordinate difference of the object during the time interval t; calculate the speed by using the distance s and the time interval t using v=s / t; Through the three-dimensional pixel coordinates xyz of the three key points, the two vectors, the dot product of the two vectors, and the length of the two vectors are calculated. The cosine value COS(θ) of the angle is calculated through the dot product of the two vectors and the length of the two vectors, and the angle θ is calculated using arccos(cos(θ)). Through the angle θ and the time interval t, the angular velocity ω is calculated using ω=∆θ / ∆t.

5. A portable three-dimensional sports technology comparative analysis method and device according to claim 1, Features: The operation method of comparing the calculation result data of the key frame with the optimal range in the self-built model module in step 4 to obtain a relative comparison result includes: Collect target reference level motion videos at different angles, mark key frames, extract time, distance, speed, angle, angular velocity and other data, use mean, median and standard deviation fusion algorithms to determine the optimal range of data, and compare it with the optimal range of data in the self-built model module to obtain relative comparison results of the data.

6. A portable three-dimensional sports technology comparative analysis method and device according to claim 1, Features: The operation method for inputting the relative comparison result into the solution library in step 5 to provide real-time feedback of solution result data includes: The solution result data includes scores, shortcomings and training solutions, etc. The solution result data content is marked as Us, where s=0, 1, 2, ..., l; l is a positive integer, indicating the maximum value of s in the solution result data content; through the historical solutions in the solution library, the scores, shortcomings and training solutions are output according to the relative comparison results; according to the comparison results, the performance of the tested person is quantitatively scored, including comprehensive scores, such as the overall score of the action completion and sub-item scores, such as speed, posture, balance, etc.; through animation rendering and visual technology, analysis videos and 3D animations are output.

7. A cloud system, Features: The data results of the measured person are sent to the solution library through the two-dimensional human posture estimation module and the three-dimensional reconstruction module; the three-dimensional sports technology is calculated and analyzed through cloud computing to execute the portable three-dimensional sports technology comparison and analysis method and device described in any one of claims 1-6.

Images