A snake-shaped running training and assessment system based on visual recognition technology

Abstract

The present invention discloses a serpentine running training and assessment system based on visual recognition technology, belonging to the field of visual recognition technology, and solves the problem that during the process of serpentine running training, training assessment is usually carried out manually, and perfect training assessment can only be carried out in the presence of a coach or professional athlete, otherwise there may be a situation of ineffective training. It includes a multi-modal visual acquisition module, a rod recognition and dynamic calibration module, an identity and action joint recognition module, a motion trajectory tracking module, a three-dimensional rod hitting detection module, and a score calculation and compensation module. The system of the present invention realizes the full process automation of serpentine running training and assessment, covering athlete identity recognition, preparation posture detection, real-time motion tracking, rod hitting detection and automatic score calculation and compensation, without manual intervention throughout the process, which significantly improves the training and assessment efficiency, reduces the interference of human factors, and ensures fairness and objectivity.

Description

Technical Field

[0001] The present invention relates to the technical field of visual recognition, and in particular to a serpentine running training and assessment system based on visual recognition technology. Background Art

[0002] Serpentine running training is a highly practical and targeted sports training program, widely used in team sports such as football, basketball, and rugby, as well as in fields such as track and field and military training. It mainly simulates the trajectory of a snake, allowing the trainer to continuously change directions during rapid movement, thereby effectively improving the body's flexibility, coordination, and reaction ability. During the training process, multiple markers, such as cone barrels, are usually set up. The trainer needs to shuttle between these markers at the fastest speed while maintaining the balance and stability of the body. This training method can fully mobilize multiple muscle groups of the body, especially the leg, waist, and core muscle groups, enhancing muscle strength and endurance. In addition, serpentine running can also exercise the athlete's visual attention and spatial perception ability, because when changing directions rapidly, the athlete needs to constantly pay attention to the position of the markers and the surrounding environment to avoid collisions. For young athletes, serpentine running training helps to cultivate their sports interests and physical fitness foundation; for professional athletes, it is an important means to improve their competitive level and optimize technical movements. By persevering in serpentine running training for a long time, not only can the athlete's breakthrough, defense, and receiving abilities in the game be improved, but also sports injuries can be effectively prevented, because it can enhance the flexibility of joints and the strength of ligaments. In short, serpentine running training is a simple and effective training method, worthy of wide promotion and application in various sports trainings.

[0003] However, during the process of serpentine running training, the training and assessment are usually carried out manually, and it is necessary to have a coach or professional athlete present to conduct a perfect training and assessment. Otherwise, ineffective training may occur.

[0004] Therefore, a serpentine running training and assessment system based on visual recognition technology is proposed to solve or alleviate the above problems. Summary of the Invention

[0005] The purpose of the present invention is to solve the deficiencies existing in the prior art, and a serpentine running training and assessment system based on visual recognition technology is proposed.

[0006] To achieve the above purpose, the present invention adopts the following technical solutions:

[0007] A serpentine running training and assessment system based on visual recognition technology, including a multi-modal visual acquisition module, a rod recognition and dynamic calibration module, an identity and action joint recognition module, a motion trajectory tracking module, a three-dimensional rod collision detection module, and a score calculation and compensation module. The multi-modal visual acquisition module is used to collect RGB images, depth information, and environmental point cloud data of athletes, and achieve multi-view spatio-temporal synchronization through the IEEE 1588v2 protocol. The rod recognition and dynamic calibration module is used to integrate the YOLOv8 model and the adaptive projection correction algorithm, complete the three-dimensional coordinate positioning of 1-7 rods by fusing lidar point cloud data, and establish a motion plane coordinate system. The identity and action joint recognition module is used to complete athlete identity authentication and preparation posture compliance detection through multi-modal biometric fusion. The motion trajectory tracking module is based on the DeepSORT algorithm and the Hungarian path matching model, calculates the three-dimensional coordinates of 14 key points of the human body in real time, constructs a serpentine running motion trajectory, and detects missed rod / out-of-order violation behaviors. The three-dimensional rod collision detection module uses a rigid body collision model and a continuous collision detection algorithm to determine a rod collision event by calculating the Euclidean distance between the ankle / knee / hip key points and the safe area of the rod. The score calculation and compensation module realizes μs-level start and stop time synchronization based on the PTPv2 protocol, combines speed projection compensation, and outputs the final assessment score.

[0008] Preferably, the multi-modal visual acquisition module is used to collect RGB images, depth information, and environmental point cloud data of athletes, and achieve multi-view spatio-temporal synchronization through the IEEE 1588v2 protocol, including the following steps.

[0009] Use a high-resolution color camera to continuously obtain a sequence of color images of athletes during serpentine running at a set frame rate.

[0010] Adopt a depth sensor to work synchronously with the RGB camera to measure the distance between objects in the scene and the camera in real time and generate a depth map.

[0011] Use equipment such as lidar to scan the training ground to obtain point cloud data containing a large amount of spatial point position information.

[0012] Achieve multi-view spatio-temporal synchronization through the IEEE 1588v2 protocol.

[0013] Preferably, the rod recognition and dynamic calibration module is used to integrate the YOLOv8 model and the adaptive projection correction algorithm, complete the three-dimensional coordinate positioning of 1-7 rods by fusing lidar point cloud data, and establish a motion plane coordinate system, including the following steps.

[0014] Collect a large amount of image data containing rods and train the original YOLOv8 model.

[0015] The collected RGB image is input into the trained YOLOv8 model. Through structures such as convolutional layers and feature pyramids, the model extracts features from the image and outputs the position information of the rods, including the category, confidence level of the rods, and the coordinates of the bounding box.

[0016] Preprocess the point cloud data collected by the lidar, including operations such as denoising and filtering.

[0017] By obtaining the extrinsic parameters of the camera and the lidar, project the point cloud data into the image coordinate system. The extrinsic parameters include the rotation matrix and the translation vector.

[0018] Combining the rod bounding box detected by the YOLOv8 model and the registered point cloud data, filter out the point cloud points located within the rod bounding box.

[0019] By taking multiple images containing calibration marks, use the Zhang Zhengyou calibration method to obtain the intrinsic and extrinsic parameters of the camera.

[0020] By continuously monitoring the changes in the site, update the projection matrix H using the following formula. , where is the calibration increment, is the historical projection matrix, is the real-time projection matrix, which can be calculated by optimization algorithms such as the least squares method. When site changes are detected, collect multiple sets of new image data and corresponding known three-dimensional coordinate data, and construct an error equation. , where e is the projection error, is the pixel coordinate in the image, is the corresponding world coordinate. By minimizing the sum of squared errors: Solve to obtain , realizing the adaptive calibration of the projection matrix H.

[0021] Utilize the point cloud information of the site in the lidar point cloud data to fit out the motion plane. Based on the fitted motion plane, establish the motion plane coordinate system.

[0022] Preferably, the identity and action joint recognition module is used to complete athlete identity authentication and preparation posture compliance detection through multi-modal biometric fusion, including the following steps.

[0023] Use a high-definition camera to obtain the facial image of the athlete. Through the face detection algorithm, locate the facial key points and extract the geometric features and texture features of the face. The geometric features include the positions, distances, and proportional relationships of the eyes, nose, and mouth, and the texture features involve information such as the texture details and color distribution of the skin.

[0024] Obtain the body depth information of the athlete through a depth sensor, combine with 3D reconstruction technology to obtain the spatial positions and pose information of various parts of the athlete's body, use a joint point detection algorithm to locate the key joint points of the athlete's body, and the key joint points include shoulders, elbows, wrists, hips, knees, and ankles, and extract features such as the positions, angles, and relative position relationships of the joint points;

[0025] Fuse the face features and body pose features to form a comprehensive biometric vector, and use the feature-level fusion method to splice or weighted fuse the two feature vectors;

[0026] Match the fused feature vector with the feature vectors in the pre-stored athlete feature library, and use a similarity measurement method to calculate the similarity between the feature vector to be recognized and the feature vectors in the library;

[0027] According to the similarity measurement result, select the athlete with the highest similarity as the recognition result;

[0028] Extract the key features of the athlete's ready pose from the collected depth information and joint point data;

[0029] Compare the extracted pose features with the preset standard ready pose feature template to determine whether the athlete's ready pose meets the specifications.

[0030] Preferably, the motion trajectory tracking module is based on the DeepSORT algorithm and the Hungarian path matching model, and calculates the three-dimensional coordinates of 14 key points of the human body in real time, constructs a snake-shaped running motion trajectory, and detects missed pole / out-of-order violation behaviors, including the following steps,

[0031] In the DeepSORT algorithm, the state of each tracking target is maintained by a Kalman filter, including information such as the position, speed, and acceleration of the target;

[0032] In each frame, use the Kalman filter to predict the position of each tracking target in the next frame;

[0033] Calculate the correlation function value between the detection target and the tracking trajectory;

[0034] Based on the calculated correlation cost matrix, use the Hungarian algorithm for optimal matching to assign the detection target to the most likely tracking trajectory;

[0035] Represent the motion trajectory of the athlete as a sequence of three-dimensional coordinates of a series of key points. In the case of multiple possible paths, use the Hungarian algorithm to match and select different paths;

[0036] Combine the depth information obtained by the depth sensor with the two-dimensional image coordinates of the key points to calculate the three-dimensional coordinates of the key points, and fuse the three-dimensional coordinates calculated from different perspectives;

[0037] In each frame, update the athlete's motion trajectory according to the matching result and the calculated three-dimensional key point coordinates, and use a smoothing algorithm to smooth the trajectory;

[0038] According to the rules of the serpentine run, define the correct passing order of the poles, compare and analyze the athlete's motion trajectory with the positions of the poles, calculate the distances between the trajectory points and each pole in three-dimensional space, determine whether the athlete passes the poles in the specified order, and judge whether there are any missed poles or out-of-order behaviors according to the relationship between the trajectory and the poles.

[0039] Preferably, the three-dimensional pole collision detection module uses a rigid body collision model and a continuous collision detection algorithm to determine the pole collision event by calculating the Euclidean distance between the ankle / knee / hip key points and the safety area of the pole, including the following steps,

[0040] Regard the ankle, knee, and hip joints of the athlete as the collision points in the rigid body model, and the positions of these points in three-dimensional space are determined by the three-dimensional coordinates calculated by the motion trajectory tracking module;

[0041] Set a safety area for each pole, usually a cylinder or other suitable shape area centered on the pole with a certain radius;

[0042] In the dynamic process of the serpentine run, continuously obtain the three-dimensional coordinate information of the human key points in each frame of image to form time series data;

[0043] Define a collision detection function to judge whether the key point enters the safety area of the pole. When the value of the collision detection function changes from 0 to 1, it is determined that a pole collision event occurs;

[0044] In the three-dimensional pole collision detection, calculate the Euclidean distance between each key point and the center of each pole;

[0045] Comprehensively consider the collision detection results of multiple key points to finally determine the pole collision event. At a certain moment, if at least one key point collides with the safety area of the pole, it is determined that a pole collision occurs.

[0046] Preferably, the performance calculation and compensation module realizes μs-level start and stop time synchronization based on the PTPv2 protocol, combines speed projection compensation, and outputs the final assessment result, including the following steps,

[0047] Realize μs-level start and stop time synchronization based on the PTPv2 protocol;

[0048] When the athlete starts running, the system determines whether the athlete has crossed the initial position of the first pole by detecting the change in the position of the athlete's feet, and uses this as the basis for recording the starting time. The starting time is determined by the device timestamp after time synchronization. When the athlete completes the slalom run and returns, the end time is determined by detecting the moment of crossing the finish line. The end time is determined by the device timestamp after time synchronization;

[0049] The slalom run result is the time difference between the starting time and the end time;

[0050] During the slalom run, the system calculates the athlete's speed in real time by obtaining the position information of the athlete at different time points and calculating the instantaneous speed;

[0051] Project the athlete's speed onto the path of the slalom run to consider the curve characteristics of the path;

[0052] Due to the delay of the detection system or the small error of time synchronization, the actually recorded time may deviate from the real time. The result is corrected through speed projection compensation;

[0053] Integrate the start and end times calculated by time synchronization and the result after speed projection compensation to obtain the final assessment result;

[0054] Feed back the final result to the athlete in time through voice or display screen, etc., and store the result data in the database for subsequent statistics and analysis.

[0055] The present invention has the following beneficial effects:

[0056] The system of the present invention realizes the full-process automation of slalom run training assessment. From athlete identity recognition, preparation posture detection, to real-time tracking and pole hitting detection during the movement process, and then to the automatic calculation and compensation of the final result, the whole process does not require manual intervention. This not only greatly improves the efficiency of training assessment, but also reduces the interference of human factors on the assessment result, ensuring the fairness and objectivity of the assessment. For example, in traditional assessment, the coach needs to manually record the time, while this system realizes μs-level start and end time synchronization through the PTPv2 protocol, combined with speed projection compensation, can accurately record the result automatically, avoiding the error of manual timing;

[0057] The system adopts a multi-modal visual acquisition module, which integrates RGB images, depth information, and environmental point cloud data, and realizes multi-view spatio-temporal synchronization through the IEEE 1588v2 protocol. This multi-dimensional data acquisition method can comprehensively and accurately capture various information of athletes during the serpentine run. Each module uses advanced algorithms to process and analyze the data. For example, the YOLOv8 model is used for pole recognition, the adaptive projection correction algorithm is used for dynamic calibration, and the DeepSORT algorithm and the Hungarian path matching model are used for motion trajectory tracking, etc., ensuring the accuracy and reliability of data processing. Taking pole recognition as an example, by fusing lidar point cloud data, the system can accurately complete the three-dimensional coordinate positioning of 1 - 7 poles, establish a motion plane coordinate system, and provide an accurate spatial reference for subsequent motion trajectory analysis and pole collision detection;

[0058] This system can not only accurately calculate the serpentine run results of athletes, but also comprehensively evaluate and analyze the training process. Through the three-dimensional pole collision detection module, pole collision events can be judged in real time to help athletes timely discover and correct problems in their movements; through the motion trajectory tracking module, missed pole / out-of-order violation behaviors can be detected, and detailed motion trajectory data can be provided for athletes to understand their path selection and direction changes during the serpentine run; at the same time, the identity and action joint recognition module can also perform compliance detection on the preparatory postures of athletes to ensure that they are in the correct posture before starting, which helps to improve the training effect and prevent sports injuries. These comprehensive evaluation information provides scientific training feedback for athletes and coaches, helping to formulate more reasonable training plans and improve training methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention, and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0060] Figure 1 It is a structural block diagram of the present invention.

[0061] 1. Multi-modal visual acquisition module; 2. Pole recognition and dynamic calibration module; 3. Identity and action joint recognition module; 4. Motion trajectory tracking module; 5. Three-dimensional pole collision detection module; 6. Result calculation and compensation module. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0062] To make the objectives, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Apparently, the described embodiments are some, but not all, of the embodiments of the present invention. The components of the embodiments of the present invention generally described and illustrated in the figures herein can be arranged and designed in a variety of different configurations.

[0063] Therefore, the detailed description of the embodiments of the present invention provided in the drawings below is not intended to limit the scope of the claimed invention, but is merely representative of selected embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the scope of protection of the present invention.

[0064] It should be noted that like reference numerals and letters denote like items in the following figures, and thus, once an item is defined in one figure, further definition and explanation thereof are not required in subsequent figures.

[0065] In the description of the present invention, it should be understood that the orientation or positional relationship indicated by the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. is based on the orientation or positional relationship shown in the drawings, or the orientation or positional relationship in which the product of the invention is customarily placed during use, or the orientation or positional relationship commonly understood by those skilled in the art. It is only for the convenience of describing the present invention and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus should not be construed as a limitation of the present invention.

[0066] In addition, the terms "first", "second", "third", etc. are only used for descriptive distinction and should not be construed as indicating or implying relative importance.

[0067] In the description of the present invention, it should also be noted that unless otherwise clearly defined and limited, the terms "set", "installed", "connected", "coupled" should be understood in a broad sense. For example, it may be a fixed connection, a detachable connection, or an integral connection; it may be a mechanical connection or an electrical connection; it may be directly connected or indirectly connected through an intermediate medium, and it may be the internal communication of two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0068] A serpentine running training and assessment system based on visual recognition technology, as Figure 1As shown, it includes a multi-modal visual acquisition module 1, a rod recognition and dynamic calibration module 2, an identity and action joint recognition module 3, a motion trajectory tracking module 4, a three-dimensional rod collision detection module 5, and a score calculation and compensation module 6. The multi-modal visual acquisition module 1 is used to collect the RGB images, depth information, and environmental point cloud data of the athlete, and achieve multi-view spatio-temporal synchronization through the IEEE 1588v2 protocol. The rod recognition and dynamic calibration module 2 is used to integrate the YOLOv8 model and the adaptive projection correction algorithm, and complete the three-dimensional coordinate positioning of 1-7 rods by fusing the lidar point cloud data, and establish a motion plane coordinate system. The identity and action joint recognition module 3 is used to complete the athlete identity authentication and the compliance detection of the ready posture through multi-modal biometric fusion. The motion trajectory tracking module 4 is based on the DeepSORT algorithm and the Hungarian path matching model, and calculates the three-dimensional coordinates of 14 key points of the human body in real time, constructs a serpentine running motion trajectory, and detects the violations of missing rods / out-of-order. The three-dimensional rod collision detection module 5 uses a rigid body collision model and a continuous collision detection algorithm to determine the rod collision event by calculating the Euclidean distance between the ankle / knee / hip key points and the safe area of the rod. The score calculation and compensation module 6 achieves μs-level start and stop time synchronization based on the PTPv2 protocol, and combines speed projection compensation to output the final assessment score.

[0069] Traditional serpentine running training and assessment rely on manual labor and require coaches or professional athletes to be present. Otherwise, ineffective training may occur. However, this system breaks through this limitation through an automated process, enabling athletes to independently conduct training and assessment and check their training results at any time. For example, athletes can take the assessment at any time, and the system automatically records the scores and detects movement problems without waiting for the coach's arrangement and participation, greatly improving the flexibility and autonomy of training, and helping athletes improve their scores more efficiently. This system covers all key links of serpentine running training and assessment, realizing full-process automation. This comprehensiveness ensures that the system can independently complete the entire assessment process from athlete identification to score output without manual intervention, greatly improving the efficiency and fairness of training and assessment. Through multi-modal visual acquisition, the system obtains rich data types, providing a basis for subsequent precise analysis; rod recognition and dynamic calibration ensure the accurate acquisition of site information; identity and action joint recognition not only identify the athlete's identity but also detect whether the ready posture is compliant, guaranteeing the training quality from the source; motion trajectory tracking monitors the athlete's dynamics in real time and constructs a complete motion trajectory, providing a basis for score calculation and motion analysis; three-dimensional rod collision detection accurately judges the rod collision event and timely discovers movement mistakes; the score calculation and compensation module 6 combines time synchronization and speed compensation to output accurate assessment scores, comprehensively reflecting the actual performance of the athlete.

[0070] Preferably, the multi-modal vision acquisition module 1 is used to collect the RGB images, depth information and environmental point cloud data of the athlete, and achieve multi-view spatio-temporal synchronization through the IEEE 1588v2 protocol, including the following steps,

[0071] Use a high-resolution color camera to continuously obtain a sequence of color images of the athlete during the slalom run at a set frame rate;

[0072] Adopt a depth sensor to work synchronously with the RGB camera to measure the distance between the object and the camera in the scene in real time and generate a depth map;

[0073] With the help of equipment such as lidar, scan the training ground to obtain point cloud data containing a large amount of spatial point position information;

[0074] Achieve multi-view spatio-temporal synchronization through the IEEE 1588v2 protocol.

[0075] During slalom run training, the movements of the athlete are complex and fast, and a single data type is difficult to comprehensively capture their motion state. Relying solely on RGB images cannot accurately obtain depth and three-dimensional position information, which may lead to misjudgments of events such as hitting the pole; only having depth information or point cloud data lacks descriptions of the athlete's appearance and detailed movements. Therefore, fusing multi-modal data and achieving spatio-temporal synchronization can make up for the deficiencies of single data, comprehensively and accurately record the athlete's movement process, provide a reliable data basis for subsequent analysis and evaluation, and thus more scientifically guide training and assessment. The multi-modal vision acquisition module 1 fuses RGB images, depth information and environmental point cloud data, and achieves multi-view spatio-temporal synchronization through the IEEE 1588v2 protocol. This fusion method enriches the data dimension, can comprehensively capture the appearance, depth and spatial position information of the athlete during the slalom run, and provides multi-angle data support for subsequent precise analysis. Spatio-temporal synchronization ensures the consistency and accuracy of multi-source data, avoids data deviation caused by asynchronous, and improves the reliability and analysis accuracy of the entire system. For example, when detecting that the athlete hits the pole, the RGB image can provide a visual basis, the depth information can determine the precise distance between the athlete and the pole, and the point cloud data can restore the three-dimensional scene. The multi-source synchronized data work together to make the pole hitting detection more accurate.

[0076] Preferably, the pole recognition and dynamic calibration module 2 is used to integrate the YOLOv8 model and the adaptive projection correction algorithm, and complete the three-dimensional coordinate positioning of 1-7 poles by fusing lidar point cloud data, and establish a motion plane coordinate system, including the following steps,

[0077] Collect a large amount of image data containing poles and train the original YOLOv8 model;

[0078] Input the collected RGB image into the trained YOLOv8 model. The model extracts features from the image through structures such as convolutional layers and feature pyramids, and outputs the position information of the rods, including the category, confidence level of the rods, and the coordinates of the bounding boxes.

[0079] Preprocess the point cloud data collected by the lidar, including operations such as denoising and filtering.

[0080] By obtaining the extrinsic parameters of the camera and the lidar, project the point cloud data into the image coordinate system. The extrinsic parameters include the rotation matrix and the translation vector.

[0081] Combine the rod bounding boxes detected by the YOLOv8 model and the registered point cloud data to filter out the point cloud points located within the rod bounding boxes.

[0082] By taking multiple images containing calibration markers, use the Zhang Zhengyou calibration method to obtain the intrinsic and extrinsic parameters of the camera.

[0083] By monitoring the changes in the site in real time, update the projection matrix H using the following formula: , where is the calibration increment, which can be calculated by optimization algorithms such as the least squares method. When the site change is detected, collect multiple groups of new image data and corresponding known three-dimensional coordinate data, and construct an error equation: , where e is the projection error. is the pixel coordinate in the image. is the corresponding world coordinate. By minimizing the sum of squared errors: Solve to obtain , to achieve adaptive calibration of the projection matrix H.

[0084] Use the point cloud information of the site in the lidar point cloud data to fit out the motion plane. Based on the fitted motion plane, establish a motion plane coordinate system.

[0085] In the serpentine running training, the position of the poles is an important basis for judging the athlete's movements and performance. If the pole position is not accurately identified or the coordinate system is not stably established, it will directly affect the judgment of the athlete's pole-rounding situation and the fairness of the performance. For example, a slight deviation in the pole position may lead to misjudgment of the athlete hitting or missing a pole. Through the YOLOv8 model and the adaptive projection correction algorithm, the system can accurately locate the poles in real time and adapt to minor changes in the venue, ensuring the rigor and accuracy of the assessment, providing reliable data support for subsequent modules, and thus comprehensively evaluating the athlete's performance. The pole recognition and dynamic calibration module 2 integrates the YOLOv8 model and the adaptive projection correction algorithm, fuses lidar point cloud data, and realizes the accurate three-dimensional coordinate positioning of 1-7 poles and the establishment of the moving plane coordinate system. The YOLOv8 model improves the accuracy and speed of pole recognition and can quickly and stably detect the pole position in complex scenarios; the adaptive projection correction algorithm adjusts the projection matrix in real time according to the venue changes, ensuring the accuracy and reliability of the coordinate positioning. This accurate positioning provides a key venue reference for subsequent motion trajectory analysis, pole-hitting detection, and performance calculation, ensuring the accuracy of the system's judgment of the athlete's movements and paths.

[0086] Preferably, the identity and action joint recognition module 3 is used to complete the athlete identity authentication and preparation posture compliance detection through multi-modal biometric fusion, including the following steps:

[0087] Use a high-definition camera to obtain the athlete's facial image, locate the facial key points through the face detection algorithm, and extract the geometric features and texture features of the face. The geometric features include the positions, distances, and proportional relationships of the eyes, nose, and mouth, and the texture features involve information such as the texture details and color distribution of the skin;

[0088] Obtain the athlete's body depth information through a depth sensor, combine it with 3D reconstruction technology to obtain the spatial positions and posture information of various parts of the athlete's body, use the joint point detection algorithm to locate the key joint points of the athlete's body, and the key joint points include the shoulders, elbows, wrists, hips, knees, and ankles, and extract features such as the positions, angles, and relative position relationships of the joint points;

[0089] Fuse the face features and body posture features to form a comprehensive biometric vector, and use the feature-level fusion method to splice or weighted fuse the two feature vectors;

[0090] Match the fused feature vector with the feature vectors in the pre-stored athlete feature library, use the similarity measurement method to calculate the similarity between the to-be-recognized feature vector and the feature vectors in the library;

[0091] According to the similarity measurement result, select the athlete with the highest similarity as the recognition result;

[0092] Extract key features of the athlete's preparatory posture from the collected depth information and joint point data;

[0093] Compare the extracted posture features with a preset standard preparatory posture feature template to determine whether the athlete's preparatory posture meets the specifications.

[0094] In training assessments, accurately identifying the athlete's identity is a prerequisite for ensuring the accuracy of results and the pertinence of training plans. If the identity is misidentified, it will lead to chaotic result records and affect the evaluation of training effects. At the same time, the preparatory posture directly affects the starting quality of the slalom run and the standardization of subsequent movements. If the preparatory posture is incorrect, it will not only affect the results but also increase the risk of sports injuries. Through multi-modal biometric fusion and posture detection, the system can accurately identify identities and standardize preparatory movements, providing a good start for subsequent training and assessments, helping athletes better demonstrate their levels and improve their results. The identity and movement joint recognition module 3 has completed the athlete identity authentication and preparatory posture compliance detection through multi-modal biometric fusion. The fusion of face features and body posture features improves the accuracy and reliability of identity recognition, avoiding errors that may occur in single-feature recognition. The preparatory posture compliance detection ensures that the athlete is in the correct posture before starting, helping to improve training effects and prevent sports injuries. This joint recognition method not only enhances the security of the system but also standardizes the athletes from the initial stage of training, ensuring the scientificity and effectiveness of training and assessments.

[0095] Preferably, the motion trajectory tracking module 4 is based on the DeepSORT algorithm and the Hungarian path matching model, calculates the three-dimensional coordinates of 14 key points of the human body in real time, constructs the slalom run motion trajectory, and detects pole-missing / out-of-order violation behaviors, including the following steps:

[0096] In the DeepSORT algorithm, the state of each tracking target is maintained by a Kalman filter, including information such as the target's position, speed, and acceleration;

[0097] In each frame, use the Kalman filter to predict the position of each tracking target in the next frame;

[0098] Calculate the correlation function value between the detection target and the tracking trajectory;

[0099] Based on the calculated correlation cost matrix, use the Hungarian algorithm for optimal matching to assign the detection target to the most likely tracking trajectory;

[0100] Represent the athlete's motion trajectory as a sequence of three-dimensional coordinates of a series of key points. In the case of multiple possible paths, use the Hungarian algorithm to match and select different paths;

[0101] Combine the depth information obtained by the depth sensor with the two-dimensional image coordinates of the key points to calculate the three-dimensional coordinates of the key points, and fuse the three-dimensional coordinates calculated from different perspectives;

[0102] In each frame, update the athlete's motion trajectory according to the matching result and the calculated three-dimensional key point coordinates, and use a smoothing algorithm to smooth the trajectory;

[0103] According to the rules of serpentine running, define the correct passing order of the poles, compare the athlete's motion trajectory with the positions of the poles, calculate the distances between the trajectory points and each pole in three-dimensional space, determine whether the athlete passes the poles in the specified order, and judge whether there are any missed poles or out-of-order behaviors according to the relationship between the trajectory and the poles.

[0104] In serpentine running training, the athlete's motion trajectory is an important basis for evaluating their performance and detecting problems. Accurate trajectory tracking can record every action detail of the athlete during the pole-around process, providing accurate data for score calculation and action analysis. For example, through the trajectory, it can be judged whether the athlete circles the poles in the specified order and whether there are any missed pole behaviors. If the trajectory tracking is inaccurate or unstable, it may lead to misjudgment and affect the fairness of the assessment. The combination of the DeepSORT algorithm and the Hungarian path matching model can effectively solve problems such as target occlusion and similar appearances in complex motion scenarios, ensuring the continuity and accuracy of trajectory tracking, thereby providing a reliable data basis for subsequent analysis and evaluation. The motion trajectory tracking module 4 is based on the DeepSORT algorithm and the Hungarian path matching model, calculates the three-dimensional coordinates of 14 key points of the human body in real time, constructs a serpentine running motion trajectory, and detects missed pole / out-of-order violations. The DeepSORT algorithm combined with the Hungarian path matching model improves the accuracy and stability of tracking and can continuously and stably track the athlete's motion trajectory in complex scenarios. The three-dimensional coordinate calculation and trajectory construction provide detailed data support for subsequent pole collision detection, score calculation, and action analysis, while the missed pole / out-of-order violation detection ensures that the athlete completes the training according to the specified path, improving the standardization and effectiveness of the training.

[0105] Preferably, the three-dimensional pole collision detection module 5 uses a rigid body collision model and a continuous collision detection algorithm to determine the pole collision event by calculating the Euclidean distance between the ankle / knee / hip key points and the safety area of the pole, including the following steps,

[0106] Regard the ankle, knee, and hip joints of the athlete as the collision points in the rigid body model, and the positions of these points in three-dimensional space are determined by the three-dimensional coordinates calculated by the motion trajectory tracking module 4;

[0107] Set a safety area for each pole, usually a cylinder or other suitable shape area centered on the pole with a certain radius;

[0108] During the dynamic process of the serpentine run, continuously obtain the three-dimensional coordinate information of the human body key points in each frame of the image to form time series data;

[0109] Define a collision detection function to determine whether the key points enter the safe area of the rod. When the value of the collision detection function changes from 0 to 1, it is determined that a rod collision event occurs;

[0110] In the three-dimensional rod collision detection, calculate the Euclidean distance between each key point and the center of each rod;

[0111] Comprehensively consider the collision detection results of multiple key points to finally determine the rod collision event. At a certain moment, if at least one key point collides with the safe area of the rod, it is determined that a rod collision occurs.

[0112] Rod collision is a common incorrect action in serpentine run training, which directly affects the athlete's performance and action standardization. Accurately detecting rod collision events is crucial for evaluating the athlete's performance and providing targeted training guidance. By calculating the Euclidean distance between the key points and the safe area of the rod, the system can objectively and accurately judge the rod collision situation, avoiding the subjectivity and errors of manual judgment. For example, in traditional assessments, coaches observing rod collisions with the naked eye may have perspective limitations and reaction delays, while this system uses three-dimensional space calculations to accurately judge rod collisions instantly, providing timely feedback to athletes, helping them improve their actions, and enhancing the training quality. The three-dimensional rod collision detection module 5 uses a rigid body collision model and a continuous collision detection algorithm to determine rod collision events by calculating the Euclidean distance between the ankle / knee / hip key points and the safe area of the rod. This method can detect rod collisions of athletes during the serpentine run in real-time and accurately, and timely discover problems in actions. The rigid body collision model regards the human body key points as collision points and combines the continuous collision detection algorithm to ensure the accuracy and timeliness of detection. This accurate detection provides timely feedback to athletes, helping them correct incorrect actions and improve training effects.

[0113] Preferably, the performance calculation and compensation module 6 realizes μs-level start and stop time synchronization based on the PTPv2 protocol, combines speed projection compensation, and outputs the final assessment result, including the following steps:

[0114] Realize μs-level start and stop time synchronization based on the PTPv2 protocol;

[0115] When the athlete starts running, the system determines whether the athlete has crossed the initial position of the first rod by detecting the position change of the athlete's foot, and uses this as the basis for recording the start time. The start time is determined by the device timestamp after time synchronization. When the athlete completes the serpentine run and returns, the end time is determined by detecting the moment of crossing the finish line, and the end time is determined by the device timestamp after time synchronization;

[0116] The result of the serpentine run is the time difference between the start time and the end time;

[0117] During the serpentine run, the system calculates the athlete's speed in real time. By obtaining the position information of the athlete at different time points, the instantaneous speed is calculated;

[0118] Project the athlete's speed onto the path of the serpentine run to take into account the curve characteristics of the path;

[0119] Due to the delay of the detection system or the slight error of time synchronization, the actually recorded time may deviate from the real time. Through speed projection compensation, the result is corrected;

[0120] Integrate the start and end times calculated by time synchronization and the result after speed projection compensation to obtain the final assessment result;

[0121] Timely feedback the final result to the athlete by means of voice or display screen, etc., and store the result data in the database for subsequent statistics and analysis.

[0122] In training assessment, the result is an important indicator to measure the athlete's performance. Accurate result calculation is crucial for evaluating the training effect and motivating athletes to improve their results. In traditional assessments, manual timing has problems such as reaction delay and perspective limitations, making it difficult to ensure the accuracy of the results. And this system realizes high-precision time synchronization and result correction through the PTPv2 protocol and speed projection compensation. It can accurately record the start and end times of athletes in complex situations, ensuring the fairness and reliability of the results. For example, in large training venues or when multiple athletes are assessed simultaneously, time synchronization and error compensation are particularly important, which can effectively avoid result deviations caused by environmental factors or system delays. The result calculation and compensation module 6 realizes μs-level start and end time synchronization based on the PTPv2 protocol. Combined with speed projection compensation, it outputs the final assessment result. The PTPv2 protocol ensures high-precision time synchronization and avoids result errors caused by time asynchronization; speed projection compensation further corrects the result deviations that may be caused by detection delays or system errors. This precise result calculation method ensures the fairness and objectivity of the assessment, enabling the athlete's result to accurately reflect their actual performance. At the same time, timely result feedback helps athletes understand their training level, adjust their training plans, and improve the training effect.

[0123] The above are only the preferred embodiments of the present invention and are not used to limit the present invention. For those skilled in the art, the present invention can have various changes and modifications. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A snake-shaped running training and assessment system based on visual recognition technology, characterized in that, The system comprises a multimodal visual acquisition module (1), a rod identification and dynamic calibration module (2), an identity and action joint recognition module (3), a motion trajectory tracking module (4), a three-dimensional rod collision detection module (5), and a performance calculation and compensation module (6). The multimodal visual acquisition module (1) is used to collect RGB images, depth information and environmental point cloud data of athletes, and obtains the RGB images, depth information and environmental point cloud data of athletes through IEEE The 1588v2 protocol realizes multi-view spatiotemporal synchronization. The rod identification and dynamic calibration module (2) is used to integrate the YOLOv8 model and the adaptive projection correction algorithm, and complete the three-dimensional coordinate positioning of 1-7 rods by fusing the laser radar point cloud data, and establish a motion plane coordinate system. The identity and action joint identification module (3) is used to complete the athlete identity authentication and preparation posture compliance detection through multi-modal biometric fusion. The motion trajectory tracking module (4) is based on the DeepSORT algorithm and the Hungarian path matching model to calculate the three-dimensional coordinates of 14 key points of the human body in real time, construct the serpentine running motion trajectory, and detect the missing pole / wrong sequence violation behavior. The three-dimensional pole collision detection module (5) adopts the rigid body collision model and the continuous collision detection algorithm to determine the pole collision event by calculating the Euclidean distance between the ankle / knee / hip key points and the pole safety area. The score calculation and compensation module (6) realizes μs-level start and end time synchronization based on the PTPv2 protocol, and outputs the final assessment score in combination with speed projection compensation.

2. The snake-shaped running training and assessment system based on visual recognition technology according to claim 1, wherein, The multimodal visual acquisition module (1) is used to collect the athlete's RGB image, depth information and environmental point cloud data, and realize multi-view spatiotemporal synchronization through the IEEE 1588v2 protocol, including the following steps: Using a high-resolution color camera, the color image sequence of the athlete during the serpentine run is continuously acquired at a set frame rate; Use a depth sensor, working synchronously with the RGB camera, to measure the distance between objects in the scene and the camera in real time and generate a depth map; With the help of laser radar equipment, the training ground is scanned to obtain point cloud data containing a large amount of spatial point location information; Multi-view spatiotemporal synchronization is achieved through the IEEE 1588v2 protocol.

3. The snake-shaped running training and assessment system based on visual recognition technology according to claim 1, characterized in that, The rod identification and dynamic calibration module (2) is used to integrate the YOLOv8 model and the adaptive projection correction algorithm, complete the three-dimensional coordinate positioning of 1-7 rods by fusing the laser radar point cloud data, and establish a motion plane coordinate system, including the following steps: Collect a large amount of image data containing rods and train the original YOLOv8 model; The collected RGB image is input into the trained YOLOv8 model. The model extracts features from the image through the convolution layer and feature pyramid structure, and outputs the position information of the rod, including the category of the rod, the confidence level, and the coordinates of the bounding box. Preprocess the point cloud data collected by the LiDAR, including denoising and filtering operations; Project the point cloud data into the image coordinate system by acquiring the external parameters of the camera and the laser radar, wherein the external parameters include the rotation matrix and the translation vector; Combined with the rod bounding boxes detected by the YOLOv8 model and the registered point cloud data, filter out the point cloud points located within the rod bounding boxes; By taking multiple images containing calibration marks and using Zhang Zhengyou's calibration method, obtain the internal and external parameters of the camera; By monitoring the changes of the site in real time, the projection matrix H is updated using the following formula: , where is the correction increment, is the historical projection matrix, is the real-time projection matrix, which can be calculated by the least squares optimization algorithm. When the site changes are detected, multiple groups of new image data and corresponding known three-dimensional coordinate data are collected to construct an error equation: , where e is the projection error, are the pixel coordinates in the image, are the corresponding world coordinates. By minimizing the sum of squared errors: the solution is obtained to achieve the adaptive correction of the projection matrix H; Utilize the point cloud information of the site in the lidar point cloud data to fit the motion plane, and based on the fitted motion plane, establish a motion plane coordinate system.

4. A snake-shaped running training and assessment system based on visual recognition technology according to claim 1, characterized in that, The identity and action joint recognition module (3) is used to complete athlete identity authentication and preparation posture compliance detection through multimodal biometric fusion, including the following steps: Use a high-definition camera to obtain the facial image of the athlete, locate the facial key points through a face detection algorithm, and extract the geometric and texture features of the face. The geometric features include the positions, distances, and proportional relationships of the eyes, nose, and mouth, and the texture features involve the texture details of the skin and the information of color distribution; Obtain the body depth information of the athlete through a depth sensor, combine it with 3D reconstruction technology to obtain the spatial positions and posture information of various parts of the athlete's body, and use a joint point detection algorithm to locate the key joint points of the athlete's body. The key joint points include the shoulders, elbows, wrists, hips, knees, and ankles, and extract the features of the positions, angles, and relative position relationships of the joint points; Fuse the face features and body posture features to form a comprehensive biometric vector, and use a feature-level fusion method to splice or weight-fuse the two feature vectors; Match the fused feature vector with the feature vectors in the pre-stored athlete feature library, and use a similarity measurement method to calculate the similarity between the feature vector to be recognized and the feature vectors in the library; According to the similarity measurement result, select the athlete with the highest similarity as the recognition result; Extract the key features of the athlete's preparation posture from the collected depth information and joint point data; Compare the extracted posture features with the preset standard preparation posture feature template to determine whether the athlete's preparation posture meets the specifications.

5. The snake-shaped running training and assessment system based on visual recognition technology according to claim 1, characterized in that, The motion trajectory tracking module (4) is based on the DeepSORT algorithm and the Hungarian path matching model, and calculates the three-dimensional coordinates of 14 key points of the human body in real time, constructs a snake-shaped running motion trajectory, and detects missed pole / out-of-order violation behaviors, including the following steps: In the DeepSORT algorithm, the state of each tracking target is maintained by a Kalman filter, including information on the position, speed, and acceleration of the target; In each frame, use the Kalman filter to predict the position of each tracking target in the next frame; Calculate the correlation function value between the detection target and the tracking trajectory; Based on the calculated correlation cost matrix, use the Hungarian algorithm for optimal matching to assign the detection target to the most likely tracking trajectory; Represent the athlete's motion trajectory as a sequence of three-dimensional coordinates of a series of key points. In the case of multiple possible paths, use the Hungarian algorithm to match and select different paths; Combine the depth information obtained by the depth sensor with the two-dimensional image coordinates of the key points, calculate the three-dimensional coordinates of the key points, and fuse the three-dimensional coordinates calculated from different perspectives; In each frame, according to the matching results and the calculated three-dimensional key point coordinates, update the athlete's movement trajectory, and use a smoothing algorithm to smooth the trajectory; According to the rules of the slalom run, define the correct passing order of the rods, compare and analyze the athlete's movement trajectory with the positions of the rods, calculate the distances between the trajectory points and each rod in three-dimensional space, determine whether the athlete passes the rods in the specified order, and judge whether there are any missed rods or out-of-order behaviors based on the relationship between the trajectory and the rods.

6. The snake-shaped running training and assessment system based on visual recognition technology according to claim 1, wherein The three-dimensional pole collision detection module (5) uses a rigid body collision model and a continuous collision detection algorithm to determine pole collision events by calculating the Euclidean distances between the ankle / knee / hip key points and the safe areas of the rods. The steps are as follows: Regard the ankle, knee, and hip joints of the athlete as the collision points in the rigid body model, and the positions of these points in three-dimensional space are determined by the three-dimensional coordinates calculated by the movement trajectory tracking module (4); Set a safe area for each rod, which is a cylindrical or other shaped area centered on the rod with a radius; During the dynamic process of the slalom run, continuously obtain the three-dimensional coordinate information of the human key points in each frame image to form time series data; Define a collision detection function to judge whether a key point enters the safe area of the rod. When the value of the collision detection function changes from 0 to 1, it is determined that a pole collision event has occurred; In three-dimensional pole collision detection, calculate the Euclidean distances between each key point and the centers of the rods; Comprehensively consider the collision detection results of multiple key points to finally determine pole collision events. At a certain moment, if at least one key point collides with the safe area of the rod, it is determined that a pole collision has occurred.

7. The snake-shaped running training and assessment system based on visual recognition technology according to claim 1, characterized in that The result calculation and compensation module (6) realizes μs-level start and stop time synchronization based on the PTPv2 protocol, combines speed projection compensation, and outputs the final assessment result. The steps are as follows: Realize μs-level start and stop time synchronization based on the PTPv2 protocol; When the athlete starts running, the system detects the position change of the athlete's foot to judge whether he has crossed the initial position of the first rod, and uses this as the basis for recording the start time. The start time is determined by the device timestamp after time synchronization. When the athlete completes the slalom run and returns, the end time is determined by detecting the moment of crossing the finish line. The end time is determined by the device timestamp after time synchronization; The slalom run result is the time difference between the start time and the end time; During the slalom run, the system calculates the athlete's speed in real time by obtaining the position information of the athlete at different time points and calculating his instantaneous speed; Project the athlete's speed onto the path of the slalom run to consider the curve characteristics of the path; Due to the delay of the detection system or the small error of time synchronization, there is a deviation between the actually recorded time and the real time. The result is corrected through speed projection compensation; Integrate the start and stop times calculated by time synchronization and the result after speed projection compensation to obtain the final assessment result; Timely feedback the final result to the athlete by voice or display, and store the result data in the database for subsequent statistics and analysis.

Images