Snakelike running training and checking system based on visual identification technology

Abstract

The invention discloses a snakelike running training examination system based on a visual identification technology, belongs to the technical field of visual identification, and solves the problems that in the snakelike running training process, training examination is generally carried out in a manual mode, perfect training examination can only be carried out under the condition that a coach or a professional athlete is on the scene, and the training efficiency is high. Otherwise, invalid training may occur. Comprising a multi-modal visual acquisition module, a rod piece identification and dynamic calibration module, an identity and action joint identification module, a motion trajectory tracking module, a three-dimensional rod collision detection module and a score calculation and compensation module. According to the system, whole-process automation of snakelike running training assessment is achieved, athlete identity recognition, preparation posture detection, real-time motion tracking, rod collision detection and automatic score calculation and compensation are covered, manual intervention is not needed in the whole process, the training assessment efficiency is remarkably improved, human factor interference is reduced, and justice and objectivity are ensured.

Description

Technical Field

[0001] The present invention relates to the field of visual recognition technology, and in particular to a serpentine running training and assessment system based on visual recognition technology. Background Art

[0002] Snake running training is a very practical and targeted sports training program, which is widely used in team sports such as football, basketball, rugby, as well as track and field, military training and other fields. It mainly simulates the trajectory of snakes, allowing trainees to constantly change directions during rapid movement, thereby effectively improving the body's flexibility, coordination and reaction ability. During the training process, multiple markers, such as cone barrels, are usually set up, and trainees need to shuttle between these markers at the fastest speed while maintaining body balance and stability. This training method can fully mobilize multiple muscle groups in the body, especially the leg, waist and core muscle groups, and enhance muscle strength and endurance. In addition, snake running can also exercise athletes' visual attention and spatial perception ability, because when changing direction quickly, athletes need to pay attention to the location of markers and the surrounding environment at all times to avoid collisions. For young athletes, snake running training helps to cultivate their sports interest and physical fitness foundation; for professional athletes, it is an important means to improve their competitive level and optimize technical movements. By adhering to snake running training for a long time, it can not only improve athletes' breakthrough, defense and response capabilities in the game, but also effectively prevent sports injuries, because it can enhance joint flexibility and ligament strength. In short, serpentine running training is a simple, easy and effective training method, which deserves to be widely promoted and applied in various sports training.

[0003] However, during serpentine running training, training assessment is usually conducted manually, and a complete training assessment can only be conducted in the presence of a coach or professional athlete, otherwise ineffective training may occur.

[0004] Therefore, a serpentine running training and assessment system based on visual recognition technology is proposed to solve or alleviate the above problems. Summary of the invention

[0005] The purpose of the present invention is to solve the shortcomings of the prior art and to propose a serpentine running training and assessment system based on visual recognition technology.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions: A serpentine running training and assessment system based on visual recognition technology includes a multimodal visual acquisition module, a pole recognition and dynamic calibration module, an identity and action joint recognition module, a motion trajectory tracking module, a three-dimensional pole collision detection module, and a performance calculation and compensation module. The multimodal visual acquisition module is used to collect the athlete's RGB image, depth information and environmental point cloud data, and pass the IEEE The 1588v2 protocol realizes multi-perspective spatiotemporal synchronization. The rod identification and dynamic calibration module is used to integrate the improved YOLOv8 model and the adaptive projection correction algorithm. By fusing the laser radar point cloud data, the three-dimensional coordinate positioning of 1-7 rods is completed, and the motion plane coordinate system is established. The identity and action joint identification module is used to complete the athlete identity authentication and preparation posture compliance detection through multi-modal biometric fusion. The motion trajectory tracking module is based on the improved DeepSORT algorithm and the Hungarian path matching model. The three-dimensional coordinates of 14 key points of the human body are calculated in real time, the serpentine running motion trajectory is constructed, and the missing pole / wrong sequence violation behavior is detected. The three-dimensional collision detection module adopts a rigid body collision model and a continuous collision detection algorithm. The Euclidean distance between the ankle / knee / hip key points and the rod safety area is calculated to determine the collision event. The score calculation and compensation module realizes μs-level start and end time synchronization based on the PTPv2 protocol, and combines speed projection compensation to output the final assessment score.

[0007] Preferably, the multimodal visual acquisition module is used to collect the athlete's RGB image, depth information and environmental point cloud data, and realize multi-view spatiotemporal synchronization through the IEEE 1588v2 protocol, including the following steps: Using a high-resolution color camera, the color image sequence of the athlete during the serpentine run is continuously acquired at a set frame rate; Use a depth sensor, working synchronously with the RGB camera, to measure the distance between objects in the scene and the camera in real time and generate a depth map; With the help of laser radar and other equipment, the training ground is scanned to obtain point cloud data containing a large amount of spatial point location information; Multi-view spatiotemporal synchronization is achieved through the IEEE 1588v2 protocol.

[0008] Preferably, the rod identification and dynamic calibration module is used to integrate the improved YOLOv8 model and the adaptive projection correction algorithm, complete the three-dimensional coordinate positioning of 1-7 rods by fusing the laser radar point cloud data, and establish the motion plane coordinate system, including the following steps: Collect a large amount of image data containing rods, and improve and train the original YOLOv8 model; The collected RGB image is input into the trained improved YOLOv8 model. The model extracts features from the image through structures such as convolutional layers and feature pyramids, and outputs the position information of the rod, including the category of the rod, the confidence level, and the coordinates of the bounding box. Preprocess the point cloud data collected by the LiDAR, including denoising, filtering and other operations; Project the point cloud data into the image coordinate system by acquiring the external parameters of the camera and the lidar, wherein the external parameters include a rotation matrix and a translation vector; Combine the rod bounding box detected by the YOLOv8 model with the registered point cloud data to filter out the point cloud points within the rod bounding box; By taking multiple images containing calibration marks, the Zhang Zhengyou calibration method is used to obtain the camera's intrinsic and extrinsic parameters; By monitoring the changes in the site in real time, the projection matrix H is updated using the following formula: ,in, To correct the increment, is the history projection matrix, It is a real-time projection matrix, which can be calculated by optimization algorithms such as the least squares method. When a site change is detected, multiple sets of new image data and corresponding known three-dimensional coordinate data are collected to construct an error equation. , where e is the projection error, is the pixel coordinate in the image, For the corresponding world coordinates, by minimizing the sum of squared errors: Solved , realize adaptive correction of the projection matrix H; The point cloud information of the site in the lidar point cloud data is used to fit the motion plane, and the motion plane coordinate system is established based on the fitted motion plane.

[0009] Preferably, the identity and action joint recognition module is used to complete the athlete identity authentication and preparation posture compliance detection through multi-modal biometric fusion, including the following steps: Use high-definition cameras to obtain athletes' facial images, locate facial key points through face detection algorithms, and extract geometric and texture features of the face. Geometric features include the position, spacing, and proportional relationship of the eyes, nose, and mouth, while texture features involve information such as skin texture details and color distribution. The depth information of the athlete's body is obtained through a depth sensor, and the spatial position and posture information of each part of the athlete's body is obtained by combining 3D reconstruction technology. The key joints of the athlete's body are located using a joint point detection algorithm. The key joints include shoulders, elbows, wrists, hips, knee joints, and ankle joints, and the position, angle, and relative position relationship of the joints are extracted; The facial features and body posture features are integrated to form a comprehensive biometric feature vector, and the two feature vectors are concatenated or weightedly fused using a feature-level fusion method; Match the fused feature vector with the feature vector in the pre-stored athlete feature library, and use a similarity measurement method to calculate the similarity between the feature vector to be identified and the feature vector in the library; According to the similarity measurement results, the athlete with the highest similarity is selected as the recognition result; Extract key features of the athlete's preparation posture from the collected depth information and joint point data; The extracted posture features are compared with the preset standard preparation posture feature template to determine whether the athlete's preparation posture meets the standards.

[0010] Preferably, the motion trajectory tracking module calculates the three-dimensional coordinates of 14 key points of the human body in real time based on the improved DeepSORT algorithm and the Hungarian path matching model, constructs the serpentine running motion trajectory, and detects the missing pole / wrong sequence violation behavior, including the following steps: In the DeepSORT algorithm, the state of each tracked target is maintained by the Kalman filter, including information such as the target's position, velocity, and acceleration; In each frame, a Kalman filter is used to predict the next frame position of each tracked target; Calculate the correlation function value between the detected target and the tracking trajectory; Based on the calculated association cost matrix, the Hungarian algorithm is used for optimal matching to assign the detected target to the most likely tracking trajectory; The athlete's motion trajectory is represented as a three-dimensional coordinate sequence of key points. When there are multiple possible paths, the Hungarian algorithm is used to match and select different paths. The depth information obtained by the depth sensor is combined with the two-dimensional image coordinates of the key points to calculate the three-dimensional coordinates of the key points, and the three-dimensional coordinates calculated under different viewing angles are fused; In each frame, the athlete's motion trajectory is updated according to the matching results and the calculated three-dimensional key point coordinates, and the trajectory is smoothed using a smoothing algorithm; According to the rules of serpentine running, the correct order of passing the poles is defined, and the athlete's movement trajectory is compared and analyzed with the position of the poles. In three-dimensional space, the distance between the trajectory point and each pole is calculated to determine whether the athlete passes through the poles in the prescribed order. Based on the relationship between the trajectory and the poles, it is determined whether there is any missing pole or wrong sequence behavior.

[0011] Preferably, the three-dimensional pole collision detection module adopts a rigid body collision model and a continuous collision detection algorithm to determine the pole collision event by calculating the Euclidean distance between the ankle / knee / hip key points and the pole safety area, including the following steps: The athlete's ankle, knee, and hip joints are considered as collision points in the rigid body model, and the positions of these points in three-dimensional space are determined by the three-dimensional coordinates calculated by the motion trajectory tracking module; Set a safety zone for each rod, usually a cylinder or other suitable shape with a certain radius centered on the rod; In the dynamic process of snake-like running, the three-dimensional coordinate information of the key points of the human body in each frame of the image is continuously obtained to form time series data; Define a collision detection function to determine whether the key point enters the safety area of ​​the rod. When the value of the collision detection function changes from 0 to 1, it is determined that a collision event has occurred. In the three-dimensional collision detection, the Euclidean distance between each key point and the center of each rod is calculated; The collision detection results of multiple key points are combined to finally determine the pole collision event. At a certain moment, if at least one key point collides with the safety area of ​​the pole, it is determined to be a pole collision.

[0012] Preferably, the score calculation and compensation module implements μs-level start and end time synchronization based on the PTPv2 protocol, combines speed projection compensation, and outputs the final assessment score, including the following steps: Achieve μs-level start and end time synchronization based on PTPv2 protocol; When the athlete starts running, the system detects the change in the position of the athlete's feet to determine whether it has crossed the initial position of the first pole, and uses this as the basis for recording the starting time. The starting time is determined by the device timestamp after time synchronization. When the athlete completes the serpentine run and returns, the end time is determined by detecting the moment of his / her crossing the finish line. The end time is determined by the device timestamp after time synchronization. The serpentine running result is the time difference between the starting time and the finishing time; During the serpentine run, the system calculates the athlete's speed in real time and calculates the athlete's instantaneous speed by obtaining the athlete's position information at different time points; The athlete's speed was projected onto the serpentine path to account for the curvilinear nature of the path; Due to the delay of the detection system or the slight error of time synchronization, the actual recorded time may deviate from the real time. The results can be corrected through speed projection compensation. The start and end times calculated by time synchronization and the results after speed projection compensation are integrated to obtain the final assessment results; The final results will be fed back to the athletes in a timely manner through voice or display screens, and the results data will be stored in the database for subsequent statistics and analysis.

[0013] The present invention has the following beneficial effects: The system of the present invention realizes the automation of the entire process of serpentine running training assessment, from athlete identification and preparation posture detection to real-time tracking and collision detection during exercise, and then to automatic calculation and compensation of the final results. The entire process does not require human intervention. This not only greatly improves the efficiency of training assessment, but also reduces the interference of human factors on the assessment results, ensuring the fairness and objectivity of the assessment. For example, in traditional assessments, coaches are required to manually record time, while this system uses the PTPv2 protocol to achieve μs-level start and end time synchronization, combined with speed projection compensation, it can accurately and automatically record results, avoiding errors caused by manual timing; The system uses a multimodal visual acquisition module, which integrates RGB images, depth information and environmental point cloud data, and realizes multi-view spatiotemporal synchronization through the IEEE 1588v2 protocol. This multi-dimensional data acquisition method can comprehensively and accurately capture various information of athletes during the serpentine running process. Each module uses advanced algorithms to process and analyze the data, such as the improved YOLOv8 model for pole identification, the adaptive projection correction algorithm for dynamic calibration, and the improved DeepSORT algorithm and Hungarian path matching model for motion trajectory tracking, etc., to ensure the accuracy and reliability of data processing. Taking pole identification as an example, by integrating laser radar point cloud data, the system can accurately complete the three-dimensional coordinate positioning of 1-7 poles, establish a motion plane coordinate system, and provide a precise spatial reference for subsequent motion trajectory analysis and pole collision detection; This system can not only accurately calculate the athlete's serpentine running results, but also conduct a comprehensive evaluation and analysis of the training process. Through the three-dimensional pole collision detection module, it can determine the pole collision event in real time, helping athletes to find and correct problems in their movements in a timely manner; through the motion trajectory tracking module, it can detect the violation of missing poles / wrong sequence, and provide athletes with detailed motion trajectory data so that they can understand their path selection and direction changes during the serpentine running process; at the same time, the identity and action joint recognition module can also conduct compliance detection of the athlete's preparation posture to ensure that they are in the correct posture before starting, which helps to improve training effects and prevent sports injuries. These comprehensive evaluation information provides scientific training feedback for athletes and coaches, which helps to formulate more reasonable training plans and improve training methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0015] Figure 1 It is a structural block diagram of the present invention.

[0016] 1. Multimodal vision acquisition module; 2. Pole recognition and dynamic calibration module; 3. Identity and action joint recognition module; 4. Motion trajectory tracking module; 5. Three-dimensional collision detection module; 6. Score calculation and compensation module. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.

[0018] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0019] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0020] In the description of the present invention, it should be understood that the terms "center", "up", "down", "left", "right", "vertical", "horizontal", "inside", "outside" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, or are the orientations or positional relationships in which the product of the invention is conventionally placed when in use, or are the orientations or positional relationships conventionally understood by those skilled in the art. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present invention.

[0021] Furthermore, the terms “first”, “second”, “third”, etc. are merely used for distinguishing descriptions and are not to be understood as indicating or implying relative importance.

[0022] In the description of the present invention, it is also necessary to explain that, unless otherwise clearly specified and limited, the terms "set", "install", "connect", and "connect" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0023] A serpentine running training and assessment system based on visual recognition technology, such as Figure 1 As shown, it includes a multimodal visual acquisition module 1, a rod recognition and dynamic calibration module 2, an identity and action joint recognition module 3, a motion trajectory tracking module 4, a three-dimensional collision detection module 5, and a performance calculation and compensation module 6. The multimodal visual acquisition module 1 is used to collect the RGB image, depth information and environmental point cloud data of the athlete, and pass the IEEE The 1588v2 protocol realizes multi-perspective spatiotemporal synchronization. The pole identification and dynamic calibration module 2 is used to integrate the improved YOLOv8 model and the adaptive projection correction algorithm. By fusing the lidar point cloud data, the three-dimensional coordinate positioning of 1-7 poles is completed, and the motion plane coordinate system is established. The identity and action joint recognition module 3 is used to complete the athlete identity authentication and preparation posture compliance detection through multi-modal biometric fusion. The motion trajectory tracking module 4 is based on the improved DeepSORT algorithm and the Hungarian path matching model. The three-dimensional coordinates of 14 key points of the human body are calculated in real time, the serpentine running trajectory is constructed, and the missing pole / wrong sequence violation behavior is detected. The three-dimensional collision detection module 5 adopts the rigid body collision model and the continuous collision detection algorithm. The Euclidean distance between the ankle / knee / hip key points and the pole safety area is calculated to determine the collision event. The score calculation and compensation module 6 realizes μs-level start and end time synchronization based on the PTPv2 protocol, and combines speed projection compensation to output the final assessment score.

[0024] Traditional serpentine training assessment relies on manual work and requires the presence of coaches or professional athletes, otherwise ineffective training may occur. This system breaks through this limitation through automated processes, allowing athletes to conduct training assessments independently and check training results at any time. For example, athletes can take assessments at any time, and the system automatically records results and detects movement problems without waiting for coaches to arrange and participate. This greatly improves the flexibility and autonomy of training and helps athletes improve their results more efficiently. This system covers all key links in serpentine training assessment and realizes full process automation. This comprehensiveness ensures that the system can independently complete the entire assessment process from athlete identification to result output without manual intervention, greatly improving the efficiency and fairness of training assessments. Through multimodal visual acquisition, the system obtains rich data types, providing a basis for subsequent precise analysis; pole recognition and dynamic calibration ensure the accurate acquisition of venue information; identity and action joint recognition not only identifies the identity of the athlete, but also detects whether the preparation posture is compliant, ensuring the quality of training from the source; motion trajectory tracking monitors the athlete's dynamics in real time, constructs a complete motion trajectory, and provides a basis for score calculation and action analysis; three-dimensional pole collision detection accurately judges pole collision events and discovers action errors in time; score calculation and compensation module 6 combines time synchronization and speed compensation to output accurate assessment results, which fully reflects the athlete's actual performance.

[0025] Preferably, the multimodal visual acquisition module 1 is used to collect the athlete's RGB image, depth information and environmental point cloud data, and realize multi-view spatiotemporal synchronization through the IEEE 1588v2 protocol, including the following steps: Using a high-resolution color camera, the color image sequence of the athlete during the serpentine run is continuously acquired at a set frame rate; Use a depth sensor, working synchronously with the RGB camera, to measure the distance between objects in the scene and the camera in real time and generate a depth map; With the help of laser radar and other equipment, the training ground is scanned to obtain point cloud data containing a large amount of spatial point location information; Multi-view spatiotemporal synchronization is achieved through the IEEE 1588v2 protocol.

[0026] In the serpentine running training, the athletes' movements are complex and fast, and a single data type is difficult to fully capture their motion state. Relying solely on RGB images cannot accurately obtain depth and three-dimensional position information, which may lead to misjudgment of events such as hitting the pole; only depth information or point cloud data lacks descriptions of the athlete's appearance and detailed movements. Therefore, fusing multimodal data and realizing spatiotemporal synchronization can make up for the shortcomings of single data, comprehensively and accurately record the athlete's movement process, and provide a reliable data basis for subsequent analysis and evaluation, thereby more scientifically guiding training and assessment. The multimodal visual acquisition module 1 integrates RGB images, depth information and environmental point cloud data, and realizes multi-view spatiotemporal synchronization through the IEEE 1588v2 protocol. This fusion method enriches the data dimension, can fully capture the appearance, depth and spatial position information of athletes during the serpentine running process, and provides multi-angle data support for subsequent precise analysis. Spatiotemporal synchronization ensures the consistency and accuracy of multi-source data, avoids data deviations caused by asynchrony, and improves the reliability and analysis accuracy of the entire system. For example, when detecting an athlete hitting a pole, RGB images can provide visual basis, depth information can determine the precise distance between the athlete and the pole, and point cloud data can restore the three-dimensional scene. Multi-source synchronous data work together to make pole collision detection more accurate.

[0027] Preferably, the rod identification and dynamic calibration module 2 is used to integrate the improved YOLOv8 model and the adaptive projection correction algorithm, complete the three-dimensional coordinate positioning of 1-7 rods by fusing the laser radar point cloud data, and establish a motion plane coordinate system, including the following steps: Collect a large amount of image data containing rods, and improve and train the original YOLOv8 model; The collected RGB image is input into the trained improved YOLOv8 model. The model extracts features from the image through structures such as convolutional layers and feature pyramids, and outputs the position information of the rod, including the category of the rod, the confidence level, and the coordinates of the bounding box. Preprocess the point cloud data collected by the LiDAR, including denoising, filtering and other operations; By obtaining the external parameters of the camera and lidar, the point cloud data is projected into the image coordinate system. The external parameters include the rotation matrix and the translation vector. Combine the rod bounding box detected by the YOLOv8 model with the registered point cloud data to filter out the point cloud points within the rod bounding box; By taking multiple images containing calibration marks, the Zhang Zhengyou calibration method is used to obtain the camera's intrinsic and extrinsic parameters; By monitoring the changes in the site in real time, the projection matrix H is updated using the following formula: ,in, To correct the increment, it can be calculated by the least squares method and other optimization algorithms. When the site changes are detected, multiple sets of new image data and corresponding known three-dimensional coordinate data are collected to construct the error equation. , where e is the projection error, is the pixel coordinate in the image, For the corresponding world coordinates, by minimizing the sum of squared errors: Solved , realize adaptive correction of the projection matrix H; The point cloud information of the site in the lidar point cloud data is used to fit the motion plane, and the motion plane coordinate system is established based on the fitted motion plane.

[0028] In serpentine running training, the position of the pole is an important basis for judging the athlete's movements and results. If the pole position is not accurately identified or the coordinate system is unstable, it will directly affect the judgment of the athlete's situation around the pole and the fairness of the results. For example, a slight deviation in the position of the pole may lead to the misjudgment of the athlete hitting the pole or missing the pole. By improving the YOLOv8 model and adaptive projection correction algorithm, the system can accurately locate the poles in real time and adapt to the slight changes in the venue, ensure the rigor and accuracy of the assessment, and provide reliable data support for subsequent modules, so as to comprehensively evaluate the performance of athletes. The pole identification and dynamic calibration module 2 integrates the improved YOLOv8 model and adaptive projection correction algorithm, integrates the laser radar point cloud data, and realizes the accurate three-dimensional coordinate positioning of 1-7 poles and the establishment of the motion plane coordinate system. The improved YOLOv8 model improves the accuracy and speed of pole identification, and can quickly and stably detect the position of the pole in complex scenes; the adaptive projection correction algorithm adjusts the projection matrix in real time according to the changes in the venue, ensuring the accuracy and reliability of coordinate positioning. This precise positioning provides a key site reference for subsequent motion trajectory analysis, pole collision detection and score calculation, ensuring the accuracy of the system's judgment of athletes' movements and paths.

[0029] Preferably, the identity and action joint recognition module 3 is used to complete the athlete identity authentication and preparation posture compliance detection through multi-modal biometric fusion, including the following steps: Use high-definition cameras to obtain facial images of athletes, locate facial key points through face detection algorithms, and extract geometric and texture features of the face. Geometric features include the position, spacing, and proportion of the eyes, nose, and mouth, while texture features involve information such as skin texture details and color distribution. The depth information of the athlete's body is obtained through the depth sensor, and the spatial position and posture information of each part of the athlete's body is obtained by combining the 3D reconstruction technology. The key joints of the athlete's body are located using the joint point detection algorithm. The key joints include shoulders, elbows, wrists, hips, knees, and ankles, and the position, angle, and relative position relationship of the joints are extracted. The facial features and body posture features are integrated to form a comprehensive biometric feature vector, and the two feature vectors are concatenated or weightedly fused using a feature-level fusion method; Match the fused feature vector with the feature vector in the pre-stored athlete feature library, and use a similarity measurement method to calculate the similarity between the feature vector to be identified and the feature vector in the library; According to the similarity measurement results, the athlete with the highest similarity is selected as the recognition result; Extract key features of the athlete's preparation posture from the collected depth information and joint point data; The extracted posture features are compared with the preset standard preparation posture feature template to determine whether the athlete's preparation posture meets the standards.

[0030] In training assessment, accurate identification of athletes is a prerequisite for ensuring the accuracy of results and the pertinence of training plans. If the identity recognition is wrong, the performance record will be confused and the evaluation of training effect will be affected. At the same time, the preparation posture directly affects the starting quality of the serpentine run and the standardization of subsequent actions. If the preparation posture is incorrect, it will not only affect the results, but also increase the risk of sports injuries. Through multimodal biometric fusion and posture detection, the system can accurately identify the identity and standardize the preparation action, providing a good start for subsequent training and assessment, which helps athletes to better play their level and improve their performance. The identity and action joint recognition module 3 completes the athlete identity authentication and preparation posture compliance detection through multimodal biometric fusion. The fusion of facial features and body posture features improves the accuracy and reliability of identity recognition and avoids the errors that may occur in single feature recognition. Preparation posture compliance detection ensures that the athlete is in the correct posture before starting, which helps to improve the training effect and prevent sports injuries. This joint recognition method not only enhances the security of the system, but also standardizes athletes from the beginning of training, ensuring the scientificity and effectiveness of training and assessment.

[0031] Preferably, the motion trajectory tracking module 4 calculates the three-dimensional coordinates of 14 key points of the human body in real time based on the improved DeepSORT algorithm and the Hungarian path matching model, constructs the serpentine running motion trajectory, and detects the missing pole / wrong order violation behavior, including the following steps: In the DeepSORT algorithm, the state of each tracked target is maintained by the Kalman filter, including information such as the target's position, velocity, and acceleration; In each frame, a Kalman filter is used to predict the next frame position of each tracked target; Calculate the correlation function value between the detected target and the tracking trajectory; Based on the calculated association cost matrix, the Hungarian algorithm is used for optimal matching to assign the detected target to the most likely tracking trajectory; The athlete's motion trajectory is represented as a three-dimensional coordinate sequence of key points. When there are multiple possible paths, the Hungarian algorithm is used to match and select different paths. The depth information obtained by the depth sensor is combined with the two-dimensional image coordinates of the key points to calculate the three-dimensional coordinates of the key points, and the three-dimensional coordinates calculated under different viewing angles are fused; In each frame, the athlete's motion trajectory is updated according to the matching results and the calculated three-dimensional key point coordinates, and the trajectory is smoothed using a smoothing algorithm; According to the rules of serpentine running, the correct order of passing the poles is defined, and the athlete's movement trajectory is compared and analyzed with the position of the poles. In three-dimensional space, the distance between the trajectory point and each pole is calculated to determine whether the athlete passes through the poles in the prescribed order. Based on the relationship between the trajectory and the poles, it is determined whether there is any missing pole or wrong sequence behavior.

[0032] In serpentine running training, the athlete's motion trajectory is an important basis for evaluating their performance and discovering problems. Accurate trajectory tracking can record every detail of the athlete's movements during the process of circling the pole, and provide accurate data for score calculation and motion analysis. For example, the trajectory can be used to determine whether the athlete circles the pole in the prescribed order, whether there is any missing pole behavior, etc. If the trajectory tracking is inaccurate or unstable, it may lead to misjudgment and affect the fairness of the assessment. The combination of the improved DeepSORT algorithm and the Hungarian path matching model can effectively solve problems such as target occlusion and similar appearance in complex motion scenes, ensure the continuity and accuracy of trajectory tracking, and thus provide a reliable data basis for subsequent analysis and evaluation. The motion trajectory tracking module 4 is based on the improved DeepSORT algorithm and the Hungarian path matching model, and calculates the three-dimensional coordinates of 14 key points of the human body in real time, constructs the serpentine running motion trajectory, and detects missing poles / wrong order violations. The improved DeepSORT algorithm combined with the Hungarian path matching model improves the accuracy and stability of tracking, and can continuously and stably track the athlete's motion trajectory in complex scenes. Three-dimensional coordinate calculation and trajectory construction provide detailed data support for subsequent pole collision detection, score calculation and motion analysis, while missed pole / wrong sequence violation detection ensures that athletes complete training according to the prescribed path, improving the standardization and effectiveness of training.

[0033] Preferably, the three-dimensional collision detection module 5 adopts a rigid body collision model and a continuous collision detection algorithm to determine the collision event by calculating the Euclidean distance between the ankle / knee / hip key points and the rod safety area, including the following steps: The athlete's ankle, knee and hip joints are regarded as collision points in the rigid body model, and the positions of these points in the three-dimensional space are determined by the three-dimensional coordinates calculated by the motion trajectory tracking module 4; Set a safety zone for each rod, usually a cylinder or other suitable shape with a certain radius centered on the rod; In the dynamic process of snake-like running, the three-dimensional coordinate information of the key points of the human body in each frame of the image is continuously obtained to form time series data; Define a collision detection function to determine whether the key point enters the safety area of ​​the rod. When the value of the collision detection function changes from 0 to 1, it is determined that a collision event has occurred. In the three-dimensional collision detection, the Euclidean distance between each key point and the center of each rod is calculated; The collision detection results of multiple key points are combined to finally determine the pole collision event. At a certain moment, if at least one key point collides with the safety area of ​​the pole, it is determined to be a pole collision.

[0034] Hitting the pole is a common wrong action in serpentine running training, which directly affects the athlete's performance and action standardization. Accurate detection of the pole hitting event is crucial for evaluating the athlete's performance and providing targeted training guidance. By calculating the Euclidean distance between the key point and the pole safety zone, the system can objectively and accurately judge the pole hitting situation, avoiding the subjectivity and error of manual judgment. For example, in the traditional assessment, the coach may have visual angle limitation and reaction delay when observing the pole hitting with the naked eye, and this system uses three-dimensional space calculation to accurately judge the pole hitting in an instant, timely feedback problems to athletes, help them improve their movements, and improve the quality of training. The three-dimensional pole hitting detection module 5 adopts a rigid body collision model and a continuous collision detection algorithm to determine the pole hitting event by calculating the Euclidean distance between the ankle / knee / hip key point and the pole safety zone. This method can detect the pole hitting situation of athletes in the serpentine running process in real time and accurately, and find problems in the action in time. The rigid body collision model regards the key point of the human body as the collision point, combined with the continuous collision detection algorithm, to ensure the accuracy and timeliness of the detection. This accurate detection provides timely feedback for athletes, helps them correct wrong actions, and improves the training effect.

[0035] Preferably, the score calculation and compensation module 6 implements μs-level start and end time synchronization based on the PTPv2 protocol, combines speed projection compensation, and outputs the final assessment score, including the following steps: Achieve μs-level start and end time synchronization based on PTPv2 protocol; When the athlete starts running, the system detects the change in the position of the athlete's feet to determine whether it has crossed the initial position of the first pole, and uses this as the basis for recording the starting time. The starting time is determined by the device timestamp after time synchronization. When the athlete completes the serpentine run and returns, the end time is determined by detecting the moment of his / her crossing the finish line. The end time is determined by the device timestamp after time synchronization. The serpentine running result is the time difference between the starting time and the finishing time; During the serpentine run, the system calculates the athlete's speed in real time and calculates the athlete's instantaneous speed by obtaining the athlete's position information at different time points; The athlete's speed was projected onto the serpentine path to account for the curvilinear nature of the path; Due to the delay of the detection system or the slight error of time synchronization, the actual recorded time may deviate from the real time. The results can be corrected through speed projection compensation. The start and end times calculated by time synchronization and the results after speed projection compensation are integrated to obtain the final assessment results; The final results will be fed back to the athletes in a timely manner through voice or display screens, and the results data will be stored in the database for subsequent statistics and analysis.

[0036] In training assessment, results are an important indicator for measuring athlete performance. Accurate result calculation is of key significance for evaluating training effects and motivating athletes to improve their results. In traditional assessments, manual timing has problems such as reaction delay and visual angle limitation, which makes it difficult to ensure the accuracy of results. However, this system achieves high-precision time synchronization and result correction through the PTPv2 protocol and speed projection compensation, and can accurately record the start and end time of athletes in complex situations to ensure the fairness and reliability of results. For example, in large training venues or when multiple athletes are assessed at the same time, time synchronization and error compensation are particularly important, which can effectively avoid the result deviation caused by environmental factors or system delays. The result calculation and compensation module 6 realizes μs-level start and end time synchronization based on the PTPv2 protocol, and outputs the final assessment results in combination with speed projection compensation. The PTPv2 protocol ensures high precision of time synchronization and avoids the result error caused by time asynchrony; speed projection compensation further corrects the result deviation that may be caused by detection delay or system error. This precise result calculation method ensures the fairness and objectivity of the assessment, so that the athlete's results can accurately reflect their actual performance. At the same time, timely result feedback helps athletes understand their training level, adjust training plans, and improve training effects.

[0037] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A serpentine running training and assessment system based on visual recognition technology, characterized in that: The system comprises a multimodal visual acquisition module (1), a rod identification and dynamic calibration module (2), an identity and action joint recognition module (3), a motion trajectory tracking module (4), a three-dimensional rod collision detection module (5), and a performance calculation and compensation module (6). The multimodal visual acquisition module (1) is used to collect RGB images, depth information and environmental point cloud data of athletes, and obtains the RGB images, depth information and environmental point cloud data of athletes through IEEE The 1588v2 protocol realizes multi-view spatiotemporal synchronization. The rod identification and dynamic calibration module (2) is used to integrate the improved YOLOv8 model and the adaptive projection correction algorithm, and complete the three-dimensional coordinate positioning of 1-7 rods by fusing the laser radar point cloud data, and establish a motion plane coordinate system. The identity and action joint identification module (3) is used to complete the athlete identity authentication and preparation posture compliance detection through multi-modal biometric fusion. The motion trajectory tracking module (4) is based on the improved DeepSORT algorithm and the Hungarian path matching model to calculate the three-dimensional coordinates of 14 key points of the human body in real time, construct the serpentine running motion trajectory, and detect the missing pole / wrong sequence violation behavior. The three-dimensional pole collision detection module (5) adopts the rigid body collision model and the continuous collision detection algorithm to determine the pole collision event by calculating the Euclidean distance between the ankle / knee / hip key points and the pole safety area. The score calculation and compensation module (6) realizes μs-level start and end time synchronization based on the PTPv2 protocol, and outputs the final assessment score in combination with speed projection compensation.

2. According to the serpentine running training and assessment system based on visual recognition technology according to claim 1, it is characterized in that: The multimodal visual acquisition module (1) is used to collect the athlete's RGB image, depth information and environmental point cloud data, and realize multi-view spatiotemporal synchronization through the IEEE 1588v2 protocol, including the following steps: Using a high-resolution color camera, the color image sequence of the athlete during the serpentine run is continuously acquired at a set frame rate; Use a depth sensor, working synchronously with the RGB camera, to measure the distance between objects in the scene and the camera in real time and generate a depth map; With the help of laser radar equipment, the training ground is scanned to obtain point cloud data containing a large amount of spatial point location information; Multi-view spatiotemporal synchronization is achieved through the IEEE 1588v2 protocol.

3. The serpentine running training and assessment system based on visual recognition technology according to claim 1 is characterized in that: The rod identification and dynamic calibration module (2) is used to integrate the improved YOLOv8 model and the adaptive projection correction algorithm, complete the three-dimensional coordinate positioning of 1-7 rods by fusing the laser radar point cloud data, and establish a motion plane coordinate system, including the following steps: Collect a large amount of image data containing rods, and improve and train the original YOLOv8 model; The collected RGB image is input into the trained improved YOLOv8 model. The model extracts features from the image through the convolution layer and feature pyramid structure, and outputs the position information of the rod, including the category of the rod, the confidence level, and the coordinates of the bounding box. Preprocess the point cloud data collected by the LiDAR, including denoising and filtering operations; Project the point cloud data into the image coordinate system by acquiring the external parameters of the camera and the lidar, wherein the external parameters include a rotation matrix and a translation vector; Combine the rod bounding box detected by the YOLOv8 model with the registered point cloud data to filter out the point cloud points within the rod bounding box; By taking multiple images containing calibration marks, the Zhang Zhengyou calibration method is used to obtain the camera's intrinsic and extrinsic parameters; By monitoring the changes in the site in real time, the projection matrix H is updated using the following formula: ,in, To correct the increment, is the history projection matrix, It is a real-time projection matrix, which can be calculated by the least squares optimization algorithm. When a site change is detected, multiple sets of new image data and corresponding known three-dimensional coordinate data are collected to construct an error equation. , where e is the projection error, is the pixel coordinate in the image, For the corresponding world coordinates, by minimizing the sum of squared errors: Solved , realize adaptive correction of the projection matrix H; The point cloud information of the site in the lidar point cloud data is used to fit the motion plane, and the motion plane coordinate system is established based on the fitted motion plane.

4. The serpentine running training and assessment system based on visual recognition technology according to claim 1 is characterized in that: The identity and action joint recognition module (3) is used to complete the athlete's identity authentication and preparation posture compliance detection through multi-modal biometric fusion, including the following steps: Use high-definition cameras to obtain athletes' facial images, locate facial key points through face detection algorithms, and extract geometric and texture features of the face. Geometric features include the position, spacing, and proportion of the eyes, nose, and mouth, while texture features involve skin texture details and color distribution information. The depth information of the athlete's body is obtained through a depth sensor, and the spatial position and posture information of each part of the athlete's body is obtained by combining the three-dimensional reconstruction technology. The key joints of the athlete's body are located using a joint point detection algorithm. The key joints include shoulders, elbows, wrists, hips, knee joints, and ankle joints, and the position, angle, and relative position relationship characteristics of the joints are extracted; The facial features and body posture features are integrated to form a comprehensive biometric feature vector, and the two feature vectors are concatenated or weightedly fused using a feature-level fusion method; Match the fused feature vector with the feature vector in the pre-stored athlete feature library, and use a similarity measurement method to calculate the similarity between the feature vector to be identified and the feature vector in the library; According to the similarity measurement results, the athlete with the highest similarity is selected as the recognition result; Extract key features of the athlete's preparation posture from the collected depth information and joint point data; The extracted posture features are compared with the preset standard preparation posture feature template to determine whether the athlete's preparation posture meets the standards.

5. The serpentine running training and assessment system based on visual recognition technology according to claim 1 is characterized in that: The motion trajectory tracking module (4) calculates the three-dimensional coordinates of 14 key points of the human body in real time based on the improved DeepSORT algorithm and the Hungarian path matching model, constructs the serpentine running motion trajectory, and detects the missing pole / wrong sequence violation behavior, including the following steps: In the DeepSORT algorithm, the state of each tracked target is maintained by the Kalman filter, including information about the target’s position, velocity, and acceleration; In each frame, a Kalman filter is used to predict the next frame position of each tracked target; Calculate the correlation function value between the detected target and the tracking trajectory; Based on the calculated association cost matrix, the Hungarian algorithm is used for optimal matching to assign the detected target to the most likely tracking trajectory; The athlete's motion trajectory is represented as a three-dimensional coordinate sequence of key points. When there are multiple possible paths, the Hungarian algorithm is used to match and select different paths. The depth information obtained by the depth sensor is combined with the two-dimensional image coordinates of the key points to calculate the three-dimensional coordinates of the key points, and the three-dimensional coordinates calculated under different viewing angles are fused; In each frame, the athlete's motion trajectory is updated according to the matching results and the calculated three-dimensional key point coordinates, and the trajectory is smoothed using a smoothing algorithm; According to the rules of serpentine running, the correct order of passing the poles is defined, and the athlete's movement trajectory is compared and analyzed with the position of the poles. In three-dimensional space, the distance between the trajectory point and each pole is calculated to determine whether the athlete passes through the poles in the prescribed order. Based on the relationship between the trajectory and the poles, it is determined whether there is any missing pole or wrong sequence behavior.

6. The serpentine running training and assessment system based on visual recognition technology according to claim 1 is characterized in that: The three-dimensional pole collision detection module (5) adopts a rigid body collision model and a continuous collision detection algorithm to determine a pole collision event by calculating the Euclidean distance between the ankle / knee / hip key points and the pole safety area, including the following steps: The athlete's ankle, knee, and hip joints are considered as collision points in the rigid body model, and the positions of these points in three-dimensional space are determined by the three-dimensional coordinates calculated by the motion trajectory tracking module (4); A safety zone is set for each rod, which is a cylinder or other shape with a radius centered on the rod; In the dynamic process of snake-like running, the three-dimensional coordinate information of the key points of the human body in each frame of the image is continuously obtained to form time series data; Define a collision detection function to determine whether the key point enters the safety area of ​​the rod. When the value of the collision detection function changes from 0 to 1, it is determined that a collision event has occurred. In the three-dimensional collision detection, the Euclidean distance between each key point and the center of each rod is calculated; The collision detection results of multiple key points are combined to finally determine the pole collision event. At a certain moment, if at least one key point collides with the safety area of ​​the pole, it is determined to be a pole collision.

7. The serpentine running training and assessment system based on visual recognition technology according to claim 1 is characterized in that: The score calculation and compensation module (6) implements μs-level start and end time synchronization based on the PTPv2 protocol, combines speed projection compensation, and outputs the final assessment score, including the following steps: Achieve μs-level start and end time synchronization based on PTPv2 protocol; When the athlete starts running, the system detects the change in the position of the athlete's feet to determine whether it has crossed the initial position of the first pole, and uses this as the basis for recording the starting time. The starting time is determined by the device timestamp after time synchronization. When the athlete completes the serpentine run and returns, the end time is determined by detecting the moment of his / her crossing the finish line. The end time is determined by the device timestamp after time synchronization. The serpentine running result is the time difference between the starting time and the finishing time; During the serpentine run, the system calculates the athlete's speed in real time and calculates the athlete's instantaneous speed by obtaining the athlete's position information at different time points; The athlete's speed was projected onto the serpentine path to account for the curvilinear nature of the path; Due to the delay of the detection system or the slight error of time synchronization, there is a deviation between the actual recorded time and the real time. The results are corrected through speed projection compensation; The start and end times calculated by time synchronization and the results after speed projection compensation are integrated to obtain the final assessment results; The final results will be fed back to the athletes in a timely manner through voice or display screen, and the results data will be stored in the database for subsequent statistics and analysis.

Images