Intelligent agent training analysis algorithm based on large model

Abstract

The invention relates to the technical field of computer vision data analysis, in particular to an agent training analysis algorithm based on a large model. The computer vision technology is used for extracting bone key points of officers and soldiers from videos, and accurate action analysis is carried out. And deep learning, especially a convolutional neural network CNN and a generative adversarial network GAN, is used for improving the accuracy of skeleton key point recognition and action recognition. The physical training technology comprises an officer and soldier training method and effect evaluation, a traditional method depends on observation of a coach, and a modern technology can improve the training efficiency and quality through video analysis, especially in a training scene needing accurate action analysis. The video analysis technology serves as a core link and is responsible for extracting action features from video data. Through advanced algorithms and models, the feature data are processed and analyzed, and detailed feedback and improvement suggestions are provided, so that coaches and officers and soldiers are helped to optimize training effects.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision data analysis, and in particular to an intelligent agent training analysis algorithm based on a large model. Background Art

[0002] With the rapid development of science and technology, computer vision technology has been widely used in many fields, including medical treatment, autonomous driving, security monitoring, etc. The physical fitness field has also gradually adopted these advanced technologies to improve training effects and competition performance. Traditional physical training methods mainly rely on the subjective judgment of coaches and the self-perception of officers and soldiers, which has certain limitations.

[0003] In traditional physical training, coaches usually observe soldiers' movements with their naked eyes to judge the correctness of their technical movements. However, this method has the following shortcomings: Inefficiency: Coaches need a lot of time and energy to observe and evaluate soldiers’ movements, especially in training with high physical activity, which makes it difficult to conduct a comprehensive and detailed analysis.

[0004] Highly subjective: Human visual observation is easily affected by subjective factors, which may lead to inconsistent evaluation results. Different coaches may have different views on the same action, thus affecting the training effect.

[0005] Incomplete data recording: Traditional methods are usually unable to accurately record and analyze soldiers’ movement data, such as movement speed, angle, amplitude, etc., which are crucial for movement improvement and skill enhancement.

[0006] In recent years, computer vision technology has made significant progress, especially in image processing, target detection, and posture estimation. Through cameras and image processing algorithms, computer vision technology can automatically analyze and identify the actions of officers and soldiers in videos. Specific progress includes: Advances in image processing technology: Advanced image processing algorithms can extract clear images of officers and soldiers from complex backgrounds and perform precise motion analysis.

[0007] Target detection and tracking technology: Through the target detection algorithm, the position of officers and soldiers can be accurately located in the video and their movement trajectory can be tracked in real time.

[0008] Posture estimation technology: Through advanced algorithms, it is possible to accurately extract the skeletal key points of soldiers from images, and then build a precise model of the soldiers' movements and conduct in-depth analysis.

[0009] Based on these technological advances, deep learning technology, especially deep learning methods based on large models, has further promoted the development of physical video analysis. Large models, such as convolutional neural networks (CNN) and generative adversarial networks (GAN), have demonstrated excellent capabilities in feature extraction and data processing. Deep learning models can automatically learn effective feature representations from large amounts of data, thereby improving the accuracy and robustness of action recognition of officers and soldiers. Specific applications include: 1. Skeleton key point recognition: The deep learning model can accurately extract the skeleton key points of officers and soldiers from the video and provide detailed motion data.

[0010] 2. Action classification and analysis: Deep learning-based methods can classify different actions and conduct in-depth analysis of soldiers’ actions.

[0011] 3. Data-driven feedback: Deep learning technology can generate specific training suggestions based on analysis results, helping coaches and soldiers make targeted improvements.

[0012] In summary, the combination of computer vision and deep learning technology provides a new solution for physical training, especially in improving training efficiency, reducing subjective bias and achieving accurate analysis. However, in practical applications, the existing technology still faces challenges such as high data quality requirements, and the need to further improve the real-time performance and accuracy of the system. Therefore, the present invention aims to provide an intelligent agent training analysis algorithm based on a large model to meet these challenges and play an important role in actual training. Summary of the invention

[0013] Computer vision technology is used to extract the skeleton key points of officers and soldiers from the video for accurate motion analysis. Deep learning, especially convolutional neural networks (CNN) and generative adversarial networks (GAN), is used to improve the accuracy of skeleton key point recognition and motion recognition. Physical training technology includes training methods and effect evaluation of officers and soldiers. Traditional methods rely on coach observation, while modern technology can improve the efficiency and quality of training through video analysis, especially in training scenarios that require accurate motion analysis. As a core link, video analysis technology is responsible for extracting motion features from video data. Through advanced algorithms and models, these feature data are processed and analyzed, and detailed feedback and improvement suggestions are provided to help coaches and officers and soldiers optimize training effects.

[0014] The purpose of this invention is to develop an advanced physical analysis and training guidance system by introducing large model technology to cope with various challenges in traditional physical training methods, thereby improving training effects and efficiency. Specific goals include: Assist in improving the accuracy of soldiers' motion analysis: Traditional motion analysis methods rely on the subjective observation of coaches, which is easily affected by personal experience and visual errors. By using deep learning models to identify key points of bones, accurate analysis of soldiers' movements can be achieved, reducing human errors and providing more accurate technical feedback.

[0015] Improve the automation and real-time performance of video analysis: In existing technologies, soldiers’ motion analysis often requires manual intervention, which is inefficient and prone to errors. A system based on a large model can automatically process and analyze video data and provide real-time feedback. This allows soldiers to obtain analysis results instantly during training, helping them quickly adjust their movements and improve training effectiveness.

[0016] Providing comprehensive analysis data: Traditional analysis methods may only provide limited information, such as whether the action is correct, while the present invention can provide more comprehensive feedback through multi-dimensional data analysis, including action quality, speed, stability, etc. This multi-dimensional analysis helps to gain a deeper understanding of the performance of officers and soldiers and formulate personalized training plans based on specific data.

[0017] Promote the establishment of intelligent training environment: With the advancement of science and technology and the popularization of intelligent equipment, physical training is also developing in the direction of intelligence and data-driven. This invention aims to provide support for the construction of intelligent training environment by integrating large model technology. The system can not only improve the scientificity and efficiency of training, but also lay the foundation for the development of future intelligent training equipment and platforms.

[0018] Meeting diverse training needs: The existing technology is usually targeted at specific physical fitness projects, but the system designed by the present invention is highly compatible and can adapt to different types of sports, such as sit-ups, pull-ups, push-ups, etc. This compatibility and scalability enables the system to be widely used in various physical fitness trainings to meet different training scenarios and needs.

[0019] An agent training analysis algorithm based on a large model includes the following steps: S1. Input physical training video The video format is MP4, AVI or MOV; the video resolution is not less than 1080p; the video encoding should adopt the H.264 or H.265 standard; the video frame rate is not less than 30fps; the mobile phone, camera or camcorder is transmitted via USB, HDMI or Wi-Fi; after receiving the video data, the video format is pre-processed by denoising, enhancing contrast or adjusting brightness; S2. Skeleton key point recognition A basic feature extraction network is built based on the CNN model of ResNet or EfficientNet. Then, a GAN module is connected after the feature extraction layer to generate diverse training samples for special postures and actions. The attention mechanism is introduced to make the model automatically focus on the recognition of skeleton key points. Based on the identified skeleton key points, the specific action is recognized by the analysis method of spatiotemporal features. S3. Action recognition with spatiotemporal features Based on the recognition of skeleton key points, the spatiotemporal features of the skeleton key point sequence are further extracted. This includes using the optical flow method to capture the motion trajectory of the key points and using models such as convolutional neural network (CNN) or long short-term memory network (LSTM) to extract features in the time dimension. Feature fusion: effectively fuse the key point position and angle of spatial features with the motion speed and acceleration of temporal features to form a complete spatiotemporal feature representation.

[0020] Action classification: Based on the extracted spatiotemporal features, the classifiers support vector machine (SVM), random forest (RF) or deep neural network (DNN) are used to classify the actions. The classifier needs to be trained with a large amount of labeled data to ensure accurate recognition of different actions. Action detail analysis: In addition to basic action classification, in-depth analysis of action details, key features of the action's starting, execution, and ending stages, and transition relationships between actions; Dataset construction: Construct a video dataset containing various physical training movements, including sit-ups, pull-ups, and push-ups, and annotate each movement.

[0021] Computer vision technology is used to extract the skeleton key points of officers and soldiers from the video and conduct accurate motion analysis. Deep learning, especially convolutional neural network (CNN) and generative adversarial network (GAN), is used to improve the accuracy of skeleton key point recognition and motion recognition. Physical training technology includes training methods and effect evaluation of officers and soldiers. Traditional methods rely on coach observation, while modern technology can improve the efficiency and quality of training through video analysis, especially in training scenarios that require accurate motion analysis. As a core link, video analysis technology is responsible for extracting motion features from video data. Through advanced algorithms and models, these feature data are processed and analyzed, and detailed feedback and improvement suggestions are provided to help coaches and officers and soldiers optimize training effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 The figure is a flow chart of the agent training analysis algorithm of the present invention. DETAILED DESCRIPTION

[0023] Main process steps Input: Receive physical fitness videos in various formats through the input module.

[0024] To ensure that the input module can process physical video data effectively and accurately, the following are the specific requirements and implementation details of physical video: Video format and resolution: The system should support mainstream video formats including but not limited to MP4, AVI, MOV, etc. to ensure wide compatibility.

[0025] The video resolution is recommended to be no less than 1080p (1920x1080 pixels) to ensure the accuracy of key point recognition and detail retention.

[0026] In practical applications, the system should automatically identify the video format and perform necessary format conversion or preprocessing to adapt to the internal processing flow.

[0027] Video coding standards: Video encoding should adopt H.264 or the more advanced H.265 standard to optimize storage space and transmission efficiency.

[0028] The system should have a built-in video decoder to support fast and accurate decoding of input videos.

[0029] Video frame rate: The video frame rate should be no less than 30fps (frames per second) to ensure smooth motion and continuity of key point recognition.

[0030] The system should be able to adjust the video frame rate according to actual needs during the processing process to adapt to different application scenarios.

[0031] Video content requirements: The video content should clearly show the entire process of officers and soldiers conducting physical training, including but not limited to the starting posture, action execution, and ending posture.

[0032] To reduce background interference, the video background should be as simple and stable as possible, avoiding frequent changes or complex scenes.

[0033] The video should contain complete timestamp information to facilitate subsequent time synchronization and motion analysis.

[0034] Input interface and equipment: The system should provide a variety of input interfaces, such as USB, HDMI, Wi-Fi, etc., so as to receive video data from various devices such as mobile phones, cameras, and camcorders.

[0035] The selection of terminals or components (such as cameras) should be determined based on the actual training environment to ensure that high-quality video data can be captured.

[0036] Video preprocessing: After receiving the video data, the system should automatically perform video preprocessing, including but not limited to denoising, contrast enhancement, brightness adjustment, etc., to improve the subsequent processing effect.

[0037] The preprocessing process should fully consider the impact of different training environments and lighting conditions on video quality, and adopt an adaptive preprocessing algorithm.

[0038] Timing control: The system should strictly control the timing of video processing to ensure that the entire process from receiving the video to outputting the analysis results is fast and efficient.

[0039] For application scenarios with high real-time requirements, the system should optimize the processing flow, reduce latency, and achieve instant feedback.

[0040] Skeleton key point recognition Aiming at the special needs of soldiers' physical training, the skeleton key point recognition module of the present invention has made significant innovations and optimizations on the basis of existing technologies to ensure high-precision and high-efficiency recognition effects in complex and changeable training environments. The specific innovations and implementation steps are as follows: (1) Customized model architecture Innovation: A deep learning model architecture optimized for the physical training of officers and soldiers is designed. This architecture combines the advantages of convolutional neural networks (CNN) and generative adversarial networks (GAN), and introduces an attention mechanism to enhance focus on key areas.

[0041] Implementation steps: First, build a basic feature extraction network based on efficient CNN models such as ResNet or EfficientNet; then, connect the GAN module after the feature extraction layer to generate more diverse training samples, especially for the special postures and movements in soldiers' training; finally, introduce the attention mechanism so that the model can automatically focus on the key areas of the skeleton key points to improve recognition accuracy.

[0042] Targeted dataset construction Innovation: A dataset containing a wealth of soldiers' physical training videos was constructed, and each video was carefully annotated to ensure the accuracy of key skeletal points. The dataset not only covers common physical training movements, but also pays special attention to the special postures and movement changes of soldiers during training.

[0043] Implementation steps: We built a high-quality training dataset by combining the company's internal training video resources with public datasets (such as MPII, COCO, etc., but emphasizing the supplement of officers and soldiers' training scenes). We used a combination of semi-automatic annotation and manual verification to ensure the accuracy and consistency of annotation.

[0044] Optimize pre-processing and post-processing processes Innovation: Based on the characteristics of soldiers’ training videos, a set of optimized pre-processing and post-processing processes were designed to further improve the recognition effect.

[0045] Implementation steps: In the pre-processing stage, in addition to conventional frame segmentation and standardization processing, illumination correction and background suppression technology are introduced to reduce the impact of illumination changes and complex backgrounds on recognition results. In the post-processing stage, spatiotemporal consistency verification and key point trajectory smoothing technology are used to ensure the stability and continuity of recognition results.

[0046] Balance between real-time performance and accuracy Innovation: Through algorithm optimization and hardware acceleration technology, a good balance between real-time and accuracy of skeleton key point recognition is achieved.

[0047] Implementation steps: Use parallel processing technology and optimization algorithms to reduce computational complexity and increase processing speed. At the same time, use high-performance computing resources such as GPUs to accelerate deep learning models to ensure that high recognition accuracy can be maintained in application scenarios with high real-time requirements.

[0048] Action recognition: Based on the identified skeleton key points, the specific actions of officers and soldiers are identified using the analysis method of spatiotemporal features.

[0049] 4. Data analysis: Comprehensively analyze the identified motion data to generate feedback on motion quality, speed, stability, etc.

[0050] Feedback and guidance: Provide specific training suggestions based on the analysis results to help coaches and soldiers improve their techniques.

[0051] Main innovations Large model skeleton key point recognition Technical refinement requirements: Model selection: Use large deep learning models with billions of parameters, such as ResNet and EfficientNet, combined with generative adversarial networks (GANs) for refined training to capture subtle changes in key points of the human skeleton in the video.

[0052] Data preprocessing: Perform frame segmentation and standardization (including normalization, denoising, etc.) on the input video to ensure the consistency and high quality of the model input data.

[0053] Feature extraction: Utilize the powerful feature extraction capability of convolutional neural network (CNN) to extract key features such as human body contours and edges from video frames, providing a basis for subsequent key point prediction.

[0054] Key point prediction: Through fully connected layers or regressors, combined with a large amount of labeled data training, accurate prediction of human skeleton key points in the video is achieved. The prediction results must reach pixel-level accuracy to meet the needs of subsequent action analysis.

[0055] Effect verification instructions: Experimental design: Design multiple sets of comparative experiments, including video data under different lighting conditions, different motion speeds, different background complexities, etc., to comprehensively evaluate the effect of large model skeleton key point recognition.

[0056] Evaluation indicators: Key point positioning accuracy (such as average precision AP), recall rate, F1 score, etc. are used as evaluation indicators to ensure the high accuracy and robustness of the recognition results.

[0057] Result analysis: Through experimental data analysis, the significant advantages of the large model in the recognition of skeleton key points are verified, such as the recognition accuracy is improved by XX% and the robustness is enhanced by XX% compared with traditional methods or small models. At the same time, the influence of different factors on the recognition effect is analyzed, and corresponding optimization strategies are proposed.

[0058] Action recognition method based on spatiotemporal features Technical refinement requirements: Feature extraction: Based on the recognition of skeleton key points, the spatiotemporal features of the skeleton key point sequence are further extracted. This includes using the optical flow method to capture the motion trajectory of the key points, and using models such as convolutional neural networks (CNN) or long short-term memory networks (LSTM) to extract features in the time dimension.

[0059] Feature fusion: effectively integrate spatial features (such as key point position, angle, etc.) with temporal features (such as motion speed, acceleration, etc.) to form a complete spatiotemporal feature representation.

[0060] Action classification: Based on the extracted spatiotemporal features, a classifier (such as support vector machine SVM, random forest RF or deep neural network DNN) is used to classify the action. The classifier needs to be trained with a large amount of labeled data to ensure accurate recognition of different actions.

[0061] Action detail analysis: In addition to basic action classification, in-depth analysis of action details should be conducted, such as the key features of the starting, execution and ending stages of the action, as well as the transition relationship between actions.

[0062] Effect verification instructions: Dataset construction: Build a video dataset containing a variety of physical training movements (such as sit-ups, pull-ups, push-ups, etc.), and accurately label each movement.

[0063] Experimental design: Design comparative experiments to compare the performance of different spatiotemporal feature extraction methods and classifiers in action recognition. At the same time, consider the impact of different lighting conditions, perspective changes, occlusion and other factors on the recognition effect.

[0064] Evaluation indicators: Accuracy, precision, recall and F1 score are used as evaluation indicators to comprehensively evaluate the effect of action recognition.

[0065] Result analysis: Through experimental data analysis, the significant advantages of spatiotemporal feature action recognition method in improving action classification accuracy and detail analysis ability are verified. At the same time, the influence of different factors on the recognition effect is analyzed, and corresponding optimization strategies are proposed.

[0066] Multi-dimensional data feedback: Provides comprehensive data feedback in multiple dimensions including movement quality, speed, stability, etc. to help deeply understand the performance of officers and soldiers.

[0067] Physical fitness video motion analysis: The system receives videos of soldiers performing sit-ups, pull-ups, and push-ups, and provides data feedback such as the number of movements and speed through skeletal key point recognition and motion analysis modules.

[0068] The present invention solves the following technical problems through the above technical solution: Real-time data processing: How to ensure that the system can perform real-time skeletal key point recognition and motion analysis when processing high-definition video.

[0069] Model accuracy and robustness: How to improve the recognition accuracy of deep learning models in various environments and lighting conditions.

[0070] Compatibility issues: How can the system support multiple video formats and perform pre-processing effectively?

[0071] Practicality of data feedback: How to convert complex analytical data into simple and easy-to-understand feedback for coaches and soldiers to use.

[0072] The technical effects achieved include: Improve the accuracy of motion analysis: The application of large models greatly improves the accuracy of skeletal key point recognition and motion analysis.

[0073] Improve training efficiency: Automated data processing and analysis reduces the time of manual intervention and improves training efficiency.

[0074] Comprehensive data feedback: Provide multi-dimensional data analysis to help coaches and soldiers fully understand action performance and improve training methods.

[0075] Wide range of application scenarios: The system has high compatibility and can adapt to the training needs of physical fitness projects such as sit-ups and pull-ups, especially providing accurate feedback in military training.

[0076] The present invention provides an agent training analysis algorithm based on a large model, which aims to improve the accuracy and efficiency of soldiers' action analysis, especially in military training. The system includes the following main modules: Input module: Physical fitness videos in various formats are received through the input module. Video data can come from mobile phones, cameras and other devices.

[0077] It supports multiple video formats, such as MP4, AVI, etc., and has pre-processing functions, such as video editing, enhancement, etc., to facilitate the processing of subsequent modules.

[0078] Skeleton key point recognition module: In the skeleton key point recognition module, we innovatively used a method combining convolutional neural network (CNN) and generative adversarial network (GAN) in deep learning technology to significantly improve the prediction accuracy of the position of each joint point of the human body. This technology selection is not a simple conversion of conventional technology, but is based on a deep understanding of the current technology development trend and a precise grasp of actual needs. Model selection and optimization: We carefully selected CNN models suitable for image feature extraction, such as ResNet or EfficientNet, which have demonstrated excellent performance in the field of image recognition. At the same time, in order to further improve the accuracy of key point recognition, we introduced the GAN model for refined training. The GAN model can generate data that is closer to the real distribution through adversarial learning between the generator and the discriminator, thereby enhancing the model's adaptability to complex backgrounds and motion states.

[0079] Feature extraction and key point prediction: After video preprocessing, the CNN model is used to extract rich feature information from each frame. These features not only contain the contour information of the human body, but also the potential location information of the key points. Then, through the fully connected layer or regressor, we predict the precise location of each joint point of the human body based on the extracted features.

[0080] Effect verification and innovation analysis: Effect verification: In order to verify the performance of this module, we constructed a video dataset containing a variety of physical training movements and accurately annotated the key points of each movement. Through comparative experiments, we found that this module is significantly better than traditional methods and models that only use CNN or GAN in terms of key point recognition accuracy, recall rate, and F1 score. In particular, this module shows stronger robustness and accuracy when dealing with complex backgrounds and fast-moving scenes.

[0081] Innovation analysis: The innovation of this module is to combine CNN with GAN to achieve refined prediction of key point positions. The powerful feature extraction capability of CNN and the generation capability of GAN complement each other, so that the model can still maintain high recognition accuracy in complex scenes. In addition, we have further improved the accuracy and robustness of recognition through technical means such as data enhancement and model fusion. These innovations not only solve the pain points in existing technologies, but also provide a more reliable foundation for subsequent action recognition and data analysis.

[0082] Algorithm steps: Video Preprocessing Specific technical requirements and steps: In the video preprocessing stage, we first perform frame segmentation on the input video to obtain each frame image in the video. The specific technical requirements for frame segmentation are as follows: Frame rate matching: Based on system requirements and video content, we set an appropriate frame rate (such as 30fps or higher) to ensure the continuity and accuracy of motion analysis. Through the video decoder, we divide the video into a series of individual frames according to this frame rate.

[0083] Frame integrity check: During the frame segmentation process, the system will perform a frame integrity check to ensure that each frame of the image is intact and avoid image quality problems caused by transmission errors or file corruption.

[0084] Next, each frame of the image is normalized to improve the accuracy of subsequent feature extraction. The specific requirements for normalization are as follows: Image resizing: To ensure consistency in deep learning model processing, we resize all frame images to the size required by the model input (e.g., 224x224 pixels). This step ensures that the model can extract the same amount of feature information when processing different videos.

[0085] Pixel value standardization: Pixel value standardization is performed according to the given formula. Specifically, we normalize the original image (denoted as ) minus the mean value of the image pixels (denoted as ), and then divided by the standard deviation of the image pixels (denoted as ), and obtain the standardized image (denoted as ). The standardization formula is as follows: Feature extraction: Use convolutional neural network (CNN) to extract image features.

[0086] in, is the extracted feature.

[0087] Key point prediction In the key point prediction stage, we use the fully connected layer (for classification tasks) or regressor (for regression tasks) in the deep learning model to predict the location of the human skeleton key points in the video frame. To ensure the accuracy and robustness of the prediction, we define the optimization process of the weight matrix and bias vector and their specific processing effects in detail.

[0088] Initialization of weight matrix (W) and bias vector (b): The weight matrix W is usually initialized randomly, such as Xavier initialization or He initialization. These methods aim to keep the magnitude of the gradients in forward and backward propagation relatively stable, which helps the model converge quickly.

[0089] The bias vector b is usually initialized to zero or a small random number to avoid introducing too large an offset in the early stages of training.

[0090] Optimization algorithm: We use gradient descent methods (such as SGD, Adam, etc.) to optimize the weight matrix W and the bias vector b. During the training process, the gradient of the loss function with respect to W and b is calculated through the back-propagation algorithm, and the values ​​of W and b are updated according to the gradient to minimize the loss function.

[0091] Regularization and hyperparameter tuning: To prevent overfitting, we add regularization terms (such as L1 regularization, L2 regularization, or Dropout) during the training process to improve the generalization ability of the model by limiting the size of the weight matrix W or randomly discarding the output of some neurons.

[0092] At the same time, we adjust hyperparameters such as learning rate, batch size, and number of iterations through experiments and cross-validation to find the best model configuration.

[0093] Prediction effect evaluation: The prediction effect is mainly evaluated by comparing the error between the predicted key point positions and the actual key point positions. We use a variety of evaluation indicators, such as mean square error (MSE), mean absolute error (MAE), and key point positioning accuracy, to comprehensively evaluate the prediction performance of the model.

[0094] Through a large amount of experimental data and comparative analysis, we found that this algorithm can accurately predict the positions of skeleton key points in a variety of complex scenes and lighting conditions, and has higher accuracy and robustness than existing technologies.

[0095] Action recognition module: Action recognition is based on the spatiotemporal characteristics of skeleton key points, and identifies specific actions by analyzing the changes of key points over time.

[0096] Algorithm steps: 1. Spatiotemporal feature extraction: Extracting the spatiotemporal features of the skeleton key point sequence in, is a spatiotemporal feature sequence, It's time The position of the skeleton key points at the moment.

[0097] Action classification: Use classifiers such as Long Short-Term Memory (LSTM) to classify actions in, is the action classification result, and are the weight and bias of the classifier, respectively. is through Extracted spatiotemporal features.

[0098] Data Analysis Module: The data analysis module conducts a comprehensive analysis of the identified motion data and generates feedback on the motion quality, speed, stability, etc.

[0099] Algorithm steps: 1. Action quality evaluation: The action quality is evaluated by calculating the difference between the key point trajectory and the standard trajectory.

[0100] Scoring criteria: The similarity between the key point trajectory and the standard trajectory is used as the scoring basis for action quality evaluation. We define a comprehensive scoring function that takes into account multiple dimensions such as key point position deviation, trajectory smoothness, and action completion.

[0101] Technical implementation: First, the standard trajectory of each key point is extracted from the standard action database. Then, the Euclidean distance or dynamic time warping (DTW) distance between the actual action trajectory and the standard trajectory is calculated as a quantitative indicator of the trajectory difference. Finally, these difference values ​​are combined into an action quality score according to the preset weight parameters.

[0102] For example, if the standard sit-up movement is performed, the angle between the spine and the ground should show a specific curve. By comparing the angle change curve in the actual movement with the standard curve, the quality of the movement can be evaluated.

[0103] in, is the action quality score, is the predicted key point location, is the standard key point position, is the number of key points.

[0104] 2. Motion speed evaluation: Calculate the speed at which key points move in, is the speed of action, and The time and The key point position, is the time interval.

[0105] Movement stability assessment In motion stability evaluation, we use the method of calculating the variance of key point positions to quantify the stability of motion. This method is based on the assumption that the smaller the change of key point positions during the execution of the motion, the higher the stability of the motion.

[0106] Technical implementation: Variance calculation: For each key point, we calculate its position variance during the entire action execution process. Specifically, we first record the position coordinates (x, y) of each key point at each time step, and then calculate the variance of these position coordinates to quantify the degree of fluctuation of the key point position.

[0107] Stability score: In order to convert the variance value into an intuitive stability score, we define a stability score function. This function maps the variance value to a predefined score interval, usually [0, 100], where 0 represents extremely unstable and 100 represents very stable. The specific calculation method of the score can be adjusted according to actual needs, but usually the variance value is normalized and an appropriate inverse transformation (such as an exponential function or an inverse proportional function) is applied to ensure that low variance values ​​correspond to high stability scores.

[0108] Comprehensive stability score: Considering that different key points may have different effects on stability, we can assign different weights to each key point and summarize the weighted variance values ​​to obtain a comprehensive stability score. This method can more comprehensively evaluate the stability of the entire action.

[0109] in, is the motion stability score, It's time Time The location of the key points, It is The mean position of the key points.

[0110] Feedback and Coaching Module: Provide specific training suggestions based on the analysis results to help coaches and soldiers improve their techniques.

[0111] The feedback includes suggestions for improving action quality, speed optimization, stability improvement, etc.

[0112] Example: Physical video motion analysis Video input: The input module receives videos of soldiers doing sit-ups, pull-ups, and push-ups. The videos can come from various devices, such as mobile phones, cameras, etc., and support multiple video formats.

[0113] Skeleton key point recognition: The skeleton key points of officers and soldiers in the video are recognized through the skeleton key point recognition module. Deep learning technology, such as convolutional neural network (CNN) or generative adversarial network (GAN), is used to recognize the skeleton key points of officers and soldiers in the video.

[0114] Action recognition: The action recognition module can identify the soldiers’ sit-ups, pull-ups and push-ups based on the skeleton key points. The action recognition method based on spatiotemporal features is used to first extract features from the skeleton key point sequence, and then predict the soldiers’ actions through a classifier or regressor.

[0115] Data analysis: The data analysis module conducts a comprehensive analysis of soldiers’ movements and generates feedback on movement quality, speed, stability, etc.

[0116] Feedback and guidance: Based on the analysis results, specific training suggestions are provided, such as improving the posture of sit-ups, increasing the speed of pull-ups on the horizontal bar, and enhancing the stability of push-ups.

[0117] The technical features of the present invention are described as follows: This input module significantly improves the system's compatibility and data processing capabilities, ensuring that various formats of physical fitness videos from different devices (such as smartphones, sports cameras, professional cameras, etc.) can be seamlessly accessed, thereby expanding the system's application scope. At the same time, through the built-in preprocessing function, the quality of input data is effectively improved, the computational burden of subsequent analysis modules is reduced, and the accuracy and efficiency of analysis results are significantly improved, laying a solid foundation for subsequent action recognition and data analysis.

[0118] Multi-format video support: The input module integrates a wide range of video decoder libraries, including but not limited to open source tools such as FFmpeg, and supports multiple mainstream video formats such as MP4, AVI, MOV, WMV, etc. When a video file is uploaded to the system, the input module first detects the file format and automatically calls the corresponding decoder for decoding and processing, converting the video data into a unified internal format for subsequent processing. In practical applications, this function ensures that no matter what device the trainer uses to shoot the video, the system can smoothly receive and process it, greatly improving the flexibility of the system and user experience.

[0119] Video preprocessing: Pre-processing functions include video clipping (removing useless clips), color correction (adjusting brightness, contrast, saturation, etc.), noise suppression (reducing random noise in the video), and frame rate adjustment.

[0120] a. Video editing: Provide a user interface that allows the user or the system to automatically select valid clips in the video to retain, and crop the rest. b. Color correction: Apply preset or customized color correction parameters according to the actual situation of the video content to improve the visual effect. c. Noise suppression: Use algorithms such as median filtering and Gaussian filtering to remove random noise in the video and improve video clarity. d. Frame rate adjustment: Uniformly adjust the video frame rate to a frame rate suitable for subsequent analysis and processing (such as 30fps) to ensure processing efficiency and consistency of results.

[0121] The quality of preprocessed video data is significantly improved, providing clearer and more accurate input for subsequent action recognition and data analysis, helping to improve the performance and accuracy of the overall system.

[0122] Skeleton key point recognition module: This module significantly improves the accuracy of skeleton key point recognition by introducing advanced deep learning technology, and reduces errors caused by human subjective factors. Its automated processing method not only improves efficiency, but also ensures the accuracy of motion analysis, providing a scientific basis for subsequent physical training.

[0123] The convolutional neural network (CNN) is used as the main framework, and combined with the generative adversarial network (GAN) for auxiliary training to optimize the generalization ability and recognition accuracy of the model. First, a CNN model containing multi-level convolution, pooling and fully connected layers is constructed to extract the feature representation of skeletal key points from the input video frames. Subsequently, the generator part of GAN is used to generate additional training samples to enhance the diversity of the training set and improve the robustness of the model. The CNN model is built based on classic architectures such as VGG and ResNet, and the GAN model refers to classic literature or open source implementations such as DCGAN and Pix2Pix.

[0124] Frame segmentation and feature extraction: The input video is segmented into continuous frames, and each frame is standardized (such as resizing, normalization, etc.), and then feature extraction is performed through CNN. In the preprocessing stage, FFmpeg and other tools are used for video decoding and frame segmentation. When extracting features, CNN performs convolution, activation, and pooling operations on the image layer by layer, and finally obtains a feature map containing the location information of the skeleton key points. Frame segmentation ensures the continuity analysis of the action, and feature extraction extracts key visual information, laying the foundation for subsequent key point prediction.

[0125] Key point prediction and optimization uses a fully connected layer or regressor to map the extracted features to the position coordinates of the skeleton key points. At the same time, data enhancement (such as random cropping, rotation, flipping, etc.) and model fusion (such as ensemble learning, model averaging, etc.) techniques are used to further improve the accuracy and stability of recognition. A regressor is added after the fully connected layer to output the two-dimensional or three-dimensional coordinates of the key points. Data enhancement is performed dynamically during the training process to simulate video data in different training environments. Model fusion reduces the limitations of a single model by training multiple models and integrating their prediction results. Taking the video of soldiers doing push-up training as an example, this module can accurately identify the positions of key points such as shoulders, elbows, and wrists, providing accurate data for subsequent action analysis and feedback.

[0126] Timing control and performance optimization, through parallel processing and batch processing technology to optimize the operating efficiency of the model, to ensure that the recognition results can still be output in real time when processing high-definition video. Use GPU acceleration technology to parallel process the feature extraction and key point prediction tasks of multiple video frames. At the same time, by optimizing the algorithm and reducing redundant calculations, the overall processing speed and efficiency are improved. These optimization measures significantly improve the real-time performance and processing capabilities of the system, ensuring efficient application in actual training scenarios.

[0127] Action recognition module, this module can accurately distinguish and identify complex sports movements through action recognition methods based on spatiotemporal features, providing efficient recognition functions, which helps coaches and soldiers to conduct detailed action analysis and improvement. This method not only improves the accuracy of action recognition, but also enhances the practicality of the system, making the training process more scientific and refined.

[0128] For spatiotemporal feature extraction, the optical flow method (such as Lucas-Kanade optical flow method, Farneback optical flow method, etc.) is combined with trajectory coding technology (such as DTW dynamic time warping, Fisher Vector coding, etc.) to comprehensively extract spatiotemporal features of the skeleton key point sequence. The optical flow method is used to capture the motion trajectory of the key points, and the trajectory coding is used to convert the trajectory information into a feature representation that can be used for classification. First, the optical flow is calculated for the position of the skeleton key points in each frame of the image to obtain its motion trajectory. Then, the trajectory coding technology is used to encode the trajectory information to form a spatiotemporal feature vector. This process requires adjusting the optical flow method parameters and coding strategies based on actual conditions to ensure the accuracy and effectiveness of the extracted features. For the optical flow method and trajectory coding technology, please refer to the relevant implementations in open source libraries such as OpenCV and Scikit-learn.

[0129] Action classification and prediction: Long short-term memory network (LSTM) or similar recurrent neural network (RNN) is used as a classifier to learn and classify the extracted spatiotemporal features to predict the specific actions of officers and soldiers. At the same time, technical means such as multi-task learning (such as predicting action category and action quality at the same time) and transfer learning (using pre-trained models for fine-tuning) are introduced to improve recognition accuracy and generalization ability. Build an LSTM model and set appropriate input and output layers, hidden layer structures and loss functions. Use the training data set to train the model and optimize the model parameters through the back propagation algorithm. During the training process, technical means such as data enhancement (such as adding noise, changing lighting conditions, etc.) and early stopping can be used to prevent overfitting. After training, use the test data set to evaluate the model performance and make necessary fine-tuning.

[0130] For the construction and training of LSTM models, please refer to the relevant tutorials and documents in deep learning frameworks such as TensorFlow and PyTorch.

[0131] Results evaluation and optimization: By comparing the predicted results with the true labels, we calculate the accuracy, recall, F1 score and other evaluation indicators to measure the performance of action recognition. At the same time, we use visualization tools such as confusion matrix to analyze the misclassification and identify and correct potential problems in the model. Run the model on the test data set, record the predicted results, and compare them with the true labels. Calculate the evaluation indicators and draw visualization charts such as confusion matrix. Adjust the model structure and parameter settings according to the evaluation results to optimize the recognition performance. This process requires multiple iterations and fine-tuning based on the actual situation. Take the physical training videos of soldiers doing sit-ups, pull-ups and push-ups as an example to show the effect of the action recognition module in actual application. Through comparative analysis, we verify the advantages of the module in improving the accuracy and efficiency of action recognition. At the same time, combined with the results of the data analysis module, we provide specific training suggestions and improvement directions for coaches and soldiers.

[0132] Data analysis module: This module integrates multiple submodules (action quality assessment submodule, action speed assessment submodule, and action stability assessment submodule) to provide comprehensive action analysis results, helping coaches and soldiers to deeply understand the soldiers' physical training performance, so as to make more targeted technical improvements. The generated detailed feedback report not only improves the pertinence and effectiveness of training, but also enhances the scientificity and systematicness of the training process.

[0133] The action quality assessment submodule adopts an assessment method based on the comparison between key point trajectories and standard trajectories. It quantitatively assesses the action quality by calculating the difference between the key point trajectory and the preset standard trajectory (such as Euclidean distance, Manhattan distance, etc.). First, a standard trajectory database for various physical training actions is established. Then, for each action to be evaluated, its key point trajectory is extracted and compared with the corresponding standard trajectory to calculate the difference value. Finally, a score or rating of the action quality is given according to the size of the difference value. This method can refer to relevant research literature in the field of sports biomechanics.

[0134] The motion speed evaluation submodule evaluates the motion speed by calculating the displacement of key points per unit time. Specifically, linear interpolation, differentiation and other methods can be used to calculate the instantaneous speed of key points, and then the average speed or speed change curve of the entire action can be obtained. First, the key point trajectory is timestamped to ensure that the time information of each key point is accurate. Then, a suitable mathematical method is used to calculate the instantaneous speed of the key points, and the speed evaluation result of the entire action is obtained based on this. The speed evaluation method can refer to the kinematic formulas and numerical analysis methods in physics, as well as scientific computing libraries such as SciPy in Python.

[0135] The motion stability evaluation submodule evaluates the stability of the motion by calculating the variance or standard deviation of the key point position in the time series. The smaller the variance or standard deviation, the more stable the motion. For each key point, its position information is recorded during the entire motion execution process. Then, the position variance or standard deviation of the key point at all time points is calculated. Finally, the stability evaluation results of all key points are combined to give a stability score or rating for the entire motion. The stability evaluation method can refer to the variance and standard deviation calculation methods in statistics, as well as related data analysis software and libraries (such as Excel, Python's Pandas library, etc.).

[0136] In actual applications, the data analysis module can analyze videos of soldiers performing physical training such as sit-ups, pull-ups, and push-ups. For example, when evaluating the quality of sit-ups, the system first extracts the trajectories of the soldiers' skeleton key points and compares them with the standard trajectories of sit-ups. By calculating the difference value, the system can give the soldiers' scores and specific improvement suggestions on the quality of sit-ups (such as adjusting breathing rhythm, strengthening abdominal muscle contraction, etc.). At the same time, the system will also evaluate the soldiers' sit-up speed and stability, and provide a comprehensive data analysis report to help coaches and soldiers make targeted training adjustments and optimizations.

[0137] Feedback and guidance module, this module aims to provide coaches and soldiers with specific and actionable training suggestions based on the results of the data analysis module, helping soldiers to quickly improve their technical level. Through personalized guidance, the training plan is optimized for different sports and training needs, thereby significantly improving the training effect. Based on the evaluation results of movement quality, speed and stability, an intelligent training suggestion generation algorithm is designed. The algorithm will comprehensively consider multiple evaluation dimensions, such as movement error types, speed deficiencies, stability issues, etc., to generate targeted training suggestions.

[0138] First, the evaluation results output by the data analysis module are analyzed in detail to identify the specific problems and shortcomings of officers and soldiers in training. Recommendation library construction: Based on professional physical training theory and practical experience, a complete training recommendation library is constructed. The recommendation library should contain solutions and training methods for different types of problems. Intelligent matching: Using the intelligent matching algorithm, the evaluation results are matched with the content in the training recommendation library to select the training recommendations that best suit the current situation of officers and soldiers. Personalized adjustment: Considering the personal characteristics, physical fitness level and training goals of officers and soldiers, the training recommendations are personalized to ensure their pertinence and effectiveness. The training recommendations are presented to coaches and officers and soldiers in an intuitive way in the form of feedback with pictures and texts that are easy to understand. At the same time, multimedia materials such as video demonstrations and animation simulations are provided to help officers and soldiers better understand and master the key points of training. Design feedback templates: Design corresponding feedback templates based on the content and characteristics of the training recommendations. The template should contain elements such as text descriptions, chart displays, and video links. Generate feedback reports: Combine training recommendations with feedback templates to generate detailed feedback reports. The report should be clear and focused, so that coaches and soldiers can quickly understand the problem and solution. Multi-channel feedback: Feedback reports are sent to coaches and soldiers through multiple channels (such as APP, web pages, emails, etc.) to ensure timely communication and receipt of information.

[0139] Glossary Large models: usually refers to artificial intelligence models with billions or more parameters, which are trained on large-scale data sets through deep learning algorithms. These models usually have very strong representation capabilities and can handle complex tasks such as image recognition, natural language processing, speech recognition, etc.

[0140] Skeleton key point recognition: Use deep learning technology to identify the locations of skeleton key points of officers and soldiers in the video.

[0141] Action recognition: Identify soldiers’ specific actions based on skeleton key points.

[0142] Data Analysis: Analyze soldiers’ movements and provide data feedback to evaluate movement quality, speed and stability.

Claims

1. An agent training and analysis algorithm based on a large model, characterized by The steps include: S1. Input physical training video The video format is MP4, AVI or MOV; the video resolution is not less than 1080p; the video encoding should adopt the H.264 or H.265 standard; the video frame rate is not less than 30fps; the mobile phone, camera or camcorder is transmitted via USB, HDMI or Wi-Fi; after receiving the video data, the video format is pre-processed by denoising, enhancing contrast or adjusting brightness; S2. Skeleton key point recognition Build a basic feature extraction network based on the CNN model of ResNet or EfficientNet; Then, the GAN module is connected after the feature extraction layer to generate diverse training samples for special postures and actions. The attention mechanism is introduced to make the model automatically focus on the recognition of skeleton key points. Based on the recognized skeleton key points, the specific action is recognized by the analysis method of spatiotemporal features. S3. Action recognition with spatiotemporal features Based on the recognition of skeleton key points, the spatiotemporal features of skeleton key point sequences are further extracted; This includes using the optical flow method to capture the motion trajectory of key points, and using models such as convolutional neural networks (CNN) or long short-term memory networks (LSTM) to extract features in the time dimension; feature Fusion: effectively integrate the key point positions and angles of spatial features with the motion speed and acceleration of temporal features to form a complete spatiotemporal feature representation; Action classification: Based on the extracted spatiotemporal features, the classifiers support vector machine (SVM), random forest (RF) or deep neural network (DNN) are used to classify the actions. The classifier needs to be trained with a large amount of labeled data to ensure accurate recognition of different actions. Action detail analysis: In addition to basic action classification, in-depth analysis of action details, key features of the action's starting, execution, and ending stages, and transition relationships between actions; Dataset construction: Construct a video dataset containing various physical training movements, including sit-ups, pull-ups, and push-ups, and annotate each movement.

2. According to the large model-based intelligent agent training analysis algorithm of claim 1, it is characterized in that The video dataset in S3 is preprocessed: First, perform frame segmentation on the input video to obtain each frame image in the video; Frame rate matching: set the frame rate to above 30fps; perform standard processing on each frame of the image; Image resizing: resize all frame images to the size specified by the model input; Pixel value standardization: standardize pixel values ​​according to the given formula; Specifically, we take the original image , each pixel value in the original image minus the mean value of the image pixels , and then divided by the standard deviation of the image pixels , get the standardized image , the standardized formula is as follows: Feature extraction: Use convolutional neural network (CNN) to extract image features; in, is the extracted feature; Key point prediction: Use the fully connected layer or regressor in the deep learning model to predict the location of the key points of the human skeleton in the video frame; Initialization of weight matrix W and bias vector b: The weight matrix W is usually initialized randomly, using Xavier initialization or He initialization; the bias vector b is usually initialized to zero or a small random number; Optimization algorithm: We use the gradient descent method SGD or Adam to optimize the weight matrix W and the bias vector b; During the training process, the gradient of the loss function with respect to W and b is calculated through the back-propagation algorithm, and the values ​​of W and b are updated according to the gradient to minimize the loss function; Regularization and hyperparameter tuning: Regularization terms, such as L1 regularization, L2 regularization or Dropout, are added during the training process by limiting the size of the weight matrix W or randomly discarding the output of some neurons; the prediction effect is evaluated by comparing the error between the predicted key point position and the actual key point position; the mean square error MSE, mean absolute error MAE and key point positioning accuracy are used.

3. According to the large model-based agent training analysis algorithm of claim 1, it is characterized in that The action recognition steps in S3 are: Spatiotemporal feature extraction: extracting the spatiotemporal features of the skeleton key point sequence; in is a spatiotemporal feature sequence, It's time The position of the skeleton key points at the moment; Action classification: Use the long short-term memory network LSTM of the classifier to classify the actions; in, is the action classification result, and are the weight and bias of the classifier, respectively. is through Extracted spatiotemporal features; Data Analysis Module: The data analysis module conducts a comprehensive analysis of the identified motion data and generates feedback on the quality, speed, stability, etc. of the motion; Action quality evaluation: The action quality is evaluated by calculating the difference between the key point trajectory and the standard trajectory; The action quality assessment uses the similarity between the key point trajectory and the standard trajectory as the scoring basis; a comprehensive scoring function is defined, which takes into account multiple dimensions such as key point position deviation, trajectory smoothness, action completion, etc. First, the standard trajectory of each key point is extracted from the standard action database. Then, the Euclidean distance or dynamic time warping distance between the actual action trajectory and the standard trajectory is calculated as a quantitative indicator of the trajectory difference. Finally, these difference values ​​are combined into an action quality score according to the preset weight parameters. By comparing the angle change curve in the actual action with the standard curve, the quality of the action can be evaluated; in, is the action quality score, is the predicted key point location, is the standard key point position, is the number of key points;Motion speed evaluation: calculate the speed at which the key points move; in, is the speed of action, and The time and The key point position, is the time interval; Movement stability assessment: In the motion stability assessment, the method of calculating the variance of key point positions is used to quantify the stability of the motion. During the execution of the motion, the smaller the change in the key point position, the higher the stability of the motion; For each key point, we calculate its position variance during the entire action execution process; first record the position coordinates (x, y) of each key point at each time step, and then calculate the variance of these position coordinates to quantify the degree of fluctuation of the key point position; in, is the motion stability score, It's time Time The location of the key points, It is The mean position of the key points.

Images