Sports action analysis method and system based on image recognition

Abstract

The invention discloses a sports action analysis method and system based on image recognition, and relates to the technical field of action recognition, and the method comprises the steps: after analyzing the time sequence change characteristics of human body key points in adjacent image frames, completing the human body key points which are judged to be shielded, obtaining the posture of the completed key points, and obtaining the posture of the key points; based on the structured key point postures and the completed key point postures, a time sequence posture feature sequence reflecting the change of the human body postures along with time is constructed, after continuous evolution analysis is conducted on the time sequence posture feature sequence, motion feature vectors used for representing the sports motion process are extracted, analysis and judgment are conducted on the motion feature vectors, and the motion feature vectors are obtained. Acquiring a sports action type; according to the method, through action feature extraction, the integrity and smoothness of a posture sequence in a complex scene are enhanced, and the accuracy and efficiency of sports action recognition are greatly improved.

Description

Technical Field

[0001] This invention relates to the field of motion recognition technology, and in particular to a sports motion analysis method and system based on image recognition. Background Technology

[0002] With the rapid development of computer vision methods, image recognition-based sports motion analysis technology has shown great application potential in fields such as sports training, sports rehabilitation, and health monitoring. In the past, motion analysis methods mainly relied on image processing algorithms, such as edge detection and optical flow. In recent years, with the emergence of deep learning, especially convolutional neural networks (CNN), human key point detection has made great progress, which can accurately identify the position of various key points of the human body and provide a foundation for subsequent motion analysis. At the same time, through time series analysis, motion recognition can more comprehensively capture the dynamic changes of the human body, thereby achieving more accurate motion type recognition.

[0003] Nevertheless, existing sports motion analysis methods still have room for improvement. First, when faced with situations where some key points are missing due to occlusion, they fail to fully explore the motion dynamics between adjacent frames, leading to detection errors for some key points. Second, they fail to fully consider the rationality of human anatomical structure, thus affecting the accuracy of posture analysis results. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides a sports motion analysis method based on image recognition to solve the problems of key point detection errors and their impact on posture analysis.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] In a first aspect, the present invention provides a sports motion analysis method based on image recognition, comprising,

[0008] A continuous sequence of image frames depicting human motion is acquired and preprocessed to obtain the target image sequence.

[0009] Perform human keypoint detection on each image frame in the target image sequence to obtain the location information and corresponding confidence scores of the human keypoints;

[0010] After constructing the skeleton topology of human key points, structural constraints are applied to the position information of human key points to obtain the structured key point pose.

[0011] A preset judgment threshold is set, and human key points with confidence scores below the judgment threshold are occluded to obtain the human key points that are judged to be occluded.

[0012] After analyzing the temporal variation characteristics of human key points in adjacent image frames, the key points that are determined to be occluded are filled in to obtain the pose of the filled key points.

[0013] Based on the structured keypoint pose and the completed keypoint pose, a temporal pose feature sequence reflecting the change of human pose over time is constructed.

[0014] After performing continuous evolution analysis on the temporal posture feature sequence, motion feature vectors are extracted to characterize the process of sports movements;

[0015] The type of sports movement is obtained by analyzing and determining the feature vector of the movement.

[0016] As a preferred embodiment of the sports motion analysis method based on image recognition described in this invention, the preprocessing includes denoising and brightness normalization.

[0017] As a preferred embodiment of the sports motion analysis method based on image recognition described in this invention, the step of obtaining the location information and corresponding confidence scores of key human body points specifically includes:

[0018] Each image frame in the target image sequence is input into a convolutional neural network for feature extraction. Based on the feature extraction results, the location of human key points is calculated to obtain the location information of human key points.

[0019] Based on the response intensity of human body key points in the image frame, a confidence calculation method is used to calculate the confidence score for the location information of each human body key point.

[0020] As a preferred embodiment of the sports motion analysis method based on image recognition described in this invention, the step of obtaining structured keypoint poses specifically includes:

[0021] The connection relationships between key points in the human body are defined, and key points with joint connections are connected in an orderly manner according to the distribution pattern of the human skeleton to form a skeletal topology.

[0022] Based on the skeletal topology of human key points, the spatial distance variation range and joint angle variation range of related human key points are limited.

[0023] After the constraints are met, the positional information of key points on the human body is corrected based on the rationality of the human anatomical structure to obtain the structured key point pose.

[0024] As a preferred embodiment of the sports motion analysis method based on image recognition described in this invention, the occlusion determination refers to setting a determination threshold, comparing the confidence level of human key points with the determination threshold one by one, and determining that the human key points are occluded in the image frame when the confidence level of the human key points is less than the determination threshold, thereby obtaining the human key points determined to be occluded.

[0025] As a preferred embodiment of the sports motion analysis method based on image recognition described in this invention, the step of performing completion processing on key human body points determined to be occluded specifically involves:

[0026] Based on the location information of human key points in adjacent image frames in the target image sequence, the direction and magnitude of position change of human key points in the continuous time dimension are calculated to obtain the temporal change characteristics of human key points.

[0027] Based on the temporal change characteristics of human body key points, the position information of human body key points that are identified as occluded is predicted and corrected to generate the pose of the completed key points.

[0028] As a preferred embodiment of the sports motion analysis method based on image recognition described in this invention, the step of constructing a temporal posture feature sequence reflecting the change of human posture over time based on structured keypoint postures and the completed keypoint postures specifically includes:

[0029] Based on the temporal order of image frames in the target image sequence, the structured keypoint pose and the completed keypoint pose corresponding to each image frame are fused to form a complete keypoint pose.

[0030] Based on the connection relationship of human key points in the skeleton topology, the pose of complete key points is rearranged according to topological constraints to form a pose representation vector.

[0031] Based on the relationship between pose representation vectors between adjacent image frames, the positional changes, velocity changes, and joint angle changes of human key points in the time dimension are calculated to obtain multidimensional temporal change features.

[0032] Multidimensional temporal variation features are cascaded in chronological order to form a temporal attitude feature sequence.

[0033] As a preferred embodiment of the sports motion analysis method based on image recognition described in this invention, the step of extracting motion feature vectors to characterize the sports motion process specifically involves:

[0034] Based on the temporal posture feature sequence, we analyze whether the human posture meets the continuous evolution requirement within a continuous time range, and perform consistency constraint processing on the human posture features that deviate from the continuous evolution requirement.

[0035] After completing the consistency constraint processing, the temporal posture feature sequence is subjected to overall feature encoding processing to obtain the action feature vector.

[0036] As a preferred embodiment of the sports motion analysis method based on image recognition described in this invention, the step of obtaining the sports motion type refers to constructing a feature pattern library describing the sports motion type, matching the motion feature vector with the sports motion type in the feature pattern library, and obtaining the corresponding sports motion type.

[0037] Secondly, the present invention provides a sports motion analysis system based on image recognition, comprising,

[0038] The preprocessing module acquires and preprocesses a continuous sequence of human motion images to obtain the target image sequence.

[0039] The detection module performs human keypoint detection on each image frame in the target image sequence to obtain the location information and corresponding confidence scores of the human keypoints.

[0040] The constraint module constructs the skeleton topology of human key points and then applies structural constraints to the position information of the human key points to obtain the pose of the structured key points.

[0041] The judgment module has a preset judgment threshold. It performs occlusion judgment on human key points with confidence levels lower than the judgment threshold and obtains the human key points that are judged to be occluded.

[0042] The completion module analyzes the temporal change characteristics of human key points in adjacent image frames and then performs completion processing on human key points that are determined to be occluded to obtain the pose of the completed key points.

[0043] The module constructs a temporal pose feature sequence that reflects the change of human pose over time, based on the structured keypoint pose and the completed keypoint pose.

[0044] The extraction module performs continuous evolution analysis on the temporal posture feature sequence and extracts motion feature vectors to characterize the process of sports movements.

[0045] The analysis module performs matching analysis on the motion feature vectors to obtain the type of sports motion.

[0046] The beneficial effects of this invention are as follows: By constructing a skeleton topology structure that conforms to human anatomy and applying structural constraints, the physical rationality and noise resistance of single-frame pose estimation are effectively improved. Based on the confidence threshold, occlusion key points are identified, clearly distinguishing between real missing points and detection failures, providing a reliable basis for subsequent processing. On this basis, the occlusion points are completed by integrating temporal motion continuity and static structural priors, enhancing the integrity and smoothness of pose sequences in complex scenes. By performing continuous evolution analysis and principal component dimensionality reduction on temporal pose features, compact, stable and highly discriminative action feature vectors are extracted, significantly improving the accuracy and efficiency of sports action recognition. Attached Figure Description

[0047] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This is a flowchart of a sports motion analysis method based on image recognition.

[0049] Figure 2 A flowchart for obtaining the pose of structured key points.

[0050] Figure 3 A flowchart for obtaining the pose of the completed key points.

[0051] Figure 4 A flowchart for obtaining the type of sports movement. Detailed Implementation

[0052] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0053] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0054] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0055] Reference Figures 1-4This is one embodiment of the present invention, which provides a sports motion analysis method based on image recognition, including the following steps:

[0056] S1. Acquire a continuous sequence of human motion images and preprocess them to obtain the target image sequence.

[0057] It should be noted that the process involves continuously capturing images of the human body during movement using an image acquisition device, obtaining image frames; arranging these image frames in chronological order to form a continuous sequence of image frames containing human motion information; and sequentially reading each image frame according to the chronological order of the continuous image frame sequence. The read image frames are then subjected to denoising and brightness normalization. Specifically, in the denoising process, a preset-size pixel neighborhood (3×3) is selected centered on each pixel in the image frame, and the median of the brightness values ​​of all pixels within this neighborhood is calculated as the statistical result. This statistical result is then used to replace the center pixel pair. The corresponding pixel brightness values ​​are updated to reduce the impact of isolated noise points and high-frequency noise on the image frame, thereby obtaining a denoised image frame. Brightness normalization is then performed on the denoised image frame. This normalization process includes calculating the maximum and minimum brightness values ​​of all pixels in the image frame, and proportionally converting the brightness values ​​of each pixel in the image frame based on these values ​​to ensure that the brightness values ​​of each pixel in the image frame are within a uniform brightness range. By performing denoising and brightness normalization on each image frame, a target image sequence is formed.

[0058] To further explain, the 3×3 pixel neighborhood reflects the local short-distance pixel relationship. When performing denoising processing in consecutive image frames, it helps to maintain the continuity of brightness changes between adjacent image frames and avoids abrupt changes in human pose details over time due to excessive smoothing. This provides a stable foundation for subsequent construction of temporal pose feature sequences and continuous evolution analysis.

[0059] S2. Perform human keypoint detection on each image frame in the target image sequence to obtain the location information and corresponding confidence scores of the human keypoints.

[0060] S2.1 It should be noted that each image frame in the target image sequence is input into a convolutional neural network for feature extraction. Based on the feature extraction results, the location of human key points is calculated to obtain the location information of human key points.

[0061] It should be noted that historical human motion image frames were collected, and each image frame corresponds to human keypoint annotation information. The human keypoint annotation information includes the labeled coordinates of the human keypoints in the image frame. The labeled coordinates of the human keypoints are organized into coordinate vectors, and the coordinate vectors are used as the supervision targets. The historical human motion image frames and the corresponding supervision targets are used as the training set. The historical human motion image frames are obtained from publicly available video datasets, publicly available sports action videos, and actual human motion videos captured by camera equipment. The image frames cover different human postures, movement speeds, and shooting angles.

[0062] The convolutional neural network (CNN) employs a hierarchical feature extraction architecture, consisting of an input layer, a convolutional stage, and an output layer. The input layer receives image frames with a size of 256×256 and 3 channels. The convolutional stage includes multiple convolutional operations, non-linear activation operations, and downsampling operations. Each convolutional operation uses a 3×3 kernel with a stride of 1. Edge padding maintains the feature map size. The number of kernels increases progressively from shallow to deep layers, for example, 32, 64, 128, and 256, to enhance the representation of local structural features and overall pose characteristics of the human body. The non-linear activation operation uses a modified linear unit function to perform element-wise mapping on the convolutional output feature values. The downsampling operation uses a 2×2 region partitioning method with a stride of 2. The output layer uses a 1×1 kernel. The parameters of the CNN are designed to achieve point-by-point calculation of the response intensity of human keypoints without destroying spatial structural information, thus meeting the needs of subsequent human keypoint localization and confidence calculation. S2.2 It should be noted that the training set is input into the convolutional neural network for forward operation to generate a spatial response distribution related to human key points. The difference between the spatial response distribution and the supervised target is calculated to obtain the training error. Based on the training error, the weights of the convolutional kernel in the convolutional neural network are updated by backpropagation. The forward operation, difference calculation and parameter update are repeated until the change in training error between adjacent iterations is less than the convergence threshold, and the trained convolutional neural network is obtained.

[0063] Furthermore, the convergence threshold is set to identify whether the change in training error of the convolutional neural network has entered a stable convergence state during continuous training iterations. The difference in training error between adjacent iterations is statistically analyzed, and the standard deviation of the difference is taken as the convergence threshold.

[0064] S2.3. It should be noted that, according to the time sequence of the target image sequence, each image frame is read sequentially, the pixel brightness values ​​of the image frames are arranged, and input into the trained convolutional neural network. The first convolutional layer of the convolutional neural network slides through the convolutional kernel in the pixel space of the image frame corresponding to the input data, performs element-wise multiplication of the pixel brightness values ​​within the coverage area of ​​the convolutional kernel with the convolutional kernel weights, and then sums them to generate the first layer feature map. Each feature value in the first layer feature map is read one by one, and the read feature value is input into the nonlinear activation function (ReLU) for calculation to obtain the nonlinear feature map. The nonlinear feature map is divided into multiple regions of the same size and non-overlapping, and the average value of the feature values ​​in the region is calculated to obtain the aggregated feature value. The aggregated feature value is written into the spatial position corresponding to the region to complete the downsampling operation. Downsampling operation is performed on all regions of the nonlinear feature map in sequence to generate the downsampled feature map. The convolutional neural network passes the downsampled feature map to the subsequent convolutional layer, and the subsequent convolutional layer repeats the convolution operation, nonlinear activation operation, and downsampling operation to obtain the human pose feature extraction result.

[0065] S2.4. It should be noted that, according to the spatial order of the extracted human posture features, each spatial location in the extracted human posture features is traversed and read to obtain the feature vector corresponding to each spatial location. The feature values ​​of each dimension in the feature vector are squared and then summed to obtain the response intensity of the human keypoint. The response intensity of each human keypoint is written into the corresponding spatial location; the response intensity of all spatial locations is compared, and the spatial location with the largest response intensity is selected as the location of the human keypoint in the image frame. The coordinate value of the location is used as the position information of the human keypoint.

[0066] Furthermore, the location of human key points in the image frame is read, and the response intensity of human key points corresponding to the location is obtained; based on the response intensity of human key points in all spatial locations, the average response intensity is calculated; the response intensity of human key points at the location is compared with the average response intensity to obtain the confidence level of human key point location information.

[0067] S3. After constructing the skeleton topology of human body key points, apply structural constraints to the position information of human body key points to obtain the structured key point pose.

[0068] S3.1 It should be noted that the location information of key points in the human body is read, and the connection relationships between key points are defined one by one according to the actual connection relationships between joints in the human anatomical structure. Specifically, after obtaining the location information of key points, key points are classified into trunk key points, upper limb key points, and lower limb key points according to the human anatomical structure. Based on the natural connection methods of bones and joints in the human anatomical structure, connection relationships are established between key points of adjacent joints. The connection relationship is used to indicate the existence of two key points in the human skeletal structure. Direct joint connections are established. When establishing connections, the hierarchical relationship between key points in the anatomical structure is used as the basis. Adjacent key points on the same limb segment are defined as a pair of connections. For example, the shoulder key point and the elbow key point are defined as a pair of connections, and the elbow key point and the wrist key point are defined as a pair of connections. Following the example, corresponding connections are established between trunk key points and upper limb key points, and between trunk key points and lower limb key points. Each pair of connections is recorded as a skeletal connection edge. The key points and skeletal connections are combined to form a skeletal topology.

[0069] Based on the skeletal connection edges defined in the skeleton topology, pairs of human keypoints with connections are identified; for each pair of human keypoints with connections, the spatial distance in the image frame is calculated using the following formula:

[0070] ;

[0071] in, Indicates the first Key points of the individual body and the first Spatial distance of individual human key points within an image frame Indicates the first The horizontal coordinates of individual human body key points in an image frame. Indicates the first The horizontal coordinates of individual human body key points in an image frame. Indicates the first The vertical coordinates of individual human key points in an image frame. Indicates the first The vertical coordinates of individual human body key points in an image frame.

[0072] The baseline spatial distance is obtained by averaging the spatial distances of all key human body points. Based on the extension and bending characteristics of corresponding bone segments in the human anatomy during actual movement, an allowable variation ratio (0.2) for the baseline spatial distance is set. This allowable variation ratio characterizes the reasonable range of change in bone length relative to the baseline spatial distance during movement. Based on the baseline spatial distance and the allowable variation ratio, the lower and upper limits of the spatial distance between key human body points are calculated using the following formula:

[0073] ;

[0074] ;

[0075] in, This represents the lower limit of spatial distance. This represents the upper limit of spatial distance. Indicates the reference spatial distance. Indicates the allowable percentage of variation.

[0076] The lower and upper limits of the spatial distance between human keypoint pairs constitute the allowable range of spatial distance variation; by limiting the spatial distance between human keypoint pairs in an image frame to within the allowable range of spatial distance variation, the range of spatial distance variation of human keypoints is limited.

[0077] The process of obtaining the allowable variation ratio is as follows: the change in the spatial distance of key points of the human body in the same skeletal segment relative to the baseline spatial distance is statistically analyzed, and the standard deviation of the change is used as the allowable variation ratio. This is used to ensure that the spatial distance of key points of the human body has sufficient constraints while taking into account the natural posture changes during human movement.

[0078] S3.2 It should be noted that, based on the connection relationship between adjacent bone connection edges in the skeleton topology, human keypoint triples formed by two bone connection edges sharing the same human keypoint are identified; the joint angle is calculated for the human keypoint triples using the following formula:

[0079] ;

[0080] in, Indicates the first The horizontal coordinates of individual human body key points in an image frame. Indicates the first The horizontal coordinates of individual human body key points in an image frame. Indicates the first The vertical coordinates of individual human key points in an image frame. Indicates the first The vertical coordinates of individual human key points in an image frame. Indicates the joint angle, Indicates the first The horizontal coordinates of individual human body key points in an image frame. Indicates the first The vertical coordinates of individual human body key points in an image frame.

[0081] For all human keypoint triplet joint angles, the minimum and maximum joint angles are identified. Based on the degrees of freedom of the corresponding joints in the human anatomy during actual movement, an angle margin is preset. The angle margin is used to characterize the additional fluctuation range that the joint angle is allowed to generate under normal movement. The difference between the minimum joint angle and the angle margin is calculated to obtain the lower limit of the joint angle. The sum of the maximum joint angle and the angle margin is obtained to obtain the upper limit of the joint angle. The lower and upper limits of the joint angle constitute the allowable variation range of the joint angle. The joint angles of the human keypoint triplet in the image frame are limited to the allowable variation range of the joint angle, thus completing the limitation of the joint angle variation range.

[0082] To further explain, setting the angle margin to 10° can achieve a balance between the effectiveness of joint angle constraints, the naturalness of human posture, and the tolerance of image recognition errors, providing a stable and reasonable structural constraint basis for subsequent human key point posture correction and sports motion analysis.

[0083] When the spatial distance or joint angle between key human points is detected to exceed the corresponding allowable range of variation, the corresponding key human point position information is determined to be inconsistent with the requirements of human anatomical structure rationality. For the key human point position information determined to be inconsistent with the requirements, the position information of the key human points is adjusted so that the adjusted key human point position information meets the limits of spatial distance variation range and joint angle variation range. By checking the key human point position information one by one and making necessary corrections, the position information of all key human points is made to conform to the rational constraints of human anatomical structure in the overall structure, thereby obtaining a structured key human point posture with clear skeletal structural relationships and spatial consistency.

[0084] S4. Preset judgment threshold, perform occlusion judgment on human key points with confidence level lower than the judgment threshold, and obtain the human key points judged as occluded.

[0085] It should be noted that a threshold for detecting the reliability of human keypoints is preset. At the same time, the confidence level of human keypoints is extracted from each image frame of the target image sequence. The confidence level is compared with the threshold one by one. When the confidence level of human keypoints is less than the threshold, it is determined that the human keypoints are occluded in the image frame. When the confidence level of human keypoints is greater than or equal to the threshold, it is determined that the human keypoints are reliably detected in the image frame. Based on the comparison results, the human keypoints determined to be occluded are extracted for subsequent human keypoint completion processing.

[0086] The decision threshold is set to distinguish whether human key points are in a reliable detection state or are occluded in an image frame. Historical human motion image frames are collected, and the confidence level of human key points is calculated. The mean and standard deviation of the confidence level are calculated, and the difference between the mean and standard deviation is taken as the decision threshold.

[0087] S5. After analyzing the temporal change characteristics of human key points in adjacent image frames, perform completion processing on human key points that are determined to be occluded, and obtain the pose of the completed key points.

[0088] It should be noted that, based on the location information of human key points in adjacent image frames in the target image sequence, human key points with the same number in two consecutive frames are paired, and the position difference of each paired human key point is calculated to obtain the position change amplitude. The position change direction is obtained by calculating the coordinate difference of human key points in two adjacent frames. The position change amplitude and position change direction constitute the temporal change characteristics of human key points.

[0089] Following the temporal order of the target image sequence, the positional information of the human keypoints identified as occluded in the previous and next image frames is read, and the corresponding temporal change features of the human keypoints are extracted. Based on the positional information of the human keypoints in the previous image frame, combined with the direction and magnitude of positional change, the positional information of the human keypoints identified as occluded in the current image frame is calculated (consistent with the positional information extraction method in step S2.1), obtaining the occlusion positional information. The consistency of the occlusion positional information with the positional information of adjacent human keypoints in the skeleton topology is checked. When the occlusion positional information meets the limits of the spatial distance change range and the joint angle change range between human keypoints, the occlusion positional information is confirmed as a valid completion result. When the occlusion positional information exceeds the limits, the occlusion positional information is corrected. Specifically, based on the connection relationship between the human keypoints identified as occluded and adjacent known human keypoints, the actual spatial distance between the occlusion positional information and adjacent human keypoints and the corresponding joint angle are calculated. The actual spatial distance is compared with the limited... The spatial distance lower limit and spatial distance upper limit are compared. When the actual spatial distance is less than the lower limit, the occlusion position information is extrapolated and adjusted along the direction from the adjacent human keypoints to the occlusion position information, so that the corrected actual spatial distance equals the lower limit. When the actual spatial distance is greater than the upper limit, the occlusion position information is retracted and adjusted in the opposite direction, so that the corrected actual spatial distance equals the upper limit, thus completing the spatial distance correction. Based on the corrected human keypoint position information, the joint angle is recalculated and compared with the lower and upper limits. When the joint angle exceeds the allowable range of joint angle variation, the predicted position information is rotated and corrected using the adjacent human keypoints as the rotation center, so that the corrected joint angle falls within the allowable range of joint angle variation, thus completing the joint angle correction. Through spatial distance correction and joint angle correction, the predicted position information of the human keypoints identified as occluded simultaneously meets the spatial distance variation range limit and the joint angle variation range limit, thus obtaining the completed keypoint pose.

[0090] S6. Based on the structured keypoint pose and the completed keypoint pose, construct a temporal pose feature sequence that reflects the change of human pose over time.

[0091] For each image frame in the target image sequence, the structured keypoint pose is extracted. The structured keypoint pose contains the position information of all unoccluded human keypoints. For human keypoints identified as occluded in the same image frame, the corresponding completed keypoint pose is read. The completed keypoint pose contains the position information of the occluded keypoint. The position information of the occluded keypoints in each image frame is replaced with the completed position information, while the unoccluded human keypoints retain their original position information from the structured keypoint pose, forming a complete keypoint pose representation for the image frame. According to the connection relationship of human keypoints in the skeleton topology, all human keypoints in the complete keypoint pose are traversed and read. Based on the predefined connection order and hierarchical relationship of human keypoints in the skeleton topology, the complete keypoint pose is... The positional information of human keypoints in the point pose is processed by sequential mapping. The human keypoints on different skeletal branches of the torso, upper limbs and lower limbs are arranged sequentially according to the connection path in the skeleton topology. During the arrangement process, the adjacent human keypoints on the same skeletal branch are kept in a fixed front-to-back order, and the human keypoints of different skeletal branches are uniformly sorted according to the hierarchical relationship in the skeleton topology. This ensures that the complete keypoint pose is consistent in the keypoint arrangement order and connection structure. Then, the rearranged human keypoint positional information is vectorized and combined in a predetermined order to form a pose representation vector with a fixed keypoint order and structural consistency.

[0092] Based on the temporal order of adjacent image frames in the target image sequence, the pose representation vectors corresponding to adjacent image frames are read sequentially. The positional information of the same human keypoint in the pose representation vector within the adjacent image frames is calculated by difference to obtain the positional change amplitude of the human keypoint in the temporal dimension. Simultaneously, based on the connection relationship of human keypoints in the skeleton topology, combinations of human keypoints constituting joints are selected, and the angles between corresponding human keypoints in adjacent image frames are calculated. The joint angles between adjacent image frames are then calculated by difference to obtain the joint angle change amplitude. The positional change amplitude and joint angle change amplitude are organized according to the human keypoint number and temporal order to form a multidimensional temporal change feature characterizing the dynamic changes of human posture in the temporal dimension. The multidimensional temporal change features corresponding to each time position are continuously stitched together to form a temporal pose feature sequence that is continuously arranged in the temporal dimension and can completely reflect the continuous evolution of human posture over time. During the stitching process, the fixed order of positional change and joint angle change in the multidimensional temporal change features is maintained in the feature dimension, ensuring structural comparability of the multidimensional temporal change features at different time positions.

[0093] S7. After performing continuous evolution analysis on the temporal posture feature sequence, extract the motion feature vector to characterize the sports movement process.

[0094] It should be noted that the multidimensional temporal variation features in the temporal pose feature sequence include the amplitude of position change and the amplitude of joint angle change. A preset threshold is set for the amplitude of position change and a preset threshold is set for the amplitude of joint angle change. When either amplitude exceeds the corresponding preset threshold, it is determined that the requirement of continuous evolution is not met, and the corresponding adjacent image frame is marked as an anomaly. Position and joint angle corrections are performed on the marked adjacent image frames. Specifically, based on the marked adjacent image frames, the human keypoint position information of the previous and subsequent image frames is summed, and the average value is taken to obtain the corrected position. Similarly, the joint angles of the previous and subsequent image frames are summed, and the average value is taken to obtain the corrected joint angles. The corrected positions and joint angles are applied to the marked adjacent image frames to complete the consistency constraint of the temporal pose feature sequence.

[0095] The position change threshold and joint angle threshold are set to distinguish between normal continuous changes and abnormal abrupt changes; the average displacement value and standard deviation of human key points between adjacent image frames are calculated, and the sum of the average displacement value and standard deviation is used as the position change threshold.

[0096] Calculate the mean and standard deviation of the joint angles between adjacent image frames, and use the sum of the mean and standard deviation as the joint angle threshold.

[0097] S7.2. It should be noted that after completing the consistency constraint processing, the human body keypoint position change information and joint angle information of each image frame are extracted from the temporal pose feature sequence and organized into a pose feature vector. The pose feature vector of each image frame is standardized. Specifically, the mean and standard deviation of all dimensions in the pose feature vector are calculated. For each eigenvalue in the pose feature vector, the mean of the corresponding dimension is subtracted from each eigenvalue, and the quotient is calculated with the standard deviation to obtain the standardized pose feature vector. The covariance matrix of the standardized pose feature vector is calculated using the following formula:

[0098] ;

[0099] in, Represents the covariance matrix. This represents the number of attitude features in the attitude feature vector. This represents the transpose of the attitude feature matrix. Represents the attitude feature matrix;

[0100] The pose feature matrix is ​​constructed based on the standardized pose feature vectors, where each column represents a pose feature and each row represents a feature dimension.

[0101] Eigenvalue decomposition is performed on the covariance matrix to obtain eigenvalues ​​and eigenvectors. The eigenvalues ​​are sorted from largest to smallest, and the eigenvectors corresponding to the first N (10) eigenvalues ​​are selected. The selected N eigenvectors are combined into a projection matrix by column. The pose feature matrix is ​​multiplied by the projection matrix to obtain the action feature matrix. The action feature matrix is ​​expanded row by row to obtain the action feature vectors. Each image frame is traversed in turn to obtain the action feature vector of each image frame.

[0102] To further explain, selecting the top 10 features can balance computational efficiency and information preservation after dimensionality reduction, avoiding the loss of important features due to excessive dimensionality reduction.

[0103] S8. Analyze and determine the motion feature vectors to obtain the sports motion analysis results.

[0104] A feature pattern library describing sports movement types is constructed. Specifically, based on the sports movement types to be identified, such as running, jumping, and push-ups, image frames of each sports movement type are captured at different time points using video equipment. For each sports movement type, the posture information of key human points is extracted from the captured image frames, and after standardization and feature extraction (consistent with the action feature vector extraction method in step S7.2), a feature pattern vector for the sports movement type is obtained. Each sports movement type and its corresponding feature pattern vector are stored to obtain the feature pattern library describing sports movement types. The cosine similarity is calculated between the action feature vector of each image frame and all feature pattern vectors in the feature pattern library to obtain the cosine similarity value between the action feature vector and all feature pattern vectors. The cosine similarity calculation formula is:

[0105] ;

[0106] in, This represents the cosine similarity value between the action feature vector and the feature pattern vector. Represents the feature pattern vector. This represents the action feature vector.

[0107] Based on the cosine similarity value between the action feature vector and all feature pattern vectors, the feature pattern vector with the highest cosine similarity value is selected, and the corresponding sports action type is extracted.

[0108] This embodiment also provides a sports motion analysis system based on image recognition, including:

[0109] The preprocessing module acquires and preprocesses a continuous sequence of human motion images to obtain the target image sequence.

[0110] The detection module performs human keypoint detection on each image frame in the target image sequence to obtain the location information and corresponding confidence scores of the human keypoints.

[0111] The constraint module constructs the skeleton topology of human key points and then applies structural constraints to the position information of the human key points to obtain the pose of the structured key points.

[0112] The judgment module has a preset judgment threshold. It performs occlusion judgment on human key points with confidence levels lower than the judgment threshold and obtains the human key points that are judged to be occluded.

[0113] The completion module analyzes the temporal change characteristics of human key points in adjacent image frames and then performs completion processing on human key points that are determined to be occluded to obtain the pose of the completed key points.

[0114] The module constructs a temporal pose feature sequence that reflects the change of human pose over time, based on the structured keypoint pose and the completed keypoint pose.

[0115] The extraction module performs continuous evolution analysis on the temporal posture feature sequence and extracts motion feature vectors to characterize the process of sports movements.

[0116] The analysis module performs matching analysis on the motion feature vectors to obtain the type of sports motion.

[0117] This embodiment also provides a computer device applicable to the sports motion analysis method based on image recognition, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the sports motion analysis method based on image recognition as proposed in the above embodiment.

[0118] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0119] This embodiment also provides a storage medium storing a computer program that, when executed by a processor, implements the image recognition-based sports motion analysis method as proposed in the above embodiments. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0120] In summary, this invention improves the physical rationality and noise resistance of single-frame pose estimation by constructing a skeleton topology structure that conforms to human anatomy and applying structural constraints. It identifies occlusion key points based on confidence thresholds, clearly distinguishing between actual missing points and detection failures, providing a reliable basis for subsequent processing. Furthermore, it integrates temporal motion continuity and static structural priors to complete occlusion points, enhancing the integrity and smoothness of pose sequences in complex scenes. Finally, it extracts compact, stable, and highly discriminative action feature vectors by performing continuous evolution analysis and principal component dimensionality reduction on temporal pose features, significantly improving the accuracy and efficiency of sports action recognition.

[0121] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A sports motion analysis method based on image recognition, characterized in that: include, A continuous sequence of image frames depicting human motion is acquired and preprocessed to obtain the target image sequence. Perform human keypoint detection on each image frame in the target image sequence to obtain the location information and corresponding confidence scores of the human keypoints; After constructing the skeleton topology of human key points, structural constraints are applied to the position information of human key points to obtain the structured key point pose. A preset judgment threshold is set, and human key points with confidence scores below the judgment threshold are occluded to obtain the human key points that are judged to be occluded. After analyzing the temporal variation characteristics of human key points in adjacent image frames, the key points that are determined to be occluded are filled in to obtain the pose of the filled key points. Based on the structured keypoint pose and the completed keypoint pose, a temporal pose feature sequence reflecting the change of human pose over time is constructed. After performing continuous evolution analysis on the temporal posture feature sequence, motion feature vectors are extracted to characterize the process of sports movements; The type of sports movement is obtained by analyzing and determining the feature vector of the movement.

2. The sports motion analysis method based on image recognition as described in claim 1, characterized in that: The preprocessing includes noise reduction and brightness normalization.

3. The sports motion analysis method based on image recognition as described in claim 2, characterized in that: The acquisition of the location information and corresponding confidence levels of key human body points specifically involves: Each image frame in the target image sequence is input into a convolutional neural network for feature extraction. Based on the feature extraction results, the location of human key points is calculated to obtain the location information of human key points. Based on the response intensity of human body key points in the image frame, a confidence calculation method is used to calculate the confidence score for the location information of each human body key point.

4. The sports motion analysis method based on image recognition as described in claim 3, characterized in that: The acquisition of the structured keypoint pose specifically involves: The connection relationships between key points in the human body are defined, and key points with joint connections are connected in an orderly manner according to the distribution pattern of the human skeleton to form a skeletal topology. Based on the skeletal topology of human key points, the spatial distance variation range and joint angle variation range of related human key points are limited. After the constraints are met, the positional information of key points on the human body is corrected based on the rationality of the human anatomical structure to obtain the structured key point pose.

5. The sports motion analysis method based on image recognition as described in claim 4, characterized in that: The occlusion determination refers to setting a determination threshold, comparing the confidence level of human key points with the determination threshold one by one, and determining that the human key points are occluded in the image frame when the confidence level of the human key points is less than the determination threshold, and obtaining the human key points that are determined to be occluded.

6. The sports motion analysis method based on image recognition as described in claim 5, characterized in that: The process of completing key human body points that are determined to be occluded specifically involves: Based on the location information of human key points in adjacent image frames in the target image sequence, the direction and magnitude of position change of human key points in the continuous time dimension are calculated to obtain the temporal change characteristics of human key points. Based on the temporal change characteristics of human body key points, the position information of human body key points that are identified as occluded is predicted and corrected to generate the pose of the completed key points.

7. The sports motion analysis method based on image recognition as described in claim 6, characterized in that: The construction of a temporal pose feature sequence reflecting the change of human pose over time is based on structured keypoint poses and the completed keypoint poses, specifically as follows: Based on the temporal order of image frames in the target image sequence, the structured keypoint pose and the completed keypoint pose corresponding to each image frame are fused to form a complete keypoint pose. Based on the connection relationship of human key points in the skeleton topology, the pose of complete key points is rearranged according to topological constraints to form a pose representation vector. Based on the relationship between pose representation vectors between adjacent image frames, the positional changes, velocity changes, and joint angle changes of human key points in the time dimension are calculated to obtain multidimensional temporal change features. Multidimensional temporal variation features are cascaded in chronological order to form a temporal attitude feature sequence.

8. The sports motion analysis method based on image recognition as described in claim 7, characterized in that: The extraction of motion feature vectors used to characterize the process of sports movements specifically involves: Based on the temporal posture feature sequence, we analyze whether the human posture meets the continuous evolution requirement within a continuous time range, and perform consistency constraint processing on the human posture features that deviate from the continuous evolution requirement. After completing the consistency constraint processing, the temporal posture feature sequence is subjected to overall feature encoding processing to obtain the action feature vector.

9. The sports motion analysis method based on image recognition as described in claim 8, characterized in that: The process of obtaining sports action types refers to constructing a feature pattern library that describes sports action types, matching action feature vectors with sports action types in the feature pattern library, and obtaining the corresponding sports action types.

10. A sports motion analysis system based on image recognition, based on the sports motion analysis method based on image recognition according to any one of claims 1 to 9, characterized in that: include, The preprocessing module acquires and preprocesses a continuous sequence of human motion images to obtain the target image sequence. The detection module performs human keypoint detection on each image frame in the target image sequence to obtain the location information and corresponding confidence scores of the human keypoints. The constraint module constructs the skeleton topology of human key points and then applies structural constraints to the position information of the human key points to obtain the pose of the structured key points. The judgment module has a preset judgment threshold. It performs occlusion judgment on human key points with confidence levels lower than the judgment threshold and obtains the human key points that are judged to be occluded. The completion module analyzes the temporal change characteristics of human key points in adjacent image frames and then performs completion processing on human key points that are determined to be occluded to obtain the pose of the completed key points. The module constructs a temporal pose feature sequence that reflects the change of human pose over time, based on the structured keypoint pose and the completed keypoint pose. The extraction module performs continuous evolution analysis on the temporal posture feature sequence and extracts motion feature vectors to characterize the process of sports movements. The analysis module performs matching analysis on the motion feature vectors to obtain the type of sports motion.

Images