A method for detecting and evaluating riding posture

Abstract

A method for detecting and evaluating horse riding postures includes: S1, two cameras respectively capturing images, and when a person riding a horse is detected in the image, a video clip is captured; S2, human posture data of each frame in the video is extracted through OpenPose, and three-dimensional reconstruction is performed based on the coordinate point information obtained from two different camera positions, and the coordinate information of the human joint points under the three-dimensional coordinate axis is further obtained; S3, considering the confidence of the acquired data and the key parts of the standard action, a suitable combination of joint points is selected, the joint point coordinate information is extracted, and the corresponding angle is further calculated; S4, based on the actual collected joint point angle information, the test data is compared with the standard data, and the standard degree is analyzed based on the results. The device used in the present invention is simple, easy to operate, and low in cost. It can identify and evaluate the posture of horse riding learners, correct the learners' posture, and can also be used in the assessment and scoring of related sports.

Description

Technical field:

[0001] The present invention relates to the technical field of information collection and analysis, and in particular to a method for detecting and evaluating a riding posture. Background technology:

[0002] Analyzing human motion based on visual information is one of the key applications of computer vision technology in sports, involving theories and techniques from multiple fields, including computer vision, pattern recognition, and intelligent sports. Currently, relevant technologies have matured in certain sports, such as running, diving, table tennis, and volleyball. However, in the more traditional sport of horseback riding, posture recognition algorithms are still relatively unavailable, making it difficult to accurately and clearly assess a learner's learning status and, consequently, to provide targeted, personalized training to address weaknesses. This posture recognition system can be used in horseback riding practice areas to identify and assess learners' posture, enabling posture correction. It can also be incorporated into assessment and scoring for related sports, offering significant practical significance for the cultural heritage of traditional sports and promising broad application prospects. Summary of the invention:

[0003] In response to the shortcomings of the existing technology, the present invention provides a riding posture detection and evaluation method, which utilizes computer vision, three-dimensional reconstruction technology, OpenPose posture recognition algorithm, etc. to apply them to the recognition of riding posture, so as to detect and correct the riding posture of horse learners, thereby promoting the inheritance of traditional culture.

[0004] The present invention provides the following technical solutions:

[0005] A method for detecting and evaluating riding postures uses a camera and a processing, analysis, and evaluation system. The camera uploads recorded videos to the processing, analysis, and evaluation system. The system includes a feature extraction module, a posture detection module, a three-dimensional reconstruction module, and a calculation and analysis module.

[0006] A method for detecting and evaluating a riding posture comprises the following steps:

[0007] S1, two cameras capture images respectively. When a person riding a horse is detected in the image, a video clip is captured;

[0008] S2, extracts the human body posture data of each frame in the video through OpenPose, and performs 3D reconstruction based on the coordinate point information obtained from two different camera positions to further obtain the coordinate information of the human body joint points under the 3D coordinate axis;

[0009] S3, comprehensively considering the confidence of the acquired data and the key parts of the standard movement, selects the appropriate joint point combination, extracts the joint point coordinate information, and further calculates the corresponding angle;

[0010] S4, based on the actual collected joint angle information, compare the test data with the standard data, and analyze the standardization degree based on the results.

[0011] In step S1, the two cameras are fixed at the same horizontal height with a wide field of view, and their viewing angles are perpendicular. The cameras capture real-time footage and use a convolutional neural network (CNN) to identify the target image. When both cameras detect a rider on horseback, a video clip is captured. The captured clip begins when the rider fully enters the frame and ends when the rider is about to exit. With simple preprocessing, two videos with the same motion timing but different angles are obtained.

[0012] In step S2, the two-dimensional coordinates and confidence scores of each key point in each frame of the acquired video are obtained through OpenPose, and the corresponding numbers of each part are: 0-nose, 1-neck, 2-right shoulder, 3-right elbow, 4-right hand, 5-left shoulder, 6-left elbow, 7-left hand, 8-mid-crotch, 9-right hip, 10-right knee, 11-right ankle, 12-left hip, 13-left knee, 14-left ankle, 15-right eye, 16-left eye, 17-right ear, 18-left ear, 19-left big toe, 20-left little toe, 21-left heel, 22-right big toe, 23-right little toe, 24-right heel.

[0013] Afterwards, the 3D reconstruction of the human skeleton is performed based on the coordinate information of the two-dimensional skeletal joints of the human body obtained from two frames captured by two cameras at the same time. For each camera, its coordinates in the world coordinate system and image coordinates can be expressed as follows:

[0014]

[0015] Where Z c is the distance from the optical center to the image plane; f is the focal length of the lens; u, v are the coordinates of the pixel coordinate system; R, T are the rotation and translation matrices of the image plane in the world coordinate system; dx, dy are the physical lengths corresponding to the pixels; X w , Y w , Z w is the coordinate of the object in the world coordinate system; u0, v0 are the pixel coordinates of the image center; K, M are the camera intrinsic parameter matrix and extrinsic parameter matrix respectively.

[0016] This step obtains the 3D coordinates of the human skeleton's joints in the world coordinate system. This allows further processing, such as calculating inter-limb angles, to eliminate the visual errors introduced by the 2D coordinates, resulting in more accurate data. The joints and corresponding parameters obtained using OpenPose are used as auxiliary parameters—critical joint angles.

[0017] In step S3, since each joint angle is composed of three joint points, the confidence data of the key joint angle is obtained by calculating the average of the data of the three adjacent joint points according to the confidence data of each joint point given by OpenPose. Let the confidence of the joint angle i be S i .

[0018] According to the key points of the standard riding posture, the joint angles corresponding to each joint point are sorted and assigned values ​​according to their importance. The least important importance is 0, and the most important importance is n, which means the heaviest weight. In step S2, the auxiliary parameters are numbered 0 to 11, so n = 11. Let the importance of joint point i be M i .

[0019] The confidence of the comprehensive joint point and the importance of the relevant limb movements in practice are weighted separately to obtain the parameter A i is the preferred degree of joint angle i.

[0020]

[0021] The priority of each joint angle is sorted, and m joint angles are selected as valid data. After that, their angles are further calculated.

[0022] Taking the joint angle numbered 0 as an example, the coordinates of its constituent joint points are A1(x1, y1, z1), A2(x2, y2, z2), and A3(x3, y3, z3). According to the distance formula between two points

[0023]

[0024] Calculate the distance b between points A1 and A2, the distance c between points A1 and A3, and the distance a between points A2 and A3, and use the law of cosines to find the included angle:

[0025] CosA=(b*b+c*ca*a) / 2*b*c (4)

[0026] Finally, the angle information of the key joint points is obtained.

[0027] In step S4, since the detection of riding posture is a dynamic process, there are static and dynamic requirements for the standards of riding posture.

[0028] For the relatively static movement standard, take the trot riding posture as an example, the shoulder and neck and hips are required to be kept on the same vertical line, so the standard constant v0 = [x1, x2, x3] is taken, and the vector v formed by joint points 1 and 8 is taken i，k =[y1, y2, y3] as the comparison data, where i represents the i-th frame and k represents the limb part number. Then calculate the vector v composed of limb k in the i-th framei，k The cosine similarity of the standard vector v0 is denoted as C i，k , the calculation formula is as follows:

[0029]

[0030] Calculate the corresponding cosine similarity of each frame and take the average value as C k , which represents the action similarity of limb k during this period of time:

[0031]

[0032] After obtaining the above data, the system scores the user's riding static posture standard, w k Indicates the weight of each limb in the entire posture score. The result is best when it is infinitely close to 1, and worst when it is infinitely close to 0:

[0033]

[0034] The dynamic motion of horseback riding can be analyzed as cyclical. For example, the shoulder rotation angle is used to determine the changing trend of the angle at joint 0. Two peak values ​​are selected, each encompassing three to four cycles. The data between these two frames is used as test data. Standard data is then obtained using the same method. After obtaining the two sets of data, each is formed into a vector. The lengths of the two vectors are then compared. If they match, no action is taken. If not, the shorter vector is padded to bring the two lengths into alignment.

[0035] Let's assume the vector obtained from the group with fewer frames is X, with length a, and the vector obtained from the group with more frames is Y, with length b. The difference in length is (ba). Then, in vector X, a data point is added every 1 / (b-a+1) intervals, and its value is the average of the two adjacent data points. The total number of data points added is (ba).

[0036] After the number of frames of the control test set data is consistent with that of the standard set data, it is assumed that the vector A of the angle data corresponding to the changed test action j =[a 0，j , a 1，j ,...a i，j ], vector B of angle data corresponding to standard action j =[b 0，j , b 1，j ,...b i，j ], where i represents the i-th frame, j represents the joint angle numbered j, μ i Indicates a i and b iThen calculate its standardized Euclidean distance, denoted as C j , the calculation formula is as follows:

[0037]

[0038] After obtaining the above data, the system scores the user's riding dynamic posture standard, w j Indicates the weight of each limb in the entire posture score. The result is best when it is infinitely close to 1, and worst when it is infinitely close to 0:

[0039]

[0040] The above technical solution can provide a riding posture detection and evaluation method and system. By applying the existing high-performance posture recognition algorithm to the recognition and comparison of riding postures, the accuracy in detecting and judging the standard degree of the user's riding posture is improved, providing a more standardized, digital, and more accurate and objective scoring system for riding learning and training. It can not only make timely judgments and corrections on students' riding technical movements during the teaching process, but also conduct objective and quantitative evaluations during rider assessments and competitions.

[0041] The advantages of the present invention are:

[0042] The real-time dynamics of the site are obtained through two fixed cameras, and three-dimensional reconstruction is performed based on the plane coordinate information in the picture to obtain the three-dimensional coordinate information of the key nodes, so as to further analyze the target's motion.

[0043] The method is simple in device, easy to operate, and has low cost requirements. It performs quantitative calculations through image information, thereby being used to detect the standard degree of the user's riding posture.

[0044] This posture recognition system is mainly suitable for horse riding practice fields to identify and evaluate the posture of horse riding learners and correct the learners' posture. It can also be used in the assessment and scoring of related sports. It has great practical significance and broad application space for the cultural inheritance of traditional sports. Description of the drawings:

[0045] In order to more clearly illustrate the technical solutions implemented in the present invention, the steps designed for the main process of the invention are illustrated below with a flow chart. The following drawings only show some examples of the present invention and should not be regarded as limiting the scope. Any equivalent changes and modifications made by any technician in this field without departing from the concept and principle of the present invention are within the scope of protection of the present invention.

[0046] Figure 1 It is a schematic diagram of the system structure for implementing the method of the present invention.

[0047] Figure 2 It is a flow chart of the detection method of the present invention.

[0048] Figure 3 It is a schematic diagram of an application scenario of the present invention.

[0049] Figure 4 It is a schematic diagram of the key nodes of the human body of the present invention.

[0050] Figure 5 After establishing the three-dimensional coordinates, Figure 3 Auxiliary parameters for structure settings.

[0051] Figure 6a-6b This is a schematic diagram of the test photo after joint recognition, where Figure 6a It's the original picture. Figure 6b Identification diagram. Specific implementation method:

[0052] In order to have a clearer understanding of the technical features, purposes and effects of the present invention, specific embodiments of the present invention are now described in detail with reference to the accompanying drawings.

[0053] See also Figure 1 、 Figure 2 A method and system for detecting and evaluating riding postures includes a camera and a processing, analysis, and evaluation system. The camera uploads recorded video to the processing, analysis, and evaluation system, which includes a feature extraction module, a posture detection module, a 3D reconstruction module, and a computational analysis module. When using the system, each of these modules is applied one by one.

[0054] In S1, when the device is powered on, two cameras capture images. When a person on horseback is detected, a video clip is captured. Importantly, the two cameras must be positioned in a fixed position, and the data must be imported into the backend in advance to facilitate subsequent 3D reconstruction. In the actual application environment, to facilitate calculations and analysis, the cameras are placed at the same level with a wide field of view, and their viewing angles are perpendicular.

[0055] In step S2, after acquiring the test video, the backend uses OpenPose technology to extract human pose data for each frame in the video. It then selects appropriate joint combinations based on the equestrian posture standard and extracts joint information. It then performs 3D reconstruction based on the coordinate information obtained from two different camera positions.

[0056] After obtaining the coordinate information of the human body joint points under the three-dimensional coordinate axis, the angle information of the key limb movements is calculated.

[0057] In step S3, the confidence of the acquired data and the key parts of the standard action are comprehensively considered to select a suitable combination of joint points, extract the joint point coordinate information, and further calculate the corresponding angle;

[0058] In step S4, the system extracts key limb information based on preset posture standards, compares the static and dynamic posture data with the corresponding standard data, performs weighted calculations, and finally obtains two evaluation scores for the static and dynamic postures of the user's riding posture and outputs them.

[0059] See also Figure 3 The scenario layout envisioned by this invention is shown in the figure. Two cameras, fixed perpendicularly and in close proximity, capture real-time video data. When the rider's position overlaps within the cameras' fields of view, the video information is captured. The acquired data is sent to the backend for subsequent processing.

[0060] See also Figure 4 The OpenPose open source project used in this paper was developed in the early days with Caffe as the framework. It can realize posture estimation of human body movements, facial expressions, finger movements, etc. It is applicable to single and multiple people and has good stability. It is the world's first real-time multi-person 2D posture estimation application based on deep learning. Its output result poseKeypoints is a matrix of n x25x 3, where n is the number of people recognized in the image. 25x3 represents Figure 3 The x, y axis positions and confidence scores of each point in the image are shown in the figure 0-24. In practice, to minimize the error, we control the camera to capture only one target person.

[0061] See also Figure 5 After completing step S3, the 3D coordinates of the human skeleton joints in the world coordinate system are obtained. This allows further calculation of inter-limb angles to eliminate the visual errors introduced by 2D coordinates, resulting in more accurate data. The joint points and corresponding parameters obtained using OpenPose are used as auxiliary parameters—the key joint angles.

[0062] According to the actual situation, the more important joint point movements in the riding posture are selected, and the ones with lower confidence are discarded according to the obtained data. The angle information of the key limb movements is calculated based on the related limb vectors. Taking the joint angle numbered 0 as an example, the coordinates of its component joint points are A1 (x1, y1, z1), A2 (x2, y2, z2), and A3 (x3, y3, z3). According to the distance formula between two points

[0063]

[0064] Calculate the distance b between points A1 and A2, the distance c between points A1 and A3, and the distance a between points A2 and A3, and use the law of cosines to find the included angle:

[0065] CosA=(b*b+c*ca*a) / 2*b*c (4)

[0066] Finally, the angle information of the key joint points is obtained.

[0067] With reference to the accompanying drawings, the method of using the present invention is as follows: The two input devices, namely the cameras, are stably positioned to clearly and completely capture the user's riding video. The processing, analysis, and evaluation system then extracts, processes, calculates, and analyzes the data through various modules, ultimately outputting a scoring result that is displayed on a human-computer interface. The user can then adjust their riding posture based on the results.

[0068] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

[0069] Please refer to Figure 6. The left side shows the original test image, and the right side shows the visualization output of the joint points after testing. As can be seen from the figure, parts of the character's body are obscured, so it is necessary to extract relevant valid data for reference. After importing the test video, the coordinate position and confidence level of each joint point in each frame are extracted. Then, a three-dimensional reconstruction is performed, and appropriate motion trajectories are selected for comparison, comprehensively considering their actual value and the confidence level of the data. Next, the similarity of the selected data sets is evaluated, and the final score is assigned using the corresponding formula.

Claims

1. A method for detecting and evaluating riding posture, characterized in that: The following steps are involved: S1, two cameras capture images respectively. When a person riding a horse is detected in the image, a video clip is captured; S2, extracts the human body posture data of each frame in the video through OpenPose, and performs 3D reconstruction based on the coordinate point information obtained from two different camera positions to further obtain the coordinate information of the human body joint points under the 3D coordinate axis; S3, comprehensively considers the confidence of the acquired data and the key parts of the standard action, selects the appropriate joint point combination, extracts the joint point coordinate information, and further calculates the corresponding angle; since each joint angle is composed of three joint points, according to the confidence data of each joint point given by OpenPose, the confidence data of the key joint angle is obtained by averaging; let the confidence of joint angle i be S i ; According to the key points of the standard riding posture, the joint angles corresponding to each joint point are sorted and assigned values ​​according to their importance. The least important importance is 0, and the most important importance is n, which means the heaviest weight. In step S2, the auxiliary parameters are numbered from 0 to 11, so n = 11; let the importance of joint point i be M i ; The confidence of the comprehensive joint point and the importance of the relevant limb movements in practice are weighted separately to obtain the parameter A i is the preferred degree of joint angle i: Sort the priority of each joint angle and select m joint angles as valid data to complete the data screening; then further calculate their angles; S4, based on the actual collected joint angle information, compare the test data with the standard data, and analyze the standard degree based on the results; for the detection method of the dynamic movement of the riding posture, since its dynamic movement can be regarded as a periodic motion; obtain the change data of the target joint movement angle, select two peak data, which contain three to four cycles, and select the data between the two frames as the test data, and obtain the standard data in the same way; after obtaining the two sets of data, respectively form their data into vectors, and then compare the lengths of the two sets of vectors. If the lengths are consistent, no processing is performed; if they are inconsistent, fill the data in the set of vectors with the shorter length to make the lengths of the two consistent; let the vector obtained from the group with fewer frames be X, with a length of a, and the vector obtained from the group with more frames be Y, with a length of b, then their length difference is (ba); then in vector X, add a data every 1 / (b-a+1), whose value is the average of the two adjacent data, and the total number of added data is (ba); after completing the length unification, use the standardized Euclidean distance for comparison.

2. The riding posture detection and evaluation method according to claim 1, characterized in that: In step S1, the two cameras must be fixed at the same horizontal height with a wide field of view, and their viewing directions are vertical. When the device is started, the two cameras respectively capture images and use the convolutional neural network (CNN) to identify whether there is a target image in the image. When both cameras detect that there is a person riding a horse in the image, a video clip is captured. The captured clip starts when the rider fully enters the image and ends when the rider is about to leave the image. After simple preprocessing, two videos with the same motion time but different angles are finally obtained.

3. The riding posture detection and evaluation method according to claim 1, characterized in that: In step S2, the two-dimensional coordinates and confidence scores of each key point in each frame of the acquired video are obtained through OpenPose, and the corresponding numbers of each part are: 0-nose, 1-neck, 2-right shoulder, 3-right elbow, 4-right hand, 5-left shoulder, 6-left elbow, 7-left hand, 8-mid-crotch, 9-right hip, 10-right knee, 11-right ankle, 12-left hip, 13-left knee, 14-left ankle, 15-right eye, 16-left eye, 17-right ear, 18-left ear, 19-left big toe, 20-left little toe, 21-left heel, 22-right big toe, 23-right little toe, 24-right heel; Afterwards, the 3D reconstruction of the human skeleton is performed based on the coordinate information of the two-dimensional skeletal joints of the human body obtained from two frames captured by two cameras at the same time. For each camera, its coordinates in the world coordinate system and image coordinates can be expressed as follows: Where Z c is the distance from the optical center to the image plane; f is the focal length of the lens; u, v are the coordinates of the pixel coordinate system; R, T are the rotation and translation matrices of the image plane in the world coordinate system; d x , d y is the physical length corresponding to the pixel; X w , Y w , Z w is the coordinate of the object in the world coordinate system; u0, v0 are the pixel coordinates of the image center; K, M are the camera intrinsic parameter matrix and extrinsic parameter matrix respectively; This step obtains the three-dimensional coordinates of the human skeleton joints in the world coordinate system. This allows further processing, such as calculating the angles between limbs, to eliminate the visual errors caused by the two-dimensional coordinates, making the data more accurate. The joint points and corresponding parameters obtained by OpenPose are used as auxiliary parameters—the key joint angles.

4. The riding posture detection and evaluation method according to claim 1, characterized in that: In step S4, since the detection of riding posture is a dynamic process, there are static and dynamic requirements for the standards of riding posture; For the relatively static movement standard, the shoulder, neck and hip are required to be kept on the same vertical line, so the standard constant v0 = [x1, x2, x3] is taken, and the vector v composed of joint points 1 and 8 is taken. i，k =[y1, y2, y3] as the comparison data, where i represents the i-th frame and k represents the limb part number; then calculate the vector v composed of limb k in the i-th frame i，k The cosine similarity of the standard vector v0 is denoted as C i，k , the calculation formula is as follows: Calculate the corresponding cosine similarity of each frame and take the average value as C k , which represents the action similarity of limb k during this period of time: After obtaining the above data, the system scores the user's riding static posture standard, w k Indicates the weight of each limb in the entire posture score. The result is best when it is infinitely close to 1, and worst when it is infinitely close to 0: The dynamic movements of horseback riding were analyzed as cyclical motions. Taking the shoulder rotation angle as an example, the changing trend of the movement angle of joint 0 was obtained. Two peak data points were selected, each containing three to four cycles. The data between these two frames was selected as test data. Standard data was obtained using the same method. After obtaining the two sets of data, they were respectively combined into vectors. The lengths of the two vectors were then compared. If the lengths were consistent, no processing was performed. If not, the shorter vector was padded with data to make the two lengths consistent. Suppose the vector obtained from the group with fewer frames is X, with a length of a, and the vector obtained from the group with more frames is Y, with a length of b. The difference in length is (ba). Then, in vector X, a data point is added every 1 / (b-a+1), and its value is the average of the two adjacent data points. The total number of data points added is (ba). After the number of frames of the control test set data is consistent with that of the standard set data, it is assumed that the vector A of the angle data corresponding to the changed test action j =[a 0，j , a 1，j ,...a i，j ], vector B of angle data corresponding to standard action j =[b 0，j , b 1，j ,...b i，j ], where i represents the i-th frame, j represents the joint angle numbered j, μ i Indicates a i and b i The mean of the two values; then calculate the standardized Euclidean distance, denoted as C j , the calculation formula is as follows: After obtaining the above data, the system scores the user's riding dynamic posture standard, w j Indicates the weight of each limb in the entire posture score. The result is best when it is infinitely close to 1, and worst when it is infinitely close to 0:

Images