Deep learning and exercise physiology-based stationary exercise equipment training correction method

Abstract

The one or more embodiments of the specification provide a fixed fitness equipment training error correction method based on deep learning and exercise physiology, comprising the following steps: S1: image acquisition position determination; S2: detector training; S3: data acquisition; S4: data noise reduction, using a tracking method to define the trajectory of a specific marker, and removing the specific marker that is not in the commonly used trajectory; S5: human body data acquisition, using a lightweight human body pose estimation method to detect human body key points, and collecting human body key points at different angles according to different fixed equipment; S6: judging the human body motion state based on human body exercise physiology, and constructing exercise physiology rules; S7: result output; A non-contact image collector is added on the fixed fitness equipment, and the fitness venue is conveniently and intelligently upgraded without incurring high costs, and the tracking algorithm combined with a high-precision deep learning algorithm improves the effectiveness and robustness of the fixed fitness equipment error correction method.

Description

Technical Field

[0001] This specification relates to the technical field of error correction methods for fitness equipment, and more particularly to a training error correction method for fixed fitness equipment based on deep learning and exercise physiology. Background Technology

[0002] Personal trainers are an effective way to provide tailored training plans and movement guidance based on individual physical conditions. However, most people cannot afford professional exercise guidance services, and the fixed and inflexible hours of personal trainers are unsuitable for busy working professionals and young people with the strongest fitness needs. Learning exercise posture online is currently the most mainstream method of fitness guidance. However, without professional supervision during exercise, it's difficult to recognize errors and pinpoint key areas of incorrect posture, making it impossible to adjust posture and potentially causing irreversible damage to muscles and bones. Smart devices are divided into wearable and contactless smart devices. Wearable standalone devices (such as smartwatches) suffer from low data depth and poor reliability, while wearable wrap-around devices (such as fitness suits) have issues with slow heat dissipation and a feeling of constriction. Contactless smart devices are a new type of smart fitness equipment, but current methods in this field mostly use traditional machine learning to extract human posture, which has poor robustness and slow inference speed, making it difficult to meet the needs of real-time fitness guidance. Even deep learning methods only focus on the accuracy of human posture judgment and cannot provide specific exercise behavior guidance for fixed fitness equipment. In general, current mainstream fitness posture guidance methods have drawbacks such as high cost, low flexibility, poor guidance effect, and inability to provide detailed guidance for fixed fitness equipment, thus failing to meet the diverse fitness needs of users. Summary of the Invention

[0003] In view of this, the purpose of one or more embodiments of this specification is to propose a training error correction method for fixed fitness equipment based on deep learning and exercise physiology, so as to solve the problems of high cost, low flexibility, poor guidance effect, and inability to provide detailed guidance for fixed fitness equipment.

[0004] To achieve the above objectives, one or more embodiments of this specification provide a method for error correction during training on fixed fitness equipment based on deep learning and exercise physiology, comprising the following steps:

[0005] S1: Image acquisition location determination: Exercise equipment needs to establish exercise physiology rules to determine the state of movement posture. For exercise equipment, it is necessary to select the location of the image acquisition device so that the acquired image can completely capture the entire movement process of the exercise equipment and the exerciser. The image acquisition device is set up according to the selected location.

[0006] S2: Detector training involves constructing a high-quality labeled dataset of specific markers and training the YOLOX algorithm to obtain a target detector capable of detecting specific markers.

[0007] S3: Data acquisition, detect and identify key points of the device with specific markers in each frame of the video data stream acquired by the image acquisition device set up in S1, obtain the detection data frame of the specific markers, and record the coordinates of the upper left and lower right corners of the detection data frame and the detection confidence.

[0008] S4: Data noise reduction. The tracking method is used to define the trajectory of specific markers, remove specific markers that are not on the commonly used trajectory, and match and compare them with the detection results in S2 based on the commonly used trajectory data of specific markers of equipment movement. Then, the specific markers that are missing in the commonly used trajectory are added to make the specific markers stably represent the key points of the fitness equipment.

[0009] S5: Human data acquisition. Lightweight human pose estimation method is used to detect key points of human skeleton. When the image acquisition device acquires images of fitness equipment, it does not need to capture the entire skeleton of the fitness person. It only needs to capture one or two angles. For different fixed equipment, key points of the human body need to be acquired from different angles.

[0010] S6: Based on human movement physiology, determine the human movement state, construct movement physiology rules, determine the positional relationship between the key points of the fitness equipment after the tracking algorithm denoises and the key points of the human skeleton, and determine whether the exerciser's movement posture is correct from the perspectives of sitting posture requirements and movement process.

[0011] S7: Output the results. Based on the exercise physiology rules in S6, output the prompts and reasons for the incorrect exercise posture of the fitness participant in the current frame. Repeat S3-S6 until the input video stream ends.

[0012] In S4, the trajectory set of the input video stream is initially the high-resolution detection box in the first frame (the detection confidence is greater than the threshold is called the high-resolution detection box). When the detection box on each frame matches the predicted box obtained by using historical information in the trajectory set, the information of the corresponding matching trajectory in the trajectory set is updated using the detection box of the current frame. Kalman filter predicts the position of the existing trajectory in the current frame. The specific marker prediction box of the existing trajectory and the detector detection box construct an association matrix for matching.

[0013] The correlation matrix for the existing trajectory prediction boxes and detection boxes in the current frame can be represented as:

[0014]

[0015] Calculate the predicted bounding box from the existing trajectory set using Kalman filtering. The IoU distance matrix between the target detection box Z and the target detection box Z, C v (O,Z,V track ) Calculate the trajectory history velocity direction V track Consistency of velocity direction between historical trajectory observation O and new observation Z;

[0016] The formula for calculating the velocity between the coordinates (x1, y1) of the center point of the historical trajectory observation O and the coordinates (x2, y2) of the center point of the new observation Z is as follows.

[0017]

[0018] Using the velocity calculation formula above, and then using radians to calculate the velocity direction θ between the historical observation O and the new observation Z, det Its definition is as follows:

[0019]

[0020] C v (O,Z,V track The velocity direction consistency Δθ = |θ| between all trajectories and the detection was calculated. det -θ track |,θ track The calculation calculates the velocity direction between the most recently successfully matched observation and the third most recently successfully matched observation. The specific definition of the velocity direction is the same as θ. det Consistent;

[0021] Furthermore, the trajectories in the trajectory set input by S4 are matched with the target detection boxes using formula (1). The trajectories in the existing trajectory set are divided into successfully matched trajectories and unsuccessfully matched trajectories according to the matching results. The detection boxes are divided into successfully matched detections and unsuccessfully matched detections according to the matching results. The successfully matched detections and trajectories are used for subsequent trajectory updates, and the unsuccessfully matched detections are used for subsequent trajectory recovery with the unsuccessfully matched trajectories.

[0022] The original IoU calculation method was modified, and an observation amplification method named EIoU was proposed to solve the problem of small overlap between the predicted and observed positions, so that the trajectory and detection can be matched. When the irregular movement of the fitness equipment causes the overlap between the predicted and observed positions to be small, the size of the observation box and the predicted box is appropriately amplified according to the confidence of the box. The specific operation is as follows.

[0023] o=(x,y,w,h)→(x,y,w+2bws,h+2bhs) (4);

[0024] Where o represents the observation / prediction box to be magnified, (x,y) are the coordinates of the center point of the observation / prediction box, (w,h) are the width and height of the observation / prediction box, s is the confidence level of the observation / prediction box, and b is the magnification parameter of the observation magnification method.

[0025] In formula (1), the unmatched detections and trajectories in the matching results are used to recover the trajectory using the following correlation matrix;

[0026]

[0027] in, Calculate the predicted bounding boxes of the set of unmatched trajectories using Kalman filtering. The EIoU distance matrix between the unmatched target detection box Z' and C v (O',Z',V' track Calculate the historical velocity direction V' of the unmatched trajectory. track Consistency of velocity direction between historical trajectory observations O' and Z' that did not match successfully.

[0028] The exercise physiology rules for fitness equipment in S6 mainly use the formulas for calculating the distance between two points, the angle between two lines, and the distance between a point and a line. Given a point A = (x a ,y a ) and B = (x b ,y b The formula for calculating the distance d(A,B) between two points is defined as follows:

[0029]

[0030] Given point B and the line The coordinates of the two points C and D are given, and the distance from the point to the line is given. The calculation formula is as follows:

[0031]

[0032] Given a straight line The coordinates of the two points A and B, and the line The coordinates of the two points C and D, and the angle between the two lines The calculation formula is as follows:

[0033]

[0034] Regarding the requirement that the lower back of the exerciser must be close to the back of the exercise equipment in the seated leg extension posture, the following exercise physiology rules are established: the angle between the lower back and the back of the exercise equipment should be as small as possible, or even parallel, and the distance between the right shoulder and the back of the exercise equipment should be as close as possible. If these conditions are met, the exerciser's posture is considered incorrect. The criteria for judgment are as follows:

[0035]

[0036] in, This represents the straight line connecting the key points of the right shoulder and right hip of the human body. This represents the straight line connecting two specific markers on the seat back. ε1 represents the angle between two lines, and p1 represents the angle threshold for determining when a person's back is close to the back of the exercise equipment. ε1 represents the straight-line distance from the right shoulder key point of the human body to the back of the seat, and ε2 represents the distance threshold for judging when the back of the fitness equipment is close to the waist of the fitness person.

[0037] Regarding the requirement that the instep hooks onto the rotation axis in the seated leg extension posture, the following exercise physiology rule was established: when the key point (q5) of the rotation axis of the fitness equipment is close to the right knee (p3) and right ankle (p4) of the exerciser, the exerciser's movement is judged to meet the exercise physiology rule. The judgment criteria are as follows:

[0038]

[0039] in, This represents a straight line connecting the key points of the right knee and right ankle in the human body. ε3 represents the distance from the point on the line connecting the key point of the fitness equipment's rotation axis to the key point of the right knee-right ankle on the human body. ε3 represents the distance threshold when the instep hooks onto the rotation axis.

[0040] To address the requirement that the back of the knee should be positioned as close as possible to the edge of the exercise equipment seat in the seated leg extension posture, the following exercise physiology rule was established: when the distance between the key point (q4) at the lower end of the exercise equipment seat and the exerciser's right knee (p3) is relatively close, the exerciser's movement is considered to meet the exercise physiology rule. The judgment criteria are as follows:

[0041] d(q4,p3)<ε4 (11);

[0042] Where d(q4,p3) represents the distance between the key point q4 at the bottom of the fitness equipment seat and the right knee p3 of the fitness user, and ε4 represents the threshold distance between the two points when the knee socket position is as close as possible to the edge of the fitness equipment seat.

[0043] To address the requirement that the knee joint angle at the lowest point of the ankle in a seated leg flexion-extension exercise should be between 65° and 80°, this invention establishes the following exercise physiology rule: when the angle between the thigh and lower leg (knee joint angle) of the exerciser is between 65° and 80°, the exerciser's movement is judged to meet the exercise physiology rule. The judgment criteria are as follows:

[0044]

[0045] in, This represents the straight line connecting the key points of the right hip and right knee at the lowest point of the ankle. This represents the straight line connecting the key points of the right knee and right ankle at the lowest point of the ankle. The angle between the line connecting the key points of the right hip and right knee and the line connecting the key points of the right knee and right ankle at the lowest point of the ankle is called the knee joint angle.

[0046] To address the requirement that the knee angle at the highest point of the ankle in seated leg flexion and extension exercises should be between 170° and 180°, the following exercise physiology rule was established: when the angle between the thigh and lower leg (knee angle) of the exerciser is between 170° and 180°, the exerciser's movement is considered to meet the exercise physiology rule. The criteria for judgment are as follows:

[0047]

[0048] in, The straight line connecting the key points of the right hip and right knee at the highest point of the ankle represents the position of the ankle. This represents the straight line connecting the key points of the right knee and right ankle at the highest point of the ankle. The angle between the line connecting the key points of the right hip and right knee and the line connecting the key points of the right knee and right ankle at the highest point of the ankle is called the knee joint angle.

[0049] To address the two requirements that the knee joint angle should be between 65° and 80° at the lowest point of the ankle and between 170° and 180° at the highest point of the ankle, the definition of the highest point of the ankle is specified, which is that the knee joint angle of the fitness participant is the largest at time t2 among consecutive time points t1, t2, and t3. The definition of the lowest point is similar to that of the highest point, and will not be repeated here.

[0050]

[0051] Where t1 < t2 < t3, This represents the knee joint angle of the fitness participant at time t1. This represents the knee joint angle of the fitness participant at time t2. This represents the knee angle of the fitness participant at time t3.

[0052] To address the requirement that the positions of the back, kneecaps, and thighs remain unchanged during seated leg flexion and extension exercises, this invention establishes the following exercise physiology rules: Within consecutive time points t1 and t2, the following conditions must be met to ensure that the exerciser's movement satisfies the exercise physiology rules: the angle between the exerciser's lower back and the backrest of the exercise equipment is minimized; the relative change in the distance between the exerciser's right shoulder and the backrest of the exercise equipment is minimized; the relative change in the distance between the lower key point (q4) of the exercise equipment seat and the exerciser's right knee is minimized; and the relative change in the distance between the upper key point (q3) of the exercise equipment seat and the exerciser's right hip is minimized. The criteria for this judgment are as follows:

[0053]

[0054] in, The straight line connecting the key points of the right shoulder and right hip of the fitness participant at time t1. The straight line connecting the two specific markers on the seat back The angle between the two lines, The straight line connecting the key points of the right shoulder and right hip of the fitness participant at time t2 represents the line. The straight line connecting the two specific markers on the seat back The angle between the two lines, ε6, represents the threshold value required for the angle between the two lines to remain unchanged on the back side. γ represents the rate of change of the linear distance from the right shoulder key point to the seat back at time t2 relative to time t1, where γ1 represents the threshold of the rate of change of linear distance required for the back not to change. γt2 represents the rate of change of distance between the key point q4 at the bottom of the fitness equipment seat and the right knee p3 of the exerciser at time t2 relative to time t1, where γ2 represents the threshold for the rate of change of linear distance required for no change in the popliteal fossa. The rate of change of distance between the key point q3 at the top of the fitness equipment seat and the right hip p2 of the fitness participant at time t2 relative to time t1, where γ3 represents the threshold of the rate of change of linear distance required for the thigh not to change.

[0055] As can be seen from the above, the fixed fitness equipment training error correction method based on deep learning and exercise physiology provided in one or more embodiments of this specification achieves real-time and efficient accurate posture correction for fitness personnel using fixed fitness equipment. Specifically, this invention only adds a contactless image acquisition device to the fixed fitness equipment, realizing convenient and intelligent upgrades for fitness venues without incurring high costs. The image acquisition device, combined with a lightweight specific marker detector and a human posture recognition algorithm, enables real-time output of the fixed fitness equipment error correction method. The tracking algorithm, combined with a high-precision deep learning algorithm, improves the effectiveness and robustness of the fixed fitness equipment error correction method. It is understood that this invention is the first training error correction method for fixed fitness equipment, with broad market potential and application scenarios, and has already been piloted in some application scenarios. Attached Figure Description

[0056] To more clearly illustrate the technical solutions in one or more embodiments of this specification or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only one or more embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0057] Figure 1 This is a schematic diagram illustrating the placement of the image acquisition device for a specific fixed fitness equipment according to the present invention;

[0058] Figure 2 This is a schematic diagram showing the location and numbering of key points of the human body in the result of the human posture estimation method in this invention;

[0059] Figure 3 This is a diagram showing the results of the fixed fitness equipment training error correction method defined in this invention;

[0060] Figure 4 This is a schematic diagram of the exercise physiology rules defined by the present invention for specific fixed fitness equipment;

[0061] Figure 5 This is a schematic diagram of EIoU operation calculation as defined in this invention;

[0062] Figure 6 This is a summary diagram of the first results of the fixed fitness equipment training error correction method defined in this invention;

[0063] Figure 7 This is a summary diagram of the second results of the fixed fitness equipment training error correction method defined in this invention;

[0064] Figure 8 This is a summary diagram of the third result of the fixed fitness equipment training error correction method defined in this invention. Detailed Implementation

[0065] To make the objectives, technical solutions, and advantages of this disclosure clearer, the following detailed description is provided in conjunction with specific embodiments.

[0066] It should be noted that, unless otherwise defined, the technical or scientific terms used in one or more embodiments of this specification should have the ordinary meaning understood by one of ordinary skill in the art to which this disclosure pertains. The terms "first," "second," and similar words used in one or more embodiments of this specification do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0067] like Figure 1-8 As shown, a training error correction method for fixed fitness equipment based on deep learning and exercise physiology is provided, including the following steps: S1, different exercise physiology rules need to be constructed for different fixed fitness equipment to determine whether the exercise posture is correct. For a certain fixed fitness equipment, it is necessary to select a suitable image acquisition device position and set the positional relationship between the image acquisition device and the fixed fitness equipment so that the acquired image can completely capture the fixed fitness equipment and the exerciser's body.

[0068] S2. To accurately locate the position of the fixed fitness equipment, this invention uses specific landmarks to pinpoint the key locations of the equipment, thus requiring a specific landmark detector. This invention constructs a high-quality labeled dataset of specific landmarks and trains the mainstream detector YOLOX from scratch, resulting in a target detector capable of detecting specific landmarks.

[0069] S3, for each frame of the input video stream, detect the fixed instrument key points with specific markers, provide the detection box of the specific markers, and record the coordinates of the upper left and lower right corners of the detection box and the detection confidence.

[0070] S4. The target detector for detecting specific markers has detection errors, misidentifying areas in the background that resemble specific markers as such. These specific markers represent key points of the fixed apparatus. Detecting the wrong position of a specific marker will cause the exercise training error correction method to fail. A tracking method is used to remove specific markers that are not on the commonly used trajectory and to supplement specific markers that are missing from the commonly used trajectory, so that the specific markers can stably represent the key points of the fixed apparatus.

[0071] S5 uses a lightweight human pose estimation method to detect key points of the human skeleton. When the image acquisition device acquires images of different fixed equipment, it does not need to capture the entire skeleton of the fitness person. It only needs to capture an angle to complete the pose judgment task.

[0072] S6 constructs exercise physiology rules, determines the positional relationship between the key points of the fixed device and the key points of the human skeleton after the tracking algorithm denoises, and judges whether the exerciser's exercise posture is standardized from the perspectives of sitting posture requirements and exercise process.

[0073] S7, based on exercise physiology rules, outputs a prompt and reason for the incorrect exercise posture of the fitness person in the current frame, and repeats S3-S6 until the input video stream ends.

[0074] Data noise reduction methods:

[0075] The target detector for detecting specific landmarks suffers from detection errors, misidentifying regions in the background that resemble these landmarks as such. These landmarks represent key points on the fitness equipment, and misdetecting their positions can render exercise training error correction methods ineffective. A tracking algorithm is used to remove specific landmarks not found on commonly used tracks and to supplement those missing from those tracks, ensuring that specific landmarks can reliably represent key points on the fitness equipment.

[0076] The initial trajectory set of the input video stream consists of high-resolution detection boxes (boxes with a detection confidence greater than a threshold are called high-resolution detection boxes). When a detection box in each frame matches a predicted box obtained from historical information in the trajectory set, the information of the corresponding matching trajectory in the trajectory set is updated using the detection box from the current frame. Kalman filtering predicts the position of existing trajectories in the trajectory set within the current frame, and the predicted boxes of specific markers on existing trajectories are matched with the detector detection boxes using an association matrix. For the existing trajectories and detections in the current frame, the association matrix can be represented as follows:

[0077]

[0078] Calculate the predicted bounding box from the existing trajectory set using Kalman filtering. The IoU distance matrix between the target detection box Z and the target detection box Z, C v (O,Z,V track) Calculate the trajectory history velocity direction V track The velocity direction is consistent with that of the historical trajectory observation O and the new observation Z. The formula for calculating the velocity between the center point coordinates (x1, y1) of the historical trajectory observation O and the center point coordinates (x2, y2) of the new observation Z is as follows.

[0079]

[0080] Using the velocity calculation formula above, and then using radians to calculate the velocity direction θ between the historical observation O and the new observation Z, det Its definition is as follows:

[0081]

[0082] C v (O,Z,V track The velocity direction consistency Δθ = |θ| between all trajectories and the detection was calculated. det -θ track |,θ track The calculation calculates the velocity direction between the most recently successfully matched observation and the third most recently successfully matched observation. The specific definition of the velocity direction is the same as θ. det Consistent.

[0083] The existing trajectory set is matched with the target detection boxes using a cost matrix (Formula (1)). The existing trajectory set is divided into successfully matched trajectories and unsuccessfully matched trajectories according to the matching result. The detection boxes are also divided into successfully matched detections and unsuccessfully matched detections according to the matching result. The successfully matched detections and trajectories are used for subsequent trajectory updates, and the unsuccessfully matched detections are used for subsequent trajectory recovery with the unsuccessfully matched trajectories.

[0084] The non-linear motion of fitness equipment makes it difficult for Kalman filters to accurately capture the target's motion information. This leads to a discrepancy between the trajectory position estimated by the Kalman filter and the actual position, resulting in a small or even non-overlapping degree between the predicted and observed positions. When the overlap between the predicted and observed positions is very small, due to the limitation of the IoU threshold, current practices discard these Hungarian matching results, which can cause trajectory matching errors or failures. This paper modifies the original IoU calculation method and proposes an observation amplification method called EIoU to solve the problem of small overlap between the predicted and observed positions, thus aiding in trajectory and detection matching. When the irregular motion of the fitness equipment results in a small overlap between the predicted and observed positions, the size of the observed and predicted bounding boxes is appropriately amplified based on the confidence level of the bounding boxes. The specific operation is as follows:

[0085] o=(x,y,w,h)→(x,y,w+2bws,h+2bhs) (4)

[0086] Where o represents the observation / prediction box to be magnified, (x,y) are the coordinates of the center point of the observation / prediction box, (w,h) are the width and height of the observation / prediction box, s is the confidence level of the observation / prediction box, and b is the magnification parameter of the observation magnification method.

[0087] Where (x, y) are the center coordinates of the observation / prediction box, (w, h) are the width and height of the observation / prediction box, s is the confidence level of the observation / prediction box, and b is the magnification parameter of the observation magnification method. Figure 6 The diagram below illustrates the calculation operations for IoU and EIoU. Formula (1) shows the unmatched detections and trajectories in the matching results, and the following correlation matrix is ​​used for trajectory recovery.

[0088]

[0089] in, Calculate the predicted bounding boxes of the set of unmatched trajectories using Kalman filtering. The EIoU distance matrix between the unmatched target detection box Z' and C v (O',Z',V' track Calculate the historical velocity direction V' of the unmatched trajectory. track Consistency of velocity direction between historical trajectory observations O' and Z' that did not match successfully.

[0090] Trajectories that failed to match according to formula (1) are matched with detections using cost matrix (5). Trajectories that failed to match according to formula (1) are divided into successfully matched trajectories and unsuccessfully matched trajectories based on the matching results. Detections that failed to match according to formula (1) are also divided into successfully matched detections and unsuccessfully matched detections based on the matching results. Successfully matched detections and trajectories are used for subsequent trajectory updates. Trajectories that failed to match according to formula (5) are matched with unsuccessfully matched detections according to formula (5) using matching rules specific to fitness equipment.

[0091] To ensure the continuity of specific marker trajectories, for trajectories that fail to match in formula (5), matching rules are formulated for specific fitness equipment to help associate unmatched trajectories with unmatched detections. Trajectories that fail to recover in formula (5) are mostly due to the rapid movement of movable parts within the fitness equipment. In this case, there is a certain distance between the unmatched trajectory and the unmatched detection, resulting in the EIoU magnified prediction box and observation box still failing to match. Therefore, we can use matching rules formulated for specific fitness equipment. Taking the matching rules for seated leg extension as an example, the unmatched trajectory in formula (5) is the axis of rotation. We can define the detection closest to the ankle keypoint among human keypoints and with a distance less than a certain threshold as the detection matching the specific marker trajectory on the axis of rotation.

[0092] Exercise physiology rules:

[0093] To achieve targeted movement correction using fitness equipment, it is necessary to develop exercise physiology rules for each type of equipment. This involves establishing connections between key points on the human body at specific image capture angles and key points on the fitness equipment represented by specific markers. The correctness of the exerciser's posture is judged from the perspectives of seated posture requirements and the movement process. This involves the exercise physiology rules for fixed fitness equipment, thus enabling movement correction methods for corresponding equipment. Fixed fitness equipment includes: seated leg extensions, seated lat pulldowns, and seated leg press machines, etc.

[0094] Taking the physiological rules of seated leg flexion and extension exercises as an example, Figure 4 A schematic diagram illustrating the exercise physiology rules of a seated leg extension machine is provided. The image acquisition angle of the image capture device under the seated leg extension trainer is shown below. Figure 1 As shown, this shooting angle can capture... Figure 2 The right shoulder, a key point in the middle of the human body ( Figure 2 Key point #2 in the middle), right hip ( Figure 2 The 9th key point of the human body), the right knee ( Figure 2 The 10th key point of the human body), right ankle ( Figure 2 The 11th human body key point in the diagram, represented by four human body key points on the horizontal and vertical axes, can be represented as p = [p1, p2, p3, p4]. The seated leg extension trainer has five specific markers representing key points on the fitness equipment. These five key points, arranged in descending order of their horizontal axes, can be represented as q = [q1, q2, q3, q4, q5]. See details... Figure 5 The location of the white coordinate point in the image.

[0095] The formulation of exercise physiology rules takes into account the instructions for use of fixed fitness equipment and human physiological limits. The exercise physiology rules for each fitness equipment mainly use formulas for calculating the distance between two points, the angle between two lines, and the distance between a point and a line. Given a point A = (x a ,y a ) and B = (x b ,y b The formula for calculating the distance d(A,B) between two points is defined as follows:

[0096]

[0097] Given point B and the line The coordinates of the two points C and D are given, and the distance from the point to the line is given. The calculation formula is as follows:

[0098]

[0099] Given a straight line The coordinates of the two points A and B, and the line The coordinates of the two points C and D, and the angle between the two lines The calculation formula is as follows:

[0100]

[0101] To address the requirement that the lower back of the exerciser must be close to the back of the exercise equipment in the seated leg extension posture, the following exercise physiology rules have been established: the angle between the lower back of the exerciser and the back of the exercise equipment should be as small as possible or even parallel, and the distance between the right shoulder of the exerciser and the back of the exercise equipment should be as close as possible. If these conditions are met, the exerciser's movement is considered to meet the exercise physiology rules; otherwise, the exercise posture is considered incorrect.

[0102]

[0103] in, This represents the straight line connecting the key points of the right shoulder and right hip of the human body. This represents the straight line connecting two specific markers on the seat back. ε1 represents the angle between two lines, and p1 represents the angle threshold for determining when a person's back is close to the back of the exercise equipment. ε1 represents the straight-line distance from the right shoulder key point of the human body to the back of the seat, and ε2 represents the distance threshold for judging when the back of the fitness equipment is close to the waist of the fitness person.

[0104] In response to the requirement that the instep hooks onto the rotation axis in the seated leg flexion and extension posture, the following exercise physiology rule was established: when the key point (q5) of the rotation axis of the fitness equipment is close to the right knee and right ankle of the fitness person, the fitness person's movement is judged to meet the exercise physiology rule.

[0105]

[0106] in, This represents a straight line connecting the key points of the right knee and right ankle in the human body. ε3 represents the distance from the point on the line connecting the key point of the fitness equipment's rotation axis to the key point of the right knee-right ankle on the human body. ε3 represents the distance threshold when the instep hooks onto the rotation axis.

[0107] In response to the requirement that the knee bend should be positioned close to the edge of the fitness equipment seat in the seated leg extension posture, the following exercise physiology rule was established: when the distance between the key point (q4) at the bottom of the fitness equipment seat and the right knee of the exerciser is close, the exerciser's movement is judged to meet the exercise physiology rule.

[0108] d(q4,p3)<ε4 (11)

[0109] Where d(q4,p3) represents the distance between the key point q4 at the bottom of the fitness equipment seat and the right knee p3 of the fitness user, and ε4 represents the threshold distance between the two points when the knee socket position is as close as possible to the edge of the fitness equipment seat.

[0110] For seated leg flexion and extension exercises, the knee angle should be between 65° and 80° at the lowest point of the ankle.

[0111] Based on the requirements, the following exercise physiology rules were established: when the angle between the thigh and lower leg (knee joint angle) of a fitness enthusiast is between 65° and 80°, the fitness enthusiast's movement is judged to meet the exercise physiology rules.

[0112]

[0113] in, This represents the straight line connecting the key points of the right hip and right knee at the lowest point of the ankle. This represents the straight line connecting the key points of the right knee and right ankle at the lowest point of the ankle. The angle between the line connecting the key points of the right hip and right knee and the line connecting the key points of the right knee and right ankle at the lowest point of the ankle is called the knee joint angle.

[0114] To address the requirement that the knee angle at the highest point of the ankle in seated leg flexion and extension exercises should be between 170° and 180°, the following exercise physiology rule was established: when the angle between the thigh and lower leg (knee angle) of the exerciser is between 170° and 180°, the exerciser's movement is considered to meet the exercise physiology rule.

[0115]

[0116] in, The straight line connecting the key points of the right hip and right knee at the highest point of the ankle represents the position of the ankle. This represents the straight line connecting the key points of the right knee and right ankle at the highest point of the ankle. The angle between the line connecting the key points of the right hip and right knee and the line connecting the key points of the right knee and right ankle at the highest point of the ankle is called the knee joint angle.

[0117] Regarding the two requirements that the knee joint angle should be between 65° and 80° at the lowest point of the ankle and between 170° and 180° at the highest point of the ankle, the definition of the highest point of the ankle is given, that is, the knee joint angle of the fitness person is the largest at time t2 among consecutive time points t1, t2 and t3. The definition of the lowest point is similar to that of the highest point, and will not be repeated.

[0118]

[0119] Where t1 < t2 < t3, This represents the knee joint angle of the fitness participant at time t1. This represents the knee joint angle of the fitness participant at time t2. This represents the knee angle of the fitness participant at time t3.

[0120] To address the requirement that the positions of the back, knees, and thighs remain unchanged during seated leg flexion and extension exercises, the following exercise physiology rules were established: If, within consecutive time points t1 and t2, the angle between the exerciser's lower back and the backrest of the exercise equipment is minimized, the relative change in the distance between the exerciser's right shoulder and the backrest of the exercise equipment is minimized, the relative change in the distance between the lower key point (q4) of the exercise equipment seat and the exerciser's right knee is minimized, and the relative change in the distance between the upper key point (q3) of the exercise equipment seat and the exerciser's right hip is minimized, then the exerciser's movement is considered to satisfy the exercise physiology rules.

[0121]

[0122] in, The straight line connecting the key points of the right shoulder and right hip of the fitness participant at time t1. The straight line connecting the two specific markers on the seat back The angle between the two lines, The straight line connecting the key points of the right shoulder and right hip of the fitness participant at time t2 represents the line. The straight line connecting the two specific markers on the seat back The angle between the two lines, ε6, represents the threshold value required for the angle between the two lines to remain unchanged on the back side. γ represents the rate of change of the linear distance from the right shoulder key point to the seat back at time t2 relative to time t1, where γ1 represents the threshold of the rate of change of linear distance required for the back not to change. γt2 represents the rate of change of distance between the key point q4 at the bottom of the fitness equipment seat and the right knee p3 of the exerciser at time t2 relative to time t1, where γ2 represents the threshold for the rate of change of linear distance required for no change in the popliteal fossa. The rate of change of distance between the key point q3 at the top of the fitness equipment seat and the right hip p2 of the fitness participant at time t2 relative to time t1, where γ3 represents the threshold of the rate of change of linear distance required for the thigh not to change.

[0123] Multiple sets of fitness videos were collected under different gym environments, lighting conditions, background crowd density, resolutions, frame rates, and error types to evaluate the effectiveness of the proposed error correction method for fixed fitness equipment. Figure 6-8The report demonstrates the accuracy of the error correction method for fixed fitness equipment under different shooting angles on various fitness equipment. It achieved an accuracy of 95% on multiple fitness equipment, including the seated leg press machine, and an error identification accuracy of over 90% on all fitness equipment, providing strong evidence for the effectiveness and robustness of the fixed fitness equipment error correction method.

[0124] Figure 3 The results of a defined error correction method for fixed fitness equipment training on seated leg flexion and extension exercises are presented. It accurately locates key points on the fitness equipment and key points on the human body at specific angles, robustly removing all background noise from specific markers. When the user's back is not close to the backrest of the fitness equipment seat, the error is identified and clearly indicated. When the user's feet are close to the rotation axis, the error is accurately identified without misjudgment. When the user's knees are far from the key point under the fitness equipment seat, it indicates that the current fitness equipment is not suitable for the user's height; the user should adjust the backrest position so that their knees are close to the key point under the seat, and the error is identified and clearly indicated. When the user's knee joint angle is between 170° and 180° at the highest point of the ankle in the frame image, it conforms to the rules of exercise physiology, and the error is accurately identified without misjudgment. When the user's knee joint angle is between 65° and 80° in the frame image, it cannot be determined whether the knee joint angle is at the lowest point of the ankle, but the error is accurately identified without misjudgment. During exercise, the user's back, knees, and thighs did not change significantly, and the system accurately identified all incorrect movements without misjudgment. This application demonstrates satisfactory performance, accurately identifying all types of incorrect movements in the absence of misjudgment of correct movements, thus proving the effectiveness of the error correction method for fixed fitness equipment. Furthermore, as the first movement error correction method for fixed fitness equipment, it provides a feasible and robust solution for subsequent movement error correction, and hopefully lays a solid foundation for promoting the intelligent upgrading of fitness equipment.

[0125] It should be noted that the methods of one or more embodiments of this specification can be executed by a single device, such as a computer or server. The methods of this embodiment can also be applied in a distributed scenario, where multiple devices cooperate to complete the task. In such a distributed scenario, one of these devices may execute only one or more steps of the methods of one or more embodiments of this specification, and the multiple devices will interact with each other to complete the method described.

[0126] The apparatus described above is used to implement the corresponding methods in the foregoing embodiments and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0127] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of this disclosure (including the claims) is limited to these examples; within the framework of this disclosure, technical features in the above embodiments or different embodiments may also be implemented, and there are many other variations of different aspects of one or more embodiments of this specification as described above, which are not provided in detail for the sake of brevity.

[0128] Although this disclosure has been described in conjunction with specific embodiments thereof, many substitutions, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description.

[0129] One or more embodiments of this specification are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of one or more embodiments of this specification should be included within the scope of protection of this disclosure.

Claims

1. A fixed fitness equipment training error correction method based on deep learning and exercise physiology, characterized in that, Includes the following steps: S1: Image acquisition location determination: Exercise equipment needs to establish exercise physiology rules to determine the state of movement posture. For exercise equipment, it is necessary to select the location of the image acquisition device so that the acquired image can completely capture the entire movement process of the exercise equipment and the exerciser. The image acquisition device is set up according to the selected location. S2: Detector training involves constructing a high-quality labeled dataset of specific markers and training the YOLOX algorithm to obtain a target detector capable of detecting specific markers. S3: Data acquisition, detect and identify key points of the device with specific markers in each frame of the video data stream acquired by the image acquisition device set up in S1, obtain the detection data frame of the specific markers, and record the coordinates of the upper left and lower right corners of the detection data frame and the detection confidence. S4: Data noise reduction. The tracking method is used to define the trajectory of specific markers, remove specific markers that are not on the commonly used trajectory, and match and compare them with the detection results in S2 based on the commonly used trajectory data of specific markers of equipment movement. Then, the specific markers that are missing in the commonly used trajectory are added to make the specific markers stably represent the key points of the fitness equipment. S5: Human data acquisition. Lightweight human pose estimation method is used to detect key points of human skeleton. When the image acquisition device acquires images of fitness equipment, it does not need to capture the entire skeleton of the fitness person. It only needs to capture one or two angles. For different fixed equipment, key points of the human body need to be acquired from different angles. S6: Based on human movement physiology, determine the human movement state, construct movement physiology rules, determine the positional relationship between the key points of the fitness equipment after the tracking algorithm denoises and the key points of the human skeleton, and determine whether the exerciser's movement posture is correct from the perspectives of sitting posture requirements and movement process. S7: Output the results. Based on the exercise physiology rules in S6, output the prompts and reasons for the incorrect exercise posture of the fitness participant in the current frame. Repeat S3-S6 until the input video stream ends.

2. The fixed fitness equipment training error correction method based on deep learning and exercise physiology according to claim 1, characterized in that, In S4, the trajectory set of the input video stream is initially the high-resolution detection box in the first frame (the detection confidence is greater than the threshold is called the high-resolution detection box). When the detection box on each frame matches the predicted box obtained by using historical information in the trajectory set, the information of the corresponding matching trajectory in the trajectory set is updated using the detection box of the current frame. Kalman filter predicts the position of the existing trajectory in the current frame. The specific marker prediction box of the existing trajectory and the detector detection box construct an association matrix for matching. The correlation matrix for the existing trajectory prediction boxes and detection boxes in the current frame can be represented as: Calculate the predicted bounding box from the existing trajectory set using Kalman filtering. The IoU distance matrix between the target detection box Z and the target detection box Z, C v (O,Z,V track ) Calculate the trajectory history velocity direction V track Consistency of velocity direction between historical trajectory observation O and new observation Z; The formula for calculating the velocity between the coordinates (x1, y1) of the center point of the historical trajectory observation O and the coordinates (x2, y2) of the center point of the new observation Z is as follows. Using the velocity calculation formula above, and then using radians to calculate the velocity direction θ between the historical observation O and the new observation Z, det Its definition is as follows: C v (O,Z,V track The velocity direction consistency Δθ = |θ| between all trajectories and the detection was calculated. det -θ track |,θ track The calculation calculates the velocity direction between the most recently successfully matched observation and the third most recently successfully matched observation. The specific definition of the velocity direction is the same as θ. det Consistent; Furthermore, the trajectories in the trajectory set input by S4 are matched with the target detection boxes using formula (1). The existing trajectory set is divided into successfully matched trajectories and unsuccessfully matched trajectories according to the matching results. The detection boxes are divided into successfully matched detections and unsuccessfully matched detections according to the matching results. The successfully matched detections and trajectories are used for subsequent trajectory updates, and the unsuccessfully matched detections are used for subsequent trajectory recovery with the unsuccessfully matched trajectories.

3. The fixed fitness equipment training error correction method based on deep learning and exercise physiology according to claim 2, characterized in that, The original IoU calculation method was modified, and an observation amplification method named EIoU was proposed to solve the problem of small overlap between the predicted and observed positions, so that the trajectory and detection can be matched. When the irregular movement of the fitness equipment causes the overlap between the predicted and observed positions to be small, the size of the observation box and the predicted box is appropriately amplified according to the confidence of the box. The specific operation is as follows. o=(x,y,w,h)→(x,y,w+2bws,h+2bhs) (4); Where o represents the observation / prediction box to be magnified, (x,y) are the coordinates of the center point of the observation / prediction box, (w,h) are the width and height of the observation / prediction box, s is the confidence level of the observation / prediction box, and b is the magnification parameter of the observation magnification method.

4. The fixed fitness equipment training error correction method based on deep learning and exercise physiology according to claim 3, characterized in that, In formula (1), the unmatched detections and trajectories in the matching results are used to recover the trajectory using the following correlation matrix; in, Calculate the predicted bounding boxes of the set of unmatched trajectories using Kalman filtering. The EIoU distance matrix between the unmatched target detection box Z' and C v (O',Z',V' track Calculate the historical velocity direction V' of the unmatched trajectory. track Consistency of velocity direction between historical trajectory observations O' and Z' that did not match successfully.

5. The fixed fitness equipment training error correction method based on deep learning and exercise physiology according to claim 1, characterized in that, The exercise physiology rules for fitness equipment in S6 mainly use the formulas for calculating the distance between two points, the angle between two lines, and the distance between a point and a line. Given a point A = (x a ,y a ) and B = (x b ,y b The formula for calculating the distance d(A,B) between two points is defined as follows: Given point B and the line The coordinates of the two points C and D are shown above. The distance from the point to the line is... The calculation formula is as follows: Given a straight line The coordinates of the two points A and B, and the line The coordinates of the two points C and D, and the angle between the two lines The calculation formula is as follows:

6. The fixed fitness equipment training error correction method based on deep learning and exercise physiology according to claim 5, characterized in that, Regarding the requirement that the lower back of the exerciser must be close to the back of the exercise equipment in the seated leg extension posture, the following exercise physiology rules are established: the angle between the lower back and the back of the exercise equipment should be as small as possible, or even parallel, and the distance between the right shoulder and the back of the exercise equipment should be as close as possible. If these conditions are met, the exerciser's posture is considered incorrect. The criteria for judgment are as follows: in, This represents the straight line connecting the key points of the right shoulder and right hip of the human body. This represents the straight line connecting two specific markers on the seat back. ε1 represents the angle between two lines, and p1 represents the angle threshold for determining when a person's back is close to the back of the exercise equipment. ε1 represents the straight-line distance from the right shoulder key point of the human body to the back of the seat, and ε2 represents the distance threshold for judging when the back of the fitness equipment is close to the waist of the fitness person.

7. The fixed fitness equipment training error correction method based on deep learning and exercise physiology according to claim 6, characterized in that, Regarding the requirement that the instep hooks onto the rotation axis in the seated leg extension posture, the following exercise physiology rule was established: when the key point (q5) of the rotation axis of the fitness equipment is close to the right knee (p3) and right ankle (p4) of the exerciser, the exerciser's movement is judged to meet the exercise physiology rule. The judgment criteria are as follows: in, This represents a straight line connecting the key points of the right knee and right ankle in the human body. ε3 represents the distance from the point on the line connecting the key point of the rotation axis of the fitness equipment to the key point of the right knee-right ankle of the human body. ε3 represents the distance threshold when the instep hooks onto the rotation axis. To address the requirement that the back of the knee should be positioned as close as possible to the edge of the exercise equipment seat in the seated leg extension posture, the following exercise physiology rule was established: when the distance between the key point (q4) at the lower end of the exercise equipment seat and the exerciser's right knee (p3) is relatively close, the exerciser's movement is considered to meet the exercise physiology rule. The judgment criteria are as follows: d(q4,p3)<ε4 (11); Where d(q4,p3) represents the distance between the key point q4 at the bottom of the fitness equipment seat and the right knee p3 of the fitness user, and ε4 represents the threshold distance between the two points when the knee socket position is as close as possible to the edge of the fitness equipment seat.

8. The fixed fitness equipment training error correction method based on deep learning and exercise physiology according to claim 7, characterized in that, To address the requirement that the knee joint angle at the lowest point of the ankle in a seated leg flexion-extension exercise should be between 65° and 80°, this invention establishes the following exercise physiology rule: when the angle between the thigh and lower leg (knee joint angle) of the exerciser is between 65° and 80°, the exerciser's movement is judged to meet the exercise physiology rule. The judgment criteria are as follows: in, This represents the straight line connecting the key points of the right hip and right knee at the lowest point of the ankle. This represents the straight line connecting the key points of the right knee and right ankle at the lowest point of the ankle. The angle between the line connecting the key points of the right hip and right knee and the line connecting the key points of the right knee and right ankle at the lowest point of the ankle is called the knee joint angle. To address the requirement that the knee angle at the highest point of the ankle in seated leg flexion and extension exercises should be between 170° and 180°, the following exercise physiology rule was established: when the angle between the thigh and lower leg (knee angle) of the exerciser is between 170° and 180°, the exerciser's movement is considered to meet the exercise physiology rule. The criteria for judgment are as follows: in, The straight line connecting the key points of the right hip and right knee at the highest point of the ankle represents the position of the ankle. This represents the straight line connecting the key points of the right knee and right ankle at the highest point of the ankle. The angle between the line connecting the key points of the right hip and right knee and the line connecting the key points of the right knee and right ankle at the highest point of the ankle is called the knee joint angle.

9. The fixed fitness equipment training error correction method based on deep learning and exercise physiology according to claim 8, characterized in that, To address the two requirements that the knee joint angle should be between 65° and 80° at the lowest point of the ankle and between 170° and 180° at the highest point of the ankle, the definition of the highest point of the ankle is specified, which is that the knee joint angle of the fitness participant is the largest at time t2 among consecutive time points t1, t2, and t3. The definition of the lowest point is similar to that of the highest point, and will not be repeated here. Where t1 < t2 < t3, This represents the knee joint angle of the fitness participant at time t1. This represents the knee joint angle of the fitness participant at time t2. This represents the knee angle of the fitness participant at time t3.

10. The fixed fitness equipment training error correction method based on deep learning and exercise physiology according to claim 8, characterized in that, To address the requirement that the positions of the back, kneecaps, and thighs remain unchanged during seated leg flexion and extension exercises, this invention establishes the following exercise physiology rules: Within consecutive time points t1 and t2, the following conditions must be met to ensure that the exerciser's movement satisfies the exercise physiology rules: the angle between the exerciser's lower back and the backrest of the exercise equipment is minimized; the relative change in the distance between the exerciser's right shoulder and the backrest of the exercise equipment is minimized; the relative change in the distance between the lower key point (q4) of the exercise equipment seat and the exerciser's right knee is minimized; and the relative change in the distance between the upper key point (q3) of the exercise equipment seat and the exerciser's right hip is minimized. The criteria for this judgment are as follows: in, The straight line connecting the key points of the right shoulder and right hip of the fitness participant at time t1. The straight line connecting the two specific markers on the seat back The angle between the two lines, The straight line connecting the key points of the right shoulder and right hip of the fitness participant at time t2 represents the line. The straight line connecting the two specific markers on the seat back The angle between the two lines, ε6, represents the threshold value required for the angle between the two lines to remain unchanged on the back side. γ represents the rate of change of the linear distance from the right shoulder key point to the seat back at time t2 relative to time t1, where γ1 represents the threshold of the rate of change of linear distance required for the back not to change. γ2 represents the rate of change of distance between the key point q4 at the bottom of the fitness equipment seat and the right knee p3 of the exerciser at time t2 relative to time t1. γ2 represents the threshold of the rate of change of linear distance required for the popliteal fossa to remain unchanged. The rate of change of distance between the key point q3 at the top of the fitness equipment seat and the right hip p2 of the fitness participant at time t2 relative to time t1, where γ3 represents the threshold of the rate of change of linear distance required for the thigh not to change.

Images