An elderly balance exercise guidance method and device based on pose estimation

Abstract

The application discloses a kind of old person balance exercise guidance method and device based on pose estimation.The device includes image acquisition module, extraction and matching module and indication early warning module.The guidance method first establishes the exercise action standard library based on professional person, then obtains the human body key point positioning of exerciser by real-time pose estimation technology, uses dynamic time planning to align exercise action sequence and standard action sequence, to judge whether the action is standard, and carries out the guidance of action improvement.Using the way of weighted calculation of the center of gravity and support area of each body part, the center of gravity position is judged, the balance state of the exerciser is monitored, and the alarm is sent in time.A kind of old person balance exercise guidance solution with safety guarantee and simple and easy to realize is provided.

Description

Technical Field

[0001] This invention belongs to the field of smart home technology and relates to the monitoring and guidance of home exercise for the elderly. Specifically, it relates to a method and device for guiding balance exercise for the elderly based on posture estimation. Background Technology

[0002] With the increasing global aging trend, falls and related injuries among the elderly have become a global public health problem, posing a serious threat to their quality of life and health. Balance training for the elderly is a health program designed to improve their balance and reduce the risk of falls, which is of great significance for improving their quality of life.

[0003] As a product of modern technology and the digital revolution, smart homes have profoundly changed the way we live at home. Backed by the Internet of Things, artificial intelligence, sensing technology, and automated control, smart home systems allow home devices, facilities, and media devices to connect with each other, enabling automation, intelligence, and remote control. By combining smart homes with health exercises for the elderly, it is possible to greatly facilitate home-based exercise for seniors and improve the quality of their workouts.

[0004] The goals of balance training for older adults include improving muscle strength, flexibility, coordination, and postural control, thereby enhancing their ability to adapt to uneven surfaces, unstable environments, and various challenges in daily life. Currently, there is a lack of professional guidance on balance training for older adults, which may lead them to choose inappropriate training methods or excessively high training intensity, increasing the risk of injury. Furthermore, older adults living alone require fall detection and reminders during balance training. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention proposes a method and device for guiding balance exercises for the elderly based on posture estimation. By combining image acquisition equipment to identify the movement sequence of the elderly during balance exercises, remote guidance is achieved through comparison with a standard movement library. At the same time, the invention introduces the identification of balance and center of gravity, and issues timely fall warnings to ensure exercise safety.

[0006] A method for guiding balance exercise in older adults based on posture estimation, specifically including the following steps:

[0007] Step 1: Collect standard balance exercise image data, extract standard movement sequences, and establish a standard exercise movement library.

[0008] Step 2: Collect real-time image data of the exerciser's balance training.

[0009] Step 3: The deep learning-based human pose estimation model performs target recognition on the balance exercise image data of the exerciser. After recognizing the human body position, it performs human key point regression and outputs the exerciser's action sequence based on the human key point coordinate vector space.

[0010] Step 4: Use Dynamic Time Warping (DTW) to compare the similarity between the exerciser's movement sequence and the standard movement sequence to determine if the exerciser's movements meet the requirements. When the distance between the exerciser's movement sequence and the standard movement sequence exceeds the standard threshold, remind the exerciser and provide suggestions on how to improve the movement based on the direction of deviation.

[0011] Step 5: Calculate the exerciser's center of gravity and support area in real time using key points on the human body. When the distance between the projection point of the center of gravity and the edge of the support area is lower than the alarm threshold, it indicates a risk of falling and triggers a fall alarm.

[0012] The SupportArea is:

[0013] SupportArea=Distance(LeftFoot,RightFoot)*FootLength

[0014] The `Distance()` function calculates the distance between two points, `FootLength` represents the width of the foot, and `LeftFoot` and `RightFoot` represent the coordinates of the centers of the left and right feet, respectively. The calculation of the body's center of gravity assigns weights based on the distribution of key points on the body, and then performs a weighted sum.

[0015]

[0016] Where H represents the number of extracted human body key points, (CehterOfMass_X, CehterOfMass_Y) represents the coordinates of the body's center of gravity projected onto the ground, (X_h, Y_h) represents the coordinates of the h-th human body key point projected onto the ground, and Weight_h represents the assigned weight.

[0017] A balance exercise guidance device for the elderly based on posture estimation includes an image acquisition module, an extraction and matching module, and an indication and warning module.

[0018] The image acquisition module uses a video capture device to collect real-time image data of the exerciser's balance training and transmits it to the extraction and matching module.

[0019] The extraction and matching module stores a standard exercise movement library. It identifies key points of the human body based on the balance exercise image data of the exerciser to obtain an action sequence. It compares the similarity with the standard exercise movement sequence stored in the standard exercise movement library to obtain the name of the exerciser's action and determine whether the exerciser's action meets the requirements. At the same time, it calculates the exerciser's current center of gravity and support area and feeds the processing results back to the indication and warning module.

[0020] The warning module includes a display screen and a speaker. When the distance between the exerciser's movement sequence and the standard movement sequence exceeds a threshold, the speaker alerts the exerciser, and the display screen displays suggestions for improving the movement based on the direction of deviation. When the distance between the projection point of the body's center of gravity and the edge of the support area is lower than an alarm threshold, a fall alarm is triggered by the speaker.

[0021] The present invention has the following beneficial effects:

[0022] This invention uses human posture estimation technology to obtain key points on the human body, and employs dynamic time planning to align exercise movement sequences with standard movement sequences to determine whether the movements are standard. It provides specific guidance for elderly individuals' exercise, and by applying different weighted calculations to each key point, it more accurately calculates the body's center of gravity, better assessing whether the elderly are in a balanced state during training. This helps improve the elderly's balance and postural control abilities, thereby significantly reducing the risk of falls and injuries. The device has a simple structure, small footprint, and low cost, meeting the needs of the elderly for balance training anytime, anywhere at home. Attached Figure Description

[0023] Figure 1 This is a system block diagram of a balance exercise guidance device for the elderly based on posture estimation;

[0024] Figure 2 Flowchart of a method for guiding balance exercise in older adults based on posture estimation;

[0025] Figure 3 This is a schematic diagram illustrating the output of the optimal path based on the dynamic programming matrix. Detailed Implementation

[0026] The present invention will be further explained below with reference to the accompanying drawings;

[0027] like Figure 1 As shown, a balance exercise guidance device for the elderly based on posture estimation includes an image acquisition module, an extraction and matching module, and an indication and warning module, used to achieve, for example... Figure 2 The following are the specific steps for the balance exercise guidance method for the elderly:

[0028] Step 1: Collect standard balance exercise image data demonstrated by professionals, extract standard movement sequences, and establish a standard exercise movement library.

[0029] Step 2: Collect real-time image data of the exerciser's balance training using a camera.

[0030] Step 3: Use OpenPose to perform target recognition on the balance exercise image data of the exerciser to obtain the human body position, and then perform human keypoint regression to output the exerciser's movement sequence based on the coordinate vector space of the human keypoints. The human keypoints include the head, neck, shoulders, elbows, wrists, hips, knees, and ankles.

[0031] Step 4: Since the speed of the exerciser's movements may differ from the speed of the professional's demonstration, Dynamic Time Warping (DTW) is used to compare the similarity between the exerciser's movement sequence and the standard movement sequence to determine whether the exerciser's movements meet the requirements. When the distance between the exerciser's movement sequence and the standard movement sequence exceeds the standard threshold, the exerciser is alerted, and suggestions for improvement are given based on the direction of deviation, such as raising the hands or lowering the waist.

[0032] The dynamic time warping method specifically includes the following steps:

[0033] s4.1 Choose a distance metric to measure action similarity. Available distance metrics include Euclidean distance, Manhattan distance, and cosine distance.

[0034] s4.2 Construct a distance matrix D of size m×n, where m and n are the lengths of the standard movement sequence and the exerciser's movement sequence, respectively. Calculate the distance between each movement in the two sequences in turn, and fill the distance values ​​into the distance matrix starting from the top left corner in the order of the movements.

[0035] s4.3 Construct a dynamic programming matrix T of the same size as the distance matrix, and initialize the elements of the dynamic programming matrix to infinity. Start filling the values ​​in the dynamic programming matrix from the top left corner.

[0036] ① The first column of the dynamic programming matrix T, T(i,1)=D(i,1)+D(i-1,1).

[0037] ②The first row of the dynamic programming matrix T, T(1,j)=D(1,j)+D(1,j-1).

[0038] ③ For other positions in the dynamic programming matrix T, T(i,j) = T(i,j) + min[D(i-1,j),D(i-1,j-1),D(i,j-1)]

[0039] Where D(i,j) and T(i,j) represent the element values ​​of the i-th row and j-th column in the distance matrix D and the dynamic programming matrix T, respectively, i = 1, 2, ..., m, j = 1, 2, ..., n.

[0040] s4.4. Starting from the bottom right corner T(m,n) of the dynamic programming matrix T, find the element with the smallest value among T(m-1,n), T(m,n-1), and T(m-1,n-1). After selecting this element, continue selecting the smallest value from the three elements adjacent to its left, until the endpoint is reached, which is the top left corner T(1,1) of the dynamic programming matrix T. Figure 3 As shown.

[0041] s4.5 Output all selected points from T(1,1) to T(m,n), the similarity sequence between the exerciser's action sequence and the standard action sequence, and determine whether the exerciser's action meets the requirements.

[0042] Step 5: Calculate the exerciser's center of gravity and support area in real time using key points on the human body. When the distance between the projection point of the center of gravity and the edge of the support area is lower than the alarm threshold, it indicates a risk of falling and triggers a fall alarm.

[0043] The SupportArea is:

[0044] SupportArea=Distance(LeftFoot,RightFoot)*FootLength

[0045] The Distance() function is used to calculate the distance between two points. FootLength represents the width of the foot, and LeftFoot and RightFoot represent the coordinates of the center of the left and right feet, respectively.

[0046] The calculation of the body's center of gravity first divides the body into similar geometric shapes, estimates the volume of each part, assigns weights to the coordinates of key points in different parts of the body, and finally performs a weighted summation:

[0047]

[0048] Where H represents the number of extracted human body key points, (CehterOfMass_X, CehterOfMass_Y) represents the coordinates of the body's center of gravity projected onto the ground, (X_h, Y_h) represents the coordinates of the h-th human body key point projected onto the ground, and Weight_h represents the assigned weight.

Claims

1. A method for guiding balance exercise in the elderly based on posture estimation, characterized in that: Specifically, the following steps are included: Step 1: Collect standard balance exercise image data, extract standard movement sequences, and establish a standard exercise movement library; Step 2: Real-time acquisition of balance training image data of the trainees; Step 3: The deep learning-based human pose estimation model performs target recognition on the balance exercise image data of the exerciser. After recognizing the human body position, it performs human key point regression and outputs the exerciser's action sequence based on the human key point coordinate vector space. Step 4: Compare the similarity between the exerciser's movement sequence and the standard movement sequence to determine whether the exerciser's movements meet the requirements; When the distance between the exerciser's movement sequence and the standard movement sequence exceeds the standard threshold, the exerciser is alerted, and suggestions for improving the movement are provided based on the direction of deviation. Step 5: Calculate the exerciser's center of gravity and support area in real time through key points on the human body. When the distance between the projection point of the center of gravity and the edge of the support area is lower than the alarm threshold, it indicates a risk of falling and a fall alarm is issued. The SupportArea is: SupportArea = Distance(LeftFoot, RightFoot) * FootLength; The `Distance()` function calculates the distance between two points, `FootLength` represents the width of the foot, and `LeftFoot` and `RightFoot` represent the coordinates of the centers of the left and right feet, respectively. The calculation of the body's center of gravity is based on weighted distribution of key points on the body, and finally a weighted sum is performed. CharacterOfMass_X = ; CharacterOfMass_Y = ; Where H represents the number of extracted human body key points, (CehterOfMass_X, CehterOfMass_Y) represent the coordinates of the body's center of gravity projected onto the ground, and ( , () represents the coordinates of the h-th person's key points projected onto the ground. This indicates the assigned weight.

2. The method for guiding balance exercise in the elderly based on posture estimation as described in claim 1, characterized in that: Target recognition was performed on image data of balance exercises by exercisers using OpenPose.

3. The method for guiding balance exercise in the elderly based on posture estimation as described in claim 1, characterized in that: The key points of the human body include the head, neck, shoulders, elbows, wrists, hips, knees, and ankles.

4. The method for guiding balance exercise in the elderly based on posture estimation as described in claim 1, characterized in that: Dynamic time warping is used to compare the similarity between the exerciser's movement sequence and the standard movement sequence.

5. The method for guiding balance exercise in the elderly based on posture estimation as described in claim 4, characterized in that: The dynamic time warping method specifically includes the following steps: s4.1 Choose a distance metric to measure action similarity; s4.2 Construct a distance matrix D of size m×n, where m and n are the lengths of the standard movement sequence and the exerciser's movement sequence, respectively. Calculate the distance between each movement in the two sequences in turn, and fill the distance values ​​into the distance matrix starting from the top left corner in the order of the movements. s4.3 Construct a dynamic programming matrix T of the same size as the distance matrix, and initialize the elements in the dynamic programming matrix to infinity; fill the values ​​in the dynamic programming matrix sequentially starting from the top left corner; ① The first column of the dynamic programming matrix T, T(i,1)=D(i,1)+D(i-1,1); ② The first row of the dynamic programming matrix T is T(1,j)=D(1,j)+D(1,j-1); ③ For other positions in the dynamic programming matrix T, T(i,j) = T(i,j) + min[D(i-1,j), D(i-1,j-1), D(i,j-1)] Where D(i,j) and T(i,j) represent the element values ​​in the i-th row and j-th column of the distance matrix D and the dynamic programming matrix T, respectively, i=1,2,…m, j=1,2,…n; s4.

4. Starting from the bottom right corner T(m,n) of the dynamic programming matrix T, find the element with the smallest value among T(m-1,n), T(m,n-1), and T(m-1,n-1). After selecting this element, continue to select the smallest value among the three elements adjacent to the top left of this element until the endpoint is reached, which is the top left corner T(1,1) of the dynamic programming matrix T. s4.5 Output all selected points from T(1,1) to T(m,n), the similarity sequence between the exerciser's action sequence and the standard action sequence, and determine whether the exerciser's action meets the requirements.

6. The method for guiding balance exercise in the elderly based on posture estimation as described in claim 5, characterized in that: The distance metric is Euclidean distance, Manhattan distance, or cosine distance.

7. A device for guiding balance exercise in the elderly based on posture estimation, characterized in that: The method for guiding balance exercises for the elderly as described in any one of claims 1 to 6 includes an image acquisition module, an extraction and matching module, and an indication and warning module. The image acquisition module uses a video capture device to collect real-time image data of the exerciser's balance exercise and transmits it to the extraction and matching module; The extraction and matching module stores a standard exercise movement library. It identifies key points of the human body based on the balance exercise image data of the exerciser to obtain an action sequence. It compares the similarity with the standard exercise movement sequence stored in the standard exercise movement library to obtain the name of the exerciser's action and determine whether the exerciser's action meets the requirements. At the same time, it calculates the exerciser's current center of gravity and support area and feeds the processing results back to the indication and warning module. The warning module includes a display screen and a speaker. When the distance between the exerciser's movement sequence and the standard movement sequence exceeds a threshold, the exerciser is alerted through the speaker, and the method to improve the movement is prompted on the display screen according to the direction of deviation. When the distance between the projection point of the body's center of gravity and the edge of the support area is lower than the alarm threshold, a fall alarm is issued through the speaker.

Images