Balance test method and system based on images
The Microsoft Kinect V2 sensor collects and processes three-dimensional motion data and calculates key biomechanical parameters, solving the subjectivity and cost problems of existing balance testing methods, and providing an efficient and personalized balance training solution, suitable for a variety of environments.
Patent Information
- Application Number
- CN202410821033.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-24
- Publication Date
- 2024-11-08
AI Technical Summary
The existing balanced testing methods are subjective, costly and complex in operation, and are difficult to popularize in daily rehabilitation practices and family environments, and lack objective quantitative assessments.
The Microsoft Kinect V2 sensor is used to collect and process three-dimensional motion data, and the key biomechanical parameters such as COM, XCOM and PLOS are calculated through digital filtering and machine learning algorithms, providing quantitative balanced training intensity evaluation and personalized training solutions.
Improve the accuracy, efficiency and personalization of balanced training, and provides low-cost, easy-to-implement balanced ability assessment and training programs, suitable for clinical, family and community environments, significantly optimizing training results and accessibility.
Smart Images

Figure CN118902385A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of balance testing, and in particular relates to an image-based balance testing method and system. Background Art
[0002] Balance testing plays a key role in clinical, research and rehabilitation fields. First, it is an important means of evaluating an individual's balance ability and can help identify potential balance problems, especially for the elderly, athletes or people with special health conditions. Balance testing can also be used to monitor changes in balance over time to help develop personalized rehabilitation plans and interventions. In rehabilitation, by assessing balance, medical professionals can better understand the patient's rehabilitation progress and adjust treatment plans to improve rehabilitation effectiveness. Secondly, balance testing also plays an important role in the field of sports and sports. By performing balance tests regularly, athletes can assess their sports skills and the risk of potential injuries, thereby developing more effective training plans and preventive measures. Balance testing also plays a key role in improving sports performance and reducing the risk of sports injuries. In addition, for people involved in specific activities or occupations, balance testing can help predict and prevent balance-related accidents, such as falls and injuries.
[0003] Through the above analysis, the problems and defects of the prior art are as follows:
[0004] (1) Subjectivity and accuracy issues: Current balance test methods, such as the Belfast Balance Scale (BBS) and the ABC, mainly rely on questionnaires and subjective judgments of professionals, are easily influenced by personal experience and judgment criteria, and lack objective quantitative evaluation.
[0005] (2) High cost and requirement of professional equipment: Although motion capture systems and force plates can provide accurate three-dimensional motion data and measurements of the center of mass (COM) and center of pressure (COP), providing a scientific basis for balance assessment, these instruments are expensive and require professional operation, limiting their popularity in daily rehabilitation practice and home environments.
[0006] (3) Complex operation and resource consumption: The traditional balance test process is cumbersome and requires professionals to operate and interpret the results. The test is also usually time-consuming, which makes it difficult to implement in a fast-paced clinical environment and for individuals who wish to train at home. Summary of the invention
[0007] In view of the problems existing in the prior art, the present invention provides an image-based balance testing method and system.
[0008] The present invention is implemented as follows: a balance test method based on image, comprising:
[0009] Step 1: Equipment Installation and Preparation
[0010] The Microsoft Kinect V2 sensor was selected as the motion capture device and installed in a position that can fully cover the test area. The acquisition frequency is 30 frames per second to obtain high-definition RGB depth information and infrared images.
[0011] Step 2: 3D motion data acquisition module
[0012] The Kinect sensor is used to capture the three-dimensional position data of the main body parts such as the head, torso and lower limbs in real time.
[0013] Step 3: 3D motion data processing module
[0014] The collected data were input into specially developed software for processing, first preprocessed by digital filtering method, and then the key biomechanical parameters of COM, XCOM and PLOS were calculated.
[0015] Step 4: Data Analysis and Evaluation
[0016] The processed data will be comprehensively analyzed and the balance training intensity will be quantitatively evaluated using XCOM and PLOS, and a balance training program will be customized for the individual accordingly.
[0017] Furthermore, the three-dimensional motion data acquisition module:
[0018] The Kinect sensor is used to capture the three-dimensional position data of the main body parts such as the head, torso and lower limbs in real time.
[0019] Furthermore, the three-dimensional motion data processing module:
[0020] The collected data were input into specially developed software for processing, first preprocessed by digital filtering method, and then the key biomechanical parameters of COM, XCOM and PLOS were calculated.
[0021] Further, the specific calculation method of the three-dimensional motion PLOS is as follows:
[0022] Determine the location of COM:
[0023] In a static standing position, the center of mass COG, or COM, is projected onto the ground between the legs. It is calculated by adding the mass of each body part multiplied by its position relative to a reference point and dividing by the total mass:
[0024]
[0025] Among them, COM static is the center of mass of the entire body at rest, m i is the mass of the ith body part, ri is the 3D position of the COM of the ith body part relative to the reference point, M total is the total mass of the entire body;
[0026] Predict the maximum range of COM movement in AP and ML directions:
[0027] ΔCOM=l·sinθ
[0028] Where l is the distance from the body part to the support point, and θ is the angle at which the body is tilted;
[0029] Assuming that COM should remain inside the supporting substrate BOS:
[0030] PLOS=COM static +ΔCOM.
[0031] Furthermore, the specific calculation method of XCOM is as follows:
[0032] Calculate the characteristic frequency ω0: ω0 is the natural frequency of the inverted pendulum model, which is used to measure the response speed of the system to disturbances;
[0033] Calculated from the gravitational acceleration g and the vertical distance (l) of COM relative to the supporting base BOS:
[0034]
[0035] Calculation of the extrapolated center of mass XCOM:
[0036]
[0037] Among them, COM position is the position of the center of mass in the global coordinate system.
[0038] Further, the data analysis and evaluation:
[0039] The processed data will be comprehensively analyzed and the balance training intensity will be quantitatively evaluated using XCOM and PLOS, and a balance training program will be customized for the individual accordingly.
[0040] Another object of the present invention is to provide an image-based balance test system, comprising:
[0041] The device installation module is used to select the Microsoft Kinect V2 sensor as the motion capture device and install it in a position that can fully cover the test area. The whole body movement of the individual is captured at a frequency of 30 frames per second, and high-definition RGB depth information and infrared images are obtained.
[0042] The three-dimensional motion data acquisition module is used to capture the three-dimensional position data of the main body parts of the head, torso and lower limbs in real time through the Kinect sensor.
[0043] The three-dimensional motion data processing module inputs the collected data into specially developed software for processing. It is first pre-processed by digital filtering method, and then the key biomechanical parameters of COM, XCOM and PLOS are calculated.
[0044] The data analysis and evaluation module is used to quantify and evaluate the intensity of balance training using XCOM and PLOS, and customize balance training programs for individuals accordingly.
[0045] Another object of the present invention is to provide a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the image-based balance test method.
[0046] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the steps of the image-based balance test method.
[0047] Another object of the present invention is to provide an information data processing terminal, wherein the information data processing terminal is used to implement the image-based balance test system.
[0048] In combination with the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solutions to be protected by the present invention are as follows:
[0049] First, the present invention significantly improves the accuracy, efficiency and personalization level of balance training through a series of carefully designed steps. By using the Microsoft Kinect V2 sensor to capture three-dimensional motion data and combining advanced data processing and analysis technology, this technical solution can not only provide customized balance training programs for individuals, but also provide real-time feedback on training effects, thereby significantly optimizing the balance training process and improving training results.
[0050] Step 1: Equipment installation and preparation stage, by selecting a high-precision Microsoft KinectV2 sensor and ensuring that it fully covers the test area. Compared with traditional motion capture technology, this invention significantly improves the accuracy and coverage of data collection. This step ensures that the subtle movement changes of individuals during balance training can be captured, providing high-quality raw data for subsequent data analysis, which is difficult to achieve with traditional technology.
[0051] Step 2: The 3D motion data acquisition module captures the 3D position data of the head, torso, lower limbs and other major body parts of the individual in real time when performing balance training tasks. Compared with the prior art, the present invention can provide more comprehensive and detailed motion data. This advantage comes from the high acquisition frequency (30 frames per second) of the Kinect sensor and the acquisition of high-definition RGB depth information and infrared images, which makes the analysis of motion data more accurate and provides personalized training.
[0052] Step 3: The 3D motion data processing module optimizes the data analysis process through advanced data processing techniques (such as digital filtering) and machine learning algorithms, and calculates key biomechanical parameters such as COM, XCOM and PLOS. The innovation of this step is that it can accurately analyze and evaluate the balance ability of individuals and quantify the training intensity. This method significantly improves the scientificity and pertinence of the training program design, making the training more in line with the actual needs of individuals.
[0053] Step 4: Data analysis and evaluation phase. This technical solution can not only quantitatively evaluate the intensity of balance training through comprehensive processed data, but also customize balance training programs according to individual performance. This customized training program can ensure that each training is carried out at the correct intensity to maximize the training effect. Compared with traditional balance training methods, this data-driven personalized training program design significantly improves the effectiveness and efficiency of training.
[0054] Compared with the existing balance training test method, the present invention has the following advantages:
[0055] 1. Provide quantified training targets
[0056] Using POSZ can provide physical therapists and clinical exercise physiologists with a clear quantitative goal, allowing them to specifically report individual progress (e.g., user A has reached 80% of the anterior-posterior stability zone limit). This method of quantifying progress is superior to the difficulty level or score report in traditional rehabilitation training, making the evaluation of training effects more accurate and intuitive.
[0057] 2. Real-time performance feedback
[0058] By combining POSZ with virtual reality (VR), individuals can be provided with real-time feedback on their performance. This helps users adjust their movements in a timely manner, thereby improving their balance ability. This method makes the training process more interactive and helps adjust the difficulty of balance training.
[0059] 3. Standardization of training programs
[0060] The application of POSZ promotes the standardization of balance training programs and training. In the past, clinicians or rehabilitation therapists judged which tasks were the "most difficult tasks" based on personal experience. This experience-based judgment leads to inconsistent training intensity because different clinicians have different understandings of the definition of "most difficult". The present invention can be combined with intelligent technologies such as OpenPose and Blazepose (Zheng et al., 2023; Bazarevsky et al., 2020) to help clinicians more objectively and accurately judge the intensity of balance training and whether training tasks need to be adjusted to achieve the desired training effect. By monitoring POSZ, clinicians can adjust training tasks based on quantitative data rather than personal experience alone, thereby providing a more personalized and scientific training method, avoiding the traditional trial and error method, and improving training efficiency.
[0061] 4. Comparability of rehabilitation training programs
[0062] The quantitative definition of POSZ allows objective comparisons between different balance training methods, an advantage that enables researchers and clinicians to more effectively evaluate and select the most appropriate training program.
[0063] 5. Development of adaptive and intuitive balance training programs
[0064] The balance test method for rehabilitation training based on POSZ can automatically adjust the difficulty as the individual's balance ability improves, making the training process more personalized and effective. Future balance training can accumulate more points based on the POSZ value achieved by the individual, or require the individual to maintain a specific POSZ value for additional rewards. This adaptive training model brings new features to the field of balance training, making it more attractive and challenging.
[0065] The present invention significantly optimizes the accuracy and personalization of balance training by combining high-precision three-dimensional motion capture technology and advanced data processing methods, and provides a low-cost, high-efficiency balance training and assessment solution. This system is expected to be widely used in a variety of environments such as medical rehabilitation, home and community, and is particularly suitable for people who need to continuously monitor and improve their balance ability, such as the elderly and rehabilitation patients. It can not only provide rehabilitation therapists with real-time and accurate balance ability assessment data, but also help them design personalized training programs for more patients, thereby improving rehabilitation efficiency and reducing the cost of social rehabilitation services, and has significant social and economic value.
[0066] The present invention proposes an innovative image-based balance testing method, which is designed for accurately evaluating and improving the balance ability of individuals in rehabilitation training. By combining an advanced markerless motion capture system (Microsoft Kinect V2 sensor) and in-depth biomechanical analysis, this method can capture and process individual motion data and extract key biomechanical parameters to evaluate balance strength. Special emphasis is placed on the calculation of the extrapolated center of mass (XCOM) and the predicted point of instability (PLOS), and these two core factors are used to customize effective balance training programs for individuals, aiming to improve the individual's balance control ability and overall health level. Using this advanced image-based balance testing method, a tailored balance training program is provided for individuals, including a schematic diagram of the equipment connection built for the real environment of rehabilitation training. This technical solution covers the entire process from equipment installation and preparation, three-dimensional motion data acquisition and processing, to data analysis and evaluation.
[0067] The present invention provides a balance ability assessment and training system based on low-cost motion capture technology, which aims to provide individuals with an objective, easily accessible and cost-effective balance test and training program, and mainly solves the following technical problems:
[0068] (1) Achieve objective and quantitative evaluation: By adopting low-cost depth sensors and motion capture technology, the present invention provides an accurate, data-based evaluation method for balance ability, overcoming the subjectivity and inconsistency problems in traditional evaluation.
[0069] (2) Reduced cost and equipment dependence: Compared with high-end motion capture systems and force plates, the present invention uses economical and practical depth sensing technology, which significantly reduces the cost of balance testing and training, making it more suitable for clinical and home environments.
[0070] (3) Simplify the operation process and improve accessibility: The present invention is easy to operate and does not require complicated equipment settings and professional skills, so that balance testing and training are not limited to professional medical or research institutions, but can also be easily implemented by ordinary individuals and families.
[0071] In summary, the present invention solves the problems of subjectivity, high cost and operational complexity of traditional balance test methods by introducing innovative technical solutions, and provides an objective, economical and easy-to-implement balance ability assessment and training method. This method can not only promote the improvement of individual health and rehabilitation effects, but also help promote the implementation of balance testing and training in a wide range of environments, including clinical, family and community.
[0072] Second, the working principle of the image-based balance test method provided by the present invention is described in detail as follows:
[0073] 1. Equipment installation and preparation
[0074] Select the Microsoft Kinect V2 sensor: Microsoft Kinect V2 is a motion capture device that integrates an RGB camera, a depth sensor, and an infrared camera. It can capture motion data in three-dimensional space in real time.
[0075] Installation location: Install the Kinect V2 sensor in a position that fully covers the test area to ensure that the tester's main body parts (such as head, torso, and lower limbs) can be effectively captured by the sensor when performing the balance test.
[0076] Acquisition frequency: Set the acquisition frequency to 30 frames per second to ensure the continuity and accuracy of the data. At the same time, the sensor can obtain high-definition RGB depth information and infrared images, providing a rich source of information for subsequent data processing.
[0077] 2. 3D motion data acquisition module
[0078] Real-time capture: Using the RGB camera and depth sensor of the Kinect V2 sensor, the 3D position data of the tester's head, torso, lower limbs and other major body parts are captured in real time. These data are usually expressed in the form of points, forming a series of 3D coordinate points.
[0079] Data synchronization: Ensure that the data collected by the RGB camera and depth sensor can be synchronized for subsequent data processing and analysis.
[0080] 3. 3D motion data processing module
[0081] Digital filtering: The collected 3D position data is input into specially developed software for processing. First, the data is pre-processed by digital filtering methods (such as Kalman filtering, median filtering, etc.) to eliminate noise and interference and improve the accuracy of the data.
[0082] Calculation of key biomechanical parameters: Based on the preprocessed data, the software calculates key biomechanical parameters such as COM (center of mass), XCOM (extra center of mass) and PLOS (pressure center line). These parameters are important indicators for evaluating the balance state.
[0083] 4. Data Analysis and Evaluation
[0084] Quantitative evaluation: The calculated XCOM and PLOS parameters are used to quantitatively evaluate the balance of the test subject. By comparing the differences between these parameters and standard values or reference values, the test subject's balance ability can be evaluated.
[0085] Training intensity assessment: Evaluate the intensity of balance training based on the changes in parameters such as XCOM and PLOS. If the parameters change significantly, it means that the training intensity is moderate; if the parameters do not change significantly, it is necessary to adjust the training plan or increase the training intensity.
[0086] Customized training program: Based on the tester's balance ability and training intensity assessment results, a personalized balance training program is customized for the tester. These programs can include specific training movements, training frequency, and training intensity, etc., designed to help the tester improve balance ability and reduce the risk of falls.
[0087] In general, this image-based balance testing method captures and analyzes the test subject's three-dimensional motion data in real time, uses key biomechanical parameters to quantitatively evaluate the balance state, and customizes personalized balance training programs based on the evaluation results, providing an effective and convenient way to improve balance ability. BRIEF DESCRIPTION OF THE DRAWINGS
[0088] Figure 1 It is a flow chart of an image-based balance testing method provided by an embodiment of the present invention.
[0089] Figure 2 It is a structural block diagram of an image-based balance test system provided by an embodiment of the present invention.
[0090] Figure 3 It is a schematic diagram of the composition of the evaluation system provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0091] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.
[0092] like Figure 1 As shown, an image-based balance test method provided by an embodiment of the present invention includes the following steps:
[0093] S101: Equipment Installation and Preparation
[0094] The Microsoft Kinect V2 sensor was selected as the motion capture device and installed in a position that can fully cover the test area. The acquisition frequency is 30 frames per second to obtain high-definition RGB depth information and infrared images;
[0095] S102: 3D motion data acquisition module
[0096] The Kinect sensor is used to capture the three-dimensional position data of the main body parts of the head, torso, and lower limbs in real time;
[0097] S103: 3D motion data processing module
[0098] The collected data were input into specially developed software for processing, first preprocessed by digital filtering methods, and then the key biomechanical parameters of COM, XCOM and PLOS were calculated;
[0099] S104: Data Analysis and Evaluation
[0100] The processed data will be comprehensively analyzed and the balance training intensity will be quantitatively evaluated using XCOM and PLOS, and a balance training program will be customized for the individual accordingly.
[0101] The three-dimensional motion data acquisition module provided by the embodiment of the present invention:
[0102] The Kinect sensor is used to capture the three-dimensional position data of the main body parts such as the head, torso and lower limbs in real time.
[0103] The three-dimensional motion data processing module provided by the embodiment of the present invention:
[0104] The collected data were input into specially developed software for processing, first preprocessed by digital filtering method, and then the key biomechanical parameters of COM, XCOM and PLOS were calculated.
[0105] The specific calculation method of PLOS for three-dimensional motion provided by the embodiment of the present invention is as follows:
[0106] Determine the location of COM:
[0107] In a static standing position, the center of mass COG, or COM, is projected onto the ground between the legs. It is calculated by adding the product of the mass of each body part and its position relative to a reference point and then dividing by the total mass:
[0108]
[0109] Among them, COM static is the center of mass of the entire body at rest, m i is the mass of the ith body part, r i is the 3D position of the COM of the ith body part relative to the reference point, M total is the total mass of the entire body;
[0110] Predict the maximum range of COM movement in AP and ML directions:
[0111] ΔCOM=l·sinθ
[0112] Where l is the distance from the body part to the support point, and θ is the angle at which the body is tilted;
[0113] Assuming that COM should remain inside the supporting substrate BOS:
[0114] PLOS=COM static +ΔCOM.
[0115] The specific calculation method of XCOM provided by the embodiment of the present invention is as follows:
[0116] Calculate the characteristic frequency ω0: ω0 is the natural frequency of the inverted pendulum model, which is used to measure the response speed of the system to disturbances;
[0117] Calculated from the gravitational acceleration g and the vertical distance (l) of COM relative to the supporting base BOS:
[0118]
[0119] Calculation of the extrapolated center of mass XCOM:
[0120]
[0121] Among them, COM position is the position of the center of mass in the global coordinate system.
[0122] Data analysis and evaluation provided by the embodiments of the present invention:
[0123] The processed data will be comprehensively analyzed and the balance training intensity will be quantitatively evaluated using XCOM and PLOS, and a balance training program will be customized for the individual accordingly.
[0124] like Figure 2 As shown, an embodiment of the present invention provides an image-based balance test system, comprising:
[0125] The device installation module is used to select the Microsoft Kinect V2 sensor as the motion capture device and install it in a position that can fully cover the test area; it captures the individual's full body movement at a frequency of 30 frames per second and obtains high-definition RGB depth information and infrared images.
[0126] The three-dimensional motion data acquisition module is used to capture the three-dimensional position data of the main body parts of the head, torso and lower limbs in real time through the Kinect sensor.
[0127] The three-dimensional motion data processing module is used to input the collected data into specially developed software for processing. It is first pre-processed by digital filtering method, and then the key biomechanical parameters of COM, XCOM and PLOS are calculated.
[0128] The data analysis and evaluation module is used to quantify and evaluate the intensity of balance training using XCOM and PLOS, and customize balance training programs for individuals accordingly.
[0129] Another object of the present invention is to provide a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the image-based balance test method.
[0130] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the steps of the image-based balance test method.
[0131] Another object of the present invention is to provide an information data processing terminal, wherein the information data processing terminal is used to implement the image-based balance test system.
[0132] The present invention is specifically implemented:
[0133] The present invention proposes an innovative image-based balance testing method, which is designed for accurately evaluating and improving the balance ability of individuals in rehabilitation training. By combining an advanced markerless motion capture system (Microsoft Kinect V2 sensor) and in-depth biomechanical analysis, this method can capture and process individual motion data and extract key biomechanical parameters to evaluate balance strength. And special emphasis is placed on the calculation of the extrapolated center of mass (XCOM) and the predicted point of instability (PLOS). These two core factors are used to customize effective balance training programs for individuals, aiming to improve the individual's balance control ability and overall health level. Utilizing this advanced image-based balance testing method, a tailored balance training program is provided for individuals, including a schematic diagram of the equipment connection set up for the real environment of rehabilitation training. This technical solution covers the entire process from equipment installation and preparation, three-dimensional motion data collection and processing, to data analysis and evaluation. The detailed test method is as follows (such as Figure 3 (shown)
[0134] Step 1: Equipment installation and preparation: A Microsoft Kinect V2 sensor was selected as the motion capture device and installed in a position that fully covered the test area. The equipment settings ensured that the individual's full body motion could be accurately captured at a frequency of 30 frames per second, while obtaining high-definition RGB depth information and infrared images.
[0135] Step 2: Three-dimensional motion data acquisition module: In the prepared test environment, the individual performs a series of balance training tasks (such as standing on one leg, leaning forward and backward, etc.), and the three-dimensional position data of major body parts such as the head, torso, and lower limbs are captured in real time through the Kinect sensor.
[0136] Step 3: 3D motion data processing module (PLOS and XCOM): The collected data is input into a specially developed software for processing, which is pre-processed by digital filtering and other methods, and then key biomechanical parameters such as COM, XCOM and PLOS are calculated. The software optimizes the data analysis process based on the principles of motion biomechanics and machine learning algorithms.
[0137] Step 4: Data analysis and evaluation: After comprehensive analysis of the processed data, XCOM and PLOS are used to quantitatively evaluate the intensity of balance training, and a balance training program is customized for the individual. This technical solution can adjust the training difficulty according to the specific performance of the individual to ensure the maximum training effect.
[0138] Through this data-based assessment method, the present invention not only improves the personalization and accuracy of balance training, but also provides rehabilitation professionals with a powerful tool to monitor and adjust the patient's training plan. The innovation and practicality of the present invention make it an important advancement in the field of rehabilitation training, and it is expected to significantly improve the balance ability and quality of life of individuals.
[0139] The following is a further explanation of each step:
[0140] Step 1: (Equipment installation and preparation)
[0141] First, determine the three-dimensional motion capture system. The present invention adopts a high-precision three-dimensional motion capture system (Microsoft Kinect V2 sensor) based on unmarked points. The device should be fixed in a position that can fully cover the test area and adjusted to a suitable angle to capture full-body movements. The Kinect sensor can collect RGB depth information at a frequency of 30 frames per second. The sensor has a built-in RGB camera and an infrared sensor to obtain raw data from the environment, where the maximum resolution of the RGB camera reaches 3840×2160 and the infrared camera is 640×576. The infrared sensor projects a specific spot light pattern into the environment, which is then captured by the infrared depth sensor. The microprocessor inside the camera uses the principle of structured light to analyze the difference between the emitted and reflected spot patterns. In addition, the sensor also uses the time of flight (ToF) method to estimate the distance of the object from the sensor by measuring the time it takes for a light wave or other signal to be emitted from the transmitter and detected by the receiver. The sensor can also perform highly sophisticated image processing technology. The sensor first captures the raw data of the environment, then analyzes it pixel by pixel, identifies pixels with the same color saturation, brightness (or intensity), and depth (that is, the distance from the sensor), and clusters these pixels with similar attributes together to accurately distinguish the boundaries of body parts of different objects in the image. In this way, the sensor is able to create a more accurate and detailed three-dimensional model, providing a basis for subsequent body part tracking and motion analysis.
[0142] The model uses multiple decision trees to analyze each pixel in parallel and select the result that best represents the characteristics of a specific body part or movement. The system is able to calculate the center point of all pixels (i.e. the center point of each body part, the center of mass) to determine its precise position in three-dimensional space.
[0143] Step 2: (3D motion data acquisition module)
[0144] After the test environment is ready, the test subjects will undergo a series of balance training. Balance training tasks are the core tool for assessing individual balance ability. By simulating balance challenges in daily life, they directly test the individual's ability to maintain stability and adjust body posture. These tasks, such as standing on one leg and tilting forward and backward, not only test the individual's balance control under static and dynamic conditions, but also reveal their adaptability to complex balance challenges. Effectively assessing and improving balance ability is essential to reducing the risk of falling, improving athletic performance, and promoting overall health. By integrating balance training tasks and 3D motion capture technology, the individual's balance performance can be accurately quantified, and personalized training programs can be designed for them to achieve significant improvements in balance ability. In order to capture the 3D position data of major body parts (such as head, torso, and lower limbs) in real time during these activities, an advanced markerless motion capture system, the Microsoft Kinect sensor, is used.
[0145] The Kinect sensor collects RGB depth data of the test individual at a frequency of 30 frames per second, which can record the subtle changes of each action in detail. These data include not only precise spatial coordinates, but also time series information, ensuring the accuracy of subsequent analysis. In order to improve data quality, the collected raw data will be pre-processed through a digital filter to remove environmental noise and other interference factors to ensure data purification and accuracy.
[0146] During data collection, the sensor analyzes each pixel and groups similar pixels together based on the consistency of color saturation, brightness, and depth, a step that is critical for distinguishing different body parts and understanding their location in three-dimensional space. Through this highly sophisticated image processing technology, Kinect is able to create an accurate three-dimensional body model, providing a deep understanding of the test's balance ability.
[0147] The 3D position data is then transferred to a computer configured with high-performance hardware, equipped with an Intel i7-9700 processor, 16GB RAM, and an NVIDIA GeForce RTX 2080 graphics processing unit to minimize processing latency, and further analyzed and processed through a body tracking software development kit (SDK) developed by Microsoft.
[0148] Step 3: (3D motion data processing module: PLOS and XCOM)
[0149] The collected three-dimensional position data is processed using the developed software. The three-dimensional data is pre-processed to ensure the accuracy and reliability of the data by removing noise and unnecessary interference through digital filters, and defining each part of the body to accurately calculate its dynamic characteristics. Subsequently, the software uses machine learning algorithms, especially multiple decision trees to analyze each pixel in parallel to identify and classify the dynamics of different body parts. This method improves the accuracy and efficiency of the analysis, allowing the software to quickly and accurately calculate the key biomechanical parameters of the characteristic moment based on these pre-processed data, including but not limited to joint angles, link lengths, and center of mass (COM). These parameters are crucial for evaluating balance ability. COM is used to evaluate the two key factors that determine the strength of balance in rehabilitation training, namely PLOS and XCOM.
[0150] Predicted Point of Loss of Stability (PLOS): PLOS defines the region where the center of mass (COM) of an individual's body is expected to be when performing balance training. The basic assumption in determining the boundaries of PLOS is that individuals perform balance training in a standing position and the movement starts from a two-legged standing position. In the static standing position, the center of gravity (COG), which is the projection of the COM on the ground, is located between the legs. When an individual leans forward, backward, or sideways, the COG moves accordingly, and its maximum excursion points in the anterior-posterior (AP) and lateral (ML) directions form an ellipse-like vertex, which is similar to a cone (calculated based on the developer's previous data). Because this stability cone is not accurately represented in people with balance problems, using theoretical stability limits to evaluate is not ideal. Although theoretical stability limits can help identify potential limits, by defining the range of movement of the COM in a quantitative way, PLOS can not only help understand the stability level of an individual when performing a specific balance task, but also guide the design of training programs to ensure that the training tasks are both challenging and within the stability limits, thereby promoting the improvement of balance ability and maximizing rehabilitation effects. The specific calculation method of PLOS is as follows:
[0151] Determine the location of COM: In a static standing position, the center of gravity (COG), the projection of COM on the ground, is located between the legs. It can be calculated by adding the product of the mass of each part of the body and its position relative to a reference point, and then dividing by the total mass:
[0152]
[0153] Among them, COM static is the center of mass of the entire body at rest, m i is the mass of the ith body part, r iis the 3D position of the COM of the ith body part relative to the reference point, M total It is the total mass of the whole body. The mass of each part of the body and the position of COM can be calculated by sports biomechanics methods and software.
[0154] Estimate the range of motion of the COM: In static standing or specific balance training situations, predict the maximum range of motion of the COM in the AP and ML directions based on the movements performed by the individual (e.g., body leaning, reaching, etc.):
[0155] ΔCOM=l·sinθ
[0156] Where l is the distance from the body part to the support point, and θ is the angle at which the body is tilted.
[0157] Determine the point of instability (PLOS): The instability point can be determined by predicting the potential displacement of the COM when it is disturbed. One or more instability points determine the instability boundary (similar to a cone), assuming that the COM should remain inside the base of support (BOS) to maintain equilibrium:
[0158] PLOS=COM static +ΔCOM
[0159] Extrapolated center of mass (XCOM): It is an important metric used to assess individual stability in sports biomechanics, proposed by Hof in 2005 [1]. The concept is based on the inverted pendulum model. By considering the velocity of COM relative to the height of the base of support (BOS) and the interaction of various parts of the body, a new COM value is generated to simulate the dynamics of an individual when performing a balance task. The stability metric is the sum of the body's center of mass COG and velocity, divided by the system's eigenfrequency. This means that when an individual performs a more challenging balance task, such as transitioning from standing on two feet to standing on one foot on the toes, the value of XCOM will increase. This shows that XCOM not only reflects the stability of the body under static conditions, but also reveals the stability level of an individual when performing more complex dynamic balance tasks.
[0160] XCOM has been shown to be a very useful stability measure in gait research. However, its application in balance rehabilitation training is relatively rare. Individuals with larger XCOM excursions during a given task show lower stability and control than those with smaller XCOM excursions. XCOM displacement in the anteroposterior (AP) and lateral (ML) directions provides information about the range of COM movement during a task, which is extremely important for evaluating the effectiveness of balance training and adjusting the difficulty of training.
[0161] XCOM can be calculated directly from the position data captured by Kinect. By analyzing Kinect data, the position and speed of COM can be obtained, where the speed of COM is defined as the rate of change of COM position, and the length is defined as the distance from the COM position to the rotation point. The processing and analysis of these data can accurately calculate XCOM, and then evaluate the stability level of individuals during balance rehabilitation training. The specific calculation method of XCOM is as follows:
[0162] Calculate the eigenfrequency (ω0): ω0 is the natural frequency of the inverted pendulum model and is a measure of how quickly the system responds to a disturbance. It is calculated from the gravitational acceleration (g) and the vertical distance (l) of the COM relative to the base of support (BOS):
[0163]
[0164] Calculation of the extrapolated center of mass (XCOM): XCOM is calculated by taking into account the center of mass velocity (V COM ), the predicted position of the COM relative to the BOS when . It is obtained by dividing the velocity of the center of mass by the characteristic frequency, reflecting the tendency of the center of mass to move:
[0165]
[0166] Among them, COM position is the position of the center of mass in the global coordinate system.
[0167] Step 4 (Data Analysis and Evaluation)
[0168] After the acquisition and processing of the three-dimensional motion data is completed, the last step is to conduct in-depth analysis and evaluation of these data to quantify the intensity of balance training and provide targeted training recommendations for individuals. The POSZ concept introduced here provides a quantitative tool to evaluate and define the intensity of balance training tasks.
[0169] Percentage OverStabilityZone (POSZ)
[0170] POSZ is a measure of the percentage of the extrapolated center of mass (XCOM) that can be located within the boundaries of the predicted point of instability (PLOS) when the individual is standing (feet shoulder-width apart). This means that when the excursion of XCOM is within the PLOS, it indicates that the balance task being performed is relatively low intensity. Balance training tasks should be moderately to highly challenging, encouraging individuals to move their COM beyond their accustomed range.
[0171] POSZ allows the intensity of a balance task to be defined in a quantitative manner. POSZ is the percentage of real-time XCOM excursions over PLOS. This variable combines the scores for the anteroposterior (AP) and lateral (ML) directions to provide a single measure of the intensity of a balance training exercise. This single score allows different balance exercises to be compared based on their motor tasks, as some exercises are more challenging for stability in the anteroposterior direction, while other games are more challenging in the lateral direction. The specific calculation of POSZ is as follows:
[0172] Calculate the offset of XCOM in AP and ML directions:
[0173]
[0174]
[0175] Combine the scores in AP and ML to calculate the comprehensive POSZ score:
[0176]
[0177] Therefore, an individual who records a lower POSZ percentage demonstrates better stability control. Conversely, an individual with a higher score reflects poorer balance control during balance training. Finally, this quantitative measure will aid in calculating appropriate thresholds for classifying low, moderate, and high intensity balance training.
[0178] Through POSZ, not only can the intensity of balance training be quantified, but the difficulty of training can also be adjusted according to individual performance to achieve the best training effect. This data-based evaluation method not only increases the personalization and precision of training, but also provides rehabilitation professionals with a powerful tool to monitor the patient's progress and adjust the training plan to ensure that each individual can be effectively trained in balance training suitable for their ability level.
[0179] The technical solution of the present invention can be used not only alone, but also in combination with other assessment tools and methods to provide a more comprehensive analysis of individual balance ability, as follows:
[0180] Combined with physical assessment tools: This technology solution can be used in combination with traditional physical assessment tools (such as force plates, dynamometers, etc.) to obtain more comprehensive information about an individual's balance ability. For example, by comparing the motion data collected by the Kinect sensor with the pressure distribution data recorded by the force plate, an individual's center of gravity control ability and stability can be more accurately assessed.
[0181] Combined with questionnaires and subjective assessments: This technology solution can also be combined with questionnaires and subjective assessment methods to obtain feedback on individual balance feelings and training experience. This multi-dimensional assessment method can help trainers and rehabilitation professionals more comprehensively understand individual needs and progress, thereby designing more personalized and effective training plans.
[0182] By combining this technical solution with other assessment tools and methods, it is not only possible to evaluate and understand an individual's balance ability from multiple dimensions, but also to adjust and optimize the training program based on the comprehensive assessment results. This comprehensive assessment and personalized training method will greatly improve the effect of balance training and promote the rapid improvement of individual balance ability.
[0183] In summary, the present invention provides a method for evaluating the balance strength of rehabilitation training using markerless motion capture technology. By accurately calculating PLOS and XCOM, and developing a comprehensive balance scoring system, this technology not only enhances the interactivity and fun of rehabilitation training, but also ensures the safety and effectiveness of individuals when performing physical activities. In this way, the present invention promotes progress in the fields of sports biomechanics and individual ergonomics, and provides new tools and methods for developing more inclusive, adaptable and educationally valuable physical games.
[0184] It should be noted that the embodiments of the present invention can be implemented by hardware, software, or a combination of software and hardware. The hardware portion can be implemented using dedicated logic; the software portion can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated design hardware. A person of ordinary skill in the art will appreciate that the above-mentioned devices and methods can be implemented using computer executable instructions and / or contained in a processor control code, such as a carrier medium such as a disk, CD or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier.
[0185] The device and its modules of the present invention can be implemented by hardware circuits such as very large scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, etc., or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., and can also be implemented by various types of processors. This image-based balance test method utilizes the Microsoft Kinect V2 sensor for data acquisition and processing, and the specific steps are as follows:
[0186] 1. Equipment installation and preparation:
[0187] A Microsoft Kinect V2 sensor was selected and mounted in a location that would provide full coverage of the test area.
[0188] The sensor acquisition frequency is set to 30 frames per second to obtain high-definition RGB depth information and infrared images.
[0189] 2. Three-dimensional motion data acquisition module:
[0190] The Kinect sensor is used to capture the three-dimensional position data of major body parts such as the head, torso and lower limbs in real time.
[0191] 3. Three-dimensional motion data processing module:
[0192] The collected data are input into specially developed software for processing.
[0193] The data is first preprocessed by digital filtering method to remove noise and unnecessary interference.
[0194] Subsequently, key biomechanical parameters such as COM (center of mass), XCOM (anteroposterior reference point of support) and PLOS (anteroposterior line of support) were calculated.
[0195] 4. Data analysis and evaluation:
[0196] The processed data were comprehensively analyzed and the intensity of balance training was quantitatively evaluated using parameters such as XCOM and PLOS.
[0197] Based on the assessment results, a personalized training program is customized for the individual to suit their balance training needs.
[0198] Through the above steps, this method can solve the technical problems existing in the existing technology and achieve significant technical progress. By using the efficient data acquisition and processing of the Kinect sensor, combined with specially developed software to analyze and evaluate the data, accurate quantification of individual balance status and formulation of personalized training programs are achieved, providing an effective technical means for balance training.
[0199] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with the technical field within the technical scope disclosed by the present invention and within the spirit and principle of the present invention should be covered by the protection scope of the present invention.
Claims
1. A balance test method based on image, characterized in that: The following steps are involved: Step 1: Equipment Installation and Preparation The Microsoft Kinect V2 sensor was selected as the motion capture device and installed in a position that can fully cover the test area. The acquisition frequency is 30 frames per second to obtain high-definition RGB depth information and infrared images. Step 2: 3D motion data acquisition module The Kinect sensor is used to capture the three-dimensional position data of the main body parts such as the head, torso and lower limbs in real time. Step 3: 3D motion data processing module The collected data were input into specially developed software for processing, first preprocessed by digital filtering method, and then the key biomechanical parameters of COM, XCOM and PLOS were calculated. Step 4: Data Analysis and Evaluation The processed data will be comprehensively analyzed and the balance training intensity will be quantitatively evaluated using XCOM and PLOS, and a balance training program will be customized for the individual accordingly.
2. The image-based balance test method according to claim 1, characterized in that: The three-dimensional motion data acquisition module: The Kinect sensor is used to capture the three-dimensional position data of the main body parts such as the head, torso and lower limbs in real time.
3. The image-based balance test method according to claim 1, characterized in that: The three-dimensional motion data processing module: The collected data were input into specially developed software for processing, first preprocessed by digital filtering method, and then the key biomechanical parameters of COM, XCOM and PLOS were calculated.
4. The image-based balance test method according to claim 3, characterized in that: The specific calculation method of the three-dimensional motion PLOS is as follows: Determine the location of COM: In a static standing position, the center of mass COG, or COM, is projected onto the ground between the legs. It is calculated by adding the product of the mass of each body part and its position relative to a reference point and then dividing by the total mass: Among them, COM static is the center of mass of the entire body at rest, m i is the mass of the ith body part, r i is the 3D position of the COM of the ith body part relative to the reference point, M total is the total mass of the entire body; Predict the maximum range of COM movement in AP and ML directions: ΔCOM=l·sinθ Where l is the distance from the body part to the support point, and θ is the angle at which the body is tilted; Assuming that COM should remain inside the supporting substrate BOS: PLOS=COM static +ΔCOM。 5. The image-based balance test method according to claim 3, characterized in that: The specific calculation method of XCOM is as follows: Calculate the characteristic frequency ω0: ω0 is the natural frequency of the inverted pendulum model, which is used to measure the response speed of the system to disturbances; Calculated from the gravitational acceleration g and the vertical distance (l) of COM relative to the supporting base BOS: g=9.81m·s 2 Calculation of the extrapolated center of mass XCOM: Among them, COM position is the position of the center of mass in the global coordinate system.
6. The image-based balance test method according to claim 1, wherein: The data analysis and evaluation: The processed data will be comprehensively analyzed and the balance training intensity will be quantitatively evaluated using XCOM and PLOS, and a balance training program will be customized for the individual accordingly.
7. An image-based balance test system for implementing the image-based balance test method according to any one of claims 1 to 6, characterized in that: The image-based balance test system includes: The device installation module is used to select the Microsoft Kinect V2 sensor as the motion capture device and install it in a position that can fully cover the test area; capture the individual's full body movement at a frequency of 30 frames per second, and obtain high-definition RGB depth information and infrared images. The three-dimensional motion data acquisition module is used to capture the three-dimensional position data of the main body parts of the head, torso and lower limbs in real time through the Kinect sensor. The three-dimensional motion data processing module is used to input the collected data into specially developed software for processing. It is first pre-processed by digital filtering method, and then the key biomechanical parameters of COM, XCOM and PLOS are calculated. The data analysis and evaluation module is used to quantify and evaluate the intensity of balance training using XCOM and PLOS, and customize balance training programs for individuals accordingly.
8. A computer device, characterized in that: The computer device includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the image-based balance test method according to any one of claims 1 to 6.
9. A computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the steps of the image-based balance test method according to any one of claims 1 to 6.
10. An information data processing terminal, characterized in that: The information data processing terminal is used to implement the image-based balance test system as described in claim 7.
