Lower limb movement monitoring and guiding method

Abstract

The invention discloses a lower limb movement monitoring and guiding method. A hardware part comprises a flexible induction sheet array, a smart phone, a built-in RGB-D camera of the smart phone and an edge calculation module. The flexible induction sheet array comprises five flexible induction sheets, each flexible induction sheet comprises a skin contact layer, a sensor layer, a flexible FPC substrate and a protective layer, the sensor layer is provided with a three-axis MEMS acceleration sensor, a PVDF piezoelectric film sensor, a flexible FPC substrate integrated signal conditioning module and a Bluetooth communication module; the flexible induction sheet is provided with an anatomical positioning mark; the lower limb movement monitoring and guiding method comprises the following steps: S1, cleaning; s2, a step of attaching a flexible induction sheet; s3, an automatic positioning calibration step; s4, a data transmission step; s5, a data fusion and calculation analysis step; s6, a normalization evaluation step; and S7, a feedback and optimization step. Quantitative assessment of lower limb movement can be effectively achieved, and the monitoring and guiding effect is good.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent remote motion monitoring, and particularly to a lower limb motion monitoring and guiding method. Background Art

[0002] The Chinese invention patent application with the patent application number: 202210306201.9 and the technical solution name: A method for monitoring human lower limb motion based on IMU discloses the following technical solution. Specifically: A method for monitoring human lower limb motion based on IMU includes: detecting whether the current posture of the monitored object conforms to the reference posture according to the acceleration data measured by the IMU; in the case where the current posture conforms to the reference posture, obtaining the reference Euler angles according to the Euler angles of the IMU; obtaining the motion angles when the monitored object moves from the reference posture, including: if the motion axis of the lower limb when the monitored object moves from the reference posture is the X-axis of the IMU coordinate system, resolving the Euler angles of the IMU in the ZYX order, and taking the difference between the roll angle in the resolved Euler angles and the roll angle in the reference Euler angles as the motion angle; if the motion axis of the lower limb when the monitored object moves from the reference posture is the Y-axis of the IMU coordinate system, resolving the Euler angles of the IMU in the ZXY order, and taking the difference between the pitch angle in the resolved Euler angles and the pitch angle in the reference Euler angles as the motion angle.

[0003] It should be noted that for the above-mentioned method for monitoring human lower limb motion based on IMU, it has the following defects. Specifically: It cannot achieve a quantitative evaluation of the standardization of lower limb motion actions, and the monitoring and guiding effect is poor. Summary of the Invention

[0004] The purpose of the present invention is to provide a lower limb motion monitoring and guiding method for the deficiencies of the prior art. This lower limb motion monitoring and guiding method can effectively achieve a quantitative evaluation of the number of times, speed, radian, and action standardization of the user's lower limb motion, and has a good monitoring and guiding effect.

[0005] To achieve the above purpose, the present invention is realized through the following technical solutions.

[0006] A lower limb motion monitoring and guiding method. The hardware part of this lower limb motion monitoring and guiding method includes a flexible sensing sheet array and a smart phone. The smart phone is built-in with an RGB-D camera and an edge computing module; The flexible sensor sheet array includes five flexible sensor sheets; each flexible sensor sheet includes a skin contact layer, a sensor layer, a flexible FPC substrate, and a protective layer that are stacked in sequence. The sensor layer has a three-axis MEMS acceleration sensor and a PVDF piezoelectric film sensor. The wiring pattern of the flexible FPC substrate is serpentine wiring, and the flexible FPC substrate integrates a signal conditioning module and a Bluetooth communication module. The three-axis MEMS acceleration sensor and the PVDF piezoelectric film sensor are electrically connected to the input end of the signal conditioning module respectively, and the output end of the signal conditioning module is electrically connected to the Bluetooth communication module; each flexible sensor sheet is provided with an anatomical positioning mark, and the anatomical positioning mark is a visible fluorescent mark point; The lower limb movement monitoring and guidance method includes the following steps, specifically: Step S1, clean the skin of the user's lower limb and ensure no grease residue; Step S2, attach the flexible sensor sheet to the skin surface of the user's lower limb through the skin contact layer, and the attachment points of the five flexible sensor sheets are respectively: Rectus femoris: the midpoint of the line connecting the anterior superior iliac spine and the upper edge of the patella; Biceps femoris: 1 / 3 of the distance from the ischial tuberosity to the head of the fibula; Tibialis anterior: 5 cm below the tibial tuberosity, 2 cm lateral to the tibial crest; Gastrocnemius: 10 cm below the midpoint of the popliteal crease, the highest point of the gastrocnemius muscle belly; Upper edge of the patella: the center of the upper pole of the patella; Step S3, scan the visible fluorescent mark points of each flexible sensor sheet through a smart phone to complete automatic positioning and calibration, and automatically adapt to the bone proportions of users with different body types through the visible fluorescent mark points and the phone's SLAM technology; Step S4, each flexible sensor sheet is connected to the smart phone via Bluetooth through the Bluetooth communication module, and each flexible sensor sheet uses TDMA time division multiple access technology to transmit data to the smart phone; Step S5, the smart phone performs fusion processing on the user image data collected by the RGB-D camera, the motion acceleration data and motion frequency data collected by each three-axis MEMS acceleration sensor, and the muscle contraction pressure change data collected by each PVDF piezoelectric film sensor through a multi-modal data fusion algorithm, and feeds the fused data back to the motion parameter calculation engine; The motion parameter calculation engine is based on Kalman filter fused data. The motion parameter calculation engine is used to calculate and analyze the fused data to obtain specific motion feedback results; Step S6, normative evaluation: compare the motion trajectory of the user with a standard template by DTW to output a similarity score; Among them, if it is detected that the knee joint valgus > 8° or the center of gravity deviation > 20%, a real-time alarm is triggered; Step S7, Feedback and Optimization: Generate a training report and automatically upgrade the training difficulty according to the passing rate.

[0007] Among them, the signal conditioning module is an AD8232 chip.

[0008] Among them, the protective layer includes a nano waterproof coating and an electromagnetic shielding layer.

[0009] Among them, in the step S3, the automatic positioning and calibration includes a static calibration step and a dynamic calibration step; The static calibration step is: the user maintains a standing posture for 3 seconds to establish a reference model for bone length; The dynamic calibration step is: perform the standard squat action 2 times to calibrate the movement range of each joint.

[0010] Among them, the process of generating the reference model for bone length is: Bone reconstruction step: Convert the RGB-D camera image into a three-dimensional coordinate system through SLAM technology; Data mapping step: Perform Kalman filter fusion on the IMU sensor data and the visual bone key points; Real-time rendering step: Render the model using the Unity 3D engine and the ARKit framework.

[0011] Among them, in the step S7, compare the AR image of the user's real-time action with the virtual bone line annotation to clearly know the action deviation.

[0012] Among them, in the step S6, the scoring metrics include: Joint angle deviation metric: the angle of knee joint valgus; Movement rhythm metric: the ratio of current speed to target speed; Muscle activation metric: the ratio of quadriceps femoris to hamstring muscle force.

[0013] Among them, the dynamic weight allocation in the scoring algorithm logic is: Base score (60%): joint angle deviation; Bonus score (30%): muscle synergy; Penalty item (10%): detection of compensatory movements; The threshold setting in the scoring algorithm logic is: Excellent (≥85 points): all joint deviations <5°, force ratio error <10%; Qualified (60 - 84 points): main joint deviation <10°, secondary joint deviation <15°; Warning (<60 points): trigger a voice prompt and pause counting.

[0014] Compared with the prior art, the present invention has the following beneficial effects, specifically: the present invention integrates a three-axis MEMS acceleration sensor and a PVDF piezoelectric film sensor through a flexible sensing sheet to realize two-dimensional monitoring of kinematics (acceleration) and dynamics (pressure), and realizes the fusion processing of sensor data and motion image data acquired by an RGB-D camera through a multimodal data fusion algorithm, so as to realize lower limb motion monitoring and guidance of multimodal sensor fusion; in addition, after the motion parameter calculation engine feeds back the feedback result, the lower limb motion monitoring and guidance method also realizes the scoring and guidance of the user's lower limb motion through normative evaluation steps, feedback and optimization steps. Therefore, the lower limb motion monitoring and guidance method of the present invention can effectively realize the quantitative evaluation of the number of lower limb motions, speed, arc and action norms of the user, and the monitoring and guidance effect is good. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The present invention will be described below in conjunction with the accompanying drawings, but the embodiments in the accompanying drawings do not constitute a limitation of the present invention.

[0016] Figure 1 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0017] The present invention is described below in conjunction with specific implementation methods.

[0018] Embodiment 1, a lower limb movement monitoring and guidance method, the hardware part of the lower limb movement monitoring and guidance method includes a flexible sensor array, a smart phone, and the smart phone has a built-in RGB-D camera and an edge computing module.

[0019] Specifically, the flexible sensing sheet array includes five flexible sensing sheets; the flexible sensing sheets include a skin contact layer, a sensor layer, a flexible FPC substrate and a protective layer which are stacked in sequence; the sensor layer has a three-axis MEMS acceleration sensor and a PVDF piezoelectric film sensor; the wiring method of the flexible FPC substrate is serpentine wiring and the flexible FPC substrate is integrated with a signal conditioning module and a Bluetooth communication module; the three-axis MEMS acceleration sensor and the PVDF piezoelectric film sensor are electrically connected to the input end of the signal conditioning module respectively, and the output end of the signal conditioning module is electrically connected to the Bluetooth communication module; each flexible sensing sheet is provided with an anatomical positioning mark, which is a visible fluorescent marking point. For the three-axis MEMS acceleration sensor, it is used to measure the acceleration and frequency of the movement and identify the action cycle; for the PVDF piezoelectric film sensor, it is used to detect the change of muscle contraction pressure to evaluate the balance of force.

[0020] It should be explained that the signal conditioning module may be an AD8232 chip, and is used to achieve sensor signal amplification and filtering.

[0021] In addition, the protective layer includes a nano waterproof coating and an electromagnetic shielding layer; for the nano waterproof coating, it is required to meet the IP67 protection level and be able to withstand sweat immersion; for the electromagnetic shielding layer, it is used to reduce the interference of the mobile phone radio frequency signal to improve the stability of signal transmission.

[0022] It should be noted that as Figure 1 shown, the lower limb movement monitoring and guiding method includes the following steps, specifically: Step S1: Clean the lower limb skin of the user and ensure no oil residue. Step S2: Attach the flexible sensing sheet to the surface of the user's lower limb skin through the skin contact layer, and the attachment points of the five flexible sensing sheets are respectively: Rectus femoris: The midpoint of the line connecting the anterior superior iliac spine and the upper edge of the patella. Biceps femoris: The posterior 1 / 3 of the line connecting the ischial tuberosity and the head of the fibula. Tibialis anterior: 5 cm below the tibial tuberosity, 2 cm lateral to the tibial crest. Gastrocnemius: 10 cm below the midpoint of the popliteal crease, the highest point of the gastrocnemius muscle belly. Upper edge of the patella: The center of the upper pole of the patella. Step S3: Automatically locate and calibrate by scanning the visible fluorescent marker points of each flexible sensing sheet through a smart phone, and automatically adapt to the bone proportions of users with different body types through the visible fluorescent marker points and the phone's SLAM technology. Step S4: Each flexible sensing sheet is connected to the smart phone via a Bluetooth communication module, and each flexible sensing sheet uses TDMA time division multiple access technology to transmit data to the smart phone. Step S5: The smart phone performs fusion processing on the user image data collected by the RGB-D camera, the motion acceleration data and motion frequency data collected by each three-axis MEMS acceleration sensor, and the muscle contraction pressure change data collected by each PVDF piezoelectric film sensor, and feeds the fused data back to the motion parameter calculation engine. The motion parameter calculation engine is based on Kalman filter fusion data. The motion parameter calculation engine is used to calculate and analyze the fused data to obtain specific motion feedback results. Step S6: Normative evaluation: Compare the user's motion trajectory with the standard template by DTW to output a similarity score. Among them, if it is detected that the knee joint valgus > 8° or the center of gravity deviation > 20%, a real-time alarm is triggered. Step S7: Feedback and optimization: Generate a training report and automatically upgrade the training difficulty according to the passing rate.

[0023] For the lower limb movement monitoring and guidance method of the first embodiment, the kinematic (acceleration) and dynamic (pressure) two-dimensional monitoring is realized by integrating a three-axis MEMS acceleration sensor and a PVDF piezoelectric film sensor through a flexible sensing sheet, and the fusion processing of the sensor data and the motion image data obtained by the RGB-D camera is realized through a multi-modal data fusion algorithm to realize the lower limb movement monitoring and guidance of multi-modal sensor fusion; moreover, after the motion parameter calculation engine feeds back the result, the lower limb movement monitoring and guidance method also realizes the scoring and guidance of the user's lower limb movement through a normative evaluation step and a feedback and optimization step.

[0024] Based on the above situation, it can be seen that the lower limb movement monitoring and guidance method of the present invention can effectively realize the quantitative evaluation of the number of times, speed, arc and action standardization of the user's lower limb movement, and the monitoring and guidance effect is good.

[0025] Embodiment 2, the difference between this Embodiment 2 and Embodiment 1 is that in the step S3, the automatic positioning and calibration includes a static calibration step and a dynamic calibration step; The static calibration step is: the user maintains a standing posture for 3 seconds to establish a skeletal length reference model; The dynamic calibration step is: perform the standard squat action 2 times to calibrate the movement range of each joint.

[0026] Embodiment 3, the difference between this Embodiment 3 and Embodiment 1 is that the process of generating the skeletal length reference model is: Skeletal reconstruction step: convert the RGB-D camera image into a three-dimensional coordinate system through SLAM technology; Data mapping step: perform Kalman filter fusion on the IMU sensor data and the visual skeletal key points; Real-time rendering step: render the model using the Unity 3D engine and the ARKit framework.

[0027] Embodiment 4, the difference between this Embodiment 4 and Embodiment 1 is that in the step S7, the AR image of the user's real-time action is compared with the virtual skeletal line annotation to clearly know the action deviation.

[0028] Embodiment 5, the difference between this Embodiment 5 and Embodiment 1 is that in the step S6, the scoring indicators include: Joint angle deviation index: the angle of knee joint valgus; Movement rhythm index: the ratio of the current speed to the target speed; Muscle activation index: the ratio of the quadriceps femoris to the hamstring muscle force.

[0029] In addition, the dynamic weight distribution in the scoring algorithm logic is: Base score (60%): Joint angle deviation; Bonus score (30%): Muscle synergy; Penalty item (10%): Compensatory movement detection; The threshold in the scoring algorithm logic is set as: Excellent (≥85 points): Deviation of all joints <5°, error of force application ratio <10%; Qualified (60 - 84 points): Deviation of major joints <10°, deviation of minor joints <15°; Warning (<60 points): Trigger voice prompt and pause counting.

[0030] The above content is only a preferred embodiment of the present invention. For those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation manner and application scope. The content of this specification should not be construed as a limitation to the present invention.

Claims

1. A method for monitoring and guiding lower limb movement, the hardware part of which includes a flexible sensor array, a smart phone, and a built-in RGB-D camera and an edge computing module in the smart phone; The flexible sensing sheet array includes five flexible sensing sheets; the flexible sensing sheet includes a skin contact layer, a sensor layer, a flexible FPC substrate and a protective layer which are sequentially stacked, the sensor layer has a three-axis MEMS acceleration sensor and a PVDF piezoelectric film sensor, the wiring method of the flexible FPC substrate is serpentine wiring and the flexible FPC substrate is integrated with a signal conditioning module and a Bluetooth communication module, the three-axis MEMS acceleration sensor and the PVDF piezoelectric film sensor are electrically connected to the input end of the signal conditioning module respectively, and the output end of the signal conditioning module is electrically connected to the Bluetooth communication module; each flexible sensing sheet is respectively provided with an anatomical positioning mark, and the anatomical positioning mark is a visible fluorescent marking point; The lower limb movement monitoring and guidance method includes the following steps, specifically: Step S1, cleaning the skin of the user's lower limbs and ensuring that there is no grease residue; Step S2: attach the flexible sensing sheet to the skin surface of the user's lower limbs through the skin contact layer, and the five attachment points of the flexible sensing sheet are: Rectus femoris: midpoint of the line between the anterior superior iliac spine and the upper edge of the patella; Biceps femoris: 1 / 3 of the line connecting the ischial tuberosity and the fibular head; Tibialis anterior muscle: 5 cm below the tibial tuberosity and 2 cm lateral to the tibial crest; Gastrocnemius: 10 cm below the midpoint of the popliteal crease, the highest point of the gastrocnemius muscle belly; Upper edge of patella: the center of the upper pole of patella; Step S3, scanning the visible fluorescent marking points of each flexible sensor sheet by a smart phone to complete automatic positioning calibration, and automatically adapting to the bone proportions of users of different body shapes through the visible fluorescent marking points and the mobile phone SLAM technology; Step S4, each flexible sensing sheet is connected to the smart phone via Bluetooth through the Bluetooth communication module, and each flexible sensing sheet transmits data to the smart phone using TDMA time division multiple access technology; Step S5: The smart phone fuses the user image data collected by the RGB-D camera, the motion acceleration data and motion frequency data collected by each three-axis MEMS acceleration sensor, and the muscle contraction pressure change data collected by each PVDF piezoelectric film sensor through a multimodal data fusion algorithm, and feeds the fused data back to the motion parameter calculation engine; The motion parameter calculation engine is based on Kalman filter fusion data. The motion parameter calculation engine is used to calculate and analyze the fused data to obtain specific motion feedback results; Step S6, normative evaluation: perform DTW comparison between the user's motion trajectory and the standard template, and output a similarity score; If the knee joint is detected to be bent inward >8° or the center of gravity is shifted >20%, a real-time alarm is triggered; Step S7, feedback and optimization: Generate a training report and automatically upgrade the training difficulty according to the achievement rate.

2. A method for monitoring and guiding lower limb movement according to claim 1, characterized in that: The signal conditioning module is the AD8232 chip.

3. A method for monitoring and guiding lower limb movement according to claim 1, characterized in that: The protective layer includes a nano waterproof coating and an electromagnetic shielding layer.

4. A method for monitoring and guiding lower limb movement according to claim 1, characterized in that: In step S3, the automatic positioning calibration includes a static calibration step and a dynamic calibration step; The static calibration steps are as follows: the user maintains a standing position for 3 seconds to establish a bone length benchmark model; The steps of dynamic calibration are: perform the standard squat movement twice and calibrate the range of motion of each joint.

5. A method for monitoring and guiding lower limb movement according to claim 4, characterized in that: The process of generating the bone length benchmark model is as follows: Skeleton reconstruction steps: Convert the RGB-D camera image into a three-dimensional coordinate system through SLAM technology; Data mapping step: Kalman filter fusion of IMU sensor data and visual skeleton key points; Real-time rendering steps: Use Unity 3D engine and ARKit framework to render the model.

6. A method for monitoring and guiding lower limb movement according to claim 1, characterized in that: In step S7, the AR image of the user's real-time action is compared with the virtual skeleton line annotation to clearly understand the action deviation.

7. A method for monitoring and guiding lower limb movement according to claim 1, characterized in that: In step S6, the scoring indicators include: Joint angle deviation index: knee joint inward angle; Movement rhythm index: current speed / target speed ratio; Muscle activation index: quadriceps / hamstring force ratio.

8. A method for monitoring and guiding lower limb movement according to claim 7, characterized in that: The dynamic weight distribution in the scoring algorithm logic is: Basic score (60%): joint angle deviation; Additional points (30%): Muscle synergy; Penalty item (10%): Compensatory action detection; The thresholds in the scoring algorithm logic are set as: Excellent (≥85 points): All joint deviations are <5°, and force ratio error is <10%; Pass (60-84 points): major joint deviation <10°, minor joint deviation <15°; Warning (<60 points): Triggers a voice prompt and pauses the count.

Images